Aims

Results of former research (Stes, Gijbels & Van Petegem, submitted to Higher Education) showed that the teaching approach of university teachers is not determined by context and demographic variables. This no-predetermination opens up perspectives for professional development: a shift towards a (more) student-focused teaching approach can probably be promoted by training. Recent research (e.g. Gibbs & Coffey, 2004; Postareff et al., in press) supports this idea. However, altering academics’ ideas about teaching is not easy (Kember & Gow, 1994). In this study we investigate whether there are differences in teaching approaches between two groups of respondents: one who participated in a training programme and one who did not. Our first research question runs as follows: Does a generic instructional development programme have an impact on teaching approach? This question is important since research evidence suggests that the way teachers approach their teaching has important implications for their students’ approaches to learning (Trigwell et al., 1999; Kember & Gow, 1994). However, it is not the approach to teaching as such, but in particular the way students perceive the teaching, that affects student learning (Ramsden, 1997; Entwistle, 1998). So, as a next step we investigate whether there are differences in teaching behaviour, as perceived by students, between teachers who participated in training and those who did not. Our second research question becomes: Is there an impact of a generic instructional development programme on the teaching behaviour of teachers, as perceived by students?

Methodology

Data were gathered according to a quasi-experimental pre-test/post-test-design. Initially, 23 teachers, enrolled on a forthcoming teacher training programme at the University of Antwerp, were asked to participate in this study. To form a control group they were asked to find a colleague, not enrolled in the programme but otherwise comparable (in terms of class size, level of expertise, teaching discipline, academic status, teaching experience and age) to participate as well. The final sample of 50 teachers (23 teachers belonging to the experimental group, 27 to the control group) selected one of the courses they were responsible for. The students of these courses were involved in this study too. Shortly before the start of the training, teachers and students were questioned anonymously (n teachers=48, n students=1058). All teachers completed Trigwell and Prosser’s (1996) Approaches to Teaching Inventory (ATI); their students were presented part 2 of Entwistle’s (2005) Experiences of Teaching and Learning Questionnaire (ETLQ). A post-test, using the same two instruments, was conducted shortly after the completion of the programme (n teachers=42, n students=1028).

First, t-tests on the differential scores were run twice, using as test values zero and the mean differential value of the control group. Secondly, paired samples t-tests and an analysis of covariance (ANCOVA) were conducted to assess the effect size of training on teaching approaches and behaviour. The results concerning teaching approaches are limited to the subscale ‘conceptual change/student-focused approach’ (CCSF) of the ATI-questionnaire, since the ‘information transmission/teacher-focused’ scale had an insufficient reliability on the pre-test. Further, a detailed examination of the differential scores for teaching approach indicated one clear outlier, being therefore emitted from the analyses.

Findings

In comparing the results on the pre- and post-test the control teachers did not increase significantly on the CCSF scale (test value=0, t(18)=1.222, p=.24), whereas their experimental colleagues’ results showed a clear boost (test value=0, t(20)=3.629, p<.01). Moreover, the increase for the experimental teachers varies significantly from the one reported by the control group (test value=0.2979, t(20)=2.149, p<.05). A paired samples t-test indicates a small effect for the control group (d=.28), while we see a large effect size for the experimental teachers (d=.79). These findings hint towards an effect of teacher training on CCSF as teaching approach. An ANCOVA reveals no significant effect from training (F(1,37) = 1.25, MSE = .56, p=.27). Moreover, the effect size is rather small with training explaining 3.3% of the variance in CCSF.

Is there an impact of teacher training on the teaching behaviour, as perceived by students? Results are limited to the subscales of part 2 of the ETLQ-questionnaire which showed consistent reliability (Cronbach’s alpha>.70) for both the pre- and post-test: ‘Clear aims, organisations, alignment and integration’, ‘Assessment for understanding and feedback’ and ‘Interest, enjoyment and perceived relevance’. Few significant effects could be traced. This shows that, on short term, training does not change teaching as perceived by students. There were nonetheless two exceptions. A t-test for assessment for understanding and feedback provides no significant results (experimental group: test value=0, t(15)=1.032, p=.32; control group: test value=0, t(14)= -.666, p=.52). Yet, both groups differ significantly (test value=-1.46, t(15)=2.30, p<.05). The scale can be subdivided into ‘clarity and feedback about assessment’ on the one hand and ‘assessment for understanding’ on the other, of which only the first scale results reliable. The experimental teachers vary from the control group on clarity and feedback about assessment (test value=-0.1989, t(15)=2.824, p<.05). This element of teaching thus seems most prompt to change due to a training.

Theoretical and educational significance of the research

In recent years educational development in higher education has become an important topic. In spite of the importance attached to it, evaluations are generally limited to measures of participants’ satisfaction: little is known about the real impact on daily teaching practice (Wilson & Berne, 1999). The lack of systematic programme evaluation is an ongoing concern (Ho, 1998; Rust, 1998). Furthermore, students are seldom involved in impact research (Ho, Watkins & Kelly, 2001). This study made a first step towards meeting these concerns. We measured the impact of a generic instructional development programme on teaching approaches and teaching behaviour as perceived by students, thereby using a quasi-experimental pre-test/post-test-design. Our results are promising: a training programme does indeed have a significant impact on teaching approaches. The impact on students’ perceptions of teaching behaviour was less distinct, namely limited to perceptions with regard to assessment. Further research on factors influencing these impact results (e.g. the teaching context, teacher’s motivation for teaching) is needed to help understand the implications concerning the design of educational development programmes.