What teachers have in mind is actually an important research field regarding the quality of teaching and learning activities in classrooms. One field about teachers mind is concerned the construction and formulation of competence profiles; it has recently become a significant issue for teacher training and continuous education. We conceptualize teaching competence as the capacity to act in a professional, meaningful, and functionally appropriate manner in a particular situation. The measurement of this competence profile on a given level defines a standard. In our work, we develop and assess standards for vocational teachers. Drawing on an expert rating study (N=793), we have developed a theoretical framework that describes 45 holistic competence profiles rated as successful teaching elements in vocational schools. Further, we have constructed a video-based diagnostic instrument that informs teachers how well they master a standard.

We present results of the validation study with 159 vocational teachers and 42 non-teachers. Participants viewed two film vignettes on group work and then answered a set of questions about various criteria: eight general dimensions of teaching quality (e.g., sympathetic understanding, effectiveness, motivation, etc.) and five standard specific dimensions (e.g., comprehensibility of the introduction, effectiveness of the group-formation, etc.). Based on a confirmatory factor analysis, we constructed 13 scales (alphas >.65; < .89). The results of a cluster analysis show that respondents with different teaching backgrounds evaluate the teachers’ actions differently. We discuss the complexity of the definition of expert-profiles and especially the question of how to designate an expert with respect to a specific standard. In addition, we present how we tackle these challenges, e.g. by remodeling our diagnostic instrument. Further, we provide an insight into our theoretical concept of “pedagogical necessity” that focuses on the situational aspects of standard oriented teachers’ actions.

The search of competencies in teaching and teacher training has a long history (see Winne, Berliner & Gage, 1975; Heath & Nielsen, 1974; Shulman, 1987; INTSAC-Standards, 1992 etc.). At the present time fundamental work has been done by Zeichner (2006) who states that everything we know from expert teachers has to be integrated into teacher training programs; and a breakthrough is given by Darling-Hammond (2006) who proposes to develop teaching standards from exemplary teacher education programs. In German-speaking Europe at least three research groups are working on the development of teaching standards, the first one is exploring comprehensive field experiences to find possibilities to improve mathematics and science instructions (see Prenzel & Ostermeier, 2006), the second focuses on the investigation of the effectiveness of pedagogical content knowledge and our group has set the goal to develop diagnostic instruments for measuring the quality of different standards in the field context (i.e. teaching environment).

We understand teaching competencies to be the capacity to act in a professional, meaningful and functionally appropriate manner in a given teaching situation. The measurement of this competence on a given level (ranging from optimum to minimum) defines a standard (Oser & Oelkers, 2001). By the means of a quasi-delphi study we inductively developed a set of 45 such teaching competence profiles (standards) for vocational school teachers in collaboration with expert teachers and vocational education experts. They are classified in four main groups and two to three corresponding sub groups (see table 1)

These standards are formulated on a general level of abstraction. An important number of vocational school teachers (N=793) rated each standard in terms of its importance, frequency of application in the own classroom, difficulty of deployment and desired importance for teacher training. Based on these ratings, the standards were clustered, identifying types of standards with different profiles. At the concluding evaluation however, 98% of the questioned teachers confirmed the meaningfulness of the generated competence profiles by judging all 45 standards to be ‘important’ or ‘rather important’. The results of this evaluation study are recapitulated in our presentation.

On the base of these affirmative findings we further filmed authentic teaching situations at vocational schools and produced standard specific film vignettes each showing a holistic, autonomous and integral classroom situation with respect to a given standard from three perspectives simultaneously (the teacher’s view, the student’s view and the overall view). These film vignettes are the principal components of the generated computer-controlled diagnostic instruments to evaluate the teaching quality of vocational schools and to measure the degree of attained quality.

Apart from providing a short insight into such a film vignette and the prototype of the diagnostic tool the evaluation of this prototype is presented: A considerable number  (N=201) of vocational school teachers (experts, professionals and novices) as well as of non-teachers viewed two film vignettes of the above mentioned standard ‘group-work’ and rated the observed teaching quality with respect to a set of items. These items correspond to eight general dimensions of teaching quality (sympathetic understanding, effectivity and pacing, engagement and motivation, task difficulty, flexibility, autonomy and control, differentiation and complexity, vision and task significance)(see Helmke & Weinert, 1997) or to one of the standard specific dimensions (comprehensibility of the introduction, task instruction, effectiveness of the group-formation, integration of the results into the continuous classroom work, general instruction). A confirmatory factor analysis (CFA) highly validates these dimensions – all measurement models satisfy current cut off criteria (χ²/df ≤ 2.5; GFI ≥ .9; AGFI ≥ .9; RMSEA ≤ .05)(see Backhaus, Erichson, Plinke & Weiber, 2006, p.382) – and allows the generation of scales with satisfactory to highly satisfactory Cronbach’s Alphas ranging from .650 to .890. Further, a cluster analysis confirms the hypothesis that respondents with different teaching backgrounds evaluate the observed teacher’s actions differently. The confirmation of our hypothesis that a profile of the expert’s ranking of the film vignettes can be compiled turned out more complicated than expected. We therefore picture the challenges of the designation of expert-profiles and particularly discuss the difficulties that occur by defining expert teachers in general but also in the field of vocational schools. We also show pathways how to tackle these challenges, and how we are remodelling our diagnostic instrument.

We will discuss problems and difficulties to work out and to work with such a diagnostic instrument. What does it mean to perceive and to judge other’s teaching and to infer from this judgement to the teacher’s own competencies? How can such diagnostic instruments be misused? What number of criteria to be judged is meaningful for such a tool? These and similar questions shall be part of the discussion.