In a large number of educational systems school self-evaluations have become a common phenomenon. Although many stakeholders welcome these school self-evaluations, there is considerable doubt regarding their quality. The question as to (the explanation) of the differences which currently exist between schools has so far remained unanswered or rests on assumptions. When it comes to determining which criteria can be used to evaluate the quality of a self-evaluation, few research results are available. A general agreed upon set of criteria is strongly needed however. Therefore it is of essential importance to gather arguments, experiences and opinions of relevant stakeholders. This paper reports on a Delphi study that initiated a communication process between Flemish (Belgian) stakeholders in order to explore different points of view and arguments. The research method consisted of consulting a group of well-selected experts in two rounds. It resulted in a set of criteria for meta-evaluation. These involve the nature of action points in which a school self evaluation should result according to the respondents and a list of agreed upon worthwhile effects. Necessary conditions put forward by the respondents in order to realise these effects are described. The operational framework that is described enables us to identify the strengths and weaknesses of (Flemish) schools with regard to relevant aspects of school self-evaluations.

In the last ten years, school self-evaluations have become a common phenomenon in a large number of educational systems (McBeath & Myers, 2002; Devos & Verhoeven, 2003; Meuret & Morlaix, 2003). Governments, universities, research institutions and schools themselves have developed tools and methodologies that enable schools to evaluate their own functioning. One the most important aims of these initiatives is to solicit organizational learning (Argyris & Schön, 1996; Friedman, Lipshitz & Overmeer, 2001; Paslowsky, Forslin & Reinhardt, 2001; Patriotta, 2004). Although stakeholders welcome school self-evaluations, there is considerable doubt regarding their quality (Nevo, 2001; Van Petegem, Verhoeven, Buvens & Vanhoof, 2005). For instance, conducting self-evaluations turns to be an activity which schools do not necessarily want or are not able to implement. Flemish (Belgian) policy-makers stipulate that schools often come to positive evaluations of their own self evaluations while, in reality they are still at the start of a long process of development. As such, the importance of increasing the willingness and the capacities of schools to achieve high-quality self-evaluations (and organizational learning) cannot be overestimated.

At present it is unclear as to what extent schools possess sufficient organizational effectiveness and capacity for self-evaluation to draw up strategies, methodologies and criteria for high-quality self-evaluation for themselves. The question as to the (explanations of) differences which currently exist between schools has so far remained unanswered or rests on assumptions (Rogers & Hough, 1995; Robinson & Cousins, 2004). This makes it difficult to estimate what can be expected from schools in this regard. Amongst policy makers there seems to be unanimity with regard to the importance of school self-evaluation as an element in quality development in schools, but when it comes to determining which criteria can be used to evaluate the quality of a self-evaluation things become more complicated. A combination of self-evaluation and school audits, for instance, presupposes a minimum acceptable quality of self-evaluation and the tools to record that quality. In other words, an array of tools needs to be developed for meta-evaluation (Scriven, 1969).

The above requires a general agreed upon set of criteria to judge the quality of a self-evaluation. Therefore it is of essential importance to gather different arguments, experiences and opinions of relevant stakeholders (teachers, principals, inspectors, educational policy makers, educational counsellors, self-evaluation counsellors and researchers). A Delphi study is an appropriate research technique to initiate a communication process between these stakeholders and to explore different points of view and arguments. The research method consists of consulting a group of well-selected experts in a number of rounds (Adler & Ziglio, 1996). It is an approved technique to make a valuable contribution to the solution of a problem by illuminating it from different points of view. The Delhi-study consisted of two question rounds including 15 Flemish experts. In the first round a written questionnaire was used; the second round consisted of in-depth interviews. The guiding research questions were: (1) How can the quality of a self-evaluation be operationalised? What are appropriate evaluation criteria? (2) What differences can be found between schools’ self evaluations and how can these be explained?

The results section of the paper firstly describes the criteria for meta-evaluation. It focuses on the nature of action points and agreed upon worthwhile effects in which a school self evaluation should result according to the respondents. Next, attention is given to the necessary features of the self evaluation process if one wants to create effective learning opportunities. Necessary conditions put forward by the respondents in order to realise worthwhile effects are described.

The study confirms the importance of the key factors promoting policy effectiveness and organisational learning. The concept of ‘policy making capacities’ appears to be a promising tool for studying, monitoring and developing the quality of self-evaluations. According to Flemish stakeholders factors such as effective communication, the innovative, responsive and reflective capacity of the school and the integrated character of the self-evaluation have a predictive value with respect to the quality of school self evaluations. Furthermore, school self evaluation is not only referred to as an act of policy but also as an act of research. The findings of the present study clearly show that implementing high quality school self evaluations is not only a matter of giving schools the necessary tools and technical self-evaluation skills. Promoting policy making capacities of schools and research skills is an appropriate strategy too in order to improve the quality of self-evaluations. The operational framework that is described enables us to identify the strengths and weaknesses of (Flemish) schools with regard to relevant aspects of school self-evaluations.