The PISA Project has had a strong repercussion in all countries were it has been implemented, especially in those where the scores obtained are beneath the European average, as is the case of Spain. Unfortunately, at least in our country, this repercussion has not exceeded the limits of debate in public media and little is being done to change educational practices so as to obtain better results in the next assessment.

With this research we try to demonstrate that the PISA items can not only be useful to evaluate the progression of a given educational system every four years. The epistemological and psycho-educational bases of PISA can also be helpful to promote changes in the educational conceptions, discourses, and practices regarding both assessment and instructional practices of Secondary Education teachers when they are invited to participate in research process as partners.

Our starting point is a case study constituted by eight secondary school teachers, involved in a two-year long research. During this period we negotiated with them what we called a Guide for Task analysis from the PISA Perspective (GTAPISA).

To analyse the changes we used a consensual system of categories, collecting these dimensions: Degree of authenticity of assessment tasks and class sessions; degree of complexity of the contents included in the assessment task, and degree of coherence and promotion of learning autonomy.

Results allow us to confirm our objectives. Indeed, the teachers’ incorporation as research partners facilitated the change in their conceptions from a view oriented to “Assessment of Learning” to a view oriented to “Assessment for Learning”.

On the other hand, the study shows that the counselling-action process also promotes evident changes in the assessment tasks elaborated and, to a lesser extent, in the Teaching Units development.

 

Introduction

 

The PISA Project has had a strong repercussion in all countries were it has been implemented, especially in those where the scores obtained are beneath the European average, as is the case of Spain. Unfortunately, at least in our country, this repercussion has not exceeded the limits of debate in public media and little is being done to change educational practices so as to obtain better results in the next assessment. Indeed, if applying PISA is not accompanied by measures to introduce real change, the repeated failure of some countries in comparison with others (with different starting points), could generate a sort of “learned helplessness” that leads to resignation, discouragement and passivity. Precisely, emphasizing learning processes and not only factual and conceptual contents, changing educational practices based on a expository model, modifying epistemological conceptions which are very deeply rooted in repetition and reproduction patterns is really complex.

With this research we try to demonstrate that the PISA items can not only be useful to evaluate the progression of a given educational system every four years. The epistemological and psycho-educational bases of PISA can also be helpful to promote changes in the educational conceptions, discourses, and practices regarding both assessment and instructional practices of Secondary Education teachers when they are invited to participate in research process as partners, as Perry, Phillips & Dowler (2004) or Boekaerts & Corno (2005) have showed in their studies.

 

Context

 

Research (i.e. McDonald et al., 1995) has shown that assessment has retroactive effects on the way students learn, but also on the way teachers teach. In a previous study presented in the XI EARLI Conference (Monereo & Castelló, 2005), we showed that introducing superficial changes in teachers’ usual assessment tests (whose characteristics placed them in a distant position from PISA) was not enough to change the way they evaluate. To obtain a relevant change in teachers’ discourses, conceptions and practices changes must be introduced which are in line with the main PISA dimensions: authenticity, complexity and autonomy. With this goal in mind we created a system of counselling-action that involves teachers in our research, taking advantage of the social and educational relevance of the PISA Project.

 

Research questions & objectives

 

This study aims to provide an answer to the following main research questions:

1- Can a counselling-action system based on the PISA Project change teachers’ conceptions and assessment practices?

2- To what degree and in relation to which variables will a change in assessment practices have effects in the teachers’ instructional practices?

 

Research methodology & design

 

Our starting point is a case study constituted by eight secondary school teachers of science, maths, English as a foreign language, French as a foreign language, Sociolinguistics, History and Greek as an antique language, involved in a two-year long research. During this period we negotiated with them what we called a Guide for Task analysis from the PISA Perspective (GTAPISA).

The procedure we followed during these two years is reflected in the table below:

 

To analyse of data we were proceeded in the following way: the questionnaires, in-depth interviews and focus group were transcribed and categorized, using a judge system and the Kaplan index; to assess tasks and record class sessions, we used a consensual system of categories, collecting these dimensions:

 

a) Degree of authenticity of assessment tasks and class sessions: 9 items refer to their degree of realism, degree of relevance for students and ecological proximity.

b) Degree of complexity of the contents included in the assessment task. 10 items about the typology of contents included, subject orientation and degree of coherence.

c) Promotion of learning autonomy. 10 items refer to promotion of students’ planning, self-regulation and self-assessment.

 

Every recorded assessment task and class session was analysed independently by four researchers, admitting only the categories with a 75% of agreement.

Results

 

Results allow us to confirm our objectives. Indeed, the teachers’ incorporation as research partners facilitated the change in their conceptions from a view – according the fortunate expression of Birenbaum et al. (2006) – oriented to “Assessment of Learning” (summative, inauthentic, context independent) to a view oriented to “Assessment for Learning” (formative, authentic, context embedded).

On the other hand, the study shows that the counselling-action process also promotes evident changes in the assessment tasks elaborated and, to a lesser extent, in the Teaching Units development. The more revealing changes appear in all content complexity sub-dimensions and in realism (authenticity); the sub-dimensions that change less are relevance (authenticity) and self-regulation (autonomy). 

 

Practical & theoretical relevance of results

 

We think that the results obtained in this research imply two kinds of contributions:

 

a) From a theoretical point of view, these results support the interest of establishing more intense collaborations with teachers with the objective of producing substantial changes in their conceptions and educational practices, turning teachers into innovators, as suggested by Randi & Corno (1997).

b) From a practical point of view, this research provides some useful analysis and reflective tools to revise educational tasks (GTAPISA) and, in the same way, it suggests a set of dimensions and indicators to evaluate the degree of authenticity, complexity and autonomy of assessment and instructional activities.