Modelling is a fundamental ability in science. The present paper describes an attempt to develop and validate a series of open-ended diagnostic tests for measuring University students’ modelling ability. We have designed tests for measuring three aspects of the modelling ability: a) specific modelling skills, b) knowledge about the modelling process, and c) meta-modelling knowledge. The twelve tests developed were used to assess the effectiveness of an inquiry oriented modelling-based curriculum that was implemented in the frame of a blended e-learning course at the University of Cyprus, during the spring semester of 2006. Our sample consisted of seventeen pre-service teachers who attended the course. We analytically describe the assessment procedure for one of the tests and summarize the results of the remaining tests. Combined qualitative (phenomenographic analysis) and quantitative analysis (Multivariate Analysis of Variance-repeated measures) indicates the quality of the tests and provides some insights on the effectiveness of the instructional intervention for the development of students’ modelling ability and for the transfer of this ability in new unfamiliar contexts.

Modeling is a fundamental thinking skill in science (Schwarz & White, 2005). Teaching and learning about modeling needs to identify and promote a number of critical elements (Justi & Gilbert, 2002). According to Papaevripidou et al. (2006),it is possible to define modeling ability, as a set of a) specific modeling skills (i. model formulation, ii. identification of model components, iii. Ability to compare and contrast models of the same phenomenon and identify advantages and disadvantages, iv. model evaluation through contrasting it to the real phenomenon and formulating ideas for improvement) b) knowledge about the modeling process (ability of the learner to explicitly describe and reflect on the major steps of the modeling-based cycle) and c) meta-modeling knowledge (appreciation of the purpose and utility of scientific models).

The aim of the presented research is to develop an approach to assess modeling ability and to use it to assess the effectiveness of an inquiry oriented modeling-based curriculum.

The participants of this study were seventeen pre-service teachers of a science course, which adopted a blended e-learning approach, at the University of Cyprus (spring semester, 2006).

The intervention was based on an iterative procedure, which involved the learner in an active process of constructing and deploying a model. This process comprised of two main stages: a) the construction of a model for the phenomenon, and b) the study of the phenomenon through systematic real world observations. A series of specially designed open-ended diagnostic tests, which were administered to the pre-service teachers before and after instruction were used as a means for assessing their modeling ability (table 1).

Due to space limitation we will only describe the assessment process for one of the tests, namely, “the solar system” test, which required students to observe a model of the solar system and identify weather it is useful to build such constructions. Table 2 summarizes the results of the phenomenographic analysis of students’ answers in this test. After carefully analysing the student’s answers we created categories of responses which were finally ranked into a hierarchical list. The results presented in the table indicate the possibility that two categories belong to the same level. For each level in the table, an authentic student response is presented (in italics). For example categories I and I are considered hierarchically equal, they belong to level 1 and are scientifically accepted. The results of this analysis indicate the transfer of students answers from lower levels of the table to upper ones as a result of the teaching intervention. Phenomenographic analysis helped us form criteria of assessment scored with one point each: Through a model someone can (a) represent, (b) visualize, (c) understand a phenomenon, (d) check the validity of a theory

The application of these criteria to each student’s response led to the quantitative analysis of the results. The results of the statistical analysis are also presented in Table 1. One issue that warrants mention is the fact that there are statistically significant differences in the students’ performances on all tests prior to and after instruction for all aspects of modeling. Moreover, the effect of the instruction (η²) for each of the aforementioned modeling aspects varies from 26,9% (test 2b) to 90,1% (test 3b).

Our research question refers to the implementation of an instructional intervention for the development of pre-teachers’ modeling ability. The implementation of the modeling cycle concerning moon phases in the activity sequence successfully led to the development of modeling ability. The effectiveness of our intervention was demonstrated by both the qualitative results of the phenomenographic analysis and the quantitative analysis of the student responses to the diagnostic tests. The statistical analysis conducted for the comparison of pre- and post-tests was based on tests that referred to a number of other systems besides moon phases (course content). This ascertained that the diagnostic tests isolated the effect of modeling ability from the particular context of the curriculum. The analysis therefore provided evidence that the instructional intervention was useful not only for the development of modeling ability but also for the transfer of this ability in new unfamiliar contexts

Additionally, we have presented a method of assessing modeling ability. It warrants mention that when reviewing the literature about modeling there is a lack of assessment approaches concerning the modeling ability. The twelve open-ended tests that we have developed and validated (table 1) allow for controlling students reasoning (qualitative analysis). We do not claim that the presented tests and the process for assessing them is the panacea. We do however suggest that this process could constitute a basis for further efforts for assessing modeling ability. From our point of view, the advantage of this procedure is that it allows for the transformation of qualitative responses into quantitative data and therefore promotes the combination of qualitative and quantitative analysis (Howe, 1988).

Howe, K. (1988). Against the quantitative-qualitative incompatibility thesis. Educational Research, 17(8), 10-16.

Justi, R., & Gilbert, J. (2002). Science teachers’ knowledge about and attitudes towards the use of models and modelling in learning science. International Journal of Science Education, 24(12), 1273–1292.

Marton, F. (1981). Phenomenography-describing conceptions of the world around us. Instructional Science, 10, 177-200.

Papaevripidou, M., Constantinou, C. P., & Zacharia, Z. (submitted). Facilitating the development of the modeling ability of fifth graders: A comparison of the effectiveness of two instructional approaches or investigating the effectiveness of an explicit and an implicit modeling-based instructional approach towards the development of the modeling ability of fifth graders. Journal of the Learning Sciences.

Schwarz, C. V., & White, B. Y. (2005). Metamodeling knowledge: Developing students’ understanding of scientific modeling. Cognition and Instruction, 23(2), 165–205.