Even in the 1980s and 90s, when Hungary was internationally renowned for student achievement in tha application of science knowledge in school-related situations (Keeves, 1992), experts warned that, as regards using school-based knowledge in real-life situations was not effective (B. Németh és Csapó, 1994; B. Németh, 2000; 2002). Having observed several years of steady decline in Hungarian students’ science achievements (as reported by Beaton et al., 1996a; 1996b; Martin, Mullis, Gonzalez and Chrostowski. 2003), the University of Szeged Center for Research on Learning and Instruction launched a research project to examine changes in students’ use of their school-based knowledge in solving real-life problems by repeating 1999 survey and comparing the 1999 and the 2006 data sets collected in two age groups, 13- and 17-year-olds (7th and 11th grades). Data were also collected on the development of reading literacy, inductive reasoning and complex problem solving, as well as attitudes to school subjects.

SAMPLES

The test was administered to 3464 students in grade 7 and 2426 students in grade 11. The subjects studied in 282 classes in 155 schools. The sample was representative with respect to territorial coverage, gender and socio-cultural background.

INSTRUMENT

The test contained 31 open ended questions. Each task presents an everyday phenomenon and asks for an explanation (e.g. “How does paint protect objects made of iron from corrosion (rusting)?”) The answers to the questions are not directly taught in science classes, but they can be found by applying the knowledge learned there to the given situation. Student’s responses were evaluated according to a coding manual on a three-point (0, 1, 2) scale.

The test proved a reliable instrument in both age groups, with Cronbach α values somewhat higher than those of the IEA-TIMSS surveys.

RESULTS

In 2006, the mean performance was 26 %p in grade 7 and 39 %p in grade 11. Performances are distributed along an asymmetric curve skewed to the left. 93% of 13-year-olds and 73% of 17-year-olds achieved below 50%p. Only .4 and 3.7% of the sample achieved higher than 70 %p.

These results are lower that those in 1999 (Table 1). The analysis of the tasks reveals a complex picture. In 2006, performances were significantly higher than in 1999 on nine tasks in grade 7 and twelve in grade 11.  The differences between the age groups on the individual tasks vary greatly in both data sets, with a somewhat broader range in 2006 (1.3-38.1 %p) than in 1999 (2.5-33.9 %p). Difficulty indices calculated with the partial credit model (ConQuest) show that different tasks proved difficult at the two assessment points. The difference between the 1999 and the 2006 difficulty indices of two items is very small, 0.01 logit. In 2006, 15 items were more and 14 less difficult than in 1999.

Weak correlations with attitudes towards science suggest that the poor application of knowledge does not depend on students’ like or dislike of sciences.

The application of science knowledge is moderately correlated with complex problem solving, inductive reasoning and reading literacy (Table 3), and the coefficients are somewhat stronger in the older age group. A regression analysis revealed that the knowledge domains examined explain 24.7% (grade 7) and 30.6% (grade 11) of the variance of the application of science knowledge. Complex problem solving was the largest contributor to the explained variance in both age groups.

The application of science knowledge is moderately correlated with complex problem solving, inductive reasoning and reading literacy (Table 2), and the coefficients are somewhat stronger in the older age group. A regression analysis revealed that the knowledge domains examined explain 24.7% (grade 7) and 30.6% (grade 11) of the variance of the application of science knowledge. Complex problem solving was the largest contributor to the explained variance in both age groups.

Table 2. Regression analyses by age group

Discussion

Because tasks required explanations for simple, everyday phenomena the backgrounds (concepts) of which are had been taught is science classes, high performances could be expected from well prepared students in the secondary school at least. Therefore students’ actual performance can be evaluated as poor: they are not effective in recognising and applying what was learnt in class in different, real-life situations.

Between 1999 and 2006, the applicability of science knowledge decreased by 5-8 %p. However, mean performances on tasks show that students could rely more on some concepts and less on others to explain the given problems in 2006. This suggests that curricular changes did not produce an overall effect, but they did facilitate learning in the case of a few concepts.

The analysis revealed that application is weakly related to attitudes, but it is markedly influenced by the developmental level of inductive reasoning, complex problem solving and reading literacy.

The findings suggest that the knowledge tapped by the instrument originates from learning outside the school and draws mostly on a growing body of experiences and on the development of abilities, a finding worth considering for a more effective science education.