Proficient students are assumed to select appropriate strategies and adjust behavior to changing task demands, making use of the awareness of previously knowledge and selecting appropriate study behavior. Metacognition was found to be instrumental in challenging tasks in mathematics, not overtaxing the capacity and skills of children and in relatively new strategies that are being acquired.This symposium focuses on the challenges and potentials of metacognition in mathematics. In a first presentation Marja Vauras analyses developmental interactions of word problem solving, metacognitive knowledge and metacognitive experiences in elementary school students as a function of gender, taks difficulty and mathematical proficiency. In the next presentation Veenman elaborates on the role of metacognitive skills in different types of learning tasks in the domain of mathematics in secondary school students. In addition Tempelaar analyses the role of self-perceived metacognitive knowledge, skills, and attitudes, in learning mathematics at a high-school level. Finally Mei-Shiu Chiu investigates in grade 5-7 students the levels of meta-cognitive knowledge in achieving deep approaches to

Within a longitudinal design, we examined the associations between mathematical problem solving (MPS), metacognitive knowledge (MCK) and metacognitive experiences (ME) of elementary school students as a function of gender, task difficulty and mathematical proficiency. Although the notion of ME was introduced by Flavell in 1979, empirical studies are recent and still rare. Metacognitive experiences are feelings, estimates, and thoughts about cognition during the task, and contribute to students’ self-regulation of learning. ME differ from MCK, that is, declarative knowledge regarding goals, persons, tasks, and strategies, as well as from metacognitive skills. In this presentation, we (1) offer some new evidence on interactions between ME and task performance, and (2) present results on developmental associations between MCK, ME and math problem solving.

The participants were 10-year old, 4^th grade students (n = 436), following the mainstream curriculum, and they we tested on math problem solving and metacognition at three time points over a one year period. A year later, we collected same data from a comparable cohort, and this cohort is used to validate outcomes. In sum, our overall findings suggest a relatively accurate calibration of judgments already at young age, and indicate context-dependent nature of ME compared to more stable MCK. Gender effects showed over-optimism in boys and uncertainty in girls, which may have an important bearing on observed differences between later mathematical interest and proficiency. Correlations between MCK and ME were low, indicating that they capture notably different aspects of metacognition. Preliminary path analyses revealed that ME had a significant relation to MPS only within the same task context, but MCK was directly related also to later MPS. Multi-level analyses comprising the full longitudinal data will be presented at the conference, and discussed in relation to previous research evidence and educational implications.

Within a longitudinal design, we examined the associations between mathematical problem solving (MPS), metacognitive knowledge (MCK) and metacognitive experiences (ME) of elementary school students as a function of gender, task difficulty and mathematical proficiency. The importance of metacognition in math problem solving has been widely recognized in literature (e.g., De Corte et al. 2000), but studies on long-term development between metacognition and math problem solving in young students are scarce. Further, although the notion of ME was introduced by Flavell already in 1979, empirical studies are quite recent and still rare (e.g., Efklides 2001). Metacognitive experiences are feelings, estimates, and thoughts about cognition during the task, and contribute to students’ self-regulation of learning. ME differ from MCK, that is, declarative knowledge regarding goals, persons, tasks, and strategies, as well as from metacognitive skills (Efklides 2001; Salonen et al. 2005).

Efklides et al’s (1998) study showed that ME correlates with performance only in moderate difficulty math tasks, and later Atras & Efklides’ (see, Salonen et al. 2005) found unrealistic ME of low-achievers, who reported low Feeling of Difficulty (FOD) and felt satisfied with their performance despite actual performance. Evidence on gender effects is so far scarce. Efklides, Papadaki et al. (1999) found only weak gender effect on FOD with 7th to 9th students, whereas Merenluoto (2001) observed strong effects on Estimate of Certainty (EOC) with 17-year-old girls skilled in math, uncertainty not related to actual task performance. In general, MEs can be seen as relatively sensitive indices, but context (e.g. task difficulty) dependent, which may be important in understanding why metacognition often fails to take control of cognition (e.g. Efklides et al. 1999; cf. Veenman & Elshout 1999). In this presentation, we offer some new evidence on interactions between ME and task performance as a function of gender and math proficiency within a longitudinal study.

Extensive evidence signify low-achievers’ ineffective use of, e.g., metacognitive strategies (see, e.g., Meichenbaum & Biemiller 1998). Our results have indicated that early (1^st grade) metacognitive proficiency importantly predicts later proficiency in MPS at the 3^rd grade (Vauras et al. 1999). Annevirta and Vauras (2001) also found distinct developmental paths (progressive and regressive) of MCK from kindergarten to the end of the 3^rd grade, and later, Annevirta et al. (2006) found that rapid increase in MCK contributed to cognitive development. In this presentation, we give evidence on developmental associations both between MCK and MPS, and between MCK, ME and MPS solving as a function of gender.

Method

The participants were 10-year old, 4^th grade students (n = 436), following the mainstream curriculum. One year later, we collected same data from a comparable cohort (n = 440), and this cohort is used to validate outcomes.

Word problem solving (WPS) was assessed at three time points at the 4^th and 5^th grade with 15 one- and multi-step word problems, structurally identical at each time point and demanding acute realistic consideration (e.g., Kajamies et al. 2006; cf., Verschaffel et al. 2000). The total number of correct solution steps was used as an indication of WPS skills (max 86).  Inter-rater agreement was high.

Math proficiency. Math proficiency was computed by using both WPS and arithmetic skill test (RMAT). RMAT is a time-limited test (max 56), from simple computations to algebraic tasks (Räsänen 1992), and comparable to the WRAT-R (Jastak & Wilkinson 1984), but following more closely the Finnish math curriculum. Both math variables were equally weighted in math proficiency.

Metacognitive experiences (ME). After each word problem, the students were asked to assess their Estimation of Certainty (EOC) (1 = very certain, 5 = very uncertain), and Feeling of Difficulty (FOD)(1 = very difficult, 5 = very easy) on a five-point Likert-scale (Kinnunen et al. 2006). They were also invited to explain their estimates in own words.

Metacognitive knowledge (MCK) was assessed by a series of verbally and pictorially presented tasks (n=9) in a classroom condition; the test was modified from our individually administered MCK test (e.g., Annevirta & Vauras 2001). In each task, the pictures depicted familiar situations, and the student selected the picture indicating the best way to remember, learn or understand in this particular task, and to select an explanation best related to her/his choice. The sum of combined scores (alternative + explanation) was used as an indication of MCK (max 54, min 9). Inter-rater agreement was satisfactory, but more detailed reliability analyses on the classroom test version will be carried out.

Results

ME and task performance. Significant correlations between ME and task performance (p = .47** for both FOD and EOC) were found. Only in novel task-types, EOC was over-optimistic, the students seeming to lack proper assessment criteria. Higher uncertainty and feeling of difficulty were estimated by mathematically weaker students, and vice versa. Over-optimism was gender dependent and detected in (even above average) boys. Low certainty estimates of mathematically skilled girls, irrespective their solution accuracy, was further found. The verbal explanations will be used to clarify the results.

Developmental associations between MCK, ME and MPS. Correlations between MCK and ME were low, indicating that they capture notably different aspects of metacognition. Preliminary path analyses indicate that ME had a significant direct relation to MPS only within the same task context, but MCK (1^st measurement) had a significant direct and indirect relations to MPS at each time point. Models for boys and girls did not markedly differ. Further analyses will include MCK, ME and MPS at each time point, and these models constitute the main pattern of results presented at the conference.

Conclusions and discussion. In sum, our findings suggest a relatively accurate calibration of judgments already at young age, and indicate context-dependent nature of ME compared to more stable MCK (cf. Efklides et al. 1999). Gender effects showed over-optimism in boys and uncertainty in girls, which may have an important bearing on observed differences between later mathematical interest and proficiency. Our pending multi-level analyses comprising the full longitudinal data will be presented at the conference, and discussed in relation to previous research evidence and educational implications.

Instruction Learning Episodes (ILEPs) are different types of learning tasks, characterized by being either productive or reproductive of nature, by being either knowledge or skill based, by being metacognitive or not, and by aiming at near or far transfer. This study is an improved replication of an earlier study, and it focuses on how different ILEPs within the discipline of math differentially draw on metacognitive skills. Twenty 14-15 yrs old secondary-school students completed a series of probability-calculus tasks while thinking aloud, with each task representing a different ILEP. Metacognitive activities were assessed for each ILEP task through protocol analyses. Results show a differentiation in metacognitive activity between ILEPs.

Learning tasks may differ in various respects, even within one discipline. Elshout-Mohr, Van Hout-Wolters, and Broekkamp (1999) categorized learning tasks on five dimensions: orientation towards (1) cognitive or non-cognitive learning, (2) towards reproduction or production, (3) towards declarative knowledge or skills, (4) towards metacognition or not, (5) towards near or far transfer. Based on this categorization, they described eight types of learning tasks (Instruction Learning Episodes or ILEPs) that regularly occur in the educational practice. These ILEPs are represented by the following scheme:

Learning goals of these ILEPs are: 1) recognition or reproduction of declarative knowledge (facts); 2) making declarative knowledge productive (deep processing); 3) making declarative knowledge productive and transferable (abstraction); 4) reproduction of procedural knowledge or skills (routines); 5) making procedural knowledge or skills (cognitive strategies); 6) making procedural knowledge or skills productive and transferable (expertise); 7) making metacognitive knowledge productive and transferable (reflection); and 8) making metacognitive skills productive and transferable (self regulation). The five dimensions characterize the learning goals or products for each ILEP (product demands). These dimensions, however, do not describe the process demands for each ILEP, that is, which learning activities are used for obtaining the learning product of an ILEP. As metacognition appears to be the most relevant mediator of manifold learning processes (Wang, Haertel & Walberg, 1990), this research focuses on the role of metacognitive skills in attaining learning products of different ILEPs. To be clear: metacognition is not studied here as learning goal and outcome of ILEP 7 and 8, but as a means for attaining learning products in al different ILEPs.

In an earlier study presented at EARLI in Cyprus 2005 we have shown that ILEPs differentially draw on metacognitive skills. We wanted to investigate, however, to what extent these results could be replicated when differences between ILEPs would be endorsed by even more explicit instructions prior to each ILEP task.

Research question

Do different learning tasks (ILEPs) rely to a different extent on specific metacognitive activities? Furthermore, do more explicit instructions for each ILEP task elicit differential metacognitive skills to the same extent as was the case in the earlier study.

Method

Twenty students from secondary education (14-15 yrs old) participated. In two sessions all participants performed a series of seven probability calculus tasks, which corresponded with seven ILEPs. ILEP 6 (expertise) could not be included due to time limits. Each task started with an instruction explicitly stating the product demands of the particular ILEP. For instance, the instruction for ILEP 1 (facs) was: “Read the text so that you can recognize the meaning of important concepts in the text. Afterwards you will have to answer multiple-choice questions about these concepts.” This ILEP instruction was more explicit than in the earlier study, because it referred to the expected outcome of task performance. Depending on the particular ILEP, the learning task consisted of studying text, solving problems, estimation of acquired knowledge, and applying a stepwise action plan. During the learning tasks participants were required to think aloud, which protocols were analyzed later according to the scoring method of Veenman and Elshout (1995). For each ILEP separately, it was judged what the participant’s quality of metacognitive activity was on: 1) Orientation activities (task analysis, paraphrasing, goal setting, activation of prior knowledge), on 2) Planning & systematical orderliness (designing a study or action plan en its systematical execution), on 3) Monitoring & evaluation (comprehension monitoring, detection of mistakes), and on 4) Elaboration & reflection (recapitulating, learning from ones behavior).

Results and conclusions

ANOVAs on metacognitive activities with ILEP task as within-subjects variable show significant overall differences between ILEPs for Orientation, Planning & systematical orderliness, and Monitoring & evaluation, but not for Elaboration & reflection. Orientation was higher for productive relative to reproductive tasks, and higher for skill relative to knowledge tasks. This result was much in line with the earlier study. However, the contrast between declarative knowledge and skills for Planning & systematical orderliness was now restricted to the metacognitive ILEPs, while in the earlier study it also referred to other ILEPs. In line with the earlier study, Monitoring & evaluation appeared to be dominant for skill tasks, and in particular to the acquisition of metacognitive skills. In both studies no significant differences in Elaboration and reflection between ILEPs were found.

            The conclusion that ILEP tasks within the discipline of math differentially draw on metacognitive skills was reconfirmed by the results of this study with more explicit ILEP instructions. Moreover, differences in metacognitive activity express themselves on the dimensional level, not on the level of individual ILEPs. Most results from the present and the former study converged. Only for Planning & systematical orderliness the declarative knowledge vs. skills contrast was restricted to the metacognitive ILEPs in the present study. This study shows what kind of metacognitive skills students in inclined to invest while performing a task with certain demand characteristics.  For instance, students are hardly applying metacognitive skills when performing ILEP 1 (facts), or even ILEP 2 (deep processing). They draw more on metacognitive skills whenever skill acquisition is involved, in particular while performing ILEP 8 (self-regulation). At present, these results cannot be generalized to domains other than math. But they underline that the investment of metacognitive skills may vary depending on task conditions and task demand. Further research should reveal whether such differential application of metacognitive skills also pertains to other domains. 

In this empirical study, we investigate the relationships between self-perceived effort in learning, measured effort in learning, and learning outcomes at the one side, and a range of self-report measures related to achievement motivation, implicit theories about intelligence, and metacognition, at the other side, of university students learning mathematics and statistics. The prime focuses of the study are the investigation of both the dependency of metacognitive self-perceptions on implicit theories, as well as the explanatory power of metacognition for subject specific achievement motivations, which in their turn explain effort and performance. Implicit theories are meaning systems about personal attributes as e.g. intelligence (Dweck, 2000). Prototypical examples are the concept of entity theory, that assumes intelligence to be a fixed, nonmalleable traitlike entity, and the concept of incremental theory, where intelligence is portrayed as something that can be increased through one’s efforts. Students’ metacognitive abilities are operationalised by the recently developed self-report instrument Awareness of Independent Learning Inventory (Elshout-Mohr et al., 2005; Tempelaar, 2006), that presumes metacognition to be a three dimensional construct, comprising knowledge, skills, and attitudes. Expectancy-value models form the basis of an instrument measuring achievement motivations and self-perceived effort (Schau et al, 1995; Tempelaar et al, 2007). Schau’s expectancy-value model distinguishes two expectancy factors dealing with students’ beliefs about their own ability and perceived task difficulty, a construct expressing subjective task-value, an affective task-related attitude, and the constructs interest and effort. Both achievement motivations and self-perceived effort are measured ex ante and ex post, in order to be able to observe developments during the learning episode. The relationships are investigated using structural equation modelling. Subjects in this study are 1500 first year students in an economics or business program, participating in an introductory course mathematics and statistics.

Summary

Metacognitive abilities are an important determinant of learning effort, both with regard to estimated or perceived effort, as with regard to actual (measured) effort (Efklides et al., 2006). At the same time, social-cognitive theories on self-perceptions of intelligence indicate that different implicit theories correspond to different views on the role of effort in learning. According to Dweck (1999), people develop at a very young age self-theories: meaning systems, mostly implicit in nature, about their personal attributes. These views about e.g. intelligence can take radically different forms; Dweck distinguishes the concept of entity theory, that assumes intelligence to be a fixed, nonmalleable traitlike entity, and the concept of incremental theory, where intelligence is portrayed as something that can be increased through one’s efforts. Connected with both views on the nature of intelligence, are views on the role of effort in learning. The entity theorist will regard effort as a negative thing, since it signals a lack of intelligence, being of fixed size. In contrast, the incrementalist will regard effort as a positive thing, since it is the only means to increase intelligence.

In this empirical study, we combine the social-cognitive model on the role of effort in learning (Dweck, 1999) with cognitive conceptions based on metacognition research (Efklides et al., 2006). We do so by investigating the relationships between self-perceived effort in learning, measured effort in learning, and learning outcomes at the one side, and a range of self-report measures related to achievement motivation, implicit theories about intelligence, and metacognition, at the other side, of university students learning mathematics and statistics. The prime focuses of the study are the investigation of both the dependency of metacgontive self-perceptions on implicit theories, as well as the explanatory power of metacognition for subject specific achievement motivations, which in their turn explain effort and performance. Implicit theories are measured with scales developed by Dweck (1999). Students’ metacognitive abilities are operationalised by the recently developed self-report instrument Awareness of Independent Learning Inventory (Elshout-Mohr et al., 2005; Tempelaar, 2006), that presumes metacognition to be a three dimensional construct, comprising knowledge, skills, and attitudes. Expectancy-value models form the basis of an instrument measuring achievement motivations and self-perceived effort (Schau et al, 1995; Tempelaar et al, 2007). Schau’s version of the modern expectancy-value model distinguishes two expectancy factors dealing with students’ beliefs about their own ability and perceived task difficulty, a construct expressing subjective task-value, an affective task-related attitude, and the constructs interest and effort. Both achievement motivations and self-perceived effort are measured ex ante and ex post, in order to be able to observe developments during the learning episode. Actual effort is generated as log-data by an electronic tutorial system used in the course as a replacement of practicals, and thus measure a specific component of students’ learning efforts. 

The relationships are investigated using structural equation modelling, assuming the following causal structure amongst measured constructs: implicit theories => metacognition => ex ante achievement motivations => planned effort => actual effort => ex post achievement motivations => perceived effort & performance. Subjects in this study are 1500 first year students in an economics and business program, participating in an introductory course mathematics and statistics.

References

Dweck, C.S. (1999). Self-theories: Their role in motivation, personality, and development. Philadelphia: Psychology press.

Efklides, A.; Kourkoulou, A.; Mitsiou, F.; & Ziliaskopoulou, D. (2006). Metacognitive knowledge of effort, personality factors, and mood state: their relationship with effort-related experiences. Metacognition and Learning, 1, 33-49.

Elshout-Mohr, M., Meijer, J., Daalen-Kapteijns, M. v.  Meeus, W., & Tempelaar, D. (2005). Construction and validation of a questionnaire on metacognition. Manuscript submitted for publication.

Schau, C.; Stevens, J., Dauphinee, T. L., & Del Vecchio, A. (1995). The development and validation of the Survey of Attitudes Toward Statistics. Educational and psychological measurement, 55, 868-875.

Tempelaar, D. T. (2006). The role of metacognition in business education. Industry and Higher Education, forthcoming.

Tempelaar, D. T., Gijselaers, W. H., Schim van der Loeff, S., & Nijhuis, J. G. (2007). A structural equation model analyzing the relationship of student personality factors and achievement motivations, in a range of academic subjects. Contemporary Educational Psychology, forthcoming.

The study aims to develop a scale which indicates the ascending levels of meta-cognitive knowledge in achieving a deep approach to mathematics learning. The contents of meta-cognitive knowledge were obtained by interviewing 65 Grade-5 children in Taiwan for their perceptions of the meanings of mathematics and learning mathematics. The contents were grouped into 14 themes, for each of which scale items were designed. The scale, with an additional six items of deep approaches, was filled in by 667 Grades 5-7 students. A factor analysis identified six factors. Correlations between the respective six factors and the deep approach were used to determine the levels of each factor, with the largest correlation being the highest/sixth level/factor. The analysis results show that each level contains the components of motivation and strategy. The components for motivation in Levels 1-6 were low confidence, destiny, aspiration/attainment, vocation, liberty, and interest in/beauty of mathematics. The components for strategy in Levels 1-6 were anxiety, avoidance, effort, pragmatism, creation, and understanding. Levels 1-2 showed negative relationships with students’ mathematics achievement, Chinese achievement, and perceptions of pleasant learning in mathematics classroom, while Levels 3-6 showed positive relationships respectively. The present findings broaden the knowledge of the interaction between motivation and strategy in meta-cognitive knowledge. It also identifies a step-by-step process in achieving a deep approach to mathematics learning, and therefore can elaborate the theories of learning approaches. Based on the understanding of students’ levels of meta-cognitive knowledge in mathematics classroom, teachers can design pedagogies which not only deal with the inhibitions found in motivation and use of strategy at the lower levels, but also to cultivate the presence of the more positive motivation and strategy at the higher levels so as to achieve deep learning approaches and achievement.

Aim. In addition to attainment, the deep-learning approach appears to be another important indicator of learning outcomes in mathematics. The study therefore aims to develop a scale which indicates the ascending levels of meta-cognitive knowledge in achieving a deep approach to mathematics learning. Whilst most studies or theories of learning approaches focus on issues of dichotomy or trichotomy as the framework of such approaches, the step-by-step process to achieve deep approaches can be clarified or elaborated by studies of meta-cognition, with its significant focus on the contents and mechanism of higher-order thinking and meaningful learning.

Methodology. A research design of mixed qualitative and quantitative research methods were utilized to collect and analyze data. In the qualitative study, the participants were 64 Grade-5 students from four classes in a primary school in Taiwan. They were chosen by balancing their classes, gender, and mathematics achievement, and interviewed individually. The interview questions were: ‘What is mathematics?’ ‘What does mathematics mean to you?’ ‘What do you mean by learning mathematics?’ and ‘What is mathematics look like?’ The students were first asked the initial question, and then asked the other questions alternatively, in order to make sure that all related issues had been raised by the students. All the interviews were conducted by the author and audio-recorded. In the development of instrument, two researchers categorized the students’ replies to the interview questions into 14 themes: Confidence, destiny, aspiration/attainment, vocation, liberty, interest, beauty of mathematics, anxiety, avoidance, effort, pragmatism, creation, and understanding. Disagreement was resolved by discussion. Based on the 14 themes and related literature, 31 survey items for each theme were created. A further six items related to deep approaches to learning mathematics, and two items related to the pleasantness of learning in a mathematics classroom. The 40 items, in total, were placed at a random order in the questionnaire. In the quantitative study, the participants were 667 Grades 5-7 students from five classes in two primary schools and 15 classes from two junior high schools in Taiwan. The classes were all mixed ability. They filled in the questionnaire under their teachers’ supervision. The students’ achievement scores of mathematics and Chinese were collected from their last school examination. The achievement scores were standardized into Z scores, for each class.

Findings. A factor analysis was performed to establish the validity of the designed items, and select proper items; this procedure formed a 29-item scale and identified six factors/subscales. Correlations between the respective six factors and the deep approach were used to determine the levels of each factor, with the largest correlation being the highest/sixth level/factor. The coefficients of internal reliability (Cronbach’s alpha) for Levels 1-6 were .80, .59, .85, .81, .84, and .89 respectively. Each level contains the components of motivation and strategy. The motivation components of Levels 1-6 were low confidence, destiny, aspiration/attainment, vocation, liberty, and interest in/beauty of mathematics. The strategy components of Levels 1-6 were anxiety, avoidance, effort, pragmatism, creation, and understanding. The correlations were calculated between the respective six levels and the respective three other student characteristics: mathematics achievement, Chinese achievement, and perceptions of pleasantness of learning in the mathematics classroom. Levels 1-2 were found negatively related to the three student characteristics, while Levels 3-6 had positive relationships with the three student characteristics respectively.

Theoretical and educational significance of the research. The present findings broaden the knowledge of the interaction between motivation and strategy use of meta-cognitive knowledge. They also identify a step-by-step process in order to achieve a deep approach to mathematics learning, and therefore elaborate the theories of learning approaches. Based on the findings, teachers can deepen their understanding of the reasons for a students’ use of a particular strategy, or judge at what levels of meta-cognitive knowledge students are working. Based on such understanding, teachers may be encouraged to design teaching activities or initiate classroom dialogues which not only deal with inhibitions found in the use of motivation and use of strategies in the lower levels, but also cultivate the positive aspects of the use in the higher levels, in order to achieve deep learning approaches and achievement.