Not often is the nature of learning and instruction 
explored by the means of qualitative research 
methodologies. This is surprising as qualitative 
approaches can provide a variety of perspectives on, 
and in-depth understanding of, student knowledge and 
learning processes in classroom settings. The aim of this
 symposium is to compare and contrast four different 
qualitative methods in the exploration of student 
knowledge and learning. The first method focuses on 
software-based concept mapping as an additional 
interview step to allow for data reduction and member 
checking with the interviewee at the same time. Second, 
“intentional analysis” is introduced as a method that
 aims at descriptions of through what context an object, 
question or task is interpreted by the student. Third, the 
study of qualitative differences in student learning is 
investigated with a “phenomenographic approach”. 
Finally, a method of video analysis is demonstrated that 
taps into verbal and gesture production in processes of 
science learning. All four methods will be discussed from 
qualitative and quantitative perspectives and will 
indicate how they can inform our field of learning and 
instruction.

The aim of this paper is to report from a study on conceptual change with regard to the shape of the earth over an extended period. For this purpose, seven children have been studied through three subsequent interviews. The first interview was conducted individually when the children were four years old. A couple of weeks later, six of the children were interviewed in pairs with a globe as a point of reference. A year later, each child was interviewed individually again. In total, seventeen interviews have been analysed.

The children came from a Montessori class and had the Earth and the solar system as an ongoing project. Thus, they had got a lot of information about the solar system. The aim of the study is to describe the ways they structure this information.

The data were interpreted and analysed using “intentional analysis”. A basis is taken in Donald Davidson’s theory of meaning, basically the ideas of ‘triangulation’ and ‘the principle of charity’. Such analysis aims at accounting for cognitive functioning as well as situational aspects.

The results concern children’s meaning making, described as the problems children encounter when engaged in discussions about the Earth. Using triangulation for identifying discourse and cognitive oriented recourses made it possible to model children’s meaning making. It was  possible to account for situational aspects in terms of how these were conceptualised by the children as well as their cognitive resources in terms of the models they constructed.

Background

Since the 1970s, it has been acknowledged in research that when children enter school they often have formed alternative frameworks of the subject taught grounded in their everyday experience. Alternative frameworks have been found conflict with ideas taught in school. Research in this area aims at describing such alternative ideas and beliefs and investigating the way they influence learning and how they relate to conceptual development (Driver & Easley, 1978).

Massive critiques have been raised towards studies within this field of research. One such critique concerns the clinical method developed by Jean Piaget (1929/1951) often used in studies on alternative frameworks. It is stated that the method rests on the assumption that language mirrors conceptions and conceptual structures in an unproblematic way and that the role of situational aspects and artefacts in learning and communication is not taken into account (e.g., Schoultz, Säljö, & Wyndhamn, 2001).

In a study of children’s conceptions of the Earth (Halldén, Petersson, Scheja, Ehrlén, Haglund, Österlind, and Stenlund, 2002) it was admitted that artefacts play an important role in children’s reasoning. Nevertheless, these did not totally shape and constrain communication. It was concluded that it is possible to make inferences about children’s conceptions from interview data. The data was interpreted from an intentional perspective. In the study, descriptions were presented of the ways children construct and invent ‘compounded models’ of the Earth using information from different sources (for a detailed discussion, see Halldén et al., 2002).

The study

The aim of the present study is to capture the process of conceptual change with regard to the shape of the Earth over an extended period of time (c.f., Halldén, Larsson, & Haglund, in press). For this purpose, seven children have been studied through three subsequent interviews. The first interview was conducted individually when the children were four years old. A couple of weeks later, six of the children were interviewed in pairs with a globe as a point of reference. A year later, each child was interviewed individually again. Thus, in total, seventeen  interviews have been analysed.

The children came from a Montessori class and had the Earth and the solar system as an ongoing project. Thus, they had got a lot of information about the solar system. The aim of the study is to describe how they structure this information. Thus, which problems do the children actualise and how do they solve them in the different interviews?

Method of analysis

The data were interpreted and analysed from an intentional perspective. Such analysis aims at descriptions of  into what context an object, question or task is interpreted by the child (c.f., Halldén, 1999; Scheja, 2004). This approach makes use of an analytical distinction between the child’s cognitions and his/her conceptions of situational features. Interpreting then, means moving back and forth between hypothesised cognitive and discourse oriented resources respectively (Halldén, Haglund & Strömdahl in press).  

In hypothesising about discourse and cognitive oriented resources, a basis is taken in Donald Davidson’s theory of meaning; basically the ideas of ‘triangulation’ and ‘the principle of charity’ (1984). Utterances are regarded as acts and ascribing meaning to acts is according to Davidson (1984) a triangulation between a speaker, an interpreter and a point of reference in a shared world. In triangulation a necessary conditions is the assumption of some kind of coherence with regard to the speaker, that is, the principle of charity

Results

The results concern children’s meaning making, described as the problems children encounter when engaged in discussions about the Earth. Using triangulation for identifying discourse and cognitive oriented recourses made it possible to model children’s meaning making. Thus it was possible to account for situational aspects in terms of how these were conceptualised by the children as well as their cognitive resources in terms of the models they constructed.



References

Davidson, D. (1984). Inquiries into truth and interpretation. Oxford: Oxford University Press.

Driver, R., & Easley, J. (1978). Pupils and paradigms: A review of literature related to concept development in adolescent students. Studies in Science Education, 5, 61-84.

Halldén, O. (1999), Conceptual change and contextualization, In W. Schnotz, S. Vosniadou, & M. Carretero, New perspectives on Conceptual change. (pp.53-65). Amsterdam, Pergamon.

Halldén, O., Haglund, L. & Strömdahl, H (in press). Conceptions and contexts. On the interpretation of interviews and observational data. To appear in Educational Psychologist

Halldén, O., Larsson, Å., & Haglund, L. (manuscript). On the emergence of a conception: Metaphors and models in research on conceptual change.

Halldén, O., Petersson, G., Scheja, M., Ehrlén, K., Haglund, L., Österlind, K., & Stenlund, A. (2002). Situating the question of conceptual change. In M. Limón and L. Mason (Eds.), Reconsidering conceptual change: Issues in theory and practice, 137-148. Dordrecht: Kluwer Academic Publishers.

Piaget, J. (1929/1951). The child’s conception of the world. Maryland: Littlefield Adams  Quality Paperbacks.

Scheja. M. (2006). Delayed understanding and staying in phase: Students’ perceptions of       their study situation. Higher Education, 52, 421-445.

Schoultz, J., Säljö, R., & Wyndhamn, J. (2001). Heavenly talk: Discourse, artefacts, and children's understanding of elementary astronomy. Human Development, 44, 103-118.

Our approach to describing qualitative differences in learning focuses very much on differences in meaning, regardless whether these differences are differences within or between individuals. We are thus not primarily interested in differences between individuals, and we do not make any assumptions about the stability or generalizability of the meanings expressed by them. We are simply interested in what different meanings of a certain phenomenon that can be found. So, how do we find these differing meanings?

The answers given by the students to our (very) open-ended questions make up the “pool of meaning”. At the word level, all the answers are different, but we are looking for answers that differ at the word level, but not in terms of meaning, and which differ from other answers both at the word level and in terms of meaning. We are looking for critical differences between answers. These differences are supposed to be critical in relation to the object of learning (what the students are expected to learn). The object of learning is not defined as clearly in the beginning of the analysis as it is later on. In a way the object of learning gets defined through our efforts to relate different answers to it.

From a pedagogical point of view, however, it might be more interesting to know what the learners can do than what they have in their minds. What does it take to know "Why certain things are expensive and others cheap”, for instance? Is it primarily “knowing that” or “knowing how”? (cf. Ryle, 1949) We would suggest in accordance with Broudy (1988) that it is primarily “knowing with” or even better: “seeing with”. A conceptual entity like “price as a function of demand and supply” is understood best as a powerful tool for seeing a certain class of phenomena. The learner has to discern variation in the price of, demand for, and supply of, a product. This means that we are not trying to look into the minds of the learners to figure out what kind of mental representations reside there; rather, we are trying to see what the learners see. “What the learners see” amounts to what they discern and focus on simultaneously.

What does it take to see something in a certain way? Our point of departure is that there is potentially more information in everything we encounter than we can possibly receive. This is why attention must necessarily be selective (Gibson & Gibson, 1955).  If we were able to extract all the information there is in a certain situation, we would, of course, see it in the same way. But we do not. In our own research tradition it has been shown in countless studies that whatever people encounter, they see it in a number of qualitatively different ways (Marton & Booth, 1997). With reference to the example of “price as a function of demand and supply” we can argue that although each answer to a certain question might be unique, on another level we find a limited number of qualitatively different ways of responding to the question. That other level is the level of meaning or the level of “ways of seeing”.

We thus make a distinction between “the meaning” and “the expression”: the same meaning can be expressed in different ways. Furthermore, meaning is defined in terms of what is discerned and focused on simultaneously. In the above example we refer to how price is seen as a function of the characteristics of the product, of the demand for it, of the supply of it, or of both demand and supply. For the latter, the students have to discern variation in supply and variation in demand and focus on both simultaneously.

What this example points to is that perception is seen as discernment (and not construction, for instance), that our concern is primarily the differences between different “ways of seeing”. If we apply our way of reasoning to the question “what changes in conceptual change?”, for instance, our answer is different from the answers suggested by other theorists. In our view it is the world experienced, the world seen, the world lived that changes. The change is outside the individual (as seen from their own perspective) so to speak; it is in the world that surrounds them. But to the extent that the world remains the same to an observer, the learner must have changed in order to see the same world differently. Yes, indeed, the learner demonstrates that they are not only capable of seeing something in one way (as they did on the first occasion), but also capable of seeing the same thing in another way (as they do on the second occasion).  How can we define a way of seeing, then? “A way of seeing something” can be defined in terms of what features are discerned and focused on simultaneously, but what does it take to discern a feature? According to the line of reasoning developed elsewhere (see Marton & Tsui, 2004), discerning a feature amounts to noticing certain kinds of differences, and noticing certain kinds of differences amounts to making certain distinctions. Therefore, the learner shows that they are capable of seeing something in a certain way by making certain distinctions. We do not ask the question: Do they really see something in the way they claim seeing it? Instead, we ask the question: What way of seeing something is expressed by making certain distinctions? In this way we can describe the different ways in which other people are capable of seeing something. We can describe what they (possibly) see. We can describe the seen, experienced, lived world, which is actually our object of research. People do not see, or experience, mental representations; they see, or experience, things in the world, and so if we want to describe their experiences, we have to describe how the world appears to them. Accordingly, what it is that changes in conceptual change is the world perceived and the learner’s capability of perceiving it. But these two things are actually two sides of the same thing: the experience of the world and the experienced world.

The presentation will be focused on the methodology of analysis of classroom productions (utterances and gestures of teacher and students). The chosen perspective is the knowledge involved in classroom practices. We consider that the taught knowledge is staged in the classroom by teacher and the students’ joint action. Following Grice, we distinguish between conventional (taught knowledge) and understood meanings (students’ or teacher’s understanding). Here, we propose a methodology to analyse these types of knowledge: taught knowledge and knowledge understood by the students or the teacher at two time scales, the meso scale (about ten minutes) at which ideas are developed in the class, and the micro scale (about seconds), which is the scale of interactions between individuals (teacher and students). The collected data, which this methodology works out, consist of several hours of video recordings of physics and chemistry classrooms at 10^th  and 12^th grades during teaching sequences on a specific part of the official curriculum (in physics and in chemistry). The meso scale analysis is based on a thematic approach. The discursive productions (including gestures) can be divided into units of about ten minutes. The micro scale analysis is based on a decomposition of knowledge into “facets”. Processing facets (without recomposing them to get a concept meaning for example) leads the researcher to characterize classroom according to the density and continuity of the taught knowledge. Treatments involving meso and micro levels allow the researcher to compare this taught knowledge to the meanings constructed by the students.

This presentation will be focused on the methodology of analysis of classroom productions (teacher and students’ utterances and gestures). This methodology comes within the scope of relating teaching, students’ understanding and learning. The chosen perspective is the knowledge involved in classroom. The taught knowledge is staged in the classroom by a joint action of the teacher and the students. Following Grice, conventional and understood meanings are distinguished. On one hand, the conventional meaning, which is the taught knowledge, is reconstructed by the researcher from: (1) disciplinary knowledge (mainly the knowledge to be taught: official curriculum, textbooks and if necessary more advanced disciplinary knowledge) and (2) classroom practice, for example when the teacher proposes formulation like” we’ll not say force exerted by the system X on the system Y any more but force of X on Y”. Consequently, the reconstruction of the taught knowledge (conventional meaning) involves an external referent which is the disciplinary knowledge. On the other hand, the knowledge understood by a student in the classroom is reconstructed from the student’s perspective; in this case the referent is internal however this reconstruction necessitates taking into account the student’s “history”. Student’s meanings and taught knowledge can be different even if they are reconstructed by the researcher from productions done in the same period of time and in the same classroom.

We propose a methodology to analyse these types of knowledge: taught knowledge and knowledge understood by the students (or the teacher) at two time scales, the meso scale, at which ideas are developed in the class (about ten minutes), and the micro scale, which is the scale between class participants’ interactions (about seconds).

The collected data consist of video recordings of physics and chemistry classrooms at 10^th and 12^th grades during a teaching sequence on a specific part of the official curriculum. Ten hours were collected in physics (mechanics) and the same amount in chemistry (chemical equilibrium). Other data like questionnaires, students’ written productions, teacher’s interviews were also collected. This presentation will deal with video analysis.

The meso scale analysis is based on a thematic approach. The discursive productions (with gestures) can be divided into units of about ten minutes called themes. Themes have a structure with frontiers (most of them have an introduction and a conclusion with words like “look”, “then”, “that’s all”, or a silence and/or a change of teacher’s position, or an action such erasing the blackboard, etc.), and a thematic coherence (most of utterances are related to a same theme). Themes structure the taught knowledge and give precise information on its evolution with time (called chronogenesis). This structure is directly elaborated from video data and their transcriptions. Such a structure makes comparison possible between knowledge that is taught in different classrooms about a similar part of an official curriculum in terms of order and duration.

The micro scale analysis is based on a decomposition of knowledge into “facets”. A facet is an element of knowledge (or strategy) that can be used by the teacher or the student in a given situation which cannot be decomposed without loosing its meaning (smallest semantic unit). It is formulated as a sentence. In our case a facet is constructed a priori and a posteriori. The a priori construction is based on the knowledge to be taught or the student’s conceptions analyses A facet can be a correct or incorrect knowledge. A posteriori, during the video data analysis (teacher or students’ productions), some facets can be constructed to take into account a new element of knowledge or strategy involved in teacher or student’s production. This analysis is also directly elaborated from the video data and their transcriptions. In the case of taught knowledge, we distinguish three uses of facets: an element of knowledge (1) introduced for the first time in the classroom, (2) re-used in a new field of application, (3) other re-use. This systematic analysis leads us to get sets of elements of knowledge or strategy on a theme, a session, or a whole sequence. Similar analysis can be done for students’ facets. These aspects of re-use are relevant to relate teaching and learning.

Different treatments can be done.

A first treatment is carried out only from facet analysis at micro scale for taught knowledge. It concerns the whole sequence: the number of new facets in relation with duration (density of new knowledge), the re-used facets of a certain type (those concerning force or relations force-modification of the velocity, etc.) in different classrooms for a teaching sequence on the same part of the official curriculum (continuity of knowledge). Density and continuity of the taught knowledge are both relevant to differentiate classroom practices. This treatment is directly done with facets without recomposing them.

The second treatment articulates analyses at the meso and micro scales. Recomposing the facets for each theme allows us (1) to check the coherence between the analyses at both scales for taught knowledge and students’ understanding as well; (2) to compare the difference of meanings between taught knowledge and students’ understanding at a same period or at different periods of time.

Our results concern the relevance of the methodology which allows the researcher to compare classrooms practice from the knowledge perspective and students’ understanding.

The focus of this presentation is to introduce concept mapping as second step of the interview process that allows for data reduction and member check with the interviewee simultaneously. Concept mapping software (i.e., IHMC Cmap Tool software) is used as essential part of this method to account for computer based-data collection, which in turn allows for the efficient use of qualitative data analysis software. The data collection entails two steps: (1) During the interview, the interviewer captures and reduces the interviewee’s answers in writing. The concept map software is used in this process; for each (smallest) semantic unit of an answer a digital card is created (concept) which contains the written reduction of the semantic unit. (2) The digital cards are connected with links and links are provided with thematic labels. It is the interviewee who is making the decision which cards should be linked with each other and which theme should be used as label. This linking and labeling process is used to member check (validate) with the interviewee the reduction of his/her answers (content of the digital cards) and to make corrections if necessary. The graphic files of the created concept maps can be later uploaded into software programs for the analysis of qualitative data (e.g., ATLAS.ti).

This method was successfully used in two interview studies with fourth and sixth graders exploring their personal epistemology (i.e., conceptions about knowledge and knowing). 140 concept maps were generated following this method of data reduction and member checking. The methodologically comparison with traditional interview methods revealed that the presented methods is more accurate in the reduction process and outcome of interview data. My presentation will entail a demonstration of this new method as it was used in these two studies and the methodological comparison with traditional interview methods and analysis.

Introduction

The aim of this presentation is to demonstrate how software based concept mapping can be added to an interview process to allow for data reduction and member checking with the participant simultaneously. Furthermore, a methodological comparison of this technique with traditional forms of interview analysis will be part of the presentation. This new technique was successfully implemented in two interview studies on children’s personal epistemology (i.e., conceptions about knowledge and knowing) and the results look very promising.



Research background: Two interview studies on children’s personal epistemology

Personal epistemology, the conceptions of individuals about knowledge and knowing play a crucial role in various aspects of cognitive processes and learning, such as argumentation, problem-solving, and conceptual change learning. Despite this importance, children’s personal epistemology is rarely subject to empirical research.

The first interview study was conducted with ninety-eight fourth graders in German elementary schools. The analysis revealed not only that children around the age of ten are able to verbalize epistemic beliefs, but that the latter are more diverse and profound than initially expected by the research community, such as different knowledge definitions, beliefs about the origin, acquisition, and verification of knowledge.

In the second study, twenty fourth and sixth grade students in the U.S. were interviewed regarding their conceptions about knowledge and knowing in science, mathematics, and reading. The interview questions were anchored in previous lessons in these school subjects. The analyses revealed that students’ personal epistemology is domain-specific and differs according to the grade level.

In both studies it was demonstrated that children held similar epistemological positions as adults, such as absolutist, multiplists, and evalutivist (Chandler, Hallet, & Sokol, 2002; Kuhn & Weinstock, 2002); except that these are based on a less broad and abstract conceptualization of knowledge. Furthermore, the results imply that teachers’ personal epistemology and classroom education in general have an impact on students’ personal epistemology.

Method: Concept mapping as a method of data reduction and member checking

The data collection process encompassed two steps. First, the semi-structured interview was conducted. While the participants answered the posed questions their responses were recorded by the researcher. For this, the interface of the IHMC Cmap Tools software was used to create digital cards on which the participants’ answers were put in writing. This process represented a form of data reduction. Complex and narrative answers were summarized in the form of complete sentences, such as an exploration of the origin of knowledge. Enumerations and simple descriptions were summarized in headwords and/or reduced sentences, such as “I know how to swim, count, write, etc.” Participants had full visual access to the recording of their answers, as they were purposefully seated next to the researcher. This data reduction was verbally verified with the participant in the subsequent step.

The second step of the data collection was described as concept mapping. Here, the reduced data on the digital cards were verbally validated (i.e., member check). That is, the text on each card was read aloud to the participants who, interestingly, were all keen to read the text simultaneously. Then, the digital cards were connected with links and links were provided with thematic labels. It was the interviewee who was making the decision which cards should be linked with each other and which theme should be used as label. This linking and labeling process is used to member check (validate) with the interviewee the reduction of his/her answers (content of the digital cards) and to make corrections if necessary. It is noteworthy, that the interviewees paid remarkable attention to the purpose of this process. They did not hesitate to correct the reductions. For example, one student corrected the reduction from “She maybe learned it in kindergarten” to “She maybe learned it in a good kindergarten”. Their attention was also evident in pointing out spelling mistakes to the researcher. The linking of the cards and labeling of the links on the software’s interface were implemented by the researcher. This process of concept mapping allowed getting into a dialog of verbal validation that was beyond a simple yes- or no- answer. The data collection with the participants took approximately twenty-five to thirty minutes.

Conclusion

Concept mapping as an additional interview step can account for both the reduction of data and member checking with research participants simultaneously. This method was successfully used in two different interview studies with fourth and sixth grade students. Its effectiveness was methodologically compared with traditional interview analysis. The methodologically comparison with traditional interview methods revealed that the presented methods is more accurate in the reduction outcome of interview data and, therefore, looks promising. This method also allows for a totally computer-supported data collection and analysis (e.g., IHMC Cmap Tool software; ATLAS.ti, software used for the analysis of qualitative data). This, in turn, allows the conduct of large-scale qualitative research study as larger amounts of qualitative data can be handled more effectively.

References

Chandler, M. J., Hallett, D., & Sokol, B. W. (2002). Competing claims about competing knowledge claims. In B. K. Hofer & P.R. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing (pp. 145-168). Mahwah, NJ: Lawrence Erlbaum Associates.

Kuhn, D. & Weinstock, M. (2002). What is epistemological thinking and why does it matter? In B.K. Hofer & P.R. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing (pp. 121-144). Mahwah, NJ: Lawrence Erlbaum Associates.