| Abstract |
With the success of computer and video games, educators develop interest in using the motivational aspect of games for educational purposes. Mixing educational objectives without loosing the gaming experience is a hot topic of research and development. The E.U. ELEKTRA project aims to develop a game including a pedagogical background and a rich technological environment game in terms of visual design, interaction possibilities, and storytelling. One of the pedagogical challenges is the support of metacognitive activity, including the use of certitude degrees. In the game the learner-player will have to solve enigma gaining knowledge and understanding about eclipses, the theme of the game. The player will also have to gain the confidence of a non playing character. It is that confidence that will be measured with certitude degrees about solving the enigma. This paper will present a proposition for mixing metacognition and game play. It will also present an experiment regarding the formulation of the certitude on a scale to be used by the player. It was suggested not to use verbal expressions (like sure, not sure,...) but to use subjective probabilities (like 100%, 80%,...) instead. We organised an experiment proving the lack of reliability when using verbal expressions. In this experiment 23 subjects passed a double writing test about grammar in their native language. They were asked to write down a spoken missing word to be written within a typed sentence and to associate their certitude having the correct spelling. The first time they used a verbal expressions to describe their certitude. The second time they used subjective probabilities. The experiment showed that verbal expressions can take different values from person to person but also from question to question for the same person. The metacognitive activity will be based on dialogues based on the good use of those judgements. |
| Summary |
INTRODUCTION One of ELEKTRA’s E.U. project aims is to ban games with sequences of pure gaming or playing followed by sequences of pure teaching. In the ELEKTRA game the learner-player will have to solve enigma gaining knowledge and understanding about eclipses, the educational theme of the game. The player will also have to gain the confidence of a non playing character (NPC). It is that confidence that will be measured with certitude degrees about solving the enigma. The game will provide two numerical metacognitive feed-backs based on Leclercq & Poumay (2003) works: the mean certitude degree of correct answers and the mean certitude degree of incorrect answers. They will be respectively named confidence and imprudence. The player will have to have a confidence above 50% and a imprudence below 50%. Metacognitive feed backs will also be provided by the NPC. AIMS To collect the certitude degree the learner-player associate with his answer, we could use verbal expression (sure, not sure,...) or numerical expression (0%, 20%,...100%). Based on Brown & Shuford (1973), Shuford, Albert & Massengill (1966), De Finetti (1965, 1970) and Leclercq (1982) it was suggested to use subjective probabilities (like 100%, 80%,...). “A subjective probability or personal probability, describes an individual’s personal judgement about how likely a particular event is to occur” (Easton & McColl, 1997). “It is not based on any precise computation but is often a reasonable assessment by a knowledgeable person.” A person’s subjective probability of an event describes his/her degree of belief in an event. To achieve the game the player will not only solve enigma but to solve them with a well made estimation of the success rate for each attempt made to try to solve that enigma. In other word the player must use the confidence degrees with realism. This means associate certitude below 50% with the failed attempts and certitude above 50% with the successful attempts. And this before having the result of each attempt. This will be like betting on the result of each attempt. Using verbal expression like ‘sure’ and ‘not sure’ might fit better in the game flow but will end in unusable judgements. To prove this an experiment proving the lack of reliability of verbal expressions was organised. METHODOLOGY In this experiment 23 subjects passed a double writing test about grammar in their native language. They received a sheet with 20 sentences each with a missing word. The complete sentence was red aloud. Each participant was asked to write down the missing word. With every word each participant was asked to give his certitude of having the correct spelling of the word using the verbal expression ‘sure’ or ‘not sure’. We collected all answers. We also collected information of feelings and methods used by the participants when using verbal expressions. Then a second test was organised. The same 20 sentences but this time certitude of having the correct spelling for each word was to be given with subjective probabilities : 0%, 20%, 40%, 60%, 80% and 100%. When the given answer changed from the first to the second test the result were disregarded. FINDINGS When using the verbal expression ‘sure’ the participants associated certitude ranging from 40% to 100%. They were only 3 times out of 65 when 0% and 20% were used. The mean certitude when using the verbal expression ‘sure’ was 74%. When using the verbal expression ‘not sure’ the certitude associated by the participants ranged also from 0% to 100%. There was no significant variation between the use of 20%, 40%, 60%, 80% and 100%. This proves the lack of reliability when using verbal expressions instead of subjective probabilities. The qualitative survey about the use of verbal expressions concluded that even if a participant defines general rules for using ‘sure’ and ‘not sure’ during a test those rules are different from person to person. Even if rules were defined participants admit not to follow them for every question. Thus they are different rules or values associated with verbal expressions like ‘sure’ and ‘not sure’ and those values change from person to person and even from question to question for the same person. THEORETICAL AND EDUCATIONAL SIGNIFICANCE OF THE RESEARCH The lack of reliability of verbal expressions to measure the certitude a person associates to an answer is confirmed inter- and intra-person. Verbal expressions like ‘sure’, ‘fairly sure’, ‘poorly sure’, and so on should not be used to evaluated the confidence associated with an answer. More experiment will be conducted to confirm those result and to investigate the stability of those findings regarding the age and the gender of the participants. Within the ELEKTRA E.U. project more research will also take place regarding the effect of using subjective probabilities on game play. The implementation of subjective probabilities will be made as if it was a gambling within the game itself. REFERENCES Brown, T. A. and Shuford, E. H. (1973). Quantifying uncertainty into numerical probabilities for the reporting of intelligence. Number No. R-1185-ARPA, July 1973. De Finetti, B. (1965). Methods of discriminating levels of partial knowledge concerning a test item. Bristish Journal of Math. & Statist. Psychol., 18, 87-123. De Finetti, B. (1970). Logical foundations and measurement of sub jective probability. Acta Psychologica, Amsterdam, 34(2-3):129–145. Easton, V. J., and McColl, J. H. (1997). Statisitic glossary. Retrieved October 23, 2006, from http://www.stats.gla.ac.uk/steps/glossary/probability.html Leclercq, D. (1982). Confidence marking, its use in testing. In Postlethwaite & Choppin (Eds.), Evaluation in education: An international review series (Vol. 6-2, p. 161-287). Oxford: Pergamon. Leclercq, D. and Poumay, M. (2003). La connaissance partielle chez l’apprenant : pourquoi et comment la mesurer. In Gagnayre, E. A.,, L’evaluation de l’education therapeutique du patient. Paris : IPCEM. Shuford, E. H., Albert, A., and Massengill, H. E. (1966, Jun). Admissible probability measurement procedures. Psychometrika, 31 (2), 125-45. |
