Learning environments typically contain devices that are included to support learners (Hannafin, Hall, Land & Hill, 1994). These support devices become especially important when learners have to deal with. However, research indicates that students tend to not use the support offered. Or, when they do use it, they often do so inadequately (see Stahl, Schworm, Fischer, & Wallace, 2003; Clarebout & Elen, 2006). Different reasons can underlie this non- or inadequate use (Perkins, 1985). First, the support itself may be poorly designed and hence not be beneficial for students. Second, students’ may be unfamiliar with the support, they may not know why and how to use the support devices. Third, the use of the support assumes learners to comply to a learning environment. For a variety of reasons learners may be inclined to be non-compliant.

The different papers in this symposium will address factors that may influence the use of support devices. The different papers stress different aspects with respect to this issue: external variables such as training are addressed as well as specific learner characteristics that may influence tool use.

Learners often neglect support (glossaries, help sites etc.) in computer-based learning environments since they experience it as an unrelated add-on. We assumed that prompts presenting situated strategic and metacognitive support (‘situated prompts’) lead to an increased support use resulting in a higher cognitive load of learners, which than causes a lower learning performance if learners have a lower thematic expertise. To scrutinize the differential effect of situated prompts we conducted an experiment with 69 students (undergraduates vs. graduates). Students learned either with a learning environment with or without situated prompts. As expected, learning with situated prompts resulted in an increased support usage. Furthermore, two interaction effects occurred. (1) Graduates learned slightly better with a programme including situated prompts compared to a programme without prompts whereas undergraduates performed better without situated prompts. (2) Undergraduates stated a higher perceived cognitive load if they learned with a program with situated prompts compared to undergraduates learning with a program without situated prompts. In the group of graduates no differences occurred concerning the perceived cognitive load. The results are interpreted within the framework of cognitive load theory.

Recent instructional theories focus on authentic learning environments as an incentive for learning. The general assumption is that authentic learning environments offer the opportunity to apply acquired theoretical knowledge to real life tasks and prevent inert knowledge. Successful self-regulated learning within authentic learning environments imposes high regulatory demands on the learner. Consequently researchers investigated the effectiveness of different support strategies. We focus on two kinds of support strategies in complex learning environments, which have a known positive effect on learning results:

(a) Metacognitive prompts. Metacognitive prompts are instructions integrated in the learning context, which ask learners to carry out specific metacognitive activities.

(b) Strategic prompts. Strategic prompts are purely informative and not related to the learning situation. The learners have to connect the general information to the content-specific task requirements.

Despite strategic and metacognitive prompts being formally integrated in the learning process, learners often characterize them as ‘additional’ information and therefore as ‘optional’, resulting in non-sufficient usage of prompts and ignoring further help sites, which are related to those prompts. One solution for this problem would be the authentic integration of prompts to form a unified learning environment in which these prompts are experienced as an integral part (situated prompts).

Situated prompts can only be realised if the learner works in a fully authentic learning environment which are tailored to increase – in terms of Cognitive Load Theory (Sweller, 1994) - the germane load and hence the learning outcome. Because of learners’ limited cognitive resources in the working memory, a proper cognitive resources allocation is critical to learning, because if a learner is required to devote mental resources to activities not directly linked to information processing, the learning process may be disturbed. However, if a learning environment is set up to be fully authentic it must necessarily have some features that are not relevant for the learning task.

Based on reasoning previously outlined, we assume the following hypothesis. A possibility to reduce the insufficient usage of support would be the authentic integration of prompts. But the authentic integration could lead to a reduction of the learning success if learners have not enough cognitive capacities to process the support and the content of a learning environment adequately at the same time. Particularly the probability of a reduced learning success arises, if learners have a low previous knowledge, because the learning content requires more cognitive resources compared to learners with a higher previous knowledge.

We conducted an experiment to test our hypothesis. First we integrated situated metacognitive and strategic prompts in a complex authentic learning environment for cost and sales accounting called JOKER (JOKER, 2006). Second 67 students in the second and forth year (undergraduates vs. graduates, factor 1) of their economic study programme learned with JOKER. Half of the students of each study level worked with a conventional version whereas the remaining students learned with version containing situated prompts (factor 2). Besides the situated prompts no other differences existed between both versions.

A questionnaire with 26 items about the cost and sales accounting knowledge was administered to the participants one week before they learned with JOKER. After working with JOKER, participants processed a questionnaire containing two measures: one to assess students’ learning outcome and one about students’ ratings referring their perceived cognitive load. The learning outcome was measured by administering the same knowledge questionnaire as in the first testing. To assess objective data about participants’ use of additional information and the study time, log files were measured.

As expected learning with a programme containing situated prompts promoted the usage of further help sites. Using log files, a 2x2-ANOVA showed that students learning with the situated prompts used the help sites more frequently than students learning without situated support (F_help (1, 63) = 8.71, p_help =.004_, h²_help = 12.2).

A second 2x2-ANOVA showed that undergraduates perceived a significantly higher cognitive load than graduates (F (1, 63) = 17.59, p < .001, h² = .218) especially if undergraduates learned with the programme containing situated support (F (1, 63) = 3.02, p = .044, h² = .046).

The results of the learning outcome were examined by a (2x2)x2-ANOVA with repeated measure (within-factor: pre-post-testing). A significant interaction between the within-factor and both between-factors was revealed (F(1, 63) = 3.19, p = .04; h² = .048). Undergraduates using the version with situated prompts had no learning effect, whereas all other groups had a positive learning outcome.

As intended, in case of learning with situated prompts, learners use additional information more frequently. But is to notice, instructional support in authentic learning environments may have a negative effect on learning success if learners have no capacities to process them adequately, especially if learners have low previous knowledge. Therefore it is most important to develop the right balance between instructional guidance and support in complex authentic learning environments.

In this study the effects of visualization tools on argumentation skills, knowledge acquisition, and motivation during hypermedia learning were examined. Participants in this experiment had to complete an argumentation task on environmental issues by using a hypermedia learning environment as resource. In one condition, participants were provided with a graphical concept-mapping tool in order to complete an argumentation task. In a second condition, a simple two-columned word-pad has been given. Results suggest that a graphical argumentation tool can enhance learners’ motivation, but has no influence on knowledge acquisition or quality of arguments. Overall, results reveal that the assignment of an argumentation task to hypermedia learning environments was an effective instructional strategy that led to enhanced knowledge acquisition.

In hypermedia learning environments the use of external representations is an established way to support learners (e.g. Jonassen, 1996; Jonassen, Beissner & Yacci, 1993). There are several functions these tools address like providing overviews, restructuring content or fostering of elaboration processes. In this research, we examine the role of external tools for argumentation tasks as a possible rationale for enhancing and deepening learning with non-linear information media. Providing an argumentation task is one possible strategy for enhancing reflection processes and Critical Thinking (Ennis, 1987; Voss & Means, 1991). For visualization of arguments within computer-based learning environments the use of graphic tools or concept mapping software is very common.

This study was designed in order to analyze the influences of two different tools for visualization of argumentation structures during hypertext-based learning on knowledge acquisition, motivation, and argumentation quality. On one hand a graphical concept-mapping tool, on the other hand a two-columned text-based argumentation tool has been provided. While concept-mapping tools often are preferred due to their semi-graphical representation of information, this possible advantage is more than questionable from a cognitive science point of view. It may rather increase cognitive load compared with basic text-based notation tools although the graphical organization might support organization of one’s argumentation structure.

For this experiment, we developed a hypermedia information system on marine pollution. Participants were randomly assigned to one of two conditions (graphical argumentation tool vs. text-based argumentation tool). All participants had to conduct an argumentative task on marine pollution and oil. They were also advised to collect necessary information in the hypermedia information system. The text based tool had a two-columned interface, one for pro and one for contra arguments. The graphic based tool was operationalised by providing the concept-mapping software Inspiration®. Here, participants were advised to develop their argumentation structure by drawing notes and connecting them with arrows indexed with “+” and “-“ for pro and contra statements. As dependant measures, structural knowledge acquisition (by means of a networking task) and motivation were assessed in pre- and posttest. Furthermore, argumentation quality was assessed.

Results reveal a significant interaction between time of measurement and the kind of argumentation tool. While the graphic based tool seemed to maintain motivation there was a decrease in the group with the text-based tool. While there was no difference between both experimental groups in knowledge acquisition, there was a general increase between the pre- and posttest. Subjects’ arguments were rated on the criteria of “relevance”, “specificity”, and “distinctiveness” (cf. Voss & Means, 1991). An overall scale “quality of argumentation” did also not reveal any significant differences between both argumentation tools.

Results suggest that graphic argument visualization maintained subjects’ motivation although there was a general decrease in motivation regarding the treatment task. This indicates on the one hand, that the given argumentation task was not very challenging and needs to be modified for further uses. On the other hand, the use of the graphic based tool seems to provide a challenging component that probably compensates the motivational decrease with an increased playfulness. Still we need to mention that this effect could possibly be caused by curiosity as well. Overall there was no effect of the independent variable on knowledge acquisition, although a significant increase in knowledge between the pre- and posttest was found. The evaluation of the balance of pro/contra-arguments suggests a clear preference for the text based tool. However, this might be a potential bias caused by its design. All in all the study suggests that the combination of argumentation tasks and learning with hypertext is an effective approach for knowledge acquisition. The use of graphic based tools for external visualization purposes might help to maintain learners’ intrinsic motivation in a challenging and playful way, despite the fact that we could not find a surplus value in cognitive support.

Successful learning is mainly based on metacognitive activities which has to be performed and constantly monitored during learning. Research reveals that many learners have difficulties in performing these metacognitive activities spontaneously which most probably results in lower learning outcomes. So, the key issue of the study is to develop effective metacognitive instructions by means of computer support. Based on earlier research a computer assisted training session was developed and evaluated experimentally. With this support device students are explicitly instructed to activate their repertoire of metacognitive knowledge and skills which should further enhance learning and transfer.

To test this assumption students of the experimental group (n=29) were instructed in a short computer assisted training session lasting about 30 minutes why metacognitive activities are useful and how to apply them during learning. Students of the control group (n=27) were not trained why and how to use metacognitive activities, but rather they were instructed by a computer device how to organize an appropriate learning place, which lasted also 30 minutes. After the training sessions the students’ learning task was to learn about the “psychological theories of using pictures in multimedia learning environments” within 60 minutes. Immediately afterwards learning outcome and transfer were obtained by questionnaire. Altogether 57 university students were participating, counterbalanced according to their prior knowledge as well as  metacognitive knowledge. As expected students of the experimental group showed better transfer performance compared to the control group. However, training did not increase metacognitive and strategic behavior measured by subjective ratings.

Based on recent research in the field of learning and instruction successful learning is not a matter of trial & error but rather a set and specific sequence of metacognitive activities which has to be performed. Ideally, a successful learner performs different metacognitive activities during learning. He first analyses the situation before he starts with the execution of information processing. The learner will orientate by glancing through task, instruction and resources. He will specify the learning goals or even break them down into sub goals and plan the ongoing procedure. Based on this analysis the student has to search for the relevant information and judge whether the information found is really relevant to reach the learning goals. He then has to extract the information (maybe not all information available is necessary to process) and to elaborate it. At the end of the learning he has to evaluate the learning product, again with respect to the learning goals. These activities are constantly monitored and controlled.

Research reveals that many learners have difficulties in performing these metacognitive activities spontaneously which most probably results in lower learning outcomes. So, the key issue of the research project is to develop effective metacognitive instructions by means of computer devices. Based on earlier research a computer assisted training session was developed and evaluated experimentally. With this support device students are explicitly instructed to activate their repertoire of metacognitive knowledge and skills which should further enhance learning and transfer.

 The aim of this study is to investigate experimentally the effects of a computer assisted metacognitive training. Students of the experimental group (n=29) were instructed in a short computer assisted training session why metacognitive activities are useful and how to apply them during learning. Students of the control group (n=27) were not trained why and how to use metacognitive activities, but rather they were instructed also by a computer device how to organize an appropriate learning place. Both training sessions lasted about 30 minutes. After the particular training session the students’ learning task was to learn about the “psychological theories of using pictures in multimedia learning environments” within 60 minutes. Immediately afterwards learning outcome and transfer were obtained by questionnaire. Altogether 57 university students were participating, counterbalanced according to their prior knowledge as well as their metacognitive knowledge.

As expected students of the experimental group showed better learning outcomes compared to the control group. Although there were no significant differences in respect to recall and factual knowledge the metacognitive instructed group showed higher transfer performance (t_1,54=1.778, p₁=.041, d=.56). 

Further, it was hypothesized that metacognitive instruction would increase metacognitive and strategic behavior during learning. To test this assumption the scales of the retrospective questionnaires (a modified version of the LIST-Questionnaire) were analysed with respect to group differences. No treatment effects were found for the metacognitive scales (i.e. Orientation & Planning, and Monitoring). Furthermore, scores for the cognitive learning scales did also not differ significantly (i.e. Organisation, Elaboration, and Rehearsal. However, one has to keep in mind, that these results are based on students’ subjective ratings which turned out to be not a valid metacognitive assessment method at all (Veenman, 2003).

 Hypotheses were partly confirmed by results. Participants who were instructed by a computer device about why and how to use metacognitive activities reached significantly better transfer performance. Results are in line with other research (s. Bannert, 2005, 2006, in press), were different learning materials were used (hypermedia-based as well as other learning topics) and, much more important, different instructional measures (metacognitive prompting) were investigated.

References.

Bannert, M. (2005). Designing metacognitive support for hypermedia learning. In T. Okamoto, D. Albert, T. Honda & F.W. Hesse (eds.). The 2nd Joint Workshop of Cognition and Learning through Media-Communication for Advanced e-Learning (S. 11-16). Sophia University, Tokyo, Japan.

Bannert, M. (2006). Metacognitive Instruction to Improve Self-Directed Learning. Paper presented at the AERA-Annual Meeting Conference, San Francisco, USA.

Bannert, M. (in press). Effects of Reflection Prompts when Learning with Hypermedia. Journal of Educational Computing Research.

Veenman, M. V. (2003). The assessment of metacognitive skills: What can be learned from multi-method designs? 10th Euro­pean Conference for Research on Learning and Instruction, August 28 – September 1, 2003, Padua, Italy.

Introduction

This research addresses issues in the design of online tool support within collaborative scientific inquiry learning environments. These environments basically enable students to learn science by doing science, offering resources to develop a deep understanding of science content by engaging in scientific inquiry processes such as hypothesis generation, experiment design, and data analysis. While these processes already are quite challenging, task complexity is further enhanced by the fact that students have to regulate their own inquiry. That is, students have to plan a series of experiments, monitor progress and comprehension, and evaluate their inquiry learning processes and knowledge gains.

The student-centered designs utilized in collaborative inquiry learning environments tacitly assume that students are proficient self-regulators. However, research has shown that poor self-regulatory skills often get in the way of students’ learning within these environments (De Jong & Van Joolingen, 1998; Land, 2000). Most contemporary inquiry learning environments therefore offer some kind of regulative scaffolding. Such support typically combines regulative hints and explanations with facilities for students to record, monitor and evaluate their own plans, hypotheses, and experimental findings (e.g., Slotta, 2004).

Manlove, Lazonder, and De Jong (2006) evaluated the use and usefulness of a regulative support tool called the Process Coordinator (PC). Their study showed that the PC had significant positive effects on learning outcomes. However, closer examination of the data revealed that the tool supported students in initial planning and general task understanding, but was little used for monitoring, and not at all for evaluation.

The current research seeks to promote increased monitoring and evaluation activity with a re-design of the PC. The most noticeable modifications include the use of cues and prompts to promote these regulative processes (planning was still supported through a hierarchy of goals, and hints). Since note taking is a central activity in monitoring in online learning environments (Land, 2000), cues were added to encourage students to write down their thoughts, plans, and intermediate outcomes in the PC (cf. Butler & Winne, 1995). Cues appeared as popups in the environment, either after a certain amount of time had elapsed or when students switched to a different activity. Prompts were added to the PC’s note taking form to stimulate students to check out their actions and comprehension (Davis, 2000; Lin & Lehman, 1999). Prompts came in the form of self-explanation questions (e.g., “How do your experiments relate to the overall research question?”) and supported students in the act of monitoring itself. Evaluation was supported by a report editing tool that allowed students to copy the contents of their notes directly into a report template.

The effectiveness of the renewed PC was evaluated in experimental study wherein half of the student groups (PC+) received a “full” version of the PC that included goals and hints to support planning, plus cues and prompts for monitoring, and a report template for evaluation. The other groups (PC–) worked with a PC from which all regulative directions were removed. PC+ groups were expected to achieve higher learning outcomes and produce more instances of planning, monitoring, and evaluating than PC– groups.

Method

Seventy-six high school students (age 16-18) worked in dyads formed by track ability matching. Subsequent random allocation of dyads to conditions resulted in 20 PC+ groups and 18 PC– groups. Following an introduction to the learning environment and its tools, all dyads worked for four fifty-minute lessons on an inquiry task within fluid dynamics that invited them to discover which factors influence the time to empty a water tank. Gained insights had to be articulated in a runnable system dynamic model and a lab report.

Results

Data analysis is currently being performed and will be completed by January 2007. Preliminary results indicate that PC– groups tended to create better models than PC+ groups. However, ongoing analysis of students’ lab reports suggest that PC+ groups produced more well-structured reports and gave a more complete account of the inquiry activities and outcomes. Analysis of report contents will shed light on the difference in knowledge gains across conditions.

Logfile analysis revealed that PC+ groups consulted the PC just as often as PC– groups did. Alleged sustainedness in PC use did not appear to bear out either: PC+ groups mainly used the PC during the initial stages of their inquiry until task understanding was reached. These findings further suggests that the cues did not promote note taking, and frequency counts indeed showed that the total number of notes did not differ across conditions. However, cues did seem to stimulate PC+ groups to consult the hints and explanations in the PC.

Discussion

These preliminary results suggest that the regulative scaffolds did not bring about higher instances of regulation. Why? One explanation might be that sustained high levels of regulation are unrealistic in an inquiry task that spans multiple lessons; perhaps students’ need for regulative guidance reduces once initial task understanding is reached. Secondly, the scaffolds that were meant to ease the burden of regulating the inquiry might, paradoxically, have added to the cognitive overload students already experience in collaborative inquiry learning environments (cf. Land, 2000). If so, less-comprehensive and less-demanding ways to support student regulation during inquiry learning are called for. This signals interesting directions for future research that might eventually yield important guidelines for the design of online tool support.

In this contribution tool use in a computer-based learning environment is looked at. Environmental and learner characteristics are investigated as influencing factors of tool use. To study the influence of these factors an experimental design was used. The two treatment groups got access to tools, while the control group did not. To study the influence of external factors, one of these groups additionally, received advice on how to use these tools. With respect to the learner related variables the following variables were considered: metacognition, motivation, help seeking behavior and instructional conceptions were measured.

Perkins (1985) pointed out that students do not always take opportunities offered. In this contribution offering tools to students for helping them learn is seen as an opportunity for learners to increase or facilitate their learning. In line with Perkins (1985), research indeed reveals that tools are seldomly used and if used often in an inadequate way (Aleven, Stahl, Schworm, Fischer & Wallace, 2003; Clarebout & Elen, 2006). It seems that learners are not always capable of making adequate decisions with respect to their learning process, and hence do not see the different opportunities available to them.

Perkins (1985) indicates that three conditions have to be met to increase the probability that opportunities are taken, or in this case that tools will be used:

1. the opportunity is there: this means that the tools provided are indeed functional to students learning process.

2. learners recognize the opportunity: the tools provided to the learners are indeed recognized as functional for their learning process. This implies that students posses the necessary metacognitive skills to identify when they would be benefit from using a tool, and that students conceive the tools as an opportunity, and hence functional to their learning. Students' instructional conceptions about the tools should calibrate with the function of the tools

3. learners are sufficiently motivated to use the tools.

These conditions point to the interactive nature of this issue; both learner (see for instance, Clarebout & Elen, 2006) and instructional variables play a role (e.g., Gräsel, Fischer, & Mandl, 2001).

Starting from Perkins' framework (1985), this contribution investigates on the one hand the influence of advice, and on the other hand the influence of student characteristics on tool use. The influence of advice is studied as this provides a means to inform students about the functionalities of the different tools. In other words, it is hypothesized that this advice may help learners to recognize the opportunity offered by this tool, and hence that learners will use the tools more extensively.

Consequently, the following research questions are addressed:

a. Do tools contribute to students' learning performance?

b. Does providing advice on tools increases tool use?

c. Do students' metacognitive and motivational aspects influence tool use?

d. Do these students' variables moderate the effect of the advice?

Participants

216 first year educational science students participated. Students received 2 credit points for a course on Learning and Instruction. On average students were 19 year old.

Instruments

Learning environment. A text about obesities was selected and adapted for a macromedia program. Three versions of the program were built. A first one, offered different computer screens with text (control group). In the two other versions different tools were included in the environment, namely a dictionary, access to the goals, example questions, help with interpreting figures and text, etc. In one of these versions additionally, students received, before seeing the text advice on the different tools and what the tools' function was (TA group). This was not the case for the other condition (T group).

Metacognitive variables: Part of the Learning Style Inventory of Vermunt (1992) was use d to measure students' regulation skills. It was opted to only use those items measuring students' self-regulation activities, given the low reliabilities in previous studies for the external and no regulation scale (Clarebout & Elen, in press). The instructional conceptions were measured through the ICON-questionnaire (Clarebout, Sarfo, & Elen, 2004).

Motivation: Students' task versus performance orientation was measured through a Dutch version of Elliot's instrument

Procedure

During the first course of Learning and Instruction, students filled out the questionnaire with the metacognitive and motivational items. They were asked to register for two sessions in the computer room. During the first 'computer' session, students (20 at a time), entered the room and were given a short text on the computer on obesitas. This test aimed at measuring students' prior knowledge about the subject. Students were randomly assigned to one of the three conditions. They were given the instruction that they had to read the text carefully because they would get questions afterwards, knowledge, insight and application questions. They got 10' to read the text. After completing the test, they were given a second, longer text. They got 20' to read the text and also had to answer a similar test. In a third session (2 weeks later), again a text with a test was offered.

Results

Reliabilities of the different instruments are overall good (between .768 and .932). As such the scales were further used in the analyses. An ANOVA with the conditions and the score on the pre-test revealed no significant differences between the three groups on the pre-test (p = .101; eta² = .01).

With respect to the first research questions, the results reveal that the two experimental groups outperform the control group, hence it can be concluded that the tools are functional.