Within the frame of Cognitive Load Theory (CLT), one of the best investigated effects of instructional design is the modality effect, which refers to the advantage of audiovisual compared to visual only presentation of textual and pictorial information. Usually this effect is explained by the lower amount of extraneous cognitive load imposed by the audiovisual presentation, which leads to more available resources for knowledge acquisition. Given this, the modality effect should be related to the total amount of working memory capacity which varies between individuals.

In two 2x2 experimental aptitude-treatment-interaction studies we investigated in the assumed interaction between working memory capacity (high vs. low) and presentation modality (visual vs. audiovisual) on knowledge acquisition and perceived cognitive load. Both studies revealed interaction effects on both learning and cognitive load, indicating that the modality effect indeed is a capacity effect as proposed by CLT.

Within the frame of Cognitive Load Theory, one of the best investigated effects of instructional design is the modality effect, which refers to the advantage of audiovisual compared to visual only presentation of textual and pictorial information (Ginns, 2005). Usually this effect is explained by the lower amount of extraneous cognitive load imposed by the audiovisual presentation, which leads to more available resources for knowledge acquisition. Given this, the modality effect should be related to the total amount of working memory capacity which varies between individuals. While learners with a low amount of working memory capacity should be affected by the higher demands of visual only presentation and perform lower on knowledge acquisition tests, learners with a high amount of working memory capacity should be able to invest more capacity in the learning process, even if the presentation format is more demanding. This should compensate the modality effect and result in equal performance independent from the presentation modality. The same effect should be expected for the amount of perceived cognitive load, as measured by mental effort scales. Here also the expected difference with respect to the presentation modality should be observable for low but not for high capacity learners. However, this assumed interaction effect between working memory capacity and presentation format has not been tested experimentally to date.

Method

In two experimental studies with 34 (study 1) and 46 (study 2) university students we tested the interaction hypothesis using different learning materials varying in their presentation modality and different capacity measures. In the first study the learning material comprised six visual representations with different representation formats like texts, pictures, tables and formulas with an additional help for the integration of these representations, which was either presented visually or auditively. In the second study the learning environment consisted of 10 text-picture-combinations, where the modality of the text was either in visual or auditive format. Both studies were conducted as aptitude-treatment-interaction studies (Cronbach & Snow, 1977). The individual amount of working memory capacity was at a time measured in a pretest. In the first study a delayed recall test for verbal, numerical and pictorial information was used (Jäger & Althoff, 1983), whereas in the second study the capacity was measured by a numerical updating test, which required storing of as well as operating with numbers in a matrix (Oberauer, Süß, Schulze, Wilhelm & Wittmann, 2000). The participants of each study were classified based on this pre-measurement as high vs low capacity learners by median split. This classification served as aptitude variable. In each experiment the treatment variable was set up by varying the presentation modality (visual vs audiovisual) of identical information. Following Mayer (2001) knowledge acquisition was tested on different levels of understanding (retention, comprehension, transfer) and (only in Experiment 2) cognitive load was assessed by subjective rating scales (Brünken, Plass & Leutner, 2003; Paas, van Merrienboer & Adam, 1993).

Results

The results of study 1 showed the expected interaction effect (F_1,34 = 7.82, p = .009, η² = .21) as well as a superiority for the audiovisual modality on the comprehension level of knowledge acquisition (F_1,34 = 3.78, p = .06, η² = .11) but not on the retention level. Here no main effects and no interaction could be observed.

Study 2 showed a similar effect pattern with an interaction effect on the overall knowledge acquisition (F_1,46 = 3.84, p = .06, η² = .09) and the transfer level scores (F_1,46 = 7.0, p = .01, η² = .15). Moreover, we found a capacity main effect on the mental effort scale (F_1,46 = 4.91, p = .03, η² = .11) with higher scores of mental effort for learners with a higher amount of cognitive capacity and an interaction effect on the extraneous load scale (F_1,46 = 5.62, p = .02, η² = .11): : learners with low capacity scores perceived higher extraneous load in the visual only condition, whereas learners with high capacity showed a reverse pattern with higher scores of extraneous load for the audiovisual condition.

Discussion

In line with cognitive load theory, the results of our experiments could demonstrate that the modality effect is moderated by the individual amount of working memory capacity. This provides empirical evidence for the theoretical assumption of the modality effect to be a capacity effect and moreover for the assumption that working memory capacity demands can be varied by the instructional design of learning materials. This results in both, differences in knowledge acquisition and perceived cognitive load. Moreover the study highlights the important role of the learners´ cognitive prerequisites for the effect of instructional design variations on learning.

References

Brünken, R., Plass, J. L., & Leutner, D. (2003). Direct measurement of cognitive load in multimedia learning. Educational Psychologist, 38, 53-61.

Cronbach, L. J., & Snow, R. E. (1977). Aptitudes and instructional methods. New York: Irvington.

Ginns, P. (2005). Meta-analysis of the modality effect. Learning and Instruction, 15, 313-331.

Jäger, A. O., & Althoff, K. (1983). Der Wilde-Intelligenz-Test (WIT). Göttingen: Hogrefe.

Mayer, R. E. (2001). Multimedia Learning. Cambridge, UK: Cambridge University Press.

Oberauer, K., Süß, H.-M., Schulze, R., Wilhelm, O., & Wittmann, W. W. (2000). Working memory capacity - facets of a cognitive ability construct. Personality and Individual Differences, 29, 1017-1045.

Paas, F.G.W.C., & van Merriënboer, J.J.G., & Adam, J.J. (1994). Measurement of cognitive load in instructional research. Perceptual and Motor Skills, 79, 419-430