| Proposal Type: | Symposium |
|---|---|
| Domain: | Assessment and Evaluation |
| SIG: | Assessment and Evaluation |
| Type | Submitted Symposium |
| Title | Conditions for assessment to enhance learning |
| Abstract | Today, there is consensus that assessment is one of the most salient contextual variables that has influence on students’ approaches to learning and their learning results is (e.g. Crooks & Mahalski, 1985; Ramsden, 1992; Scouller & Prosser, 1994; Thomas & Bain, 1984). However, empirical studies regarding the effects of the learning and assessment environment show that the expected learning outcomes are not always demonstrated (Segers, 1996). Recent research shows that the way the learning and assessment environment is perceived by the students affects to a large extent how students cope with the learning environment and consequently their learning results (Segers & Dochy, 2001). This means that investigating the way the learning and assessment environment is perceived by the students seems to be crucial for interpreting their learning outcomes, both in terms of approaches to learning and study results (Segers, et al., 2003). In this symposium 4 papers will be presented in order to enlarge our understanding of the relationship between students perceptions of the learning and assessment environment and the conditions under which this contributes to the development of deep approaches to learning and better learning results. |
| Equipment |
PC and projector |
| Keywords | Assessment Learning processes/strategies |
| Chair list | |||||
|---|---|---|---|---|---|
| Name | Surname | Institution | Country | EARLI Number | |
| Mien | Segers | University of Leiden | Netherlands | segers@fsw.leidenuniv.nl | |
| Organiser list | |||||
|---|---|---|---|---|---|
| Name | Surname | Institution | Country | EARLI Number | |
| David | Gijbels | University of Antwerp | Belgium | david.gijbels@ua.ac.be | |
| Mien | Segers | University of Leiden | Netherlands | segers@fsw.leidenuniv.nl | |
| Discussant list | |||||
|---|---|---|---|---|---|
| Name | Surname | Institution | Country | EARLI Number | |
| Sari | Lindblom-Ylanne | University of Helsinki | Norway | sari.lindblom-ylanne@helsinki.fi | |
| Paper Details |
|---|
| Title | Students’ perceptions of assessment demands and their approaches to learning in a constructivist learning and assessment environment |
|---|---|
| Abstract | Recent research shows that, as students interpret the demands of the assessment tasks, they vary their study approaches in order to cope with the assessment tasks. Three research questions are central in the present paper: (1)Can a constructivist learning-assessment environment change students’ perceptions of assessment demands towards more assessment of deep learning skills? (2) Can a constructivist learning-assessment environment change students’ approaches to learning towards a more deep approach? (3) Are students’ changes in perceptions of assessment demands related to changes in students’ approaches to learning? Students following the course ‘Education and psychology’ of the teacher training program at the |
| Summary | Introduction An important challenge for today’s higher education remains the development and implementation of teaching practices that will foster in students the skill to acquire and apply their knowledge efficiently, think critically, analyze, synthesize and make inferences. It is said that students should adopt more deep approaches to learning in order to achieve these goals. The concept of the deep approach is associated with students’ intentions to understand and construct the meaning of the content to be learned, whereas the concept of the surface approach refers to students’ intentions to learn by memorizing and reproducing the factual contents of the study materials. Overall, it is claimed that ‘constructivist’ or ‘new’ learning environments have the potential to improve these educational outcomes for students in higher education (Lea, Stephenson, & Objectives Up till now, only a few studies have presented empirical evidence for the relation between students’ initial study approaches and perceptions of assessment demands, their perceptions of the demands of new modes of assessment, and their actual learning strategies in a constructivist learning environment. Three research questions guide the present study: 1) Can a constructivist learning-assessment environment change students’ perceptions of assessment demands towards more assessment of deep learning skills? 2) Can a constructivist learning-assessment environment change students’ approaches to learning towards a more deep approach? 3) Are students’ changes in perceptions of assessment demands related to changes in students’ approaches to learning? Method Students following the course ‘Education and psychology’ of the teacher training program at the University of Antwerp were asked during the first (pre-test) lesson of the course to complete two questionnaires; the Revised Study-Process-Questionnaire (R-SPQ- Results & Conclusions Results of the paired sampled t-tests indicated that students indeed do change their perceptions of assessment demands towards more assessment of deep learning skills. However, the results also indicated that students did not change their approach to learning towards a more deep approach. On the contrary, students seem to develop more surface approaches to learning during the course. Correlational analyses indicated that only changes of perceptions of assessment demands towards more surface skills are significantly related to changes in approaches to learning, towards a more surface approach to learning. The results point us to the complexity of the relationship between the learning-assessment environment, the students’ perceptions of assessment demands, and students’ approaches to learning. It does not seem easy to change students’ approaches to learning in the desired (deep) direction. Several explanations are discussed in the paper. References Biggs, J., Kember, D., & Leung D.Y.P. (2001). The revised two-factor study process questionnaire: R-SPQ- Crooks, T. J., & Mahalski, P. A. (1985). Relationships among assessment practices, study methods, and grades obtained. Research and Development in Higher Education, 8, 234-240. Lea, S. J., Stephenson, D., & Newble, D. I., & Jaeger, K. (1983). The effects of assessment and examinations on the learning of medical students. Medical Education, 20, 162-175. Nijhuis, J., Segers, M., & Gijselaers, W. (2005). Influence of redesigning a learning environment on student perceptions and learning strategies. Learning Environments Research, 8, 67-93. Ramsden, P. (1979). Student learning and the perception of the academic environment. Higher Education, 8, 411-427. Ramsden, P. (1992). Learning to teach in higher education. Scouller, K., & Prosser, M. (1994). Students’ experiences in studying for multiple choice question examinations. Studies in Higher Education, 19, 267-279. Segers, M., Nijhuis, J., & Gijselaers, W. (in press). Redesigning a learning and assessment environment: The influence on students’ perceptions of assessment demands and their learning strategies. Studies in Educational Evaluation. Thomas, P. R., & Bain, J. D. (1984). Contextual dependence of learning approaches: The effects of assessments. Human Learning, 3, 227-240. Wilson, K. L., & Fowler, J. L. (2005). Assessing the impact of learning environments on students’ approaches to learning: Comparing conventional and action learning designs. Assessment and Evaluation in Higher Education, 30, 87-101. |
| Keywords | Assessment Higher education Student perceptions |
| Appendices | |
| Authors | ||||||
|---|---|---|---|---|---|---|
| Name | Surname | Institution | Country | EARLI Number | Presenting | |
| David | Gijbels | University of Antwerp | Belgium | david.gijbels@ua.ac.be | * | |
| Mien | Segers | University of Leiden | Netherlands | segers@fsw.leidenuniv.nl | ||
| Elke | Struyf | University of Antwerp | Belgium | elke.struyf@ua.ac.be | ||
| Title | Students’ perceptions of traditional assessment in relation to their study-results in a constructivist learning environment |
|---|---|
| Abstract | In the present study students’ perceptions of assessment are topic of research. Two questions are central in this study: (1) How do students perceive the ‘traditional’ assessment in the constructivist learning environment (i.e. the cognitive processes assessed in the traditional assessment)? (2) In what way are students’ perceptions of assessment related to their assessment results? Students’ perceptions of assessment were measured by means of 15 questions from the dimension Cognitive process of the Assessment Preference Inventory, (Birenbaum, 1994). Students’ assessment results were measured by means of their final exam, which consisted of both multiple-choice and open-ended questions. The constructivist learning environment used in this study concerned problem-based learning (PBL). 208 first-year Law students following a PBL-course on the topic of public law participated in the study. Results were analysed by means of descriptive statistics for the measures used in the present study and analysis of variance were conducted to probe into the relationships between students’ assessment preferences and their study results. Results showed that students perceived their assessment primarily an a measurement consisting of comprehension- and application-based questions that required the drawing of conclusions, problem solving, analysis, interpretation and critical thinking. The measurement was also considered, secondarily, as a measurement of reproduction based questions. Despite this, there was a clear correspondence between the intended level of cognitive processes and the perceived level of processes in the assessment in only 40 % of the cases. Though students with a matching perception scored slightly better on the assessment of outcomes compared to students with a misperception, the differences were marginal and not significant. Additional interviews with students about the assessment revealed some insights in how students identify these questions and how they (do not) cope with it. Several explanations are discussed in the paper. |
| Summary | Background In the last decades developments in society and a shift to a constructivist learning paradigm combined with the implementation of powerful learning environments changed the role of assessment in education (National Research Council, 2001). The most fundamental change in the view of assessment is represented by the notion of assessment as a tool for learning (Dochy & McDowell, 1997). Despite the fact that a lot of schools successfully implement new learning environments in which new modes of assessment forms show up well in many cases, final exams containing multiple choice questions and open ended or essay questions are still used (Kuhlemeier, de Jonge & Kremers, 2004). Though, the effects of such new environments do not always demonstrate the desired outcomes (Segers, 1996). In fact research results show that educational change only becomes effective if the students’ perceptions are changing accordingly (Lawness & Richardson, 2002). Scouller and Prosser (1994) investigated students’ perceptions of a multiple choice question examination, consisting mostly of reproduction-oriented questions, to investigate the students’ abilities to recall information, their general orientation towards their studies and their study strategies. The students’ perceptions do not always seem to be correct: and as a consequence students used inappropriate study strategies to learn for their examination. Nevertheless, in this study no correlations were found between perceptions of the multiple choice questions and the resulting grades. In an other study by Scouller (1998), relationships were found between students’ preferences, perceptions and performance outcomes. Poorer performance, either on the multiple choice questions or on the essays, was related to the use of an unsuitable study approach due to an incorrect perception of the assessment. Better performance on the essays (assessing higher levels of cognitive processing) was positively related to a perception of essays as assessing higher levels of cognitive processing and to the use of a suitable study approach. In the above mentioned studies, the multiple choice examinations intentionally assessed the lower levels, and the assignment essays the higher levels, of cognitive processing. It is not always evident that students perceive assessments in the ways that were intended by the staff. Results from a study by MacLellan (2001), using a questionnaire asking students and teaching staff about their assessments, showed that there are differences in perceptions between students and staff of the use and purposes of the assessment and the cognitive level measured by the assessment. Research questions In the present study students’ assessment perceptions are topic of research. Two questions are central in this study: (1) How do students perceive the ‘traditional’ assessment in the constructivist learning environment (i.e. the cognitive processes assessed in the traditional assessment)? (2) In what way are students’ perceptions of assessment related to their assessment results? Method To measure the students’ perceptions of the assessment 15 questions from the dimension Cognitive process of the Assessment Preference Inventory, (Birenbaum, 1994) were used. Basically, this questionnaire asked students which cognitive processes (remembering, understanding, applying, and analysing, evaluating and creating) they thought were assessed by the assessment. Students’ assessment results were measured by means of their final exam, which consisted of both multiple-choice and open-ended questions. 25 % of the assessment consisted of reproduction/knowledge based questions (37.5 % of all multiple choice questions), 20 % of comprehension based questions (30 % of all multiple choice questions), and 55 % of application based questions (32.5 % of all multiple choice questions and all open ended questions). The constructivist learning environment used in this study concerned problem-based learning (PBL). 208 first-year Law students following a PBL-course on the topic of public law participated in the study. Results and conclusions Results were analysed by means of descriptive statistics for the measures used in the present study and analysis of variance were conducted to probe into the relationships between students’ perceptions of assessment and their assessment results. Results showed that students perceived their assessment primarily as a measurement consisting of comprehension- and application-based questions that required the drawing of conclusions, problem solving, analysis, interpretation and critical thinking. The measurement was also considered, secondarily, as a measurement of reproduction based questions. Despite this, there was a clear correspondence between the intended level of cognitive processes and the perceived level of processes in the assessment in only 40 % of the cases. Though students with a matching perception scored slightly better on the assessment of outcomes compared to students with a misperception, the differences were marginal and not significant. Additional interviews with students about the assessment revealed some insights in how students identify these questions and how they (do not) cope with it. Several explanations are discussed in the paper. References: Birenbaum, M. (1994). Toward adaptive assessment - the student’s angle, Studies in Educational Evaluation, 20, 239-255. Dochy, F., & McDowell, L. (1997). Assessment as a tool for learning. Studies in Educational Evaluation, 23 (4), 279-298. Kuhlemeier, H., de Jonge, A., & Kremers, E. (2004). Flexibilisering van centrale examens. Cito: Lawness, C.J., & Richardson, J.T.E. (2002). Approaches to studying and perceptions of academic quality in distance education. Higher Education, 44, 257-282. MacLellan, E. (2001). Assessment for learning: the differing perceptions of tutors and students. Assessment & Evaluation in Higher Education, 26 (4), 307-318. Messick, S. (1994). The interplay of evidence and consequences in the validation performance assessments. Educational Researcher, 23 (2), 13-22. National Research Council. (2001). Knowing what students know: The science & design of educational assessment. Committee on the foundation of Assessment. J. Pelligrino, N. Chudowski & R. Glaser (Eds). Board on Testing and Assessment, Center for Education. Division of Behavioral and Social Sciences and Education. Scouller, K. (1998). The influence of assessment method on students’ learning approaches: Multiple choice question examination versus assignment essay. Higher Education, 35, 453-472. Scouller, K.M., & Prosser, M. (1994). Students’ experiences in studying for multiple choice question examinations. Studies in Higher Education, 19 (3), 267-279. Segers, M. (1996). Assessment in a problem-based economics curriculum. In M. Birenbaum & F. Dochy (Eds.), Alternatives in Assessment of Achievements, Learning Processes and Prior Learning (pp. 201-226). |
| Keywords | Assessment Higher education Student perceptions |
| Appendices | |
| Authors | ||||||
|---|---|---|---|---|---|---|
| Name | Surname | Institution | Country | EARLI Number | Presenting | |
| Gerard | van de Watering | Technische Universiteit Eindhoven | Netherlands | g.a.v.d.watering@tue.nl | * | |
| David | Gijbels | University of Antwerp | Belgium | david.gijbels@ua.ac.be | ||
| Janine | van der Rijt | University of Maastricht | Netherlands | janine.vanderrijt@edit.unimaas.nl | ||
| Filip | Dochy | University of Leuven | Belgium | filip.dochy@ped.kuleuven.be | ||
| Title | Students’ perceptions of the assessment practice and their approaches to learning in a portfolio assessment context |
|---|---|
| Abstract | There is a widespread claim that assessment steers learning. A few empirical studies indicate that students’ perceptions of the assessment demands are related to their learning approaches, however not always in the expected direction. In order to gain more insight into the elements of the assessment practice influencing student learning, we used the Gibbs & colleagues framework. We focused on students’ perceptions of the amount and distribution of effort, assessment and learning, quantity and timing of feedback, the quality of feedback and the use of feedback. The setting for this research is a HE competency-based program on Applied Sciences that implemented portfolio assessment. The students completed two questionnaires at the end of the academic year: the Revised SPQ (Study process questionnaire) to measure their approaches to learning and the AEQ (Assessment Experience Questionnaire) to measure their perceptions of the assessment practice. The results show that in comparison with students with a surface approach, students who adopt a deep approach to learning indicate they make more use of the feedback and they perceive the portfolio assessment as motivating and enhancing learning. It is the feedback practices as perceived by the students and the perception of the assessment as motivating and supporting learning, that are related to student learning approaches and not the time and effort students devote to the assessment. Results of the MANOVA analyses indicate no significant differences in learning approaches and perceptions between first, second year students and students following the shortened program neither between students with different prior education. The results confirm that when implementing new modes of assessment such as portfolio assessment, feedback is essential. Moreover, it is not the assessment design but how it is implemented and how students’ learning is stimulated through the assessment that influences the quality of learning. |
| Summary | Introduction It is argued that in society there is a growing need for competencies such as critical thinking, aptitude for self-management, learning, reflective thinking and the ability to solve novel problems (Kember, Charlesworth, Davies, McKay & Stott, 1997). Kember et al. (1997) argue that graduates frequently lack the very competencies consistent with a deep approach to learning. These qualities are the ability to appropriately engage with, and respond to, the professional situations which they encounter, to understand the structural complexity of the task and the rationale behind facts, and to seek meanings. In education, approaches to learning have been the subject of study for many years and in many disciplines. There is a general consensus that one of the most salient contextual variables to influence students' approaches to learning is the assessment method. It is argued that the students’ perceptions, rather than the instructional and assessment methods themselves, affect student learning most directly (Trigwell & Prosser, 1991). Two strands of research can be discerned. A first strand of research focusses on the students’ perceptions of the assessment demands (e.g. Scouller, 1998; Scouller & Prosser, 1994; Segers, Nijhuis, Gijselaers, in press). The results of these studies are inconclusive. A second strand of research can be described as ‘ assessment for learning’ research, aiming to unravel the conditions for assessment to foster learning. Based on prior research on formative assessment and the role of feedback Gibbs & Simpson (2004) describe conditions for assessment to enhance student learning. The conditions refer to different elements of the assessment practice: the amount of study time and effort; the quality, quantity and use of feedback and the motivating and stimulating role of assessment. Although these conditions are based on evidence from prior research, to date is it unclear how powerful they are in explaining the impact of assessment on students’ learning approaches. Research question Two research questions guide the present study: 1) Is there a relation between students’ perceptions of the assessment practice and their approaches to learning? 2) Are their differences between different groups of students: first, second year students and students of the shortened program; students with different prior education? Setting The study is conducted in a competency-based HE program on Applied Sciences . The instructional approach is based on socio-constructivist principles with students working collaboratively on projects, supported during lectures, just-in-time activities and a study coaching trajectory. The proofs of students’ competence development, the goals aimed for, the actions undertaken and their reflection are presented in the digital portfolio. Based on this portfolio, the student makes a selection to develop an assessment dossier. At the end of each year, this assessment dossier is discussed with an assessment committee consisting of teachers and representatives from the professional field. Method At the end of the academic year the students were asked to complete two questionnaires; the Revised Study-Process-Questionnaire (R-SPQ- Results & Conclusions With respect to students’ learning approaches, it is clear that students employ more deep learning approaches than surface learning approaches. The quality of the feedback and the use of feedback are the most positive perceived aspects of the assessment practice. Students perceive the amount and distribution of effort as the least positive aspect of the assessment practice. The results of the correlation analysis indicate a significant positive correlation between a deep approach to learning and what students do with the feedback they receive, their perception of portfolio assessment as stimulating learning and being motivating. There is a negative relation between students’ surface approach to learning and the perception of the quantity and timing of feedback, of the quality of the feedback, what students’ do with the feedback; how they perceive portfolio assessment stimulates their learning, how it motivates them and how clear they perceive what is required from them in the portfolio assessment. The results of the MANOVA analyses indicate that there were no main differences in deep nor surface learning approaches between first year students, second year students and students participating in the shortened program; between students with different prior education nor any interaction effects. Additionally, students’ perception of the assessment practice were found not to differ significantly between the different groups of students. No significant interaction effects could be found. The results confirm that when implementing new modes of assessment such as portfolio assessment, feedback is essential. Moreover, it is not the assessment design as such but how it is implemented and how students’ learning is motivated and stimulated through the assessment that influences the quality of learning. References Kember, D., Charlesworth, M., Davies, H., McKay, J., & Stott, V. (1997). Evaluating the effectiveness of educational innovations: Using the Study Process Questionnaire to show that meaningful learning occurs. Studies in Educational Evaluation, 2, 141–157. Nijhuis, J., Segers, M., & Gijselaers, W. (2005). Influence of redesigning a learning environment on student perceptions and learning strategies. Learning Environments Research, 8, 67-93. Scouller, K. (1998). The Influence of assessment method on students’ learning approaches: Multiple choice question examination versus assignment essay. Higher Education, 4, 453–472. Scouller, K., & Prosser, M. (1994). Students’ experiences in studying for multiple choice question examinations. Studies in Higher Education, 19, 267–279. Segers, M., Nijhuis, J., & Gijselaers, W. (in press). Redesigning a learning and assessment environment: The influence on students’ perceptions of assessment demands and their learning strategies. Studies in Educational Evaluation. Trigwell, K., & Prosser, M. (1991). Improving the quality of student learning: The influence of learning context and student approaches to learning on learning outcomes. Higher Education, 22, 251–266. |
| Keywords | Assessment Student perceptions Study approaches |
| Appendices | |
| Authors | ||||||
|---|---|---|---|---|---|---|
| Name | Surname | Institution | Country | EARLI Number | Presenting | |
| Mien | Segers | University of Leiden | Netherlands | segers@fsw.leidenuniv.nl | * | |
| David | Gijbels | University of Antwerp | Belgium | david.gijbels@ua.ac.be | ||
| Marieke | Thurlings | University of Leiden | Netherlands | m.c.g.thurlings@umail.leidenuniv.nl | ||
| Title | Are assessment environments gendered? An analysis of the learning responses of male and female students to different assessment environments |
|---|---|
| Abstract | The A version of the Course Experience Questionnaire (the OSCEQ) is administered to all third year undergraduates at A parallel study has examined the responses of students in three undergraduate programmes in each of three universities (including The paper will raise questions about common assumptions about the benefits to all students of particular features of assessment environments in the light of analysis of gender differences. |
| Summary | Introduction and objectives The Study 1 A version of the Course Experience Questionnaire (Ramsden, 1991) adapted for Study 2 A parallel study has examined the responses of students in three undergraduate programmes in each of three universities (including Discussion The paper will raise questions about the differential benefits to male and female students of particular features of assessment environments and help to explain the gender differences in student performance experienced at ReferencesDavies, B. & Harré, R. (1989) Explaining the Oxbridge figures. Oxford Review of Education, 15, 3, pp221-225 Dunbar-Goddet, H. & Gibbs, G. (2006) A methodology for evaluating the effects of programme assessment environments on student learning. European Association for Research into Learning and Instruction, Assessment Gibbs, G. (2006) Changing Gibbs, G. & Simpson, C. (2003) Measuring the response of students to assessment: the Assessment Experience Questionnaire. 11th International Improving Student Learning Symposium, Gibbs, G. & Simpson, C. (2004) Conditions under which assessment supports student learning. Learning and Teaching in Higher Education 1, pp3-31. McCrum, N.G. (1996) Gender and social inequality at Ramsden, P. (1991). A performance indicator of teaching quality in higher education: The Course Experience Questionnaire. Studies in Higher Education, 16, 129-150. Trigwell, K. and Ashwin, P. (2003) Undergraduate Students’ Experience of Learning at the |
| Keywords | Assessment Gender |
| Appendices | |
| Authors | ||||||
|---|---|---|---|---|---|---|
| Name | Surname | Institution | Country | EARLI Number | Presenting | |
| Graham | Gibbs | University of Oxford | United Kingdom | graham.gibbs@learning.ox.ac.uk | * | |
| Gill | Turner | University of Oxford | United Kingdom | gill.turner@learning.ox.ac.uk | ||
| Harriet | Dunbar-Goddet | University of Oxford | United Kingdom | harriet.dunbar-goddet@learning.ox.ac.uk | ||

