Proposal view
Proposal Type: Individual Paper 
Domain: Assessment and Evaluation 
SIG: Assessment and Evaluation 
Type Submitted Paper 
Equipment Slide projector
Paper Details
Title The Construction of Scientific Explanations and Its Effect on Student Learning
Abstract One fundamental activity in scientific inquiry is the construction of scientific explanations. It is assumed that construction of explanations helps students understand the nature of scientific knowledge in terms of its connection to evidence and its uncertainty and subjectivity to change. In this paper, we analyze the quality of students’ written explanations in eight middle-school classrooms and explore the link between the quality of students’ written explanations and their performance on assessments focusing on the topic studied. More specifically, we asked whether the quality of the students’ written explanations found in the students’ science notebooks was related to the observed performance in a set of posttest assessments focusing on the content of the science unit studied.
Summary  
Objective

One fundamental activity in scientific inquiry is the construction of scientific explanations. It is assumed that construction of explanations helps students to understand the nature of scientific knowledge in terms of its connection to evidence and its uncertainty and subjectivity to change (Duschl, 2003). In this paper, we focused on the question, Is there a relationship between the quality of students’ written explanations and their performance on the topic studied? More specifically, we analyze the quality of the written explanations found in the students’ science notebooks and their observed performance in diverse posttest assessments. We also explore an instructional aspect about explanations by asking: What instructional prompts, if any, best promoted high quality explanations?

Analyzing Scientific Explanations

Analysis focused on three aspects: (1) Quality of Explanations according to its three components (Claim: statement that answers a scientific problem; Evidence: data that supports a claim, and Reasoning: justification that shows why the data counts as evidence to support the claim; Kenyon & Reiser, 2006; Tzou, 2006). (2) Quality of Communication focusing on characteristics that have been considered important in students’ scientific communications, and (3) Student’s Level of Understanding focusing on students’ understanding about density based on a conceptual progress trajectory of density.

For each explanation component we developed criteria to capture its quality, communication characteristics, and an overall level of understanding. For each component identified we asked a set of coding questions (Table 1).

 

Table 1. Examples of Questions to Score Quality of Scientific Explanations.




































Component
Examples of Coding Questions
Examples of Scoring Criteria

Claim
How does the claim address the main idea(s) tapped in the investigation conducted?


·   Does not address

· Partially addresses

· Addresses all main ideas

Evidence
What type of evidence did the student provide?

 
·   No evidence

· Anecdotal/opinion/everyday examples

· Investigation data

 
If specific examples provided, is there enough evidence to support the claim?
· One data provided

· Two data points

· More than two data points

Reasoning
Is the evidence provided aligned with the claim?


·   No

· Partially

· Completely

 
Is the evidence provided connected to the claim?


·   No link

· Link indicated by connected words

· Link elaborated



 

Method

Participants. Twelve middle school physical science classrooms participated in the study in which a unit on relative-density was taught.

Procedure. Students’ science notebooks were used as the main source of information. Notebooks were collected at the end of the school year. Within each classroom, nine students’ notebooks were randomly selected from strata based on the students’ scores on the post-test (top-, medium-, and low-proficient).

Using an Access computer program each notebook has been analyzed on six aspects: Problem, Vocabulary, Background, Method, Reporting results, and Conclusions. We focus on the analysis of the “Conclusions” where scientific explanations are embedded.

Students were also administered different types of assessments in a pretest-posttest design, including a multiple-choice test, one performance assessment, one predict-observe-explain, and one open-ended question.

Results

We provide information about Investigation 7 (Table 2). Averaged interrater reliability over explanation components was .92. From the sample scored (n = 72) we found that only 18.1 percent of students provided explanations with the three expected components. Only 12.5 percent provided claims with supporting evidence. The majority (40.3 percent) provided only claims without any supporting evidence, and 9.7 percent provided only data. Nineteen percent of students in the sample analyzed did not provide any “form” of explanation.

 

Table 2. Percent of Students’ Explanations Provided by Type and by Classroom










































































Type of “Explanation”
Classrooms

1
2
3
4
5
6
7
8

Explanations
11.1
0.0
0.0
0.0
55.6
0.0
77.8
0.0

Claims & evidence
0.0
77.8
11.1
0.0
11.1
0.0
0.0
0.0

Only claims
66.7
22.2
22.2
77.8
0.0
33.3
11.1
88.9

Only evidence


0.0
0.0
66.7
0.0
0.0
0.0
11.1
0.0

No explanation
22.2
0.0
0.0
22.2
33.3
66.7
0.0
11.1

 

 

We have collected information on each of the explanation components, quality of claim, evidence and reasoning and created an explanation composite score. This score was used to link students’ quality of explanations with their performance (Table 3).

 

Table 3. Descriptives of Students’ Scores on Two Assessments Only


















































































































Type of Assessment
 
 
Classrooms

Max
n*
1
2
3
4
5
6
7
8

Multiple-Choice Post


43
9
 
 
 
 
 
 
 
 

    Mean
 
 
24.78
20.55
27.33
24.67
27.33
20.00
28.22
26.77

    Standard Deviation
 
 
8.55
7.07
7.65
6.26
8.39
9.06
8.59
7.84

 
 
 
 
 
 
 
 
 
 
 

Performance Assessment
32
9
 
 
 
 
 
 
 
 

    Mean
 
 
18.12
11.75
16.66
17.50
18.50
12.57
20.78
16.33

    Standard Deviation
 
 
10.24
2.05
9.31
7.74
9.63
5.74
6.03
7.77

* n per classroom

To address the effect of writing explanations and student learning we looked into the correlations between the explanation score and the student performance scores (Table 4). We focused on correlations that involved the eight groups due to the small sample size within each group. The pattern observed indicates a significant positive relation between the quality of the students’ explanations and students’ performance at the end of the investigations. However, the magnitude of the correlations varied according to the type of assessment at hand.

 

Table 4. Correlations between the Explanation Composite Score and the Diverse Assessments






























 
n
Explanation Composite Score

Multiple-Choice Post
66
0.245*

Predict-Observe-Explain


57
0.331*

Performance Assessment


60
0.325*

Short Open-Ended (WTSF)
62
0.269*

*      Correlation significant at 0.05.

**   Correlation significant at 0.01.

 

We concluded that engaging students in the construction of scientific explanations seems to be linked to the students’ high performance in the post-test. The paper will provide information about the quality of explanations over time by analyzing two more investigations, and about instructional strategies that seems to be more effective in eliciting explanations.

Importance of the study

This study tests a scientific inquiry instruction premise: the construction of explanations by students after conducting an investigation. It not only provides evidence on whether the premise holds, but also presents evidence on the effects of involving students in the construction of scientific explanations on their learning. Furthermore, it will provide information on what practices (e.g., type of prompt) observed seemed to be the most effective.
Keywords Assessment
Assessment methods
Science education
Appendices
Authors
Name Surname Institution Country e-mail EARLI Number Presenting
Maria Araceli Ruiz-Primo University of Colorado at Boulder United States maria.ruiz-primo@cudenver.edu   *  
Min Li University of Washington United States minli@u.washington.edu    
Shin Ping Tsai University of Washington United States sptsai@u.washington.edu    
Visit NQcontent
© European Association for Research on Learning and Instruction, 2010 All rights reserved.