The main focus of this presentation is the result from a study investigating dimensions of grades by discerning achievement and non-achievement in grades. Achievement may reflect a subject-specific dimension of grades while non-achievement may reflect a common dimension of grades. The distinction between indirect and direct influences in grades is also made. Non-achievement may influence grades indirectly, by influencing student subject knowledge, which in turn influences grades. On the other hand, non-achievement may not influence student subject knowledge but still influence grades which may be interpreted as a direct influence in grades.

The data analysed were taken from the ETF (Evaluation Through Follow-up) an ongoing longitudinal project in Sweden. The participants were 8874 ninth grade students, who left compulsory school in 2003. Two measurements were used, subject grades and national test scores in Swedish, English and mathematics from the end of compulsory school. Confirmatory factor analysis (CFA) was used, and a two-level model was executed in order to control for differences between schools.

The result indicated that grades measure both achievement and non-achievement. The influence of achievement in grades seems to vary both concerning subject grades and national tests and between schools. Non-achievement seems to influence grades in all subjects and without any significant differences between schools which may support the hypothesis of a common factor in grades. 

Summary 

Introduction and aim

In Sweden grades are expected to measure student subject knowledge or achievement but research has indicated that grades also measure non-achievement like student effort, skills and temperament (Alexander, 1935; Brookhart, 1991; Selghed, 2004). On the one hand, student achievement may reflect a specific dimension of grades. On the other hand, non-achievement like effort, temperament, interest, communication skills may reflect common dimensions of grades in the sense that they are common to several or all grades. It is also possible to distinguish indirect and direct influences of grades. Non-achievement may influence grades in an indirect way, where aspects of non-achievement influence student subject knowledge, which in turn influences grades. Non-achievement may also influence grades in a direct way, where student subject knowledge is not influenced by non-achievement, but still influences grades. Non-achievement like effort, work completion, teamwork and on-time submissions may influence grades in a direct way while student social competence, communication skills and attitude may influence grades in an indirect way. The purpose of the current study is to discern achievement and non-achievement in grades where the distinction of direct and indirect influences of grades is made.

Method

The subjects were 8874 ninth grade students, born in 1987 who left compulsory school in 2003. Data used in this study comes from the ETF (Evaluation Through Follow-up) which is a Swedish ongoing longitudinal project. The ETF database contains extensive administrative data from schools like grades and scores on national tests in core subjects and also questionnaire data from the students.

Two measures were used in this study, subject grades and national test scores from the end of compulsory school in three subjects, Swedish, English and mathematics, which constituted the manifest variables. Swedish and English had four manifest variables each; subject grades in Swedish (SGSW) and English (SGEN) and three national test scores in Swedish (TSSW1-3) and English (TSEN1-3) while mathematics had two manifest variables; subject grades (SGMA) and only one national test score (TSMA).

In the present study confirmatory factor analysis (CFA) was used in order to investigate whether grades measure both achievement and non-achievement. Four latent variables were created, three hypothesized to measure subject-specific achievement (AchSw, AchEn and AchMa) and one hypothesized to measure non-achievement (NoAch). In order to discern achievement and non-achievement in grades, the three latent variables AchSw, AchEn and AchMa, were related to the subject grades SGSW, SGEN and SGMA respectively, and scores from national tests TSSW1-3, TSEN1-3 and TSMA, while the non-achievement latent variable, NoAch, was related only to the subject grades SGSW, SGEN and SGMA. This model was hypothesized to separate achievement and non-achievement in grades since AchSw, AchEn and AchMa were supposed to reflect achievement, a specific dimension of grades, while NoAch was supposed to reflect a common dimension of grades, interpreted as non-achievement.

As measures of model fit, the X² goodness-of-fit test and RMSEA were used.

Internal missing was handled by missing data modelling (Muthén, Kaplan & Hollis, 1987). Mplus, version 3 (Muthén & Muthén, 2004) was used and STREAMS (Gustafsson & Stahle, 2005) is a modelling front-end program and was used to execute the analyses.

Findings

Four steps were executed. First, measurement models were computed for Swedish and English but not for mathematics since the measurement model were non-identified. Secondly, a three-factor model (C) was computed including the three latent achievement variables (AchSw, AchEn and AchMa), with covariance between them, related to respectively subject grades (SGSW, SGEN and SGMA) and national test scores (TSSW1-3, TSEN1-3 and TSMA). Fit indices were acceptable (X²(32, 8874)=819.19;RMSEA=.053) but improvable and supported the hypothesis that an additional common latent variable may exist. Thirdly, a four-factor model (D) was executed with the same relations as in the previous model (C) but with the non-achievement latent variable, NoAch, included in the model. NoAch was related only to the subject grades (SGSW, SGEN and SGMA). This model, with covariances between AchSw, AchEn and AchMa, indicated a better fit (X²(29, 8874)=563.16; RMSEA= .046) which support the hypothesis that grades have something in common in addition to subject knowledge or achievement. Finally, a two-level structural model (E) was executed in order to control for effects of differences between schools. Covariances were fitted between all the manifest variables on the second level while the relations between the latent and manifest variables remained (model D) on the first level. This model indicated an even better fit (X²(29. 8874)=462.79;RMSEA=.041). This model indicated an additional improvement which supports the hypothesis that differences between schools concerning grades exist.

Theoretical and Educational significance of the research

The hypothesis of the existence of a common factor in grades seemed to be supported in this study. The common non-achievement factor, NoAch, explained variance in all the subject grades, Swedish, English and mathematics. However, this non-achievement factor needs further investigations which will be done in future studies.

The latent variables accounted for a considerable proportion of the variance in subject grades and national tests but the result did leave a great amount of variance not accounted for. Thus, it seems reasonable to believe that the latent variables of the different subjects may be related to other latent variables, which may form a hierarchal structure, and explain additional variance in subject grades and national tests. A possible interpretation of unexplained variance may be that student characteristics like social background, sex and national background is not accounted for in this study.

This study also indicates that school differences concerning achievement in grades and national tests exist. It also seems that non-achievement influence grades in all subjects, Swedish, English and mathematics, and without any significant differences between schools.