In order to coach teachers adequately, researchers need to supply them with the tools that can capture students’ successive attempts at self-regulation. Ideally, teachers should be able to determine the zone of proximal development in self-regulation. In order to do that, they need to predict expected outcomes in terms of the self-regulation strategies that their students are capable of using. How can they be coached to achieve this? First, teachers need insight into the logical structure of SR development in a domain. Second, they need a set of assessment tools that capture their students’ current level of self-regulated learning. Researchers need to provide teachers with this information and with the necessary assessment tools. Unfortunately neither the information nor the tools are available at present. Predictive approaches offer the opportunity to model self-regulation in the classroom. They are capable of discovering complex relationships and interactions in the inputs (predictors) and outcomes. They help us understand complex relationships across self-regulation components in a systematic fashion. As such, these approaches will allow researchers to explain and predict learning outcomes, by examining the pattern of relations between the different types of self-regulation strategies that students use habitually in a given domain (automaticity) and the effect of favorable and unfavorable learning conditions on their strategy use.

Teachers need tools that can capture students’ successive attempts at self-regulation. These tools should be sensitive enough to provide information about the level of self-regulation that the students have attained. Teachers need this information in order to feed it back to the students when they discuss progress and to determine the zone of proximal development. Researchers need this information as well in order to study the pattern of relations between the different types of self-regulation strategies that students use habitually in a given domain (automaticity) and the effect that favorable and unfavorable learning conditions have on their strategy use. Are predictive approaches able to examine in detail the multiple elements of the self-regulation model in an integrated fashion? Are they able to link all the elements in the model to the objectives and expected outcomes?

There have been several attempts to gain insight into the multiple components of self-regulated learning. Multiple assessment instruments have been designed to describe students’ attempts at self-regulation. Many of these instruments have been criticized because they are removed from the context where the actual attempts at self-regulation occur and therefore the SR strategies that are measured are susceptible to distortions. Recently software and on-task probes have been used to assess self-regulation in real time. Some researchers featured mainly on the cognitive and meta-cognitive traces that students leave behind when they are making attempts at self-regulation (Winne and Perry, 2006), others focused on the motivational aspects (Ainley, 2006; Vollmeyer and Rheinberg, 2006 and still others on the social aspects of self-regulation (Jarvela, 2006; Hadwin, 2006). These are promising new tools that invite students to report on their thoughts, feelings and actions while they are self-regulating the learning process. Unfortunately, the information that is accrued with these tools is still fragmentary and does not reveal the essential links between elements of the self-regulation model.

In my presentation I will focus on self-regulation of the writing process. I will argue that this is a complex process and that to deal adequately with this complexity researchers need a model of self-regulation that is grafted on the domain they are studying. In other words, when studying self-regulated writing the model of self-regulation should encompass the multiple components of self-regulated writing and should be supported by custom-built, integrative assessment instruments that can capture self-regulated writing in real time. What is more, the multiple components of self-regulated writing should be described as a system. I will introduce Boekaerts’ (1997) six component model of self-regulation as a powerful model for discovering the underlying patterns of self-regulation in the writing domain. The model allows researchers to carefully describe the structural properties of SR in the writing domain. It also allows researchers to spell out the links that exist between the different components in the model (dynamic interactions) and to link these components to the outcome variable (writing ability). Finally, the insights that the model provides allows researchers to design integrative assessment instruments to measure the self-regulation strategies that students use in the writing domain. I will argue that the Neural Network approach is ideal to process the data accrued with the integrative assessment instruments because it is sensitive enough to detect meaningful links in the data and to determine the relative strengths of predictors.

 

           As agreed with the conference manager Mr. Csaba Csíkos there is no separate slot for expert panel discussions. Accordingly we used the symposium slot. Panelists agreed to write an abstract and short summary because they don’t know exactly what to say nine months in advance. In order to fill the 600 words required, we replicated this text several times.  As agreed with the conference manager Mr. Csaba Csíkos there is no separate slot for expert panel discussions. Accordingly we used the symposium slot. Panelists agreed to write an abstract and short summary because they don’t know exactly what to say nine months in advance. In order to fill the 600 words required, we replicated this text several times. As agreed with the conference manager Mr. Csaba Csíkos there is no separate slot for expert panel discussions. Accordingly we used the symposium slot. Panelists agreed to write an abstract and short summary because they don’t know exactly what to say nine months in advance. In order to fill the 600 words required, we replicated this text several times. As agreed with the conference manager Mr. Csaba Csíkos there is no separate slot for expert panel discussions. Accordingly we used the symposium slot. Panelists agreed to write an abstract and short summary because they don’t know exactly what to say nine months in advance. In order to fill the 600 words required, we replicated this text several times.

This presentation will describe the application of neural networks (NNs) in modeling self-regulation in the classroom. This machine-learning technique, developed to mirror human brain processing, is an iterative process that is capable of discovering complex relationships and interactions in the inputs and outcomes. The goal of NNs is often to maximize classification accuracy, regardless of understanding the relative strengths of predictors. However, recent work has demonstrated that NNs in combination with more traditional statistical techniques can provide novel insights into complex relationships to identify which variables are driving the model (Costigan, 2003). We will present an introduction to NNs, including definitions, data requirements, and parameter settings. Included in the presentation will be a discussion of relative strengths and limitations, linking NNs in a sequential analytic chain to maximize modeling. We will also present a case example of this application in educational research.

 

Yes. This machine-learning approach can be configured in such a way as to examine relationships at various levels of modeling. To investigate SRL, we will (a) examine each component of SRL independently to identify the strongest relationships within each and then subsequently combine these single-component models into a multidimensional model, using only the strongest predictors. Strengths of this two-step approach include (a) model parsimony by reducing variables within each component prior to combining component; and (b) results in single component models that researchers can use to further investigate, replicate, and understand individual aspects of SRL in a systematic fashion. In addition, we will build the combined, multidimensional model, using all variables available from all components. The advantage to this latter method is that complex relationships across SRL components may be derived, which would have been missed in the single component modeling process.

 

 

 

NNs are historically thought of as “black box” techniques in that there was no easy way to understand the variables and interactions of most importance to the predictive model. However, with recent advances in computing, NNs are more easily configured and conducted. Additional machine-learning techniques can also be used to understand the variables that are most influential in the model (see below).

 

 

Since model validation is critical in establishing the optimal solution for a neural network, a number of critical steps are taken to examine the results and the model proposed: (a) methodologies to “see” inside the neural networks “black-box” and establish the relative importance of weights by modelling the predictive values using a rule-induction technique to develop classification systems in which the “decision rules” identify the variables of importance; (b) in order to validate the neural network model, it is important to partition the data into two samples: training/testing sample and validation sample. This technique, common in more traditional statistical approaches, protects against over-fitting of the model; (c) causal relationships among components/constructs in this model as well as the underlying variables within these components/constructs are examined; (d) finally, in order to evaluate the performance of the neural network system, a number of measures provide a means of determining the quality of the solutions. These measures will be explained in the context of the discussion.

 

 

Any assessment of self-regulated learning is faced with a highly complex system of interacting elements which have to be regarded as the dynamics in the individual’s autonomous regulation. Because of this complexity predictive assessment requires a description of the structural characteristics as well as of the functions of structural transformations. Although a structural approach may help, a comprehensive evaluation of the assessment concepts that are used is hardly possible, due to their diversity in elaboration and formalisation and due to the diversity and partial incomparableness of their underlying theoretical concepts. For this reason, I suggest that in any further development of predictive systems approaches emphasis should be given to the development of actual “systems” and that the respective implications should be examined. In addition, I would suggest to frame the differential assessment approach in a mode that encompasses the multiple facets of the phenomenon so that it can cope successfully with complexity.

Any assessment of self-regulated learning is faced with a highly complex system of dynamically interacting elements: The first level includes the basic cognitive perspective with general and specific knowledge and skills and their specific connection to various strategies. It also contains the basic motivational and emotional perspective with emotions, moods and their specific connection to motives, motivations and intentions. At the second level, the cognitive and motivational/emotional perspectives interact, giving rise to learning interests, learning styles, orientations and attributions, which are embedded at the third level in specific contents, domains and social contexts as learning environments. Finally, encompassing all levels, these interactions have to be regarded as a dynamic system, whose specificity is included in the autonomous regulation of the individual.

 

Taking into account such complexity, all approaches, and in particular predictive approaches to the assessment of SRL, have to explain at which level and in which perspective they examine the multiple elements of self-regulation. Any integrative assessment founded on a comprehensive model of SRL can only be informative, if it describes, predicts and differentiates information in a structured mode. However, as the choice of level and perspective determine the characteristic type of input as well as the frame of objectives and expected outcomes of an assessment, its fit with the selected conceptual model is essential for the validity as well as the accuracy of the intended prediction. As a consequence, the quality of a predictive system and its sensitivity to meaningful classification has its limits in the quality of the founding concept it is calibrated on.

 

Along the above line of arguments, the first requirement of systemic predictive assessment is its well defined structure. The second requirement concerns the assignment of parameters, apt to designate simultaneously its structural properties as well as the functions that describe the dynamics within the structure. Accordingly, assessing SR in learning implies that the characteristics of its structure are described as well as how predictive assessment of the dynamics in SR will take place. It implies the format of a functional description of structural transformations.

 

Viewing self-regulation in learning in the above structural perspective (rather than attempting to understand SRL as just another special type or model of learning) could serve as the via regia towards a more insightful understanding of the dynamically interacting processes inside the “black box” named SR. In addition, opening a window on the processes connecting self-regulation to context variables could help in detecting the frame and, with special regard to predictions, the zone of development in which self-regulation in learning may take place.

 

At a first glance, the ideas about assessing self-regulated learning as described in the foregoing paragraphs may allow a more formal and perhaps a more systematic and more precise evaluation of existing assessment tools. However, because of their elaborated diversity and formalisation, a comprehensive or at least a somewhat representative evaluation (even of the diverse predictive systems) is hardly possible in reality. Furthermore, as most assessment tools are bound to a certain model or at least to a family of models of SRL, we need to define the respective tertia comparationis for any comparative analysis. This mainly depends on the properties of the models to compare. Even if we refer to relatively general aspects, such as “effectiveness” or “efficiency” we have to meaningfully define and operationalize the constructs we want to compare. For this reason, it might be more productive to look ahead and construct systematically new parameterised assessment tools instead of looking back and use comparative evaluations of (partly incomparable) instruments. Recent discussions have highlighted that “ calibration” and “content-sensitivity” are key issues of predictive instruments, which ultimately refer to important characteristics of validity in any assessment tool.

With respect to calibration, I would like to emphasise that the characteristic assessment tools will have to be located within the model’s definition of self-regulation. If we calibrate our predictive tool e.g. on Boekaert’s model of self-regulated learning, the dynamics of switching between the cycles of the learning path and the well-being path are key characteristics of the top level embedded in lower level cognitive and emotional appraisal processes within the frame of an information processing model. In Nenniger’s model on self-directed learning dynamic switching is not considered, but the spiral recursivity of the ongoing self-direction in learning coincides with the cycles of the learning path. In contrast, in goal oriented models (e.g. sensu Pintrich or Kuhl), where we find the straight forward oriented dynamics of Heckhausen’s “Rubicon model”, such characteristics are not explicitly considered and therefore it makes no sense to compare them in this perspective.

With respect to content-sensitivity, validity often refers to the scope of application. Many models of self-regulated learning (e.g. in the tradition of the expectancy x value approach of motivation [Atkinson, Heckhausen, Nuttin]) are ultimately oriented at a general understanding of success or failure and are rather content (and culture) insensitive, meaning that in these models content represents an arbitrary placeholder. Therefore, assessment tools related to the above models might be applicable e.g. to predict the impact of actual interestedness at a certain moment of ongoing self-regulated learning. In contrast, predictions about regulatory effects of interest (e.g. sensu Schiefele) are from their understanding of interest as interactions of person and matter explicitly linked with the specificity of a content.

 

In conclusion: In any further development of a predictive systems approach to the assessment of self-regulated learning emphasis should be given to “systems” together with the respective implications. This also means that any discussion restricted to methodology or even technology is too narrow for the nature of this issue. My line of argument proposes to frame a differential approach that encompasses, on the one hand, the multiple facets of the phenomenon, including level, aspect and dynamics of a self-regulation model, as well as content, social environment and didactical concept. On the other hand, it recommends an adaptive hierarchical procedure within the assessment tool which allows us to cope successfully with the complexity by a choice of straightforward analyses of each single facet.