The Sakai Virtual Research Environment (VRE) for Educational Research project’s aim was to evaluate the Sakai VRE for supporting collaborative geographically-distributed educational research projects in the UK. In this paper we describe one specific instrument of this project, the use of ‘video diaries’. The aim of this instrument was two-fold, first to develop a new and novel approach for the gathering of ‘user requirements’ to inform software developers. The second was to analyze and theorize the role that evaluators need to develop to move between the two distinct worlds of educational researchers and software developers. The gathering of ‘user requirements’ is crucial for future development of the virtual research environments for use by large collaborative and geographically-distributed research teams in a new e-Research driven environment.

The Joint Information Systems Committee’s (JISC) Sakai VRE for Educational Research project worked in collaboration with the UK’s Economic Social Research Council’s Teaching and Learning Research Programme to evaluate Sakai for use as a VRE for large collaborative educational research projects. Sakai is an open source community developed project which provides an online access controlled suite of modular collaboration tools which geographically-distributed research teams can use to support their research. Ten educational research projects were recruited for the Sakai VRE evaluation project.

Four tasks were developed which would give the evaluators and the software developers some insight into both the user requirements of educational researchers and the nature of collaboration within the discipline of educational research. Two of the tasks were individual, one being broad semi structured questions, the second a structured questionnaire which participants undertook themselves. The other two tasks were collaborative tasks, one involving the nature of the interaction between the participants and their workplaces and how the VRE fitted into that context. The other task involved a group discussion about the nature of collaboration in educational research and how VRE technologies fitted into that.

Our findings show a positive and enthusiastic response from the researchers who were willing to engage with the tasks. The use of ‘video diaries’ with self-engaged researchers rather than just audio recordings provided us with rich contextual information and background about the behaviors and attitudes of educational researchers. The videos provided us with images, including facial expression, body language and gesture as well as the audio of the researcher’s workplace discourse without any effects of bias from having an outside observer or interviewer steering the interaction.

The lack of an observer or an interviewer throughout the tasks may have given the participants more of an opportunity to be more open and/or more critical about the VRE and their thoughts on collaboration throughout the tasks.

Boundary objects are, according Star and Griesemer, “…are both plastic enough to adapt to local needs and constraints of the several parties employing them, yet robust enough to maintain a common identity across sites. They are weakly structured in common use, and become strongly structured in individual site use” (Star & Griesemer, 1989:393). Thus in our case the idea of boundary object is the video tape of educational researchers talking about their lived experiences both individually and in collaborative team situations. These video objects were able to move between the two worlds of the educational researchers and the software developers.

One way of analyzing these boundary crossing video objects is provided by Kinti and Hayward (2006) who suggest three classes of “coordination problems that inhibit the development of processes of interagency working for knowledge co-configuration in a collaborative team of experts”. These are Internal Coordination Problems (ICP) characterized as interaction breakdowns within the team itself, External Coordination Problems (ECP) that occur due to breakdowns between the team and end users’ and their environment, and Latent Coordination problems (LCP) which they describe as “breakdowns in the interactions between the team and actors’ home work systems, that are not directly under the control of the team but have a negative impact of the work of the team”.

These three types of coordination problems can be uncovered in our two separate disciplinary groups. Initially the making of the video tape provides a forum for the open discussion of educational researchers own practice in relation to the VRE, both individually and collaboratively, thus giving them a space to discuss within their teams their own ICP’s. The nature of the tasks will also allow the participants to highlight the ECP’s they are having with the VRE. The video then crosses the boundary from educational researchers to software developers. The developers can then use the video as a focus to highlight the user’s issues with the VRE system. This may then confront ICP’s within the software developer’s team and ECP’s which come from the users experiences with the VRE. The video objects may also be able to show some of the LCP’s that users have to deal with both within their own institutions and within the context of their own discipline, thus the video may be able to also capture the disciplinary norms that educational researchers.

This analysis highlights the affiliation networks but not their intentional networks (Nardi et al, 2000) that both the educational researchers and the software developers are involved with and move between. These include their institutions, their disciplines, their funders, and their projects.

References

Kinti, I. and Hayward, G. (2006) “Managing scientists is like herding cats”: Challenges of coordinating interagency collaborative teamwork, presented at American Educational Research Association conference, San Francisco, 7-11 April.

Nardi, B., Whittaker, S. and Schwarz, H. It's Not What You Know, It's Who You Know: Work in the Information Age, First Monday, volume 5, number 5 (May 2000) accessed on 15-11-07, URL: http://firstmonday.org/issues/issue5_5/nardi/index.html

Star, S. L. and Grisemer, J. R. (1989) Institutional ecology, ‘Translations’ and Boundary Objects, Social Studies of Science, 19 (3), 387-420.