The TAO framework is an open-source project which provides a very general and open architecture for computer-assisted test development and delivery. As upcoming evaluation needs will imply the collaboration among a large number of stakeholders situated at different institutional levels and with very different needs for assessment tools, the TAO framework has the ambition to provide a modular and versatile framework for collaborative distributed test development and delivery with the potential to be extended and adapted to virtually every evaluation purpose that could be handled by the means of computer-based assessment. In order to illustrate the power and the flexibility implied by the TAO framework, several use cases of the architecture will be presented. Future developments and use cases of the TAO framework will be discussed.

The TAO platform for computer-assisted testing (from the French acronym for CBA – “testing assisté par ordinateur”) is an open-source project which provides a very general and open architecture for computer-assisted test development and delivery (Martin et al., 2005, 2006). As upcoming evaluation needs will imply the collaboration among a large number of stakeholders situated at different institutional levels and with very different needs for assessment tools, the TAO framework has the ambition to provide a modular and versatile framework for collaborative distributed test development and delivery with the potential to be extended and adapted to virtually every evaluation purpose that could be handled by the means of computer-based assessment. In order to reach this goal, the testing problem has been broken down into its constitutive parts, which are considered as separate knowledge domains which have to be interconnected and coordinated in order to build a complete assessment process (for the moment, six different knowledge domains are considered: (1) item development and management, (2) test development and management, (3) subject management, (4) group management, (5) test delivery and (6) results management). Of course, depending on the context, more knowledge domains might be identified or required. As we shall see further, the inclusion of these in the platform will be enabled using the modular aspect of the platform architecture, the different knowledge domains having been implemented under the form of multiple specialized modules connected in a peer-to-peer network

Each knowledge domain corresponds to an independent data domain that can be managed by different actors possibly distributed in various locations. For instance, the management of subjects and their characteristics may be managed centrally by a dedicated administration (Ministry of National Education, for instance), while groups can be managed by teachers from their classroom computer or from the school central server. The most important and universal information about subjects are the login and the password they should type in at delivery time to be able to pass the test they have been allocated to. Groups are composed of subjects and are assigned a series of tests. Tests are in turn composed of items that contain multimedia content. The results can be put in relation with any feature of groups, subjects, tests, and items.

This top-level model enables the separation of concern with respect to the different types of resources involved in the assessment process. This separation is the keystone of modularity and component reusability in the creation, management, and delivery of tests. Indeed, separation between subjects and groups is a convenient way to preserve privacy. Doing so, personal data are considered separately from the strict CBA problem (represented by the group-test relation). For analysis purposes, results can be put in relation with the other types of resources. This enables one to compare results (scores, endorsements, behavioural data etc.) and other result properties with characteristics of any other resource that was involved in the assessment process. This is of course required to ensure very rich result analysis capabilities. Tests can be delivered according to particular schedules, or organised in the form of campaigns gathering several different tests. Constraints regarding the number of possible passes per subject should also be established, as well as multi-pass result combination policies.

As the TAO architecture should fit into many different use cases, there is no central node in the system (the whole network of TAO nodes is an open-system with no global consistency) and it is assumed that there cannot be an a priori model for each data domain that reaches consensus within all the actors. On the contrary, each module owner or user is free to build the model that best corresponds to his own view. In order to support this model diversity, TAO modules are equipped with advanced modelling functionalities allowing users to build dynamically their own model and share data organized according to different models.

In order to illustrate the power and the flexibility implied by the abovementioned software architecture which is provided by the TAO framework, several use cases of the architecture will be presented: the use of TAO as a tool for adaptive testing in the field of the evaluation of language skills (comprehension of written and spoken language), the use of TAO in large scale assessments of scholastic aptitudes as they can been found in the context of school monitoring programs, the use of TAO as a learning tool in mathematics classes, the use of TAO as a tool for dynamic evaluation of scientific literacy, as well as other already realized or potential applications. The different parts of the testing process as they have been instantiated for these use cases in the TAO framework will be presented, as well as the required hardware and server infrastructure. Future developments and use cases of the TAO framework will be discussed.

References:

Martin, R., Latour, T., Burton, R., Busana, G., & Vandenabeele, L. (2005). Covering different levels of evaluation needs by an internet-based computer-assisted testing framework for collaborative distributed test development and delivery. In P. Kommers & G. Richards (Eds.), Proceedings of world conference on educational multimedia, hypermedia and telecommunications 2005 (pp. 3715-3720). Chesapeake, VA: AACE.

Martin, R., Latour, T., Burton, R., Busana, G., & Vandenabeele, L. (2006). Tao: Several use cases of a collaborative, internet-based computer-assisted testing platform. In P. Kommers & G. Richards (Eds.), Proceedings of world conference on educational multimedia, hypermedia and telecommunications 2006 (pp. 2216-2221). Chesapeake, VA: AACE.