Interoperability and Reusability
Many tutors have reservations about the time required to implement e-assessment and of its educational effectiveness. To address these concerns there has been considerable focus on the development and implementation of question banks (Bull and Dalziel, 2003). A question bank is a collection of questions that can be reused and shared across a learning community (usually within a particular subject discipline). Question banks allow users to contribute their own questions as well as reusing questions created by others tutors. This approach is useful for two main reasons. Firstly, there is concern over the effectiveness of CAA. This is because many of the questions are multiple choice and it can be difficult to draft MCQs that test students' understanding of particular concepts (as opposed to questions that jog memory recall) (Bull and McKenna, 1999). This requires both creativity and hard work before a question can be used effectively to measure students' learning (Haladyna, 1997). Therefore, reusing questions that have specifically been drafted to test students' understanding can help with this problem. Secondly, by sharing and reusing questions, tutors can benefit from economies of scale (Littlejohn, 2003).
The problem is how easy is it for you to reuse a question within your own VLE? Will this involve completely rewriting and restructuring the question? Until recently, the answer to this question has been 'yes': the reuse of question tests has simply not been worth the time invested in making a question reusable. Now, however, interoperability standards have been developed and implemented to allow questions to be easily transported from one VLE or e-assessment system to another without losing their structure. One such standard is IMS Question and Test Interoperability (QTI).
QTI is a framework for transferring assessment questions, tests and results from one VLE to another (Sclater and Cross, 2003). QTI has four main areas: the assessment tasks; the results after a test has been taken; the ordering of questions and tasks; and the processing of test scores (taking into account weighting factors, etc). This is useful for three main reasons. Firstly, it allows tutors to develop online assessments using a range of question types. Secondly, tutors can share assessment information across different e-assessment systems. Thirdly students' results can be sent to centralised, institutional student record systems.
A number of these test banks have already been set up. These include:
COLA - a bank of items and assessments for Scottish Further Education (www.coleg.org.uk/ requires membership) and
Electronics and Electrical Engineering Assessment Network (e3an) - a bank of peer-reviewed questions being reused across UK Higher Education
Universities Medical Assessment Partnership (UMAP) - being developed between a consortium of Medical Schools in England
COLA Project Sclater and MacDonald (2004: 208) "The assessments had to work in a range of VLEs and it was expected that the VLEs would accept these item types if they were marked up using the IMS Questions and Test Interoperability v1.2 (QTI) specification (IMS, 2002)
COLA Project Sclater and MacDonald (2004: 211) "It was necessary to develop a program to convert the items and assessments from the word templates to the QTI format. This task was carried out by the JISC-funded Technologies for Online Interoperable Assessment (TOIA) project which had the necessary expertise in QTI in collaboration with an expert group representing the four main VLEs"
