Author: Michael Begg, michael.begg@ed.ac.uk

Author: David Dewhurst, d.dewhurst@ed.ac.uk

JISC e-Learning Activity Area: e-Assessment

Higher Education Academy Subject Centre: Medicine, Dentistry and Veterinary Medicine

This case study illustrates...use of specialist software, an effect on exam results, an effect on student personal development, innovation in learning and teaching, an influence on educational research, staff satisfaction with e-learning, a positive effect on retention, an influence on policy, use of resources, modifications to learning spaces, management of learning assets

Tangible Benefits

What tangible benefits did this e-learning approach produce?

• Evaluation of OSCA as an effective mechanism for delivering online assessment in the College has taken a number of forms. At assessment events themselves, external examiners question student groups as to how they found the exam. The examiners themselves have taken the time to effectively "sit" the exam themselves. The informally gathered, anecdotal feedback from these sessions has been wholly positive.
• Formal questionnaires were completed by students via the integrated evaluation engine within the medical VLE, EEMeC and these contained questions relating to the OSCE assessments. The feedback from students suggests that the question-delivery interface is clear, the navigation through question sections is clear, and the range of question types and their presentation is also satisfactory. They, on the whole, perceive it to be an appropriate and "professional" way of conducting exams.
• The use of virtual OSCE stations (as developed and delivered via OSCA) has reduced the number of physical OSCE stations and consequently the manpower needed to support them. Since OSCE examinations are auto-marked has also resulted in considerable savings in staff time. The facility to support grouped marking (the alignment of same and similar responses for grouped checking and marking) has also contributed to the decision to continue and evolve the use of OSCA.
• The chances of student collusion outside the examination venue are significantly reduced as even large-scale events require only two back-to-back sittings.
• With each event comes new questions which contribute to a growing bank of questions that can be drawn upon for future events. This should lead to more efficient creation of future questions, as authors can model new questions on tried and tested question types.

Did implementation of this e-learning approach have any disadvantages or drawbacks?

• The system requires a significant amount of staff development work by learning technologists and teaching fellows in acquiring suitable resources for digital delivery, and composing of questions in ways that "make sense" at a system level.
• The reliance upon limited physical space within the institution suitable for delivery of online assessment may soon lead to constraints upon how much can actually be delivered via the system.

How did this e-learning approach accord with or differ from any relevant departmental and/or institutional strategies?

Although begun as a College level resource - which solicited a degree of discord with the institutional level support for QMP - there is now a perceived acceptance not only of the College's need for OSCA, but that the application may be of use outwith the College. The possible friction that the dual systems may have given rise to have been in part abated by the efforts of the teaching fellow who, through the course of evaluating the systems in use within the College maintained good dialogue with central IT support bodies as well as local College staff. OSCA has certainly impacted on assessment policy in the College and if adopted more widely may impact on University assessment policy.