Author: Simon Wilkinson, simon.wilkinson@nottingham.ac.uk

Author: Heather Rai, heather.rai@nottingham.ac.uk

JISC e-Learning Activity Area: e-Assessment

Higher Education Academy Subject Centre: Medicine, Dentistry and Veterinary Medicine

This case study illustrates...use of specialist software, an effect on exam results, student satisfaction with e-learning, innovation in learning and teaching, an influence on educational research, staff personal development, a positive effect on recruitment, a positive effect on retention, an influence on policy, use of resources, modifications to learning spaces, management of learning assets, an effect on social equality

Background & Context

Why did you use this e-learning approach?

The use of high-stakes summative assessment is important for all stakeholders. From the students' point of view failure at examination time can have serious progression/career implications and can be costly in terms of wasted fees. From the staff perspective being able to consistently and rigorously defend standards is at the heart of any robust assessment process. However, a key interface between these two groups lies in the accommodation of various forms of disabilities. Being able to pass exams is critical to progression through all degrees, but failure to adequately address potentially discriminating features of an assessment can result in students with certain forms of disabilities being unfairly disadvantaged. This case study describes steps taken to enhance an online assessment system TouchStone to afford a wider range of accommodations. The main aim being to focus on the measurement of subject matter understanding rather than a students' ability to interact with a particular assessment format.

What was the context in which you used this e-learning approach?

A number of new laws have been enacted to protect the interests of individuals who study in higher/further education and have various disabilities: Disability Discrimination Act (DDA) (1992) and Special Educational Needs and Disability Act (SENDA) (2001) in the UK, and Individuals with Disabilities Education Act (1991) in the US. Analysis of applicants awarded places at the University of Nottingham reveals self-reported rates of disability ranging from 3% to 9% across different faculties. This case-study is written from the perspective of the Medical School within the Faculty of Medicine and Health Science. Reported numbers of students with some form of disability are low at 3%. Interest in accommodating individuals with disabilities has grown as the number of online summative exams has increased. As Konur (2007) notes that examinations emerge as a more critical issue for all stakeholders (staff and students). Without proper consideration the move towards increased online assessment could exacerbate any potential disadvantages for individuals with certain forms of disability.

What was the design?

The priority was to create a design that could be: a) implemented quickly before each exam with no specialist personnel required, and b) could be used on any computer used for examinations rather than being limited to specific machines. Also, because of the uniqueness of examinee requirements the system must permit accommodation at the individual level rather than whole groups. Training staff and the use of client-side cookies to remember settings were all considered but in the end the most effective and robust solution considered was to create an additional database table to hold individual requirements. Utilising an extra table within the main CAA system had the advantage that accommodations can be 'distributed' to any client machine that the candidate sits at for an exam and is easy to centrally administer. A new simple interface was then added to the system (see figure below) to allow non-technical staff to easily set student preferences.

Staff involved in the design included disability support officers from the Disability Policy Unit regarding the most common types of accommodations that might be required by students. Administrative personnel who timetable exams were also involved to make them aware of the range of customisations that can be performed in the system and finally IT support staff who may be required to actually set up individual examinee profiles.

How did you implement and embed this e-learning approach?

The first step towards embedding the approach was to talk with experts from the Disability Policy Unit within the University to try and better understand the range of accommodations that might be required. This was a two-way process whereby the unit advised the programming team on what in theory would be needed, and conversely as the design was implemented the programming team confirmed back with the unit what was actually possible with the technology.

The design is now formally embedded within an official departmental/institutional policy. Firstly an assessment method is approved at school teaching committee level. The method (i.e. online examination) will then be publicised to students via a VLE. If any students are worried that the format of the exam may unfairly disadvantage them then they are referred to the Disability Policy Unit for assessment. This unit will then report back to a faculty-based administrative unit with recommendations for any exam adjustment and then this will be relayed to an IT person to alter the examinees' profile within the CAA system (see Case Study: University of Nottingham, e-Assessment, Medicine).

Importantly the student is then informed that the changes have been made and asked to evaluate these adjustments by accessing any of the many online formative assessments via the VLE.

Currently accommodations are only made for individuals which have been properly assessed by the Disability Policy Unit. Thus far this has not caused any problems with the non-disabled student body.

Building on the success of personal adaptation within the CAA system, the University of Nottingham Medical School has now embarked upon a re-design programme to allow customisation of the VLE. Although learning through a VLE is not subject to the same time pressures of a summative exam, any adjustment that can result in faster or more readable text for dyslexic or colour blind users must be considered advantageous.