Author: Syeve Jones, s.r.jones@leedsmet.ac.uk

JISC e-Learning Activity Area: e-Assessment

Higher Education Academy Subject Centre: Hospitality, Leisure, Sport and Tourism

This case study illustrates...an effect on exam results, an effect on student personal development, innovation in learning and teaching, staff satisfaction with e-learning, use of resources

Background & Context

This example looks at the use of CAA on a module with large numbers of students. The assessments comprise 5 stage tests worth 10% each and an end exam worth 50%. Tests have both formative and summative aspects. Drivers for developing this approach were to give quick feedback to students from early on; to introduce them to HE assessment processes; and to save staff considerable time marking 350 scripts.

Why did you use this e-learning approach?

This approach was adopted on courses in Events Management and Tourism Management for a number of reasons. These included the University assessment, learning and teaching strategy that stressed the use of computer assisted assessment (CAA). One of the specific targets of the strategy is to 'expand the use of computer aided assessment for both formative and summative assessment, with at least one appropriate and well designed CAA assignment in each undergraduate course by 2007'.

Additionally, we wanted to give early and rapid feedback to level 1 students in semester one as to their progress in a higher education environment.

Finally, we wished to introduce students to the rigid deadlines demanded by the University against the expectations of a number of students arriving from further education, where deadlines for assessments could be negotiated.

What was the context in which you used this e-learning approach?

This approach to e- learning is used on the module 'Applied Technology and Finance' taken in semester one of level 1 by students on both tourism and events management courses. There are a large number of students taking this module (in 2006-07 around 350 students were enrolled on this module) with a proportionate number of staff looking after workshop/seminar groups.

Finance and the use of technology subject areas can often cause some stress and difficulties to students who may have struggled at school with numeracy and IT literacy.

Prior to the adoption of this approach, assignment submission was on paper and took a considerable amount of time to mark and feedback to students. In addition to this, it was not possible to give the students multiple opportunities to receive feedback; the students completed one end of module assessment and the mark and feedback was returned three to four weeks later.

This approach to e- learning has presented few problems or challenges in terms of implementation since the drivers for change came from staff teaching on the module. These staff subscribed to the principles of rapid feedback and frequent feedback described in the introduction to this approach. There was some concern around the final assessment involving all 350 students and whether the virtual learning environment (VLE) would cope with this level of traffic.

What was the design?

The students were introduced to this approach at the very beginning of the module with full details of what was expected of them detailed in a module handbook given out in paper form and available on the VLE. The students were inducted into an approach that asked them to complete prescribed reading and activities for each week and supplied them with supplementary activities and reading that they were encouraged to undertake. Every two weeks starting at week three the students had to take a test (summative assessment) with the questions based on the activities of the previous two weeks along with a small number of the questions based on the supplementary reading and activities. The students received their mark immediately on completion of the test and feedback was made available at the end of the week when the tests took place. At the end of the module a formally scheduled exam was held using the VLE which all students took on the same morning.

During the initial development of this approach the design process was led by WebCT champions who also happened to be members of the module team. Subsequently all tutors on the module have taken part in the incremental development of this activity and the dissemination of this approach.

How did you implement and embed this e-learning approach?

The roll out of this approach was relatively straightforward with few issues. No other staff other than the module team were involved and the students were thoroughly briefed before assessments and before the end exam. The VLE support team did however monitor the impact on resources of the final exam; this proved to be remarkably low.

Initial evaluation involved meetings of the module development team on a regular basis to focus on implementation issues and to discuss the outcomes of any discussions with students on this approach. In the initial pilot of this approach students' opinions were sought on a regular basis.

In initial trials of this approach a number of students missing tests asked tutors if they could take the test the following week. Since the desks were akin to examinations this was not normally possible, but the decision was taken to allow students to take the tests during the week that their normal class took place. In the first week the students could retake the test just by asking, but in subsequent test sessions only plausible excuses were accepted culminating with the need for the student to submit mitigation for their absence from the final test. The main reason for this was that a number of students were not used to strict University processes and the team wanted the chance to reiterate this over a number of weeks.

Although the tests and the final exam were held in exam like conditions, a small number of students complained that they were aware of students overlooking the computer screens of other students taking the test. This was made possible because initial tests and exams were presented to all students in the same way. After this complaint, question banks were extended and questions were selected at random by the VLE - typically this meant that students would get 20 questions presented randomly from a bank of 30 questions. At the same time the decision was taken to randomise the order that the potential answers appeared in for each multiple choice question. Interestingly, large number of students commented on this approach and the initial complainants noted that students said it was virtually impossible to look at answers on the computers of those around them.