Author: Pru Marriott, pmarriott@glam.ac.uk

JISC e-Learning Activity Area: e-Assessment

Higher Education Academy Subject Centre: Business, Management, Accountancy and Finance

This case study illustrates...an effect on learning, an effect on exam results, an effect on student personal development, student satisfaction with e-learning, innovation in learning and teaching, use of resources

Background & Context

The use of phased online summative assessment on a first year undergraduate accounting module.

Why did you use this e-learning approach?

The use of phased online summative assessment is recognised as an important and powerful assessment practice which supports high-quality learning and teaching. The rationale for the change in assessment was in response to students' poor performance, lack of participation in the learning process and superficial engagement in the topic area and focussed on:

• measuring student development and identifying areas where additional support is needed
• providing students with timely feedback on their progress in order to facilitate the self- reflective process that impacts on their personal development
• facilitating high quality learning through the provision of effective and timely feedback that enables students to advance their learning and be actively involved in their own learning

What was the context in which you used this e-learning approach?

The students in the case study are first year accounting and finance degree students studying six 20 credit compulsory modules: Financial Accounting, Management Accounting & IT, Quantitative & Research Skills for Accountants, The Economic Environment, Organisational Behaviour and Business & Company Law. The delivery of the undergraduate programme is linear and the module is taught for 24 weeks over the academic year. Weekly class contact consists of a 2-hour workshop and a 1-hour lecture.

Sixty students (33 male and 27 female) were enrolled on the module at the beginning of the academic year and all had achieved at least 220 points at 'A' level, although very few of them had studied accounting to this level.

The case study concerns first year accounting and finance degree students enrolled on the Financial Accounting module assessed by an in-class time-constrained case study, weighted 30%, and an end of year 3 hours' time-constrained examination.

Changing the method of assessment in a significant way was challenging in terms of how the students would engage with the revised assessment practice and how effective it would be in improving student learning. Given the need for the multiple delivery of tests, the potential for collusion between students was identified.

What was the design?

QMP was used to develop 5 phased online summative assessments. At the commencement of the module students were issued with a lecture programme which highlighted the dates of their online summative assessments and the topic areas to be tested (see Table 1 below).

Table 1 - Assessment Timetable and Topics Tested

A bank of 400 questions was created, separated into topic groupings, and comprised multiple-choice, multiple-response, true/false, yes/no and text match questions. Five online assessments were written from these questions and typically students would receive a test of 30 questions taken from a topic grouping comprising of 60 questions.

The feedback to each test included a score for the test, a summary of the questions asked and the student answers to those questions and an explanation of the correct answers.

A QMP expert, employed at the university, was instrumental in the initial creation of the question bank, from questions provided by the tutor, and responsible for delivering a series of training seminars on the use of QMP. Ongoing technical support was also provided throughout the duration of the case study.

How did you implement and embed this e-learning approach?

Sixty students were enrolled on the module and all students were given the option of taking the tests online or paper based. One student within the group had a visual impairment which precluded her from sitting all the tests online and another requested a paper-based copy, out of preference, for the first test only - he voluntarily reverted to online assessment thereafter.

Prior to each test, hard copies of test papers were prepared as a backup should there be any technical issues which would prevent online delivery of the assessment. The contingency plan was not needed as all tests were successfully delivered online.

All the tests were time-constrained (50 minutes) and sat under exam conditions in a supervised on-campus computer lab during the students' workshop sessions on pre-determined days during the academic year.

The limited size of computer labs required multiple deliveries (3 groups of 20 students) of the tests which were undertaken during the same weeks but at different times. To prevent unauthorised and unscheduled access the tests were password-protected with the name and password being changed for each sitting.

To prevent collusion between students during and after each session, the tests comprised questions which were randomly selected from the question bank and then randomly ordered on the assessment papers. Additionally, the answers linked to the multiple-choice and multiple response type questions were 'shuffled'. These safeguards resulted in students receiving a unique set of questions each time and removed the opportunity for unfair practice.

An anticipated issue was raised by the external examiner concerning the relative difficulty of each test. This was addressed by the design of the assessments where each combination of test comprised an equal number of questions testing topic specific knowledge and an equal distribution of difficulty levels. The levels included easy, medium and difficult. The chance of any test being significantly more difficult than any other was removed.