Practical Considerations
Practical considerations include having sufficient and sufficiently robust information technology systems available for students who are required to engage in e-assessment, as well as ensuring that students have sufficient access to the facilities. For security reasons it is important that summative assessments are not placed on part of the IT infrastructure where students can find them, and that results are posted securely. However, security is less of an issue when the assessment is purely formative.
Sim et al (2004) provide information about the British standard code of practice governing the use of information technology in the delivery of assessments (BS7988 (2002)) to which institutions should adhere. This code of practice for the use of information technology in the delivery of assessment recommends that students take a break after 1.5 hours. This has implications for invigilation of examinations.
Choosing a Software Package
The main considerations here will be in terms of cost efficacy, whether proprietary software has to be bought or whether software is available free or bundled with an institutional VLE/MLE. The amount of time required for developing the assessment items, including the programming that might be required will be an important consideration, as will the level of IT support required from the institution and locally within departments. the considerations, cost efficiency, time for development. A key issue will revolve around whether the assessment is to be formative, which can be conducted locally and probably requiring little in the way of backing, or whether it is summative requiring a large degree of institutional support to maintain quality assurance issues.
Security
Many versions of computer software for e-assessment allow for questions to be presented to students in a different order, and can provide the key and distracters in varying. If sufficient questions have been compiled of sufficient integrity then they can sit different tests, and for mathematically underpinned assessments, the programmes also provide the means to create, at little cost, variants of the same questions. All of this allows for students to sit in adjoining desks in computer laboratories that will at other times be used for other learning activities. This is fairly straightforward for on-campus students, but could be more problematic for students taking courses at a distance. However, Rovai (2000) suggests that this difficulty can be overcome by using 'proctored testing' where academics arrange for students to sit online assessments under test conditions in alternative venues. However, Sim et al (2004: 224) believe that:
'If security measures are in place there is no evidence to suggest that the integrity of the examination is more compromised by delivery over the Internet than by paper.'
Accessibility
When creating e-assessments that are to be as inclusive as possible, there are a variety of things to bear in mind:
What pedagogical issues are there to consider?
Design for All (where this can be achieved while not reducing the overall effectiveness of the assessment for other students)
Selection of mode of assessment and subsequent assessment design with accessibility in mind
Should enrich the experience for all users, not increase accessibility for some by reducing experience for others
Is not 'the lowest common denominator'
Is not bland and anti-technology
Basic principles are easy to apply
e-Assessment (as an alternative to paper-based assessments) can benefit those with mobility difficulties, illnesses, mental health issues and so on
Think about alternatives of equivalence
Not a case of when all else fails, thinking about possible alternatives will improve the whole assessment process
Encourages lateral thinking about choice of mode of assessment and content
Not 'only' for students with disabilities - all students have preferences, strengths and weaknesses
Can be costly and time consuming, particularly in the short term, but can reap dividends later in terms of marking time and administration year after year
Make use of student input to the design - nobody knows better what is achievable, what will aid the learning process and what will best improve the assessment experience
Validity
Alternatives and modifications need to be authorised by the validating body - the exam board, university board or whatever. The degree of difficulty with this varies tremendously between boards, but most will be interested in facilitating the assessment process for the widest possible cohort of students
What are the common mistakes?
Often we inadvertently assess things which are not a part of our stated outcomes, but instead are a function of the students' physical or social capabilities - we need to determine our policy on each of these prior to setting the assessment, and to communicate this policy to the students in time for them to make representations if they have issues with any of them:Speed of writing or typing are not normally an outcome of an assessment (except keyboard skills tests etc) - imagine a student with a very slow typing speed in an essay-style exam - what would be the best course of action to allow them to exhibit their knowledge? Sometimes extra time would be appropriate, sometimes the use of another input method such as voice or a scribe, and sometimes a completely different but equivalent mode of assessment
We should determine prior to each assessment whether or not we are adding or deducting marks due to spelling and grammar. If this is the case, we need to consider how we will view the work of dyslexic or blind students (for example, a blind student using voice input software will not know whether the programme has inserted the word 'court' or 'caught' - should we penalise a student for this?
Are we assessing manual dexterity? Should a student be marked down because they cannot pour liquid from one flask into another, despite knowing everything about the chemical reaction involved?
Are we assessing visual acuity? Auditory perception? Short-term memory? Physical endurance?
Where can I get further assistance on this subject?
The Teachability project is a resource booklet describing a step-by-step audit process towards making a course or module more inclusive for disabled students, including of course the consideration of assessment.
The Web Content Accessibility Guidelines (WCAG) published by the Web Accessibility Initiative (WAI) are a checklist for making web pages accessible (a full debate about the usefulness of 'pure' accessibility as against usability can also be found on the WAI website). Although they are oriented towards web pages they are extremely pertinent to any online materials and therefore to e-assessment.
Is there a simple 'checklist' of considerations I should make?
Not a definitive one, as every student and cohort of students will have different needs and every assessment and institution will necessarily be different. However, there are some considerations which may be considered to affect a greater proportion of students and therefore may merit the most immediate attention (NB This is not an excuse to forget about any other needs your students may have!)Navigation - can your assessment be completed without using a mouse? Can check boxes be checked using the keyboard alone? Are drop-down menus accessible using only the keyboard?
Keyboard dexterity - do users need to be able to press more than one key simultaneously? If text is entered incorrectly or by accident, can the user backtrack and correct their actions? Are there alternatives for users who cannot access drag-and-drop features (NB there is often no need to remove drag-and-drop exercises because some students cannot access them, it is often far more beneficial to create alternatives)
Audio content - are alternatives, transcripts or subtitles available?
Language - is the assessment, particularly the instructions and navigation, clear and precise? Have you used 'trick' multiple-choice questions which use similarly spelled words as spoilers, which may confuse dyslexic students?
Have you examined your images and graphics at high magnification to check they are still legible? Images of text in particular become pixelated at increased magnification, meaning they become illegible to vision impaired students.
Does the layout make sense to screen reader users? Screen readers progress linearly through a page only if it has been coded to allow this, otherwise they dart about and can be very confusing. Tables in particular can be very difficult for screen reader users to cope with if badly designed. Read aloud your table row by row and see if the question is still achievable without having the table available visually. Do images and graphics have text tags or alternatives for students that cannot see them?
Be careful with use of colour - do not use colour alone to convey meaning. Be sparing with bright colours - think of magnifier users who may view your pretty little lime green heading sixteen times larger than you imagined!
Is it possible to give the student control over text size, font, and colour? If not, have you selected options that will be available to as many students as possible (sans-serif font, minimum 12 point font size, 1.5 line spacing etc)
If you are not sure whether you have succeeded in making your assessment more inclusive and accessible, test it with students - but if you are seeking disabled students to test your assessment with, please approach them via the student support staff, as they invariably get pulled in a number of directions at once!
