Assessing e-Portfolios: Guidance for Practitioners
'Students are increasingly engaged in a broad range of activities that make up the wider student experience: engaging in activities, volunteering, ...Graduates want and need the ability to communicate these skills and experiences to their future employers.'
Streeting (2007)
There is debate whether e-portfolios should or should not be assessed; much of this is linked to different ideas about the primary role and purpose of e-portfolio. One approach is for tutors to ask: What is being assessed: the product or the process? As Barrett and Carney (2005) state many of our assessment tools and procedures are focused on the product of learning rather than the process. From an HE perspective, the outcomes of the REAP project are critical in informing this debate in the context of re-designing assessment practices.
Links
There are those who favour using e-portfolios for assessment as a means of capturing valuable material developed from a process of learning. Assessment also ensures the engagement of all students and staff (as seen for example, in some of the participating groups in the FILE-PASS project) as well as raising the awareness of Personal Development Planning (PDP) supported by e-portfolios. Furthermore, e-portfolios may provide an opportunity for the sector to measure and record non-formal learning and update our, often antiquated, assessment procedures.
In contrast, those who do not favour assessing e-portfolios stress that the process of reflecting on learning is too personal and would require complex evaluation procedures. Arguments persist about over-assessment in the sector and that if PDP is about helping students to be independent learners, then the assessment of PDP is removing this element of independence (Atlay 2005). Also, learners are not very keen on being assessed (summatively) on their reflections - 'if this is my personal reflection, how can you give me 3 out of 5?' (David Tosh and Jeff Haywood of SCROLLA, 'Students and e-portfolios: What's in it for me?', ALT-C 2005).
'If used for formative assessment purposes, rather than summative evaluation, portfolios can be powerful devices for learning'
Barrett and Carney (2005)
Barrett and Carney draw the distinction between using the portfolio product for the summative assessment of learning rather than using the process of the development of the e-portfolio to support the formative assessment for learning. In the table below, they emphasise that formative assessment is essential for e-portfolio development as feedback allows the learner to reflect, change and improve work.
Portfolios used for Assessment of Learning
Portfolios that support Assessment for Learning
Purpose of portfolio prescribed by institution
Purpose of portfolio agreed upon with learner
Artefacts mandated by institution to determine outcomes of instruction
Artefacts selected by learner to tell the story of their learning
Portfolio usually developed at the end of a class, term or programme - time limited
Portfolio maintained on an ongoing basis throughout the class, term or programme - time flexible
Portfolio and/or artefacts usually "scored" based on a rubric and quantitative data is collected for external audiences
Portfolio and artefacts reviewed with learner and used to provide feedback to improve learning
Portfolio is usually structured around a set of outcomes, goals or standards
Portfolio organization is determined by learner or negotiated with mentor/adviser/teacher
Sometimes used to make high stakes decisions
Rarely used for high stakes decisions
Summative - what has been learned to date? (Past to present)
Formative - what are the learning needs in the future? (Present to future)
Requires Extrinsic motivation
Fosters Intrinsic motivation - engages the learner
Audience: external - little choice
Audience: learner, family, friends - learner can choose
Formative and summative assessment of e-portfolios (Barrett 2004b)
Barrett (2004b) drawing upon the work of Paulson and Paulson (1994) highlight this dichotomy when discussing the selection of artefacts for a portfolio from the positivist or constructivist approach. They argue that a positivist approach 'puts a premium on the selection of items that reflect outside standards and interests... the constructivist approach puts a premium on the selection of items that reflect learning from the student's perspective'. Barrett (2004a) also suggests that using e-portfolios for assessment may have an influence on how learners perceive the purpose of the e-portfolio process. She suggests that assessing the e-portfolio changes the purpose for the learners from a tool that can support lifelong learning development to a 'high-stakes' document that will be judged against a set of prescribed learning objectives. Thus, Barrett (2004a) suggests that learners will perceive an e-portfolio that is used for assessment purposes as 'something that is done to them rather than something they WANT to maintain as a lifelong learning tool'.
Issues relating to the use of e-portfolios for summative assessment
- Interoperability between e-portfolio and e-assessment systems
- Transferability between learning providers and external awarding bodies
- Security and authentication of user information and of assessment decisions
- Acceptability and credibility of data authenticated by Awarding Bodies and accrediting institutions
- A range of ‘views’ on to the data, with associated accesses and permissions
- Efficiency of assessment process – potentially a benefit, currently a risk
- Problem of piloting new systems in high-stakes summative assessment contexts
- Designing assessment strategies to make effective use of the new tools and systems (current lack of innovative work in the use of technologies to capture evidence)
Beetham, 'e-portfolios in post-16 learning in the UK: developments, issues and opportunities'
In the FILE-PASS Project in Institution C, nine 16-19 year old 'isolated' learners were introduced to a blog tool and asked to keep a blog during their programme of studies. This was not seen by the tutor but related to a reflective assessment on how the blog-writing had contributed to their development. The students appeared to gain confidence in writing and what they wrote. In interviews it was noted that the learners felt a sense of ownership of their blogs and how it had impacted on their ability to think about themselves and their behaviour. The blogs were not assessed or seen by the tutor. The tutor sensed a greater sense of ownership and pride than in previous paper-based portfolios which had been retained at all times by the tutor.
Approaches and Methods
Assessing e-portfolios presents a new challenge for tutors. Traditional assessment methods are not appropriate so alternative, and sometimes time consuming, approaches may need to be explored, e.g. engaging learners in debate and dialogue. Atlay (2005) suggests addressing the following issues when considering assessing e-portfolios:
- What are we assessing - is it the product or the process?
- Are we giving it a grade - should we use pass/fail or what?
- What weighting do we give it?
- How does it fit in with our existing approach to assessment?
Baume (2003) stresses that tutors must ensure that their students know that they will not be penalised for including less capable work. Learners need to know that they are being marked on being able to learn from their experiences and their identification of their future training needs: 'Students are likely to put considerable effort into presentation so you need to show them examples of - or tell them about - the minimum (and perhaps maximum) standards of presentation that will be appropriate'.
Part of this debate will also include discussion about the handling and processing of online portfolios (Stefani et al 2007). Institutions will need to re-consider their regulations for the handling of online assessments, for example:
- Are systems sufficiently robust to handle a large number of online submissions of e-portfolios?
- What will happen if tutors refuse to mark online e-portfolios?
- How will tutors comment on online portfolios and return them to students in line with current institutional assessment policies?
- Will all awarding bodies accept online e-portfolios? (Beetham and Strivens 2005)
- Will external examiners, auditors, moderators be able to access the e-portfolio online?
- Are there options for printing online?
There is also some concern about the authenticity of digital evidence. Some institutions may decide to retain both a paper-based portfolio as well as an online e-portfolio. If this is chosen as an option at any institution, staff and students will need guidance on how these will interrelate and how to support both.
In some cases, for example, in the School of Pharmacy at the University of Strathclyde, student could not graduate if the e-portfolio had not been completed. However, no marks were allocated because the primary purpose was PDP.
Providing feedback
The development of an e-portfolio requires feedback from peers, tutors and potentially employers. In Case Studies 1 and 2 of the ELP project, tutors found the e-portfolio systems' feedback mechanism onerous. preferring to provide formative feedback face-to-face.
Managing the Assessment of e-Portfolios
When introducing e-portfolios as an assessment tool, you will need to consider tutor reaction, system functionality, administrative procedures for handling online assignments and external examiner and professional body perspectives. Institutions will need to re-consider their regulations for the handling of online assessments. Some considerations are presented here, adapted from Stefani et al (2007).
Tutor
- How will tutors access the e-portfolio, mark online and add annotations and feedback?
- How will tutors return assessed e-portfolios to learners in line with institutional assessment policies?
- What will happen if tutors refuse to mark online? What support will be available to tutors marking online?
- How will tutors know if the digital evidence is authentic?
- Is it appropriate to have both a paper-based as well as a digital copy of the portfolio and if so, how will the two interrelate?
System
- Are systems sufficiently robust to handle a large number of online submissions of e-portfolios and handle them simultaneously?
- Who is allowed access to the e-portfolio?
- Are there options for printing?
- Does your system support anonymous marking?
External Examiners and professional bodies
- Will all awarding bodies accept e-portfolios? (Beetham and Strivens 2005)
- Will external examiners/auditors/moderators be able to access the e-portfolio?
