3.7.3. Developing your Evaluation Plan

This is taken from ILRT's Evaluation Toolkit, which can be found at http://www.ltss.bris.ac.uk/jcalt/. You may find it useful when developing your evaluation plan.

An evaluation plan will help you

• identify the intended participants or audience for the evaluation (stakeholder analysis)

• define your evaluation questions

• focus on defining the complexity or scale of the proposed evaluation study

• take into account the associated time and resource (both financial and human) implications.

There are five principle questions to consider:

• What are you evaluating?

• What are your reasons for carrying out the evaluation?

• What is the context within which your evaluation takes place?

• Who is the evaluation for?

• What evaluation questions are you going to ask?

Step 1. What are you evaluating?

It is possible to evaluate both processes and products, depending on your focus of interest. You may want to evaluate the success of your MLE development against the project plan, alternatively you might be interested in understanding the barriers and enablers to implementation. It is important to be clear about the focus of your evaluation as this will influence the data collection used and the findings. Provide a brief description of what your evaluation will focus on.

Step 2. What are your reasons for carrying out the evaluation?

The table below lists a range of reasons why you might want to carry out an evaluation. Consider these in relation to your MLE development and decide which are appropriate. For example you may be focusing on ?selecting? as part of your MLE development might be about choosing and integrating a commercial VLE with other systems in the institutions. Alternatively you may want to understand the issues surrounding MLE development in which case the nature of your evaluation will be more ?explorative?. A common focus for evaluation is trying to assess value for money and cost effectiveness.

Step 3. Context of the evaluation

This allows you to consider the resources and expertise which you have available to carry out your evaluation and timescales and any constraints.

Step 4. Who are you doing the evaluation for?

This section will help you to define key stakeholders in your evaluation. Although ?stakeholders? are sometimes defined as the groups who have the ability to influence the situation, here the term is used to refer to anyone who has a stake in the evaluation process or the activity/implementation.

The table below lists a set of potential stakeholders and some ideas for what their concerns might be. The list of stakeholders and of concerns is not intended to be authoritative or exhaustive; users should adapt the list to their own local circumstances, adding in-groups and/or concerns as appropriate.

• Select the stakeholders who are important for your evaluation (or add your own stakeholders).

• Select appropriate concerns or add to the list with your own examples.

• Remember you are invariably one of the stakeholders and you should include this in your choice. You will need to decide which of the categories you fit into.

• We recommend that you limit your choice to a maximum of two or three stakeholders only and would also suggest that you allocate one of these as the primary focus for the evaluation.

• Try and also identify a maximum of three concerns from your list which you can work with in the next stage of the process

Step 5. Devising the right evaluation question

The questions prioritised as the focus for your evaluation can have a large impact on the design of the study. Additionally, the different stakeholder groups will see certain types of questions and the resultant data collected as more persuasive than others. For these reasons, it is important to be aware of the range of different types of questions that could be asked. The table below gives you some examples of the types of questions you might consider asking.