Measuring Benefits
Groundhog Day
'We need new IS capability to deliver business benefits; a business case is prepared; it's approved on the basis of the benefits; the project delivers new capability; it's closed and we move on to another requirement. But something is missing - we've concentrated on delivering the capability, and ignored the benefits, when what we wanted in the first place was the benefits, without which there's no point in delivering the capability.'
Benefits are only likely to be realised after the individual project outputs move into operational use. In fact the benefit may only accrue after a considerable period of operation.
If one of the desired benefits was 'improved retention and achievement' it is likely that the measurement of this will only be practical at the end of an academic year and that the academic year in which the project delivers into operational use may not yield a true picture thus it may take at least two years to take meaningful measurements of such a benefit.
Where staff are finding the transition to new working practices challenging it can be helpful to use early benefit analysis to overcome resistance.
It is the job of the benefits manager to ensure that the operational use of project outputs runs smoothly and remains focussed on the achievement of the benefits as listed in the Business Case. Identifying early indicators that the benefit will or will not reach the target measurements is one task that will allow remedial action to be taken if a shortfall is forecast.
It can be useful to go back to the Benefits Profile at each stage boundary as a way of ensuring that relevant measurements are being taken in the way that was originally specified and in a way that is comparable with the original measurements taken during the before-change period.
When attributing benefits to investments it is also important to identify, and, if possible quantify, any achievements attributable to other factors. It is also important to have been realistic about the likely benefit achievement of deliberately introducing multiple factors.
MSP
In MSP methodology this is covered by (chapters given in brackets):
- Benefits Realisation Management (7)
- The Business Case (10)
- Quality Measurement (12)
- Defining a Programme (15)
- Managing the Tranches (16)
- Delivering the Capability (17)
- Realising the Benefits (18)
- Closing a Programme (19)
- OGC MSP website
For example:
It is forecast that refurbishing the building environment will increase retention by 3%
It is forecast that having better pastoral care facilities would also increase retention by 3%
It is forecast that introducing more student control of course content via web2.0 technology would increase retention by 3%
Would introducing all three factors increase retention by 9%? It is unlikely. Implementing multiple projects at once can lead to diminishing returns and may make attributing benefits to individual projects a subjective nightmare with potential for conflict between champions of the different projects.
Good programme management should reduce the incidence of this type of possible double counting.
