What are the challenges? - For institutions

1 Senior level sponsorship and engagement

BI systems struggle to be funded and to be implemented properly without strong sponsorship from the top of the organisation. It would appear that a BI implementation is not one of those things that works well when bottom led.

Some senior managers understand in theory how effective data management will help planning but don't know how to achieve this. Others simply don't think joining up data can achieve change. Some are good at articulating strategy, but are unable to set meaningful targets; whilst others ask for unrealistic data or reports.

2 Tendency towards bespoke rather than 'off the peg' system development

Underestimation of effort involved in creation of a data warehouse (and possibly other BI models too) causes institutions to try to do it themselves. This is almost always more expensive initially and certainly more expensive over the years when support is considered.

Some institutions think their needs are so specific they can't use an 'off the shelf' system from a vendor as a starting point, but need to develop a system 'from scratch' tailored exactly to themselves. This is rarely true. Vendor systems bring good value for money since development costs have been spread over many institutions, experience of what works so avoiding common mistakes, and can in any case be configured to provide a more personalised system.

There is a clear link between this institutional challenge and the 'inadequate core knowledge and experience across the sector' reported in the Challenges for the sector (No. 1)

3 Data quality

Most institutions interviewed have to draw data manually and at speed from dispersed systems in order to create meaningful reports. This jeopardises accuracy, reliability, consistency and is time consuming. Data may be 'owned' by those who operate it - this is fine operationally but not strategically.

You only get good predictions out if you put accurate data in. While it may be possible to control the quality of internal data sources this won't be so with external sources so some 'health warning' ought to be attached to data where the quality is dubious.

4 Data Definitions

Considerable concern has been expressed about consistent data definitions between external bodies (eg HEFCE & SLC) - see Sector challenge No.3 However, some also found that failure to understand data and related questions led to problems with source data, inconsistency of reporting and difficulties in obtaining consistent KPI data at a detailed (department and course) level within the institution, leading to difficulties with internal performance benchmarking.

This situation is compounded when institutions either have as a legacy issue, or continue to wish to have different systems or modules from different vendors (perhaps becasuse each is believed to be the best particular fit for the specific function in question). There is currently a lack of common standards that would allow these to sit together seamlessly.

5 Who should 'control' the BI system?

There is clearly an issue to be resolved regarding which department should have "control" of the BI system? By default it is likely the IT department will look after the working of the system but they will not be the consumers of the majority of the data, so who decides what service level agreements etc should be in place - those who have to deliver them or those who benefit and who, as a result, should pay?

Substantial change management programmes are taking place within many institutions. People realise that this requires a two-way conversation between service & academic departments which may each have different priorities. Planners need to continue to support departments and communicate and present data as effectively as possible; and managers need to own data and recognise it as a corporate asset.

The BI system will be the most flexible if it can respond quickly to changing requirements. This could mean that users need to be able to link to additional/new data sources themselves, however to preserve data integrity it may be best to limit the ability to make such links to professional administrators. There are difficult issues to resolve regarding a sensible balance point.

6 Business Case

Some institutions are finding it hard to quantify efficiency savings and return on investment (ROI) in terms of decision-making, and presenting a convincing business case. This may become a particularly serious issue in times of budget constraint where there may be a requirement to justify purchase of a BI system by showing how it will save money, and how much will be saved etc. And yet institutions may struggle to gather the data required for this justification without a BI system, thus resulting in something of a 'chicken and egg' situation.

7 Security

Obviously personal data must remain secure but too much security can reduce the potential value of the system because people don't have access to the data they need. A sensible balance is required and one which can be difficult to achieve within complex organisational structures.

8 Modelling

A number of institutions contacted asked for guidance for modelling on, for example: quotas for staying within HESA contract; data security; data handling; compliance etc. Successfully utilising your BI system for modelling and scenario planning purposes forms part of the 6th and final stage of our Maturity Model and, if realised, has the potential to extract the maximum benefit to the institution from their investment in BI.