How much evidence currently exists within the RM literature relating to the area of benefits measurement?
As already noted the profession's best practice standards subscribe to measuring the efficiencies of the records management programme, but there is little available methodology documented to support such activity. Statements concerning efficiency are repeatedly made in the literature, more so in the professional magazines and vendor information than the academic journals. The statements are often articulated in terms of improvements such as 'decreased storage space' or 'reduced turnaround times for fulfilling customer orders'. The statements tend to be general or tenuous (or both).
Within the literature, in more general pieces the potential benefits of records management initiatives are illustrated with statistics that are frequently based on industry surveys. The industry surveys are often based on questions that are reflective: asking the respondent to reflect and give their opinion as to how they carry out information related tasks and how long they think they spend on them. The survey information provides a snapshot of industry opinions and perceptions of various issues relating to records, information, knowledge and content management.
The reasons for adopting such an approach are easy to understand. Surveys and questionnaires are a well established, relatively cheap way of gathering information on a particular subject - especially when seeking to understand the views and opinions of a large number of individuals. In the absence of any other established methodologies for gathering data on process performance prior to and after the implementation of records management improvements there may also appear to be little choice other than to ask the opinions of those involved.
However, there are clearly also significant limitations implicit with 'data' gathered in this way. How long people think they spend filing (for example) may vary considerably from how long they actually spend filing. Activities such as filing or searching for information are, unless very carefully defined; open to a considerable degree of interpretation. For some transferring an email from their inbox to a subject folder may count as a filing activity whereas others may think only of the formal addition of new information to a registered file as being relevant. Plus, we should also recognise that for most people, activities such as filing and searching for information count as 'background tasks', part and parcel of modern office life and not, therefore, something they give much consideration to. As such, any retrospective musings as to how long it takes or how frequently they are performed are likely to be vague at best. The fact that these tasks also take the form of 'chores' for many may also inadvertently steer them towards an overestimate of the amount of time it takes (or 'wastes' in their eyes) during the day. Lastly of course it is possible that the recipient of the questionnaire may have their own reasons for wishing to interpret and respond to the survey in particular ways: perhaps they hope that stating a high figure for the amount of time 'wasted' during these tasks will add weight to their argument for extra administrative help within their department, or view it as a convenient means of blaming 'the system' for failure to achieve targets etc.
As Asprey (2004, pp.7-31)6 states rather succinctly in his opinion piece: 'Industry analysts, product suppliers and systems integrators often publish statistical information, about the average time spent by people locating information within organisations, and the benefits of enabling systems. These statistics might prove helpful when building the business case, but the methods used to derive the statistics may also need to be examined. Practitioners need to determine whether the analyses are relevant to the specific enterprise, and whether the statistics can be qualified and quantified.'
None of these are reasons not to undertake such reflective, user-focused data collection. Indeed user-perception may have an important role to play in terms of measuring 'user satisfaction' with any improvement undertaken. How popular a new system or process is with those who use it and how successful it is perceived by them to be can be almost as important to know as how successful it has actually been (particularly when both sets of data are combined) but it should be noted that they are different and that we should be careful of applying the same weight of evidence on user surveys as on empirical data derived from an accurate and repeatable process.
There are also repeated cases within some of the literature where statistics are quoted but no further reference or a partial reference to support the origin of the information is cited. Without provenance the information provided lacks weight and substance and common sense suggests that it should therefore be used with caution (though this seldom seems to be the case). Such statements that cannot withstand further scrutiny weaken the literature available and the records management profession's continued reliance on them despite their obvious flaws does its professional credentials no favours.
