3.4. Finding and using information
Finding out information can be time consuming, so it is important to be clear about what information you want to gather, how you are going to find it and how it will be used. For example if you have a clear understanding of different stakeholders and their agendas, this can be built into your dissemination strategy for the project and will also give you a clearer idea of who should be involved in the MLE project and at which stages. Similarly a knowledge of internal enablers (such as potential MLE champions, existing projects which might be linked more closely with the MLE work, funding sources which could be combined, etc) and external enablers (new portal software which might be included, guidelines and resources for MLE implementation, funding sources which could be used to supplement internal funds, etc) will allow you to consider how and when these might be used throughout the plan to best effect. The same is true with barriers and potential problems which might impact on the success of the MLE implementation.
This section outlines some of the techniques for fact finding and their pros and cons. These are only illustrative, a host of other data collection methods and techniques are of course possible and the references in the resource sub-section give pointers to further examples which you might want to consider. Also see the section 'Gathering requirements' on 'Trawling for information'. If you are one individual working on this it is unlikely that you will have either the time or the resources to use these in-depth however it is still useful to outline them for you as they will give you an idea of the range of methods and approaches that can be used.
- Surveys and questionnaires
Surveys and questionnaires are a standard means of data collection. They have the advantage of being easy to collate, with simple response and closed question types being amenable to quantitative analysis. However good questionnaire design is a specialised skill and requires considerable time investment to get meaningful results. Furthermore, there are often low response rates to questionnaires. One final point is the importance of careful targeting of respondents, which can be particularly difficult in terms of MLE developments where it is difficult to pin down who key stakeholders are and where they are located in different parts of institutions.
- Interviews
Interviews with key stakeholders can be a valuable mechanism for both initial fact finding and ongoing analysis of stakeholder views and perspectives on your MLE development. It is important to spend some time establishing who key stakeholders might be and their relevance or interest in the project outcomes. However, interviews can be both time consuming in terms of arranging, doing the interviews and then analysis of the results. You need to be clear about both the purpose and the format of the interviews.
- Focus groups
An alternative to interviews is to carry out a series of focus groups of representative stakeholders. As with interviews it can be difficult to find an appropriate time to get users together for the focus group. You may want to employ a range of strategies for organising focus groups in a way that is confluent with other demands on participants' time, such as carrying out a focus group at the end of an existing meeting or linking it in with a staff development event. Both interviews and focus groups can be used to provide a range of views about the organisation, political and economic issues facing your institution and the challenges and opportunities you might face with your MLE development. It is useful to build up different models of the nature of your organisation to capture these different aspects. A methodology for this is outlined in the modelling section, where the different metaphors for description organisations and the kinds of information they yield is discussed.
- Participant observation
Another technique for gathering data is observation; this can be used in a variety of circumstances such as observing the use of new software to establish usability issues or exploration of general navigational issues. Observation can also be carried out on user engagement in workshops or the use of materials to identify their effectiveness. Data can be gathered by a researcher taking notes on specific activities or outcomes and might be supported by a audio or video recording of the session. However, it is worth noting that the latter adds considerably to the analysis time in terms of transcribing and interpreting the recordings. If you do decide to carry out this more detailed analysis, you may need to identify additional sources of funding as it is unlikely that you will have core funding from your organisation to do this.
- Process mapping
Process mapping has been used by a lot of projects as a useful means of identifying existing structures and practices. One mechanism for achieving this is to build a systems picture of the institution having already obtained information about the management, service and department structures and processes and then using this as a means of developing a model. This technique has been used extensively for example as a precursor for establishing user requirements and specifications for Managed Learning Environments, but also in the development of specific software systems, Web sites and portals for specific purposes. See also the Process Review infoKit.
- Project groups
Project working groups, meetings and committees can all be a valuable source of information. Analysis can include interpretation of the discussions held during the meeting, as well as analysis of documented meeting minutes. You may also want to consider using these as pseudo-focus groups for feedback on particular aspects of the project. This is a useful way of addressing the issue of the difficulty of getting different stakeholders together.
- Document analysis
Analysis of documentation more generally can provide valuable insights into existing practices and structures. A variety of sources can be used such as minutes of other meetings and committees of relevance as discussed above, but also project plans and reports, departmental documents, strategic and operational plans, policy documents, handbooks and guidelines. In addition organisational charts and functional charts and role specifications are valuable data sources. These can provide a snapshot of current activities and also a comparison of how structures and functions have changed over time which can provide insights into the impact technology has had and the ways they are being embedding in learning and teaching and also the impact on organisational structures, roles and functions.
- Interpreting and using your findings
There are a variety of ways in which the data you collect can be interpreted and used. It can provide a mechanism for undertaking an audit of current activity, for example by providing a snap shot of activities of relevance to your MLE development across an institution or an indication of the degree of uptake of a VLE across different subject domains. It can also be used as a mapping of existing MLE-related activities against structures and functions or as a means of monitoring change and benchmarking over time.
This section is not intended to provide a detailed outline of different analytical techniques but rather picks out common examples and approaches.
In terms of interpretation of the transcripts gathered from focus groups and interviews then a useful starting point is to go through the material and simply draw out the key themes and issues. You might like to begin by highlighting keywords and then move to trying to group related issues together. If you want a richer interpretation of the data there are a variety of research methodologies you can use, commonly used techniques include pre-coded classifications (ie classifications that others have already developed) or a grounded-theory approach (where themes emerge from the data). In addition, to these qualitative approaches a range of quantitative methods can be applied to material gathered through closed questionnaires or via systems usage and Weblog statistics, using a host of standard statistic tests and output mechanisms. There are a range of software tools which can be used to collate, manage and analyse such results, the most widely used being SPSS, but you may find that using a spreadsheet is adequate for your purposes. Again bear in mind the time investment against the benefits with learning new methods and familiarising yourself with different software. Similarly for analysis of qualitative data there are a range of software tools for managing and analysis such as NVIVO which helps in terms of coding, classification and analysis.
If possible it is a good idea to collect your information from a variety of sources and use a multi-modal data collection and analysis process as this is likely to provide you with a richer picture of what is happening. You may also want to return to the information you have gathered at a number of points during your project, reusing it for a variety of different purposes. Overall remember to be clear about what you want to find out, how you are going to do this and how it will be used.
Developing a fact-finding plan
In an ideal world you would have the time and resources to pick a rich mixture of the data collection methods outlined above. In reality your MLE project will be limited both in terms of budget and staffing and will be bound by time constraints. In addition it will need to fit in with other institutional activities and the external context as outlined in earlier sub-sections. Below are a series of questions you may wish to consider to help you devise a practical and realistic mechanism for finding out relevant information.
What do you want to know and how will it be used?
What sources are you going to use?
What methods are you going to use?
What resources do you have available to support this data collection?
Who is going to coordinate the data collection?
How is it going to be collected?
Are there any ethical or political issues?
If you are collecting data from more than one source how will these be combined?
Which aspects of the overall MLE development plan will this information feed into?
How else might the information be re-used elsewhere in the organisation?
Follow this link for key resources for this section (these open in a new window)
Follow this link for more resources for this section (these open in a new window)
