Grounded Guidelines/Case Study: Educational Psychology
Evaluation
Use the back arrow in your browser to return to the main document
All the Group 1 respondents to the post-seminar task questionnaire said they liked the speed of exchange which email allowed, and most also mentioned the convenience of asynchronous exchange, which meant not having to wait till someone was in, and having time to think about replies. Comments during subsequent interviews reiterated these points.
It's quite useful because we don't meet together that often I suppose, so if it was something towards the end of the week we needed to know about for Monday or Tuesday, then sometimes we would send emails on Thursday or Friday to each other, to the whole group. [Trainee (F), focus group discussion] |
It's a very speedy way of finding information. [Trainee (F), focus group discussion] |
Well I used it because it was easier than the phone, and, it's the best thing where they're not really important burning things but you think, I can send this, I can ask, I need to know something about the service in a nursery or parent workshops or something, and I can just type it and I can send it out and lots of people will see it, and maybe come back to me with particular resources or things I should think about before I go and do anything. [Group 1 trainee (F), interview] |
The trainees found it a positive experience. They certainly commended it to us as something they wanted to continue with. [Course tutor, interview] |
The successful joint production of the seminar paper was also spontaneously mentioned in the questionnaire by almost all of Group 1 as a surprising (and positive) outcome from use of the system.
In contrast, Group 2 made no mention in their questionnaire responses of the benefits of asynchronicity (perhaps reflecting paucity of use of the system), and had a less positive reaction to the resource as a whole, not least because of the impact on usage of the initial problems with the setting up of the system.
If you're sending a message to everyone, it's not that reliable because you know not everyone's using it in the same way. I think that's a problem for the tutors and for ourselves. [Group 2 trainee (F), interview] |
These general evaluations were an accurate reflection of the problems that all had had with the operation of the system, but also of individuals' different ways of coping with these. Thus, throughout the initial five-month period of monitoring of usage of the conferencing system there was one category of message in addition to the others (see 'Character of the Communications' above) which had a consistently strong presence. This category was exchanges about network connections or software difficulties, either pointing out problems and seeking advice; or else announcing the solution of problems and the establishing of connections. During 11 out of the 17 weeks from the start-up of the system until the summer vacation this was the most frequent or joint most frequent category of message. In other words, then, all other activity during this time took place against a background of perceived connection problems, though individuals varied in their reaction to these.
I had a lot of problems initially because I didn't have it at my base and we couldn't set it up and we didn't know how to set up and we tried, and there were some negative attitudes towards it because of that. [Group 2 trainee (F), interview] |
Yo J,
[Group 1 trainee (F), email archive, main conference] |
Don't know what I've done differently this time but I can now access First Class will someone now please send me a message ! [Group 2 trainee (F), email archive, main conference] |
|
Hello all, [Group 2 trainee (F), email archive, main conference |
|
Hi there - nice try with the data but sadly my system can't read it, argh! Try sending by firstclass if you can and I should be able to read it (alternatively give me a hard copy on monday and I'll put it in monday night!) [Group 1 trainee (F), email archive, main conference] |
M, I got a phone call from AO and have managed to sort out my new identity!! and a password - I'm back on line! Thanks for your help [Group 1 trainee (F), email archive, main conference] |
Hi G, I think I've finally made it onto first class. I'm looking for the Mars Bar!! Let me know if you get this message. That means I am a fully paid up member, H [Group 2 trainee (F), email archive, main conference] |
Finally got connected - I hope! Desperate for a Mars Bar. Please send messages to prove the connection is not a figment of my imagination. Your deepest Ayrshire jungle correspondent, H. [Group 2 trainee (F), email archive, main conference] |
In line with this, all the Group 1 respondents to the post-seminar task questionnaire reported having experienced access problems due to network difficulties or lack of availability of machines, and noted that they considered dealing with these problems as key requirements for the success of such systems. Group 2 placed even greater emphasis than Group 1 on technical problems, and reportedly split into pairs to carry out the seminar task in order to get around them, hence the greater incidence of working together noted in the logs, and the lower use of email.
In part, these reactions were a genuine reflection of the start-up difficulties noted in the Background section above. However, the perception that there were problems using the system persisted beyond the point at which connections from placement offices had been reliably established.
I didn't have it at all initially, so I couldn't do it at all. But now they've sorted it out in my office but I couldn't get on at all to it and they had all the right package and they phoned up the people that do First Class or whatever and they still couldn't explain to us what we were doing wrong. [Trainee (F), focus group discussion] |
This was particularly true for Group 2, which contained the person who had the greatest difficulty establishing connection at the outset. Despite the fact that these problems were solved by the time the Group 2 seminar task was set, they still evinced a greater reluctance to make use of the conferencing system - and indeed apparently still had doubts about its reliability months later.
Dear L, [Group 2 trainee (F), email archive, main conference] |
Given the functions of email for Group 1, one factor which might have contributed to this longer-term effect (and thus to the broader differences between the groups) was that Group 2 reported more failures in sending and receiving attachments. These failures were to some extent due to mismatches between the resident word processing software (Word and ClarisWorks) on the group members' machines. They also appeared, however, to reflect some lack of understanding of how to circumvent these difficulties (e.g. by standardising software, or by using rtf files as the exchange format).
Dear group 2 [Group 2 trainee (F), email archive, main conference] |
Here is my part of the seminar paper. Hope it transfers to both apple and PC machines. H. [Group 2 trainee (F), email archive, main conference] |
some of us were working on IBM, some of us were working on Mac and therefore we weren't able to always read the attachments which other people sent. So invariably, I would say that we used the fax machine more. [Trainee (F), focus group discussion] |
...we couldn't send a p.c. attachment to an Apple Mac, 'cos all the Education Department use Apple Macs. [Trainee (F), focus group discussion] |
Male 1: Surely we swapped, did we not swap text? [Trainees (M/F), focus group discussion] |
There were also reports of continuing problems with access to machines, especially in the placement offices, where demand for resources meant that they were frequently tied up in ways that meant linking to the conferencing system was not possible. However, there were also some signs amongst these reports that, perhaps because of initial negative attitudes, some people were not trying that hard to make connections.
I've not read any of my messages for ages because our machine is used all the time for the database now, so I don't know if something really interesting happens. [Trainee (F), focus group discussion] |
I had a problem in that the phone line wasn't just my phone line that I was disconnecting in order to do the email line. It was also another psychologist's. So I couldn't actually use the machine unless she wasn't going to be using the telephone which was really awful to begin with. [Trainee (F), focus group discussion] |
Access is an issue because of the modem problem, although I mean it I think it's not that hard obviously to put one piece of wire into a slot, but because I'm so, you know, basic in my knowledge of how to do things if something goes wrong, I just give up. [Group 2 trainee (F), interview] |
Whatever their precise nature, the implication is that technical difficulties led Group 2 to adopt a structure for the seminar task which minimised the need for exchange, and that this was the primary reason for the differences in interaction patterns between Group 1 and Group 2, both at the time and subsequently. If so, it is not clear whether these technical difficulties were more perceived or actual. One possibility is that Group 2 had greater lack of confidence in the system because they had fewer early frequent users: three in Group 1, against one, or at most two in Group 2. As a result, Group 1 may have had a "critical mass" of users which was sufficient to persevere and overcome any lack of confidence in the system, whereas Group 2 did not. In a sense, though, whether this account is accurate matters less than the fact that, whatever the cause, due to technical problems Group 2 negotiated a working procedure which left the conferencing system without any clear function. As a consequence, in contrast to Group 1, they used it much less, and this became an established pattern.
As noted above (see 'Continuity with curricula'), the system was not seen as addressing in any direct fashion the primary educational objectives of the course. However, there was one more secondary objective that it was seen as fulfilling, namely the more generic concern of learning how to handle such technologies in preparation for professional employment. Thus responses to the post-seminar task questionnaire indicated that members of Group 1 all felt that learning about conferencing had been the most useful aspect of the seminar task, and that keeping in touch or discussing were the least useful aspects (cf. the relative lack of use of the conference for this, and the availability of face-to-face contact). Group 2 at that time perceived no real advantages to having the system, but subsequently there was a more general agreement that having had experience of conferencing might be useful for helping to establish the kind of information exchange resources available to other groups of professionals.
I think they're fascinating 'cos I'm a sad thing, but you read all these interesting comments from psychologists all round the country and they're all dealing with different issues, loads and loads of different issues and there's lots of really helpful advice for English psychologists in there and if First Class could be like that throughout the whole of Scotland, it would be wonderful. [Trainee (F), focus group discussion] |
Moreover, for some of the trainees there was the additional motivation of not only knowing how to use such systems, but also having the edge of being amongst the first in Service offices to have such knowledge, something that was unlikely to be a barrier to subsequent promotion.
I mean that's why it was set up, the Scottish Office pays the phone bills for it, to allow them, practising psychologists, to conference with each other, but the ones in my practice base didn't use it and it was only when I came along and we were allowed to get access to it, I showed them how to use it. They hadn't been using it. [Trainee (M), focus group discussion] |
However, the comparison to the existing activity of professional educational psychologists may have been a double-edged sword, something that was actually demotivating for some of the trainees, because they saw the potential redundancy of a skill that was not generally valued within the culture of which they were shortly to become established part.
...we tend to be very telephone based ... just familiarity ... and the people that we're looking for wouldn't be logging on regularly enough to ensure that they got that message ... you need more redundancy in your communications systems than that. You couldn't rely on people having picked up that email. Which is a fatal flaw with this. You may be taken and keen on it and use it a lot, but unless you're mixing with people, communicating and working with people who are equally familiar, and systematic in checking their emails, then it falls down ... You can check the history of a message and see if its been picked up, but that's more complicated in the end for a lot of psychologists than just picking up the phone. [Group 1 trainee (F), interview] |
Thus whether the system was seen as a salient and useful part of the course provision may have been influenced by calculations about the likelihood of the culture changing, and the possibility of contributing to that process of change. This too may have had a bearing on the extent to which individual trainees made use of the system.
