The Role of Virtual Learning Environments in the Online Delivery of Staff Development
Report 1: Review of Experiences of Delivering TALiSMAN Online Courses
0 O V E R V I E W
For two years, the SHEFC funded TALiSMAN project has provided a programme of training and awareness to enable staff at Scottish Higher Education Institutions to benefit from the Metropolitan Area Networks which link them. A significant part of this programme (>600 training places) has been delivered online.
Here we describe two alternative approaches to delivering online staff development which we have followed. The report will discuss our experiences of these two approaches and highlight issues inherent in the design and delivery of staff development programmes online. We will recommend ways of maximising the learning experience whilst retaining a flexible and scalable delivery structure.
A second report of this project will extend these findings and discuss the suitability and potential role of commercial and freely available Virtual Learning Environments in the delivery of online staff development programmes.
1.2 The Scottish MANs
and UMI Projects
To complement this investment in infrastructure, SHEFC funded a number of projects under the Use of MANs Initiative (UMI, ). These projects were intended to encourage the use of the newly created MANs. The projects funded under UMI phase one fell into three strands:
The TALiSMAN (Teaching and Learning in Scottish Metropolitan Area Networks ) project was set up in June 1996 under the staff development strand of UMI to encourage the use of networks by staff at the 21 Higher Education Institutions (HEIs) connected to the Scottish MANs. A programme of events providing a combination of training and awareness was developed and delivered to the 21 institutions throughout Scotland.
Whilst most of the training was to be delivered face to face it was anticipated that some of this training would be delivered online to:
As part of its' first year activities, TALiSMAN carried out a Training Needs Analysis (TNA) surveying the training requirements of academic and research staff at all 21 HE Institutions in Scotland . The TNA confirmed a willingness to accept this type of training: "Although staff expressed a strong preference for traditional methods of training and support, most were willing to receive training by a variety of non-traditional methods including network-supported open and flexible learning... ...TALiSMAN should offer a mix of traditional courses and training via on-demand distant methods such as network-delivered open and flexible learning "
1.3 Other Relevant
In early 1998, SHEFC began a comprehensive review of their C and IT provision by carrying out a consultation exercise with representatives from the 21 Scottish HEI's and other relevant bodies. The results of this consultation were reported in a circular letter in April 1998 . On staff development, the authors were clear about the potential role of C and IT: "The most widely recognised need, for the further development of the use of C and IT in higher education and especially in Teaching, Learning and Assessment (TL and A), is an extended range of staff development activities. These range from the provision of training in basic IT skills for all staff to the rethinking of the processes of TL and A in the light of the enhanced facilities that C and IT can provide. Many responses emphasised the need to embed the C and IT development in the wider issue of the reformulation of the processes of teaching and learning, moving from conventional delivery mechanisms to more independent, resource-based learning, facilitated and supported by academic staff. The programme should be pedagogy-led rather than being technology-led." The report goes on to highlight the role of new technologies in facilitating this staff development: "The important contributions of CTI, LTDI and TALiSMAN to increasing the awareness and abilities of academic staff in the Scottish HE sector were recognised. In the next stage of development, the need was expressed to exploit C and IT itself (through means such as videoconferencing, and the production and use of IT-based staff development materials) to bring the development activities closer to the individuals who need them, and to allow a wide participation from experts across the sector."
The report of the National Committee of Inquiry into Higher Education (the Dearing Report ) was concerned with all aspects of Higher Education, but some of its findings relate to the use of C and IT in Teaching and Learning. Discussing students and learning, the report recognises the central role of C and IT along with the attendant need for appropriate staff development (Recommendation 9). Furthermore it is thought important to formalise such training within a proposed "Institute for Learning and Teaching in Higher Education" (Recommendation 14). On staff development, the report stresses that whilst C&IT has much to offer, it is important to develop its potential carefully - with appropriate training, recognising the role already played by TALiSMAN: "Training and support in the use of C and IT is an issue in its own right. In such training we imagine that institutions will wish to draw on materials already developed in the sector such as the Netskills project at the University of Newcastle and the TALiSMAN activities being carried out by Heriot-Watt University on behalf of the Scottish Institutions" (para. 14.19).
The recent "Evaluation of the Computers in Teaching Initiative and Teaching and Learning Technology Support Network" report  scrutinises the current role of these initiatives in promoting the use of C and IT within the UK Higher Education community. This report recognised the importance of C and IT in enabling the development of new learning strategies, and satisfying the changing expectations of students and employers (para. 5).
Each of these studies recognises the future importance of C&IT to staff development and specifically the role of networks in making staff development flexible.
1.4 TALiSMAN Online
A second report from this project will examine a range of other Virtual Learning Environments (VLE's) and evaluate their potential and suitability for the delivery of staff development online within the UK HE community, with special attention paid to the issues identified here.
2 O U R A P P R O A C H
2.1 The TALiSMAN
In designing the online course materials we examined two existing courses: 'Using the Internet for Teaching and Learning', developed by Jackie Galbraith at MEDC (University of Paisley) for Scottish FE, and the short 'Finding Learning Opportunities on the Web' (FLOW) course run by the Open University . We followed the structure of the MEDC course (6 weeks, staged delivery) and drew on the overall aims of the FLOW course, however we used our own content (revised from a face to face course) and delivery mechanism.
The initial implementation of the course (http://www.talisman.hw.ac.uk/online/100397/ - see Appendix B for an outline of the course) consisted of six 'lessons' made available at weekly intervals (usually on a Tuesday morning). Each lesson was announced by a brief email which summarised the week's activities. Usually, a reminder was sent out on the Friday urging those who had not yet participated to complete the coursework. Completion of the course required a commitment of 2-3hrs per week self-study, with activities mainly centred on the discussion forum. The courses were not accredited.
After requests from some participants, an extra break was added halfway through the course to provide a break for Easter. The course was advertised at TALiSMAN events, by mailshot to our Institutional contacts and from the TALiSMAN WWW site. 52 participants signed up and all registration and administration was carried out through email. Prior to commencement of the course, participants were given access to the course environment to enable them to familiarise themselves with its structure and interface. The course material was predominantly simple web pages, including links to example sites, and a course discussion area utilising the discussion software Dialogue . After the course, feedback was collected by means of an electronic feedback form. This form is included as Appendix C.
A further 62 participants signed up for the second run of the course which started in May 1997 (http://www.talisman.hw.ac.uk/online/120597/). For this run of the course, very little content was altered, however significant changes were made to the environment and administration of the course. As before, we felt unable to 'force' participants to actively take part in the course, but we did ask them to give a commitment to the course by completing a 'declaration of intent' before starting. We also elicited participant information through a course 'expectation' questionnaire as well as detailed feedback questionnaires (all collected electronically) filled in during and after the course.
During the summer, a more significant revision of the course was undertaken. Changes were introduced to improve the structure, reflecting feedback from the initial two courses. The most significant change involved introducing an academic subject focus to the course material.
2.2 Academic Subject
Table 1: Academic Subject-based Courses
These subject based courses each attracted approximately 20 to 25 participants. The Statistics and Art and Design courses, were advertised throughout UK HE in order to enrol enough participants from these rather small communities.
For the subject courses, help was sought from subject experts to deliver part of the course. Week 2 (Finding Resources) was replaced by a lesson delivered by an external tutor with a specific subject focus. The external tutors were from Information Gateway Projects (EEVL, SOSIG, OMNI, and Biz/Ed ) or Computers in Teaching Initiative (CTI) Centres (Statistics, Land Use and Environmental Sciences and Art and Design ) and they drew largely on material from their own site to provide alternative content for that week. In some cases we also provided alternative material for week 4 (Developing Web-based Resources). A significant advantage of using external tutors is that it gave courses 'authority' and a more diverse input.
The structure of the course environment was also changed from being essentially one web page per week to being a series of short pages; mostly viewable as single screens and with a simple navigation structure maintained across the whole course. This change reduced the amount of text in any one page and emphasised the common structure running through the course. An example screen is shown below [Figure 1]. This shows learning material and navigation links for week 5 of one of the academic subject-focus courses.
As different courses were being run in parallel, we decided to move to a more robust discussion system and chose WebBoard . Whilst being a more sophisticated package than Dialogue, WebBoard was found to be somewhat daunting by those who had never used computer based conferencing systems before. An example screen from WebBoard is shown below [Figure 2]. For the standard view of WebBoard, the message thread is displayed on the left and individual messages on the right, buttons for submissions to the discussion are placed along the top of the browser window, below the standard toolbar and address panel.
Figure 2: Example screen from WebBoard
The course environment which we created, whilst simple, was robust and flexible enough to act as an effective delivery vehicle for the TALiSMAN Online Course. If the course was to be delivered on a larger scale, then it would help to have administrative tools that could be used to manage the course efficiently. Furthermore, if the course was accredited, then delivery and administration of assessment would require new tools.
Online Study Centre
The TALiSMAN Online Study Centre was created as a way of keeping our original courses accessible. We envisioned a single interface allowing access to adapted versions of our face-to-face training courses. Given its intended use, the requirements of the new environment were different to those for the original online course. This environment should provide tools for communication, student tracking and course administration. It must allow the management of a large number of students, and be flexible enough to incorporate different styles of course material. Finally, creation of online materials must be simple and quick. Rather than follow the 'home made' approach of our initial online course, we decided to use a commercial Virtual Learning Environment (VLE) package for the management and delivery of the courses within the online study centre.
One of several packages we considered was WebCT (Web Course Tools) developed at the University of British Columbia . We chose WebCT because of the flexibility and customisability it offered. Learning materials can be created easily by experienced web developers and novices. Our second report will discuss WebCT and other VLEs in greater detail.
For the TALiSMAN Online Study Centre (OSC), we required a single environment, offering access to a range of courses, with shared communication facilities, a calendar, web based assessments and other features. Whereas some packages allowed no customisation, WebCT allowed us to include or discard individual tools and features as required. For instance we felt it inappropriate to force participants to use the WebCT email system for one-to-one communication, but felt the internal discussion forum was useful for one-to-many interactions. WebCT allowed us to separate these two modes of communication, unlike other systems, where one-to-one and one-to-many conferencing modes are integrated and cannot be offered separately.
The homepage of the Online Study Centre is shown below [Figure 3]. Generic materials reside in the left section whilst the right hand section holds the courses available, along with tools for the learner: student records, password administration, the course discussion area, an event calendar, student homepages and a keyword search.
As with the online course, the courses offered within the Study Centre were not accredited. Furthermore, no formal tutoring was provided (although there is tutor support). Users register to access a course or courses and are given a username and password. This initially provided users with access to individual courses and to the relevant course discussion forum. This was changed slightly so that registration provided access to all courses. Participants have to motivate themselves to complete the course - there is no tutor initiating discussions on specific topics at specific times. In this way, the materials are entirely self-paced and resemble open learning.
The Online Study Centre was expected to perform a dual role - that of providing access to our materials, and also to serve as a test bed for us to investigate how different course structures and different content could be fitted into a standard template. Of particular concern to us was whether the constraints imposed by WebCT on learning materials in any way compromised their quality or usability.
The OSC initially hosted two halves of a course 'New Opportunities in Teaching and Learning' which we had delivered on site at 19 of the 21 Institutions we serve. These courses ('Using Communications in Teaching and Learning' and 'Using the WWW in Teaching and Learning') represented approximately three hours face to face teaching each. The materials for the Communications course were already relatively interactive whilst those for the WWW course were more text based, having originally been designed as a stand alone booklet, to complement the face to face taught course. In fact the 'Using the WWW in Teaching and Learning' materials had already been utilised to create the original TALiSMAN Online Course. In addition to these two courses, the TALiSMAN Online Study Centre also contained a short 'Introduction to email' course which had been developed but never delivered and a web authoring skills course which represented approximately 2 days face-to-face teaching time. An example screen showing learning material is shown in Figure 4. The screen is divided into three frames, along the top are buttons giving access to general course tools such as the glossary and search facilities, along with navigation buttons. The main left hand frame provides an outline for the course and highlights the current page. The main right hand pane contains the learning material itself.
Figure 4: Learning Material in the Online Study Centre
In many ways, the TALiSMAN Online Study Centre was developed as a complementary approach to that used by the Online Course. The course materials were made available as open learning, in keeping with the initial concept of the Centre. Rather than groups of participants being given the learning material as a cohort, access to the Centre and its courses was provided - 'on demand' and the pace of progression through the course was entirely under the control of the participant. Tutor support was given as needed, but there was no formal course structure, or delivery schedule. The discussion area was used not to discuss specific topics (at specific times), but to provide a forum for users of the Centre to raise their own issues. The growing archive of discussions provides an additional resource for participants; here they will find advice on using the Centre, interesting issues, further materials and so on. We expected users to progress through different courses in the Centre, gaining a range of skills through the different materials.
The screenshot below shows the course discussion area [Figure 5]. The left hand frame allows the user to decide how messages are displayed. The right hand frames show message titles (top panel) and individual messages (bottom panel).
Our experience suggests that some materials were better suited to WebCT than others. Initially, materials were put into WebCT 'raw' and subsequently reworked. Some materials, such as the Simple HTML Authoring course worked well in this format. Course materials with a more rigid and uniform structure, such as the support texts for the 'Using the WWW in Teaching and Learning' Introductory Course worked less well. This is unsurprising, as predominantly text-based materials are known to work less well on screen. Most recently, we have created materials directly for use within WebCT. This has allowed us to create a style that meets both the constraints of WebCT and the needs of the materials themselves and these courses have proved popular with users of the Online Study Centre.
3 Course Review
The original online course has now been delivered on 5 different occasions to a total of 284 participants. A slightly greater number of people have signed up for the TALiSMAN Online Study Centre (333 in October 1998). Formal feedback on the TALISMAN Online Course has been collected through electronic feedback forms. For the TALiSMAN Online Study Centre, only informal feedback has been collected - no formal evaluation has yet been carried out by TALiSMAN.
Online Course Feedback
Feedback from individual courses was used to revise subsequent courses. It was partly responsible for the introduction of an academic subject focus, input from external tutors and a change in use of the discussion forum. Feedback also led us to change the way we advertised the course and to clarify its aims.
Feedback was mainly positive, with most respondents indicating that the course met their expectations, that the level of tutor support was appropriate, and that they would be happy to recommend the course to others. The table below summarises the responses to these questions [Table 2].
Table 2: Quantitative Feedback
Respondents were asked directly about a number of features of the course, the most and least useful aspects of the course, whether they would make any changes to the course, and any other general comments. The responses collected highlighted a number of issues as discussed below.
Expectations for the Online Course
Some participants expected practical training in the creation of web pages. Whilst we felt that this would have constituted a logical part of the course, it was thought impractical (in such a short course) to attempt to pass on a skill which people learn in very different ways. As other TALiSMAN courses covered web authoring it was felt acceptable to omit it.
Finally, a number of participants expected to be led to teaching materials tailored for their courses. As a generic course, we felt it more appropriate to pass on transferable skills for finding and evaluating learning materials and resources, leaving the academic to use their personal expertise in their own individual field to find specific resources. The subject specific courses delivered in the second year of the project were designed to address this issue. Course participants were directed to CTI centres and Information Gateways, many of which provide libraries of links to materials they have identified as useful and appropriate. Unfortunately, such resources are not available in all subject areas.
Many participants found that it was impossible to set aside 2-3 hours without being interrupted (e.g. by students or phone calls). Others reported that they had worked through the materials in their own time (staying late after work) to avoid interruption.
In planning the course, we were acutely aware that devoting sufficient time to the course by the participants would be a problem. We felt it appropriate to structure the delivery of the course (one lesson a week) rather than just to present the materials as self study. One advantage of this approach is that the regular delivery of chunks of materials can act as a guide through the course content. However, as our courses were not accredited, lecturers were never released from other tasks and there was no carrot which we could offer (or stick we could wield) to encourage or force participants to complete the course on time. Inevitably, a busy lecturer faced with spending 3 hours on the online course, or clearing a backlog of marking will choose the latter and our participants often emailed apologies for non-participation with reasons such as 'I've just been given extra teaching...' or 'It's exam time and I've got lots of marking ' A number of our participants asked whether the course materials would be available after the end of the course, or suggested that the courses themselves could, in future, be run over a longer period.
The main casualty of this inevitable constraint on participation time was the discussion forum. Although we tried to stress the benefits of discussion, we felt it was never used to its full potential, with a generally low level of use in all courses (though discussions in the subject specific courses were slightly more active). Some respondents felt that participants were shy and would have benefited from a face to face session at the start of the course (this was tried for some of the courses but seemed to have little effect on the usage of the discussion forum). A few users remarked that WebBoard was confusing as an interface, but this in itself did not seem to have an effect on the level of use. Realistically, six weeks is too short a time to expect students to become familiar and comfortable with any interface. The choice of discussion forum can significantly affect the way in which online discussion occurs. Although we felt that the Dialogue discussion forum was appropriate for the level of course and number of users, we were concerned that its rudimentary structuring was of little use for reading discussions retrospectively. As we knew that participants might only look in on the discussion area once a week, we felt that such structuring was an important feature. Before the subject specific courses were delivered, we decided to change to a more robust system. We chose to use WebBoard, which provided a simple and familiar interface. WebBoard was easy to integrate with the existing web pages and administration was simple. One criticism of WebBoard was that by being more rigidly structured and clearly presented, discussion may have been stifled, because participants felt that their submissions would come under greater scrutiny.
An alternative possibility might have been to use email as the main discussion medium. Email is familiar to virtually everyone in the community, so no familiarisation would have been required. As most participants check their email every day, they would be able to keep in touch easily with the discussion as it progressed. In practice, this model is unsatisfactory. Most participants tried to set aside time to work through the course in a single session per week, rather than several short sessions. Whilst single email messages might be passed over, and left unread, a single visit to the WebBoard would allow the participant to catch up on many discussions at once. Also, not every participant would be interested in every discussion thread and may come to resent the arrival of emails they considered to be irrelevant. Although we did use email to announce and pace each week of the course, but did not use it to deliver 'content'. If the level of discussion in the forum had been higher, it might have been appropriate to send an email digest of discussions to each participant, once or twice a week.
Online Study Centre Feedback
As the discussion forum is not used for tutor led discussions, the level of use tends to be low. Participants in the study centre realise that they are engaged in self-paced learning and tend to work independently. Although there are many messages in the system, users are given the option to display only new messages. There tend to be periods of activity as a topic is discussed, then a lull before the next point is raised. The main use of the bulletin board tends to be for technical queries, but it is also used to swap web addresses, allow users to air their own interests and to ask for general advice. As these are answered, they become part of the growing resource of the Study Centre. New users come across answers to commonly asked questions in the bulletin board. This seems to work quite well; if it weren't satisfactory, it would be an easy task to abstract the material to a FAQ (Frequently Asked Questions) page. The Online Study Centre has been running for almost one year now and will soon be internally evaluated as part of TALiSMAN's ongoing analysis of its' courses. The results of this evaluation will be made available when complete.
From the questionnaire responses, many positive aspects of the Centre are highlighted. This user worked the whole way through courses, re-used the materials, and contributed to the discussion forums and resource database. This user felt the level of tutor support appropriate, felt that materials were of an appropriate level and was happy to (indeed already had) recommended the Centre to others. The user was critical of some of the course structuring and provided suggestions for more interactivity which was felt to be lacking.
The simple structure adopted for the TALiSMAN online course attracted no criticism. The Online Study Centre environment created within WebCT has been criticised by many users who dislike the navigation structure and the way some of the tools function. With the extra functionality being offered, the structure of the Online Study Centre materials is necessarily more complex and perhaps requires some familiarisation or induction period.
As a further part of the external evaluation, a 'typical' user was commissioned to work through all courses in the Online Study Centre and produce a case report. This user had already participated in one of the staged delivery TALiSMAN online courses (and hinted that devoting sufficient time had been difficult). The commissioned evaluator was also generally positive about the Online Study Centre (would recommend it to others) but outlined a number of criticisms. Navigation round the courses was felt to be poor - with 'Home' buttons on different pages having different destinations - confusing to the first time user. This failing is inherent in the structure of WebCT, but we had felt that as the materials were self-paced, familiarisation with the course structure could be gradual, rather than requiring a formal induction. A similar criticism was levelled against the homepages - although potentially useful, few participants create personal homepages and this is probably because the tools provided for this purpose by WebCT are not particularly good. Making a homepage for users of the study centre was entirely voluntary. Had we wanted to make this compulsory, we would have investigated and provided more appropriate tools. The final negative comment regarded a technical issue. The 'Web Images' course required software which was not available for the platform used by this evaluator, and there was some degree of frustration at the amount of time wasted before this was realised. This highlights the need for some formal technical support. Although we do give some guidance about the recommended specification of machines used for viewing the Online Study Centre and do our best to ensure that the learning materials can be fully utilised by as many participants as possible, there can be unanticipated difficulties. Again, with experience, this could be rectified.
In each of the cases cited above, interactivity (or at least activity) is cited as a key issue. Finding a balance between increasing interactive components and maintaining a focus for the course is vital. It is sobering to note how intolerant our users were of materials not specifically designed for online delivery.
4 F I N D I N G S A N D L E S S O N S L E A R N E D
Over the last two years TALiSMAN has delivered a range of learning materials (task based, self-paced, discussion centred etc.) over the WWW using a variety of delivery and administration strategies. Some of the approaches we tried have worked well, whilst others were less successful. This section shall examine some of our findings, the lessons we have learned and highlight some of the issues inherent in the use of the WWW and Virtual Learning Environments for staff development.
The special circumstances of our courses should be reiterated. Had our courses been accredited, we would have been able to explore many strategies (such as collaborative working and assessment) more fully. Also, had these courses been run over longer periods, we would have been able to investigate optimum rates of delivery (some course participants asked for longer courses, others for more time between lessons) and course structure.
In our experience, a single tutor can support a large number of students, but the tutoring itself will require an almost full-time commitment; answering individual queries, checking regularly on discussion forums etc. Rather than all activity on a course being concentrated into a single face to face meeting each week, online courses effectively go on all day every day. The tutor will need to check the discussion areas at least twice a day and reply to email queries within a day of receipt, as well as prepare lessons and assessments, carry out administration and send out 2 or 3 messages a week to pace the learning process. For the online course, I spent about 2 days per week supporting the course, though had I been able to spare the time, I could easily have spent more time tutoring.
The way in which the discussion forum is used can also be important. If the forum is relatively unstructured, it is the tutors' responsibility to ensure that all questions raised are answered promptly. If a question is raised and not immediately responded to by other students then it becomes in danger of being lost in the discussion forum as new questions are raised. However a single tutor immediately answering all points raised in the discussion forum can stifle discussion initiated by the students. The use of a team of tutors would be helpful as students will appreciate the variety of styles and will perceive that experts are answering their questions. Tutors can adopt alternative viewpoints to encourage students to think, rather than unreservedly accepting the view of a single tutor.
The assessment facilities offered by WebCT are rudimentary, but as they are already built in, they are easily utilised. There are a number of standalone assessment tools available (see section 5) but implementing them can be time consuming and technically demanding, and this can deter course developers from integrating assessment into web based courses.
Format of the Course
A further issue regarding WebCT and other VLEs is their unsuitability for offline working. If the participant pays for their Internet access, then they are likely to want to minimise the time they spend online. Software such as WebWhacker allows groups of web pages to be collected for offline viewing. The learner need only log on to initially collect the pages and subsequently to take part in any online (synchronous or asynchronous) activities. This strategy does not work for WebCT because the web pages are stored in a database and cannot be collected by offline browsing software such as WebWhacker.
We should recognise that efficient delivery of accredited staff development is likely to have a significant modular component and it would make sense to consider the scalability of any system. Within reason, (again, there is no sense in compromising the materials just to make them fit a specific template) the learning materials should have a common structure (objectives learning, materials, tasks, discussion, assessment, reinforcement etc). Students will learn more efficiently if they are familiar with the structure - an important consideration if we remember that online students require more guidance. Within that structure, there should be clear sign-posting of a route through the learning material. This guidance can serve to pace the learning process. The use of simple reinforcement exercises can also reassure the student that they have satisfactorily completed that section of the material.
The online course delivered by TALiSMAN consisted of self-made web pages (conforming to a simple standard template) supported by a discussion forum (which could be changed). Administration and course announcements used email. Student records were kept in Microsoft Excel, which was also used to collect and collate feedback form data. Adding new materials would have been relatively easy, but adding large numbers of new students would have strained the administration structure. Any formal accreditation would have required more extensive administration (student tracking, collection of assessments etc.), and this would certainly have overstretched the existing structures. The WebCT environment used for the Study Centre has many administrative tools built in and once participants have been registered with the Centre, the WebCT system itself holds student records, showing access logs, student progress and the scores of any assessment carried out. One of the main advantages offered by networks for the delivery of staff development is the scalability. For efficiency, large scale courses must be modular, easy to manage and easy to administer.
Standards for these factors are being developed by the Instructional Management Systems (IMS) Project . The IMS project is an international initiative whose task is to recommend and create a standard architecture for online learning. The framework will be flexible and allow interoperability without constraining developers. Materials which comply with the IMS standard criteria will be branded as IMS-enabled. IMS standards will cover not just the learning material, but also methods of administration, student tracking, communication, assessment, structure and interface.
5 S Y N T H E S I S A N D C O N C L U S I O N S
The main purpose of this report has been to review our experiences of delivering staff development over networks. Can we now make some general conclusions regarding the design and delivery of online learning materials? This section of the report will reconsider some of the key issues and provide some pointers to the role of networks in the delivery of staff development within UK Higher Education. The second report of this project will review the suitability of commercial VLEs for this purpose.
5.1 Considerations for Online Delivery of Staff Development
It can be advantageous to have some face to face sessions in a course. A face to face induction session can activate a learning community (and can provide an opportunity for remedial training and necessary familiarisation with the online learning environment). Occasional face to face sessions can also structure open learning materials, if there are progression deadlines for each face to face event. Learner homepages, where individuals can tell the rest of their group about themselves (and if appropriate) make available some of their own work (a portfolio) are also of use in maintaining and building this community.
Frequently Asked Questions (FAQ)
For accredited staff development in HE, most learners will be graduates and will have considerable study skills already, but these may be ill-suited to online learning. Learning material which works well 'on screen' is likely to be radically different to traditional paper-based materials and require new study skills. Basic IT skills are an essential prerequisite for any course which utilises computers and online courses obviously involve intensive use. Learners should be comfortable with navigation through learning materials, working with multiple windows and electronic note-taking. Where contribution to discussion forums is expected as part of the course, some familiarisation is essential to ensure that participants do not come to see the technology as a barrier to their participation.
Even amongst those familiar with computers, there will be some who are disinclined to take part in course discussions and co-operative work depending on their learning style. Learners who may be vocal in face to face tutorials may be inhibited by the permanence of a discussion forum submission. In contrast, learners who would normally be quiet in class might welcome the chance to reflect on their ideas and prepare a formal submission. For courses conducted entirely online, it may be possible to organise synchronous online discussions (in addition to asynchronous activities). Synchronous sessions can be used to 'brainstorm' at the start of asynchronous discussions. Also, their informality makes them attractive to those who would normally feel stifled by the rigid structure of asynchronous forums.
In order to build strong communities, it is important that learners are streamed according to their initial knowledge. Within large courses, students can be allocated to groups of similar ability. Any progressive course structure would also alleviate this problem, as learners would be expected to have fulfilled certain prerequisites for a course.
We found that online course participants harbour very different expectations to face to face students. Managing these expectations is critical - whether by encouraging student-student communication or by anticipating the students desires and providing an extra resource for them to refer to; exaggerated expectations only become a problem if they are not addressed. Allied to this, it is important that course objectives are clearly stated, that navigation through the learning material is intuitive and also that activities (such as simple reinforcement exercises) demonstrate when/if objectives have been met.
Online students also require more explicit guidance on progress through the course - signposts which indicate the expected route through materials, and also the expected progression rate. These signposts may be concrete (as with staged delivery), formal (as with specific assessment deadlines) and informal (as with prompts from the tutor).
Assessing progress and performance in online courses requires new strategies which accurately gauge the success of the learning process. Assessment of activities is likely to be just as important as assessment of essays and projects.
At Glasgow Caledonian University, a course in learning and teaching for new lecturers is now delivered online . This course uses web pages, three face to face tutorials, assignments and a discussion forum. The course has been delivered twice in 1997-1998 and changes planned for further delivery. As with our online course, the course leaders were disappointed with participation rates, and the changes they have recommended for the next delivery concur with the findings of this report: prepare participants (clarify expectations and familiarise them with components), make materials more interactive and introduce more activities, and encourage more use of discussion forums.
The new opportunities offered by networks for the delivery of staff development look capable of providing the flexibility required by workplace learning and professional development. Exploiting the World Wide Web for the delivery of these courses requires careful design of materials and courses, together with imaginative use of technology to build communities. Finally, managing the learning experience effectively is vital. Within reasonable constraints, Virtual Learning Environments can support and enhance effective learning, but although they provide new opportunities, their implementation and use must be carefully planned and adequately supported. This report has examined one (WebCT) and compared it to the facilities offered by traditionally created and administered web pages. The second report from this project will examine different VLEs and evaluate their suitability to the delivery of online staff development, given the issues we have discussed in this report.
6 R E F E R E N C E S
7 A P P E N D I C E SAppendix A: Example Course Flyer
Online Course: Using the WWW for Teaching & Learning
Launch Date :10 March 1997. 2nd cohort runs from 12 May 1997
Who should attend:
Last updated by Colin Milligan 12/3/97 - Produced by TALiSMAN
Appendix B: Course Structure
Week One - 16.5.97
Appendix C: Course Feedback Forms
If you are not using a forms-capable browser, please email your response to this form to: firstname.lastname@example.org
Having taken part in the TALiSMAN Online Course, 'Using the WWW in Teaching and Learning', we would now like to ask for your comments on the course. Can you please answer the questions below. Some of the questions require answers. For others, response is optional.
1. Name: _________________________
4. Would you object to your comments being used in future by TALiSMAN?
6. Had you used the WWW before participating in this course?
7. Had you previously used discussion forums before participating in
8. What were your expectations of the course?
9. Did the course meet your expectations?
10. Would you recommend the course to others?
11. Would you suggest changes to the scope or level of the course?
If Yes, what would you change? (optional)
12. On average, how much time do you think you spent on the course each
13. What do you think was the best aspect of the course? (optional)
14. What do you think was the least useful aspect of the course? (optional)
15. Did you find the tutor support provided adequate?
16. Did you find the course useful in relation to your work?
17. Do you have any immediate plans to use the WWW in your Teaching and Learning? (optional)
18. Do you have any general comments? (optional)
19. Did you find 'dialogue' easy to use?
20. Did you find 'dialogue' robust enough to cope with the level of
21. TALiSMAN is about to run this course again with modifications to content and delivery. We are keen to improve the course, can you suggest any improvements for this or future TALiSMAN OnLine courses? (optional)
Thank you for filling in the form, if you have any further questions regarding the course, this feedback, or any of TALiSMAN's activities, please do not hesitate to get in touch.
This document was added to the Education-line database on 16 November 1999