Purpose of Case-study

The results of the audit carried out by the Big Blue project team suggest that, prior to students undertaking an Information Skills programme very little assessment of their existing skills/knowledge is carried out. This case study aims to focus on the pre and post self-assessment embedded in the Library Skills unit, which forms part of Information skills (SK1003). This is a compulsory Induction module for all First Year MA, BD, BTh, BLE, MRM and CEM Students (Arts Divinity and Social Science students). The course can be viewed at http://www.abdn.ac.uk/infoskills/ and the specific section relating to Library Information skills at http://www.abdn.ac.uk/infoskills/libskills/part3.hti. The case study was carried out by :

Background

The Directorate of Information Services and Systems (DISS), a converged service incorporating the Library, Computing and Management Information services, has been active in the delivery of on-line Information skills, both IT and Library, as part of a 1st year undergraduate induction module since the mid 1990's. These units sat alongside, 'Learning strategies' to form a compulsory module 'Academic skills.' The status of this module has grown over the years and is now regarded by the University as an essential cornerstone for students entering Higher education. It is also seen as an essential building block in assisting students with their development of a range of transferable skills. This course was reviewed during 2000/2001, and in the light of consultation and feedback some key changes were made. With regard to the Library Information skills unit, student feedback recommended that the existing generic content should be supplemented by relevant subject specific information sources. These recommendations were adopted, and from the academic session 2001 the revised and renamed 'Information skills' module was re-launched. It comprises 3 sections:

Prior to embarking on the module all students undertake a self-assessment questionnaire in both the Library and IT skills units. This is then followed at the end of each unit with a post evaluation questionnaire, which includes both a self-assessment section and a unit evaluation section.

Delivery

One of the impressive features of the Information Skills module is its large-scale delivery. Compulsory for over 1000 undergraduates, it is also used, in part, by other Faculties. This reflects a substantial level of commitment from the University, which, in practical terms, is translated into the provision of resources necessary for the support, delivery and administration of the module. The module is delivered at the beginning of the first term in a tightly timetabled fashion. Attendance at workshops is compulsory until they have achieved the set objectives. However, once the workshops have finished, students have until May to complete the self-paced assignments, exercises and quizzes. The module is delivered using a variety of media.

For the purposes of this case-study the focus will primarily be on the development of the content of the current Library Information skills unit and in particular the inclusion of pre and post self-assessment. Key to the development of pre and post assessment was the appreciation that these activities would assist with the measurement of the unit's success.

Unit description

The Library Information skills unit has been developed as a collaborative effort, with input from Library staff, the Learning Technology Unit and academic staff. The current unit was initially conceived as an internal project bid. It was submitted as part of a bidding process for resources to develop new teaching initiatives. The bid was required to illustrate, not only its indicative content, but also sound pedagogical principles with clearly identified learning outcomes and opportunities for student feedback and progression. The learning outcomes were tied in closely with QAA benchmarking statements. e.g. Students can:

The generic content was principally written by the Faculty Information Consultant for the Arts faculty. Working alongside her was a member of the Learning Technology Unit who developed the infrastructure and design of the package using WebCT. Part of the project proposal was the inclusion of a pilot, which formed the basis of an interim evaluation report. The principle aim of the pilot was to assess the success of the unit in meeting its projected learning outcomes. The pilot tracked the progress of 1st year Engineering and English undergraduates during February and March 2001. Online feedback was obtained from students in advance of, and on completion of the course. In addition, students were asked to assess their own knowledge in a pre and post unit questionnaire. Even though not all students completed both questionnaires, the results indicate that the number of students assessing themselves as a 'Beginner' dropped significantly on completion of the unit, with the majority shifting their self-assessment to 'Intermediate' or 'Advanced'. The degree of subjectivity which underpins this self-assessment is to some extent balanced by the results from Skills Assessment exercises. These exercises aim to check whether the student has met the learning outcomes for each section. The scores from the Skills Assessment exercise confirmed that the majority of students, having worked thorough the unit, had improved their skills. Most achieved scores of 80% and above. Very few failed to reach 50%.

Content of the pre and post self assessment questionnaires

The content of both questionnaires essentially cover the same suite of topics. Students are asked to rank their knowledge as 'Beginner', 'Intermediate' or 'Advanced' in relation to a list of 5 skills. Each of the skills is mapped to the learning objective for each section of the unit, e.g.

The results (as of 27/1/02) confirm the findings from the pilot. The majority of students assess their skills level as moving from ' Beginner' to 'Intermediate' on completion of the unit. A small percentage identify their level as either 'Beginner' or 'Advanced'.

In addition, comments received from students as part of the unit evaluation questionnaire corroborate the perception that their knowledge, familiarity and skill levels have improved. e.g. 'I know feel more confident about using the library system and information retrieval' 'I can now make efficient use of the library' '…increased my knowledge on how to find things'

Liaison with teaching departments

The inclusion of exercises using subject specific resources is seen as an opportunity for teaching staff to tie the unit more closely to their academic programme. However, the degree of integration of the unit into teaching programmes is variable. Some teaching staff have embraced the unit and regard it as a useful tool to embed awareness, and use of, subject specific information resources into their curriculum. One lecturer from the Department of Film studies, within the Department of English has collaborated closely with the Library to develop subject specific exercises which tie in with assignments and encourage knowledge and use of key websites. This example of collaboration has been regarded as successful by both parties and is seen as an effective method for encouraging students to exploit the Library's resources. The English department has taken that model a step further and developed an additional unit which explores use and knowledge of information resources within their 1st year English courses. The course can be accessed at: http://www.abdn.ac.uk:8900/public/englib/index.html To log on, use 'guest' as both ID and password.

Conclusion: Measuring the success

It is difficult to anticipate the longer-term impact the unit may have on the information- seeking behaviour of students. The pre and post self-assessment exercises indicate that the unit is successful in achieving its learning outcomes and anecdotal evidence suggests that many 1st year undergraduate students no longer require support in basic use and access to Library services and resources. However, Library staff will have to wait until the end of May (the completion date for the unit), before taking the opportunity to reflect and review the success of the unit and its impact on the skills of this year's intake.