Teachers' and pupils' perceptions of the use of Integrated Learning Systems in English and mathematics education
Chrysanthi Gkolia and Alan Jervis
Faculty of Education, University of Manchester, e-mail: email@example.com ; firstname.lastname@example.org
Paper presented at the British Educational Research Association Annual Conference, Leeds University, 13-15 September 2001
This paper reports on a small-scale study of teachers' and pupils' attitudes towards the use of Integrated Learning Systems in English and Mathematics education in secondary schools. The study was designed to explore the opinions of the teachers who are involved in the implementation of Integrated Learning Systems and the opinions of the pupils who have been using the system as an educational support tool in addition to their normal classroom activities.
The research investigated the pedagogical and practical issues involved in the integration of the software as an additional tool into the teaching and learning activities of the traditional classroom. The research part of the paper is based on semi-structured interviews taken from teachers and pupils from four different secondary schools.
The findings from this research are discussed and compared with the software vendors' claims and other research in the field. Within this context, an analysis of the factors that support and inhibit the implementation of Integrated Learning Systems in both the teaching and learning activities of the schools is provided.
There has been a prolific expansion of educational software in recent years. Software applications are now used in many diverse settings and can be an alternative or supplementary learning medium which can be utilised to serve all kinds of educational goals (Dickson & Parsons, 1998).
Computer Assisted Instruction (CAI) was one of the earliest and most significant applications of computer technology to education. The computer industry itself was among the first to use CAI in the late 1950's to train industry personnel (Mills & Ragan, 1994). But although computers entered the United States' schools in the 1960's and Britain's schools a few years later, "the computer was initially the object of study in its own right..." (Underwood & Underwood, 1990, p.5).
In 1970's individualised instruction gave a new direction to CAI software development and prompted research in the field of Intelligent Tutoring Systems (ITS). ITS are considered to be an educational sub-product of Artificial Intelligence (AI). Although AI derives form CAI, it differs from the latter in the use of AI tools in order to know what they teach, whom they teach, and how to teach (Mendes et al, 1996). To some extent all the products of AI claim to possess 'intelligence'. (MacKenzie, 1990). Many of the strategies of effective instruction have been incorporated into educational software resulting in the construction of Integrated Learning Systems (ILS) (Mills & Ragan, 1994).
Many researchers and commentators (e.g. Underwood, 1997; Bagley, 1996; NCET, 1994; Mills, 1994; Steeg, 1998; Becker, 1992) give an analytical description of an ILS and its main components. These are: curriculum or subject content, a pupil record system and a management system.
The research questions and the methodology
This research project explored the attitudes of the teachers who are involved in the implementation of Integrated Learning Systems in schools and the attitudes of the pupils who have been using the system as an educational supporting tool in their normal classroom activities. The study attempted to find answers to the following questions:
1) How do teachers perceive the educational goals of the system?
2) What are the practical problems that schools experience in their efforts to integrate Integrated Learning Systems into the traditional classroom activities?
3) How do pupils assess the value of their interaction with the system?
4) How does the use of ILS alter the social interactions and the patterns of behaviour of the traditional classroom settings?
5) What do teachers think about the learning outcomes of its use?
The present study made use of semi-structured interviews.
The sample consisted of six teachers from four different secondary (11-16) schools and seven pupils, from three different secondary schools. Four were IT teachers, one a mathematics teacher and one an English teacher. Four of the pupils were male and three were female. Three of the schools used Successmaker while the fourth used Global.
Results and Discussion
The recorded and unrecorded data that were collected from the semi-structured interviews were scrutinised, grouped and compared. Following that, the data were interpreted further to extract the main points of interest reported by the interviewees.
a) Is the ILS of greater benefit for pupils of a certain ability range than others?
Vendors claim that ILS can respond successfully to the individual needs of every pupil independent of the band of ability they fall into.
The research that has been conducted so far on ILS, though of low academic rigour, suggests that ILS are of greater benefit for pupils at either ends of the ability distribution.
Teachers in the study held the belief that ILS can be beneficial for pupils from the whole range of abilities but they mentioned that for the moment they are using the software to help the low achieving pupils to catch up with the rest of the class. Some of them suggested that ILS can be more beneficial for the low-ability pupils in terms of motivation for learning and enhancement of confidence. However, teachers also suggested that if there were more hardware and time available at their disposal, they would probably use the ILS to meet the advanced needs of the gifted and talented pupils as well.
b) Learning gains
According to Bentley (1991, p.25), "The majority of the research has been done by the very companies that market the systems, which creates serious conflict of interest and raises questions as to the reliability of the product of such a company."
Henry Becker, one of the leading researchers on ILS in the USA, conducted a critical review and meta-analysis of thirty evaluation reports in 1992 (Becker, 1992). In 1994 the NCET in Britain began conducting a national trial of two ILS packages, the American Successmaker and the British Global in twelve UK schools. According to Wood et al (1999, p.95) effects, from NCET's research and similarly from Becker's meta-analysis, "range from strong, significant and positive, through non-significant and neutral, to relatively small but consistent negative impacts of ILS on outcome criteria."
Teachers taking part in the study had no clear picture of the effects of the ILS on the academic achievement of the pupils. Their views were mainly based on readings of past evaluations of the systems, which they tended to accept uncritically, rather than on their personal experience of the ILS.
On the contrary, pupils appeared to be very enthusiastic about their learning gains. They felt that they had made progress unusually fast on the system - that is unusual compared to the rate of progress in the traditional classroom. However, this progress was measured by the system's internal tests and not by the school's standards - and this raises an important question concerning the objectivity of the vendor's claims about the learning gains attained by the users of ILS and their transferability. One possible explanation for greater caution expressed by the teachers may be that pupils are not aware of the cost and other constraints on the implementation of the ILS.
c) Comparison of teaching practices
The pupils interviewed mentioned the difference of teaching practices and they showed a preference for their teacher. They felt that their teacher could respond more effectively to their difficulties than the system usually can and the teacher could answer their questions in more detail. As one of the pupils explained, "you can't talk to the system...but with the teacher you can talk...the teacher explains with more detail." (Pupil C1).
The teaching approach that the ILS uses is based on the assumption that newly acquired skills have to be practiced repeatedly until they become automatic. According to Wood et al (1999, p.92), "ILSs operate on a neo-behaviouristic model of learning which uses automatic task selection, guided practice and feedback to deliver core curriculum content and skills through individualised tutoring and practice." Although this approach, which is mainly based on the behaviouristic learning model, might be successful for the acquisition of very specific skills, it is probably not appropriate for teaching higher order cognitive skills.
Pupils seemed to appreciate the feedback and reward systems of the ILS. According to the majority of the teachers, the instant feedback that the system gives to pupils on every attempt is something they are not likely to get in the traditional classroom and keeps them more motivated and active.
d) The management system
The pupil record system and the ability of the majority of the existing ILS to produce printed records of each pupil's progress on almost every activity is claimed by the vendors to be one of their stronger selling points.
Teachers found this particular feature of the ILS the most time-consuming part of the management system to deal with. The ILS is capable of producing vast amounts of data on the pupils' progress. Teachers claimed that the time that is required to go through every pupil's printed reports carefully is unrealistic keeping in mind the tight time schedule of their job. They believed that if they had the time thoroughly to examine the system's records of achievement, this would prove to be valuable in their own assessment of the pupils' progress. Teachers in the study were unable to make extensive use of this feedback information.
e) Supervision and intervention
Teachers believed that the role of the teacher in the use of ILS is important. As one of the teachers mentioned "as in any other classroom activity the teacher makes the difference...and even on a program like that, the personality of the teacher in the classroom makes significant difference." (Teacher C). Teachers distinguished the role of administrator who is responsible for the supervision of pupils while on task from that of support teacher who is responsible for helping pupils understand the learning objectives of the ILS before each session and the progress they had made after each session.
However, although pupils mentioned that they were always supervised while on task either by their teachers or members of the IT staff, they said that teachers did not intervene in the ILS session in any other way. This contradiction between the teachers' suggestions and the pupils' experiences implies that although the teachers have often a clear picture of what effective use of the ILS requires of them, they often fail to meet these demands, probably due to shortage of time and lack of organisation.
f) Transferability and sustainability of learning gains
According to Ngeow (1998) transferability of knowledge is vital to learner success. She suggests that "learners who have mastered the content knowledge of a particular domain are fully capable of displaying sophisticated use of effective strategies in new situations..." (Ngeow, 1998, p.2).
Most teachers did not comment on this issue based on their personal experience but on published evaluations, which they had come across before the implementation of ILS in their school and they tended to accept them uncritically. The researcher felt that the teachers were predisposed towards the success of the implementation of the ILS, since they were aware of the school's investment of time and money and so tended to rely on the evaluations of ILS that had encouraging results.
g) Behavioural gains
The majority of teachers in the study, based on personal observation, believed that pupils are more highly motivated to work on the ILS but some of them attributed this fact to the means of delivery of the material - the computer. They thought that almost anything that runs on a computer would attract pupils' interest.
An interesting point raised by one of the teachers was that use of the ILS resulted in improved behaviour, increased motivation and confidence in the classroom and it may be that these are the real gains that are transferred back to the 'normal' classroom. It is possible that what the ILS has to offer primarily is an enhancement of motivation for learning and of confidence rather than improvement of specific skills.
On the same subject, pupils admitted having better concentration when working on the system rather than attending a lesson in the traditional classroom. As one of the pupils said: "You are looking at the system not at somebody else...when you're in the classroom you're looking everywhere." (Pupil C2).
h) Sustainability of motivation
According to the teachers studied, motivation for work on the ILS seemed to decrease with the number of sessions. This was attributed to a number of factors. One of them is that an ILS loses its novelty value for the pupils after some time. Another reason, which was mentioned by the mathematics teacher but was supported by most of the other teachers as well was that when pupils work on the ILS it usually means that "...they've missed 20 minutes worth of teaching, so they are behind where their friends are...in the beginning of the year they were all very happy...but towards the end of the year they weren't happy anymore because they were missing (the normal lesson)." (Teacher D1).
On the other hand, most of the pupils did not report to the researcher any of the above concerns and were enthusiastic about using the system in the future. However, they did mention that occasionally their interest decreased owing to the repetitive nature of the tasks presented to them by the program. As pupil B2 claimed, "Sometimes it (the ILS) gets on your nerves when it tells you all these examples and goes on for ages."
The Successmaker software is manufactured in the United States and embodies cultural and linguistic differences from British or International English. A British English version of it has been produced, though some of the schools are still using the older versions of the software.
Both teachers and pupils complained about the American expressions that the system often uses, especially about the American accent in the literacy modules of the ILS.
"Yeah...it was a bit American...some words were strange." (Pupil C1)
According to some teachers the mistakes that the system is trying to pick up in the assessment tasks are American in nature. "The major difficulty is how Americanised it is. In English, the mistakes it (the system) is looking for are classical American mistakes which British students do not usually make" (Teacher D1).
j) Teacher training
Teachers seemed to be generally, satisfied with the training they had received when the ILS software was first set up. However as one of the teachers mentioned "...the training was very good but ... people move on, people's responsibilities change and you need new training or updating because people forget." (Teacher A). The training offered by the Research Machines PLC is concerned with the administration and the management of the software but not with the difficulties which the school and the teaching staff are likely to face while trying to integrate the ILS or the pedagogical issues that the implementation procedure of the ILS is likely to raise.
The majority of the teachers agreed that the cost of the ILS software, its set-up, maintenance and licensing usually consume a very large part of the annual school budget. It is exactly because of this high cost that the teachers' expectations of the ILS are high and that they admitted to feeling that, for what the ILS costs, they have yet to realize and exploit the full potential of the system.
Keeping all that in mind, one might wonder "...would that additional cost of equipment and software be better spent on smaller class sizes, on raising teacher salaries to attract more educated people into the teaching profession, on more print materials, on inservice training for handling classroom management problems or on training for using other innovative instructional programs?" (Becker, 1992, p.3).
The research project on which the present brief report is based on provided some answers to the research questions stated in the methodology but also generated many other questions that are worthy of further investigation.
Some of them have to do with the learning gains attained by the use of ILS.
What are the learning outcomes from the use of the ILS?
Are the gains transferable to traditional classroom settings? Are they sustainable over a long period of time? What is their effect on the formal school achievement record?
Longitudinal, independent and in-depth investigations on the above issues are yet to be seen.
Bagley, M. (1996 April/May). "Integrated Learning Systems". Educational Computing and Technology, 17, 8-12.
Becker, J. H. (1992). "Computer-Based Integrated Learning Systems in the Elementary and Middle grades: a critical review and synthesis of evaluation reports". Journal of Educational Computing Research, 8(1), 1-41.
Bentley, E. (1991). "Integrated Learning Systems: the problems with the solution". Contemporary Education, 63(1), 24-27.
Dickson, S. and Parsons, D. (1998). "The odd couple - skill learning and computer software". British Journal of Educational Technology, 29(4), 371-374.
MacKenzie, S. I. (1990). "Courseware evaluation: where is the intelligence?". Journal of Computer Assisted Learning, 6, 273-285.
Mendes, C. D. M., Nunes, V. G. M., Andreucci, A. C. (1996). "On the use of Intelligent Tutoring Systems for teaching and learning mathematics". Humanistic Mathematics Network Journal, 14, 45-49.
Mills, C. S. (1994). "Integrating computer technology in classrooms: Teacher concerns when implementing an Integrated Learning System". Site 99: Society for Information Technology & Teacher Education International Conference (10th, San Antonio), 7.
Mills, C. S. & Ragan, J. T. (1994). "Adapting instruction to individual learner differences: A research paradigm for Computer-Based Instruction". Proceedings of selected research and development presentations at the 1994 National Convention of the Association for Educational Communications and technology (16th, Nashville), 23.
NCET. (1994). Integrated Learning Systems: A report of the pilot evaluation of ILS in the UK. Coventry: NCET.
Ngeow, K. Y. (1998). "Motivation and Transfer in Language Learning. ERIC Digest". ERIC Clearinghouse.
Steeg, T. (1998). "Getting under the skin of an ILS". Micromath, 14, 15-20.
Underwood, D. M. J. (1997). "Integrated Learning Systems: where does the management take place?"" Education and Information Technologies, 2, 275-286.
Underwood, D. M. J. and Underwood, G. (1990). Computers and Learning: Helping children acquire thinking skills. Great Britain: Basil Blackwell Ltd.
Wood, D., Underwood, J., Avis, P. (1999). "Integrated learning systems in the classroom". Computers and Education, 33(2), 91-108.
This document was added to the Education-line database on 10 October 2003