Education-line Home Page

 

NATIONAL FOUNDATION FOR EDUCATIONAL RESEARCH

 

School Improvement through ICT: Limitations and Possibilities

Peter Rudd

Paper presented at European Conference on Educational Research (ECER) University of Edinburgh, 22nd September 2000

 

ABSTRACT

In recent years the literature on school improvement has flourished and much the same can be said about the literature relating to the impact of ICT on teaching and learning. However, reading across these two bodies of educational writings, one is left with the impression that these are two very discrete fields of enquiry.

Writings on school improvement and school effectiveness have tended to concentrate on aspects of measuring and raising 'standards' in schools - and although there has been some emphasis on the key characteristics of successful schools, ICT has rarely been discussed as one of these characteristics or as a major element in any drive towards improving school and student performance. Similarly, although the ICT literature has made considerable strides in terms of, for example, examining the links between ICT and student learning and motivation, the role of ICT as a direct factor in bringing about improvements in school and student performance has been largely neglected.

As increasing amounts of money are spent on ICT and new learning infrastructures in schools, it is important that these two areas of educational research and practice should be brought closer together. This article calls for more direct research and discussion on the issue of how ICT can contribute to processes of school improvement. It examines the ways in which findings from these two sets of literature converge and diverge, and provides examples of how insights from both areas might be brought together, to the advantage of schools, teachers, students and parents.

 

INTRODUCTION: TWO DISCRETE AREAS OF RESEARCH

In recent years there has been a great deal of research activity and writing on, firstly, school improvement and school effectiveness and, secondly, on the use of Information and Communications Technology (ICT) in schools. It would not be an exaggeration to say that these are two areas of educational research where great strides have been made in terms of finding out more about the links between processes of teaching and learning, and also in terms of producing findings that are useful to practitioners and policy-makers.

However, to a large extent, the similarities end there and reading across these two bodies of research literature one is left with the impression that these are two distinct, almost unrelated areas of educational research. Writings on school improvement and school effectiveness have tended to concentrate on aspects of measuring and raising 'standards' or learning outcomes in schools - and although there has been some emphasis on the key characteristics of successful schools, ICT has rarely been discussed as one of these characteristics or as an essential element in any drive towards improving school and student performance. Similarly, although the ICT literature has made considerable advances in terms of, for example, examining the links between ICT and student motivation, the role of ICT as a direct factor in bringing about improvements in school and student performance has been under-researched.

In many ways this tendency against linking these two major areas is understandable. These are two of the most extensive, demanding and complex fields of educational research and practice and it is not surprising that those involved have had to devote much of their energy to making progress within their own particular specialist area. However, given the current importance of (and the amount of spending on) ICT initiatives in schools, and the fact that policy makers in the United Kingdom particularly, but in other countries too, are expecting returns from these initiatives in the form of improved 'standards' of student performance, the time is right for a bringing together of research from these two areas of education.

The primary aim of this article is to encourage more direct research and discussion on the issue of how ICT can contribute to processes of school improvement. It examines the ways in which findings from these two sets of literature converge and diverge, and provides examples of how insights from both areas might be brought together, to the advantage of all the relevant stakeholders.

It is acknowledged from the outset that encouraging such collaborations is not an easy task. However, the central importance of new technologies and their effects on teaching and learning means that attempts to assess directly how ICT can contribute to school improvement and the quality of educational experiences must be put in train now. Policies to supply and implement new technologies to enhance teaching and learning are forging ahead and it is, arguably, the responsibility of educational researchers and practitioners to evaluate such policies in rigorous and meaningful ways and to make recommendations on how these and future initiatives can best be shaped to maximise school and student performance.

 

ICT WITHIN THE SCHOOL IMPROVEMENT LITERATURE

It is beyond the scope of this article to discuss in any detail the very expansive literature that exists on school effectiveness and school improvement (SESI) and there are in any case a number of overviews available for readers interested in developing their understanding of this field (e.g. Stoll and Riley 1999a; 1999b). However, it is possible to sum up briefly some of the main developments in this area and to consider how, if at all, the impact of ICT has been featured in these writings.

Mortimore (1998) has put together a collection of writings, based upon over twenty years of research, that represents a kind of 'history' of the development of school effectiveness and school improvement work. He notes that 'it is apparent as one reads them that terminology has changed, methods of inquiry have advanced and statistical techniques have become ever more sophisticated' (Mortimore, 1998, p.5). The Fifteen Thousand Hours study of secondary schooling carried out in 1979 by Mortimore and his colleagues:

broke new ground by creating a methodology to identify a set of student outcomes; to measure, in a sample of schools, the range of effectiveness by taking account of the different backgrounds and attributes of pupils entering these schools, and then endeavouring to 'backward map', to identify the characteristics of the seemingly most effective schools in the sample (Mortimore 1998, p.3; Rutter et al., 1979).

This research, partially replicated in primary schools in the School Matters study (Mortimore, 1995), brought together a number of variables that could impact upon student performance. In some ways this has now become almost the basic methodology for looking at school effectiveness, though there are debates about what variables should feature (and the relative impact of the 'school effect' as compared to the effects of social disadvantage) and whether and how multi-level models should be used.

The 'input' variables tend to include gender, the ethnic characteristics of students and measures of prior attainment. Socio-economic indicators such as social class of parents and the percentage of students eligible for free school meals have also been used. To date, levels of ICT usage or computer literacy or any similar type of variable have not been utilised to any significant extent in these types of considerations. This is understandable given that, firstly, it has been difficult enough to establish correlations or links between the inputs listed above and outcomes (such as examination results) and, secondly, there are a number of difficulties in defining ICT provision, usage, ability or competence. However, this may be the right time to commence work on defining some such variable and building it in to models or assessments of school or student performance.

Another advance was the use of 'value added' measures of school and student performance - measures that attempt to isolate the school's contribution to improved student outputs from the effects of external influences. Although there are always difficulties in comparing the performance of one school with that of another, because of fundamental differences in contexts and intakes, this did facilitate some general comparative work and, partly as a result of this, in the 1990s some reviews of SESI research were able to suggest that there were a number of common characteristics in successful schools. Sammons and her colleagues (1995) reviewed and summarised these in an influential paper entitled Key Characteristics of Successful Schools. Eleven characteristics were identified in this study. Given that the SESI literature had not yet embraced the new technologies as a direct influence in successful schools, it was not surprising that this list did not include ICT in any shape or form. It seems that the SESI research in general was having difficulties in taking into account the impact of resource issues, whether these resources were financial, human or technological.

MacGilchrist and her colleagues moved the 'key characteristics' approach forward considerably by stressing that there was no single blueprint for improving schools (1997, p.xv). Following the work of Gardner, these writers identified nine 'intelligences' that 'when used in combination enable a school to have the capacity to achieve its goals successfully' (p.xvii). It is noticeable that their list of nine intelligences did not include technical or technological intelligence or any intelligence that can be directly linked with, or encompasses, ICT. Rather, the emphasis was on things like 'academic', 'emotional' and 'spiritual' intelligence. These authors, of course, are not claiming that their list is comprehensive, nor does their analysis preclude the use of ICT to improve schooling: but this is an interesting omission, one which supports the view expressed earlier, that academics working in this area have not yet fully been able to get to grips with the fundamental significance and potential of ICT as a tool, or a set of tools, for school improvement.

Interestingly, at the same time as this 'holistic' or combined intelligences approach was being developed, there was also an emerging recognition that analyses of school performance often masked differences in performance or outcomes within schools. It was shown that differences between departments within a school could be highly significant in any assessment of a school's effectiveness (Sammons et al., 1997). School improvement strategies, from this point onwards, had to address issues of departmental and teacher effectiveness as well as whole-school issues. In this way SESI research was continuing to focus down onto activity within the classroom as the main influence upon student and school performance outcomes. This was a significant development in this field: it created the possibility of, but did not yet establish, a focus and a coverage that could include the impact of ICT at classroom and teacher levels.

In recent years this situation has been complicated by a trend towards 'internationalisation', with the use, often encouraged by policy-makers, of school-based case studies from individual countries (e.g. Townsend et al., 1999; see also Teddlie and Reynolds, 2000). These studies confirm that, overall, school improvement and school effectiveness research has made a number of methodological and theoretical advances. There has been much useful work on how learning outcomes (and inputs) can be measured, on how attempts at school improvement can be evaluated and refined, and on 'what works' in the classroom. These are complex areas and it is not surprising that some dimensions of school life, such as the uses of ICT, have not yet been given as much attention as they deserve.

 

SCHOOL IMPROVEMENT AND THE ICT LITERATURE

The literature on ICT in schools and classrooms has expanded almost as rapidly as the new technologies have been developed and put to use in schools and in other educational contexts. Overall, as with the SESI literature, this research is too extensive to summarise in detail here, but for the purposes of this article two types of research on the impact of ICT can usefully be identified. Firstly, research that deals with specific ICT projects, programmes or interventions; and, secondly, research that has a more general concern with the relationships between ICT, teaching, learning and enhancing the quality of education. Arguably, the first type of research is mainly empirical, and the latter type predominantly theoretical, but there is a considerable overlap and, in any case, theoretical or conceptual developments are often based on, or are use to frame, empirical investigations.

Taking the empirical research first, in the past decade, and more recently too, there seems to have been something of an emphasis on relatively small-scale, but nonetheless useful, studies of particular initiatives or projects. Examples include studies of Independent Learning Systems (ILS), of the use of laptops in the classroom, of the impact of Internet or web-based learning and communication, and of school- or LEA-based ICT initiatives. Such studies have often included considerations of how the use of computer hardware and software has impacted upon learning outcomes, but there have frequently and inevitably been difficulties in establishing hard evidence of positive outcomes. For example, the author of a study of Independent Learning Systems concluded that:

The systems evaluated were generally seen by students and teachers as being successful at teaching core mathematical and English skills but only in some instances did this translate into success in tests or examination (Wood, 1998, p.1).

Often evaluations of ICT interventions and initiatives find enthusiasm on the part of teachers and students for using computers, but are not able to establish hard data links between the ICT project and improved student outcomes. This difficulty occurs because, as noted in the previous section, there are many factors impacting upon student achievement and it is very difficult to isolate the impact of an ICT-based intervention from other influences upon achievement.

Studies of this type, to an extent, have to rely upon reported or reputational findings and consequently have been criticised for relying only upon anecdotal evidence: 'There has been a constant griping about there being a real lack of research-based evidence to support the positive effects of information and communications technology' (Selinger et al., Times Educational Supplement, 14 April, 2000). Such 'griping', as these researchers suggest, is rather misplaced because, despite the challenges, there are some very useful and positive findings emerging from these studies. Whilst the ideal would be to establish causal links or positive correlations between ICT inputs and learning outcomes this is difficult, and sometimes impossible, because of the multiplicity of variables impacting upon student outcomes. In any case, the perceptions of teachers and students relating to ICT usage are also very important as these will themselves impact upon motivation, attitudes and learning at classroom and individual levels.

It is also evident that international evaluations of these sorts of interventions can be very useful. Many of the issues and difficulties to do with using ICT to enhance learning experiences transcend national and cultural boundaries and are common to all countries with commensurate levels of technology (see, for example, Selinger and Pearson, 1999, on international issues relating to telematics). An indication of this is the fact that the International Association for the Evaluation of Educational Achievement (IEA) is currently organising a major international comparative study - the Second Information Technology in Education Study - to collect detailed information about ICT practices in schools in participating countries.

The second broad type of research on ICT and its impact consists of general considerations of the relations between ICT, teaching and learning, as opposed to evaluations of specific projects or interventions. It is possible to identify a selection of important substantive strands in this literature, including the following:

This list is by no means comprehensive, but in terms of the search for ways of linking ICT and school improvement, some of the most important developments have been in these strands of this literature, particularly in the way that a focus on the importance of classroom activity has been developed.

Overall, much of the evidence that has formed the basis of this literature is founded upon the perceived benefits (or disadvantages) of using particular types of hardware or software or of particular ICT projects or strategies. Few direct links with school effectiveness and school improvement processes have been made. However, there has been a tendency in these writings in recent years, as with the SESI literature, to move the focus onto teaching and learning in the classroom. The work on classroom organisation, computers and pedagogical styles, student motivation and achievement, and on practical strategies for using ICT to develop learning, and to provide a good foundation for making such links in the future.

 

CONVERGING AND DIVERGING TENDENCIES

In recent years some larger-scale studies of the effects of ICT upon teaching and learning have been initiated, notably the IEA study mentioned above, the IMPACT studies funded by the British Educational and Communications Technology Agency (BECTa) and ongoing evaluations of the UK National Grid for Learning, but it would appear that while the SESI research literature has included a focus upon broad processes of school improvement and learning outcomes, the ICT literature has featured an emphasis on smaller-scale evaluations. School effectiveness research necessarily requires broad considerations of all the major factors that can influence student and institutional achievement and progress. ICT research has been able to concentrate on a particular dimension of schooling, but nonetheless has had to take account of a complex backcloth of contextual and institutional factors that may facilitate or hinder the use of new technologies.

There have also been some differences in the methodologies used, with SESI studies initially tending to work with large samples, often based on student year groups as cohorts. School effectiveness research has primarily been based on quantitative approaches, though in recent years there has been a growing recognition that statistical data, especially 'hard' examination performance data, can usefully be complemented with 'soft' attitudinal information, such as student opinions on the quality of their educational experience.

Although actual and potential areas of overlap or convergence in these two areas of enquiry are difficult to identify, primarily because the two fields have developed in isolation from one another, there are at least three possible areas of convergence:

The narrowing of the research focus onto 'what works in the classroom', along with a renewed interest in 'teacher effectiveness', has been evident in recent investigations in both of these areas. It is clear that, despite some very dramatic changes in the contexts and conditions for teaching and learning, the teacher continues to have a pivotal position in both school improvement initiatives and in maximising benefits from the deployment of new technologies. This is therefore the obvious starting point for any future collaborative work between ICT and school improvement researchers and practitioners.

A number of influential studies in both sets of literature stress the need for monitoring and evaluation. This is very evident in the school improvement field in particular, where there has been a rapid expansion in the collection and use of performance data at departmental, school and LEA levels. This expansion has included the increased use of 'value added' data and the use of external inspection as a form of monitoring alongside internal, formative processes of school self-evaluation. Something similar has happened in studies of the impact of ICT on teaching and learning, though admittedly not on the same scale, in that investigations of the influence of the new technologies have stressed the importance of teacher and student evaluation. The argument has been made that the users of ICT must be allowed and indeed encouraged to feed back their views if ICT provision is to be made more effective in terms of the achievement of school and student learning goals. The trend towards 'practitioners as researchers' is one part of this recognition of the importance of user evaluation.

Finally, both areas of research have made more use recently of international comparative studies. The reasons for this are obvious: although there are some institutional differences between national education systems, and different pedagogical cultures, much can be learned from the examination of school improvement and ICT projects at this level. Whilst it is not always possible to transfer a successful programme or initiative to another context, there can be no doubt that lessons can be learned from particular projects around the world.

 

SCHOOL IMPROVEMENT THROUGH ICT: LIMITATIONS AND DIFFICULTIES

What is immediately evident is that the use of ICT in schools is not a simple panacea for solving problems of underachievement, nor is it a straightforward way of raising standards of student performance. The investment of ICT resources in schools, and the development of accompanying teacher and student skills, should enhance the overall effectiveness of a school and should also improve levels of academic performance, but there can be no guarantees that these things will happen.

There is a small but potentially important literature on the possibility of ICT having negative effects on teaching and learning - and, in discussing the prospects for combining school improvement work with research on the effects of ICT, it is important to bear these critiques in mind. At the extreme, there have been claims that ICT could 'wreck' schools: the American scientist Seymour Papert once said that putting a computer into a school is 'like putting a jet engine on a stagecoach - you gain a temporary increase in speed, but then the whole thing is wrecked!' (Kenny, Times Educational Supplement, 24 March, 2000).

This is an exaggerated analogy, but there does need to be an awareness that there are a number of possible negative aspects to the use of ICT in schools. In particular the student-teacher role has changed and may have become depersonalised. To an extent this relationship has become less direct, less personal and less straightforward because it is now mediated by innovative forms of dissemination and new technology. There are new difficulties of classroom organisation and management and the teacher has had to take on an additional role as change agent or manager of change.

Davis and her colleagues make some astute comments about the possible contribution of ICT to the quality of learning:

It would appear that the conditions for classroom learning can be improved by information technology tools. But, equally, teachers can use information technology to create a new set of mundane tasks which negate the opportunities for quality learning (Davis et al., 1997).

It is clear that ICT projects cannot simply be 'bolted on', or 'imported' in, to school development or school improvement processes.

An additional difficulty is that of methodology and the fundamental problem of establishing a link between ICT and improved school or student performance. As suggested previously, it is very difficult to establish any kind of causal link between an 'input variable' such as levels of ICT provision, or ICT literacy, and 'outputs' such as performance outcomes, whether these are at the institutional or the individual level. The essential problem here is, of course, one that affects all areas of educational research: the complex and multivariate nature of processes of teaching, learning and assessment. To put this issue another way: it might be possible to show that student performance has improved significantly over a period of time in a school which has overseen the implementation of an ICT initiative, with an appropriate increase in hardware and human resources, but it is likely to be impossible, because there are so many other factors to consider in the school, to prove that a definite link exists between increased ICT use (as cause) and improved performance (as effect).

A final difficulty arises from the fact that these two fields of enquiry have, historically, developed independently and with very little reference to one another. They have developed their own methodologies, terminologies and conceptual frameworks and, whilst there are fairly well established interpersonal networks within each of the fields, these networks have not overlapped or interacted with each other. Some researchers may need to change their approaches. For example, SESI researchers may need to recognise and acknowledge that the scale of ICT usage in schools now means that this should be included in any model or analysis of whole-school effectiveness. Similarly, ICT evaluators may need to develop a more wide-ranging awareness of how the new technologies can contribute to broader processes of school improvement, and also of what other factors are at play in such processes. Appropriate intellectual, organisational and practical frameworks need to be put in place so as to encourage collaborations across these two areas.

 

SCHOOL IMPROVEMENT THROUGH ICT: POTENTIAL AND POSSIBILITIES

It will not be easy to overcome the problems described in the previous section. However, researchers from both fields have made positive steps in recent years and it is clear that ICT, despite the problems that inevitably arise with the implementation of new initiatives, has enormous potential for enhancing teaching and learning.

There are many ways in which ICT has contributed, and can contribute, to processes of school improvement. This can occur particularly through the use of new technology as an aid to independent learning, as a motivator of students of all abilities, as a set of tools for professional development and as a set of innovative mechanisms for assessment and monitoring. Further research is needed in all of these areas.

The methodological difficulties described above are not insurmountable. The development and use of multi-level modelling and other advanced statistical techniques means that there are now improved possibilities for linking ICT experience, knowledge and skills with learning outcomes. Whilst it may often be impossible to establish a causal link, there is no reason why associations cannot be demonstrated. In addition, the judicious combination of quantitative data and qualitative information, and collaboration between teachers and researchers, will help to provide meaningful pictures of 'what works' in the classroom.

What comes out very strongly from a reading of the literature on ICT and classroom activity - and this is probably the key to how ICT and school improvement research can be brought together - is a stress on the need for human interaction and structured teaching and learning to accompany the use of new technologies in educational settings. Learning involving ICT applications must be carefully planned, clearly set out and well sequenced (and in this respect the requirements for a successful lesson using ICT are no different from those of a good lesson generally).

Fundamental changes are taking place, but clearly the role of the teacher remains pivotal. The key point is that students must not just simply be left to use the technology (Cox, 1999, p.33) with the assumption that standards of academic performance will improve. Social interaction between students and students, and between teachers and students, is a vital part of the process of learning through the use of computer technology: 'learning is as, if not more, likely to take place via the interactions of pupils with peers and the teacher while using ICT applications as it is via the interactions with ICT itself' (Pachler, 1999, p.14). Just as there is a need to base school improvement on teaching and learning in the classroom, so there is a need to embed students' ICT work in a context of meaningful interpersonal support.

 

CONCLUSIONS: POINTERS FOR FUTURE RESEARCH AND PRACTICE

There are three current educational developments on which future collaboration between SESI and ICT researchers might usefully be based. These are:

Essentially, both school improvement planning and projects involving new technologies in schools have the same overall goals: the enhancement of the student learning experience alongside, and as a contribution to, improved educational outcomes. Although ICT research on the one hand, and SESI research on the other, have so far developed independently of one another, there are good grounds for thinking that the insights developed from each of these fields can be brought together and that collaboration between the two sets of researchers is both possible and desirable. The challenge for practitioners and researchers in the next few years is to assess, evaluate and advise on how ICT can best be used to maximise the effectiveness of schools and the educational experiences and achievements of individual students within them.

Correspondence: Peter Rudd, Senior Research Officer, School Improvement Research Centre, National Foundation for Educational Research, The Mere, Upton Park, Slough, SLY 2DQ, UK. Telephone +44 (0)1753 574123. Email: p.rudd@nfer.ac.uk

 

REFERENCES

COX, M.J. (1999). The Effects of Information Technology on Students' Motivation: Final Report. Coventry: NCET.

COX, M.J. (1999). 'Motivating pupils through the use of ICT.' In: LEASK, M. and PACHLER, N. (Eds) Learning to Teach Using ICT in the Secondary School. London: Routledge.

DAVIS, N., DESFORGES, C., JESSEL, J., SOMEKH, B., TAYLOR, C. and VAUGHAN, G. (1996). 'Can quality in learning be enhanced through the use of IT?' In: SOMEKH, B. and DAVIS, N. (Eds) (1996). Using Information Technology Effectively in Teaching and Learning: Studies in Pre-service and In-service Teacher Education. London: Routledge.

HARRIS, S. (1999). INSET for IT: a Review of the Literature Relating to Preparation For and Use of IT in Schools. Slough: NFER.

KENNY, J. (2000). 'They're taking over!' Times Educ. Suppl., 24 March, 27-8.

LEASK, M. and LITCHFIELD, D. (1999). 'Using ICT in your particular subject.' In: LEASK, M. and PACHLER, N. (Eds) Learning to Teach Using ICT in the Secondary School. London: Routledge.

MacGILCHRIST, B., MYERS, K. and REED, J. (1997). The Intelligent School. London: Paul Chapman.

MORTIMORE, P. (1995). School Matters: the Junior Years. Sage: London.

MORTIMORE, P. (1998). The Road to Improvement: Reflections on School Effectiveness (Contexts in Learning Series). Lisse: Swets & Zeitlinger.

PACHLER, N. (1999). 'Theories of learning and ICT.' In: LEASK, M. and PACHLER, N. (Eds) Learning to Teach Using ICT in the Secondary School. London: Routledge.

RUTTER, M. et al. (1979). Fifteen Thousand Hours.

SAMMONS, P., HILLMAN, J. and MORTIMORE, P. (1995). Key Characteristics of Effective Schools: a Review of School Effectiveness Research. London: University of London, Institute of Education.

SAMMONS, P., THOMAS, S. and MORTIMORE, P. with WALKER, A., HIND, A., SMEES, R., BAUSOR, J. and CAIRNS, R. (1997). Forging Links: Effective Schools and Effective Departments. London: Paul Chapman.

SELINGER, M. (1999). 'ICT and Classroom Management.' In: LEASK, M. and PACHLER, N. (Eds) Learning to Teach Using ICT in the Secondary School. London: Routledge.

SELINGER, M. and PEARSON, J. (Eds) (1999). Telematics in Education. Oxford: Pergamon.

SELINGER, M., WYNN and MUMTAZ (2000). [Article] in Times Educational Supplement On-line, 14 April, p.31.

SOMEKH, B. and DAVIS, N. (Eds) (1996). Using Information Technology Effectively in Teaching and Learning: Studies in Pre-service and In-service Teacher Education. London: Routledge.

STOLL, L. and RILEY, K. (1998). 'From infancy to adolescence: school effectiveness and school improvement in England since 1995.' In: TOWNSEND, T., CLARKE, P. and AINSCOW, M. (Eds) Third Millennium Schools: a World of Difference in Effectiveness and Improvement. Lisse: Swets & Zeitlinger..

STOLL, L. and RILEY, K. (1999). 'School effectiveness and improvement - recent research', Management in Education, 3, 2, 16-22.

TEDDLIE, C. and REYNOLDS, D. (2000). The International Handbook of School Effectiveness Research. London: Falmer Press.

TOWNSEND, T., CLARKE, P. and AINSCOW, M. (Eds) (1998). Third Millennium Schools: a World of Difference in Effectiveness and Improvement. Lisse: Swets & Zeitlinger..

WOOD, D. (1998). The UK Integrated Learning System (ILS) Evaluations, Final Report. Coventry: British Educational and Communications Technology Agency.

This document was added to the Education-line database on 28 June 2001