Evaluating School Self-Evaluation
by Peter Rudd and Deborah Davies
National Association for Educational Research
Paper presented at the British Educational Research Association Conference, Cardiff University, 7-10 September 2000
School self-evaluation now sits alongside, and has been embraced by, external inspection as a major mechanism for monitoring and raising standards of achievement in schools.
Peter Rudd and Deborah Davies of the NFER report on a project that has mapped out what schools and LEAs have been doing in terms of school self-evaluation and explain what the main benefits have been for those involved in such processes.
During the mid-1990s and certainly from 1993, the year in which Ofsted first started to carry out school visits, external inspection was seen as the main driving force in terms of the evaluation of school and pupil performance. Several years on, however, it is apparent that the processes and frameworks used as a basis for inspection have been modified so as to take greater account of a growing drive for internal, self- evaluation, arising from the desire of schools and teachers to assess for themselves how well they are doing. By 1996, for example, a new inspection framework was introduced which gave greater emphasis to a school's own evaluation of its strengths and weaknesses (Earley et al., 1996).
Further, recent publications from Ofsted such as School Evaluation Matters (1998) have focused directly upon self-evaluation and have offered advice to schools about the processes involved in carrying out evaluation. It is clear that Ofsted now views external inspection and self-evaluation as complementary activities. Thus the Handbook for Inspecting series (1999a, 1999b) states that 'OFSTED is committed to promoting self-evaluation as a key aspect of the work of schools'. The nature of the relationship between Ofsted inspections and school self-evaluation is described as follows: 'It is advantageous to base school self-evaluation on the same criteria as those used in all schools by inspectors. A common language has developed about the work of schools, expressed through the criteria. Teachers and governors know that the criteria reflect things that matter' (Ofsted, 1999b, p.138).
This makes sense and it is true that a majority of schools view the inspection criteria as being a suitable basis for self-evaluation, but our research revealed that some difficulties still remain, arising from the fact that self-evaluation and school inspection are not the same thing. As we investigated what was happening 'on the ground' it became evident that there were still some tensions between the (external) requirement for inspection and (internal) school-based desires for self-evaluation and improvement. This is why a number of schools and Local Education Authorities (LEAs) have made use of frameworks other than that supplied by Ofsted, including quality assurance standards, such as 'Total Quality Management', British Standards indicator BS 5750, 'Investors in People' and school-driven frameworks such as that suggested by MacBeath (1999) in his influential work Schools Must Speak for Themselves. Typically, schools or LEAs make use of a combination of elements of the Ofsted framework (including relevant checklists from the Ofsted Handbooks) and customised, LEA-produced evaluation tools or elements from other frameworks such as those mentioned above.
The project reported on here was carried out from 1999 to 2000 and was part of the Local Government Association Educational Research Programme. The main aim of the research was to conduct a mapping and critical examination of what schools, in many cases supported by their LEAs, were doing in terms of self-evaluation and self-review, given the developing policy focus on these activities. What were the benefits for schools and LEAs of carrying out self-evaluation and what issues still need to be resolved? We aimed to find out whether, beneath the rhetoric, there really was beginning to develop in schools a culture of self-evaluation in which target-setting and related processes could take root and flourish, a culture of self-evaluation which might eventually supersede external inspection as the predominant way of driving a school forward.
The project involved a review of key documents and reports relating to school self-evaluation and self-review, along with fieldwork visits to LEAs and schools. A programme of case-study work was carried out involving in-depth interviews with LEA personnel and with senior management teams and selected teaching staff in schools that were in the process of self-evaluation. Nine LEAs were featured in the review (one of these was located in Wales and the remaining eight were English), with documentary evidence collected from a further five English LEAs. These were LEAs that appeared to be 'very active' in the field of school self-evaluation, as indicated by the number and contents of relevant LEA documents held in the NFER's Educational Management and Information Exchange (EMIE) database. Care was taken to ensure that a range of different LEA types was featured in the review. A senior officer or adviser, with direct responsibility for school self-evaluation, was interviewed in each of these nine LEAs and a total of 27 school staff, from 23 schools in these authorities, were also interviewed. The usual pattern was to visit one primary and one secondary school in each authority, though there were some variations on this. The schools visited were usually those that had been suggested by the LEA interviewee as being particularly 'active' in terms of self-evaluation.
LEA perspectives on school self-evaluation
Many of the LEA officers reported how useful school self-evaluation processes were, both for the authority and for their schools. The main positive aspects of school self-evaluation, as described by these officers in the interviews, can be summarised as follows.
1. The need for monitoring and evaluation provides the LEA with a useful 'way in' to their schools. In other words, the requirement for school self-evaluation provides reasons for initiating and maintaining close contact, on the part of LEA personnel, with schools and their staff.
2. The implementation of school self-evaluation helps the LEA to develop an overview of how their schools are performing. There is a dual purpose to school self-evaluation in this respect. Firstly, there is a professional development function - LEA personnel, in working with schools, can help to train staff in methods of data collection, evaluation and analysis. Secondly, the collection of evaluation data is, in itself, useful to the LEA. Such data can be used not only to see how pupils and teachers in a particular school are doing, but also to develop an overall picture of how the borough's schools are performing.
3. School self-evaluation processes help to facilitate the development of positive working relationships between LEAs and their schools. The great majority of LEA officers interviewed had a clear view that they were working with schools: 'Ofsted's vision for school self-evaluation seems to be inspection. [But] inspection is something that is done to you; self-evaluation is something you are part of'. School self-evaluation was a relatively new, but very useful, mechanism for the encouragement of good working relations, sometimes in areas where there had previously been hostility between schools and the LEA. The key to these good working relationships appeared to be the use of consultation and the maintenance of dialogue between school staff and LEA personnel.
4. From 1998, as a requirement of the School Standards and Framework Act of that year, LEAs found themselves having to write Educational Development Plans. Some LEAs have had concerns about how schools can feed in to their aims and goals as expressed in the EDP - and school self-evaluation, for many advisors and officers, represents an important link or bridge between the Education Development Plan and School (or Institutional) Development Plans.
5. Several LEA interviewees indicated that school self-evaluation had the effect of informing and supporting the Ofsted inspection process. Sometimes evaluation was directly linked to inspection requirements, but, whatever form of self-evaluation or self-review was adopted in an authority, it usually had a 'knock on' effect of helping a school to 'know where it is at' prior to an inspection.
Although there were many positive aspects to school self-evaluation processes for local education authorities, and the culture of LEAs and schools was changing towards increased self-evaluation and self-review, there remained a number of problems and issues to be dealt with. A major source of tension was the need for the LEA to achieve a balance between 'managing' and 'supporting' schools. Just how far could LEA management and intervention in these processes go? To what extent could the schools be relied upon to carry out rigorous and demanding self-assessments? One LEA interviewee felt that 'not enough of the schools were asking themselves difficult questions'. Similarly, there was an issue of whether school self-evaluation processes should LEA-driven or school-driven. Whilst many LEA personnel expressed a desire for schools to have 'ownership' of the process of self-evaluation, and that such evaluation should be from the 'bottom up', several also acknowledged that self-evaluation is, in practice, largely LEA-led. Only two of the authorities actually made direct use of MacBeath's Schools Speak for Themselves framework, though all were aware of it. There were also some resourcing issues. Should the schools be made to pay for supported school self-review? How should the provision of such support by the LEA be funded or organised? Some LEAs charged schools a rate per hour for advisory staff; others have taken a broader approach, providing a number of standard days for schools, without explicit charges.
School perspectives on school self-evaluation
Interviews with staff in schools revealed that self-evaluation, in its various forms, has been a positive experience for headteachers and class teachers in a number of ways. The benefits to schools of self-evaluation were reported as being that:
1. School self-evaluation can help bring about a change in the culture of a school, helping to formalise and to extend existing processes of evaluating teaching and learning and data analysis. One headteacher outlined how he was using a whole-school evaluative approach which rested on the ability to change: 'I believe in a thinking, changing school and a thinking, changing teacher who will develop a thinking, changing child'. One aspect of changing school culture was an increased willingness to use methods of evaluation that had not necessarily been used previously in the school including, for example, the technique of classroom observation by peers.
2. Teachers' professional development can benefit from a school's commitment to self-evaluation, particularly in an institution where staff are encouraged to share expertise with colleagues and to take up training opportunities. Some schools had adopted an explicit approach, using packages such as 'Investors in People'; while for others professional review took place within a more general framework such as an LEA package.
3. For some headteachers, particularly those recently appointed, school self-evaluation has provided a means to learn about their school and to organise change. In other words, the processes and mechanisms provided school senior managers with a framework (and 'levers') for the management of change. One headteacher approached self-evaluation as being an important part of a process of strategic planning: 'it's fundamental to where you are, what you are achieving and where to move forward. If you don't [self-evaluate] you stagnate'.
4. Schools can develop their own agenda for self-evaluation, enabling teachers to focus on aspects of the school that they identify as areas for improvement. Furthermore, the internal agenda set within schools can also help promote ownership among teachers of their self-evaluation activity. While it was clear that much of the impetus for self-evaluation was generated by headteachers, particularly in the early stages, the majority were keen to encourage teachers to become involved in these processes.
5. Many school interviewees said that they had benefited from having the support of a 'critical friend', whether an LEA advisor, consultant or colleague from another school. A critical friend who is external to the school can help teachers identify areas for development, meet the demands of a timetable for implementing and evaluating activities and can 'ask difficult questions'.
6. School self-evaluation can be used to encourage community involvement. Parents, pupils and governors can provide useful feedback, inform classroom practice and help to set the agenda for change. There was evidence that self-evaluation had afforded some schools the opportunity to involve pupils and parents in the process. Several school interviewees said that their planned 'next step' in the evaluation process was to seek the views of parents or pupils. As one headteacher said: 'children know what helps their learning and what doesn't'.
7. Self-evaluation packages and programmes, whether developed 'in-house' by LEAs or 'bought in', can provide schools with a range of tools for implementing evaluation activities. These may take the form of questionnaires for parents and pupils, observation checklists, files for recording data, or some other format. 'Toolkits' for schools avoid the need for teachers to 're-invent the wheel' and can facilitate the sharing of information across institutions.
Although it is clear that self-evaluation can have a positive impact on many aspects of school life, self-evaluation also presents school communities with a variety of difficulties. For example, the degree to which schools have ownership over their self-evaluation activity varies considerably. It is clear that much of what happens in schools relating to self-evaluation has not, to date at least, followed a 'bottom-up' model as outlined by MacBeath (1999). This apparent dichotomy between 'official' and 'unofficial' or 'internal' and 'external' aspects of school self-evaluation was highlighted in a good proportion of the school interviews. A further difficulty lies in the extent to which school self-evaluation is embedded in the school - in other words, whether or not ownership is confined to the headteacher or senior management team. Once again, there was little evidence to suggest that schools had adopted a genuinely 'bottom-up' approach despite many headteachers' acknowledgements that they needed to 'have the staff on board'.
There was evidence in one or two schools of there being suspicions among staff as to the purpose of school self-evaluation, sometimes related to the appraisal system or to national plans for the introduction of performance related pay. For example, a primary school headteacher had encountered some difficulty in overcoming negative perceptions of self-evaluation among staff whom she described as 'battered by inspections'. However, she felt that initial anxieties had diminished: 'it's okay when teachers realise that it's a mirror rather than a stick'. Anxiety about the impact of self-evaluation on teachers' workload was a more widespread concern. Staff in some schools felt that they were suffering initiative fatigue. Further, a lack of resources for self-evaluation, in the form of time, training and material support, often constrained schools' ability to participate in self-evaluation. In some schools staff have been required 'to fit school self-evaluation in around existing commitments'. Teachers in primary schools reported that the lack of non-contact time available has impacted on self-evaluation work, particularly the organisation of classroom observation.
On occasion, school self-evaluation can present difficulties for schools' relationship with their LEA. Some school staff felt that their LEA had not yet developed a coherent message, with a united working approach across all departments and services. Different sections within an LEA sometimes had different views on the purposes self-evaluation and on how evaluation linked with school improvement. Conflicting messages were felt by some teachers to hamper their efforts to implement self-evaluation. One headteacher described the overall support she had received from her advisor as 'brilliant and exceptional', but she was critical that there was 'no joined-up thinking in the LEA.'
Despite the difficulties of implementing (and acting upon) school self-evaluation strategies, there was much evidence from our research to suggest that schools and LEAs are some way towards embracing such processes. In this respect it seems likely, although the 'Self-inspecting school' (Ferguson et al., 2000) may still be some way off, that school self-evaluation will be developed further as an essential mechanism for monitoring and improving teaching and learning at the institutional level.
However, the relationship between self-evaluation and preparation for inspections needs to be considered particularly carefully. School self-evaluation has several functions and evaluation activities can be seen as being shaped by at least four sets of influences: (1) the impact of government initiatives, such as the introduction of the literacy and numeracy hours and national requirements for target setting; (2) the demands of Ofsted Inspections; (3) the role of the LEA in processes of school improvement; and (4) the purposes of school self-evaluation as defined by schools themselves.
Is the purpose of self-evaluation to prepare for inspection, to assist in the process of achieving performance targets, to 'raise standards', to assist teachers' professional development or, as is often the case, some combination of these things? (see Saunders, 1999). One LEA officer raised some important issues about the influence of Ofsted requirements upon school self-evaluation processes, expressing concerns about why schools wanted to be involved in school self-evaluation. His own LEA had 'pushed' the Ofsted package, but was concerned that schools might be adopting it as a limited preliminary inspection process rather than as a broad tool for self-evaluation and professional development. The central question that needs to be asked by LEA officers, school managers and staff, is 'who or what is school self-evaluation for?'
DAVIES, D. and RUDD, P. (forthcoming). Evaluating School Self-Evaluation. Slough: NFER.
EARLEY, P., FIDLER, B. and OUSTON, J. (1996). Improvement Through Inspection? Complementary Approaches to School Development. London: David Fulton.
FERGUSON, N., EARLEY, P., FIDLER, B. and OUSTON, J. (2000). Improving Schools and Inspection: the Self-Inspecting School. London: Paul Chapman.
MacBEATH, J. (1999). Schools Must Speak for Themselves: the Case for School Self-evaluation. London: Routledge.
OFFICE FOR STANDARDS IN EDUCATION (1998). School Evaluation Matters. London: OFSTED.
OFFICE FOR STANDARDS IN EDUCATION (1999a). Handbook for Inspecting Primary and Nursery Schools with Guidance on Self-evaluation (Inspecting Schools). London: The Stationery Office.
OFFICE FOR STANDARDS IN EDUCATION (1999b). Handbook for Inspecting Secondary Schools with Guidance on Self-evaluation (Inspecting Schools). London: The Stationery Office.
SAUNDERS, L. (1999). 'Who or what is school 'self'-evaluation for?' School Effectiveness and School Improvement, 10, 4, 414-29.
Address for correspondence
Peter Rudd or Deborah Davies, NFER, The Mere, Upton Park, Slough, Berkshire, SL1 2DQ.
Tel. 01753 574123: Email firstname.lastname@example.org, email@example.com.