Using research and evidence to improve teaching and learning in the training of professionals – an example from teacher training in England
Paper presented at British Educational Research Association Annual Conference, University of Manchester, 16-18 September 2004
The Teacher Training Agency (TTA) in England is a government agency with a national strategic role in improving the quality of teacher training.
In 2002, the TTA made a decision to allocate resources to support four goals related to the improvement in quality of teacher training - the codification of the knowledge base, the identification and recording of good practice in the preparation of teachers and the dissemination of this good practice using web-based technology. A team within the TTA - the Effective Practices and Research Dissemination team (EPRD) - was established to work with providers of initial teacher training to establish and disseminate the evidence base for practice.
Whilst this paper focuses on the strategy the EPRD team developed - now in its third year and in a consolidation phase following two years of development and implementation- the international and national context for the drive to achieve evidence-informed practice is common to other professional groups.
Background - the context for 21st century professional practice
Increasingly It is recognised that policy-making and professional practice should be based on the best evidence available (Davies et al, 2000; Gough, 2003). To achieve this goal of evidence-informed1 policy and practice, government departments and agencies as well as professional groups are developing processes to take stock of what evidence exists and what the quality of that evidence is. Increasingly sophisticated web-based means are being used to make the evidence base available to stakeholders. One example across professional groups (see Campbell and Cochrane Collaborations below) is that web-libraries of ‘systematic reviews’ or ‘research syntheses’ of the evidence for policy and practice are being built. Systematic reviews use explicit and agreed processes to synthesise the evidence provided by individual studies. Because of the clarity of these processes, reviews can be ‘topped up’ as new studies emerge. Traditional literature reviews are not easy to update systematically unlike systematic reviews. They do not as do systematic reviews, explicitly state the processes used to identify studies for the review and the inclusion and exclusion criteria used nor are studies included assigned quality ratings.
Examples of initiatives synthesising evidence to underpin policy and practice include the following:
The Cochrane collaboration (www.cochrane.org)2 is ‘an international not for profit organisation providing up-to-date information about the effects of health care with the Cochrane Library containing regularly up-dated evidence-based health care databases’. 10,000 professionals in 89 countries have been involved over the ten years of development to date.
The Campbell Collaboration (www. campbellcollaboration.org) is an embryonic ‘not for profit organisation that aims to help people make well informed decisions about the effects of interventions in the social, behavioural and educational arenas. C2’s objectives are to prepare, maintain and disseminate systematic reviews of studies of interventions’ in areas such as education, criminal justice, social policy and social care.
In the US
in 2002 the ‘What Works Clearing House’ (www.w-w-c.org) was developed to focus on major studies which have been vetted according to specific ‘scientific’ criteria. (Their website provides detailed information).
In the UK
The Health Development Agency (www.hda-online.org.uk) is ‘supporting evidence-based working for better health’ with the website containing items relating to public health including ‘evidence-based briefing documents’.
The National Electronic Library of Health (NELH, www.nelh.nhs.org) provides a vast array of health related information; supports the National Institute for Clinical Excellence (NICE) which ‘makes recommendations on treatments and care using the best available evidence’; produces Bandolier (a journal providing ‘evidence-based thinking about health care’ and subscribes to the Cochrane Library so that all individuals (patients or professionals) with a .uk email address can access this evidence base.
In the field of education in England, the Department of Education and Skills (DFES) funds the National Education Research Forum (NERF, www.nerf-uk.org) which produces a bulletin called ‘Evidence for Teaching and Learning’ and the Evidence for Policy and Practice Information and Co-ordinating centre (EPPI-Centre) which manages and provides training for systematic review groups in education. The growing number of systematic reviews of the evidence base for educational practice is published through the Research Evidence in Education Library (REEL) on www.eppi.ioe.ac.uk.
In funding the EPPI – Centre, the DFES focus was particularly on developing the evidence base for practice in schools. However, the evidence base for practice in school classrooms is only part of the evidence base which teacher educators need to provide the evidence base for their practice so the TTA is commissioning reviews in areas particularly relevant to teacher training. Priorities for these areas are determined through consultation with various agencies and from an annual survey through which newly qualified teachers provide feedback about the quality of their training.
The context (see figure 1) in which the strategy to ‘codify the knowledge base’ was developed was one where over the last decade, teacher educators have moved from a position of having little access electronic networks and databases to one where there is increasing expectation that government agencies and professional associations will provide resources and support electronically and where there is a plethora of materials accessible through the web – one systematic review group found a potential pool of 5,000 articles3. During this period, teacher education has moved from an historical oral tradition, where accessing evidence about what worked in education and up-to-date knowledge was done through word of mouth, learning from colleagues; reading journals and texts; undertaking research and attending conferences for some, to a position where evidence-informed practice is beginning to be supported through access to web-based materials and advice available at the time of need.
The goal of evidence-informed policy and practice in teacher education
A key part of the Teacher Training Agency (TTA) remit given by government, is to work with providers to improve the quality of teacher training.
Independent inspection data (HMI, 2001-2), indicates that the quality of teacher training is higher than ever and that the quality of educational practice of newly qualified teachers is close to that of experienced teachers. Improving the specialised knowledge base of teacher educators is seen to be one way to improve the quality of teacher training further. In addition, the remodelling of the teaching workforce and the current focus on personalised learning is expected to provide new opportunities for specialisation in the teaching workforce. This initiative requires teachers to have access to finely tuned pedagogical knowledge, diagnostic tools and subject specific specialist advice.
But the range of areas about which teacher educators and teachers need to be knowledgeable is vast. The British Education Index lists 5,000 new journal articles about educational issues each year (Sheffield and Saunders, 2004). So although staff will have specialist knowledge, keeping up-to-date is a challenging task. Some form of synthesis with good access to the evidence base is clearly necessary to provide policy makers and practitioners with high quality evidence on which to base decisions.
Evidence-based practice in education has been seen as a goal for decades (Stenhouse, 1975). The advent of web-based technologies provides at last a means of providing access to the recorded evidence base anywhere at anytime but the evidence base has first to be recorded and made available. Establishing and making available such an evidence base for teacher education has been one of the tasks assigned to the EPRD team.
Michael Fullan who has written so much on managing change and raising quality in education, suggested in 1993, that the lack of an evidence base was holding back the development of teaching as a profession: "The absence of a strong publicly stated knowledge base allows the misconception to continue that any smart person can teach." (1993, p.111). He challenged the teacher education community to identify the evidence underpinning their practice: "…a key obstacle in the evolution of teaching as a profession is an inadequately defined knowledge base about teaching and teacher education." (1993, p.113). It was around this period that web access to the Educational Resources Information Centre (ERIC) database was developed. ERIC does not attempt to provide an evidence base for practice, it simply provides access to many thousands of US based resources. In the UK, the EPPI-Centre which had been established in 1993 was funded from about 2001 to support reviews in education. In the US in 2002, the What Works Clearing House was established to provide an evidence base for practice in schools. The Campbell Collaboration was established n 1999.
Codifying the knowledge base – a strategy involving practitioners, subject groups and national agencies
For any professional group, the task of identifying the evidence base for practice is likely to be extremely demanding and complex. For teacher education, the task potentially required the identification of effective practice in the preparation of teachers for each school curriculum area as well as in priority areas such as the cross-curricular theme of citizenship, professional skills such as in behaviour management and general professional and pedagogic knowledge such as in the raising of achievement of pupils from diverse backgrounds. The identification of good practice in the structures, processes and management of initial teacher training programmes as well as in the philosophies underpinning models of training was also required.
Different forms of knowledge are identified in the literature as providing the foundation for effective teaching such as subject content knowledge, subject pedagogical knowledge, and general pedagogical knowledge and so on (Hoyle and John, 1995; Gilpin, 2003; Hirst, 1965). Consequently there is an ongoing professional debate about the extent to which professional practice can be recorded and whether it is even worth trying to do so (Slavin, 2004; Olson, 2004). Clearly any electronic evidence base will be limited in what it can provide.
The EPRD team had the freedom to develop new ways of working with providers of initial teacher training and had access to funds to support initiatives. In developing the strategy for codifying and disseminating the knowledge base for practice the team drew on the evidence base about effective forms of professional learning and transformation of practice as well as change and knowledge management theories (Eraut, 1994; Fielding et al, 2003; Rogers, 1995; Lave and Wegner, 1991; Wegner, 1998). Drawing on these theories and past experience of managing change, tenders for contracts to undertake work required contractors to include teacher education experts and to demonstrate that they had the capacity to build communities of practice where experts could work together to build new knowledge and to induct inexperienced teacher educators into the accumulated knowledge base in specific areas. Three types of communities or networks which have been developed are IPRNs (ITT Professional Resource Networks), focusing on professional thematic work; subject networks, focusing on subject specific work including the induction of new teacher educators into the knowledge base for the subject, and systematic review groups which are synthesising the evidence base for practice in priority areas. Research and Development awards to support innovation have been instigated to promote the thinking and experimentation which may lead to the good practices of the future. The TTA website provides more details (www.tta.gov.uk/eprd).
Much professional knowledge is provisional – it is continually being improved upon. The networking approach to the identification and dissemination of good practice via the web through the IPRNs and subject networks allows for this provisionality with these professional networks taking responsibility for updating the knowledge base they have published. Access to all of this work and other relevant work will shortly be available through a portal focused on materials supporting evidence-informed practice.
This provisionality of professional knowledge is also accommodated through the systematic reviewing process tested out by the Cochrane Collaboration. Whilst systematic reviews do provide access to quality assured summaries of what is the best state of knowledge in a particular area at a given point in time, they can also be easily updated – an advantage over traditional forms of literature review.
In 2002-03, The EPRD team funded nine pilot review groups through the EPPI centre
- to synthesise the evidence base in various priority areas;
- to test out the relevance of involvement in the process to the professional development of teacher educators; and
- to test the potential of review groups to provide the education system and policy makers with ‘expert knowledge banks’.
Following the successful outcomes of this pilot phase, eight review groups were funded to work over three years covering a range of priority areas. This means there is now a considerable number of educationalists in the UK who are familiar with the EPPI research synthesis approaches. This group is expected to be influential in shaping the planned development of a national strategy to bring coherence to the evidence base for educational practice.
Maximising the value of systematic reviews
Work is currently being undertaken by DFES and TTA in conjunction with review groups to develop different products from the reviews for different user groups. The basic product is a technical report which is a full report on the process and outcomes. Increasingly review groups are producing leaflets and user summaries of the key points. Seminars are held for DFES policy makers. There are also outcomes which go beyond the production of documents. For example, processes of systematic reviewing are, in some institutions, replacing the traditional forms of literature review that students undertake in MA, MPhil and PhD programmes and where teacher educators take part in review groups, they themselves can expect to become deeply knowledgeable in specific areas. As they understand the process, they can be expected to understand the strengths and weaknesses of other reviews and how the outcomes of other reviews can be used to inform their own practice. In addition, if in education, review groups are established which take long term responsibility for keeping reviews up to date, the staff involved will provide a long term resource of expert knowledge for policy makers as well as for the profession. Reviews are already being used to underpin policy developments and the work of some of the networks mentioned above.
How are systematic reviews rated within the Research Assessment Exercise (RAE) is a question academics ask. At the Campbell collaboration conference colleagues from different countries were asked how systematic review or research synthesis counted in their country’s equivalent of the English RAE. The answers varied. In some countries the work is highly rated, in others less highly rated. What does seem to be important for the academics concerned is to publish the outcomes of the systematic review through the traditional routes such as journals.
National structures supporting evidence-informed policy and practice
As other national educational agencies also have a remit to support the move to evidence-informed policy and practice, developing strategic alliances has been an important part of the EPRD team strategy. Examples of such alliances include collaboration on specific projects with
- DFES and NERF to agree guidelines with journal editors and authors about how to report studies so that systematic review groups can judge the reliability, validity and generalisability of the findings (Elbourne, in preparation);
- DFES, NERF, General Teaching Council in England (GTCE) and QCA to bring together stakeholders to develop a nationally coherent strategy for systematic reviewing in education involving colleagues from across the UK (Leask and Morris, in preparation);
- DFES, NERF, GTCE, Teaching and Learning Research Programme (TLRP) to develop interoperability of the evidence bases relevant to practice and policy-making (Sheffield and Saunders, 2004);
- DFES, GTCE, BECTA to achieve a fine grained national standard for metatagging educational resources which would be regularly updated as professional language develops;4
- subject networks to provide resources to keep the knowledge base for their area up to date;
- the National Educational Research Funders’ Forum, to provide information about the gaps in the educational knowledge base which systematic review groups and subject networks have identified so that resources can be more effectively focused. Briefing documents for the Funders’ Forum are in preparation.
The strategy to bring the evidence base for practice in teacher education together, to induct new teacher educators into the evidence base for practice (through face to face sessions) and to make the evidence base available via the web has required substantial involvement and contributions from all stakeholders. In this third year of the strategy, the focus is on publicising the existence of the resources and the further development of the web-interface to increase searchability and to bring together the different elements of the knowledge base as contributed by the different stakeholders. Every component of the strategy is subjected to independent evaluation and the evidence to date indicates that the level of use of the emerging web-based resources is beyond expectations.
In the education arena evidence-informed teaching and learning is increasingly an expectation of policy makers and external bodies charged with quality assuring standards. For professional groups in general, facing the need to move from intuitive professional practice to building processes for accumulating and accessing knowledge from research does not seem to be an optional activity.
However, the major challenge for the professions and for policy makers is finding cost effective ways to provide a credible, comprehensive, accessible and up-to-date evidence base for practice and policy-making. Medical professionals, through the Cochrane Collaboration in particular, have set high standards for other professions to follow.
Davies, T., Nutley, S., Smith, P. Eds. (2000) What Works, Evidence-based policy and practice in public services, Bristol: The Policy Press, ISBN: 1 86134191 1, 1 86134 192 X.
Elbourne, D. (in press) The paper providing advice for journal editors and authors is being developed collaboratively by NERF, TTA, the EPPI-Centre and the journal editors attending the 3rd DFES conference for journal editors. The advice will be available on the websites of all three organisations.
Eraut, M. (1994) Developing Professional Knoweldge and Competence, Falmer Press: Lewes.
Fielding, M., Eraut, M. Thorp, J., Cunningham, I., Jones, C., Craig, J. Horne, M. (2003) ‘Factors influencing the transfer of good practice: Literature Review’ Report for the Department for Education and Skills. London. University of Sussex and DEMOS.
Fullan, M. (1993) Change forces probing the depths of educational reform: The Falmer Press see chapter 6 Teacher Education: society’s missed opportunity.
Gilpin, A. (2003) ‘Professional work is an orchestration of different levels of knowledge: John, 2002’ contribution to the seminar: Meeting new teacher trainers’ needs – perspectives from research’ 13 May 2003.
Gough, D.A. (2003) ‘Systematic Research Synthesis’, In Pring and Thomas (Eds) forthcoming.
Hirst. P. (1965) ‘Liberal Education and the Nature of Knowledge’ republished in Peters, R.S., (ed.) The Philosophy of Education, Oxford: Oxford University Press.
HMI (2001-02) Annual Report of Her Majesty’s Chief Inspector of Schools: Standards and Quality in Education 2001-02, www.ofsted.gov.uk/publications.
Leask, M. and Morris, A. ( in preparation) A report on this initiative is being presented at the Fifth annual Campbell Collaboration Conference in 2005.
Hoyle, E. and John, P. (1995) Professional Knowledge and Professional Practice, London: Cassell.
Lave, J. and Wenger, E. (1991) ‘Situated Learning: Legitimate Peripheral Participation’, Cambridge, UK: Cambridge University press. ISBN 0 5214 2374 0.
Mosteller, F., Nave, B. and Mech, E. (2004) Why we need a structured abstract in education research Educational Researcher Vol.33, No.1 January-February 2004, pp 29-34, www.aera.net/pubs/er.
Olson, D. (2004) "The Triumph of Hope over Experience in the Search for "What Works: A response to Slavin", Educational Researcher Vol.33, No.1 January-February 2004, pp 24-26, www.aera.net/pubs/er.
Rogers Everett M. (1995, 4th Edition) Diffusion of innovation, New York: Free Press.
Sheffield, P. and Saunders, S. (2004) Scoping study related to development of a Web portal for users of education evidence bases, Report presented to the Teacher Training Agency – 10 May 2004, contract number: TTA32605P.
Slavin, R. (2004) "Education Research Can and Must Address" What Works" Questions", Educational Researcher Vol.33, No.1 January-February 2004, pp 27-28, www.aera.net/pubs/er.
Stenhouse, L. (1975) An Introduction to Curriculum Research and Development, London: Croom Helm.
Wenger, E. (1998) Communities of Practice: Learning, Meaning, and Identity, New York: Cambridge University Press.
The term ‘evidence-informed’ rather than ‘evidence-based’ is used as evidence from research is just one factor brought to bear in decisions about practice or in policy-making.
Funding to support the central administration comes from various sources (charities and governments buying services) but practitioners of all types, nurses, doctors and para-professionals take part in reviewing and updating the professional knowledge base as part of their professional work. As well as a secretariat in Oxford (funded through publications’ sales) there are sub-groups dealing with membership and publishing policy. There is a common misconception that this work is simply based on quantitative data. Reviews by health professionals include studies using the range of methodologies commonly used in education. The key message from the Cochrane collaboration is that professionals can work together to manage the establishing of a library of professional evidence and that there are various funding models which can support such initiatives.
Systematic review groups have reported that frequently articles do not have sufficient information about the research methodologies for an accurate assessment to be made about the quality of the claims made. Consequently, the EPRD team has commissioned the EPPI centre to develop guidelines for authors and editors to improve the quality of reporting of research. This advice was further developed with journal editors at the NERF Journal Editors Seminar in July 2004 and will be available late in 2004. Mosteller, Nave and Mech (2004) provide a US perspective on this problem.
The strategy outlined in this paper has been based on the outcomes of research from a number of research projects some of which are mentioned below. The works of the authors below have also been particularly influential.
American Educational Research Association (2003) Symposium: Consensus Panel on Teacher Education: an overviewhttp://www.aera.org.
Amidon, D., and Skyrme, S. (1997) Creating the Knowledge-Based Business, London: Business Intelligence Ltd, ISBN: 1 898 085 27 7.
Berieter, C. and Scardamalia, M. (1996) Schools as Knowledge Building Organisations. Today’s Children, tomorrow’s society: the developmental health and wealth of nations New York: Guildford.
Chang, R. and Kelly, P. (1995) Improvement Through Benchmarking, London: Kogan Page, ISBN: 0 74941655 6.
Cook, D.J., Greengold, N.J., Ellrodt A.G., and Weingarten, S.R., (1997) The relation between systematic reviews and practice guidelines, Annals of Internal Medicine 127:210-216.
Darling-Hammond, L., Wise, A., Klein, S. (1995) ‘A license to teach: building a profession for the 21st-century schools’, Summertown, Oxford: Westview Press.
Furlong, J. (2003) The knowledge debate in initial teacher education: an introduction, mimeo Teacher Training Agency ITT Support Group Oxford Conference 21-22 May 2003.
Glaser, B. and Strauss, A. (1967) The Discovery of Grounded Theory: Strategies for Qualitative Research, New York: Aldine.
Glass, G.V., (1976) Primary, secondary and meta-analysis of research, Educational Researcher 5: 3-8.
Goddard, D. and Leask, M. (1992) The Search for Quality: Planning for Improvement and Managing Change. London: Paul Chapman Publishing/Sage Publications.
Gough. D., and Elbourne, D., (2002) ‘Systematic Research Synthesis to Inform Policy, Practice and Democratic Debate’, Social Policy & Society 1:3, 225-236
Heightman, S. (2002) Trainee Teachers’ Understanding of what it is to teach for Pupil and Teacher Learning, PhD thesis, London: Brunel University.
Jones, S., Lang, G., Terrell, I., Thompson, K., Ramondt, L. (2001) ‘Establishing on-line communities for school leadership – an interim report’. Paper presented at the BERA conference 2001.
Leask, M. (2003a) Professional knowledge building in ITT: a 21st century strategy harnessing the potential of ICT, paper presented at Edinburgh: BERA Conference 2003.
Leask, M. (2003b) ‘Codifying the professional knowledge base of ITT trainers in England – the rationale for the TTA Effective Practices strategy’. Paper presented at Edinburgh: BERA Conference 2003.
Leask, M. (2000a) European Knowledge Centre Seminar Report 1 – ‘Classroom Practice and Educational Research: Using ICT to Build European Networks for Change’, Socrates Project 7115, mimeo, Bedford: De Montfort University.
Leask, M. (2000b) 'European Knowledge Centre within the European Schoolnet', Seminar 2 Report: 'Classroom practice and educational research: using ICT to build European networks for innovation and change', Socrates Project 71151, mimeo, Bedford: De Montfort University.
Leask, M. (1998) The development and embedding of new knowledge in a profession, De Montfort University, Bedford, PhD thesis.
Leask, M. (1989). ‘Teacher-Evaluators - the Missing Link’. Contribution to Hopkins D. Evaluation for Development, Milton Keynes: Open University Press, pp. 90-93.
Leask, M. (1988) Reliability and validity of teacher-evaluator approach to project evaluation MPhil thesis, Cambridge Institute of Education UK.
Leask, M. and White, C. (2003) ‘ITT professional resource networks: rationale and development’. Paper presented at BERA conference, Edinburgh, 2003.
Leask, M. and Younie, S. (2002a) ‘Communal Constructivist theory: ICT pedagogy & internationalisation of the Curriculum’, Journal for IT for Teacher Education, Vol. 10; Nos 1&2, pp117-134.
Murray, J. (2003) ‘Meeting new teacher trainers’ needs – perspectives from research’: Presentation. 13 May 2003. London: Teacher Training Agency
Oakley, A., (2003) ‘Research Evidence, Knowledge Management and Educational Practice: early lessons from a systematic approach’ London Review of Education, Vol.1, No.1.pp.21-34.
Rhine, S. (1998) ‘The Role of Research and Teachers' Knowledge Base in Professional Development’. Educational Researcher, pp. 27-31.
Schon, D. (1987) How professionals think in action, New York: Basic Books.
Simon, B. (1994) ‘Why no pedagogy in England?’ in Moon, B and Shelton Mayes, A. (1994). op cit.
US Department of Education Office of Postsecondary Education (2003) Meeting the Highly qualified Teachers challenge: The Secretary’s Second Annual Report on Teacher Quality, Washington: US Department of Education.
Von Krogh, G., Roos, J., Kleine, D., Eds. (1998) Knowing in Firms, Understanding, Managing and Measuring knowledge, London: Sage, ISBN: 0 7619 6013 9, 07619 6014 7.
Wilson, S., Floden, R., Ferrini-Mundy (2001) ‘Teacher Preparation Research: Current Knowledge, Gaps and Recommendations: A Research Report prepared for the US Department of Education by the Centre for the Study of Teaching and Policy in collaboration with Michigan State University.http://depts.washington.edu/ctpmail/pdfs/Teacher prep