Education-line Home Page

Taking the Evidence. Comparing a National Policy Review with more Conventional Policy Research

The Review of Initial Teacher Training Capacity in Wales

John Furlong, Hazel Hagger
University of Oxford

and Cerys Butcher
NIACE Dysgu Cymru

Paper presented at the British Educational Research Association Annual Conference, University of Warwick, 6-9 September 2006

'Good government is thinking government. And a good department is a thinking department. Rational thought is impossible without good evidence and to get this we need a revolution in the relations between government and the research community’ (Blunkett, 2000, para 62)

‘Policy making in a modern, complex, plural society like Britain is unwieldy and complex. It is often unscientific and irrational, whatever the claims of policy makers to the contrary’ (Ball 1990: 3)

1 Introduction - Evidence Based Policy Making in England and Wales

The mantra of ‘evidence based policy making’ has been one of the hall marks of British politics since the election of New Labour in 1997. It emerged ‘somewhere on the journey from Opposition to Government’ (Soulesbury 2001) and found its first full expression in the 1999 White Paper ‘Modernising Government’.

This growth of interest in evidence based policy making has in turn had a profound impact on the position of the social sciences in Britain. There has been an emergence of a ‘new utilitarianism’ (Young et al 2002) best expressed by the then Secretary of State for Education, David Blunkett, in his speech to the ESRC in 2000. ‘We need to be able to rely on social science and social scientists to tell us what works and why and what types of policy initiatives are likely to be most effective’ (Blunkett 2000). Others have referred to a ‘new social contract’ for social scientists; increased funding and opportunities for policy relevant research but matched by a growing emphasis on accountability, relevance and value for money (Demeritt 2000).

At the level of rhetoric at least, the evidence based policy movement is about challenging old strategies for policy formation, based as they were either on ideology or elites. This aspiration is well captured by Soulesbury (2001) of the ESRC UK Centre for Evidence based Policy and Practice, when he writes

‘…..there is something new in the air which gives both a fresh urgency and a new twist to the issues around evidence-based policy. To my mind the key factor is the shift in the nature of politics; the retreat from ideology, the dissolution of class-based party politics, the empowerment of consumers’ (9).

The rhetorical appeal of social science in providing part of the evidential base for policy is therefore that it indeed can be seen as scientific. It purports to stand outside the political process, giving policy advice based on rational evidence rather than ideology or sectional interest.

Significantly, the growing interest in the evidence based movement has not been confined to England. Wales, with its newly achieved democratic devolution has also embraced the role of evidence in the development of policy. In an early policy speech in December 2002, the First Minister, Rhodri Morgan, referred to a ‘new pluralism’ in policy making in Wales, with the Welsh Assembly seeking a broader engagement with civil society in Wales. As Daugherty (2003) suggests, this has been exemplified by the establishment within the civil service of a cross-departmental policy unit and by overtures to higher education in Wales to make research expertise available to government in Wales.

This commitment to the role of evidence in policy development has been particularly evident in the field of Education where there has been an emergence of an increasingly different set of educational policies from England. As G. Rees (2004) notes, those policies are based on

‘deep-seated social democratic virtues, albeit in changed and unfamiliar circumstances: equality of opportunity through universal provision; the necessity for the state’s role in ensuring this; the rights and obligations of citizenship; partnership between the central state, local education authorities and professional groups; and so on’ (p31)

As a result, to a greater extent than at any time previously, commentators are pointing to the emergence of a truly national education system in Wales (Egan and James 2003, Phillips 2003, G. Rees 2002; G. Rees 2004). And in the development of that distinctive agenda, the use of evidence, it is claimed, has been central. As Jane Davidson, Minister for Education and Lifelong Learning stated in the Introduction to her most recent policy document ‘Learning Country 2’ (NAfW 2006a).

‘Our policies will also continue to be evidence based. Over the last five years we have drawn both on practitioner expertise and high quality educational research. We commission independent evaluations of all our major policy initiatives. Leading research bodies have complimented us on this approach. (We) will continue to operate in this way.’

However, in one regard, the position of Wales is profoundly different from that of England and that is in relation to educational research capacity. As Furlong and White (2001) demonstrated early in the life of the new Assembly, research capacity in the field of education is extremely weak. Unlike England, or indeed Scotland, the Welsh Assembly Government’s aspiration for research based evidence is not matched by the capacity of higher education or indeed any other bodies within Wales to provide that evidence through conventional social science.

It is perhaps for this reason that rather than commissioning significant amounts of formal research, over the last five years, the Welsh Assembly Government has adopted a different strategy for assembling relevant evidence for its policy development. That strategy has been the commissioning of ‘Independent Reviews’. So far there have been a number of independent reviews in education in Wales: ‘Devolution of the Student Support System and Tuition Fee Regime in Wales’ (2005) chaired by Professor Teresa Rees, ‘Learning Pathways Through Statutory Assessment: Key Stages 2 and 3’ (2004) chaired by Richard Daugherty and the recent ‘Independent Review into Part Time Higher Education Study in Wales’ (2006) chaired by Dr Heather Graham. Each Review has thus been commissioned to focus on a key area of policy within the Welsh educational agenda. The focus of this paper is on one further independent Review: The Review of Initial Teacher Training Capacity in Wales, which was led by the current authors.

But if such Reviews are a key means by which research and other evidence is assembled on behalf of the Welsh Assembly Government, it is important to understand how they work. Each, it would seem has had a common structure. They have been chaired by an independent academic and have worked with a Support Group representing both lay and expert opinion from different relevant communities. Each has also demonstrated a strong commitment to the broadest possible consultation from across Wales. T. Rees (2002) reports having consulted over 1500 individuals and groups in the course of her first Review on student funding, while we took evidence from over 150 individuals and 50 organisations. Where the Reviews would seem to vary is the extent to which they have themselves commissioned original research (our Review concentrated more on assembling a range of existing evidence and opinion from within and outside Wales) and on the degree of support provided by seconded civil servants (our Review had relatively little such support because of staffing shortages in the relevant unit).

The processes involved are therefore rather different from conventional policy research. For one thing, what counts as ‘evidence’ is more broadly conceived. For example, our evidence included, among other things, highly technical reports from the Higher Education Funding Council for Wales (HEFCW), the General Teaching Council for Wales (GTCW) and Estyn (the inspectorate), and hand written letters from individual members of the public with a personal interest in teacher education. There is also an explicit expectation that Reviews will themselves move on from evidence to policy recommendations; in this regard, going further than conventional social scientific research.

But despite these differences, we can see that in many ways, the aim of such Reviews, particularly from the policy makers’ point of view, is to do the same thing as commissioned research. Their appeal is that they appear to be at one at the same time both democratic, consulting a wide cross section of opinion, and ‘independent’; in this sense they may be seen as sidestepping politics, moving policy making beyond ideology or sectional interest. This is how our study was presented in public. For example when first discussing our Final Report in the Education and Life Long Learning Committee, the Minister said

We are (today) discussing an independent report, on which we are now taking evidence so that the Assembly Government can give a response. We could not possibly have given a response prior to the publication of an independent report. (NAfW 2006b)

But is this actually how these Reviews work; are they really ‘independent’; are they really rational and scientific, standing outside politics? So far we have had two retrospective accounts of these Welsh Reviews –on the first Rees Review (T. Rees, 2002 and Stroud 2002) and one on the Assessment Review (Daugherty 2004). Both of these retrospective accounts present themselves as success stories and as being entirely rationally based. As T. Rees, (2002) states

‘In many senses, the investigation resembled a policy-oriented research project: we identified the questions, we collected evidence, we analysed it, we discussed possible interpretations and their implications with key actors, we came to some conclusions and we made our recommendations’

The only difference from conventional research that T. Rees notes is the great enthusiasm with which people gave evidence!

Both Rees and Daugherty therefore continue the ‘story’ of rational evidence based policy making. And while we could indeed tell a similar story, in this paper we want to do something different; we want to highlight some of the complex political processes that went on in and around our Review; processes which suggest that it was anything but outside the political arena. In this sense we would like our retrospective account to contribute to the growing literature on how research and policy processes actually interrelate; we want to contribute to a greater understanding of what Shulock (1999) calls the ‘paradox of policy analysis’, a paradox arising from ‘a mismatch between notions of how the policy process should work and its actual messy, uncertain, unstable and essentially political realities’. (p218)

Our experience is that the Review process, just like any other strategy for assembling evidence, is deeply involved in that paradox . Facing up to those contradictions and tensions in the process is, we believe, an important step in assessing the real value of incorporating the Review process within a modern democratic state.

2 The Review Process

There are many different stories that could be told about the way in which our work was ‘political’; the Review found itself inserted into many different political discourses (Foucault 1984; Olssen et al 2004). Some we experienced at a personal level (for example a chance meeting in Tesco’s with a senior civil servant who wanted to discuss our Interim Report; informal discussions with key informants in the pub and in meetings about other things). Politics at this level is perhaps the product of Wales being a small country, with many cross cutting personal and professional networks. But we were also aware that the Review was a very small part of many larger political discourses – the relationship between the Minister and the Higher Education sector in Wales; the relationship between the Higher Education Funding Council and the National Assembly; the relationship between Wales and key bodies in England such as the Department for Education and Skills (DfES) and the Training and Development Agency for Schools (TDA). Of the many stories that could be told, we want to concentrate here on just three that illustrate different ‘moments’ in the Review process. They are the setting up of the Review; the way in which we attempted to engage the Higher Education sector in Wales in the Review process; and finally our recommendations and the responses to them.

2.1 Setting up the Review.

There are two key issues that we wish to highlight in the setting up of the Review. The first concerns the Terms of Reference and the second concerns the membership of the Support Panel. Both of these were highly significant in shaping the way we worked and perhaps in the final outcomes of our Review.

The idea that there should be a Review of initial teacher education had been discussed for a number of years amongst the teacher education community in Wales; certainly since the establishment of democratic devolution in 1999. Estyn, in successive reports, had expressed concern that although most higher education based provision was ‘good’, only a very small proportion was ‘excellent’; they had also expressed concern about the variability in school based provision. (Estyn 2003; 2005). Some schools and LEAs that had begun experimenting with employment based training though the Graduate Teacher Programme were keen to expand that system from its current limited number of 50 places per year. And the Welsh Language school system had for many years expressed concern about the numbers of teachers willing and able to teach through the medium of Welsh, with particular concern being expressed about Welsh second language teaching in primary schools.

Perhaps the largest number of concerns however, were expressed by those in Higher Education. In the early 1990’s, initial teacher education in Wales , along with that in England, had been swept up into the radical reforms of the day. A system of teacher training (based on a model of government imposed ‘standards’ or competencies, to be acquired in large part through practical school experiences, combined with a strong inspection regime, with the threat of course closure for courses that failed to achieve the appropriate standard) had been imposed from London with no consultation. While there were and are some important differences from the English model in the management of the system ( for example, the absence of a TDA or its equivalent; a slightly ‘lighter touch’ inspection regime; more regular contact between the key players in the system – the inspectorate, the funding council, Welsh Assembly Government officers and representatives of HEIs) the underlying model of teacher training itself was and is the same system that was imposed by the Westminster Conservative government in 1992. With the emergence of a different agenda for many aspects of educational policy in Wales following democratic devolution, there was an appetite and a hope amongst university based teacher educators for a fundamental review of the system. From their point of view there were a number of concerns that might have merited review: the apparent under funding of the system which in some institutions appeared to be resulting in the progressive casualisation of staffing; the difficulties of establishing effective school partnerships in some parts of Wales; and perhaps most fundamentally of all, the continuing lack of recognition of a distinctive role for universities in teacher education rather than simply in teacher training.

However, in the end, it was none of these concerns that actually precipitated the establishment of the Review; instead it was the issue of the numbers of teachers being trained, a particular concern for the Welsh Assembly Government itself.

For several years prior to the establishment of the Review there had been growing concern about the apparent difficulty that some newly qualified teachers in Wales, especially primary teachers and especially those who had trained in the south of the country, were having in finding a post. Throughout the early 2000’s there were regular letters and comments in the Welsh press about the difficulties many newly qualified teachers were facing in finding permanent posts; there were reports of newly qualified teachers not being able to complete their statutory induction which at the time had to be undertaken in a full time post within one year of qualifying. In response to these concerns, the NUT was running a campaign to encourage Wales to establish an entitlement for all newly qualified teachers to a one year placement in order to complete their induction, as had been introduced in Scotland. The initial response of the Welsh Assembly Government was to change the regulations, allowing newly qualified teachers 5 years rather than one in which to complete their induction. HEIs were also given advance warning in 2004 that there would be a 5% cut in primary numbers from September 2006. Despite these moves, concerns about the oversupply of primary school teachers remained. And when combined with demographic forecasts for the future reduction in the school population, it was becoming apparent that in the future, the position might get worse rather than better. From the point off view of the Welsh Assembly Government, this was a nettle that had to be grasped.

There were therefore many competing expectations and aspirations about what questions a Review of Initial Teacher Training Provision in Wales might address. Initially, discussion with the Minister and her Civil Servants suggested that the Review should confine itself entirely to the issue of numbers of teachers needed; conversations implied that the Review should be primarily statistical in nature with clear recommendations about percentages and location of cuts needed over the next 5 to 10 years. Behind this question was the issue of whether or not it was technically possible or indeed sensible, for Wales only to focus on producing its own teachers. Our response was that addressing the issue of numbers was clearly very important but that simple percentage cuts were unlikely to be successful in that they could very easily destabilise the system; the complexities of staffing both primary and secondary courses had to be taken into account as well as the implications of any cuts in numbers for other forms of provision, such as Continuing Professional Development (CPD) and research. We were also keen to look in more detail at the substance of current provision and suggested that we address the issue of ‘quality’. This proposal was warmly accepted with two provisos. Firstly, that we should not look at the issue of ‘partnerships’ as HEFCW had recently commissioned its own Review on this matter. Secondly, that we should not address the issue of ‘content’. Following a period of consultation, officers were only now completing their latest revisions of the revised ‘standards’ for ITT and there was clearly no appetite (or indeed capacity) to entertain further discussion. The opportunities for asking many of the more fundamental questions proposed by those in higher education were therefore effectively curtailed. However, further discussion suggested that we might also examine the relationship between initial training and induction; there was also agreement that we should address the issue of the strategic management of the system.

At a certain point during these early discussions, it became apparent that the likely costs of the Review meant that it would have to undergo a procurement and competitive tendering exercise. At that point therefore, we (rightly) had no more involvement in discussing terms of reference, which after a further month were published with invitations to tender. The Oxford team successfully bid for the contract. The final list of questions the Review was required to address was as follows

1. The needs of schools in Wales

How can the need for ITT provision be calculated to achieve a better match between the supply of NQTs and the future needs of schools in Wales?

2. Course provision for Welsh needs

To what extent could or should Wales aim to provide as far as possible just for its own needs for NQTs and, as far as practicable, avoid producing NQTs from ITT in Wales unlikely to work here?

3. Adjusting provision in Wales

In the light of answers to questions 1 and 2 above, how might ITT provision be adjusted in order to ensure a supply of NQTs of the highest quality, appropriate to the needs of schools in Wales?

4. Maintaining and improving quality in ITT

What sort of high quality ITT provision should Wales aspire to develop over the next 5 to 10 years in order to continue improving standards of teaching and pupil achievement?

5. ITT and Induction/EPD

With the introduction of Induction and Early Professional Development for NQTs, is there a case for examining the balance between ITT and what can be done through Induction and EPD and how they might be better integrated?

6. Strategic planning

How might arrangements for the strategic planning of ITT provision in Wales be developed that are responsive to the needs of schools in Wales?

The Review should not consider the following issues:

a) the content and delivery of ITT in terms of meeting the standards for Qualified Teacher Status and the minimum requirements for IYY courses ( the Assembly Government is currently addressing these in a Review of Welsh Office Circular 13/98).

b)The marketing of teaching as a career, including teaching incentives

(http://www.learning.wales.gov.uk/scripts/fe/news_details.asp?NewsID=1529)

The second significant issue in the setting up of the Review was the selection of the Support Group. Each of the Independent Reviews established in Wales in recent years has had such a group, drawn from a range of interest groups and communities from across Wales. According to the press release issued in May 2004, the purpose of our Support Group was to ‘-test objectively the Review’s findings and to provide a source of expert advice and information, and to hear evidence from expert witnesses and stakeholder groups’. (NAfW 2005).

Our experience of working with our Support Group was that they were indeed an invaluable resource; they functioned both as a source of expert advice and information and played a significant role in the shaping of our final recommendations. Though the Final Report was our own, we were keen to ensure that each of our recommendations had been carefully discussed and commanded the support of the Group as a whole. Following the publication of our Review, Support Group members were also important in acting as ‘champions’ (Selby-Smith 2000) for the report within a range of different communities across Wales.

In the early stages of discussions about the Review we were consulted on who should be a member of the Support Group; we made a number of suggestions some of which were accepted. However, one key group that it was considered inappropriate to include was representation from the Higher Education Institutions (HEIs) themselves. Our reasoning for their inclusion was that given that our proposals for reform were likely to have more significance for higher education than for any other group in Wales, it was vitally important to draw them into the Review process as fully as possible. The Welsh Assembly Government took a different view, suggesting that as HEIs were the main providers, they should not be asked in effect ‘to review themselves’. In our initial proposals we also identified a number of individuals who were senior members of the Welsh language sector. Unfortunately, in the individuals finally selected by the Government for membership of the Support Group, there were no senior members of that community.

We would suggest that these two ‘exclusions’, from HEIs and from the Welsh language sector, had a major impact on the way we had to work (put simply we had to work extra hard to try to establish our credibility with these groups and probably only achieved partial success). In particular, the lack of representation from the Welsh language sector may have added an additional complexity to our relationship with HEIs, organisations and individuals in the predominantly Welsh speaking north and west Wales. It may also be that these exclusions had a significant impact on the way in which our Review was responded to once it was published, in that it had no insiders to act as ‘champions’ for it from these two key communities. These are issues that we discuss in more detail below.

2.2 Taking the evidence – listening to Higher Education

As we have already indicated, our approach to collecting evidence for our Review was broadly based. Our aim was to achieve a report that was both technically authoritative and could be seen to have taken into account the views of as many individuals and organisations as possible with an interest in the future of teacher education in Wales. Our first task was therefore to construct a website on which we published the 6 key questions from our Terms of Reference and a letter encouraging anyone with an interest in initial teacher education to write to us. In addition, we wrote to over 100 organisations setting out our Review questions and asking them to write to us. As a result of these requests we received 51 written responses ( 6 from individuals, 4 from LEAs; 6 from schools; 17 from other educational organisations; 2 from religious bodies; one from a section within the Welsh Assembly Government; 8 from HEI education departments, 2 from Vice Chancellors; 3 from trade unions; and 2 from other organisations),

In addition to written evidence, the Review Team and the Support Group met to take evidence for 7 whole days in Cardiff. During those meetings, representatives from 24 different organisations gave oral and written evidence and engaged in discussion with panel members. The Review team also conducted a number of individual meetings with representatives from lead bodies, such as DfES, HEFCW, TDA and the General Teaching Council of Wales (GTCW), and consulted over 80 different relevant policy documents. As we have already indicated, the character of this evidence varied widely, from personal, hand written letters, to highly technical reports and government policy documents.

However, although we were keen to receive comments from as broad a range of interested parties as possible, we were not even handed in our collection of evidence. From the earliest stages of our Review we were keen to give particular attention to the 7 HEIs that were and are the main providers of initial teacher education in Wales. During the course of the Review, members of the team therefore visited each of the 7 institutions, spending a day in each of them. During those visits we asked to meet with senior members of the teacher education team, representatives from partnership schools and representatives who could speak on behalf of the HEI itself. As a result we therefore had the opportunity to discuss our Review questions with over 60 colleagues including Vice Chancellors, head teachers, mentors, course leaders and lecturers. In addition we convened two one day seminars – one at the beginning of our Review and one at the end – where we invited two representatives from each HEI to contribute to a discussion of our Review questions, to examine our evidence, and to debate our emerging recommendations.

Why did we take this ‘selective’ approach to the gathering of evidence, in what was, after all, supposed to be an independent, public Review? Why did we make so much effort to focus on the needs and interests of Higher Education, the current main providers? Our response is three fold. Firstly, we were acutely aware that as outsiders to teacher education in Wales, we might not understand the many institutional, regional and linguistic complexities involved in the day to day running of teacher education courses in Wales today. Put simply, we had a great deal to learn about the detail of current provision before we could make recommendations for the future. We also recognised a political reality. If our recommendations were to stand any chance of gaining support, we had to ensure, as far as possible, that we were seen as basing our analysis on a thorough understanding of current provision in Wales. As we have already indicated, the fact that the Welsh Assembly Government chose not to include representatives from the HEIs on our Support Group made this ‘legitimisation’ process even more important than it would otherwise have been. Finally we must acknowledge that as a team we were far from neutral on the contribution of Higher Education to teacher education. Both Furlong and Hagger in particular have written extensively on teacher education over the last two decades (Furlong et al 1988; Furlong and Maynard 1995 Furlong et al 2000; Hagger et al 1993; Hagger 1997; Hagger and McIntyre 2006 ); each in their own right is well known for their commitment to the role of both schools and HEIs in the provision of high quality professional education. Given this background, it was inevitable that as a team, we took the contribution of HEIs in Wales extremely seriously. Establishing our own legitimacy was therefore both a personal as well as a professional imperative for us.

This ‘lack of neutrality’ affected our working in other ways too. From the very beginning we were acutely aware of the possible negative implications of simple percentage number cuts on HEIs. In our ongoing discussions with representatives of the Welsh Assembly Government and HEFCW we were therefore keen to ensure, that whatever the change in actual student numbers, the institutional unit should remain largely stable; an argument that was accepted. Because of our other existing professional commitments (see for example Furlong and White 2001), we were also keen, in the longer term, to see the establishment strong schools of education in Wales, capable of providing leadership in terms of CPD and research as well as initial teacher education. It is clear that this existing professional commitment also affected how we went about our work and some of the recommendations that we made. In the next section we look at these recommendations and the response to them in more detail.

2.3 The recommendations and the response.

In our final Report, we addressed each of the 6 key questions we were required to consider; in each case we reviewed the evidence available and then made our recommendations. In total the Report contains 36 recommendations. There is not the space in this current paper to address each of these; rather we focus on just three, closely interrelated recommendations that were particularly significant – our proposals for a significantly reduced number of trainee places; our proposal to move to an entirely post graduate entry route into teaching and the establishment of an alternative ‘pre-professional’ undergraduate degree; and our proposal for the establishment of three main schools of education in Wales. In each case we want to explore the political dimension of our recommendations and the response to them.

Without doubt the most controversial finding arising from our evidence was the current ‘employment rate’ amongst newly qualified teachers in Wales. Over the last 4 years, GTCW had, for the first time, been collecting and publishing robust evidence on the numbers of newly qualified teachers taking up permanent posts in Wales. With this statistical evidence it was then relatively straightforward for us to calculate the primary and secondary ‘employment rates’ for each of the last four years; i.e. the numbers of posts available for newly qualified teachers as a proportion of the numbers of training places available in any one year. Our evidence showed that for newly qualified primary teachers, the employment rate over the last four years had never been higher than 41% and in 2003/4 it had dropped to 28%. For newly qualified secondary teachers, the employment rate was on average 56% over the last four years. Our analysis also showed that the numbers of teachers trained in Wales seeking employment elsewhere in the UK was at the very most 20% of those trained, a number that was easily matched by those training elsewhere and seeking to work in Wales. Moreover, cross border flow was reducing over time, probably as a result of the introduction of higher education fees. Our evidence therefore showed, for the first time, the degree of current overproduction of teachers in Wales. When, to these figures, we added the substantial downward projection in demand for teachers that is likely to arise because of demographic trends over the next 5 – 10 years, the size of the challenge facing the sector becomes apparent.

With these challenging figures in mind we therefore made a number of interrelated proposals for the restructuring of teacher education in Wales. For primary training, we suggested that numbers needed to be reduced by up to 50% over the next five years; this, we argued, should be achieved by moving to entirely postgraduate provision. Given the current, let alone future demand for newly qualified teachers in Wales, current numbers of primary PGCE students were already more than sufficient to meet demand. However, for that to happen we argued that the capacity of HEIs to deliver these programmes would need to be protected; in our view that would be impossible if departments were only able to offer the PGCE. We therefore argued that in managing the reduced numbers, HEFCW needed to ensure that the unit of resource going to each institution should remain largely the same. We also recognised that current undergraduate courses provided a vitally important access route into higher education for some disadvantaged groups and was a very important resource in terms of access to Welsh language provision. At the same time as recommending the move to a post graduate only route into teaching we also therefore recommended the development of a new form of undergraduate provision – a new degree to be offered by departments of education, that would serve the needs of the same group of vocationally oriented young people who currently applied for the BA(Ed) degree. These courses, ‘pre-professional degrees’, we suggested, should lead to a range of different programmes of postgraduate professional training (including the PGCE) for careers involved in working with young people. We also suggested that each School of Education offering this new degree should offer some provision through the medium of Welsh.

For secondary provision, we recommended a reduction of 25% over the next five years. However, here we suggested that adjusting provision was more complex in that although there was currently significant overprovision, the challenge of maintaining viable courses across the sector, in both English and Welsh, was extremely difficult. In principle, healthy provision, we suggested, was needed in all regions of Wales (north, south west and south east), in all subjects and in both languages. However, a more detailed analysis, we suggested, could not take place until the numbers of institutions offering teacher education in Wales was addressed.

Our evidence suggested that the current financial position of many of the 7 main providers of teacher education in Wales was not strong. For the future, we argued, Wales needed a sector that was financially strong, provided a range of different sorts of provision and was able to respond effectively to the changing needs of Wales including the growing demand for teachers able to teach through the medium of Welsh. Wales, we argued, also needed a sector that had a proper research base if higher education was to take its proper role in helping to achieve the many changes that will be needed in the schools of Wales in the future.

We therefore argued that Wales needed to develop three main Schools of Education. These should be geographically distributed – North and Central, South West, South East. Each school should in principle offer the full range of provision need to meet the future needs of Wales – i.e. Primary PGCE, Secondary PGCE – both in Welsh and English plus the new undergraduate Pre Professional degree. Each School should in addition have a CPD portfolio and research capacity. We recommended that funds should be made available from HEFCW to support such a reconfiguration. Over the next 5 years, once numbers were brought more into line with actual demand for new teachers, we also recommended that the Welsh Assembly Government should seriously consider establishing a guaranteed induction placement for all newly qualified teachers.

These then were the most challenging of our recommendations, designed to respond to the significant predicted down turn in demand for teachers over the next 5-10 years in Wales, but also designed to protect and indeed strengthen provision in the longer term. How were they responded to by different communities in Wales and most particularly by the Welsh Assembly Government?

Our final Report was formally discussed by the Education and Life Long Learning Committee on the 26th of January 2006. Other bodies were also asked to give evidence to the Committee on their response to the Report. Many were highly positive. The Chief Executive of the GTCW welcomed ‘this comprehensive, well thought through and clearly argued report’ (NAfW 2006b para 105); Her Majesty’s Chief Inspector for Schools in Wales also welcomed the report, stating that ‘it offers a bold and timely vision for the future of initial teacher education and training in Wales. ..(and) addresses the major challenges facing us directly and vigorously, and suggests ways forward’ (NAfW 2006b para 120); the Chief Executive of HEFCW stated ‘we very much welcome the Furlong report’ (NAfW 2006b para 182) advising that the proposals for change were broadly achievable within the timescale suggested. Finally, the deputy chair of the Association of Directors of Education, Wales, (ADEW), stated that his members also supported the main recommendations of the report.

Others however were more critical, particularly Higher Education Wales (the Vice Chancellor’s organisation). Their representative stated that they had ‘consistently opposed one of the fundamental assumptions of this report, namely that the sole purpose of initial teacher training in Wales should be to meet the immediate needs of Welsh schools’ (NAfW 2006b para 201). They also stated that they were ‘surprised by the fact that one of the most popular degree courses in Wales, which is highly valued by many schools, and which contributes significantly to higher education targets in widening participation and, particularly, to the Welsh-medium targets. …is going to be abolished’ (NAfW 2006b para 202). However, he also indicated that if the proposals were accepted, the HE sector would be willing to work with them as long as the unit of resource were protected. The Universities Council for the Education of Teachers Cymru (UCET Cymru), on behalf of the higher education based teacher educators took a similar line.

However, probably the most significant criticisms came in written evidence received from the Steering Group for Welsh Medium Provision in Higher Education. In a letter of the 10th of March the chair of that group stated that they were particularly concerned about the impact that ending the undergraduate BA Education degree would have on the supply of Welsh teachers. ‘Members considered it highly unlikely that the various pre-professional degrees proposed would be able to match the ITT provision both in terms of the number of Welsh medium enrolments and the volume of Welsh medium provision delivered’. They also expressed concern that as undergraduate ITT provision accounted for some 31% of all Welsh medium provision within Higher Education, abandoning the degree could have a major adverse impact on current targets for overall Welsh medium undergraduate provision in Wales.

On the 29th March 2006, the Minister outlined her own response to the Report to the ELL committee. In making her response she indicated that she had taken into account various stakeholders’ reactions, including the comments made in Committee by UCET Cymru, the GTCW and Higher Education Wales.

In terms of numbers of teachers needed, the Minister accepted, for the purposes of planning, the proposed reductions of ITT student places – by 50% for primary and by 25% for secondary – to be achieved over the next 5 years. ‘However, these figures will be very much in outline only and may be subject to noticeable changes depending on the outcomes of our examination of data collection improvements’. (NAfW 2006c para 3). In addition, she also announced a short term exercise to be carried out by the Statistical Directorate ‘in order to establish confidence in the data’ Education and Lifelong Learning committee NAfW 2006c Annex A para 2). In order to support these reductions, she announced that ‘all the necessary resources from the current amount of ITT provision (would be) retained to run those courses in a sustainable manner’ (NAfW 2006d, para 5a)

Given the reduction in ITT numbers envisaged, the Minister accepted the case for reconfiguring ITT provision based on fewer individual providers than at present. However she stated that

I am not presently wedded to one particular model for this. My over-riding concern is to ensure sustainable and cost-effective delivery of ITT and ITT-related work in the light of the projected reduction in course numbers. I am therefore remitting HEFCW to produce for the Welsh Assembly Government a detailed plan (including the likely cost and timing) for a reconfiguration of HEI ITT providers in Wales (except for the Open University). The report should consider options for the best organisation of providers and the range of courses in the revised structure. (NAfW 2006c para14.2)

Finally, on the issue of undergraduate provision, the Minister concluded that a measure of undergraduate provision should be retained. Here she argued that:

A number of concerns have been raised, however, about the possibility of ending undergraduate ITT. These include that it would result in reduced diversity of intake; could shut off direct routes into teaching for some sectors of the community; might eliminate a source of supply well suited to primary teaching; and would severely damage Welsh medium intake to ITT. It should also be noted that ITT provision also contributes heavily to current Welsh medium numbers in higher education generally. Education and Lifelong Learning committee (NAfW 2006c para 15.2)

In short, following consultation both in front of the ELL Committee, and presumably outside it, the Minister accepted the broad thrust of our recommendations while rejecting some of our more specific proposals. She particularly rejected our proposal to protect provision by linking reductions in ITT numbers with a corresponding expansion of other forms of undergraduate provision. On the issue of the reconfiguring of the numbers of institutions, she declined to make a decision, passing to the Funding Council the task of drawing up a detailed plan.

4. Discussion

The description of these three ‘moments’ in the development of our Review demonstrates that, as a process, it was anything but ‘neutrally scientific’; politics was involved at every turn. Politics was involved in the setting up of the Review and the questions that were asked; it was involved in how we went about collecting evidence; it was involved in our interpretation of that evidence; and it was involved in how the Report was responded to afterwards. And as we noted in the introduction, the Review was also embedded in many other different political discourses; these three have merely been illustrations.

But should we be surprised at such an observation? And perhaps more importantly, does the fact that the Review was indeed part of a political process in this way, undermine its claims to neutrality and objectivity; does it undermine its claim to contribute to the democratic process? If the aspiration of ‘evidence based policy making’ is to bring some critical distance to the advice that is offered to policy makers, then does our analysis undermine that aspiration? Have we really moved beyond basing policy advice on ideology and sectional interest?

Our first response is that ‘no’ we should not be surprised. Any careful reflection on how a research project is carried out will reveal that decisions have to be made at every turn: in deciding what questions to ask, what evidence to collect and how to interpret that evidence. Moreover, the growing body of literature on what makes effective policy research would highlight precisely the same range of political issues that we confronted in our Review.

Buxton and Hanney, (1994) for example, talk about the importance of ‘policy maker involvement and brokerage, as key factors in enhancing utilisation’, while Selby Smith (2000) talks about the importance of ‘linkages’, between researchers, practitioners and policy makers if research is to have any impact. Our Review would largely corroborate Selby Smith’s comments when he suggests:

To stress the concept of linkages is to be concerned with facilitating the establishment of multiple areas of collaboration between researchers, policy makers and practitioners, given the multiple pathways through which research can influence policy and practice.

He goes on:

Contact between the two domains, not only at the close of a study, but also before and during its conduct, can strongly affect impact…..The stronger the linkages, the clearer the pathways of influence are likely to be and the greater the likelihood of uptake for new ideas.

Our experience would suggest that it is the concept of ‘linkages’ that marks the key difference between a Review and a more conventional academic research project. Throughout the Review process we were in constant dialogue with policy makers and practitioners within many different policy communities. But significantly, during those contacts we frequently found ourselves not only listening to representatives for the ‘evidence’ they could give, we also found ourselves trying to understand the political realities of their world and ‘sounding out’ possible policy recommendations. Unlike a conventional research project, the processes of gathering ‘evidence’ and policy formation were blurred. While such blurring may be considered by some as undermining the objectivity of our work, we would suggest that in reality it allowed us to develop more realistic and carefully grounded recommendations.

We would also agree with Selby Smith when he suggests that linkages are important in ensuring that researchers address the ‘right’ questions (Selby Smith 2000 p…). As we have described, in deciding what questions to address, we were able to engage in some debate and to a degree to shape the agenda for the Review. However, through that process, it also became clear that the Welsh Assembly Government were only ready to ‘hear’ particular issues. Whatever our personal interests, whatever the aspirations of others in the teacher education community in Wales, the overriding issue for the policy makers concerned the numbers of teachers being trained. If our Review was to be taken seriously, it was essential that we too made this the central issue to be considered. Other questions, however important to us or to other interest groups, had to take second place. We would not apologise for ‘shaping’ our questions in this way; doing so was part of the political reality that we faced.

Other studies, such as those of Jones and Trembath ( quoted in Selby Smith, 2000) have also noted the importance of ‘champions’ to support research, and to encourage its use and develop linkages within and outside the organisation. Certainly our experience was that through our Support Group we were able to recruit a number of key ‘Champions’ for our Review ( senior representatives from the Inspectorate, the General Teaching Council, Local Education Authorities and the Higher Education Funding Council) who were vitally important in brokering support within their own policy communities. During the process of collecting evidence, our work with Higher Education Institutions and with the Welsh Language Community also gained us a number of key individual supporters. Where we were less successful was in gaining formal endorsement from these two key communities. As a result, there were only limited opportunities for us to engage with the leaders of the Higher Education and Welsh Language Communities. Had this been possible, then it is our view that their opposition to some of our proposals might well have been allayed.

In understanding the Minister’s response to our Review, we might also draw on the work of Halpern of the No 10 policy unit (Halpern 2003) . who argues that for evidence to be ‘heard’ it has to be within a policy maker’s ‘Zone of Proximal Development’. It was clear that our evidence on the ‘employment rate’ amongst newly qualified teachers was a surprise to everyone; indeed it was a surprise to us and we and the General Teaching Council carefully checked the figures again and again. The degree of over capacity within the system was clearly outside the ZPD for policy makers and practitioners at all levels of the system though to date no one has successfully challenged our figures. It is therefore not surprising that the Minister’s response was cautious - commissioning of additional work in order to establish ‘confidence in the data’ while at the same time broadly accepting of our arguments.

Finally, although not all of our recommendations were taken up by the Government, we might take comfort from those who remind us that in the policy field single research studies seldom have a one to one impact. As Brown (1991) puts it:

On a good day, ideas (information) may gain a hearing amid the swirl of political considerations, but it must be a very good and rare day indeed when policy makers take their cues mainly from scientific knowledge about the state of the world they hope to change or protect (Brown, 1991p….).

As Weiss’ (1998) long term analysis of the policy process has demonstrated, research studies are much more likely to contribute to a longer term climate of opinion than to immediate change; what they can do is help to set out the terms of a debate. And it is clear than in the field of teacher education in Wales, the debate is indeed continuing (See for example Western Mail 1st August, 2006)

4. Conclusion

In the opening section of this paper we referred to what Shulock (1999) called the ‘paradox of policy analysis’, which arises from ‘a mismatch between notions of how the policy process should work and its actual messy, uncertain, unstable and essentially political realities’ (p218). What we have tried to demonstrate is that our Review was very much part of that messy, uncertain and essentially political world. Indeed, given its high profile, how could it have been otherwise? There was far too much at stake for all of the parties involved for it to be outside the political process. But does that mean that all we have is political reality? If it is naïve to understand evidence based policy making as a straightforward, technical rationalist process, is the alternative to agree with Ball, whom we quoted at the beginning of this paper, when he argues that policy making is frequently ‘unscientific and irrational, whatever the claims of policy makers to the contrary’?

Our experience would suggest that neither of these two extremes is correct. Certainly the gathering of evidence was anything but a technical rationalist process; but we would suggest that the gathering of evidence never is or can be. All research is partial and policy based research is no different. Moreover, to recognise the partiality of the process does not mean that we did not do our best to bring together such evidence as was available on the questions we were asked to address. Much of the evidence we assembled, even when it was already in the public domain, had not been brought together in that form; it was therefore not available to policy makers. Through the Review process, we were able to broaden and deepen the range of information that was available to policy makers in considering the future of teacher education in Wales.

Moreover, we would also suggest that the Review process did indeed present an opportunity to democratise policy making – at least to a degree. It was an opportunity for a wider range of ‘voices’ to be heard in the policy debate than would otherwise have been the case. Indeed, given the very broad notion of what counted as ‘evidence’, we might argue that the Review process was more ‘open’, more ‘democratic’ than many conventional research projects with their predetermined ‘scientific’ notions of what counts as evidence in the first place.

In his analysis of the first term of the Welsh Assembly Government, G Rees (2004) argues that since democratic devolution, many established power groups have continued to exert influence on educational policy in Wales – civil servants (especially the Inspectorate), professional organisations and trade unions and local education authorities. These groups were also highly influential in our Review. Nevertheless, in how we went about our work and in what we counted as evidence, it was clear that our Review, like the others that have been conducted in Wales, for all their political realities, did provide an opportunity for a broader range of voices to be heard in the political process. In the longer term, whether that opportunity for greater democracy is taken of course depends on how such Reviews are responded to by the Welsh Assembly Government. In our case, the jury is still out.

References

Ball, S. (1990) Politics and policy making in education: explorations in policy sociology. London: Routledge.

Blunkett, D. (2000) ‘Influence or Irrelevance: can social science improve government?’ Speech, February 2000, reprinted in Research Intelligence, 71, 12-21.

Brown, L. (1991) ‘Knowledge and Power: Health Services Research as a Political Resource’ in E.Ginzberg (ed) Health Services Research: Key to Health Policy. Harvard University Press, Massachusetts.

Buxton, M. and Hanney, S. (1994) Assessing Payback from Department of Health Research and Development: Preliminary Report, Vol.1: The Main Report, HERG Research Report No 19. Health Economics Research Group, Brunel University, Uxbridge.

Daugherty, R. (2003) 'Education Policy and Education Research in Wales: Changing relationships?' The Welsh Journal of Education, 12 (2), 83-98.

Daugherty, R. et.al. (2004) Learning Pathways through Statutory Assessment: Final Report of the Daugherty Assessment Review Group. Cardiff: Welsh Assembly Government.

Daugherty, R. & Jones, G. (2002) 'Research Note: The Learning Country', The Welsh Journal of Education, 11 (1), 107-113.

Daugherty, R., Phillips, R. & Rees, G. (eds) (2000a) Education Policy Making in Wales: Explorations in Devolved Governance. Cardiff: University of Wales Press.

Demeritt, D. (2000) The new social contract for science: accountability, relevance and value in UK and US science and research policy, Antipode, 32 (3), pp. 308-329.

Education and Lifelong Learning Committee (2000) Final Report: Laying the Foundations: Early Years Provision for Three Year Olds. Cardiff: National Assembly for Wales.

Egan, D. & James, R. (2003) ‘Education’ in Ormond, in J. & Jones, B. (eds) Birth of Welsh Democracy: the First Term of the National Assembly for Wales. Cardiff: Institute of Welsh Affairs.

Estyn (2003) Initial Teacher Training in Wales 1996-2002. Summary of Estyn Findings for Higher Education Funding Council Wales. Cardiff: Estyn.

Estyn (2005) The Annual Report of Her Majesty’s Chief Inspector of Education and Training in Wales. Cardiff: Estyn.

Foucault, M (1984) The Foucault Reader: An Introduction to Foucault’s Thought. Paul Rabinow (ed) London: Penguin Books.

Furlong, J., Hagger, H., Butcher, C. and Howson, J. (2006) Review of Initial Teacher Training Provision in Wales. A report to the Welsh Assembly Government. Oxford: University of Oxford Department of Educational Studies.

Furlong, J., Hirst, P., Pocklington, K. and Miles, S. (1988) Initial Teacher Training and the Role of the School. Buckingham: Open University Press

Furlong, J. and Maynard, T. (1995) Mentoring Student Teachers; The Growth of Professional Knowledge. London: Routledge.

Furlong, J. & White, P. (2002) Educational Research Capacity in Wales: A Review. Cardiff: University School of Social Sciences

Furlong, J., Barton, L., Miles, S., Whiting, C. and Whitty, G. (2000) Teacher Education in Transition. Buckingham: Open University Press.

Hagger, H., Burn, K. and McIntyre, D. (1993) The School Mentor Handbook. London: Kogan Page.

Hagger, H. (1997) ‘Enabling student teachers to gain access to the professional craft knowledge of experienced teachers’ in D. McIntyre (ed) Teacher Education Research in a New Context: The Oxford Internship Scheme. London. Paul Chapman, New BERA Dialogues, 99-133.

Hagger, H. and McIntyre, D. (2006) Learning teaching from teachers: realising the potential of school based teacher education. Buckingham: Open University Press

Halpern, D. (2003) ‘Does evidence matter?’ Talk given at Overseas Development Institute, April 2003; transcript available at www.odi.org.uk

National Assembly for Wales (2005) Press Release, W060013-Ed

National Assembly for Wales (2006a) The Learning Country 2. Cardiff: National Assembly for Wales.

National Assembly for Wales (2006b) Education and Lifelong Learning Committee; Proceedings Thursday, 26 January 2006. Cardiff: National Assembly for Wales.

National Assembly for Wales (2006c) Education and Lifelong Learning Committee 29.3.06 Initial Teacher Training - next steps, ELLS(2) 06-06(p2) . Cardiff: National Assembly for Wales.

National Assembly for Wales (2006d) Education and Lifelong Learning Committee, 29.3.06 Initial Teacher Training - Furlong Report. Cardiff: National Assembly for Wales.

Olssen, M., Codd, J. and O’Neill, A-M. (2004) Education Policy. Globalisation, Citizenship and Democracy. London: Sage

Rees, G. (2004) 'Democratic devolution and educational policy in Wales: the emergence of a national system?' Contemporary Wales 17; An Annual Review.

Rees, T. (2002) ‘The Independent Investigation Group on Student Hardship and Funding in Wales: Reflections from the Chair’, The Welsh Journal of Education 11 (1), 10-17.

Rees, T. et. al. (2005) Fair and Flexible Funding: A Welsh Model to Promote Quality and Access in Higher Education. Cardiff: Welsh Assembly Government.

Selby Smith, C. (2000) ‘The Relationships Between Research and Decision-making in Education: An Empirical Investigation’ in The Impact of Educational Research Department of Education, Training and Youth Affairs, Canberra Australia.

Shulock, N. (1999) ‘The Paradox of Policy Analysis: if it is not used, why do we produce so much of it?’, Journal of Policy Analysis and Management 18, 2, 226-244.

Soulesbury, W (2001) Evidence Based Policy: Whence it Came and Where it is Going. Working Paper 1. University of London Queen Mary College: ESRC UK Centre for Evidence Based Policy and Practice.

Stroud, D. (2002) 'Research Note: The Independent Investigation Group on Student Hardship and Funding in Wales', The Welsh Journal of Education, 10 (2), 92-104.

Weiss, C. (1998) 'Knowledge creep and decision accretion', Knowledge: Creation, Diffusion and Utilisation, 19 (1), 274-281.

Young, K., Ashby, D., Boaz, A. and Grayson, L. (2002) ‘Social Science and the Evidence Based Policy Movement’, Social Policy and Society 1:3 215-224.

This document was added to the Education-Line database on 26 January 2007