Education-line Home Page

The Effective Provision of Pre-School Education [EPPE] Project. A longitudinal study funded by the DfES (1997 – 2003)

now Effective Pre-School and Primary Education (EPPE) 3-11 2003 - 2008

Papers presented at the EPPE Symposium at the British Educational Research Association Annual Conference, University of Manchester, 16-18 September 2004

Convenor

Brenda Taggart

Chair and Discussant

Dr. Maria Evangelou
University of Oxford

Presentation One: Pedagogical and Interactional Aspects of Quality: How they are related to children’s developmental outcomes.

Professor Kathy Sylva

Presentation Two: Analyses of Family Salary Data, pre-school Duration and Quality, and Impact on Child Attainment and Progress at Entry to Primary School 1.

Professor Pam Sammons

Presentation Three: Effective Pre-School and Primary Education 3-11 (EPPE-3-11)

Brenda Taggart

September 2004

The EPPE Research Team

Principal Investigators

Professor Kathy Sylva
Department of Educational Studies, University of Oxford
00 44 (0)1865 274 008 / email kathy.sylva@edstud.ox.ac.uk

Professor Edward Melhuish
Institute for the Study of Children, Families and Social Issues
Birkbeck University of London
00 44 (0) 207 079 0834 / email e.melhuish@bbk.ac.uk

Professor Pam Sammons
University of Nottingham
00 44 (0) 0115 951 4434 / email pam.sammons@nottinghham.ac.uk

Professor Iram Siraj-Blatchford
Institute of Education, University of London
00 44 (0)207 612 6218 / email i.siraj-blatchford@ioe.ac.uk

Research Co-ordinator

Brenda Taggart
Institute of Education, University of London
00 44 (0)207 612 6219 / email b.taggart@ioe.ac.uk

EPPE 3-11 Project
Room 416
University of London
Institute of Education
20 Bedford Way
London WC1H 0AL
Tel: +44 (0)207 612 6219 Fax: +44 (0)207 612 6230
E
mail: b.taggart@ioe.ac.uk

Or visit our website on http://www.ioe.ac.uk/projects/eppe

Contents

Part One -
An Introduction to EPPE

Part Two – Linked Studies
The Linked Study in Northern Ireland 1998 – 2003
The Early Years Transition and Special Education Needs (EYTSEN) Project

Part Three
Synopsis of Key Finding and Summary of progress over the Pre-School Period

Part Four
The Key findings explained
The impact of pre-school provision
The quality and practices in pre-school centres

Pre- school ‘Quality’
What improves ‘quality’?
Pre-school practices

The quality of adult-child verbal interactions
Knowledge and understanding of the curriculum
Knowledge on how young children learn
How adults support children in resolve conflicts
Supporting children’s learning at home.

What improves ‘practice’?

Type of pre-school
The importance of home learning
Methodology

Part Five :
Overall Abstracts

Presentation One: Pedagogical and Interactional Aspects of Quality: how they are related to children’s developmental outcomes
Presenter: Professor Kathy Sylva

Presentation Two: Analyses of Family Salary Data, pre-school Duration and Quality, and Impact on Child Attainment and Progress at Entry to Primary School
Presenter: Professor Pam Sammons

Presentation Three: Effective Pre-School and Primary Education  (EPPE-3-11)
Presenter: Brenda Taggart

Technical Papers in the Series and Ordering information

The EPPE Team

The Effective Provision of Pre-School Education (EPPE) Project

Part One – An Introduction to EPPE1

This five year longitudinal study assessed the attainment and development of children between the ages of 3 to 7 years. Research began in 1997 and both quantitative and qualitative methods (including multilevel modelling) have been used to explore the effects of pre-school education on children's cognitive attainment and social/behavioural development at entry to school and any continuing effects on such outcomes two years later at the end of Key Stage 1 (age 7).

To investigate the effects of pre-school1 education for 3 and 4 year olds, the EPPE team collected a wide range of information on over 3,000 children, their parents, their home environments and the pre-school settings they attended.

Settings (141) were drawn from a range of providers (local authority day nursery, integrated2 centres, playgroups, private day nurseries, maintained nursery schools and maintained nursery classes). A sample of ‘home’ children (who had no or minimal pre-school experience) was recruited to the study at entry to school for comparison with the pre-school group. In addition to investigating the effects of pre-school provision on young children’s development, EPPE explores the characteristics of effective practice (and the pedagogy which underpin them) through twelve intensive case studies of settings with positive child outcomes.

In addition to pre-school centre effects, the study investigated the contribution to children’s development of individual and family characteristics such as gender, ethnicity, language, parental education and the educational environment of the home. The research design addresses a variety of research issues (methodological and practical) in investigating the impact of pre-school provision on children’s developmental progress.

EPPE has demonstrated the positive effects of high quality provision on children’s intellectual and social/behavioural developmental

Part Two – Linked studies

The Effective Pre-school Provision in Northern Ireland (EPPNI)

The Effective Pre-school Provision in Northern Ireland (EPPNI) shares a common methodology with the main EPPE study. EPPNI research involves 70 pre-school centres randomly selected throughout Northern Ireland. The study investigates all main types of pre-school provision attended by 3 to 4 year olds in Northern Ireland: playgroups, day nurseries, nursery classes, nursery schools and reception groups and classes.

The Early Years Transitions and Special Educational Needs (EYTSEN) Project

The EYTSEN project builds on the work of EPPE and explores evidence of possible special educational needs (SEN) amongst pre-school children. It uses a range of information to identify children who may be ‘at risk’ in terms of either cognitive or social behavioral development and investigates the links with a variety of child, parent and family characteristics.

Part Three: Synopsis of Key Finding and Summary of progress over the Pre-School Period

Key findings

Impact of attending a pre-school centre

  • Pre-school experience, compared to none, enhances children’s development.
  • The duration of attendance is important with an earlier start being related to better intellectual development and improved independence, concentration and sociability.
  • Full time attendance led to no better gains for children than part-time provision.
  • Disadvantaged children in particular can benefit significantly from good quality pre-school experiences, especially if they attend centres that cater for a mixture of children from different social backgrounds.
  • The quality and practices in pre-school centres

  • The quality of pre-school centres is directly related to better intellectual/cognitive and social/behavioural development in children.
  • Good quality can be found across all types of early years settings. However quality was higher overall in integrated settings, nursery schools and nursery classes.
  • Settings which have staff with higher qualifications, especially with a good proportion of trained teachers on the staff, show higher quality and their children make more progress.
  • Where settings view educational and social development as complementary and equal in importance, children make better all round progress.
  • Effective pedagogy includes interaction traditionally associated with the term "teaching", the provision of instructive learning environments and ‘sustained shared thinking’ to extend children’s learning.

  • Type of pre-school

  • There are significant differences between individual pre-school settings in their impact on children. Some settings are more effective than other in promoting positive child outcomes.
  • Children tend to make better intellectual progress in fully integrated centres and nursery schools.
  • The importance of home learning.

  • The quality of the learning environment of the home (where parents are actively engaged in activities with children) promoted intellectual and social development in all children. Although parent’s social class and levels of education were related to child outcomes the quality of the home learning environment was more important. The home learning environment is only moderately associated with social class. What parents do is more important than who they are.
  • Summary

    This study has demonstrated the positive effects of high quality pre-school provision on children’s intellectual and social behavioural development up to entry to primary school. The EPPE research indicates that pre-school can play an important part in combating social exclusion and promoting inclusion by offering disadvantaged children, in particular, a better start to primary school. The findings indicate pre-school has a positive impact on children’s progress over and above important family influences. The quality of the pre-school experience as well as the quantity (more terms but not necessarily more hours per day) are both influential. The results show that individual pre-school centres vary in their effectiveness in promoting intellectual progress over the pre-school period, and indicate that better outcomes are associated with some forms of provision. Likewise, the research points to the separate and significant influence of the home learning environment. These aspects (quality and quantity of pre-school and home learning environment) can be seen as more susceptible to change through policy and practitioner initiatives than other child or family characteristics, such as SES. Further analyses will explore the progress of the children who attended a pre-school centre as well as the home group over Key Stage 1. Such analyses will help to establish whether the positive impact of pre-school on young children’s development remains significant as children progress through their first years at primary school.

    Each of the Key Findings is discussed in more detail in Part Four.

    Part Four: The Key findings explained

    The impact of pre-school provision

    From analyses of children’s development during pre-school compared with ‘home’ children, EPPE found that pre-school attendance improves all children’s cognitive development and aspects of social behaviour, such as independence, concentration, cooperation, conformity and relationships with other children (peer sociability). Moreover, individual settings vary in their effectiveness with some settings fostering better child outcomes than others.

    Children with no (or limited) pre-school experience (the ‘home group’) had poorer cognitive attainment, sociability and concentration when they start school. These differences show even when we take account of differences between the pre-school and home groups in child, family and home environment characteristics.

    A number of factors associated with attendance at pre-school were also explored. EPPE show that how long a child attended pre-school (duration measured in months from entry to the study to the start of primary school) was related to positive intellectual gains. An early start at pre-school (under 3 years) was linked with better intellectual attainment and children having better relationships with other children (peer sociability) at age 3 years. These benefits continue when children start primary school. However, there was no evidence that full day attendance led to better development than half-day attendance.

    In addition to studying the overall impact on all children’s development the research explored whether pre-school had an impact on the progress of different kinds of children. For instance, was pre-school particularly beneficial to children who are more disadvantaged? EPPE shows that one in three children were ‘at risk’ of developing learning difficulties at the start of pre-school. However, this proportion fell to one in five by the time they started primary school3. This suggest that pre-school can be an effective intervention for the reduction of special educational needs (SEN), especially for the most disadvantage and vulnerable children.

    Disadvantaged children are more likely to have adverse social profiles at age 3 and school entry. The increased risk of anti-social/worried behaviour can be reduced by high quality pre-school when they were aged 3 and 4.

    Different groups of children have different needs. Results imply that specialised support in pre-schools, especially for language and pre-reading skills, can benefit children from disadvantaged backgrounds and those for whom English was an additional language.

    There is evidence of significant gender differences in young children’s intellectual and social behavioural development. At entry to pre-school, girls generally show better social development than boys, especially in co-operation/conformity and independence and concentration. Girls also show higher attainment on all cognitive outcomes. These differences persist to the start of primary school.

    EPPE has shown that pre-school has an important impact on children’s development. Whilst not eliminating disadvantage, it can help to ameliorate the effects of social disadvantage and can provide children with a better start to school. Investing in good quality pre-school provision is therefore likely to be an effective means of achieving targets concerning social exclusion and breaking cycles of disadvantage.

    The quality and practices in pre-school centres

    Pre- school ‘Quality’

    An important question for the EPPE research was whether higher quality pre-school provision makes a difference to the intellectual and social behavioural development of young children, and if so, what is essential in ensuring quality?

    Information from observations to assess the quality of each setting, using standardised rating scales4 showed significant links between higher quality and better child outcomes.

    Children in pre-school centres of high quality show reduced anti-social / worried behaviour by the time they get to school.

    EPPE findings on quality are consistent with other large-scale longitudinal research including the NICHD (National Institute of Child Health and Development) and CQO (Childcare Quality and Outcomes) studies in the US.

    Good quality pre-school education can be found in all kinds of settings irrespective of type of provider. However, the EPPE data indicate that integrated centres and nursery school provision have the highest scores on pre-school quality, while playgroups, private day nurseries and local authority centres have lower scores.

    The quality of the interactions between children and staff were particularly important; where staff showed warmth and were responsive to the individual needs of children, children showed better social behavioural outcomes. Several features of the quality rating scale were also related to increased intellectual progress and attainment at entry to school.

    What improves ‘quality’?

    There was a significant relationship between the quality of a centre and improved outcomes for children. There was also a positive relationship between the qualification levels of the staff and ratings of centre quality. The higher the qualification of staff, particular the manager of the centre, the more progress children made. Having qualified trained teachers working with children in pre-school settings (for a substantial proportion of time, and most importantly as the pedagogical leader) had the greatest impact on quality, and was linked specifically with better outcomes in pre-reading and social development.

    Pre-school ‘Practices’

    The rating scales used to assess quality showed an impact on children’s development. For instance, centres which put particular emphasis (as described in the rating scale) on the development of literacy, maths and catering for children’s individual needs promoted better outcomes for children in the subsequent development of reading and mathematics. Similarly, high scores on some aspects of the rating scale which focus on promoting positive ‘social interactions’ were linked with better sociability in children.

    In addition to the rating scale measurements of quality, EPPE conducted individual intensive case studies in 12 centres identified in the upper range of effectiveness based on the amount of progress their children made while attending them. The purpose of the case studies was to explore the practices in these centres that might help explain their greater effectiveness. This has important implications for all those working directly with young children as it describes practices linked to children making better progress.

    The case studies identified five areas that are particularly important when working with children aged 3 to 5 years. These were the quality of adult-child verbal interactions; staff knowledge and understanding of the curriculum; knowledge of how young children learn; adult’s skill in supporting children in resolving conflicts and helping parents to support children’s learning in the home.

    The quality of adult-child verbal interactions

    ‘Sustained shared thinking’ is where two or more individuals ‘work together’ in an intellectual way to solve a problem, clarify a concept, evaluate an activity, extend a narrative etc. Both parties must contribute to the thinking and it must develop and extend the understanding. It was found that the most effective settings encourage ‘sustained shared thinking’ which was most likely to occur when children were interacting 1:1 with an adult or with a single peer partner. It would appear that periods of ‘sustained shared thinking’ are a necessary pre-requisite for the most effective early years practice.

    Interestingly, information from interviews with parents suggests that in some of the very middle class case study settings (notably the private day nurseries), parents who were pro-active towards their children’s learning engaged in ‘sustained shared thinking’ with their children at home. In more disadvantaged settings staff had to be pro-active in supporting parents to develop the home learning environment.

    Knowledge and understanding of the curriculum

    Pre-school workers’ knowledge of the particular curriculum area that is being addressed is vital. A good grasp of the appropriate curriculum content linked to strategies for promoting learning in that content area is a vital component of pedagogy and it is shown to be just as important in the early years as at any later stage of education. The research found that, even in these effective settings, there were examples of inadequate knowledge and understanding of curriculum areas, especially in the teaching of the sound patterns of word e.g. rhymes. Our study shows that early years staff may need support in developing their knowledge of curriculum content and ways of introducing it to children especially in the domains of the Early Learning Goals.

    Knowledge on how young children learn

    There has been a long debate about the extent to which pre-school education should be formal or informal, often summarised by the extent to which the curriculum is or is not ‘play’ based. EPPE concludes that in the most effective centres, ‘play’ environments were used to provide the basis of instructive learning. The most effective pedagogy is both ‘teaching’ and providing freely chosen yet potentially instructive play activities.

    In effective settings, the balance of who initiated the activities, staff or child, was about equal, Children were encouraged to initiate activities as often as the staff. Similarly in effective settings the extent to which staff extended child-initiated interactions was important. Almost half of the child-initiated episodes which contained intellectual challenge, included interventions from a staff member to extend the child’s thinking. The evidence also suggests that adult ‘modelling’ is often combined with sustained periods of shared thinking, and that open-ended questioning is also associated with better cognitive achievement. However, open-ended questions made up only around 5% of the questioning used in even the ‘effective’ settings. Greater use of such open ended questions by staff is likely to benefit better intellectual and social development for pre-school children.

    In all of the case study settings, we found that the children spent most of their time in small groups. Freely chosen play activities often provided the best opportunities for adults to extend children’s thinking. It may be that extending child-initiated play, coupled with the provision of teacher-initiated group work, improves opportunities for learning.

    We found that qualified staff in the most effective settings provided children with more experience of curriculum-related activities (especially language and mathematics) and they encouraged children to engage in activities with higher intellectual challenges. While we found that the most highly qualified staff also provided the most direct teaching, we found that they were the most effective in their interactions with the children, using the most sustained shared thinking. Further, we found that less qualified staff were significantly better as pedagogues when they worked with qualified teachers.

    How adults support children in resolve conflicts

    The most effective settings adopted discipline/behaviour policies in which staff supported children in being assertive, while simultaneously rationalising and talking through their conflicts. In settings that were less effective in this respect, our observations showed that there was often no follow up on children's misbehaviour and, on many occasions, children were ‘distracted’ or simply told to stop.

    Supporting children’s learning at home.

    The most effective settings shared child-related information between parents and staff, and parents were often involved in decision making about their child’s learning programme. There were more intellectual gains for children in centres that encouraged high levels of parental involvement. More particularly, children did better where the centre shared its educational aims with parents. This enabled parents to support children at home with strategies that complemented those being undertaken in the pre-school. In more disadvantaged areas, staff in effective settings had to be proactive in influencing and supporting the home learning with the kind of activities described later in this briefing.

    What improves ‘practice’?

    The case studies reveal the practices that appear to contribute to better outcomes for children. The following factors should be considered when trying to improve the pre-school experiences of very young children.

    The settings that view cognitive and social development as complementary achieve the best all round outcomes.

    Pre-school workers need good curriculum knowledge as well as knowledge and understanding of child development. In addition, increasing formative feedback to children during activities will aid a child’s understanding.

    The most effective settings provide both adult-initiated group work and freely chosen yet potentially instructive play activities. Children’s cognitive outcomes appear to be directly related to the quantity and quality of the teacher/adult planned and initiated focused group work for supporting children’s learning.

    Behaviour policies in which staff support children in being assertive, at the same time as rationalising and talking through their conflicts lead to better socialisation for children.

    Improving practices in sharing educational aims with parents would benefit children.

    Trained teachers were most effective in their interactions with children, engaging more often in sustained shared thinking. Less well-qualified staff demonstrated significantly better practices when they were led by qualified teachers.

    The research findings support the general approach taken in Curriculum guidance for the foundation stage (CGFS).

    Type of pre-school

    Even after taking account of a child’s background and prior intellectual skills, the type of pre-school a child attends has an important effect on their developmental progress.

    It was found that integrated centres (these are centres that fully combine education with care) and nursery schools tend to promote better intellectual outcomes for children.

    Similarly, integrated centres and nursery classes tend to promote better social development even after taking account of children’s backgrounds and prior social behaviour.

    Disadvantaged children do better in settings with a mixture of children from different social backgrounds rather than in settings containing largely disadvantaged groups. This has implications for the siting of centres in areas of social disadvantage.

    The importance of home learning

    What parents and carers do makes a real difference to young children’s development. The EPPE project developed an index to measure the quality of the home learning environment (HLE). This measures a range of activities that parents undertake with pre-school children that are related to improvements in children’s learning and have a positive affect on their development. For example, reading to child, teaching songs and nursery rhymes, painting and drawing, playing with letters and numbers, visiting the library, teaching alphabet, teaching numbers, taking children on visits and creating regular opportunities for them to play with their friends at home were all associated with higher intellectual and social/behavioural scores.

    The HLE can be viewed as a ‘protective’ factor in reducing incidence of SEN. It is interesting to note that the HLE was only moderately associated with mother’s educational level. In other words what parents do with their children is more important than who parents are. Young mothers, with few qualifications can improve their children’s progress, and give them a better start at school by engaging in those activities at home that foster children’s learning. This has important implications for programmes such as Sure Start (local programmes) that target areas of high social exclusion.

    Methodology

    EPPE used the following sources of information: standardised child assessments taken over time, child profiles completed by pre-school staff, parental interviews, interviews with pre-school centre staff, quality rating scales and case study observations and interviews. The case studies included detailed documentation of naturalistic observations of staff pedagogy, and systematic structured target child observations of children’s learning. Information was also gathered and analysed using interviews with parents, staff and managers and through intensive and wide ranging documentary analysis and a literature review of pedagogy in the early years.

    These sources of data have used in statistical analyses including multilevel modelling to explore the ‘value added’ by pre-school after taking account of a range of child, parent and home background factors to produce rigorous and persuasive data for policy makers and provided practical guidance on quality for practitioners.

    Part Five : The Presentations

    Overall Abstract:

    The Effective Provision of Pre-School Education (EPPE) project is the first major European longitudinal study of a national sample of young children’s development between the ages of 3 and 7 years. The EPPE team collected a wide range of information on 3,000 children, their parents, home environments and the pre-school settings they attended. Settings (141) were drawn from a range of types of providers (local authority day nursery, integrated centres, playgroups, private day nurseries, nursery schools and nursery classes). A sample of ‘home’ children (who had no or minimal pre-school experience) were recruited to the study at entry to school for comparison with the pre-school group. In addition to investigating the effects of pre-school provision, EPPE explores the characteristics of effective practice (and the pedagogy which underpin them) through twelve intensive case studies of settings with positive child outcomes. The study has demonstrated the positive effects of high quality provision on children’s intellectual and social/behavioural developmental measured at the start of primary school. The study has had impact at national level (through research evidence to the Sure Start unit and the Treasury for the Spending Review), at local level (through work with LEAs seeking advice on reconfiguring their services) and at practitioner level (through keynote addresses, conferences, seminars and workshops).

    The three papers presented in this symposium will report on (a) a brief background to EPPE for those new to the study, highlighting some key findings. This will be followed by findings on the relationship between children’s developmental outcomes and three different observational assessments of Early Childhood Quality: the Arnett Adult/Child Interaction Scale, the Early Childhood Environmental Rating Scales (ECERS-R and the ECERS-Extension) This work is currently being used to help practitioners consider their own self audits of ‘quality’ and raising the debate about what constitutes quality in the early years settings within the context of the Curriculum Guidance for the Foundation Stage.

    The symposium will also consider (b) There is increasing interest in finding ways to present the results of educational research in ways that are meaningful and useful to policy makers and researchers. The paper provides an example of the use of effect sizes in a multilevel analysis of the impact of differences in family income on young children’s cognitive attainment at entry to primary school and makes comparisons with the impact of quality and duration of pre-school on attainment. It is argued that effect sizes can provide additional information about the relative strength of relationships amongst measures of particular interest to policy makers.

    The symposium will end (c) with a paper on the continuation of the EPPE study. This new project: Effective Pre-School and Primary Education 3-11 (EPPE 3-11, 2003 - 2008) continues to build on the extensive data collected in the original EPPE study, following the children up to the age of 11 at the end of Key Stage 2. EPPE 3-11 is interested in the continual effects of pre-school education, the characteristics of effective primary schooling, the learning trajectories of resilient and vulnerable children and the contribution of out of school learning. The new project will adopt an innovative design that will explore the identification of effective primary schools using quantitative data derived from analyses of matched data sets across Key Stages. This quantitative data will inform focus school observations which will explore the practices and process at classroom level which may influence children’s cognitive and social/behavioural development. The study combines statistical analyses, observational and interview data.

    Presentation One: Pedagogical and Interactional Aspects of Quality: How they are related to children’s developmental outcomes

    Presenter: Professor Kathy Sylva

    Abstract :

    EPPE has produced persuasive data on the effects of pre-school education on children’s cognitive and social/behavioural development between the ages of 3 and 5 years old. During the pre-school period the study assessed ‘quality’ using three observational scales of quality: the Early Childhood Environmental Rating Scale-Revised (an international instrument based on Developmentally Appropriate Practice) and the Early Childhood Environmental Rating Scale-Extension (developed by the EPPE team to be sensitive to the English early years curriculum). In addition the Arnett Adult/Child Interaction Scale (CIS) was administered to measure the sensitivity of adult’s interactions with children. The CIS measures the following: Responsiveness, Detachment, Punitiveness and Permissiveness. This paper will compare the three quality assessment scales in light of their relationship to children’s developmental outcomes.

    Presentation:

    Research Question

    When considering children’s developmental outcomes, are the effects of quality in curriculum and pedagogy different from those of the quality of adult-child interaction? In other words, can we distinguish different aspects of Early Childhood quality through differential effects on children’s development?

    Measures

    Early Childhood Environment Rating Scale-Revised (Harms et al)
    Early Childhood Environment Rating Scale-Extension (Sylva et al)
    Caregiver Interaction Scale (Arnett)

    Results

    The Presentation will include tables showing the significant associations between scores on the different aspects of quality and children’s developmental gains over a two-year period in cognitive attainment and in social behaviour and concentration.

    Discussion

    It has long been known that sensitivity and curriculum are both important; in this presentation it will be argued that they lead to different developmental outcomes in children. A well planned and stimulating curriculum/pedagogy lead to pre-reading and numeracy skills and to non-verbal reasoning. On the other hand responsive relationships lead to children’s gains in cooperation, pro-social behaviour and independence/concentration. Children thrive on what might be called ‘academic starch’ but they also thrive on sensitive interactions in the context of responsive relationships. One without the other makes an unbalanced experience for children. EPPE stress that sensitive caring is not enough. The EPPE data from the ECERS-R and the ECERS-E scales as well as Arnett’s Care giving Interaction Scale show the differing contribution of the two different strands to quality in the Early Years.

    Presentation Two: Analyses of Family Salary Data, Pre-School Duration and Quality, and Impact on Child Attainment and Progress at Entry to Primary School.

    Presenter: Professor Pam Sammons

    Abstract:

    There is increasing interest in finding ways to present the results of educational research in ways that are meaningful and useful to policy makers and researchers. Shagen & Elliot (2004) discuss the potential use and limitations of effect sizes in improving researchers and policy maker’s understanding of research data. They argued that effect sizes can provide additional information about the relative strength of relationships amongst a range of measures of particular interest to policy makers. This presentation provides examples of the use of effect sizes in a multilevel analysis of the impact of differences in family income on young children’s cognitive attainment at entry to primary school and makes comparisons with the impact of quality and duration of pre-school on attainment. The calculation of effect sizes for specific sub groups of children allow comparison with the effects on attainment levels at entry to primary school (rising 5 years) attributable to a variety of child, family or home environment characteristics. Of particular relevance for this paper are the comparisons of the size of parental salary effects with those of the influence of pre-school provision.

    The Presentation:

    This presentation summarises analyses originally prepared for HM Treasury in November 2003, leading up to the 2003/4 Spending Review Round. The data on parental salaries was derived from a questionnaire to EPPE parents administered in Autumn 2001 /spring 2002. The response rate to the parent survey was over 77%. The paper provides an example of the use of effect sizes in a multilevel analysis of the impact of differences in family income on young children’s cognitive attainment at entry to primary school and makes comparisons with the impact of quality and duration of pre-school on attainment

    A parental questionnaire survey provided indicators of mother’s and father’s salary level in the form of average yearly salary (before tax). Parents who were not employed were coded as having a zero salary level. A total parent salary measure was created combining the mother’s and father’s salary bands. To do this the mid point of each band was used except for the top band (£65000 plus) where the conservative estimate of £66,000 was chosen. The use of bands means that the salary data provide an approximate indication of relative family salary levels but cannot be treated as providing accurate information about actual gross salary. The use of bands means that it is not possible to explore the relationships between children’s attainments and more finely differentiated salary levels. Seven broad joint income categories were created, including an ‘unknown’ category for children for whom no parent salary data were collected

    Table 1 – Parent Salary bands

    Salary Band

    Range £

    n

    %

    1

    No salary

    569

    17.9

    2

    2500-15000

    485

    15.3

    3

    17500-27500

    411

    13.0

    4

    30000-35000

    271

    8.5

    5

    37500-66000

    470

    14.8

    6

    67500-132000

    173

    5.5

    7

    No data available#

    792

    25.0

    #no questionnaire returned or no response to item

    Just under one- in five children were in families with no earned income. No data were available for a quarter of the sample (no response to the survey or to the specific item in the questionnaire). A measure was also created concerning parents employment status. This covers no parent working (17.9%), mother only (10.3%), father only (13.7%), and both parents working (33.1%). Data were missing for 25% (due to no questionnaire return/response).

    The parent salary and the employment status measures were tested in multilevel models used to explore the impact of pre-school on attainment and which included a ‘home’ sample that had no pre-school experience. A range of child parent and family measures are controlled in order to assess the net impact of parent salary and pre-school while controlling for other significant predictors such as parents SES, mother’s qualification level, family size, birth weight, ethnicity and EAL status and aspects of the home learning environment.

    Correlations show the relationship between the parent income measure (continuous) and attainment in Pre-reading and Language (0.37 for both). These associations are somewhat stronger than those between parental work status (0.24-0.30) and attainment (note that these associations do not control for other measures, such as parental qualifications and SES).

    Table 2: Associations between salary, work status and child attainment

     

    Pre-reading (standardised)

    Language (standardised)

    Total Parent Salary (continuous)

    0.37**

    0.37**

    Parental Work Status

    0.24**

    0.30**

    ** Correlation is significant at the 0.01 levels

    The multilevel estimates allow the calculation of effect sizes (ES) for salary categories, while controlling for the influence of other relevant factors such as birth weight, parent highest SES, mother’s qualification levels, aspects of the home learning environment etc. Table 3 shows ES results for parental salary compared with those for the duration of pre-school. In all cases full control is made for other significant child, family and home environment measures, thus the ES are net of the influence of other factors (the direction of the relationships, positive or negative, from original estimates is also shown).

    Table 3 Comparison of Effect sizes for parental salary and pre-school attendance for attainment at entry to primary school

    Salary Group

    (Compared with no reported salary including not working and unemployed or parent absent etc)

    Pre-reading

    Language

    £2500-£15000

    0.066

    0.057

    £15100-£27500

    0.177*

    0.091

    £27600-£35000

    0.143

    0.113

    £35100-£66000

    0.315*

    0.140

    £66100 plus

    0.502*

    0.222*

    Salary NK

    0.014 negative

    0.103 negative

         

    FSM (compared to not eligible)

    0.127* negative

    0.103 negative

         

    Duration of pre-school (compared with no pre—school attended i.e. ‘home’ children)

       

    Under 1 year

    0.123

    0.456*

    1-2 years

    0.255*

    0.379*

    2-3 years

    0.361*

    0.421*

    3 years plus

    0.403*

    0.591*

    The results indicate that parental salary level is more closely related to young children’s pre-reading than their language development. The only statistically significant differences in attainment are between the no salary group and those on higher reported joint incomes (the band £35100-£66000 and the band £66000 plus). The effect size is larger for the highest salary level for pre-reading.

    The effect size associated with just under one year of pre-schooling is 0.123 for the pre-reading outcome, this a little smaller than the effect of earned family income band £17500-£27500 versus no salary income. Interestingly the effect size associated with one year of pre-school for language outcomes is significantly larger than for pre-reading, while the impact of family income for language is much lower.

    The effect sizes for parents’ employment status are generally smaller than those for family income or pre-school duration. The results are significant for pre-reading attainment, with moderate positive effects (controlling for other significant predictors) for both parents working (ES both parents working compared with no parent working 0.202 for pre-reading, 0.119 for language). Nonetheless, duration of pre-school has a stronger net impact.

    Quality and duration

    Further analyses were conducted to explore the influence of quality and duration of attendance effects on child attainment at the start of primary school. Pre school centres were divided into three groups low (bottom 20%), average (middle 60%) and high (top 20%) based on total ECERS E rating, an observational quality measure. These bands were chosen after looking at the distribution of scores to distinguish the low and high quality centres. Because the 'cut offs' were based on centre scores (considered the most appropriate approach in studying variation in center processes) the numbers of children in the three bands varied (around 16% of the sample were in centres with the lowest quality scores but around 24% in centres with the highest scores). Within each quality band children were further divided on the basis of duration of attendance. It should be noted that due to the relatively smaller numbers in the low quality band, the sub divisions by duration are broader. Therefore direct comparisons for the low quality low duration are not possible. Table 4 shows the net effects for each of the sub-groups.

    Table 4 Comparison of Effect sizes for quality and duration

    Pre-school group

    (Compared with no duration, no quality i.e. the ‘home group)

    Pre-reading

    Language

    Low quality low duration (< 24 months)

    0.254

    0.602*

    Low quality average duration (24-36 months)

    0.293*

    0.540*

    Low quality high duration (36 months plus)

    0.368*

    0.529*

    Average quality very low duration (< 12 months)

    0.153

    0.459*

    Average quality low duration (12-24 months)

    0.331*

    0.459*

    Average quality average duration (24-36 months)

    0.479*

    0.528*

    Average quality high duration (36 months plus)

    0.545*

    0.672*

    High quality very low duration (<12 months)

    0.256*

    0.338*

    High quality low duration (12-24 months)

    0.381*

    0.526*

    High quality average duration (24-36 months)

    0.346*

    0.535*

    High quality high duration (36 months plus)

    0.622*

    1.010*

    In comparison with the home group all levels of quality and duration show a significant positive effect compared with none. Overall, longer duration shows a greater benefit than low duration, however, the combination of high quality and high duration shows a particularly strong effect size(1.01) for language, and a fairly large effect for pre-reading (0.622). To try to distinguish the separate quality effect we can calculate the net difference between low quality high duration and high quality high duration. For language this gives an estimate of 0.481. For pre-reading the difference is somewhat smaller at 0.236.

    The original EPPE analyses modelled the quality measure as a continuous scale in value added analyses of children’s cognitive progress over the pre-school period (but the home group were not included in these models because they were not recruited to the study until entry to primary school). The results also confirmed a separate significant effect for quality (as measured by the ECERS -E instrument) over and above a larger effect for duration (see Sammons et al 2002).

    Summary

    The calculation of effect sizes for specific sub groups of children allow comparison with the effects on attainment levels at entry to primary school (rising 5 years) attributable to a variety of child, family or home environment characteristics. Of particular relevance for this paper are the comparisons of the size of parental salary effects with those of the influence of pre-school provision. While effect size information can allow comparison of the relative strength of different predictors in statistical models, it is argued that reference to other important information is also relevant (Elliot & Sammons, 2004). The use of effect sizes cannot compensate for a poor sample or inadequately specified model. In EPPE the large data set allowed control for a wide range of child, family and home environment factors in analyses of possible pre-school effects. In calculating centre effects, longitudinal value added analyses of cognitive progress or social behavioural development were conducted. However, to explore the influence of income, contextualised models of attainment at entry to school were considered most appropriate, since these show a stronger net influence for individual child, family and home environment factors (because these are associated with prior attainment levels at younger ages).

    The results indicate that pre-school duration and quality of centre provision have an impact on attainment irrespective of parental salary levels. Only fairly high joint salary bands show a significant net impact on attainment compared with the no salary band. Analyses reported elsewhere show positive effects for duration and quality of pre-school provision on additional outcomes (early number concepts and social behavioural measures, particularly Peer sociability and Independence and concentration, measured by teacher ratings at entry to primary school). Combined with the main EPPE findings on pre-school effectiveness (Sammons et al 2002, 203) the results of the present analyses indicate that additional investment in pre-school provision (improving quality and extending provision) may therefore be expected to have a greater impact on young children’s development than the provision of additional family income except at high levels, particularly since the impact of differences in earned income may well differ from those of increases in benefit income. In relation to the findings it should be noted that children from more advantaged backgrounds in the study experienced a longer duration of pre-school (the average duration was 21 months, but children whose mothers had a degree spent around 5 months longer in pre school than those whose mothers had no qualification. None or a reduced period of time in pre-school, therefore, can be seen as an additional disadvantage for vulnerable groups of young children.

    Elliot, K. and Sammons, P. (2004) Exploring the use of effect sizes to evaluate the impact of different influences on child outcomes. In I .Schagen & K Elliot (eds) What Does it Mean? The Use of Effect Sizes in Educational Research, (Slough: NFER ), pp 6-24.

    Sammons, P, Sylva., K, Melhuish, E, Taggart, B, and Elliot, K (2002) Measuring the impact of pre-school on children’s cognitive progress over the pre-school period. EPPE Technical Paper 8a. (London: DfES/Institute of Education)..

    Sammons, P, Sylva., K, Melhuish, E, Taggart, B, and Elliot, K (2003a) Measuring the impact of pre-school on children’s social behavioural progress over the pre-school period, EPPE Technical Paper 8b. (London: DfES/Institute of Education).

    Schagen, I & Elliot, K (eds) 2004 What Does it Mean? The Use of Effect Sizes in Educational Research, (Slough: NFER ), pp 6-24.

    Presentation Three: Effective Pre-School and Primary Education 3-11 (EPPE-3-11)

    Presenter: Brenda Taggart

    Abstract:

    The Effective Pre-School and Primary Education 3-11 (EPPE 3-11, 2003 – 2008) project continues to build on the extensive data collected in the original EPPE study, following the children up to the age of 11 at the end of Key Stage 2. EPPE 3-11 is interested in the continual effects of pre-school education, the characteristics of effective primary schooling, the learning trajectories of resilient and vulnerable children and the contribution of out of school learning. The paper will explain the innovative design that will explore the identification of effective primary schools using quantitative data derived from analyses of matched data sets across Key Stages. This quantitative data will inform focus school observations which will explore the practices and process at classroom level which may influence children’s cognitive and social/behavioural development. The study combines statistical analyses, observational and interview data.

    The Presentation:

    The main EPPE research official ended in August 2003. However, the information from the research has had impact at national, LEA and practitioner level. EPPE has provided evidence for two Treasury spending review rounds and has help secure significant additional resources for early years education and care. The roll out of the new Children’s Centre programme has been predicated on the research findings. As EPPE have a unique sample of 3,000 children, with plotted individual learning trajectories, the opportunity to use the sample to answer further research questions presented itself. The Effective Pre-School and Primary Education 3-11 (EPPE 3-11) Project is an extension to the main EPPE study and follows the same cohort of children to the end of Key Stage 2. This five year extension has been developed to explore four related themes in a series of embedded sample and research tiers.

    The four main research questions are:

    1. Do the effects of pre-school continue through to Key Stage 2?
    2. What are the characteristics of ‘effective’ primary classrooms and schools?
    3. Who are the resilient and the vulnerable children in the EPPE sample?
    4. What is the contribution of ‘out-of-school learning’ (homes, communities, internet) to children’s development?

    The following chart describes the breadth and depth of the research.

    Tier 1 – Primary School Effectiveness study

    The aim of the first tier is to compare the effectiveness (and trends in effectiveness) across Key Stage 2 of all primary schools in England. This study will provide effectiveness measures for the schools in the EPPE sample and allow us to place the schools EPPE children attend in the context of all other schools in England. This study will constitute the first major multilevel value added study of the effectiveness of primary schools in England.

    Tier 2 -The continuing effects of pre-school education

    This study aims to establish whether the effects of pre-school that were apparent at entry to school continue through to Key Stage 2 for 2400+ children in 700+ schools. Statistical models will examine the extent to which pre-school centres have any continuing impact on pupil attainment at age 11, after controlling for children's performance in relevant assessments at entry to reception (rising 5) and for social background. The study will identify children who have ‘succeeded beyond the odds’ and children whose early profiles were sound but who later underachieved, despite average or good early profiles. The study will investigate the factors associated with these resilient and vulnerable trajectories. The study will also continue to monitor the progress and development of various groups of children ‘at risk ‘of SEN as identified in the linked EYTSEN extension study.

    Tier 3 Investigating educational processes

    Approximately 1600+ EPPE children attend 125 focal schools. These EPPE children and their classmates will help us investigate in more detail the contribution to children’s outcomes of classroom and school processes. This research will also provide further information on classroom and school climate and other processes related to children’s attainment and social development. Classroom observations will be undertaken in Year 5 of the focal schools. An observational schedule based on the NICHD study developed by Professor Robert Pianta will be used. The ‘Pianta’ observation scales focus on the:

    Which gives information on: setting, curriculum, teacher behaviour, child academic behaviour, child social behaviours and

    Classroom – positive affect, self-reliance, sociable/co-operative with peers, attention, disruptive, activity level, child- teacher relationships

    Which gives information on: Richness of instructional methods, over-control, chaos, detachment, positive classroom climate, negative classroom climate, productive use of instructional time, evaluative feedback, teacher sensitivity.

    In addition an additional schedule ‘the Stipek’ scale will be administered to explore in more detail literacy and numeracy in a sub sample of school.

    The ‘Stipek’ gives information on

    General – classroom climate, classroom routines, cross-disciplinary connections, linkage to beyond the classroom, social support for learning, engagement.

    Literacy – reading as meaning making, skills in context, higher order thinking in writing, purposeful development if writing skill and instructional conversations.

    Maths – use of analysis, knowledge and understanding, problem solving, discourse and communications and locus of authority.

    The multilevel modelling techniques applied in the full sample study will be used, but in this case the range of predictor variables will include measures of classroom and school processes. Thus, we will have available more information on classroom and school processes to include in the statistical models explaining educational effectiveness and resilience.

    The Effective Provision of Pre-School (EPPE) Project Technical Papers in the Series

    Please note that some papers are now into re-prints which are slightly more expensive their original price.

    Technical Paper 1 - An Introduction to the Effective Provision of Pre-school Education (EPPE) Project ISBN: 085473 591 7 Published: Autumn 1999 Price £8.50

    Technical Paper 2 - Characteristics of the Effective Provision of Pre-School Education (EPPE) Project sample at entry to the study ISBN: 085473 592 5 Published: Autumn 1999 Price £4.00

    Technical Paper 3 - Contextualising EPPE: Interviews with Local Authority co-ordinators and centre managers ISBN: 085473 593 3 Published: Autumn 1999 Price £3.50

    Technical Paper 4 - Parent, family and child characteristics in relation to type of pre-school and socio-economic differences. ISBN: 085473 594 1 Published: Autumn 1999 Price £4.00

    Technical Paper 5 – Characteristics of the Centre in the EPPE Study: (Interviews) ISBN: 085473 595 X Published: Autumn 2000 Price £5.00

    Technical Paper 6 - Characteristics of the Centres in the EPPE Sample: Observational Profiles ISBN: 085473 596 8 Published: Autumn 1999 Price £8.50

    Technical Paper 6A - Characteristics of Pre-School Environments ISBN: 085473 597 6 Published: Autumn 1999 Price £8.50

    Technical Paper 7 - Social/behavioural and cognitive development at 3-4 years in relation to family background ISBN: 085473 598 4 Published: Spring 2001 Price £5.00

    Technical Paper 8a – Measuring the Impact of Pre-School on Children’s Cognitive Progress over the Pre-School Period. ISBN: 085473 599 2 Published: Autumn 2002 Price £8.50

    Technical Paper 8b – Measuring the Impact of Pre-School on Children’s Social/behavioural Development over the Pre-School Period. ISBN: 085473 683 2 Published March 2003 Price £8.50

    Technical Paper 9 - Report on age 6 assessment ISBN: 085473 600 X Publication Date Autumn 2004

    Technical Paper 10 - Intensive study of selected centres ISBN: 085473 601 8 Published Autumn 2003 Price £11.00

    Technical Paper 11 - Report on the continuing effects of pre-school education at age 7 ISBN: 085473 602 6 Publication Date: Autumn 2004

    Technical Paper 12 - The final report ISBN: 085473 603 4 Publication Date: Autumn 2004

    Related Publications

    The Early Childhood Environment Rating Scale: Revised Edition (1998). Harms, Clifford and Cryer

    ISBN: 08077 3751 8 Available from Teachers College Press. Columbia University. 1234 Amsterdam Avenue. New York. NY10027

    Assessing Quality in the Early Years, Early Childhood Environment Rating Scale Extension (ECERS-E):Four Curricular Subscales (2003) Sylva, Siraj-Blatchford and Taggart (2002) Trentham Books

    Early Years Transition and Special Educational Needs (EYTSEN) Technical Paper 1: Special Educational Needs across the Pre-school Period.

    EYTSEN Technical Paper 2: Special Educational Needs in the Early Primary Years: Primary school entry up to the end of Year One.

    EYTSEN Technical Paper 3: Special Educational Needs: The Parents’ Perspective

    Ordering information – For EPPE Publications

    The Bookshop at the Institute of Education. 20, Bedford Way. London WC1H OAL. Tele: 00 44 (0) 207 612 6050 Fax: 0207 612 6407 e-mail: ioe@johnsmith.co.uk, website: www.johnsmith.co.uk/ioe or The EPPE Office. The University of London, Institute of Education. 20 Bedford Way, London. WC1H OAL. U.K.Telephone 00 44 (0) 207 612 6219 / Fax. 00 44 (0) 207 612 6230 / e-mail b.taggart@ioe.ac.uk Please Note : Prices will vary according to size of publication and quantities ordered.

    The EPPE Team

    Principal Investigators

    Professor Kathy Sylva
    Department of Educational Studies, University of Oxford

    Kathy Sylva is Professor of Educational Psychology at the University of Oxford. Her research centres on evaluation of pre-school and primary educational interventions, with a special focus on the predictors of cognitive, linguistic and social outcomes in longitudinal research. She has also led research projects on literacy interventions, supporting parents, and the effects of child care on children and families.

    Professor Edward Melhuish
    Institute for the Study of Children, Families and Social Issues
    Birkbeck University of London

    Professor Edward Melhuish is a developmental psychologist. His research interests are in studying environmental influences on social, cognitive and communicative development, often using longitudinal studies. Interests include pre-school care and education; parenting; and the linking of research on child development with social policy. Previous research influenced the 1989 Children Act.

    Professor Pam Sammons
    School of Education, University of Nottingham

    Professor Sammons research interests include school effectiveness and improvement, longitudinal studies, evaluation and equity issues in education. She is a Professor of Education at the University of Nottingham and was formerly Coordinating Director of the International School Effectiveness and Improvement Centre (ISEIC) at the Institute of Education (1999-2004). She has directed a range of studies on primary and secondary schools funded by ESRC, DfEE/DfES, SOEID/SEED, DENI, OFSTED, QCA and several LEAs.

    Professor Iram Siraj-Blatchford
    Institute of Education, University of London

    Professor Siraj-Blatchford is a specialist in Early Childhood Education at the Institute of Education, University of London. Her research interests include early childhood curriculum and pedagogy. She has published on issues of Early Childhood quality and equality. She has directed projects for several bodies including the DfEE, Aga Khan Foundation, Esme Fairburn Trust and the Leverhulme Trust. She has been a consultant to UNESCO. Currently she is evaluating several Early Excellence Centres.

    Research Co-ordinator

    Brenda Taggart
    Institute of Education, University of London.

    Brenda Taggart has a background in primary education, having been a teacher, primary advisory teacher, deputy head and acting head. She has held senior positions as both an initial and in-service trainer of teachers. She has worked on a number of research projects focusing on primary and early years education. This has included work for the ESRC, SCAA, NUT, DENI, the DfEE and several LEAs.

    Notes:

    1. EPPE Technical Paper One : An Introduction to EPPE, sets the design of EPPE within the context of other research studies on the effectiveness of early education and care. Pre-school centres in this document means those centres that included 3 and 4 year olds.

    2. ‘Integrated’ settings fully combines education and care and is referred to as ‘combined’ centres in EPPE Technical Papers.

    3. See the Early Transition and Special Education Needs (EYTSEN) Report for more detail on SEN in the early years.  Published by Institute of Education.

    4. The Early Childhood Environment Rating Scales: Revised (ECERS-R) and Extension (ECERS-E).

    This document was added to the Education-line database on 18 January 2005