Protocol for a systematic review: Universal school-based programmes for improving social and emotional outcomes in children aged 3-11 years: a systematic review and meta-analysis.

Abstract

Social and emotional competence is vital for forming and sustaining good relationships, solving everyday problems, adapting behaviour and making healthy choices throughout life. It includes self-awareness, understanding and working with others, controlling emotions and caring about oneself and others. Through social and emotional learning, children and adults are able to develop the skills necessary to work and live effectively in society, yet a significant proportion of pupils struggle with one or more facets of social and emotional development (Doll, Brehm, & Zucker, 2014; Jones & Bouffard, 2012; Zins, 2004). Estimates suggest that between 9.5 and 14.2 percent of children under five experience some form of social and emotional problem which negatively impact their functioning, development and school-readiness (Brauner & Stephens, 2006). Underdeveloped social and emotional competencies can have consequences which are far reaching and long lasting (Daly, Delaney, Egan, & Baumeister, 2015; Durlak, 2015; Moffitt et al., 2011). Deficits in basic skills - such as the ability to identify emotions - can affect all stages of the lifespan, including: being rejected by others; exclusion from peer activities; being victimised; and lower peer-rated popularity (Lemerise & Arsenio, 2000; Leppänen & Hietanen, 2001; Mostow, Izard, Fine, & Trentacosta, 2002). Chronic physical aggression during primary school increases the risk of violence and delinquency through adolescence in boys (Broidy et al., 2003; Nagin & Tremblay, 1999), which in the long term can lead to destructive forms of emotion management, such as alcohol abuse. Poor self-regulation in childhood is associated with negative outcomes in adulthood including poorer health, social and economic outcomes (Compas, Connor-Smith, & Jaser, 2004; Daly et al., 2015; Kubzansky, Park, Peterson, Vokonas, & Sparrow, 2011; Moffitt et al., 2011), lack of (Nota, Soresi, & Zimmerman, 2004) and increased criminal behaviour (Henry, Caspi, Moffitt, Harrington, & Silva, 1999). Social and emotional learning (SEL) programmes aim to intervene early to address skill deficits and equip children with the social and emotional competencies they need for life (Eisenberg, Spinrad, & Eggum, 2010; Moffitt et al., 2011). Attention has increasingly turned to facilitating social and emotional learning in the classroom as a means to help children develop skills such as; empathy, emotional regulation and behaviour management strategies, that will enable them to become functioning members of society in adulthood (Humphrey, 2013; Humphrey, Lendrum, & Wigelsworth, 2010). Social and emotional learning (SEL) has been defined in different ways. Definitions tend to encompass a range of competencies, concepts and areas for development. For example, the American Collaborative for Academic, Social and Emotional Learning (CASEL, 2015) have described and defined five core competencies: self-awareness; self-management; social awareness; relationship skills; and responsible decision making. Waters & Sroufe (1983) also describe these social and emotional competencies as being important in enabling children “to generate and coordinate flexible, adaptive responses to demands and to generate and capitalize on opportunities in the environment” (p80). In the UK, The Young Foundation (McNeil, Reeder, & Rich, 2012) have identified a core set of evidence-based social and emotional capabilities shown to be important throughout the lifespan, including: communication; confidence and agency; planning and problem solving; relationships and leadership; creativity; resilience and determination; and managing feelings. Whilst core competencies vary in their definitions and scope between authors and agencies, a consistent pattern emerges demonstrating the array and reach of social emotional learning and its related competencies. It has been evidenced both nationally and internationally (Barry, Clarke, Jenkins, & Patel, 2013; Weare & Nind, 2011; Yoshikawa et al., 2015) that improving social and emotional learning allows children to connect with others and to begin to learn in a more effective way, thereby increasing their chances of success both in school and in life. Many countries have endeavoured to embed SEL into school ethos and culture. For example; in England personal and social development is addressed at the policy and practice level through the inclusion of building character and resilience as a priority by English Department of Education's (Department of Education, 2016); In America, CASEL work to support the integration of SEL in education through research, practice and policy; In Australia, SEL has been embedded in the national curriculum under personal and social capability (Collie, Martin, & Frydenberg, 2017). There is a growing consensus in academic and policy circles regarding the importance of children's social and emotional development and its links to behavioural and health outcomes (Ciarrochi, Deane, & Anderson, 2002; NICE, 2008; Petrides, Frederickson, & Furnham, 2004). Children who fail to achieve developmental milestones associated with SEL may be at risk of failing to make meaningful relationships with their peers and with the school situation (Zins, 2004; Zins & Elias, 2007). Social and emotional outcomes are related to educational outcomes (Duckworth & Seligman, 2005), emotional wellbeing (Eisenberg et al., 2010) and general life trajectories (Daly et al., 2015; Moffitt et al., 2011). Educationally, a range of social and emotional factors have been found to have an impact on educational achievement (Banerjee, Weare, & Farr, 2014; Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011), including willingness to learn, openness to new experiences and one's ability to interact with peers and teachers in the school situation. Given the increasing emphasis on SEL in both the educational research and policy arenas, there has been increased importance placed on SEL in schools and the need for facilitating optimal development of SE competence in every child. A variety of school-based programmes have been developed which aim to target, remediate and/or improve social and emotional skills in children, from the very beginning of their educational careers. Within the context of education and schooling, these programmes are particularly important for a number of reasons. Firstly, children do not learn in isolation, but rather construct meaning based on their life experiences (Vygotsky, 1980) and the relationships they have built can either facilitate or impede the learning process (Zachary, 2011; Zeidner, Matthews, & Roberts, 2012). Secondly, social and emotional skills are necessary antecedents for learning and constructing knowledge and can help ensure that children are ready, willing and able to learn. Thirdly, strong social and emotional skills enable children to work together, work with others and work alone in the school setting. There is evidence to suggest that certain groups of children are more likely to have difficulties in specific areas of social and emotional development. Children from socially disadvantaged communities and children who suffer maltreatment, are found to be less likely to develop social and emotional skills in line with their peers (Blair & Raver, 2012). Research on similarities and differences between social and emotional development in boys and girls is equivocal with some studies suggesting girls and boys develop in different ways (Lehmann, Denissen, Allemand, & Penke, 2013) while others conclude that these differences may be exaggerated (Else-Quest, Higgins, Allison, & Morton, 2012). Finally, age is likely to be a moderating factor. While early intervention has been shown to produce greater impacts this is not the case for all relevant outcomes (Sklad, Diekstra, Ritter, Ben, & Gravesteijn, 2012). Understanding the developmental trajectory of social and emotional skills and identifying key developmental stages may help identify the age at which interventions are likely to confer most benefit. This systematic review will focus on curriculum based SEL interventions delivered in preschool or primary/elementary schools aimed at improving social and emotional skills among pupils. It will include any universal programme, delivered on a whole-class or school basis. The interventions' primary goal must be to improve social and emotional competence and be delivered in a pre-school/kindergarten or primary/elementary school setting as part of the normal school day. The intervention must be structured, include a taught component and be delivered directly to children and involve the active participation of the child. The aims of the programme must be explicitly related to child gains, as opposed to those which focus solely on teacher competencies or school ethos. The intervention must also be a curriculum based social emotional learning program. Interventions may be delivered by the class teacher, other school personnel or non-school personnel. Interventions must be delivered for a minimum of one school term. Whilst interventions are varied in terms of their scope and their delivery, they typically adopt either a preventative or remediative approach and may use a range of theoretical approaches, such as Bronfenbrenner's ecological theory or the achievement model of emotional literacy (Rivers & Brackett, 2010). Many SEL interventions in preschool and primary/elementary school are based upon an implicit or explicit understanding of how social and emotional skills develop among children at this age. Rarely, however, do interventions outline a logic model which clearly sets out how the intervention creates or facilitates change in the desired outcomes. To this end, most interventions encourage and help children recognise and identify emotions or undesirable behaviours. Children are then taught, usually through scene setting, discussion and/or modelling techniques, skills to help them reflect on the problem and think through (and thus modify) the various and alternative actions they might take when faced with a difficult or conflict-related situation. A range of programmes have emerged that target children who are not meeting the developmental milestones associated with social and emotional learning. Some of these milestones include, for example: being able to understand complex and simultaneous feelings; consider multiple perspectives; empathise or sympathise with others; and recognise, regulate and express emotions effectively in order to create and sustain personal and social relationships. In so doing, SEL interventions aim to support children to enhance their positive emotions and behaviours, whilst also moderating their negative emotions and associated behaviours. Programmes such as PATHS (Promoting Alternative Thinking Strategies) are implemented widely in a number of countries and are designed to facilitate the development of self-control, emotional awareness and interpersonal problem-solving skills. PATHS does this by teaching children strategies to help them regulate their emotional response to a difficult situation by: stopping to reflect on the problem, thinking through alternative solutions and based on this, choosing an appropriate course of action. Here, the programme's aims and objectives are closely tied with the developmental achievements which have been set out in previous literature. Similarly, Roots of Empathy – a classroom based SEL intervention - seeks to promote prosocial behaviour and reduce aggressive behaviour by enhancing children's emotional recognition, empathy and emotional regulation. It is a structured curriculum-based programme that involves a mother and baby coming to class. Children observe the baby's development over the period of a school year and are taught to ‘recognise’ the baby's feelings and reflect on their own and others feelings. Learning about the baby's development, how the baby might be feeling, why they might be upset or happy and observing the loving parent-child relationship provides children with a model of responsible parenting from which to learn. The intervention relies on a mixture of modelling, discussion and storytelling to teach children about their own and others' emotions and behaviours. Based on the logic model suggested by CASEL, explicit instruction in SEL skills (provided through the intervention) combined with teacher instructional practices that are both integrated with the academic curriculum and school ethos, should lead to the acquisition of SEL skills, improved attitudes towards self, others and learning as well as an enhanced learning environment. In turn, these changes result in positive social behaviour, fewer conduct problems, lower emotional distress and improved academic performance. For the purpose of the current review, the co-primary outcomes will be behavioural, namely: improved prosocial behaviour and fewer conduct problems. Secondary outcomes will include: SEL skills (including emotional regulation, emotional recognition, empathy, problem solving); attitudes (including self-esteem, peer relationships, enjoyment of school); emotional distress and academic performance. Whilst certain background or demographic characteristics may moderate the impact of a programme on certain groups of children or families (previously discussed), other programme related factors – such as quality and fidelity of implementation – can also moderate effects. A number of programme factors emerge which could be seen to be important when implementing a programme. As mentioned in Clarke et al's (2015) recent review, parental involvement may be an area which facilitates success of an intervention. Here it was found that when parents were involved and knowledgeable about the process being undertaken children were more likely to make significant gains. Additional factors which may lead to greater successes include teachers and other school staff implementing the programme compared to instructors from outside the school (Durlak et al., 2011, p.13) which can allow children to make greater gains, perhaps because they feel calm and confident with the instructor. There is also some evidence to suggest that structured programmes which incorporate training for those delivering the program, are interactive in nature and which guide young people towards a specific set of goals are more likely to be successful (Smith, Schneider, Smith, & Ananiadou, 2004). Moreover, there is some evidence to suggest that multi-component programmes may have added benefits compared with single component programs (Adi, Killoran, Schrader McMillan, & Stewart-Brown, 2007; Catalano, Berglund, Ryan, Lonczak, & Hawkins, 2004; Humphrey et al., 2010; Wells, Barlow, & Stewart-Brown, 2003). Factors regarding programme design and implementation are also influential, with programmes which were identified as well-designed and well implemented being the most impactful (Durlak et al., 2011). Stage of programme development may affect the success of the intervention, with efficacy trials conducted under controlled conditions and often led by the programme developer resulting in greater effects than effectiveness trials conducted under more ‘real world’ conditions (Wigelsworth et al., 2016). These programme content and implementation factors will be documented in the coding framework and, if possible, analysed as moderators of intervention effect. The importance of SEL is now well recognised, and there has been significant growth in the number and type of SEL programmes offered in schools. Existing related reviews either do not focus on broad-based SEL programs or are currently out of date, or have included studies which used non-robust methodologies. The Campbell and Cochrane systematic review libraries were searched in December 2017 for completed and ongoing reviews relevant to this area. This search found a number of relevant ongoing or completed reviews: In the wider research literature, several other reviews have been conducted in the area of social and emotional learning (SEL) programmes (Browne, Gafni, Roberts, Byrne, & Majumdar, 2004; Clarke, Hussein, Morreale, Field, & Barry, 2015; Durlak et al., 2011; Payton et al., 2008; Sklad et al., 2012; Taylor, Oberle, Durlak, & Weissberg, 2017; Wigelsworth et al., 2016; Wilson & Lipsey, 2007). The most relevant of these is Durlak et al.'s (2011) meta-analysis, which focused on school-based programmes and their impact on a number of pupil outcomes including: social and emotional skills and attitudes; positive social behaviour; conduct problems; emotional distress; and academic performance. Durlak et al.'s searches were conducted up until the end of 2007 and thus the findings are now dated. Whilst more recent, Clarke et al.'s (2015) review focussed only on literature published in the UK, did not conduct meta-analyses and studies were included which did not include control groups. More recently, (Wigelsworth et al., 2016) examined the impact of developer involvement, trial stage and transferability on universal SEL interventions. This review represents an important step in explicating the complex relationships between efficacy and effectiveness, programme fidelity and transferability. However, it was limited to studies reported in English between 1995–2013, excludes pre-school programmes and excluded studies not reporting means and standard deviations, which misses an opportunity to identify gaps and bias in reporting of conducted trials. Finally, Taylor and colleagues (Taylor et al., 2017) conducted a meta-analysis of follow-up effects of SEL programs including studies reported in English up to December 2014. This review provides an analysis of the longer term impacts of SEL programs but does not include recent studies or studies in a language other than English. Since these reviews have been published, the review team is aware of a number of new evaluations conducted on interventions aimed at pupil emotional wellbeing and behaviour through universal social and emotional learning programmes. An up-to-date systematic review including more recently conducted studies is therefore needed. Additionally, whilst there are a number of ongoing or completed narrative reviews which focus on individual programmes, specific social and emotional outcomes or specific niches or facets of socio-emotional development, the reviews which have been (or are being) completed currently tend not to systematically review, map and assess the wide range of social and emotional learning programmes currently being used in schools for children aged 3–11. This proposed review will therefore be more wide-ranging and inclusive and will allow for the inclusion of more recent literature emerging in the field. Finally, this review is seeking to address a broader set of questions regarding the overall impact of universal school-based SEL programmes and, within this, has a particular focus on comparing the effectiveness of different types of intervention, and for different subgroups, in order to determine whether there are any underpinning programme-specific components that are associated with greater effects. These objectives require a broad-based review that has not been attempted by any of the reviews listed above. The objectives of the review are: Given the broad definition of social and emotional learning and the wide variety of school-based interventions that now exist, studies will be varied in terms of their design, methodology, implementation, outcomes and measurement. Four representative studies that could potentially meet the inclusion criteria for the current review, are detailed below. Randomised controlled trials or cluster randomised controlled trials will be included. We anticipate a large number of included studies and so can reasonably restrict our analysis to only the highest quality research designs. Children attending preschool or primary/elementary schools. This population is typically between the ages of three and eleven years. Where a study includes participants from both preschool or primary schools and secondary schools reasonable attempts will be made to extract or source the data for only the children attending preschool/primary/elementary school from published and unpublished documentation. If data us unavailable authors will be contacted to request summary statistics for only the participants of interest. Studies which are located in special schools or which focus on children with identified special educational needs or social, emotional or behavioural difficulties will be excluded. Studies will be included if the intervention is a universal, school- or classroom-based programme which is delivered, via a set curriculum, to a whole class or whole school and which primarily aims to improve social and emotional competencies of children. The intervention must be a curriculum-based social emotional learning programme. The intervention must be delivered directly to children and involve the active participation of the child. The aims of the programme must be explicitly related to child gains, as opposed to those which focus solely on teacher competencies or school ethos. Interventions may be delivered by the class teacher, other school personnel or non-school personnel. Interventions must be delivered for a minimum of one school term. Studies must include an inactive comparison condition that could include; Studies with an inactive control compared to two or more intervention arms can be included and the sample size of the control group will be divided by the number of eligible intervention arms to avoid double counting control group participants in any meta-analysis. The following types of studies will be excluded: Kratochwill et al. (Kratochwill, McDonald, Levin, Scalia, & Coover, 2009) represents an example of a study that is likely to be excluded. This study evaluated the Families and School Together (FAST) programme (McDonald et al., 1997). Here, children were recruited from schools and the programme delivered in the school setting. The programme aimed to improve a number of outcomes, including parental involvement, child behaviour and teacher perceptions of attainment. To do this it delivered eight weekly sessions where children and parents took part in games and activities together. Whilst this study does include some aspects of social and emotional development, the main focus was on parents who are working alongside their children. This is a multi-family, group intervention, rather than one which focusses on the child and their social and emotional development. As there is no direct taught element which is delivered to the child this study would not meet the inclusion criteria Primary outcomes Conduct problems. Secondary outcomes In SEL trials it is rare that adverse effects are considered and so we will not select adverse outcomes a priori. Instead we intend to extract any data on adverse outcomes and to provide a narrative summary of potential adverse effects. Outcome measures vary widely in terms of quality and validity. We will exclude treatment inherent measures as they risk overestimation of effect size (Slavin & Madden, 2008). The minimum standard will be that for any instruments used in included studies a full description of the scale and its scoring as well as associated reliability statistics are available. In order to find all eligible studies the following data sources will be searched and consulted. This comprehensive search will include multiple electronic databases, research registers, grey literature sources, and reference lists of reviews and relevant studies. We will not restrict the study selection in terms of language, date or publication status. Each of the following databases and trial registries will be searched using the search strings set out in appendix 1: Relevant reviews will be searched for in the following databases; We will also search grey literature and the following databases and websites will be used: Reference lists of relevant reviews and included studies will be screened and forward citation searching of included studies will be carried out through Google Scholar. These web searches will be limited to the first 2 pages of search results. Prominent authors in the field will also be contacted. In addition, a final step towards the end of analysis, a manual search of the most recent issue(s) of key journals will be conducted. To do this, the top 10 journals that have provided included studies so far, will be identified and their most recent issues will be checked. One reviewer will conduct the database searches, remove duplicates and obviously irrelevant records. We anticipate that the searches will result in a very large number of records to screen and so to ensure robustness, each report will be screened by title and abstract by two reviewers. Potentially eligible studies will then be retrieved in full text form and two reviewers will screen each full text. Any disagreements will be discussed with the wider review team until a consensus is reached. The search strategy has been developed using a modified version of the pearl harvesting approach. First, one author (JR) extracted keywords from ten randomly selected relevant studies. Each of the terms extracted were then searched individually within the thesauri in ERIC, British Education Index and Psycinfo and all additional terms added to the list of possible search terms relating to either: the participants; intervention setting; study design; or outcomes of interest. All authors then suggested additional terms that had not been identified (e.g. nursery school did not appear in any article of thesauri). Two authors (JR and JH) then screened the full list of terms and removed any duplicate, overlapping or irrelevant terms. The search string samples below therefore were generated by a combination of terms originating from the literature, terms originating from the review team and terms originating from a thesauri search. A combination of four search strings will be used relating to: 1) participants; 2) the intervention setting; 3) the study design; and 4) the outcomes of interest. A sample search strategy is provided in appendix 1. Search strings and search limits will be modified to be suitable for each database. Search for exact phrases or proximity searching will be used to increase search specificity. In addition searches for a number of known named interventions will also be undertaken (see appendix 1). All included studies will be randomised or cluster-randomised controlled trials. It is important to ensure that the effects of an individual intervention are only counted once and the following conventions will therefore apply. Where there are multiple measures reported for the same outcome, this will be dealt with by calculating an average effect size within each study for each outcome. A simple average effect size will be calculated by first calculating the effect size for each measure of a given outcome and then averaging these effect sizes within each study. The exception will be any treatment inherent measures of the outcome of interest, these measurements will be discarded as they risk overestimating the treatment effect. Where the same outcome construct is measured but across multiple time domains, such as through the collection of both post-test and further follow-up data, the main analysis will focus on synthesising the evidence relating to effect sizes at immediate post-test. Any subsequent measures of outcomes beyond immediate post-test will be meta-analysed and reported separately. Studies comparing multiple treatment and control arms will be discussed with the full author team to decide if eligible intervention arms are similar enough to combine and compare as if they are one intervention group. If not, each intervention arm will contribute separate effect sizes to the meta-analysis and the control group sample size will be split by the number of intervention arms included to avoid double counting of control participants. In the case of multiple cohorts appearing in one study we will calculate a simple average, as described above, for the omnibus meta-analysis. If different cohorts in a study fall into different subgroups then they will be considered separately in subgroup analysis but no overall summary of effect will be calculated combining subgroups in those cases. If there are sufficient eligible studies reporting multiple and dependent effect sizes (i.e. occurring in more than 20 eligible studies) then robust variance estimation will be employed. This technique calculates the variance between effect sizes to give the variable of interest a quantifiable standard error. It has been shown to calculate correct results with a minimum of 20–30 individual studies (Hedges, Tipton, & Johnson, 2010) although it performs better with an increased quantity of studies. Once eligible studies have been found, an initial analysis of programme descriptions will be undertaken for all eligible programmes. This will be used to identify the core components of programmes and to develop an overarching typology and coding frame for these. From previous studies, and as referenced in the earlier discussions of existing literature, it is anticipated that such components are likely to include: Alongside extracting data on programme components, descriptive information for each of the studies will be extracted and coded to allow for sensitivity and subgroup analysis. This will include information regarding: A coding framework has been devised and piloted (see appendix 2) and will be refined as necessary based on the above. Coding will be carried out by trained researchers. Each study will be coded by two members of the review team independently and discrepancies will be discussed and a consensus agreed. Quantitative data will be extracted to allow for calculation of effect sizes (such as mean cha