Comprehensive School Reform Program Research Articles

The effects of Success for All in the Netherlands on the reading achievement of first-grade students at risk of reading problems

This article reports outcomes of a quasi-experimental evaluation of Success for All (SfA), a comprehensive school reform program that has recently been introduced in the Netherlands. The Response to Intervention framework is used to describe how SfA supports students at different Tiers. The effects of SfA on five reading subskills were investigated for first-grade students at risk of reading problems. 299 students from two different cohorts were involved. Multilevel analyses demonstrated a significant effect of SfA on reading comprehension (ES = +0.26) in the first cohort. For the second cohort and the other reading subskills, mostly small positive effects of SfA were found, though these effects were not statistically significant. Furthermore, the relationship between tutoring intensity and reading achievement was examined. In the second cohort, a significant negative association of tutoring intensity with word and text reading skills was found. Implementation issues that may have impacted the outcomes are discussed.

Promoting students’ social behavior in primary education through Success for All lessons

Success for All (SfA) is a comprehensive school reform program with a strong emphasis on cooperative learning that aims to improve students’ social emotional learning alongside students’ cognitive learning. In the present study it was examined whether SfA led to improved students’ social behavior in Grade 1–3 of primary education. Peer sociometric data was collected for 974 students aged 6–9. Using multivariate multilevel analysis we found no significant effect of SfA on students’ pro- and antisocial behavior over time. However, a significant interaction effect was found showing that antisocial behavior of students from disadvantaged backgrounds decreased in the intervention condition in Grade 2. This is a promising finding given that the SfA program was especially developed for schools serving large numbers of disadvantaged students. Implications of the study are described.

Accommodating Change: Relating Fidelity of Implementation to Program Fit in Educational Reforms

Asking practitioners to make larger changes to their practice is often thought to lead to lower fidelity of implementation. However, salient differences between ambitious new reforms and teachers’ existing practices may also facilitate processes of conceptual change and correspondingly increase fidelity of implementation. I use survey data on the implementation of two Comprehensive School Reform programs to investigate this puzzle, presenting a series of descriptive multivariate regressions that—contrary to conventional wisdom—predominantly support a positive association between larger changes and higher fidelity. I also address alternative explanations for this finding and discuss the conceptual and empirical strengths and weaknesses, implications for future research, and potential utility for practice of each interpretation.

Riding the Implementation Curve: Teacher Buy-In and Student Academic Growth under Comprehensive School Reform Programs

AbstractWhile teachers have long been acknowledged as a key part of educational innovation, research has largely neglected the impact that their buy-in has on reform initiatives. This study addresses this problem by examining the association between teacher buy-in and student academic growth under school reform programs, with a particular focus on how that relationship changes over time. Using multiyear data from the Study of Instructional Improvement, we arrived at two important findings. Counterintuitively, we found that teacher buy-in is negatively associated with student achievement in general. However, after controlling for program age, we also found that, in schools with more mature reform programs, higher teacher buy-in had a significant positive relationship with students’ academic growth in all sampled subject areas. These findings provide useful insights into school reform policies, as well as important empirical evidence on the actual role of teacher buy-in in students’ achievement.

The Impact of a Comprehensive School Reform Policy for Weak Schools on Educational Achievement; Results of the First 4 years

This paper estimates the effects of a comprehensive school reform program on high-stakes test scores in Amsterdam. The program implements a systematic and performance-based way of working within weakly performing primary schools and integrates measures such as staff coaching, teacher evaluations and teacher schooling, and the use of new instruction methods. Difference-in-differences estimates show substantial negative effects on test scores for pupils in their final year of primary school. The program decreased test scores with 0.17 standard deviations in the first 4 years after its introduction. A potential explanation for this finding is the intensive and rigorous approach that caused an unstable work climate with increased teacher replacement.

HIGH SCHOOLS THAT WORK: PROGRAM DESCRIPTION, LITERATURE REVIEW, AND RESEARCH FINDINGS

ABSTRACTInterest in setting high educational standards is at the forefront of many school reform programs in the United States. States' pursuit of high standards can be advanced through implementation of programs such as High Schools That Work (HSTW), which emphasizes high expectations and rigorous academics for all students, including Career and Technical Education students. High Schools That Work is the largest comprehensive school reform program for high schools in the United States, with over 1,000 schools in more than 30 states currently participating. This report describes the program, previous research, and recent research studies conducted by ETS to ensure validity for the HSTW program.

Reconsidering replication: New perspectives on large-scale school improvement

The purpose of this analysis is to reconsider organizational replication as a strategy for large-scale school improvement: a strategy that features a “hub” organization collaborating with “outlet” schools to enact school-wide designs for improvement. To do so, we synthesize a leading line of research on commercial replication to construct a “knowledge-based logic” focused on the production, use, improvement, and retention of effective practices in large numbers of schools. Drawing on findings from a longitudinal case study, we then use the knowledge-based logic to structure an interpretation of Success for All, a leading comprehensive school reform program. In contrast to common assumptions of organizational replication as a strategy that yields rapid results at the expense of local and professional control, we argue that organizational replication can be understood as a long-term enterprise in which program providers and schools collaborate to produce, use, improve, and retain practical knowledge. Capitalizing on this potential, however, is contingent on both proponents and critics re-examining common assumptions about organizational replication and recognizing value in replication enterprises that they would otherwise miss.

Embedded Teacher Learning Opportunities as a Site for Reflective Practice: An Exploratory Study

Framed by sociocultural learning perspectives, this study examines whether embedded learning opportunities for teachers are more supportive of reflective practice than traditional professional development. All schools in this study implemented a comprehensive school reform program. The designs of the programs introduced embedded learning opportunities for teachers by adding teacher leadership positions and encouraging various forms of teacher collaboration. The results indicate that these two kinds of embedded learning opportunities are positively and strongly associated with teacher reflection, and the effects of these variables were twice the size of the effect of traditional professional development. Implications and limitations of the results are discussed.

Allocating More Experienced Teachers to Leadership Positions in Schools: A Double-Edged Sword?

This article presents a critical discussion of the implications of allocating the most experienced and skilled teachers in schools to teacher leadership positions. Placing such teachers in leadership positions supports school improvement initiatives, but it can also pull valuable teaching resources from the classroom. The article examines this issue by investigating how expertise is distributed between teacher leaders and regular classroom teachers in urban schools implementing comprehensive school reform programs. Results indicate that teacher leaders have substantially more postsecondary training and teaching experience than do nonleaders. The implications of these results are discussed.

Accountability Nudges Districts into Changes in Culture

We may never be able to show statistically the school district's contribution to student achievement, but we know the district plays a key role in improving teaching and learning. Without adequate district support, changes that focus solely on the school, like comprehensive school reform programs or restructuring efforts, often fail or only limp forward. Most districts have better links to professional networks than their schools because they have specialists to track particular specialties. So the district is in a superior position to identify useful innovations, decide which ones to ignore, and mobilize the intellectual and financial resources needed for improvement efforts. Perhaps most important, accountability pressures have been increasing in education for at least a quarter century. However, though policy makers might try to change it, the district is usually the legal point of accountability for most things in this country. Schools rarely are directly subjected to state and federal policy; the district filters those policies (Fairman and Firestone 2001). So, how districts interpret and act on accountability policy matters. Although there is more research on what promotes school effectiveness than on what makes an effective school district, work in the area is growing. In the last decade, a number of case studies of unusually high-achieving districts have yielded useful clues, as has research on specific levers through which districts can influence teaching and learning. Districts develop distinctive cultures that can affect how those levers get used. The most typical has been the loosely coupled culture in which districts have relatively little influence on practice. The rise of the accountability movement--as exemplified by No Child Left Behind (NCLB)--has brought about an accountability culture in many districts that creates greater coherence through centralized control. However, the accountability culture is not as effective as the student learning culture for promoting achievement. What takes a district from accountability to the student learning culture is a mix of board and community support and leadership from the top. DISTRICT CULTURES When people talk about cultures in education, they usually mean something like a pattern of shared assumptions that is learned by solving important problems. But school district cultures have three points in common: 1. District culture will concern at least two areas: Teaching and learning--that is, expectations for students and beliefs about what they can learn--and how people will work together. 2. Although cultures are usually defined by shared expectations, the extent of sharing may vary. Especially in schools and districts, cultural themes are unevenly represented. 3. A district's culture is located at the top, so an ongoing challenge is how to spread constructive cultures to every school. Although each district has its own culture, three are quite typical. Loosely Coupled Culture. The term, loosely coupled culture, reflects 40 years of research suggesting that most school districts have few common goals, shared beliefs, or effective levers for coordinating activity. A loosely coupled district lacks a shared vision for teaching and learning. A district might, for example, focus on improving the physical plant and the cafeteria system, rather than emphasizing student learning (Rosenholtz 1989). These districts don't attempt to send a common message to schools. Departments in the district, such as the curriculum office and the Title I office, often send divergent messages to schools. These districts often adopt and drop improvement initiatives quickly with no effort to coordinate new programs; initiatives often follow the money rather than a coherent philosophy (Spillane 1998). Loosely coupled districts fail to use the levers they have to influence instruction. They may not have a central curriculum; if they do, teachers don't feel required to follow it (Floden 1988). …

Improving Low-Performing High Schools: Searching for Evidence of Promise

Noting that many of the nation's high schools are beset with major problems, such as low student reading and math achievement, high dropout rates, and an inadequate supply of effective teachers, Steve Fleischman and Jessica Heppen survey a range of strategies that educators have used to improve low-performing high schools. The authors begin by showing how the standards-based school reform movement, together with the No Child Left Behind Act requirement that underperforming schools adopt reforms supported by scientifically based research, spurred policy makers, educators, and researchers to create and implement a variety of approaches to attain improvement. Fleischman and Heppen then review a number of widely adopted reform models that aim to change "business as usual" in low-performing high schools. The models include comprehensive school reform programs, dual enrollment and early college high schools, smaller learning communities, specialty (for example, career) academies, charter high schools, and education management organizations. In practice, say the authors, many of these improvement efforts overlap, defying neat distinctions. Often, reforms are combined to reinforce one another. The authors explain the theories that drive the reforms, review evidence of their reforms' effectiveness to date, and suggest what it will take to make them work well. Although the reforms are promising, the authors say, few as yet have solid evidence of systematic or sustained success. In concluding, Fleischman and Heppen emphasize that the reasons for a high school's poor performance are so complex that no one reform model or approach, no matter how powerful, can turn around low-performing schools. They also stress the need for educators to implement each reform program with fidelity to its requirements and to support it for the time required for success. Looking to the future, the authors suggest steps that decision makers, researchers, and sponsors of research can take to promote evidence-based progress in education.

A Study of Comprehensive School Reform Programs in Arizona

Background/Context The U.S. federal government has been interested in improving the performance of students who come from low-income homes since the time of Lyndon Johnson's “Great Society” initiatives in the 1960s. The current administration strongly supports the belief that good schools can be created and has funded the Comprehensive School Reform Program (CSR) to support these beliefs. This article briefly reviews literature related to school reform throughout this period, describes our research plan to study CSR elementary schools in the state of Arizona, and introduces the articles included in this special issue. Focus of Study The basic intent of this study was to inform working theories of learning, motivation, and social/emotional development in school contexts in Grades 3–5. We hoped that an emphasis on theory, contextual enactment, and participant mediation would yield a richer picture of classroom practices and motivational dynamics that might underlie student achievement and CSR effectiveness. This study focused in particular on participant perspectives (principals and students) and classroom practices associated with CSR programs in elementary schools in the state of Arizona. Research Design The research program includes interview (with principals), observation (of classroom practices), and survey measures and an adaptation of Thematic Apperception Test procedures (with students). Conclusions/Recommendations School reform initiatives can profit from more research on participant perceptions, actual classroom practices, and student mediation of those practices. These understandings can better link program design and student achievement to enhance the effectiveness of comprehensive school reform initiatives.

Comprehensive School Reform Instructional Practices Throughout a School Year: The Role of Subject Matter, Grade Level, and Time of Year

Background/ContextThe achievement effects of Comprehensive School Reform (CSR) programs have been studied through the use of input-output models, in which type of CSR program is the input and student achievement is the output. Although specific programs have been found to be more effective and evaluated more than others, teaching practices in CSR schools have received less attention. This study focuses on observations of math and reading/language arts lessons in classrooms implementing an array of CSR programs to better understand what occurs in CSR classrooms.Research QuestionThis article describes observed instructional practices and teacher-student dynamics that occurred in CSR classrooms in two different subject areas, reading/language arts and math, in Grades 3–5. Reading/language arts included several literacy-related areas such as spelling, vocabulary, phonetics, and writing. Math primarily consisted of computations and math applications. Our primary research questions were: (1) Does subject matter matter in CSR classrooms? (2) How does instruction in CSR classrooms differ among Grades 3, 4, and 5? (3) Are there major differences in classroom practices between fall and spring?PopulationTeachers (N = 104) in Grades 3–5 in 16 CSR schools, totaling 248 observation periods in math and reading/language arts lessons.Research DesignObservational study using a systematic coding system to observe student/teacher classroom behaviors and dispositions.Data AnalysisData were analyzed using basic descriptive statistics and analysis of variance procedures.ConclusionWe found that students were productively involved in assigned tasks and that classrooms were pleasant and task oriented in both mathematics and reading/language arts. Some subject matter differences were notable, particularly that math lessons were more structured and rigid than were reading/language arts lessons. Also of interest, third- and fifth-grade classrooms experienced more positive teacher-student relationships than did fourth-grade classrooms. Furthermore, instruction in the fall was more structured and more focused on basic skills than in the spring. Overall, students appeared to be engaged in learning basic facts/skills in an uninterrupted teacher-directed classroom. Students did what was asked of them, were given little choice about their social and academic tasks, and were in mostly comfortable classroom environments.

What We Learned about Research on School Reform: Considerations for Practice and Policy

Background/Context The U.S. federal government has been interested in improving the performance of students who come from low-income homes since the time of Lyndon Johnson's “Great Society” initiatives in the 1960s. The current administration strongly supports the belief that good schools can be created and has funded the Comprehensive School Reform (CSR) program to support these beliefs. This article provides information about recent school reform research and conditions of schooling. The article then reviews our research findings (drawing on all the preceding articles in the special issue) and considers implications for policy makers, principals, teachers, teacher educators, and researchers. Purpose The basic intent of this study was to inform working theories of learning, motivation, and social/emotional development in school contexts in Grades 3–5. We hoped that an emphasis on theory, contextual enactment, and participant mediation would yield a richer picture of classroom practices and motivational dynamics that might underlie student achievement and CSR effectiveness. This study focused in particular on perspectives (principals and students) and classroom practices associated with CSR programs in elementary schools in the state of Arizona. Research Design The research program includes interview (with principals), observation (of classroom practices), and survey measures, and an adaptation of Thematic Apperception Test procedures (with students). Conclusion/Recommendations School reform initiatives can profit from more research on participant perceptions, actual classroom practices, and student mediation of those practices. These understandings can better link program design and student achievement to enhance the effectiveness of CSR initiatives. We believe that it is now possible to conduct field experiments for improving normative (typical) practice in elementary school classrooms. We offer considerations for doing this research and suggest the need for attention to students, both as social beings and learners, while altering classroom practices in small measured steps.

Do Your Homework! Investigating the Role of Culturally Relevant Pedagogy in Comprehensive School Reform Models Serving Diverse Student Populations

Like the African proverb, ‘It takes a village to raise a child’, many educational researchers charge that it takes a comprehensive school reform to raise student achievement. With the passing of the No Child Left Behind legislation in 2002, national officials authorized the Comprehensive School Reform program to support low performing schools as they struggled to improve student achievement. As a result of this national effort, an increase in implementation of comprehensive school reforms is occurring in schools serving predominantly diverse student populations in urban areas. Therefore, this article explores the framework of comprehensive school reforms and challenges stakeholders to do their homework by investigating whether the school reform allows for the implementation of culturally responsive educational experiences for students.

Opening Up the Black Box: Literacy Instruction in Schools Participating in Three Comprehensive School Reform Programs

This study examines patterns of literacy instruction in schools adopting three of America’s most widely disseminated comprehensive school reform (CSR) programs (the Accelerated Schools Project, America’s Choice, and Success for All). Contrary to the view that educational innovations seldom affect teaching practices, the study found large differences in literacy instruction between teachers in America’s Choice schools and comparison schools and between teachers in Success for All schools and comparison schools. In contrast, no differences in literacy teaching practices were found between teachers in Accelerated Schools Project schools and comparison schools. On the basis of these findings and our knowledge of the implementation support strategies pursued by the CSR programs under study, we conclude that well-defined and well-specified instructional improvement programs that are strongly supported by on-site facilitators and local leaders who demand fidelity to program designs can produce large changes in teachers’ instructional practices.

Organizational Strategies for Promoting Instructional Change: Implementation Dynamics in Schools Working With Comprehensive School Reform Providers

This article develops a conceptual framework for studying how three comprehensive school reform (CSR) programs organized schools for instructional change and how the distinctive strategies they pursued affected implementation outcomes. The conceptual model views the Accelerated Schools Project as using a system of cultural control to produce instructional change, the America's Choice program as using a model of professional control, and the Success for All program as using a model of procedural control. Predictable differences in patterns of organizing for instructional improvement emerged across schools working with these three programs, and these patterns were found to be systematically related to patterns of program implementation. In particular, the two CSR programs that were organized to produce instructional standardization produced higher levels of instructional change in the schools where they worked. The results of the study suggest organizational strategies program developers can use to obtain implementation fidelity in instructional change initiatives.

Implementation: Measuring and Explaining the Fidelity of CSR Implementation

Comprehensive school reform (CSR) is only as effective as its implementation. By using data collected for the National Longitudinal Evaluation of Comprehensive School Reform (NLECSR), this article explores the factors that predict CSR model implementation and the ways that CSR model implementation varies. We found little difference in the fidelity of implementation between schools implementing a CSR program and their matched comparison schools, but we found vast differences in the fidelity of implementation between different CSR program implementation keys (the normative guidelines received from CSR program developers on which we based our implementation measures). The fidelity of implementation is predicted by multiple factors, including the CSR program implementation keys, the principal's instructional leadership, the measure of teachers' professional community, and the usefulness of the CSR developers' assistance. The change in implementation is associated with positive change in principal's instructional leadership and usefulness of the CSR developers' assistance.

A Case Study of the Partnership Schools Comprehensive School Reform (CSR) Model

This case study reports the feasibility of the Partnership Schools Comprehensive School Reform (CSR) model for school improvement in a Title I elementary school. Interviews were conducted and documents were collected for 3 years to study whether and how the school implemented key policy attributes—specificity, consistency, authority, power, and stability—that have explained the successful implementation of other CSR programs. The data also identified if and how the school implemented essential elements of teamwork, leadership, action plans, implemented activities, evaluation, and networking, which have explained improvements in programs of family and community involvement. With the implementation of the model, the case study school increased the number of families involved in students’ education at school and at home. Longitudinal achievement test scores showed that the CSR school improved the percentage of students attaining proficiency by state standards compared to schools that were matched, one each, for math, reading, and writing. The CSR school also closed its gap in test scores with the district as a whole, despite the fact that the district included several schools in more affluent neighborhoods with higher test scores in the base year. The study revealed a new factor for program development, transitioning, which extends the existing frameworks on program implementation by requiring plans and decisions about continuing a program or parts of it before the end of the CSR grant.

State-Level Support for Comprehensive School Reform: Implications for Policy and Practice

In the current context of standards-based reform and heightened accountability for school performance, state education agencies (SEAs) have an important, but not yet well-articulated, role to play in local school improvement efforts. This article starts to articulate such a role by examining the variety of approaches and strategies used by 7 SEAs to support schools and districts applying for and ultimately implementing the federal Comprehensive School Reform (CSR) program. Building from the assertion that SEAs can and do make choices about how to best implement federal programs, we explore the choices that the studied SEAs have made around (a) the integration of the CSR program with other state and federal initiatives, (b) the different levels of funding provided to schools, (c) the continuity of SEA leadership around CSR, and (d) the variety of support strategies provided to participating schools. Although these SEAs have supported CSR in different ways, we conclude that the most important state strategy for leveraging systemic advantage from CSR involves the extent to which SEA staff have stayed with the program and thus gained in-depth knowledge of the program necessary to improve CSR in their states.