High-stakes Accountability Systems Research Articles

Unveiling teachers’ work preferences: A conjoint experiment on the implications of school governance reform across three countries

School governance reforms have changed teachers’ work in many aspects and have been associated with increasing teachers’ discontent and demotivation. Research on which school policies and organizational practices teachers prefer is scarce and faces challenges. Our conjoint experiment identifies the importance given by teachers to different work dimensions altered by recent reforms and teachers’ preferences regarding school policies in three contexts. Internationally-shared preferences include qualitative teaching assessments, socially mixed classes, clear goal-setting, and collective rewards. Context-specific preferences include individual incentives in Chile, a low-pay, high-stakes accountability system, and peer support in Norway and Catalonia, which are systems with collaborative governance traditions.

Confronting the disadvantage gap: the challenges to transformative leadership in a high-stakes assessment system

ABSTRACT Students with low socio-economic status (SES) often have lower levels of academic achievement. In England, various UK governments have sought to address this disadvantage gap through reforms to national large-scale assessments, school accountability mechanisms, and educational governance structures. While scholarly research highlights the important mediatory role of educational leaders in policy implementation and student performance, typologies of successful leadership tend to emphasise ‘what works’ at the organisational level. Conversely, theories of transformative leadership acknowledge that efforts to provide an equitable education are related to, and should address how to transform, wider social power structures. With an abductive methodology, this qualitative study explores the experiences and perspectives of three academy headteachers in their work with the most disadvantaged students in their school communities. Semi-structured interview data were analysed thematically to evaluate whether and how their leadership practices might be considered transformative. The findings suggest that these headteachers are mostly transformative at the individual level which aligns with the current UK government social mobility agenda. Moreover, despite their courage to promote equity of opportunity, transformative leadership is restricted by government cuts to public expenditure, a high-stakes accountability system focused on students’ academic performance, and the limited political capital of students and headteachers.

Why do opt-out movements succeed (or fail) in low-stakes accountability systems? A case study of the Network of Dissident Schools in Catalonia

External and standardized assessments based on student results are a contested education policy among school actors. Movements of opposition have emerged in different countries, especially in those contexts with high-stakes accountability systems. However, this phenomenon has not been analyzed in soft accountability systems. The objective of this article is to study the opt-out movement in Catalonia, understood as an anti-standardization movement in a system of soft accountability. In order to do so, we adopt the case study approach as a methodological strategy, based on the triangulation of semi-structured interviews with activists (n = 14), key stakeholders (n = 3), and document and press analysis (n = 25). The results shed light on the emergence and nature of the movement, its opportunity structures, the discursive frames and the repertoires of collective action. Our results show how accountability instruments have a ‘life of their own’ beyond their policy design. In this sense, the opt-out movement in Catalonia identifies potential risks and adverse effects similar to those reported in high-stakes systems, developing a repertoire of collective action and discursive frames similar to other emerging anti-standardization movements in high-stakes contexts.

How are the ‘losers’ of the school accountability system constructed in Chile, the USA and England?

ABSTRACT Performance-based accountability systems that rank schools based on their effectiveness produce ‘winners’ and ‘losers’. Substantial evidence has pointed to the (side)effects of these classifications, particularly in the most disadvantaged communities. Whilst previous studies have compared schools under different effectiveness categories within and between countries, this qualitative study takes a cross-case comparison approach to analyse education policies, grey literature and previous research evidence to explore the mechanisms that construct ‘failing’ schools in three notable high-stakes accountability systems worldwide: Chile, the USA and England. After describing (1) the identification and classification of ‘failing’ schools; (2) the instruments used to justify these classifications; and (3) who make these judgements, we conclude that the construction of ‘failing’ schools serves the competition and differentiation required for maintaining neoliberal hierarchical and unequal market-oriented regimes. Instead of disciplining only ‘failing’ schools, these labels also provide a caution for the whole school system.

School Organizational Effectiveness and Chronic Absenteeism: Implications for Accountability

ABSTRACTChronic absenteeism in K-12 schools is strongly associated with critical educational outcomes such as student achievement and graduation. Yet, the causes of chronic absenteeism are complex, with environmental, family/individual, and school factors all affecting the likelihood of a student attending school regularly. This exploratory study examines whether school organizational effectiveness has the potential to moderate external influences on chronic absenteeism. Using school-level scores from the 5Essentials surveys, we find that, in traditional public schools, schools that are organized for effectiveness have lower rates of chronic absenteeism, while controlling for student demographics and grade level. In particular, schools with higher scores for “involved families” have lower chronic absenteeism. While charter schools in Detroit have significantly lower rates of chronic absenteeism than traditional public schools, we did not find an association between organizational effectiveness and chronic absenteeism in charter schools. This suggests that student sorting by school type may produce variation in chronic absenteeism rates that is not moderated by school actions. These findings have important implications for practice and policy, as educators seek to reduce chronic absenteeism in response to pressures from high-stakes accountability systems.

When the Achievement Gap Becomes High Stakes for Special Education Teachers: Facing a Dilemma with Integrity

Context A fundamental assumption behind a high stakes accountability system is that standardized testing, proficiency goal setting for demographic student subgroups, and sanctions would motivate teachers to focus on students whose performance had heretofore lagged. Students with disabilities became one such subgroup under the No Child Left Behind system. Special education teachers faced a novel pressure: to radically narrow the achievement gap between their students with disabilities towards proficiency or incur sanctions and corrective action for their schools and districts. Purpose The study uses the concept of “integrity” to analyze public service workers’ agency in situations of strain or crisis. Integrity consists of four overlapping domains of judgment: obligations of office, personal integrity, client needs, and prudence. Research Design The study is an in-depth multiple case study of seven teachers; 21 structured interviews, and 17 observations, augmented by a number of informal contact that included invitations to observe teacher meetings and conversations with school administrators. Findings The study found that the special education teachers faced a true dilemma. Teachers adopted contradictory solutions — some embraced the new demands, some rejected them. Both seemed equally untenable. The study reveals salient dimensions of this dilemma: how teachers related to the external moral obligation to equalize, what they chose to ‘see’ when they viewed the achievement gap; how they explained, or explained away, their agency in narrowing the gap; how they strategized and muddled through with instructional maneuvers to make the gap go away; and what they regarded, and guarded, as fields of professional responsibility and autonomous decision making. Implications What kind of accountability system would enable a collective dialogue among special education teachers in which high expectations, keen diagnosis, instructional expertise, internal responsibility for individualized learning gains, openness to external challenge, and attention to results would be the poles of the discussion? At the core, such an accountability system would validate the professionalism of the most expert teachers and avoid activating their defensiveness and demoralization. It would guard against middling expectations by making the performance of a wide spectrum of high and low performing schools or special education departments transparent. It would stay away from high pressure attached to unrealistic goals in order to discourage teachers from developing blind spots about their students, or acting with mere compliance and expediency. It would motivate a dynamic of student-centered continuous improvement in reference to a common standard, but also to low-stakes metrics that may guide iterative improvement.

Long-Term English Language Learners’ Educational Experiences in the Context of High-Stakes Accountability

Background/Context A large number of emergent bilinguals, also known as English language learners (ELLs), in secondary schools have experienced academic difficulties, grade repetition, inappropriate referrals to special education, and dropout. They are the most in danger of academic failure and in need of highly qualified instruction that meets their needs; however, they have received little attention from schools. To better understand secondary emergent bilinguals’ academic challenges, it would be necessary to examine their learning experiences throughout schooling, including language support programs and services provided, instructional practices and academic support implemented, and assessments administered to make their educational decisions. In particular, as the high-stakes accountability system has played a critical role during their course of schooling, it would be essential to examine how the accountability system has interacted with these underachieving emergent bilinguals. Purpose/Focus of Study This manuscript draws from a larger qualitative study (Kim, 2013) that investigated long-term ELLs’ history of schooling and their perceptions of language and academic learning experiences. The purpose of this analysis is to more closely examine the nature of language support received during participants’ K-12 schooling and their program placements in the context of their performance on state-mandated language and academic achievement tests. Experiences of participants who were retained and referred to and/or placed in special education serve to illustrate schools’ responses to these underachieving emergent bilinguals. Setting/participant This research study took place at a high school in central Texas. Eleven secondary emergent bilinguals who met the purpose of this research project participated in the study. The essential attributes of participants for inclusion were English language learners who (a) met the state eligibility criteria for classification as limited English proficiency, (b) had attended public schools in the United States for seven years or more, and (c) had at least a year of high school experience. Research Design A qualitative research design based on a constructivist, Naturalistic Inquiry (NI) paradigm was utilized to understand the reality of participants’ school challenges. This NI approach is appropriate to analyze concepts and themes derived from the exploratory cases of this population. Data Collection and Analysis Data sources were individual, semi-structured, and in-depth interviews and a variety of school documents that included each student's cumulative folder, Language Proficiency Assessment Committee (LPAC) documents, and academic assessment records. As the constant comparative method of data analysis was a core function of this research process, data analysis occurred concurrently with data collection. Triangulation among students’ recollections of their program placements, archival data, and informant (i.e., district bilingual and ESL professionals) interviews was used to ensure that the findings accurately reflected the actual phenomenon. Member checks, reflexive journaling, and peer debriefing were also utilized to ensure the trustworthiness of the study. Findings Participants in this study experienced multiple layers of limited opportunity to learn as they moved through the educational process. They began their schooling with inadequate bilingual education services in elementary grades, and many of them were retained and/or referred to special education, mainly due to their unsatisfactory scores on state-mandated assessments. Subsequently, participating ELLs all moved up to middle school where they received little English language support; their continued low scores on high-stakes assessments led to placement in remedial programs or intervention courses for state mandated tests. During their high school years, this pattern continued with the addition of End-Of-Course designated courses, remedial courses for preparing for state mandated tests, and credit recovery programs. Though state assessments should not be exclusively used to measure student learning, they were solely used to participating ELLs’ high-stakes instructional and placement decisions (e.g., retention, placement in remedial programs), which appeared to negatively affect participants’ educational trajectory by increasing the gaps in their opportunities to learn. Conclusions/Recommendations The participating ELLs’ experience of schooling fostered academic gaps due to their limited opportunities to learn, and they had been inadvertently excluded from the formal curriculum because schools in the district did not have adequate programs and services to address these learners’ linguistic and academic needs. Informed by social capital framework research and theory, findings suggest that participants’ gaps in learning continued to grow with each subsequent year of schooling, exacerbated by their limited access to appropriate language services and educational support, thereby rendering them struggling, low-achieving, long-term ELLs. Despite their academic challenges, participating ELLs remained eager to succeed in school, which raises a critical question regarding how well the educational system is prepared to provide these emergent bilinguals with high quality, rigorous programs that are responsive to their linguistic and academic needs.

The “Datafication” of Teaching: Can Teachers Speak Back to the Numbers?

Teachers face considerable and increasing pressure in their working lives. Labor intensification compels teachers to work faster, harder, and longer. However, teachers also experience increasing external control over what they teach and how they teach. These processes are increasingly made possible by the “datafication” of teaching, whereby the educational process is increasingly transformed into numbers that allow measurement, comparison, and the functioning of high-stakes accountability systems linked to rewards and sanctions. Although there is no question that being able to use student assessment data to support learning has an important place in teachers' repertoire of skills, “datafication” refers to the use of data in a way that has become increasingly detached from supporting learning and is much more concerned with the management of teacher performance as an end in itself. This article presents two currents of critical thought in relation to teachers' work, labor process theory and poststructural analyses grounded in the concept of performativity, and discusses them as a way of “making sense” of teachers' work and the “datafication” of teaching, with a particular focus on questions of control and resistance. It seeks to understand why, despite the pressures on teachers, teacher resistance has seldom developed in ways, at times, or on a scale that both experience and theoretical insight might have predicted. There are clearly significant differences between the two perspectives presented in this article, not least in the ways they conceptualize and explain “resistance.” However, common ground is identifiable and the two theoretical approaches can be bridged in a form that can be productive for those seeking to “speak back to the numbers.” In looking to broker this theoretical divide, I argue that frame theory, rooted within the sociology of social movements, can offer a fruitful way of theory bridging, while also providing the basis for a wider politics of transformation. The article offers several examples of grassroots initiatives formed to oppose standardized testing in England that provide practical examples of this “ideas work” in action.

Wanting the Unwanted Again: Safeguarding Against Normalizing Dehumanization and Discardability of Marginalized, “Unruly” English-Learning Latinos in Our Schools

Our current high-stakes accountability system for too long has allowed officials in positions of power and oversight to discard students viewed as too much of a liability as a practice of policy and pedagogy. This practice especially has altered how English learners are taught and the lens through which they are viewed. The authors present how a scandal in a large, urban borderland district in which “unruly” and English-language learning Latinas/os were systematically eliminated from test-taking and pushed-out illustrates how the consequences of No Child Left Behind have brought us to a tipping point in the education of the marginalized. Our education system has evolved into one in which the dehumanization of vulnerable students, their parents, and their communities has now dangerously become commonplace. In the pursuit of praxis, this essay argues that we must consider actions at the individual and local level, to bring about localized, incremental change that can result in larger cumulative movements of counter-narratives and counter-pedagogies in response to this trend of dehumanization in our schooling of ethnic, racial, and linguistic minorities.

Examining Teacher Agency: Why Did Les Leave the Building?

Effective teachers have been described as possessing a vision for their teaching. However, the current high-stakes accountability climate has provided challenges for teachers attempting to align their instruction with their vision for teaching. The goal of this study was to explore 3 in-service teachers' visions and perceived obstacles they faced as they worked to enact their vision in their respective schools. Although teachers' visions focused on adapting to fit the needs of their students, this was often at odds with their schools' recommendations. Findings from this study suggest the challenges associated with enacting one's vision given the constraints of high-stakes accountability systems.

Putting School Reform in Its Place

For decades, policymakers and researchers have struggled to understand the reasons that schools in disadvantaged contexts have relatively more trouble responding successfully to reform demands. This analysis extends theory regarding the challenges of school change in disadvantaged contexts by illustrating how the internal resources that schools rely on to respond to external policy demands can be affected by the social contexts in which they are embedded. The article draws on data from a study of five high poverty high schools’ responses to the pressures of Texas’ high stakes accountability system. The case study data illustrate how a school’s social context can precipitate instability in some schools and relative stability in others, how organizational stability in turn can affect schools’ organizational social capital, and how organizational social capital can influence schools’ ability to respond to external policy demands.

Teacher Learning in an Era of High-Stakes Accountability: Productive Tension and Critical Professional Practice

Background/Context With the installation of No Child Left Behind, teachers, particularly those who serve marginalized students, have increasingly been told what and how to teach. Previous research demonstrates that teachers can act as mediators between policy and practice, even within coercive environments such as those generated by high-stakes accountability systems. Yet we know little about how teachers who have been specially prepared to serve marginalized populations respond to accountability demands within tightly controlled contexts, such as those commonly found in “underperforming” schools. Purpose/Objective/Research Question/Focus of Study This study draws on social learning and activity theories to examine the specific factors that support equity-minded teachers to navigate accountability-driven language arts reforms, the specific barriers that might hinder teachers from serving marginalized students—particularly English Learners—in an era of accountability, and how particular contextual factors mediate teachers’ responses to accountability pressures. Setting The study was conducted in three different “underperforming” schools in California, predominately comprised of Spanish-speaking English Learners. Population/Participants/Subjects Three upper-elementary teachers, who earned their bilingual (Spanish/English) teaching credentials and Master's degrees from three different equity-focused teacher preparation programs. Research Design The research design is qualitative case study. Data collection and Analysis Data collection took place between September 2003 and December 2004 and consisted of teacher interviews, classroom observations, principal interviews, and focus group interviews. Using the constant comparative method, data analysis was ongoing. Findings/Results Initial coding identified patterns in teachers’ technical, normative, and political responses to accountability-related pressures in the area of language arts. Further analyses illuminated variations in teachers’ responses and emphasized how contextual factors, especially local leadership, mediated teacher learning and agency in the context of school change. Specifically, when principals provided teachers with opportunities to grapple with reforms they found objectionable and to apply innovations to their classroom practice, a “productive tension” led the teachers towards important professional learning and instructional improvement. Conclusions/Recommendations Findings underscore the importance of balanced leadership in an era of high- stakes accountability, particularly as it relates to teacher professionalism, learning, and agency.

Avoidable losses: High-stakes accountability and the dropout crisis.

In the state of Texas, whose standardized, high-stakes test-based accountability system became the model for the nation's most comprehensive federal education policy, more than 135,000 youth are lost from the state's high schools every year. Dropout rates are highest for African American and Latino youth, more than 60% for the students we followed. Findings from this study, which included analysis of the accountability policy in operation in high-poverty high schools in a major urban district, analysis of student-level data for more than 271,000 students in that district over a seven-year period under this policy, and extensive ethnographic analysis of life in schools under the policy, show that the state's high-stakes accountability system has a direct impact on the severity of the dropout problem. The study carries great significance for national education policy because its findings show that disaggregation of student scores by race does not lead to greater equity, but in fact puts our most vulnerable youth, the poor, the English language learners, and African American and Latino children, at risk of being pushed out of their schools so the school ratings can show "measurable improvement." High-stakes, test-based accountability leads not to equitable educational possibilities for youth, but to avoidable losses of these students from our schools.

Happy Marriage or Uneasy Alliance? The Relationship Between Comprehensive School Reform and State Accountability Systems

The purpose of this article is to examine how the relationship between comprehensive school reform (CSR) and state accountability systems helps or hinders school improvement efforts. This article draws on case study data collected in schools in 3 states that received funding to implement reforms through the federal CSR program. Findings show that the 3 states all had high-stakes accountability systems in place (albeit of varying levels of maturity) and these accountability systems often had much more of an impact on teacher practice than the CSR models themselves. In some cases, there was synergy between the models and the states' own reform agendas, but in other cases, teachers felt as if they were being asked to achieve 2 independent goals. The states varied in their levels of support of schools' CSR efforts, depending on the primacy of CSR in the state reform picture and the political will and experience of the individuals in charge of CSR at the state level. Implications for the policy and practice of states, reform design teams, and district and school educators are discussed.

Opportunity to Learn and the Accountability Agenda

When failure to a test can result in retention, mandatory remediation, or the denial of a diploma, making sure that all students have had a real opportunity to learn the material tested and to demonstrate their learning becomes paramount, Mr. Starratt argues. THE ISSUE of opportunity to learn (OTL) is most often debated from a policy perspective, namely, how to balance federal and state demands for accountability with policies ensuring that students have an opportunity to learn.1 If all students are to be held to the same standards, the argument runs, then they must have equal access to high- quality instruction. More specifically, all students should be able to attend schools that provide high-quality course offerings, instructional materials, and technologies; strong professional development opportunities for teachers; safe and secure school facilities; active nondiscrimination policies; and teaching that is both aligned with state standards and targeted toward higher-order thinking rather than memorization. Another way to address OTL is to focus on the current environment of high-stakes testing. Questions have been raised about flaws in assessment technology and how they affect conclusions drawn from the tests, about the distortions involved in reporting aggregated scores, and about the negative effects of high-stakes tests on various student populations. Although often implied, questions regarding the ethics of high-stakes accountability systems and their effects on students and teachers are seldom raised explicitly in the research on OTL. In this article, I want to demonstrate that imposing such accountability systems without fully addressing the issue of OTL is a violation of social justice. One critical moral issue related to the high-stakes testing systems currently operating in many states -- in which failure to a test can result in retention, mandatory remediation, or denial of a diploma -- is whether all students have the opportunity to learn the material on which they are being tested. My argument is not with testing, nor indeed with any particular form of state tests. Rather, my argument is with states that require students to tests or face high-stakes consequences (and the public humiliation and shame that accompany such consequences) but fail to provide adequate opportunities to learn. I would argue that such an accountability system is a form of victimization. INJUSTICE AT THE GROUND LEVEL While deliberations at the policy level can be conducted relatively dispassionately, in the classroom one can witness injustice firsthand. As an extreme example, suppose a class of 25 ninth-graders were exposed to one minute of instruction on how to solve quadratic equations. After one minute of instruction, they were given a test to solve 10 quadratic equations. Of the 25 students, two solved seven equations, thereby passing the test. These two students were the children of engineers, and their abilities had been enriched by exposure to mathematical games and puzzles in the home. Of the remaining 23 students, two solved two of the quadratic equations, indicating that they had learned some of the material (or had guessed their way into a correct solution) but had not become sufficiently proficient to pass the test. When asked why they didn't solve the other quadratic equations, they said that they ran out of time. The remaining 21 students failed to solve any of the quadratic equations. Given these test results, it is clear that most teachers and parents would say that it is unreasonable to expect one minute of instruction to provide a legitimate opportunity to learn how to solve quadratic equations. They would say that more instructional time and more time to practice solving quadratic equations would constitute a more reasonable opportunity to learn. In addition to questioning the educational inputs the students received prior to testing, the teachers and parents might ask about the length of time allotted for the test. …

All That Glitters is Not Gold: School Reform in Charlotte-Mecklenburg

In the early 1990s, the Charlotte-Mecklenburg school system (CMS) began a sweeping program of school reform that involved revamped standards, a comprehensive system of benchmark goals, increased autonomy for principals, and a high-stakes accountability system. This program has been lauded in prestigious education journals, praised by reform advocates in business and government, and cited as a model in a recently published handbook of school reform. Such national publicity notwithstanding, there has been no systematic investigation of the effects of CMS's program on academic outcomes. This article provides such an assessment by comparing CMS's progress in improving outcomes for both Black and White students with that of two other North Carolina urban districts and the state itself. With the exception of some outcomes for Advanced Placement (AP) and other higher level courses, we find no evidence that CMS's program improved academic outcomes. Trends in CMS's dropout rate—one of North Carolina's highest—generally paralleled those of the controls as did trends in SAT scores and proficiency in core high school courses. Moreover, CMS's progress in improving reading and math proficiency in Grades 3–8 lagged behind the controls'. Finally, racial imbalance in CMS increased during this time period. The many shortfalls of CMS's program raise questions whose answers will further the national dialogue about school reform and the role of standards, accountability, and assessment in such reform.

All That Glitters Is Not Gold: School Reform in Charlotte-Mecklenburg

In the early 1990s, the Charlotte-Mecklenburg school system (CMS) began a sweeping program of school reform that involved revamped standards, a comprehensive system of benchmark goals, increased autonomy for principals, and a high-stakes accountability system. This program has been lauded in prestigious education journals, praised by reform advocates in business and government, and cited as a model in a recently published handbook of school reform. Such national publicity notwithstanding, there has been no systematic investigation of the effects of CMS's program on academic outcomes. This article provides such an assessment by comparing CMS's progress in improving outcomes for both Black and White students with that of two other North Carolina urban districts and the state itself. With the exception of some outcomes for Advanced Placement (AP) and other higher level courses, we find no evidence that CMS's program improved academic outcomes. Trends in CMS's dropout rate—one of North Carolina's highest—generally paralleled those of the controls as did trends in SAT scores and proficiency in core high school courses. Moreover, CMS's progress in improving reading and math proficiency in Grades 3–8 lagged behind the controls'. Finally, racial imbalance in CMS increased during this time period. The many shortfalls of CMS's program raise questions whose answers will further the national dialogue about school reform and the role of standards, accountability, and assessment in such reform.

Accountability Requirements and Professional Development in the US Adult Basic and Literacy Education System

Even before the 2001 enactment of the No Child Left Behind legislation, the education bill that holds schools in the US accountable for student achievement, ‘adult education [had] become part and parcel of the new federal trend to encourage the setting of national education goals and standards and holding programs accountable for demonstrating achievements’ (Sticht 1998). Now, almost ten years after enacting the Workforce Investment Act (1998), the legislation that required states to report how adult students were making progress towards educational and work goals, the field is just beginning to take stock of whether accountability has helped or hurt our adult education system.In the US school system (kindergarten to 12th grade for children five to 18), several researchers have investigated the effect of stronger accountability requirements on professional development systems. Berry et al. (2003), in a study of 250 teachers and principals in schools across six Southeastern US states found that results were mixed:Although high-stakes accountability systems help focus professional development efforts on the curricular needs of students, little evidence exists to support the claim that such systems help teachers change their practice to enhance student learning...A tendency exists…to narrow the focus of professional development activities to tested subjects or provide general support that is disconnected from curricular needs. (Association for Supervision and Curriculum Development 2004:3)