Standards-based Accountability Systems Research Articles

Lessons learned: how parents respond to school mandates and sanctions

AbstractOver the past three decades, a reform movement bent on improving schools and educational outcomes through standards-based accountability systems and market-like competitive pressures has dominated policy debates. Many have examined reform policies’ effects on academic outcomes, but few have explored these policies’ influence on citizens’ political orientations. In this study, using data from an original survey, I examine whether and how No Child Left Behind’s accountability-based architecture influences parents’ attitudes towards the government and federal involvement in education. I find little evidence that diversity in parents’ lived policy experiences shapes their political orientations. However, the results of a survey experiment suggest that information linking school experience to policy and government action may increase parents’ confidence in their ability to contribute to the political process. Understanding whether and under what conditions parents use public school experiences to inform orientations towards the government can improve the design of future reforms.

Developing Instructional Managers

Significant educational ideas endure, but they also evolve over time. In the 2013s, "instructional manager" became the dominant paradigm for organization managers after researchers noticed that effective organizations usually had managers who kept a high focus on curriculum and instruction. In the first half of the 2012s, attention to instructional manager seemed to waver, displaced by discussions of organization-based management and facilitative manager.But recently instruction has surged back to the top of the manager agenda, driven by the relentless growth of standards-based accountability systems. Explicit standards of learning, coupled with heavy pressure to provide tangible evidence of success, have reaffirmed the importance of instructional manager.Nevertheless, despite general agreement that instructional manager is a critical skill, few managers and superintendents have had in-depth training for that role, especially in a standards-based environment. This Digest reviews the demands of today's instructional manager and discusses steps that universities and organization districts can take to help managers develop the necessary skills.

Measuring Opportunity to Learn and Achievement Growth

The related constructs of opportunity to learn (OTL) and achievement growth are fundamental aspects of the current large-scale assessment and accountability system in operation in the United States. For purposes of this article, OTL is defined as the degree to which a teacher dedicates instructional time and content coverage to the intended curriculum objectives emphasizing high-order cognitive processes, evidence-based instructional practices, and alternative grouping formats. Both OTL and achievement growth, although frequently talked about by education policy makers and teachers, are actually difficult to measure. Much of my research, in collaboration with other colleagues, over the past decade has concerned the measurement of these constructs in ways that facilitate teachers’ understanding of them and to advance sound policies regarding the instruction and assessment of students with disabilities (SWDs). In this article, I examine both the measurement of OTL and achievement growth, share recent research on them, and outline an agenda for future research on these two important aspects of standards-based accountability systems for all students.

Standards-Based Accountability in the United States:

Standards-based accountability (SBA) has been a primary driver of education policy in the United States for several decades. Although definitions of SBA vary, it typically includes standards that indicate what students are expected to know and be able to do, measures of student attainment of the standards, targets for performance on those measures, and a set of consequences for schools or educations based on performance. Research on SBA indicates that these policies have led to some of the consequences its advocates had hoped to achieve, such as an emphasis on equity and alignment of curriculum within and across grade levels, but that it has also produced some less desirable outcomes. This article summarizes the research on SBA in three areas: quality of standards, ways in which SBA has shaped educators’ practices, and effects on student achievement. The article identifies lessons learned from the implication of SBA in the United States and provides guidance for developing SBA systems that could promote beneficial outcomes for students.

From a Roar to a Murmur: Virginia's History & Social Science Standards, 1995–2009

The authors trace the development and implementation of Virginia's History and Social Science standards-based accountability system from 1995 to 2009. They frame the study within an examination of the political ideologies that influence policy realization and unpack the relationship between ideological and epistemological beliefs about the nature of disciplinary knowledge and arguments regarding what knowledge is of most worth and whose voices should be included. While initial policy implementation created vociferous reactions, subsequent revisions have been met with silence. Such acquiescence, the authors suggest, reflects the ways in which high stakes testing as a vehicle for assessing learning has become normalized in Virginia. This shift in beliefs about education foreshadows the potential impact of the nationwide accountability movement and raises a concern that if Virginia ceased to test history and social science, its place within the school schedule would be lost to content areas that impact Adequate Yearly Progress (AYP).

The Concept of Accountability in Education: Does the Swedish School System Apply?

- Recent years have seen a growing international debate over accountability in education, and particularly the consequences of such systems. The Swedish school system seems to share many common features with systems in countries where the accountability debate is strong. Still, in Sweden there has been limited discussion and no attention paid to the international debate. The present paper explores whether Sweden can be said to have a standards-based accountability system or not, by comparing the Swedish system with a standards- based accountability model. The conclusion is that the Swedish national assessment system may not be an accountability system in the narrower sense, but that issues of accountability are nevertheless prevalent in current discussions about the successes and failures of Swedish educational policy and practice. It is found the in-explicitness of the system is a threat to validity as assessment instruments may be used and interpreted in ways they were not intended to, with unintended consequences as a result. Keywords: Accountability, assessment, goals, consequences, compulsory education.

Searching for Vitality in Teacher Education

The hopes and dreams for our shared JTE editorship began 6 years ago over three bowls of tortilla soup. During that lunchtime meeting, through deliberation and debate, we formed our editorial vision statement. Scanning the professional horizon in fall 2003, we characterized the teacher education landscape for our American Association for Colleges of Teacher Education (AACTE) editorship proposal in the following terms: Those committed to education share a historically unprecedented goal: To ensure all children will develop the knowledge, understandings, and competencies needed to reach their full potential and to contribute to our democratic society. Furthermore, most accept that quality teachers play a crucial role in realizing this aim. Yet, in the face of this common goal, different visions for how best to realize this aim compete in the marketplace of ideas. On the one hand, some reforms promote standards-based accountability systems, market-based approaches to public education, advocacy of direct instruction and prescribed curricula, and deregulation of teacher preparation and licensure. On the other hand, other reformers encourage a broader conception of the aims and purposes of education, a view of public schools as institutions uniquely charged with shaping and creating a fairer, equitable, and just society, a preference for pedagogies and assessments that focus on conceptual understanding, and the promotion of teaching as a profession. One thing is certain in all of this mix, a consensus about teaching, public schooling, and the role of teacher education does not exist. Add to these debates an increasing number of alternative and state-sanctioned routes to teacher certification, the changing cultural and class demographics of our public schools, the relatively stable white, middle-class pool of prospective and practicing teachers, and the decreasing state appropriations for higher education, and we end up with a terrain ripe for structural transformation. While we don't believe these changes will be designed or guided solely by rational deliberations, we do maintain that the Journal of Teacher Education has and should provide a scholarly forum for the dissemination and examination of teacher education research, policy, and practice. If we were chosen to guide the Journal in upcoming years we would provide a ... setting where distinct and varied scholarly approaches could explore, interrogate, and illuminate the contours of teacher education. (1) The guiding metaphor for our editorial vision was then and has continued to be a conversation of many voices. We believe that rich conversational lines with many distinct participants allow us to crisscross the ill-structured landscape of teaching and learning to teach, to explore and thereby affirm its complexity (Lampert, 2001; Spiro, Coulson, Feltovich, & Anderson, 1988/2004). Although understanding complexity is a scholar's primary aim (and delight), such multi-vocal conversations offer those involved in the endeavor opportunities to see the terrain from different vantage points and to cross well trodden paths in new places. Out of these encounters, it was our hope that fresh ideas and practices would emerge. As we approached this, the final issue published under our editorship, we revisited our proposal statement. To some extent, we were discouraged. The statement written in 2003 could easily be written in 2009. The field continues to be in flux, and university-based teacher preparation is arguably more besieged today than when we began. We wondered about the vitality of our field: Is there any evidence or reason to believe that creative, vital, and powerful ideas and practices are being taken up in the field of teacher education? We asked individuals who hold diverse views to respond to the following question: Is there vitality in the field of teacher education? …

Testing Historical Knowledge: Standards, Multiple-Choice Questions and Student Reasoning

This article explores the reasoning employed by high school students to answer a set of multiple-choice history questions. The questions come from New York State's Global History and Geography Regents exam. The Regents exams, together with a particularly well-regarded and ambitious set of content standards, are the cornerstone of the state's standards-based accountability system. Using “think-aloud” and interviews with a small sample (n=13) of urban 10th graders, the knowledge and skills elicited by a small sample of items are explored. This article begins with a discussion of the state standards and the discipline-based knowledge and skills that they describe. It continues with a report on the extent to which the student-participants were exposed to the material included in the administered items by a teacher whose pedagogy was commensurate with state standards. The bulk of the paper focuses on two test items that were particularly effective at discriminating between high and low performers. The reasoning and knowledge employed by participants when answering these questions are explored. The findings support previous research indicating that the assumptions made by adults about what items measure are often only partially correct. In the case of the items included in this study, they appeared to evoke knowledge and skills in three domains—history content, literacy, and test-wiseness—but not the discipline-based thinking so highly prized by the standards.

Responding to the Demands of Assessment and Evaluation in Catholic Education

Public schools are experiencing a new era of assessment and evaluation with the implementation of state accountability systems and No Child Left Behind. How can Catholic schools respond by recognizing the legitimacy of evaluation and assessment, while also critically examining its appropriateness? To help address this question, the article contains a brief history of how Catholic schools have accommodated evaluation and assessment demands, followed by a critical examination of standards-based accountability systems and high-stakes testing in relation to the identity and current milieu of Catholic schools. It also presents an expanded understanding of assessment and evaluation, which draws heavily on qualitative research paradigms.

Meeting Expectations? An Empirical Investigation of a Standards-Based Assessment of Reading Comprehension

The present study was designed to examine the relationship between performance on a standards-based assessment of reading comprehension in fourth grade and performance on a diagnostic battery of component skills of reading for a cohort of children who were followed from kindergarten through fourth grade. The findings demonstrate that the relationship between performance on component skills of reading and the proficiency classifications from the standards-based assessment is generally weak. They also demonstrate that the relationship between the proficiency classifications and a norm-referenced diagnostic measure of reading comprehension is only moderate. Furthermore, the study shows that the broad proficiency-level classifications of the standards-based assessment mask significant heterogeneity in children’s performance on some of the component skills assessed, both within and across proficiency classifications. In turn, the ability of the standards-based assessment results to inform instructional interventions is low. The implications of these findings for instructional planning within standards-based accountability systems are discussed.

Emerging Accountability Systems within Part C and Section 619 Programs

Within recent years, federal legislation has required schools to develop and implement standards-based accountability systems for the purposes of evaluating their performance. Although the implementation of accountability systems for K-12 programs is well underway, the design and implementation of such systems for Part C and Section 619 programs lag significantly behind. This article will provide a brief review of accountability-related laws and current efforts to design accountability systems for infants and young children with disabilities and will discuss the challenges inherent in the design of such systems for Part C and Section 619 programs. Finally, and most importantly, the article will address the implications of these accountability initiatives for personnel preparation programs within institutions of higher education.

A Role for Technology in Professional Development? Lessons from IBM

It would seem logical that if there were a precedent for using technology successfully for training, it would be found at IBM. Thus Mr. Levy and Mr. Murnane studied the company's approach to training new to see what lessons it might hold for schools and districts wanting to rely more heavily on technology to deliver professional development. STANDARDS-BASED accountability systems challenge American educators to accomplish something that has never been done in the nation's history: teaching all children to master a demanding set of skills. The challenge makes sense today because technological changes and outsourcing have left American workers who lack strong skills unable to earn a decent living. However, the magnitude of the challenge is frequently underestimated. Working harder will not by itself allow American educators to meet this challenge. Indeed, most already work so hard that burnout is a continual danger. Instead, educators need to learn how to work together more effectively. Professional development, the primary strategy for improving the effectiveness of educators, has not worked well. As Mike Schmoker pointed out in the February 2004 Kappan, too often it consists of workshops led by an outside speaker, with little or no follow-up. These efforts do not change how well teachers teach or how effectively children learn. Nor do they improve the quality of administrators' leadership. One response to the professional development challenge has been to make greater use of technology. Initiatives that use the Internet in professional development include TeachScape, LessonLab, and the Center for Online Professional Education. What role should technology play in professional development? Can it substitute for face-to-face meetings that are expensive and hard to schedule? Can it make face-to-face meetings more valuable? Does it change how meeting time should be spent and require new skills of those who facilitate these meetings? In this article, we describe how IBM answered these questions in the process of revamping its training program for new managers. We then consider how the IBM experience can inform efforts to improve professional development for educators. At this point readers may well ask, What can educators learn from a profit-making company engaged in a business so different from teaching children? Our answer has three parts. First, for reasons we explain below, the challenges IBM face have much in common with those educators face. We want to be clear that we are using the term educators to include teachers as well as school administrators. We see teachers as managers in that they are responsible for the well-being and development of a large number of children. In the differentiated staffing models that many school reformers advocate, experienced teachers would also serve as mentors and supervisors for beginning teachers.1 Second, as a technology-based company, IBM has significant expertise in developing and implementing new ways to use computer networks in its training programs. Third, because IBM operates in an extremely competitive industry, it has both a great incentive to design efficient training programs and the freedom to do so. What Information Technology Can and Cannot Do Computers excel at conveying information. Since using information lies at the center of teachers' and administrators' jobs, it makes intuitive sense that networked computers could be valuable in improving the effectiveness of teachers and administrators. Indeed, we believe this is true. Networked computers can improve teachers' and administrators' access to a wide range of potentially useful information, including patterns in student test scores, ideas for lesson plans, and school district rules and strategies for dealing with disruptive students. Yet, for information to be useful, it needs to be interpreted in context. Suppose in the course of conversation, a friend speaks the word bill. …

Coping with conflicting demands: Student assessment in Dutch primary schools

n the Netherlands, as in many other developed countries, the educational authorities are trying to strengthen their control of educational productivity. They want schools to adopt student assessment schemes, based on norm-referenced tests. A questionnaire was submitted to primary school teachers and principals in order to obtain a description of current assessment practices and opinions. It appears that Dutch teachers and principals make only limited use of norm-referenced tests. They prefer curriculum-embedded tests, which provide much more support for teaching and learning processes. Respondents consider accountancy purposes to be of secondary importance. Several recommendations specifically directed at educational officials are given in order to facilitate the introduction of a standards-based accountability system.

Assessments and Accountability

Use of tests and assessments as key elements in five waves of educational reform during the past 50 years are reviewed. These waves include the role of tests in tracking and selection emphasized in the 1950s, the use of tests for program accountability in the 1960s, minimum competency testing programs of the 1970s, school and district accountability of the 1980s, and the standards-based accountability systems of the 1990s. Questions regarding the impact, validity, and generalizability of reported gains, and the credibility of results in high-stakes accountability uses are discussed. Emphasis is given to three issues regarding currently popular accountability systems. These are (a) the role of content standards, (b) the dual goals of high performance standards and common standards for all students, and (c) the validity of accountability models. Some suggestions for dealing with the most severe limitations of accountability are provided.

Assessments and Accountability

Use of tests and assessments as key elements in five waves of educational reform during the past 50 years are reviewed. These waves include the role of tests in tracking and selection emphasized in the 1950s, the use of tests for program accountability in the 1960s, minimum competency testing programs of the 1970s, school and district accountability of the 1980s, and the standards-based accountability systems of the 1990s. Questions regarding the impact, validity, and generalizability of reported gains, and the credibility of results in high-stakes accountability uses are discussed. Emphasis is given to three issues regarding currently popular accountability systems. These are (a) the role of content standards, (b) the dual goals of high performance standards and common standards for all students, and (c) the validity of accountability models. Some suggestions for dealing with the most severe limitations of accountability are provided.