Student Achievement Data Research Articles

Toward Accessible Assessments: The Promises and Limitations of Test Item Adaptations for Students with Disabilities and English Language Learners

When do item adaptations veer from their intent and, instead of increasing access, modify the construct being measured? This study analyzed early elementary student achievement data from a statewide field test containing both standard and adapted science items. Four student groups were included in this analysis: English language learners, students with learning disabilities, students who are deaf or hard of hearing, and a control group of non–individualized education program native English speakers. This study included a qualitative evaluation of the items by experts with backgrounds in working with students with disabilities. Small sample sizes and restricted numbers of items available made generalizations of the results difficult, but findings suggest unexpected focal group differences in performance on the adapted items. The authors suggest that attention needs to be paid to whom items are adapted for, and steps need to be taken during item development to ensure that changes have the intended effect.

Research and Policy: Lunch, Teeth, Body, and Mind: Children’s Learning and Well-Being

This article explores the research on the relationship between two particular aspects of poverty and school achievement. In particular, it examines research on food insecurity and oral healthcare among elementary-school-age children. The argument here is that these sorts of experiences of children in poverty account for some of the achievement difference between students from low-income homes and those with adequate resources for physical well-being. This fact has significant policy implications, as there are gaps in the ways policymakers engage with data; they often do not attempt to correlate student or school achievement data with material disadvantage like the ones investigated here. Reform efforts in curriculum and instruction would be more fruitful if they were combined with improving children’s overall well-being.

Assessing Student Achievement in Physical Education for Teacher Evaluation

While many teachers continue to ignore the practice of assessing student achievement in physical education, recent federal pressures to include student assessment data in teacher evaluation systems has shown that assessment of student outcomes is here to stay. Though there is a strong tradition of assessing teacher practice in physical education, standardized measures of student achievement and the requirement to show data on student learning in physical education are relatively new ideas. This article reviews issues about using student achievement data to evaluate physical education teachers. It also presents examples of assessments that could be used to document student achievement for the purpose of teacher evaluation.

Thirty-Seven and Counting: How Has AEFP Evolved from Its Origins?

It has been a busy time for the Association of Education Finance and Policy (AEFP). Over the past few years the association has acquired a new name, a new journal, and many new members. The 2012 annual conference, convened in Boston last March, proved to be the largest conference in the association’s thirty-seven-year history, with 556 members in attendance. The theme, selected by incoming president Deborah Cunningham, was “Education Finance, Policy, and Practice: The Role of Evidence in a Dynamic World,” which underscores the contemporary challenge to the association: how to apply an increasing abundance of information and sophisticated analytical tools to produce the evidence needed to guide decision making by educational policy makers and practitioners. The Boston meeting was notable not only for the number in attendance. The unique qualities and strengths of the association were in clear display: papers of unusual methodological rigor; an interdisciplinary mix of academics from the social sciences, public policy schools, and colleges of education; educational finance professionals, policy analysts, and practitioners, a mix rarely found in the same place; and sessions addressing today’s hot topics as well as issues that have endured over the years. Having said this, all indications are that AEFP is what it has always been: a small, diverse group of people tackling some really big problems. Of particular note was a trend that has been growing for years but has clearly come into full flower: the large

SMALL RURAL SCHOOL DISTRICT CONSOLIDATION IN TEXAS: AN ANALYSIS OF ITS IMPACT ON COST AND STUDENT ACHIEVEMENT

Historically, the number of public school districts in the United States has decreased despite a dramatic increase in the number of students enrolled. Although public school district consolidation has impacted districts of all sizes, since the late 1930’s smaller rural districts facing dwindling community resources have merged or consolidated with each other, resulting in fewer school districts. When school districts consolidate, all aspects of the newly-formed district are affected. Each year, lawmakers and rural public school district officials face dwindling finances, and each year these decision makers question whether to consolidate to avoid fiscal perils. Proponents tout the benefits of fiscal efficiency, a broadened curriculum, and a projected increase in student achievement. Critics argue that the community suffers when the community school closes, students are burdened with new transportation issues, increased academic opportunities do not necessarily impute to greater student performance, and a host of tangible and non-tangible arguments are put forth. This ex post facto quantitative study examines the fiscal efficiency of small, rural, consolidated school districts by comparing per-pupil expenditures with matched non-consolidated school districts in the state of Texas. The study also examines student achievement levels by comparing passing rate percentages on all Texas state assessment tests for 3rd, 5th, and 8th grade students attending these schools. For before and after consolidation comparison purposes, rural community public schools were matched according to Texas state designated “paired” protocol. Districts meeting Texas Education Agency (TEA) Snapshot criterion for Absorbing districts were matched with Joining districts. Expenditure and student achievement data for Absorbing and Joining districts were collected for the ten-year period from 1999 to 2009. A paired samples t-test measured differences in the district’s efficiency, and the Lawshe-Baker Normative t-test measured differences in student achievement. Four null hypotheses were examined with an a priori alpha level = 0.05. This study, when the data for the joining and absorbing districts was subjected to appropriate t-tests, supports other research that suggests per-pupil expenditures increased and student achievement decreased for the absorbing district. Keywords: per-pupil expenditures, student achievement, joining and absorbing districts

In Search of Tools that Inform Practice: Utilizing Program Accountability Data to Guide Program Improvement

In light of the fact that policy makers are attempting to build accountability systems that will hold teachers, and the teacher education programs that prepared them, accountable for student achievement results, failure on the part of teacher educators to repeatedly and assertively advocate for a significant role in shaping the accountability systems for teacher education programs and to structure them in ways that can lead to program improvement will undoubtedly be a missed opportunity with dire consequences that will take years, if not decades, to correct. The data analysis practices advocated in this article yield data that teacher preparation programs could use for program improvement. The authors use actual student achievement data collected through an earlier large-scale teacher induction study to demonstrate the ways in which the proposed data analysis approach could be useful to teacher educators interested in program improvement.

How Context Matters in High-Need Schools: The Effects of Teachers’ Working Conditions on Their Professional Satisfaction and Their Students’ Achievement

Background/Context Educational policy makers have begun to recognize the challenges posed by teacher turnover. Schools and students pay a price when new teachers leave the profession after only 2 or 3 years, just when they have acquired valuable teaching experience. Persistent turnover also disrupts efforts to build a strong organizational culture and to sustain coordinated instructional programs throughout the school. Retaining effective teachers is a particular challenge for schools that serve high proportions of low-income and minority students. Although some interpret these turnover patterns as evidence of teachers’ discontent with their students, recent large-scale quantitative studies provide evidence that teachers choose to leave schools with poor work environments and that these conditions are most common in schools that minority and low-income students typically attend. Thus, mounting evidence suggests that the seeming relationship between student demographics and teacher turnover is driven not by teachers’ responses to their students, but by the conditions in which they must teach and their students are obliged to learn. Purpose/Objective/Research Question/Focus of Study We build on this body of work by further examining how working conditions predict both teachers’ job satisfaction and their career plans. We use a broad conception of the context of teachers’ work, paying attention not only to narrowly defined working conditions but also to the interpersonal and organizational contexts in which teachers work. We also extend Ladd's analysis describing the relationship between the work context and student achievement. Advancing our understanding of this relationship is particularly important, given the increasing emphasis legislators place on evidence of student achievement when evaluating education policy. Specifically, we ask three research questions: (1) Do the conditions of work in Massachusetts public schools affect teachers’ satisfaction with their jobs and their career plans? (2) Are schools with better conditions of work more successful in raising student performance than schools with less supportive working conditions? (3) If the conditions of work are important, what elements of the work environment matter the most? Research Design In this article, we combine a statewide survey of school working conditions (MassTeLLS) with demographic and student achievement data from Massachusetts. We examine three primary outcomes: teacher satisfaction, teacher career intentions, and student achievement growth. From different items on the MassTeLLS, we construct a set of nine key elements that reflect the broad-based conditions in which teachers work. We fit standard regression models that describe the relationship between each outcome and both overall conditions of work and each element separately, modeling this relationship according to the properties of our outcome variables. Findings/Results We found that measures of the school environment explain away much of the apparent relationship between teacher satisfaction and student demographic characteristics. The conditions in which teachers work matter a great deal to them and, ultimately, to their students. Teachers are more satisfied and plan to stay longer in schools that have a positive work context, independent of the school's student demographic characteristics. Furthermore, although a wide range of working conditions matter to teachers, the specific elements of the work environment that matter the most to teachers are not narrowly conceived working conditions such as clean and well-maintained facilities or access to modern instructional technology. Instead, it is the social conditions—the school's culture, the principal's leadership, and relationships among colleagues—that predominate in predicting teachers’ job satisfaction and career plans. More important, providing a supportive context in which teachers can work appears to contribute to improved student achievement. We found that favorable conditions of work predict higher rates of student academic growth, even when we compare schools serving demographically similar groups of students. Conclusions/Recommendations In short, we found that the conditions of teachers’ work matter a great deal. These results align with a growing body of work examining the organizational characteristics of the schools in which teachers work. Together, these studies suggest strongly that the high turnover rates of teachers in schools with substantial populations of low-income and minority students are driven largely by teachers fleeing the dysfunctional and unsupportive work environments in the schools to which low-income and minority students are most likely to be assigned. If public education is to provide effective teachers for all students, then the schools those students attend must become places that support effective teaching and learning across all classrooms.

High School Teachers' Use of Data to Inform Instruction

The expectation that teachers will use student achievement data to improve their instruction is a major feature of national and local reform agendas. The theory of action behind data-driven decision making is a mostly causal model of professional action, whereby teachers diagnose weaknesses and implement solutions. The purpose of this article is to examine how high school teachers, situated within their policy and work contexts, use data to inform instructional decisions. Using a framework that draws upon sense-making and co-construction theories on reform implementation, we analyze qualitative data gathered in 4 urban public high schools in the United States. Findings reveal that the process of data use by teachers is complex, multilayered, and influenced by teacher interpretations and social interactions. Teachers used a variety of forms of data to inform their decision making and struggled to reconcile policies promoting data-driven decision making with local beliefs and practices. Implications for research and policy are discussed.

Is Arizona's Approach to Educating Its ELs Superior to Other Forms of Instruction?

BackgroundIn the Horne v Flores Supreme Court decision of June 25, 2009, the Court wrote that one basis for finding Arizona in compliance with federal law regarding the education of its English learners was that the state had adopted a “significantly more effective” than bilingual education instructional model for EL students—Structured English Immersion (SEI).PurposeThis paper reviews the extant research on SEI, its definitions, origins, and its effectiveness, particularly in contrast to other instructional strategies. This paper asks, Does the research bear out the Court's conclusion? What is the evidence that Arizona's program of SEI is really superior to other approaches, including bilingual or dual language education? How are Arizona's EL students faring under this “significantly more effective” instructional program?Research DesignData on the relative effectiveness of SEI are drawn from a comprehensive review of the literature. Analysis of public documents, particularly records from the Arizona English Language Learners Task Force, which was charged with selecting a research-based instructional program for English learners. Drawing from a recent ethnographic study and student achievement data, we examine the impact of structured English immersion programs on English learners in Arizona thus far, beginning with achievement outcomes.Conclusions/RecommendationsThere is no research basis for the Court's statement the SEI is “significantly more effective”; at best SEI is no better or no worse than other instructional strategies, particularly bilingual instruction, when they are both well implemented. However, SEI as implemented in Arizona carries serious negative consequences for EL students stemming from the excessive amount of time dedicated to a sole focus on English instruction, the de-emphasis on grade level academic curriculum, the discrete skills approach it employs, and the segregation of EL students from mainstream peers. Moreover, the paper argues that there are, in fact, strategies that can ameliorate these problems as well as provide an additive, rather than a subtractive, educational experience for English learner and mainstream students alike.

Transforming Special Education Teacher Education: A Reaction to the Report Transforming Teacher Education through Clinical Practice: A National Strategy to Prepare Effective Teachers

Policy makers, university teacher education faculty, school leaders, and government officials are asking the same question: How do we recruit, prepare, and retain effective teachers who will produce desired student outcomes in every classroom? This complex question garners distinct opinions depending on the queried stakeholder, but most agree that significant improvement is needed in the processes of teacher preparation and induction (Darling-Hammond, 2010; Greer & Meyen, 2009; Sykes, Bird, & Kennedy, 2010; Wang, Odell, Klecka, Spalding, & Lin, 2010). An argument can be made that the need for improvement is most urgent within the field of special education teacher preparation (Brownell, Sindelar, Kiely, & Danielson, 2010; Piper, 2007; Pugach, Blanton, & Correa, 2011; Simonsen et al., 2010; Sindelar, Brownell, & Billingsley, 2010) . To illustrate, recent achievement data for students with disabilities provides striking evidence of the critical need for improvement in areas of literacy, graduation rates, and other postsecondary outcomes (e.g. National Center for Education Statistics [NCES], 2009; Wagner, Newman, Cameto, Levine, & Garza, 2006). There is substantial variability in the numerous factors that contribute to the struggles of students with exceptionalities on measures of academic and social success (see Skiba, Poloni-Staudinger, Simmons, Feggins-Azziz, & Chung, 2005; Wagner et al., 2006). Many complex factors influence a teacher's impact on student achievement, which leads to the need for us to continue to examine and reform our current models of teacher preparation. Thus, teacher educators and practitioners must continue to investigate and evaluate the effects of new and existing policies, programmatic structures, and individual practices on outcomes of interest and disseminate those findings. Although calls to reform teacher education and P-12 instruction for children with exceptionalities are not new (Vaughn & Fuchs, 2003), it is clear that new thinking is needed to overcome traditional barriers to academic and social success for individuals with exceptionalities. However, despite the critical need for improvement, teacher preparation models within special and general education largely remain fixed to traditional methods that reflect the status quo as opposed to evidence-based practice (Boyd, Grossman, Lankford, Loeb, & Wycoff, 2009; Brownell, Griffin, Leko, & Stephens, 2011; Sykes et al., 2010).

Transforming Special Education Teacher Education: A Reaction to the Report Transforming Teacher Education through Clinical Practice: A National Strategy to Prepare Effective Teachers

Policy makers, university teacher education faculty, school leaders, and government officials are asking the same question: How do we recruit, prepare, and retain effective teachers who will produce desired student outcomes in every classroom? This complex question garners distinct opinions depending on the queried stakeholder, but most agree that significant improvement is needed in the processes of teacher preparation and induction (Darling-Hammond, 2010; Greer & Meyen, 2009; Sykes, Bird, & Kennedy, 2010; Wang, Odell, Klecka, Spalding, & Lin, 2010). An argument can be made that the need for improvement is most urgent within the field of special education teacher preparation (Brownell, Sindelar, Kiely, & Danielson, 2010; Piper, 2007; Pugach, Blanton, & Correa, 2011; Simonsen et al., 2010; Sindelar, Brownell, & Billingsley, 2010) . To illustrate, recent achievement data for students with disabilities provides striking evidence of the critical need for improvement in areas of literacy, graduation rates, and other postsecondary outcomes (e.g. National Center for Education Statistics [NCES], 2009; Wagner, Newman, Cameto, Levine, & Garza, 2006). There is substantial variability in the numerous factors that contribute to the struggles of students with exceptionalities on measures of academic and social success (see Skiba, Poloni-Staudinger, Simmons, Feggins-Azziz, & Chung, 2005; Wagner et al., 2006). Many complex factors influence a teacher's impact on student achievement, which leads to the need for us to continue to examine and reform our current models of teacher preparation. Thus, teacher educators and practitioners must continue to investigate and evaluate the effects of new and existing policies, programmatic structures, and individual practices on outcomes of interest and disseminate those findings. Although calls to reform teacher education and P-12 instruction for children with exceptionalities are not new (Vaughn & Fuchs, 2003), it is clear that new thinking is needed to overcome traditional barriers to academic and social success for individuals with exceptionalities. However, despite the critical need for improvement, teacher preparation models within special and general education largely remain fixed to traditional methods that reflect the status quo as opposed to evidence-based practice (Boyd, Grossman, Lankford, Loeb, & Wycoff, 2009; Brownell, Griffin, Leko, & Stephens, 2011; Sykes et al., 2010).

Translating Policy to Practice

This mixed methods study explores the pilot implementation of a Response to Intervention framework in the first grade classrooms in three urban schools. Two schools in a fully implemented condition (FI) with a facilitator and a partially implemented condition (PI) without a facilitator were investigated using student achievement data, field notes, teacher questionnaires, and teacher focus groups. Repeated measures ANOVA indicate that student risk levels measured by DIBELS phonemic awareness and decoding tasks were reduced in both conditions but favored the FI condition. Qualitative evidence indicates that programmatic changes were implemented in one year, but more complex paradigmatic changes did not occur.

Can Value-Added Measures of Teacher Performance Be Trusted?

We investigate whether commonly used value-added estimation strategies produce accurate estimates of teacher effects under a variety of scenarios. We estimate teacher effects in simulated student achievement data sets that mimic plausible types of student grouping and teacher assignment scenarios. We find that no one method accurately captures true teacher effects in all scenarios, and the potential for misclassifying teachers as high- or low-performing can be substantial. A dynamic ordinary least squares estimator is more robust across scenarios than other estimators. Misspecifying dynamic relationships can exacerbate estimation problems.

Developing Academic Language in English Language Learners Through Sheltered Instruction

This article describes a study examining the effects of Sheltered Instruction Observation Protocol (SIOP) model instruction on the academic language performance of middle and high school English language learners. The SIOP model is an approach for teaching content curriculum to students learning through a new language. Teachers employ techniques that make the content concepts accessible and also develop students' skills in the new language. Using a quasi‐experimental design, the research was conducted in content area and English as a second language classes in two districts in northern New Jersey over 2 years. The analysis presents student achievement data from state‐mandated language proficiency tests in the final year of the intervention, after most of the treatment teachers had completed their professional development in the SIOP model. There were statistically significant differences in the average mean scores in favor of the treatment student group on Writing, Oral Language, and Total English scores of the IDEA Language Proficiency Tests with small to medium effect sizes. The results from this study show that the SIOP model offers a promising approach to professional development that can improve the quality of instruction to English language learners and increase their English language achievement.

Do Student Achievement Outcomes Differ Across Teacher Preparation Programs? An Analysis of Teacher Education in Louisiana

Achievement outcomes for students taught by recent program completers of Louisiana’s teacher preparation programs (TPPs) are examined using hierarchical linear modeling of State student achievement data in English language arts, reading, mathematics, science, and social studies. The current year’s achievement in each content area is predicted using previous achievement data, student characteristics, classroom characteristics (e.g., percentage of students with disabilities), school characteristics, and attendance of teachers and students. The contribution of a teacher having recently completed a specific TPP is modeled at the classroom level as an indicator variable for each TPP. Results for programs with 25 or more new teachers are reported. Results demonstrate substantial overlap in confidence intervals (CI) among programs. In some instances, 68% and/or 95% CI for programs in specific content areas did not overlap results for the average new teacher or experienced teachers (i.e., they were lower than average new teachers or higher than average experienced certified teachers). Results varied across content areas for some programs.

Teacher salary and national achievement: A cross-national analysis of 30 countries

Using national teacher salary data from the Organisation for Economic Co-operation and Development (OECD) and student achievement data from the Programme for International Student Assessment (PISA), this study compared secondary school teacher salary in 30 countries and examined the relationship between average teacher salary and national achievement in mathematics and science. Although new teachers in the U.S. are paid higher than the international average, U.S. investment to the salary of experienced teachers was lower than the international average. The study also found that the countries with higher average salary for experienced teachers are more likely to have higher national achievement. However, the national average salary for new teachers was not significantly associated with national achievement level.

A Statistical Approach to Identifying Schools Demonstrating Substantial Improvement in Student Learning

The rising tide behind the school turnaround movement is significant, as national education leaders continue to call for the rapid improvement of the nation's lowest-performing schools. To date, little work has been done to identify schools that are drastically improving their performance. Using publically available school-level student achievement data from Minnesota, we begin to detail the statistical and practical steps necessary to identify schools rapidly improving their performance.

Assessing North Carolina School Administration Candidates Using an Integrated Standards-Based Portfolio

North Carolina’s educational leadership programs for Masters of School Administration (MSA) and Add-on-Licensure (AOL) candidates recently completed a mandated “revisioning” process based on the North Carolina Standards for School Executives. As part of that process the MSA/AOL portfolio replaced the School Leaders Licensure Assessment (SLLA) that was previously required. In light of the national debate on the preparation of school principals and North Carolina’s adoption of new leadership standards, this article discusses the portfolio’s integration with those standards, university leadership programs, and research on student achievement. Ultimately, that integration strengthens the portfolio’s value as a tool for developing leaders who can improve student achievement. In turn, the leadership and student achievement data acquired during the principal evaluation process can be used in the future to improve leadership standards, university leadership programs, and the portfolio assessment itself.

Teaming up: Linking collaboration networks, collective efficacy, and student achievement

Improving student achievement through teacher collaboration networks is a current focus of schools in many countries. Yet, empirical evidence on the relationship between teacher networks and student achievement and mechanisms that may explain this relationship is limited. This study examined the relationship between teacher networks and student achievement and the mediating role of teachers’ collective efficacy beliefs. Data were collected from 53 Dutch elementary schools. Using social network analysis and multiple regression we analyzed teacher survey and student achievement data. Findings indicate that well-connected teacher networks were associated with strong teacher collective efficacy, which in turn supported student achievement.

Model Selection with the Linear Mixed Model for Longitudinal Data

Model building or model selection with linear mixed models (LMMs) is complicated by the presence of both fixed effects and random effects. The fixed effects structure and random effects structure are codependent, so selection of one influences the other. Most presentations of LMM in psychology and education are based on a multilevel or hierarchical approach in which the variance-covariance matrix of the random effects is assumed to be positive definite with nonzero values for the variances. When the number of fixed effects and random effects is unknown, the predominant approach to model building is a step-up method in which one starts with a limited model (e.g., few fixed and random intercepts) and then additional fixed effects and random effects are added based on statistical tests. A model building approach that has received less attention in psychology and education is a top-down method. In the top-down method, the initial model has a single random intercept but is loaded with fixed effects (also known as an “overelaborate” model). Based on the overelaborate fixed effects model, the need for additional random effects is determined. There has been little if any examination of the ability of these methods to identify a true population model (i.e., identifying the model that generated the data). The purpose of this article is to examine the performance of the step-up and top-down model building approaches for exploratory longitudinal data analysis. Student achievement data sets from the Chicago longitudinal study serve as the populations in the simulations.