Statewide Testing Program Research Articles

The Effects of Providing Students with Revision Opportunities in Alternate Assessments

ABSTRACT Some statewide testing programs allow students to receive corrective feedback and revise their answers during testing. Despite its pedagogical benefits, the effects of providing revision opportunities remain unknown in the context of alternate assessments. Therefore, this study examined student data from a large-scale alternate assessment that allows students to make multiple attempts until they find the correct answer to multiple-choice items. The students receive partial credit based on the number of attempts being made. The effects of the multiple-attempt approach on both test characteristics and student performance were investigated. The results indicated that, despite making most items on the assessment relatively easier, the availability of partial credit improved the strength of the items in distinguishing low-achieving and high-achieving students while maintaining high internal consistency among the test items. Although the students were able to increase their scores due to the inclusion of partial credit based on the number of attempts, the relative positions of the students remained nearly the same.

Language Matters: Teacher and Parent Perceptions of Achievement Labels from Educational Tests

ABSTRACT Since the standards-based assessment practices required by the No Child Left Behind legislation, almost all students in the United States are “labeled” according to their performance on educational achievement tests. In spite of their widespread use in reporting test results, research on how achievement level labels are perceived by teachers, parents, and students is minimal. In this study, we surveyed teachers (N = 51) and parents (N = 50) regarding their perceptions of 73 achievement labels (e.g., inadequate, level 2, proficient) used in statewide testing programs. These teachers and parents also sorted the labels according to their similarity. Using multidimensional scaling, we found labels used to denote the same level of performance (e.g., basic and below proficient) were perceived to differ in important ways, including in their tone and how much achievement they convey. Additionally, some labels were perceived as more encouraging or clear than others. Teachers’ and parents’ perceptions were similar, with a few exceptions. The results have important implications for reporting results that encourage, rather than discourage, student learning.

Prevalence of SARS-CoV-2 antibodies in healthy blood donors from the state of Tyrol, Austria, in summer 2020.

BackgroundSeroepidemiological studies provide important insight into the spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV‑2) in our society. We aimed to determine seropositivity of SARS-CoV‑2 antibodies and its cross-sectional correlates in a large cohort of blood donors.MethodsIn this observational cohort study, we tested healthy blood donors residing in Tyrol, Austria, for SARS-CoV‑2 antibodies using the Abbott SARS-CoV‑2 IgG chemiluminescent microparticle immunoassay. We estimated 95% confidence intervals (95% CI) of seroprevalences using bootstrapping and tested for differences by participant characteristics using logistic regression.FindingsBetween 8 June and 4 September 2020, we screened 5345 healthy individuals at local blood donor sessions (mean age 42.7 years, SD 13.5 years, 46.7% female). Overall seroprevalence was 3.1% (95% CI 2.7–3.6%, 165 cases), which is 5.1-fold higher (95% CI 4.5–6.0%) than the case number identified by the health authorities in the state-wide testing program (0.6%; 4536 out of 757,634). Seroprevalence was higher in the district Landeck (16.6%, P < 0.001) and in individuals aged < 25 years (4.7%, P = 0.043), but did not differ by gender, blood types, or medication intake. The odds ratio for seropositivity was 2.51 for participants who had travelled to Ischgl (1.49–4.21, P = 0.001), 1.39 who had travelled to other federal states (1.00–1.93, P = 0.052), and 2.41 who had travelled abroad (1.61–3.63, P < 0.001). Compared to participants who had a suspected/confirmed SARS-CoV‑2 infection but were seronegative, seropositive participants more frequently reported loss of smell (odds ratio = 2.49, 1.32–4.68, P = 0.005) and taste (odds ratio = 2.76, 1.54–4.92, P = 0.001).ConclusionIn summer 2020, SARS-CoV‑2 seroprevalence in Tyrolean blood donors was 3.1%. Our study revealed regional variation and associations with young age, travel history and specific symptoms.

Predicting performance on a statewide reading achievement test in Spanish with aimswebPlus Spanish

AbstractThe Every Student Succeeds Act requires that states make efforts to offer statewide achievement tests in languages, other than English, that are present to a significant extent in the state. Currently, four states offer Spanish reading or language arts assessments as part of their statewide testing program. In these states, practitioners may be interested in screening students who will complete the Spanish assessments to identify those who are at risk for poor performance. Spanish versions of curriculum‐based measures of reading appear well suited for this purpose. We conducted a retrospective analysis of data collected in a high‐performing district in Texas to investigate the predictive validity and diagnostic accuracy associated with the aimswebPlus Spanish scores when predicting statewide Spanish reading assessment performance for students in Grades 3 through 5. Across grades and seasons, the aimswebPlus Spanish scores explained a small amount of variance in statewide Spanish reading assessment (r 2 = 0.17–0.30). When risk on the aimswebPlus was determined using the national 35th percentile, screening decisions were associated with poor sensitivity (range = 0.50–0.72). Decisions were associated with higher specificity (range = 0.67–0.84) with adequate specificity obtained in four of six comparisons. Limitations, practical implications, and future research directions are discussed.

Physical Fitness and Motor Competence in Upper Austrian Elementary School Children-Study Protocol and Preliminary Findings of a State-Wide Fitness Testing Program.

Motor competence and physical fitness are key components for the promotion of an active and healthy lifestyle. Poor motor competence and low physical fitness in children, therefore, are a major threat to future public health. Even though the assessment of physical fitness and motor competence per se does not enhance these entities, fitness tests can provide important information for intervention strategies. Fitness tests may also motivate children to become more active in order to increase their physical abilities. In the school-year 2016/17 the Upper Austrian government initiated the state-wide testing program “wie fit bist du” (how fit are you) in elementary schools, that examined cardiorespiratory fitness, muscular power, speed, agility, flexibility and object control skills along with the assessment of height and weight. Since the beginning of the program more than 18,000 children between 6 and 11 years of age participated in the school-based tests. The results show a significant increase in the prevalence of overweight/obesity with increasing age (p > 0.01). Overweight/obese children displayed lower motor competence and physical fitness, except for upper body strength. Further, the improvement in test performance with increasing age was less pronounced in overweight/obese children compared to their normal weight peers. In fact, distance covered during the 6-min run did not improve throughout the elementary school years in overweight/obese children. Given the importance of motor competence and physical fitness for general development and well-being, physical education should be considered a viable setting for the promotion of these entities as a majority of children can be reached independent of their socio-economic background. In order to provide adequate movement experiences that enhance motor competence and physical fitness while ensuring a motivating environment, objective information on current ability levels are required. The implementation of fitness monitoring at young ages, therefore, can be an important contributor for the promotion of an active and healthy lifestyle.

Estimating School Effects with a State Testing Program Using Transition Matrices

ABSTRACTFor the past decade, the accountability model associated with No Child Left Behind (NCLB) emphasized proficiency on end of year tests; with Every Student Succeeds Act (ESSA) the emphasis on proficiency within statewide testing programs, though now integrated with other measures of student learning, nevertheless remains a primary metric for reporting achievement. We examine transition over categories across two years for three cohorts of middle school students as one means for evaluating change. We document school effects for seven models by analyzing relations among them and comparing them to the actual change in movement across categories. We report moderate to high correlations across the transition models though they differed vastly in the (a) assumptions behind them, and (b) ranking of schools with actual transitions. The findings, therefore, need to be considered in the context of assumptions embodied in each transition model to adequately understand their worth in reporting school effects.

84Comparative Effectiveness of Social Networks-Based Strategies Versus Standard Counseling, Testing and Referral for HIV Screening in a State-Wide Testing Program

84Comparative Effectiveness of Social Networks-Based Strategies Versus Standard Counseling, Testing and Referral for HIV Screening in a State-Wide Testing Program

Thirty-Seven and Counting: How Has AEFP Evolved from Its Origins?

It has been a busy time for the Association of Education Finance and Policy (AEFP). Over the past few years the association has acquired a new name, a new journal, and many new members. The 2012 annual conference, convened in Boston last March, proved to be the largest conference in the association’s thirty-seven-year history, with 556 members in attendance. The theme, selected by incoming president Deborah Cunningham, was “Education Finance, Policy, and Practice: The Role of Evidence in a Dynamic World,” which underscores the contemporary challenge to the association: how to apply an increasing abundance of information and sophisticated analytical tools to produce the evidence needed to guide decision making by educational policy makers and practitioners. The Boston meeting was notable not only for the number in attendance. The unique qualities and strengths of the association were in clear display: papers of unusual methodological rigor; an interdisciplinary mix of academics from the social sciences, public policy schools, and colleges of education; educational finance professionals, policy analysts, and practitioners, a mix rarely found in the same place; and sessions addressing today’s hot topics as well as issues that have endured over the years. Having said this, all indications are that AEFP is what it has always been: a small, diverse group of people tackling some really big problems. Of particular note was a trend that has been growing for years but has clearly come into full flower: the large

Accountability-assessment under No Child Left Behind: Agenda, practice, and future

The push for educational reform seems to be ubiquitous. For example, educational reform efforts have been documented in Australia (Brindley, 2001), Canada (Klinger, DeLuca, & Miller, 2008), Israel (Shohamy, 2001), the UK (Broadfoot, 1996), Germany (Rupp & Vock, 2007), and the USA (Wixson et al., 2003; Chalhoub-Deville, 2009a). Political rhetoric aimed at reform often seeks to underscore that education lies at the heart of economic development, international competitiveness, and social harmony (Haertel & Herman, 2005). The demand for accountability, however, which requires states to monitor and report achievement of all students, including English language learners (ELLs), and which entails a punitive system of sanctions for schools and educators based on student performance (defined in large part by test scores) is unprecedented, at least in the USA. Increasingly educational reform efforts in the USA have mandated the use of standardized tests and expanded a system of rewards/sanctions associated with test results. As Linn (2005) observes, ‘[i]n the 1960s ... attempts to use tests for school accountability generally failed to be approved ... In the following two decades, states moved toward statewide testing programs that had higher stakes initially for students and later for schools and educators’ (p. 2). As we stated in a previous publication,

A Comparison of Approaches for Improving the Reliability of Objective Level Scores

This study is an evaluation of the psychometric issues associated with estimating objective level scores, often referred to as ‘‘subscores.’’ The article begins by introducing the concepts of reliability and validity for subscores from statewide achievement tests. These issues are discussed with reference to popular scaling techniques, classical test theory, and item response theory. Methods for increasing the reliability of subscore estimates that have been suggested in literature are then reviewed. Based on this review, an empirical study comparing some of the more promising procedures was conducted. Test score data from a large statewide testing program were analyzed in this study. The comparison of subscore augmentation approaches found that generally all methods were very successful in dramatically increasing the reliability of subscore estimates. However, this increase was accompanied by near-perfect correlations among the subscore estimates. This finding called into question the validity of the resultant subscores, and therefore the usefulness of the subscore augmentation process. Implications for practice are discussed.

April Showers Bring May Flowers

WHILE STATE legislatures have been very busy during the first four months of the year, the rate of passage for bills does tend to crescendo as the sessions progress. By April, the volume is usually at its peak, as many legislatures speed toward their dates of adjournment. If April showers help generate flowers, perhaps this rain of legislative activity will generate new growth of its own. REPORTING ON EARLY WARNING INDICATORS The Indiana legislature was particularly busy this year. One section of H.B. 1347--a very big bill with many interesting components, among them increasing the compulsory attendance age to 18--appears to seek disclosure of early warning signs related to student engagement. The provision requires schools to report data on some new indicators that are highly predictive of students' dropping out of school. These new indicators include the number and percentage of students participating in school flex programs, which allow students with chronic attendance or academic issues to meet alternative standards for attendance; the number of students who have dropped out of school and their reasons for dropping out; the number of students who have not completed sufficient credits to advance to grade 10; and the number of students suspended for any reason. ACCELERATING LEARNING Another piece of Indiana's H.B. 1347 requires the state board to design a high school diploma that can be granted to those not currently enrolled who successfully complete a high school fast-track-to-college program. Qualified individuals must be at least 19 years of age and not enrolled in a school, or at least 17 years of age and have received consent from the high schools they have most recently attended. The bill authorizes state higher education institutions to establish such programs, which offer individuals an opportunity to earn a high school diploma while earning credits toward a college degree. In addition, Section 10 of H.B. 1347 creates the Indiana Double Up for College Program, which allows districts (or school corporations, as they are called in Indiana) and state education institutions to collaborate on offering early college, dual credit, or dual enrollment programs. High school juniors and seniors may enroll in courses under the program on a full-time or part-time basis. State institutions must accept or reject a student based on the standards ordinarily used to decide student admission. ASSESSMENT Kentucky H.B. 197 creates a new section of law defining core courses and requires the state department to begin developing standardized end-of-course (EOC) examinations in algebra I, algebra II, and geometry to be piloted at the end of the 2007-08 academic year. Content teachers in the core courses and postsecondary faculty members, including subject-matter specialists, are to be involved in the development and review of items for the exams. If resources are available, EOC exams are to be available to schools in an online format. Another new Indiana law (H.B. 1240) mandates that the department and the state board review the current statewide testing program. Long-term goals for a transition to a new testing program must include developing an assessment system that is less time-consuming and less expensive while maintaining the current level of rigor, that provides prompt results, that measures individual student growth from year to year, that explores all options for diagnostic tests for use by teachers, that is compatible with a move to online testing, and that assesses writing ability. South Carolina, too, has authorized a study to determine the feasibility and cost of converting the state assessment program to a computer-based or computer-adaptive format. New provisions in H.B. 4328 require tests to include a writing assessment and multiple-choice questions designed to reflect a range of cognitive abilities beyond the knowledge level. …

Editorial: The Search for Magic Solutions

The data are beginning to accumulate on the results of the various statewide testing programs mandated by the No Child Left Behind Act. Whether we prefer to see the glass as half full or half empty, it seems to me that there is some indication of improvement in educational outcomes in general, but that that large gaps in measured achievement continue to exist across identifiable segments of our school aged population. This applies specifically to children in one or more of the categories of poor (receiving free or reduced lunches), minority, limited-English-speaking, or disabled. Even the most optimistic projections do not envision all children from all groups achieving 100% proficiency by the target year 2014. Perhaps some politicians, but I would hope very few educators, believed that the imposition of rigorous testing would lead to revolutionary changes in our schools and eventuate into some magical improvement in school outcomes. Rather, more legislators apparently thought that legislating accountability by setting high standards, improving the preparation of teachers and other professionals, mandating scientifically proven instruction (e.g., a phonics to reading), and identifying and placing contingencies on schools and districts not making adequate yearly progress (AYP), THEN we would achieve magical results by 2014. I agree in general with all of these ideas. We do need higher expectations, better trained professionals, improved curriculum, scientifically based instruction, and accountability. I must admit that a reading of articles in this journal over the past several years suggests that the jury is out on the efficacy of a phonics-based approach to reading for children and that the procedures for identifying, working with, and even taking over schools and districts in need of improvement seem harsh and, to a large extent, counterproductive, in that the emphasis seems to be on negative reinforcement and punishment rather than on positive reinforcement and reward. I have accessed information on the results for and hard of hearing children in some states and the data consistently report that the percentages of children scoring at the proficient level fall below those of the general school age population. It is clear that we will not achieve the goals for 2014. For two of the states I have seen reports that suggest part of the problem may be related to American Sign Language (ASL). The rationale was given that because ASL is a separate language it is not fair to test children in English. The implication was that children can either learn ASL or English, but not both-an incorrect and dangerous implication. This led me to think about how we have been searching for a magic solution over the centuries in the form of prevention or a cure for deafness or through some simple application of an instructional system or mode of communication. A decade ago, ASL was being advertised by some professionals as the solution to all of the educational problems of children, an unrealistic and naive position. Most professionals I know who have actually taught children actively support the use of ASL, but know we have far to go whatever mode of communication or language is used. Attempts to prevent or cure deafness have a long history. More than a century ago, A. G. Bell was advocating restricting the marriages of people in order to prevent the development of a deaf variety of the human race. The eugenics movement was carried to its final extremes by Nazi Germany with its policy of sterilizing individuals or assigning them to labor camps to work them to death under inhuman conditions. Today, we have developments in human genetics with the possibility of genetic manipulation in the near future. Cures for deafness involving highly advanced digital hearing aids and implants are becoming increasingly common. …

Views from the Classroom: Teachers' Opinions of Statewide Testing Programs

This article discusses teachers' views on state-mandatedtesting programs. An overview of the literatureis presented, as well as results from a nationwidesurvey of teachers. Findings from both suggest thathigh-stakes state-mandated testing programs can leadto instruction that contradicts teachers' views of soundeducational practice. In particular, teachers frequentlyreport that the pressure to raise test scores encouragesthem to emphasize instructional andassessment strategies that mirror the content and formatof the state test, and to devote large amounts ofclassroom time to test preparation activities. The articleconcludes that serious reconsideration must begiven to the use of high-stakes consequences in currentstatewide testing programs.

A Comparison of Linear, Fixed Common Item, and Concurrent Parameter Estimation Equating Procedures in Capturing Academic Growth

A natural method to assess academic growth, and in turn facilitate inferences regarding the effectiveness of educational policy, administration, and instruction, is to compare the performance of students on standardized tests over time. A key methodological step in this process is equating test scores from separate administrations so that the scores are comparable. Three common item response theory equating approaches are investigated: linear transformation of separate calibrations, fixed common item parameter calibration, and concurrent calibration. Using data from a high-stakes statewide testing program, differences in mean growth were found between methods depending upon both the type of ability estimate used and the type of equating procedure implemented. Furthermore, depending upon the choice of equating method, large differences in classification may occur.

Views from the Classroom: Teachers' Opinions of Statewide Testing Programs

Views from the Classroom: Teachers' Opinions of Statewide Testing Programs

Prevalence of Accommodations for Students with Disabilities Participating in a Statewide Testing Program

This study reports empirical data on the use of various test accommodations for students with disabilities participating in the Missouri Assessment Program. The findings indicate that three accommodations, extended time, small-group administration, and read-aloud, account for nearly all of the students using an accommodation. The results also demonstrate that accommodations are rarely used in isolation. Students using one accommodation are very likely to use at least one additional accommodation. Although the read-aloud accommodation was not the most frequently used, we argue that more emphasis should be given to studying the effects of this accommodation.

The East Texas Bedding Plant Pack and Garden Performance Trials

The East Texas Bedding Plant Pack and Garden Performance Trials began several years ago at the Texas A&amp;M University Agricultural Research and Extension Center at Overton (Overton Center) with the goal of providing information on greenhouse and field performance of bedding plant varieties to the local bedding plant industry and consumers of these products. The program began with local trials that have now expanded in scope with the Smith County Master Gardeners Association playing an integral role in performing the trials. Entries are received from most of the major ornamental seed companies doing business in the United States giving the regional industry access to the only comprehensive greenhouse performance trials in this part of the country. Performance evaluation data is important to this industry since it has a wholesale value of over $500 million in the northeast Texas region, of which over $100 million is bedding plant production. The field performance trials are now replicated at the Overton Center, the Dallas Arboretum and Botanical Garden (Dallas Arboretum) and the Texas A&amp;M University Agricultural Research and Extension Center at Dallas (Dallas Center), giving over 5 million consumers in the northeast Texas region the opportunity to see how promising new plants from all over the world perform in the local climate. Plants that grow well in this climate have the potential to reduce inputs needed for production and use in the home or commercial landscape. Many of the top performing varieties from the bedding plant trials are also chosen to be part of the Coordinated Education and Marketing Assistance Program (CEMAP), a statewide testing program headquartered at the Dallas Center in which entries vie for designation as Texas Superstar plants. The comprehensive benefit of the East Texas Bedding Plant Pack and Garden Performance Trials is the link between the rural bedding plant producers and the urban consumers which serves as a basis for improving the quality of life for the citizens of Texas.

Accounting for the effects of increased high school graduation expectations on pupil performance and resource allocation: results from New York State

Increases in high school graduation requirements are prompting interest in the consequences for students, taxpayers, and educators. This study examines the experiences of New York State school districts that increased student participation between 1992 and 1996 in a statewide testing program that historically has been designed for college-bound students. The study includes statistical comparisons based on the universe of school districts in the State as well as the results of an intensive set of selective case studies. The results show that many different types of districts moved to increase participation levels during the study period and that increases in participation levels were associated with (a) modest declines in the percentage of students passing the exams; (b) unchanged drop-out rates; (c) increases in spending that were similar in magnitude to increases found in districts with little or no increase in participation; and (d) increases in professional staffing levels in core instructional areas. Implications for policy are explored.

Gender and Racial/Ethnic Differences on Performance Assessments in Science

We examined whether the differences in mean scores among gender and racial/ethnic groups on science performance assessments are comparable to the differences that are typically found among these groups on traditional multiple-choice tests. To do this, several hands-on science performance assessments and other measures were administered to over 2,000 students in grades five, six, and nine as part of a field test of California’s statewide testing program. Girls tended to have higher overall mean scores than boys on the performance measures, but boys tended to score higher than girls on certain types of questions within a performance task. In contrast, differences in mean scores among racial/ethnic groups on one type of test (or question) were comparable to the differences among these groups on the other measures studied. Overall, the results suggest that the type of science test used is unlikely to have much effect on gender or racial/ethnic differences in scores.

Gender and Racial/Ethnic Differences on Performance Assessments in Science

We examined whether the differences in mean scores among gender and racial/ethnic groups on science performance assessments are comparable to the differences that are typically found among these groups on traditional multiple-choice tests. To do this, several hands-on science performance assessments and other measures were administered to over 2,000 students in grades five, six, and nine as part of a field test of California’s statewide testing program. Girls tended to have higher overall mean scores than boys on the performance measures, but boys tended to score higher than girls on certain types of questions within a performance task. In contrast, differences in mean scores among racial/ethnic groups on one type of test (or question) were comparable to the differences among these groups on the other measures studied. Overall, the results suggest that the type of science test used is unlikely to have much effect on gender or racial/ethnic differences in scores.