Stanford Achievement Test Research Articles

The Measure Matters: Examining Achievement Gaps on Cognitively Demanding Reading and Mathematics Assessments

This study presents estimates of Black–White, Hispanic–White, and income achievement gaps using data from two different types of reading and mathematics assessments: constructed‐response assessments that were likely more cognitively demanding and state achievement tests that were likely less cognitively demanding (i.e., composed solely or largely of multiple‐choice items). Specifically, the study utilized multilevel modeling of data from over 25,000 fourth‐ through eighth‐grade students participating in the 6‐state Measures of Effective Teaching (MET) study of 2009–2010, including data from the state reading and mathematics achievement tests used in MET districts at that time and data from the Stanford Achievement Test Open‐Ended Reading Assessment (SAT‐9OE) and the Balanced Assessment of Mathematics (BAM). The latter two assessments, consisting entirely of constructed‐response items, were selected by MET researchers to assess learning outcomes, such as those included in the Common Core State Standards, deemed more cognitively complex than those assessed by state achievement tests at the time. The investigator found that estimated Black–White, Hispanic–White, and income achievement gaps were smaller on the SAT‐9OE than on state reading assessments, before accounting for other relevant factors. Estimates of Black–White and Hispanic–White mathematics achievement gaps were slightly larger using BAM data, whereas the estimated income achievement gap was slightly smaller using BAM data. In later models, prior student academic achievement and average student subject‐specific prior achievement accounted for portions of these estimated achievement gaps.

Moving the Needle on Evaluating Multivariate Screening Accuracy

The assessment of screening accuracy and setting of cut points for a universal screener have traditionally been evaluated using logistic regression analysis. This analytic technique has been frequently used to evaluate the trade-offs in correct classification with misidentification of individuals who are at risk of performing poorly on a later outcome. Although useful statistically, coefficients from a multiple logistic regression can be difficult to explain to practitioners as it pertains to classification decisions. Moreover, classifications based on multivariate assessments are challenging to understand how performance on one assessment compensates for performance on another. The purpose of this article is to demonstrate and compare the use of logistic regression with classification and regression tree (CART) models in the identification of students who are at risk of reading comprehension difficulties. Data consisted of 986 Grade 1 students and 887 Grade 2 students who were administered a screening assessment at the middle of the school year as well as the 10th edition of the Stanford Achievement Test. Results indicated that CART performs comparably with logistic regression and may assist researchers and practitioners in explaining classification rules to parents and educators.

Replicating Criterion Validity in Science Content for the Combination of Critical Content Monitoring and Sentence Verification Technique

The purpose of the research was to replicate commonality analysis for two measures: critical content monitoring and sentence verification technique. Participants were 967 fourth-, fifth-, and sixth-grade students across seven public primary schools in a southeastern U.S. district. The predictor variables were administered as benchmarks 3 times in an academic year. Predictor scores were compared with science content test and reading comprehension scores from the Stanford Achievement Test–Tenth Edition abbreviated online form and a statewide accountability test. Commonality analysis results indicated that scores from both critical content monitoring and sentence verification technique added unique variance to explanatory models, replicating previous findings. In most cases, critical content monitoring scores provided the greatest percentage of unique and common variance to model explanations.

Effects of PowerPoint Presentations on Reading Comprehension of Deaf and Hard of Hearing Students in Iranian Exceptional High Schools

It is becoming increasingly difficult to ignore the role of PowerPoint presentation on reading comprehension of deaf and hard of hearing (D/HH) students in schools. Based on different studies, PowerPoint, a common multimedia tool, has a crucial role in teaching and learning through providing a suitable understanding of the text and motivating students. The present study aimed to examine the predictive effects of PowerPoint presentations on reading comprehension of D/HH students. As reported in the literature, reading levels of deaf high school students are equal to those of fourth grade hearing students. Therefore, a visual computer-mediated approach was implemented to monitor the reading comprehension progress of 20 D/HH students in Iranian high schools. The reading comprehension of the experimental and control groups were compared by conducting the Stanford Achievement Test and applying paired t-tests. Our findings indicated a significant difference between the mean reading comprehension scores of the two groups.

Assessing the Accuracy of Generalized Inferences From Comparison Group Studies Using a Within-Study Comparison Approach

Various studies have examined bias in impact estimates from comparison group studies (CGSs) of job training programs, and in education, where results are benchmarked against experimental results. Such within-study comparison (WSC) approaches investigate levels of bias in CGS-based impact estimates, as well as the success of various design and analytic strategies for reducing bias. This article reviews past literature and summarizes conditions under which CGSs replicate experimental benchmark results. It extends the framework to, and develops the methodology for, situations where results from CGSs are generalized to untreated inference populations. Past research is summarized; methods are developed to examine bias in program impact estimates based on cross-site comparisons in a multisite trial that are evaluated against site-specific experimental benchmarks. Students in Grades K-3 in 79 schools in Tennessee; students in Grades 4-8 in 82 schools in Alabama. Grades K-3 Stanford Achievement Test (SAT) in reading and math scores; Grades 4-8 SAT10 reading scores. Past studies show that bias in CGS-based estimates can be limited through strong design, with local matching, and appropriate analysis involving pretest covariates and variables that represent selection processes. Extension of the methodology to investigate accuracy of generalized estimates from CGSs shows bias from confounders and effect moderators. CGS results, when extrapolated to untreated inference populations, may be biased due to variation in outcomes and impact. Accounting for effects of confounders or moderators may reduce bias.

The Relationship of a Systemic Student Support Intervention to Academic Achievement in Urban Catholic Schools

Much of the achievement gap between rich and poor students can be attributed to out-of-school factors, yet few schools have a comprehensive, coordinated system for addressing students’ nonacademic needs. Within a group of Catholic schools located in one city, this study examines academic achievement on the Stanford Achievement Test battery in mathematics, reading, and language among 2nd through 8th grade students participating in such an intervention, and compares the results with those of similar nonparticipating students in nearby cities. Using hierarchical longitudinal growth modeling and adjusting for demographic characteristics, this study found that students in intervention schools outperformed the comparison group on average in 6th grade mathematics. Intervention students also experienced significantly higher rates of growth in achievement than the comparison group in all three subjects. The results suggest that systemic service provision models have the potential to help urban Catholic schools meet their mission of educating the whole child and serving the poorest families.

Why Are Economists Evaluating the Impact of Gifted Education?

Bui, Craig, and Imberman assessed the impact of gifted and talented programs on student achievement using regression discontinuity and random assignment to gifted magnet schools. In both analyses, they found minimal impact of the gifted and talented programs on student achievement. This commentary addresses two concerns associated with the study. First, the studies provided no description of the gifted students in the analyses, and some evidence suggests that the students in the study may not match typical conceptions of giftedness. Second, the outcome measures (Stanford Achievement Tests) may not have aligned well with the intent of the gifted program in the school district. In addition, this commentary asked why and how economists study the impact of gifted education programs. It is recommended that those who study and practice gifted education seek common ground on the goals and measurable outcomes of gifted education programs in ways satisfying to both the field of gifted education and to policy evaluators.

Academic Achievement of Deaf and Hard-of-Hearing Students in an ASL/English Bilingual Program.

There has been a scarcity of studies exploring the influence of students' American Sign Language (ASL) proficiency on their academic achievement in ASL/English bilingual programs. The aim of this study was to determine the effects of ASL proficiency on reading comprehension skills and academic achievement of 85 deaf or hard-of-hearing signing students. Two subgroups, differing in ASL proficiency, were compared on the Northwest Evaluation Association Measures of Academic Progress and the reading comprehension subtest of the Stanford Achievement Test, 10th edition. Findings suggested that students highly proficient in ASL outperformed their less proficient peers in nationally standardized measures of reading comprehension, English language use, and mathematics. Moreover, a regression model consisting of 5 predictors including variables regarding education, hearing devices, and secondary disabilities as well as ASL proficiency and home language showed that ASL proficiency was the single variable significantly predicting results on all outcome measures. This study calls for a paradigm shift in thinking about deaf education by focusing on characteristics shared among successful deaf signing readers, specifically ASL fluency.

The Benchmarking Capacity of a General Outcome Measure of Academic Language in Science and Social Studies

The amount of research evaluating the technical merits of general outcome measures of science and social studies achievement is growing. This study targeted criterion validity for critical content monitoring. Questions addressed the concurrent criterion validity of alternate presentation formats of critical content monitoring and the measure’s predictive validity. Participants were fifth-grade students ( N = 51) who completed five different forms of critical content monitoring probes as well as oral reading fluency and maze probes over three benchmarking periods. Criterion measures were the science and social studies subtests of the online abbreviated Stanford Achievement Test–10th edition. Concurrent correlation magnitudes for critical content monitoring ranged from .47 to .60. Predictive correlations for fall and winter ranged from .23 to .64. In three of four cases, commonality analyses findings favored critical content monitoring over oral reading fluency and maze as benchmarking choices. Study limitations and benchmark assessment framework implications are discussed.

Do explicit number names accelerate pre-kindergarteners’ numeracy and place value acquisition?

The purpose of this longitudinal study is to investigate whether an early childhood intervention using an explicit and transparent number naming system will have a lasting benefit to English and Spanish speaking children in their mathematics achievement related to number sense by accelerating their acquisition of concepts of numeracy and place value recognition. Statistical analysis of the results of the Stanford Achievement Test administered to school children in grades 1 through 5 at a school in Texas (USA) over a 6-year period of time showed consistent differences between those children who had participated in the intervention program by learning to count using explicit numbers names in Pre-Kindergarten and those who had not participated in the program. The advantages of this study were most apparent in children in the fourth grade, 6 years after their Pre-Kindergarten experience with explicit number names, particularly with those who had 2 years of experience with explicit number names.

Evaluation of the Effectiveness of the Alabama Math, Science, and Technology Initiative (AMSTI)

The Alabama State Department of Education (ALSDE) developed a statewide initiative to improve mathematics and science teaching and student achievement in kindergarten through grade 12 (K-12). The Alabama Math, Science, and Technology Initiative (AMSTI) is a two-year intervention intended to better align classroom practices with national and statewide teaching standards — and ultimately to improve student achievement — by providing PD, access to materials and technology, and in-school support for teachers.AMSTI, a schoolwide intervention, was introduced in 20 schools in 2002. Each year since then, the state has rolled out the program to additional schools within its 11 regions. By 2009, about 40% of the state’s 1,518 schools were designated as AMSTI schools. Funding for the program from the state legislature was $46 million in 2009.Given the policy relevance and level of investment in AMSTI, the Regional Educational Laboratory Southeast mounted a longitudinal, cluster randomized controlled trial to determine the effectiveness of AMSTI in grades 4-8, as implemented in five regions in the state. Previous evaluations of the program’s effects on students in K-12 did not use randomized controlled trials. The most recent evaluation (Miron & Maxwell 2007) reported that students in grade 5 in AMSTI schools outperformed students in non-AMSTI schools in math, science, and reading and students in grade 4 in AMSTI schools outperformed their counterparts in non-AMSTI schools in reading only. These evaluations used a study design that compared school-level test scores of AMSTI schools with non-AMSTI schools in the same district but did not establish preintervention comparability. This study’s RCT design improves on previous evaluations because it eliminates selection bias and establishes the preintervention comparability of the two groups.The effect of AMSTI on student achievement in math after one year, as measured by end-of-the-year scores on the Stanford Achievement Test Tenth Edition (SAT 10) math problem solving assessment of students in grades 4–8, was 2.06 scale score units. The difference of 0.05 standard deviation in favor of AMSTI schools is equivalent to a gain of 2 percentile points on the SAT 10 math problem solving assessment for the average control group student had the student received AMSTI. The 0.05 standard deviation is statistically significant but smaller than the effect the research team believed would be detectable by the experiment as designed. Whether this size effect is educationally important is an open question. It may be useful to convert this effect into a more policy-relevant metric — additional student progress measured in days of instruction. In these terms, the average estimated effect of AMSTI was equivalent to 28 days of additional student progress over students receiving conventional math instruction. The effect of AMSTI on student achievement in science, as measured by end-of-the-year scores on the SAT 10 science assessment, required only in grades 5 and 7, was not statistically significant after one year.AMSTI had a positive and statistically significant effect on classroom practices in math and science after one year. Based on multiple surveys in which teachers reported the number of minutes of active learning strategies used during the previous 10-day period, AMSTI math teachers averaged 49.83 more minutes, and AMSTI science teachers averaged 40.07 more minutes than control teachers. These estimated effects are equivalent to 0.47 standard deviation in math and 0.32 standard deviation in science. Although teachers in both the AMSTI and control groups reported using active learning instructional strategies, teachers in AMSTI schools reported spending more time engaged in this type of instruction.The exploratory investigation of the two-year effect of AMSTI on student achievement in science also found a statistically significant result, with a magnitude of 4.00 scale score units. This effect represents a difference of 0.13 standard deviation in favor of AMSTI schools, equivalent to a gain of 5 percentile points for the average control group student had the student received AMSTI for two years.The effect of AMSTI on teacher-reported content knowledge after one year was not statistically significant in either math or science. AMSTI did have a positive and statistically significant effect on student engagement after one year, measured on a 5-point scale ranging from “not engaged” to “fully engaged.” AMSTI teachers were more likely than control teachers to rate their students as achieving higher levels of engagement.An exploration of the differential effects of AMSTI on student achievement for subgroups of students found no statistically significant differential effects on student achievement in mathematics or science based on racial/ethnic minority status, eligibility for free or reduced-price lunch, gender, or pretest level. In reading, however, AMSTI had a statistically significant differential effect for minority and White students of 3.04 scale score points (p

PROTOCOL: Small Class Sizes for Improving Student Achievement in Primary and Secondary Schools: A Systematic Review

PROTOCOL: Small Class Sizes for Improving Student Achievement in Primary and Secondary Schools: A Systematic Review

Investigating the Efficacy of a Core Kindergarten Mathematics Curriculum to Improve Student Mathematics Learning Outcomes

This study examined the efficacy of a core kindergarten mathematics program, Early Learning in Mathematics (ELM), a 120-lesson program with four content strands: (a) number operations, (b) geometry, (c) measurement, and (d) vocabulary. The study utilized a randomized block design, with 129 classrooms randomly assigned within schools to treatment or control conditions. Measures of achievement were collected in the fall (pretest) and spring (posttest) in kindergarten and in the winter of first grade (follow-up). Although students in ELM classrooms did not differ significantly from students in control classrooms, we expected and found that the effects of ELM depended on students’ initial achievement. Initial achievement moderated condition effects for Test of Early Mathematics Ability (TEMA) scores (p =.0039), but not the Early Numeracy–Curriculum-Based Measurement (p =.1887). We found no effects on follow-up first-grade scores from the Stanford Achievement Test Tenth Edition Problem Solving subtest (p =.9737) or Procedures subtest (p =.6336).

The Development of Antonym Knowledge in American Sign Language (ASL) and Its Relationship to Reading Comprehension in English

It is unknown if the developmental path of antonym knowledge in deaf children increases continuously with age and correlates with reading comprehension, as it does in hearing children. In the current study we tested 564 students aged 4–18 on a receptive multiple‐choice American Sign Language (ASL) antonym test. A subgroup of 138 students aged 7–18 took the Stanford Achievement Test reading comprehension test. The results showed that antonym knowledge depended more strongly on age for deaf children with deaf parents than for deaf children with hearing parents. This indicates more developmentally typical acquisition for deaf children with deaf parents, consistent with early natural language exposure. Multiple regressions demonstrated that ASL antonym knowledge eliminated the advantage of deaf parents for reading. This establishes a language effect of ASL on reading comprehension in English.

The Test Matters

In this study, we examined how the relationships between one observation protocol, the Protocol for Language Arts Teaching Observation (PLATO), and value-added measures shift when different tests are used to assess student achievement. Using data from the Measures of Effective Teaching Project, we found that PLATO was more strongly related to the Stanford Achievement Test (SAT-9), the alternative assessment used by MET to assess more ambitious outcomes. We also found that the SAT-9 is more instructionally sensitive to the PLATO factor of Cognitive and Disciplinary Demand than the state tests used in MET study. This difference suggests that PLATO factors designed specifically to identify ambitious instructional practices are especially sensitive to which test is used to construct value-added scores.

Teachers’ practices: responding to governmentality in accountability testing policy

In this article, I offer an examination of findings from a recently completed qualitative study conducted in the south-eastern region of the US. In the study, nine teachers from three different schools representing two different school districts were interviewed twice about how they dealt with their students’ Stanford Achievement Test 10 (SAT 10) data to make instructional decisions. I frame the teachers’ reported practices as responses formed in the context of the governmentality undergirding current accountability testing policy. Using the lens of Foucault’s governmentality , makes it possible to see these ‘data-based’ practices as neither ‘neutral’, nor ‘objective’, but vulnerable to educators’ appropriation of ‘what is sayable and thinkable’ within a local accountability context. Teachers’ responses are seen as primarily shaped in discourses of meritocracy, deficit thinking and teachers’ professional knowledge.

Using Separate Answer Sheets with Grade 3 Students

ABSTRACTThe authors compared the performance of third-grade students testing on answer sheets with those testing on machine-scored test booklets. The 1,832 students in the nationally representative sample were assigned at the campus level to complete the Stanford Achievement Test Series, Tenth Edition in 1 of 4 conditions: (a) Form A answer sheet, (b) Form A booklet, (c) Form B answer sheet, and (d) Form B booklet. After controlling for scholastic ability, no significant differences in performance on total reading, total mathematics, and total language strands were found between students using booklets and those using answer sheets. The results of this study provide no evidence to support the need to use separate test booklets with general education third-grade students. States may consider using separate answer sheets with these students to realize potential cost and schedule efficiencies.

An Examination of the Efficacy of a Multitiered Intervention on Early Reading Outcomes for First Grade Students at Risk for Reading Difficulties

This article presents findings of an efficacy trial examining the effect of a multitiered instruction and intervention model on first grade at-risk students’ reading outcomes. Schools (N = 16) were randomly assigned to the treatment or control condition. In the fall of Grade 1, students were assigned to an instructional tier on the basis of Stanford Achievement Test–10th Edition scores (31st percentile and above = Tier 1; from the 10th to the 30th percentile = Tier 2). In both conditions, students identified as at risk (i.e., Tier 2; n = 267) received 90 min of whole group instruction (Tier 1) and an additional 30 min of daily small group intervention (Tier 2). In the treatment condition, teachers were trained to enhance core reading instruction by making instruction more explicit and increasing practice opportunities for students in Tier 1. In addition, at-risk readers were provided an additional 30-min daily small group intervention with content that was highly aligned with the Tier 1 core reading program. Results indicate significant, positive effects of the intervention on students’ decoding and first semester fluent reading and potentially positive effects on reading comprehension and total reading achievement.

The effectiveness of interventions designed to improve academic outcomes in children and adolescents in out-of-home care: a systematic review protocol

The review question is: What is the best available evidence regarding the effectiveness of interventions designed to improve academic outcomes in children and adolescents in out-of-home care? Inclusion criteria Types of participants This review will consider studies that include children and adolescents (under 18 years of age) placed (currently or formerly) in out-of-home care (OOHC). As the terminology for OOHC varies within the literature, this review will take an inclusive approach by including any placement setting that falls within the study’s definition of OOHC or according to the jurisdiction in which the study was undertaken. Examples of placement settings include: foster care, kinship care, group home, or residential care. Types of intervention(s) This review will consider studies that evaluate interventions designed to improve academic or school outcomes in children and adolescents in OOHC as defined below. Eligible interventions may include but are not limited to: tutoring programs, individualized educational support, education liaison, reading intervention programs, etc. Types of outcomes This review will consider studies that include at least one of the following outcomes: Primary outcome (academic outcomes): •Achievement or performance across a range of academic areas (reading comprehension, spelling, mathematics calculation and reasoning, writing, spelling, etc.) measured with school grades, grade point average (GPA) or scores on age standardized measurements (e.g., California Achievement Test, Stanford Achievement Test, Wide-Range Achievement Test, etc.); and teacher assessments or ratings. Secondary outcomes (school outcomes): •Attendance rates, grade retention, disciplinary referrals, suspensions, expulsions, dropout rates and graduation rates.

A Metalinguistic Awareness Test for ASL/English Bilingual Deaf Children: The TASLA-R

In this study, the researchers examined how deaf children could demonstrate their awareness of ASL by evaluating the correctness of ASL signs, sentences and discourse presented in stories in American Sign Language (ASL) signed by deaf native signers. To this end, a metalinguistic awareness test--the Test of American Sign Language—Receptive (TASLA-R), was created with eight short stories. The test consisted of 40 items presented in a multiple choice format test. Each of the 40 test items presented correct and incorrect statements about ASL formation using phonology, semantics, morphology, syntax and pragmatic aspects. One hundred and forty deaf children between the ages of 5 and 21 from four state schools for the deaf were administered the TASLA-R. Significant findings were as follows: 1) As the children got older, there was an increase in their ASL metalinguistic awareness and ability to identify incorrect ASL structures; 2) the deaf children of deaf parents group outperformed the deaf children of hearing group on the TASLA-R; 3) comparisons of TASLA-R scores between the junior high group and the high school were not significant; 4) scores on the TASLA-R showed moderate correlation to the English language, vocabulary and reading comprehension subtests on the Stanford Achievement test, 9 th Hearing Impaired edition. These findings were discussed in relation to Ellen Bialystok’s (2001) theories of metalinguistic uses of language in cognition and language development and in relation to the learning of ASL of bilingual deaf children.