Science Achievement Measures Research Articles

Validity Evidence in Science Achievement Assessments Found in a Sample of Published Research Articles on Science Teaching

The authors add to published empirical findings that an appropriate range of validity evidence for achievement tests, including science and mathematics, has either not been gathered, not reported, or is not accessible for independent review. The current study focuses on a sample of published, peer reviewed science intervention studies from a single journal using science achievement measures, over an 11-years period, with some discussion of validity, and finds results similar to previous studies that were conducted in other STEM areas and broader contexts. The consensus of the educational measurement profession is that validity is assessed by the extent to which evidence and theory support the proposed interpretation of test scores for proposed uses. What is troublesome is that a shortfall in validity evidence raises concerns about faulty, or insubstantial, test score interpretation to inform student short-term and long-term education and career trajectories, and to inform curricula intervention improvements. Discussion cautions readers that though the study findings report shortfalls in some kinds of validity evidence, this simply raises a flag for the test user to consider what kinds of validity evidence apply to her test use. It also raises a flag for the profession to explore the reasons for shortfalls.

The association between science achievement measures in schools and TIMSS science achievements in Sweden

ABSTRACTThe aim of this study was to examine the association between students’ TIMSS (Trends in Mathematics and Science Study) science achievement, and students’ school achievements, in terms of national tests and grades from school years 6 and 9. Further to examine the association with TIMSS science achievement and different subgroups of students based on their home background. The study is based on a unique possibility to analyse TIMSS 2015 data together with register data of the Swedish students’ national test results from school years 6 and 9 and their science subject grades from school years 6 and 9. The overall results show that there were moderate associations between TIMSS science achievement and school achievement measures. The association between grades and the high-stakes national tests were stronger than between grades and TIMSS. The students’ home background had a clear impact on the results as students with highly educated mothers, who comes from homes with many books and are nonimmigrants had on average higher TIMSS science achievements.

Effect of project‐based learning on high school students' state‐mandated, standardized math and science exam performance

AbstractIn a technologically driven society, math and science students in the United States are falling further and further behind their international counterparts, resulting in an influx of STEM focused, reformed K‐12 schools, including schools focused on project‐based learning (PBL). This article reports a study of the effectiveness of PBL on high school students' performance on state mandated standardized mathematics and science achievement measures. Manor New Tech High School is a nationally recognized model STEM school, with a diverse student population, where all instruction is delivered through PBL. Although there is ample research suggesting that PBL is advantageous for increasing STEM learning compared to conventional teaching approaches, there is a lack of studies randomly assigning students to receive PBL. Further, some of the effects observed for students attending project‐based schools could be due to a self‐selection bias for students or parents that choose such an alternative learning environment. This study addresses both of these concerns and found that students taught through PBL, as a group, matched performance of conventionally taught students on all science 11th grade and mathematics 9th, 10th, and 11th grade TAKS achievement measures and exceeded performance by a scale score increase of 133 for the 10th grade science TAKS measure by (B = 133.082, t = 3.102, p < .05). One possible explanation of the differences observed in this study could be the TAKS instrument used to capture student math and science achievement that interprets “real‐life applications” of content differently between math and science questions. These results align with literature on the effects of PBL and deepen our understanding of these effects by providing a controlled study with random assignments to the PBL experience. Future research looking at the effect of PBL on achievement on the PISA could be beneficial in identifying benefits of PBL implementation in schools.

Is Inquiry Science Instruction Effective for English Language Learners? A Meta-Analytic Review.

Despite being among the fastest growing segments of the student population, English Language Learners (ELLs) have yet to attain the same academic success as their English-proficient peers, particularly in science. In an effort to support the pedagogical needs of this group, educators have been urged to adopt inquiry approaches to science instruction. Whereas inquiry instruction has been shown to improve science outcomes for non-ELLs, systematic evidence in support of its effectiveness with ELLs has yet to be established. The current meta-analysis summarizes the effect of inquiry instruction on the science achievement of ELLs in elementary school. Although an analysis of 26 articles confirmed that inquiry instruction produced significantly greater impacts on measures of science achievement for ELLs compared to direct instruction, there was still a differential learning effect suggesting greater efficacy for non-ELLs compared to ELLs. Contextual factors that moderate these effects are identified and discussed.

Effects of Inquiry-Based Instruction on Science Achievement for Students with Disabilities: An Analysis of the Literature

In comparison to the past, more students with disabilities are being included in the general education classroom for science instruction. Though inquiry-based instruction has not shown to be an effective practice for students with disabilities, it is vastly becoming the dominant practice in science education. The purpose of this review is to examine the effects of inquirybased instruction on science achievement for students with disabilities. The twelve studies, meeting selection criteria, report improvement in science achievement using inquiry practices. The participants and settings, variations of inquiry-based instruction, science achievement measures, and teacher training were addressed in this review. Two major contributions have resulted from analyzing the twelve studies. First, students with disabilities require supports to participate in an inquiry-based lesson and demonstrate progress on science achievement measures. Second, science achievement improves when components of explicit instruction are utilized in both the general and special education setting for students with disabilities.

Science Achievement Gaps Begin Very Early, Persist, and Are Largely Explained by Modifiable Factors

We examined the age of onset, over-time dynamics, and mechanisms underlying science achievement gaps in U.S. elementary and middle schools. To do so, we estimated multilevel growth models that included as predictors children’s own general knowledge, reading and mathematics achievement, behavioral self-regulation, sociodemographics, other child- and family-level characteristics (e.g., parenting quality), and school-level characteristics (e.g., racial, ethnic, and economic composition; school academic climate). Analyses of a longitudinal sample of 7,757 children indicated large gaps in general knowledge already evident at kindergarten entry. Kindergarten general knowledge was the strongest predictor of first-grade general knowledge, which in turn was the strongest predictor of children’s science achievement from third to eighth grade. Large science achievement gaps were evident when science achievement measures first became available in third grade. These gaps persisted until at least the end of eighth grade. Most or all of the observed science achievement gaps were explained by the study’s many predictors. Efforts to address science achievement gaps in the United States likely require intensified early intervention efforts, particularly those delivered before the primary grades. If unaddressed, science achievement gaps emerge by kindergarten and continue until at least the end of eighth grade.

Experimental evaluations of elementary science programs: A best-evidence synthesis

This article presents a systematic review of research on the achievement outcomes of all types of approaches to teaching science in elementary schools. Study inclusion criteria included use of randomized or matched control groups, a study duration of at least 4 weeks, and use of achievement measures independent of the experimental treatment. A total of 23 studies met these criteria. Among studies evaluating inquiry-based teaching approaches, programs that used science kits did not show positive outcomes on science achievement measures (weighted ES = +0.02 in 7 studies), but inquiry-based programs that emphasized professional development but not kits did show positive outcomes (weighted ES = +0.36 in 10 studies). Technological approaches integrating video and computer resources with teaching and cooperative learning showed positive outcomes in a few small, matched studies (ES = +0.42 in 6 studies). The review concludes that science teaching methods focused on enhancing teachers' classroom instruction throughout the year, such as cooperative learning and science-reading integration, as well as approaches that give teachers technology tools to enhance instruction, have significant potential to improve science learning. © 2014 Wiley Periodicals, Inc. J Res Sci Teach 51:870–901, 2014.

Exploring the Technical Expression of Academic Knowledge: The Science-in-CTE Pilot Study

The Science-in-CTE pilot study tested a curriculum integration model that enhanced the science that oc- curs in CTE curricula. The study replicated the National Research Center for Career and Technical Ed- ucation’s (NRCCTE) Math-in-CTE experimental research design (Stone, Alfeld, & Pearson, 2008) with applied science in secondary agricultural education. The semester-length study was conducted in North Dakota with secondary agricultural educators who were randomly selected to participate in the experi- mental and control groups. The experimental treatment mirrored the Math-in-CTE model of extended professional development, partnering experimental CTE group teachers with science educators, and use of a 7-element pedagogic framework. Standardized measures of science achievement were administered to students to determine the impact of the treatment on their science knowledge and skills. The results of hierarchical linear modeling (HLM) analysis indicated that the intervention had a statistically significant positive impact on posttest science achievement for students in the 2nd, 3rd, and 4th quartiles on pretest science achievement. We interpreted the intervention’s effects as small for participants in the 2nd quartile and as moderate for those in the 3rd and 4th quartiles. Relative to the control group, the intervention ap- peared to have no impact on students in the 1st quartile on pretest science achievement.

The Impact of Science Knowledge, Reading Skill, and Reading Strategy Knowledge on More Traditional “High-Stakes” Measures of High School Students’ Science Achievement

This study examined how well cognitive abilities predict high school students’ science achievement as measured by traditional content-based tests. Students (n = 1,651) from four high schools in three states were assessed on their science knowledge, reading skill, and reading strategy knowledge. The dependent variable, content-based science achievement, was measured in terms of students’ comprehension of a science passage, science course grade, and state science test scores. The cognitive variables reliably predicted all three measures of science achievement, and there were also significant gender differences. Reading skill helped the learner compensate for deficits in science knowledge for most measures of achievement and had a larger effect on achievement scores for higher knowledge than lower knowledge students. Implications for pedagogy and science assessment are discussed.

Effects of thematic‐based, hands‐on science teaching versus a textbook approach for students with disabilities

AbstractInstitutions of higher education, states, and local K–12 districts have been challenged to improve math and science education for our nation's students. In the past few years, there has been renewed interest in inquiry‐based, activity‐oriented instruction in science for students with disabilities. Yet, there still exists the need for further empirical evidence to support instructional improvements for students with more involved emotional and behavioral disabilities. This study describes a program in which 18 middle school students with serious emotional disturbances were instructed, over the course of 8 weeks, on “Matter” by two different instructional approaches. Students in one classroom received a traditional textbook approach to science content, whereas students in another classroom received science instruction by a hands‐on, thematic approach. Over the course of instruction, data were collected regarding students' behavior and achievement. Results indicate that, overall, students in the hands‐on instructional program performed significantly better than the students in the textbook program on two of three measures of science achievement, a hands‐on assessment and a short‐answer test. The students did not differ on a multiple‐choice format test. With regard to behavior, there were no significant differences in behavioral problems found between the two groups of students over the course of the study. Findings are discussed in relation to instructional implications for students with disabilities in the science classroom. © 2005 Wiley Periodicals, Inc. J Res Sci Teach 42: 245–263, 2005

The Versatile Multiple-Choice Item

The present study used four editions of a Grade 9 science examination to test validity changes in multiple-choice items as a result of changes in item length, reading speed and difficulty, language difficulty, and the omission of leads. It discovered that such changes failed to adversely affect the items’ validity or to increase the tests’ correlation with either intelligence or reading scores. The study also produced some evidence to show that reading test scores are more valid measures of science achievement than are the grades assigned by teachers on the bases of essay items.

Factor Analyses of High School Science Achievement Measures

Factor Analyses of High School Science Achievement Measures