Maryland School Performance Assessment Program Research Articles

Integrating Economics In The K-12 Curriculum

This is a multi-year study of the effects of integrating economics into the curriculum on the Maryland School Performance Assessment Program (MSPAP) economics outcome scores. The study was carried out using State summary and disaggregated data and summary data for each school system in the State of Maryland. The 1992, 1994, 1995, 1997 and 2000 MSPAP economics outcome scores for grades 3, 5, and 8, constitute the dependent variable for the study with the level of integration of economics in the curriculum and class size, being the main explanatory variables. Using comparative static analysis and regression analysis three questions were addressed. The first question asks how school systems with different levels of integration of economics in the curriculum compare to the State average. The second question asks how these groupings compare to each other. The third question employs regression analysis to determine the effect of economics instruction and class size on the economics test scores. Other explanatory variables considered include: wealth per capita, income per capita, expenditures per pupil, student enrollments, and the level of performance on the overall battery of MSPAP tests.

Schooling and test scores: A mother-natural experiment

In the economics of education, no task has been more important or more difficult than identifying the relationship between school inputs and student performance. The literature on this topic has reached little resolution, largely owing to the endogeneity of school resources. In this paper I examine the effect of a vital but little studied component of the education production function: instructional time. To identify the impact of schooling on test scores I make use of the fact that variation in winter weather made non-trivial differences in the number of school days students received prior to taking the Maryland School Performance Assessment Program (MSPAP) exams. I find evidence that students who took exams in years with heavy snowfall performed significantly worse than their peers in the same school who took MSPAP exams in other years. I also find that performance in a subject with relatively inflexible curricula (mathematics) and students in earlier grades were most affected by snow. Both of these findings are consistent with the interpretation that education inputs in the form of instructional days improve students’ test scores.

Impact of a state performance assessment program in reading and writing

The purpose of this research was to examine the impact of the Maryland School Performance Assessment Program (MSPAP) and the Maryland Learning Outcomes (MLOs) in a number of areas including teacher, principal, and student beliefs, classroom practices, and student learning in reading and writing. A statewide sample of 90 elementary and middle schools in Maryland participated in the study. Questionnaires were administered to principals, teachers, and students. Additionally, classroom instruction and assessment materials were collected from teachers, and student performance on MSPAP over 5 years was obtained. Results indicated that principals and teachers tended to be supportive of MSPAP, reading and writing classroom practices were somewhat aligned with the goals of MSPAP, and greater performance gains in reading and writing tended to occur in schools that reported increased use of reform-oriented instruction. The methodology and interpretive outcomes in the study contribute to the growing literature base on the effects of high stakes testing.

Examining the Construct Validity for the Multiple-Content Testing Programs

The primary objective of this study was to examine the construct validity for the 2 multiple-content testing programs-the multiple-choice Comprehensive Tests of Basic Skills (CTBS/5) together with the performance-based Maryland School Performance Assessment Program (MSPAP)-by evaluating the true-score longitudinal associations among multiple-content scores in 1 school district. Each content area measure in the 2 testing programs was closely examined by the following criterion: The true-score correlation between 2 time-period measures of the same content area is higher than its longitudinal true-score correlations with other content areas. This criterion was achieved in 2 (Reading and Mathematics) of 5 CTBS/5 content subtests, as well as 1 (Language) of 6 MSPAP content subtests. The structural equation modeling has been conducted on a multitrait-multimethod correlation dataset, where the traits of Reading and Mathematics were assessed by MSPAP and the old version of CTBS/4. Although convergent validity existed in these 2 measures, there was little evidence to support discriminant validity in both measures.

The Impact of a State Performance-Based Assessment and Accountability Program on Mathematics Instruction and Student Learning: Evidence From Survey Data and School Performance

The purpose of this study was to examine the impact of the Maryland School Performance Assessment Program (MSPAP) and the Maryland Learning Outcomes (MLOs) on mathematics classroom instruction and assessment practices, professional development, and student learning. The data sources included questionnaires for principals, mathematics teachers, and students, as well as student performance on MSPAP over a 5-year period. Ninety elementary and middle schools in Maryland participated in the study. The results indicate that principals and teachers tended to support MSPAP as a tool for making changes in instruction, teachers were making some positive changes in mathematics instruction because of MSPAP (based on the questionnaire data), and the schools for which teachers reported that MSPAP had a greater impact on their mathematics instruction had greater MSPAP performance gains in mathematics over the 5 years.

Are Multiple Measures Meaningful?: Lessons From a Statewide Performance Assessment

For practical, psychometric, and pedagogical reasons, strong interest exists in developing multiple-measure constructed-response items for use in large-scale performance assessments. Items that can be scored for evidence of proficiency in 2 or more content areas raise questions, however, about the "fit" between various content areas and the possibility of sending cross-messages or confounding different content demands. To determine the factors that contribute to or compromise the effectiveness of multiscored items, in this study we combine analysis of statewide score data from the 1996 Maryland School Performance Assessment Program tests, administered at Grades 3, 5, and 8, with systematic analysis of 60 activities providing measures of writing, language usage (LU), or both, as well as one or more content areas. Although test developers to date have had greater success in creating writing/LU items that can also be scored for reading and social studies than for mathematics and science, we argue for the validity of multiple-measure items across all content areas and suggest that, across content areas, successful multiple-measure items (a) make information sources explicit and allow students to draw on both text-based and personal knowledge; (b) identify the specific content area skills or concepts being assessed; (c) permit open-ended development; (d) maintain a good fit between content demands and the rhetorical situation, creating an authentic writer-audience relationship; and (e) are uncluttered, focused, and direct, with good recall capability. Thus, we suggest that test developers reorient their concerns from the difficulty of items to identifying elements of multiple measure activities that facilitate or impede students' ability to demonstrate what they know and can do in different content areas.

How Can Technology Enhance Children's Natural Curiosity?

Summary We want children to WANT to learn. Since they have a natural curiosity about so many things in their world, it is only right to base our teaching strategies around this curiosity. If we disguise our teaching objectives as child-centered activities, our students will soon realize that learning is fun and not something to dread. People often have a natural bond with animals, especially with cats and dogs. We have used technology to support our students' inquiry as they observed a puppy and kitten throughout several months. By linking technology with our county objectives, the Maryland School Performance Assessment Program science outcomes (MSPAP) and the National Science Education Standards we have provided the students with an exciting, real-world experience.

From Perception to Practice: The Impact of Teachers' Scoring Experience on Performance-Based Instruction and Classroom Assessment

Proponents of performance assessment often cite its potential to drive instructional reform. This has been the case with the Maryland School Performance Assessment Program, a battery of performance tasks administered annually to students in Grades 3, 5, and 8. Particularly because student responses to these tasks are scored in-state each summer by nearly 700 teachers, the scoring experience is regarded as a critical opportunity for professional development in support of such reform. As one means of examining the consequential validity of performance assessment, this study investigates the impact of the scoring experience on teachers' instructional and classroom assessment practice. Although teachers almost unanimously endorse the value of scoring, our analysis of interview data, questionnaires, classroom observation, and classroom artifacts demonstrates that their appropriation of performance-based instruction may be superficial and incomplete. From this study of a cadre of teacher-scorers comes a tempering of claims for the scoring experience as well as insights into the kinds of support needed to engender real and sustained changes in teaching and learning.

Editorial

Editorial High Stakes Testing: Are the Stakes Too High? On balance, it appears to me that the impact of federal legislation and mandates on the education of deaf children has been positive. In addition to general education mandates for accountability and improved academic achievement , this includes legislation specific to our field such as the Education of the Deaf Act and special education legislation from the Education of All Handicapped Children Act of 1975 to the Individuals with Disabilities Act (IDEA) of 1977. In fact, as this editorial is being prepared, it appears likely that Congress will "fully fund" up to 40% of the excess cost of educating a disabled child, a commitment that was made in 1975 but never honored. Perhaps the greatest potential benefit is exemplified by the IDEA stipulation that all children should have access to the general, or regular, curriculum. I am convinced that more deaf children from residential schools to neighborhood programs are exposed to a regular curriculum than ever before and that access to more rigorous standards has improved academic achievement. Of course, no new development is either completely positive or negative. We have had problems, for example, with a too enthusiastic embrace in some quarters of the mainstreaming/inclusion/zero reject/integration model(s) of school placement, and too many deaf children have suffered in isolation as a result of a misapplication of the least restrictive environment principle. However, my concern in this editorial is not with issues of placement but with the possibly deleterious effects on education of deaf children of the interaction of three seemingly very positive trends: access to the general education curriculum, upgrading the general education curriculum, and accountability. Until recently, I thought the issues were relatively simple. On the surface they are, but we are facing a potential mine field. I would like to discuss a few of these issues briefly. Most readers are aware that the federal government for years has been advocating improving the academic achievement of American children in the content areas of science, mathematics, social science, and literacy and has encouraged establishment of more rigorous curricula. Comparisons have been made internationally and among the states, and the federal government has issued reports that may be considered "report cards" identifying the highest achieving states and states showing the most improvement in various categories. The emphasis is on improvement and accountability. At first glance, the situation seems to be relatively straightforward . However, as all American educators know, we do not have a system of federal education, and the relationships between the federal government and the states are quite complex. In essence, all states are committed to improving academic achievement and to accountability. "Accountability" can mean different things to different people. High stakes testing has become a catch phrase in recent years, especially as it relates to accountability. In many states, a child must pass a statewide test in order to move from one grade to another or to graduate from high school. Although many educators and test developers decry tying one measure to such important decisions, the practice is common. The unit of analysis may be a teacher (class), principal (school), school superintendent (school district), or state director of education (state). High stakes testing can impact each of these areas. The scores of his or her students may heavily influence teacher evaluation and pay. Principals may get bonuses contingent upon schoolwide improvement on the tests. Schools may lose their accreditation if their students do less well than expected . There are cases of school superintendents promising to raise school district scores or leave. There is competition across states. The stakes are very high for everyone concerned. The tests themselves can vary from state to state. Even when the same tests are used, the cutoff standards may be different . The use of different tests can lead to differences in classroom instruction. For example, the two adjacent states of Virginia and Maryland have different approaches to testing. The Virginia Standards of Learning (SOL) program tests of core areas of math, English, science, and social studies from elementary to high school are all multiple choice. The Maryland School Performance Assessment Program (MSPAP) Volume 145, No. 3, 2000 American Annals of the...

A question of choice: The implications of assessing expressive writing in multiple genres

The Maryland School Performance Assessment Program (MSPAP) tests include an expressive writing task in which students at grades 3, 5, and 8 can choose to write about any topic they wish in the form of either a story, poem, or play. This test design feature provided the opportunity to investigate what factors contribute to students' choice of genre, how scorers apply a single expressive writing rubric to a range of genres, and whether these genres constitute equivalent tasks for measurement and reporting purposes. Our study, which combined analysis of statewide score data, 300 randomly selected student texts, questionnaires given to teacher-scorers, and interviews with students, argues strongly for the validity of this choice task as a measure of expressive writing and demonstrates that choice of genre both increases writers' engagement and enhances the fairness of the assessment by giving all students the best opportunity to demonstrate proficiency in this learning outcome. By highlighting several features of student texts that complicate scoring, the study also suggests that accuracy and consistency might be improved by 1. 1) providing additional sample papers during training, 2. 2) attending to scorers' assumptions regarding several key concepts, especially “originality,” and 3. 3) adjusting the ways that training for focused holistic scoring generally takes place. The study concludes that the perceptions of students, scorers, and classroom teachers are critical to the ongoing development of writing assessments that offer students increasing control and choice.

Contextual Characteristics of Locally Dependent Open-Ended Item Clusters in a Large-Scale Performance

A current trend in educational assessment is to move away from exclusive use of multiple-choice items and to use clusters of open-ended (or free-response, constructed-response) items. The purpose of this study is to identify and explain causes of local item dependence in a large-scale performance assessment. Data for the study come from the 1991 administration of the Maryland School Performance Assessment Program. Clusters of reading and mathematics items displaying high and low local dependence were identified. Contextual characteristics (i.e., content and response requirements) were identified to differentiate locally independent and dependent item clusters. Hypothesized explanations are made for high levels of local dependence where it exists.

The Maryland School Performance Assessment Program: Performance Assessment with Psychometric Quality Suitable for High Stakes Usage

The Maryland School Performance Assessment Program (MSPAP) is an innovative performance-based testing program covering reading, writing, language usage, mathematics, science, and social studies. MSPAP is administered annually on a census basis to students in Grades 3, 5, and 8, and the results are used for high-stakes, yearly evaluations of school performance and for tracking school improvement. The present article describes the program design and highlights its psychometric characteristics with respect to scaling, equating, standard setting, score accuracy, and validity.

Economics in the pre-college curricluum

This study analyzes the effects of integrating economics into the social studies curriculum and the effects of participation in the Stock Market Game (SMG) on the Maryland School Performance Assessment Program (MSPAP) economics outcome scores. The study was carried out using state summary and disaggregated data and summary data for each school system in Maryland. The 1992 and 1994 MSPAP economics outcome scores for grades 3, 5, and 8 constituted the dependent variable with the level of integration of economics in the curriculum and the level of participation in the SMG being the main explanatory variables. Using comparative static analysis, two related but separate questions were addressed. The first question asks how school systems with different levels of integration of economics in the curriculum and with different levels of participation in the SMG compare to the state average. The second question asks how these groupings compare to each other. It is this second question that lends itself to more rigorous hypothesis testing.

Can Assessment Mirror Instruction? A Look at Peer Response and Revision in a Large-Scale Writing Test

Both recent research and practitioners' classroom experience repeatedly affirm that the opportunity to obtain peer response and revise rough drafts helps writers to improve their texts. With the god of aligning assessment with exemplary instructional practices, on the Maryland School Performance Assessment Program tests is assessed through a multiday, process-oriented framework. Results of a study analyzing students' peer response feedback and subsequent revisions through comparison of rough and final draft stage at Grades 3, 5, and 8 suggest that within this testing context, students' peer response is unengaged, minimal, and formulaic; revisions are similarly sparse and superficial, with scores on final drafts remaining the same or decreasing with greater frequency than they increase. This investigation demonstrates that elements of classroom-based writing process instruction cannot simply be inserted into otherwise unchanged assessment formats. It suggests instead a number of principle...

Project Party

Assessing a student's ability to use mathematics in solving problems should render information on the student's use of strategies for problem solving, as recommended by the NCTM's Curriculum and Evaluation Standards for School Mathematics (1989). Traditional assessment tools do not give students the opportunity to display and communicate their strategies for solving a problem but, instead, focus on attaining a single answer to the problem. Opportunities for discussion and analysis are frequently lost when trying to form the basis for clear mathematical thinking and communication. To assess the problem-solving strategies of a group of ten fourth-grade students, a four-day activity titled “Project Party” was developed. This activity is a series of performance tasks based on the ideas from the Maryland School Performance Assessment Program (1990).

A Comparison of Equal Percentile and Partial Credit Equatings for Performance‐Based Assessments Composed of Free‐Response Items

This study compares the equal percentile (EP) and partial credit (PC) equatings for raw scores derived from performance‐based assessments composed of free‐response (open‐ended) items clustered around long reading selections or multistep mathematics problems. Data are from the Maryland School Performance Assessment Program. The results suggest that Masters (1982; Wright & Masters, 1982) partial credit model may be useful for equating examinations composed of moderately easy (or not too difficult)items sharing a first principal component with at least 25% of the total variance. This conclusion appears to hold even in the presence of some level of response dependency for the items within each cluster. Although visible discrepancies were found between PC and EP equated scores in the skewed tail of the score distributions, the direction of these discrepancies is unpredictable. Therefore, it cannot be concluded from the study that the two methods give equivalent results when the distributions are markedly skewed.