Using Large-Scale Assessments to Evaluate Effectiveness of School Library Programs in California

Abstract

The authors are the joint recipients of the Phi Delta Kappa Outstanding Doctoral Dissertation Award for 2006. Theirs is the first such winner to be summarized in the Kappan, but it won't be the last. A brief summary of each year's winning study will appear in one of the late spring issues. ********** THE increased emphasis on accountability is causing educators to focus on what students have learned, rather than on what has been taught, and this has created a need for quantitative measures of the effectiveness of educational programs. Our study evaluated school library programs by comparing student achievement at schools that have libraries with achievement at schools that do not have libraries. According to the American Library Association (ALA), a school library is active, technology-rich learning environment with an array of information resources that combine effective learning and teaching strategies and activities with information access skills. (1) In addition, the ALA specifies that, for a school to be classified as having a school library, it must employ a credentialed library media teacher. We adopted these definitions for our study. We asked several research questions: Is there a difference in student achievement, as measured by standardized assessments in English/language arts and mathematics, between schools with similar demographics that either do or do not have a library program? Do criterion-referenced and norm-referenced testing instruments yield similar results? If a significant relationship exists between library programs and achievement, what factors within the programs may affect this relationship? We examined data from 4,022 California schools, including 2,589 elementary schools, 893 middle schools, and 540 high schools. To determine which of the schools had libraries, we used the California School Library Surveys for 2000-01 and 2001-02, developed and administered by the California Department of Education (CDE) with the California School Library Association, and the CDE School Library Database, which recorded the results of each school's survey. For our achievement data, we used each school's aggregate scores on the California Standards Tests (CSTs) in English/language arts and mathematics and on the California Achievement Test, 6th Edition Survey (CAT-6) in reading, language, and mathematics. The CSTs are criterion-referenced tests designed to measure student performance against the state's standards, and the CAT-6 is a nationally norm-referenced test designed to compare student achievement with that of a nationwide student population. RESEARCH METHODS While previous studies examined demographic factors independently to make comparisons among schools, research indicates that student achievement at schools with multiple challenges is actually lower than if the impact of each challenge were calculated independently. (2) To address the complexity of the schools' overall circumstances (socioeconomic status, English fluency, ethnicity, school staffing, year-round programs, and teacher credentialing), we used the California School Characteristics Index (SCI), which was developed by the CDE Policy, Evaluation, and Research Division, to compare similarly challenged schools with one another. Because the schools involved in our study differ from one another in student and staff demographics, an analysis of covariance (ANCOVA) allowed us to adjust mean performance levels to estimate how schools would have performed if they all possessed the same demographics. For this ANCOVA, the presence of a school library program was the non-linear independent variable, and the SCI, adjusted for school and student demographics, was the linear covariate. …