Unsettling Scores: An Evaluation of Hennen's Annual Public Library Ratings

Abstract

ABSTRACT This paper analyzes the results of a study of the Hennen Annual Public Library Ratings (HAPLR). This study was conducted for the U.S. National Commission on Library and Information Science (NCLIS) as part of a larger survey of library assessment instruments. HAPLR ratings are based upon public library data available from the U.S. National Center for Education Statistics (NCES). The paper examines promotion of the HAPLR rating method and construction of its scores, summarizes literature on library input and output measures, reviews published criticisms, identifies other key shortcomings of the method, and explores selected statistical trends in NCES public library data from 1999 through 2004. The article concludes that imprecision in HAPLR scores renders the method unsuitable for quantitatively assessing comparative and individual library performance. Statistical validity of the method has never been demonstrated. Input/output data that HAPLR uses are inadequate measures of library performance and quality. The scores are susceptible to inaccuracies in the compilation of NCES data. Promoters of the HAPLR methodology should fully disclose the limitations and biases of the method. These limitations also need to be understood and heeded by the public library community, which has injudiciously joined its national library identity with a statistically unsound portrayal of its collective and individual institutional strengths and weaknesses.