Model based ranking of schools

Abstract

The problem studied in this chapter is the ranking of schools in terms of quality. There are many different ways in which we can quantify if and in how far a school is better than another one. One simple way is to compare means of student outcomes at the end of the school career. But this can be extremely misleading, because it does not correct the differences in school input. If we want to correct for background characteristics of students, in order to find out how well the school does with the input material it receives, then we have to use statistical models in order to make clear on what assumptions our corrections are based. We first study linear models with fixed regression parameters, such as ANCOVA and models with separate regression parameters for each school. In the first case the regression lines are parallel and the ranking of the schools is the same for all backgrounds. In the second case schools may be ranked quite differently for boys and girls, for blue collar and white collar children, and for high-IQ and low-IQ students. In our example, a 1959 data set of 1290 children in 37 schools in the city of Groningen, it turns out that indeed slopes differ, and thus rankings of schools differ for different backgrounds. We investigate how important and how real this effect is. More recently random coefficient models have been proposed for school effect analysis by Aitkin & Longford, Raudenbusch & Bryk, Mason, Wong & Entwistle, De Leeuw & Kreft. and Goldstein. In these random coefficient models schools are considered to be a random sample from a population of schools, and we want to make statements about this population and not necessarily about the individual schools in our study, interpreted as subpopulations. In random coefficient models the residuals in the regression equations are not independent for students in the same school. As a consequence if we predict the outcome for a student in a particular school, we also have to take the other students in the school into account. This means that predictions will generally be more conservative than in fixed coefficient models (the so-called shrinkage to the mean). It is argued in the paper that random coefficient models are more appropriate for school effects analysis. Rankings of the schools in the example are also computed, both for random intercept models (which are the random coefficient version of ANCOVA) and for random slope models (which correspond with nonparallel slopes models). Criteria are discussed which one can use to decide which rank orders are the most appropriate ones.