Determining relative performance in educational evaluation

Abstract

Determining relative performance in educational evaluation requires a statistical technique for analyzing test results which considers initial individual differences. One such field-tested statistical method (Klein & Erlich, Note I) is based on regression analysis and construction of a band around the regression line to account for errors of estimation of expected scores. In this method the relative performance of the student is classified as being or expectancy level, according to whether his score falls within, or the regression band lines. The bandwidth is a function of the standard error of estimate and the consideration of Type I and Type II errors. Defining bandwidth should depend on the nature of the data and the practical scale used for scoring performance. Once bandwidth is defined and the relative performance of each observation determined, the interpretation of the results should consider the expectancy level as well as the distance of the scores from the band line-defined hereafter as the band residual score. The Program. The purpose of the program is to: classify performance scores as below, at, or above expectancy levels; report the actual, expected, residual, and band residual scores for each observation; and describe for each expectancy level the number of observations, the mean score, and the mean band residual score. Program Calculations. At Step I, the program calculates an expected score, a residual score, an expectancy level, and a band residual score for each observation. At Step 2, the number and percentage of observations falling in each level of expectancy are calculated along with the mean score in each expectancy level, and the mean band residual score in the above and below expectancy levels. Input. The program input includes the following six cards: title card (Columns 1-80); name of the unit of analysis (e.g., student, class, school) (Columns 1-16); total number of variables (maximum 10 variables), numerical order of dependent variable, and number of input tape (312); data input format within parentheses (Columns 1-80); values of the constant term and of the independent variables' (predictors) regression coefficients (IOF8.0); standard error of estimate (SE) and a multiple of SE to define the width of half band (2F8.0). Information for the last two cards can be defined by any regression program available. Output. The program provides the following output: (l) Summary table for each observation, including actual score, expected score, residual score, expectancy score, and band residual score. (2) Summary table that describes, for each expectancy level, the number and percentage of observations, the mean actual score, and the mean band residual scores (only for above and below expectancy levels). Computer and Language. The program has been run on CDC 6600 and IBM 360/91 computers using FORTRAN IV; core requirement is lOOK; execution time is less than 10 sec. Availability. The source listing of the program, its documentation, and a sample application may be obtained at no cost from Oded Erlich, School of Education, Tel-Aviv University, Tel-Aviv, Israel.