Effects of Halo-Effect Instructions on Level of Ratings and Intercorrelations

Abstract

This study is from a program of empirical examinations of the effects of format on students' ratings of effectiveness of college teaching. One crucial inadequacy of rating scales in general, and student rating scales in particular, is the The effect was defined by Stalnaker and Remmers (1928) as the tendency to allow the general impression of the ratee to influence markedly the evaluation of specific traits. Unpublished empirical evidence indicates intercorrelations of .30 or higher among most items. This implies a There is also considerable conjecture that the effect operates in students' ratings. The purpose of this exploratory study was to investigate if mere instructions ro students to rate each item independently of all other items would produce lower ratings and also lower intercorrelations than directions without this instruction. If the simple instruction to rate each item independently produced lower ratings and intercorrelarions then the cumbersome and little used forced choice format would not have to be used. Two classes in education at the University of South Florida in December, 1973 constituted the student samples. One instructor taught guidance and the other taught curriculum. Within each class, students were randomly assigned to the two treatment groups, Halo, or Conventional (no anti-Halo instruction). Ss were told they were participating in a study of the language of instructors' raring scales, and to rate the instructor and course as they ordinarily would if no study were being conducted. The rating scale was a conventional 15-item instructor and course rating scale developed at the University. Rating categories were: Far Above Average ( 5 ) , Above Average (4) . Average ( 3 ) , Below Average ( 2 ) , Far Below Average ( 1 ) . Ns, group reliability estimates based on analyses of variance, means adjusted for the five-point scale, and total unadjusted means and standard deviations, respectively, for Instructor 1 were: 24, .go, 4.07, 61.08 and 7.30 for Halo; and 26, .90, 4.11, 61.63 and 7.74 for Conventional. The same statistics for Instructor 2 were: 23, 9 1 , 3.37, 50.57 and 9.14 for Halo; and 25, .86, 3.51, 52.60 and 8.82 for Conventional. Separate analyses of variance of ratings indicated no significant differences on either total score or on any individual item for either instructor. Inspection of the items' intercorrelations within each group gave no noticeable differences in magnitudes of correlations for either instructor. The intercorrelations for each instructor ranged mostly from .OO to .60 with relatively few higher or lower. Some items for both treatments had noticeably higher intercorrelations than others but there were no noticeable differences between instructions. It is concluded that mere instruction to student raters to consider each item independently of all other items does not reduce halo effect.