An Objective Peer Evaluation Scale: Construction and Validity

Abstract

Abstract : A study was made of peer ratings obtained by having the raters evaluate their peers on objective criteria of officer excellence. A peer evaluation scale was described in which statements describing the peer were selected on the basis of content analyses of paragraphs describing outstanding and inferior candidates for the billet of Marine Corps officers. The statements were written by enlisted marines with an average of 3.5 yr of active service. The items used in the peer evaluation scale were selected from these statements on the basis of their ability to discriminate between the men ranked first, fifth, tenth, and last, in a section of approximately 15 men. This preliminary scale was administered to a second independent group of candidates and 3 indexes for item selection purposes were calculated. These indexes were: a criterion index which was a measure of the relative agreement of the item with the criterion; a total test index, which was a measure of the relative agreement of the item with the total test score; and an item variability index, which was a measure of the relative intra-item variability. By using these 3 indexes, 20 items were segregated for use in the final scale. This refined peer evaluation was administered to an independent sample of candidates. The rank-difference correlation was found between the average peer score for a candidate and his rank position in the platoon as determined by 4 line officers. An average rho coefficient of 0.85 was obtained. Norms were determined for each platoon. A chi square test indicated homogeneity of the norms.