Comparative evaluation of weighted mean and composite score for combining assessments.

Abstract

Approaches for combining scores have been based on weighted mean (WM) without consideration for psychometric characteristics of each individual assessments. This study evaluates consequences of WM and composite score (CS) approach. Data from two longitudinal cohorts (n=219) were utilized for performance in three Operative Dentistry courses as basis to compare the two score-combining methods. Four assessments (two written and two practical exams) from each course were combined using WM and CS approaches. WM scores were calculated by multiplying the score by its weight and summing across assessments. The CS approach follow a modification of the Kane and Case method, by standardizing scores, taking into account the reliability and associations between each assessment score. t-Tests and Pearson's correlation were used to evaluate the consequences of the WM and CS approaches. In addition, changes in each student's rank across WM and CS were determined. Combining scores using CS method produced lower scores and higher percentage failure in all courses compared to WM. Students ranks were changed significantly when CS was used with only 15% of the cohorts retained their ranks. CS produced a composite that is correlated with WM but still being substantively different providing meaningful and psychometrically rigorous information.