Construct validity of postgraduate conjoint assessment of master of surgery program of school of medical sciences at USM

Abstract

Introduction: A clear concept and understanding about the measure and the measuring tools is essential for good practice of assessment. Assessors need to have information about the full range of assessment tools inclusive of psychometric validity and purpose of its use. Subjective inferences drawn from the readily available data as numbers of summative scores over the years and statistical evidences of reliability and validity of assessment tools used to measure student’s performance are good sources of feedback for competent assessment program. It also provides meaningful evaluation of learning and teaching in medical education. Method: A retrospective study of 119 candidates was carried out to analyze the summative assessment scores of their certifying examination of Masters of Surgery in School of Medical Sciences (SMS) at Universiti Sains Malaysia. Subjective judgment of raw data followed by internal consistency as reliability, convergent validity and discriminant validity as constructs of individual assessment tool was analyzed. Finally each assessment tool as a measure of written or clinical construct was evaluated against six aspects of Messick’s criteria for quality control. Result: The correlation coefficient for validity and Cronbach’s alpha for reliability was evaluated for clinical measures. However, the test of internal reliability was not possible for essay being the only measure in written construct of summative assessment in surgery. All measures of clinical construct were found highly reliable with Cronbach’s alpha between 0.962-0.979. Long case and the short cases have shown excellent correlations (r=0.959 at p <0.001). Viva stood on its own and showed good correlation with long case (r=0.933 at p <0.001) as well as with short cases (r=0.926 at p <0.001). The linear regression analysis of essay measure was not predicted by any of the clinical measure. In clinical construct long case was strongly predicted by short case and vice versa (B=0.640 at p <0.001). Viva was predicted by the long case only (B=. 245 at p <. 001). All measures have positively predicted the overall performance however, the long case predominantly more than the other measure of this construct (r 2 =0.973 at p <. 001) Conclusion: Suggestions to improve the framework of assessment are proposed for future practice of competent assessment program in surgery.