Radiology faculty evaluation: Bi-institutional implementation of an experimental appraisal instrument

Abstract

The authors evaluated the reliability and validity of an experimental radiology faculty appraisal instrument. Residents from the University of Wisconsin (UW) and Indiana University (IU) were asked to use a previously developed behaviorally based, 53-item experimental faculty appraisal instrument. Twenty UW residents evaluated 29 UW faculty members and 37 IU residents evaluated 31 IU faculty members by using the experimental instrument. Residents also evaluated faculty by using their institution's existing appraisal instrument. Correlations between existing and experimental forms were .69 and .87 for UW and IU, respectively. Existing form reliabilities were .89 and .94 and experimental form reliabilities were .98 and .98 for UW and IU, respectively. Experimental form length was reduced to 30 items by eliminating the questions that correlated the least with section scores. Reliabilities of scores on the shortened form were .97 and .98 and correlated .65 and .88 with scores on the longer form for UW and IU, respectively. Ratings obtained with the existing forms correlated substantially with the experimental form, attesting to the experimental form's validity. Shortening the experimental form had a minimal effect on the reliability and validity of the data obtained. The behavior-based form was used to rate behaviors that residents believed discriminated between effective and ineffective instructors, enabling an objective and relevant assessment to be made.