Validity of In-Training Evaluation Report (ITER) in Saudi Pediatric Residency Training. Single center experience

Abstract

This article was migrated. The article was not marked as recommended. Objective: The study aimed to examine validity evidence of in-training evaluation report (ITER) at King Abdulaziz University Hospital (KAUH)'s pediatric residency program. Predictive validity of ITERs, to assess learner acquisitions of required competencies were studied. We used ITER scores to identify trainees with unprofessional conduct. Methods: One-year data for 35 residents in pediatric residency training program at KAUH were reviewed. Data were extracted from 360 ITERs completed by faculty members and specialists. ITERs' reliability was evaluated using Cronbach's alpha. ITERs scores were correlated with final Objective Structured Clinical Examinations (OSCE) and written exams scores. Professionalism composite scores from ITERs were compared with scores of trainees reported for unprofessional attitude. Results: Specialists had high variability in scoring competencies with average standard deviations (SD) of 0.85 compared with 0.15 by faculty. Reliability of ITER scores using Cronbach's alpha was 0.895. Good correlation was observed between competencies ranging between 0.86 to 0.29. There was fair correlation between ITER composite scores and those of final OSCE and written exams. There was no correlation between professionalism scores and residents reported for unprofessional conduct. Conclusion ITER scores had good validity and reliability despite variability observed between faculty and specialist when scoring competencies. ITERs failed to detect trainees with unprofessional conduct.