Professionalism Mini-Evaluation Exercise for medical residents in Japan: a pilot study

Abstract

Assessing medical professionalism among medical residents is of great importance. The Professionalism Mini-Evaluation Exercise (P-MEX) is a tool for assessing professionalism that was developed, tested for reliability and validated in Canada. Prior to the present study, no Japanese version of the P-MEX had been tested. We modified the P-MEX for use in Japan and tested it on medical residents in a Japanese teaching hospital. For each resident, eight evaluators completed the P-MEX forms. A total of 184 P-MEX forms were completed on 23 senior residents. The construct validity of the P-MEX was analysed by confirmatory factor analysis through structural equation modelling. The reliability of the P-MEX was tested using generalisability theory and a decision study. After performing the assessment and providing feedback to the residents, we conducted a survey on the residents' perceptions of the assessment. Results indicate content and construct validity. A confirmatory factor analysis revealed that factor loadings ranged from 0.58 to 0.96, indicating good construct validity except for one item (P12: Maintained appropriate boundaries with patients and colleagues). Structural equation modelling showed that adding new items developed in Japan to the P-MEX provided adequate factor validity. A decision study showed confidence intervals sufficiently narrow with as few as 10 evaluations, slightly more than the eight forms verified in Canada. Most residents stated that the items were reasonable and appropriate, the results of the assessment were consistent with their own self-evaluation and the assessment enhanced their motivation. Our study demonstrated good evidence of adequate reliability and validity of the P-MEX for the assessment of professionalism among Japanese residents. Moreover, the addition of new items developed in Japan provided adequate factor validity.