The ASK-SEAT: a competency-based assessment scale for students majoring in clinical medicine

Linxiang Huang,Pi Guo,Gang Xin,Xiaoqing Huang,Haijie Xu,Zihua Li,Weijie Zhan,Yingying Zheng,Zeting Huang,Chibin Cheng,Shaoyan Zheng

doi:10.1186/s12909-022-03140-0

Abstract

BackgroundTo validate a competency-based assessment scale for students majoring in clinical medicine, ASK-SEAT. Students’ competency growth across grade years was also examined for trends and gaps.MethodsQuestionnaires were distributed online from May through August in 2018 to Year-2 to Year-6 students who majored in clinical medicine at the Shantou University Medical College (China). Cronbach alpha values were calculated for reliability of the scale, and exploratory factor analysis employed for structural validity. Predictive validity was explored by correlating Year-4 students’ self-assessed competency ratings with their licensing examination scores (based on Kendall’s tau-b values). All students’ competency development over time was examined using the Mann-Whitney U test.ResultsA total of 760 questionnaires meeting the inclusion criteria were analyzed. The overall Cronbach’s alpha value was 0.964, and the item-total correlations were all greater than 0.520. The overall KMO measure was 0.966 and the KMO measure for each item was greater than 0.930 (P < 0.001). The eigenvalues of the top 3 components extracted were all greater than 1, explaining 55.351, 7.382, and 5.316% of data variance respectively, and 68.048% cumulatively. These components were aligned with the competency dimensions of skills (S), knowledge (K), and attitude (A). Significant and positive correlations (0.135 < Kendall’s tau-b < 0.276, p < 0.05) were found between Year-4 students’ self-rated competency levels and their scores for the licensing examination. Steady competency growth was associated with almost all indicators, with the most pronounced growth in the domain of skills. A lack of steady growth was seen in the indicators of “applying the English language” and “conducting scientific research & innovating”.ConclusionsThe ASK-SEAT, a competency-based assessment scale developed to measure medical students’ competency development shows good reliability and structural validity. For predictive validity, weak-to-moderate correlations are found between Year-4 students’ self-assessment and their performance at the national licensing examination (Year-4 students start their clinical clerkship during the 2nd semester of their 4th year of study). Year-2 to Year-6 students demonstrate steady improvement in the great majority of clinical competency indicators, except in the indicators of “applying the English language” and “conducting scientific research & innovating”.

Highlights

To validate a competency-based assessment scale for students majoring in clinical medicine, ASK-SEAT
In 1978, McGathie et al prepared a report for the World Health Organization (WHO), advocating for cultivating medical talents through competency-based medical education (CBME) in order to meet the healthcare needs of local populations worldwide [1]
Questionnaire Questionnaires created based on the 24 indicators were distributed via an online platform from May through August 2018 to Year-2 to Year-6 students at the Shantou University Medical College (SUMC)

Summary

Introduction

To validate a competency-based assessment scale for students majoring in clinical medicine, ASK-SEAT. A group of international educators refined CBME as “an outcomes-based approach to the design, implementation, assessment, and evaluation of medical education programs, using an organizing framework of competencies” [2]. Developed countries such as U.K., U.S, and Canada have developed more comprehensive competencybased frameworks [3–6]. At CanMEDS 2015, a physician competency framework endorsed by 12 Canadian medical organizations was presented which identified multiple key roles played by a competent physician [6]: a) Medical expert— applying medical knowledge, clinical skills, and professional values to provide quality patient-centered care; b) Communicator—forming relationships with patients and their families which facilitate sharing essential information for the delivery of effective health care; c) Collaborator—working effectively with other health care professionals to provide quality patient-centered care; d) Leader—engaging with others to contribute to realizing visions of quality health care systems; e) Health advocate—contributing expertise and influence to improve healthcare when partnering with communities or patient populations;

Objectives

Methods

Results

Discussion

Conclusion