Measuring competence in healthcare learners and healthcare professionals by comparing self-assessment with objective structured clinical examinations: a systematic review

Abstract

Executive summary Background The measure of clinical competence is an important aspect in the education of healthcare professionals. Two methods of assessment are typically described; an objective structured clinical examination and self-assessment. Objectives To compare the accuracy of self-assessed competence of healthcare learners and healthcare professionals with the assessment of competence using an objective structured clinical examination. Inclusion criteria Types of participants All healthcare learners and healthcare professionals including physicians, nurses, dentists, occupational therapists, physiotherapists, social workers and respiratory therapists. Types of intervention Studies in which participants were first administered a self-assessment (related to competence), followed by an objective structured clinical examination; the results of which were then compared. Types of outcomes Competence, confidence, performance, self-efficacy, knowledge and empathy. Types of studies Randomized controlled trials, non-randomized controlled trials, controlled before and after studies, cohort, case control studies and descriptive studies. Search strategy A three-step search strategy was utilized to locate both published and unpublished studies. Databases searched were: Medline, CINAHL, Embase, ERIC, Education Research Complete, Education Full Text, CBCA Education, GlobalHealth, Sociological Abstracts, Cochrane, PsycInfo, Mosby's Nursing Consult and Google Scholar. No date limit was used. Methodological quality Full papers were assessed for methodological quality by two reviewers working independently using the Joanna Briggs Institute Meta-Analysis of Statistics Assessment and Review Instrument (JBI-MAStARI). Data collection Details of each study included in the review were extracted independently by two reviewers using an adaptation of the standardized data extraction tool from JBI-MAStARI. Data synthesis Meta-analysis was not possible due to methodological and statistical heterogeneity of the included studies. Hence study findings are presented in narrative form. The data was also analyzed using 'The Four Stages of Learning' model by Noel Burch. Results The search strategy located a total of 2831 citations and 18 studies were included in the final review. No articles were removed based on the critical appraisal process. For both competence and confidence, the majority of studies did not support a positive relationship between self-assessed performance and performance on an OSCE. Conclusions Study participants' self-assessed competence or confidence was not confirmed by performance on an objective structured clinical examination. An accurate self-assessment may be threatened by over confidence and high performers tend to underestimate their ability. It is theorized that this disparity may in part be due to the stage that the learner or professional is in, with regard to knowledge and skill acquisition. Educators need to examine their evaluation methods to ensure that they are offering a varied and valid approach to assessment and evaluation. Notably, if self-assessment is to be used within programs, then learners need to be taught how to perform consistent and accurate self-assessments. Implications for practice It is important that educators understand the limitations within the evaluation of competence. Key aspects are the recognition of the stage that the learner is in with regard to skill acquisition and equipping both learners and professionals with the ability to perform consistent and accurate self-assessments. Implications for research There is a need for standardization on how outcomes are identified and measured in the area of competence. Further, identifying the leveling of an OSCE and the appropriate number of stations is required.