Development of an Objective Structured Clinical Examination Checklist and a Post-Education Questionnaire for Musculoskeletal Ultrasound Training Focusing on Volar Wrist and Carpal Tunnel Syndrome

ABSTRACTConstruct: The impact of using nonbinary checklists for scoring residents from different levels of training participating in objective structured clinical examination (OSCE) progress tests was explored. Background: OSCE progress tests typically employ similar rating instruments as traditional OSCEs. However, progress tests differ from other assessment modalities because learners from different stages of training participate in the same examination, which can pose challenges when deciding how to assign scores. In an attempt to better capture performance, nonbinary checklists were introduced in two OSCE progress tests. The purposes of this study were (a) to identify differences in the use of checklist options (e.g., done satisfactorily, attempted, or not done) by task type, (b) to analyze the impact of different scoring methods using nonbinary checklists for two OSCE progress tests (nonprocedural and procedural) for Internal Medicine residents, and (c) to determine which scoring method is better suited for a given task. Approach: A retrospective analysis examined differences in scores (n = 119) for two OSCE progress tests (procedural and nonprocedural). Scoring methods (hawk, dove, and hybrid) varied in stringency in how they awarded marks for nonbinary checklist items that were rated as done satisfactorily, attempted, or not done. Difficulty, reliability (internal consistency), item-total correlations and pass rates were compared for each OSCE using the three scoring methods. Results: Mean OSCE scores were highest using the dove method and lowest using the hawk method. The hawk method resulted in higher item-total correlations for most stations, but there were differences by task type. Overall score reliability calculated using the three methods did not differ significantly. Pass–fail status differed as a function of scoring methods and exam type, with the hawk and hybrid methods resulting in higher failure rates for the nonprocedural OSCE and the dove method resulting in a higher failure rate for the procedural OSCE. Conclusion: The use of different scoring methods for nonbinary OSCE checklists resulted in differences in mean scores and pass–fail status. The results varied with procedural and nonprocedural OSCEs.

OSCE (objective structured clinical examination) time can be a stressful time for many medical students who might feel underprepared and inexperienced. In this article we give an overview of the most useful apps for OSCE practice. -Free (offers in-app purchases, £3.99 to unlock all content) -Android (4.4/5 from 238 reviews) -Apple (4/5 from 22 reviews) They say: “Learn on the go with our comprehensive collection of high quality OSCE guides covering a wide range of commonly tested clinical skills.” Features -Guides to the following OSCEs: cardiovascular, respiratory, gastrointestinal, neurology, musculoskeletal, endocrine, obstetrics & gynaecology, ear, nose & throat, breast examination, clinical procedures and communication skills -Video demonstrations and images -Checklist mark schemes -Customisable timer Student BMJ’s verdict: This is one of the few OSCE apps on the market which includes videos and images of the clinical examinations, as well as text. Most of the content is locked until paid for. -Free - offers in-app purchase of £1.49 -Android (4/5 from 61 reviews) -Not available for Apple devices They say “Revision aide for medical OSCE examinations. Contains 47 mock OSCE sheets, which provides a checklist for students to work through in pairs, checking off each item to ensure correct procedure” Features -47 mock OSCE checklists - Stations include history-taking, practical and clinical skills - Available in test and practical mode - Timer mode - Exam mode which runs consecutive, random stations Student BMJ’s verdict You have to pay for the clinical OSCE checklists, but the free material is good quality. The app is easy to use and well designed. Being able to …

This study aimed to report the specific methods and investigate the educational effects of diagnostic musculoskeletal ultrasound training and the Objective Structured Clinical Examination (OSCE) for traditional medicine students. Scanning volar wrist and diagnosing carpal tunnel syndrome were selected for musculoskeletal ultrasound to train students to use the basic functions of the ultrasound device and scan various structures including tendons, nerves, and arteries. The students were divided into two groups: one group had 8 weeks of training with mock OSCE experience and received feedback about their scan images, and the other group had 3 weeks of training with flipped learning. The OSCE was implemented on the last day of the training. The subjective learning outcomes were analyzed as students’ evaluation with a 5-point scale, and the objective learning outcomes were analyzed using OSCE scores evaluated with a pre-validated checklist. Of the 111 students, 60 (54.1%) responded to the questionnaire. Overall satisfaction with this ultrasound training was high (4.5 ± 0.60). The average OSCE score in the 8-week group was significantly higher than that in the 3-week group. The students’ self-assessment showed no significant differences between the two groups. Proficiency in using ultrasound is affected by the practice time and feedback. Ultrasound training should be further expanded as a required curriculum to meet students’ needs and achieve learning objectives in the clinical skills education of Korean medicine colleges. Further studies are needed on ultrasound education, especially guided interventions for traditional medicine students.

Background: Implementing large-scale multi-site objective structured clinical examination (OSCEs) for national competency examination in a low resource country is challenging.Aims: To describe the first national OSCE for national competency examination of medical doctors in Indonesia and evaluate the reliability, validity, feasibility, acceptability, and educational impact.Methods: We collected electronically the OSCE scores from 49 out of 73 medical schools that participated to assess reliability and validity. We conducted electronic survey to examiners, examinees, SP trainers, and OSCE coordinators to assess feasibility, acceptability, and educational impact.Results: The Cronbach’s alpha coefficient across station was 0.79. There was strong correlation between rubric and global rating scores in each station (coefficient correlation ranges from 0.705 to 0.82). The content validity ratio was 0.97. The coefficient correlation between OSCE and MCQ was 0.335 (p = 0.00). All 49 medical schools were able to conduct OSCE simultaneously. Examiners, examinees, SP trainers, and OSCE coordinators had good perception regarding feasibility and acceptability of OSCE. Both examiners and examinees indicated good educational impact of OSCE application. The cutting score based on the borderline regression method was 61.96%. There were 67.39% of the examinees achieved similar or above the cutting score.Conclusion: With 12 stations 15 min each, the reliability coefficient across station is intermediate. Content validity is good. It is feasible and acceptable to implement large-scale multi-site OSCEs in Indonesia. Examiners and examinees perceive good educational impact on OSCE implementation.

To develop a valid and reliable tool for an objective structured clinical examination (OSCE) on hoarseness. To pilot-test the feasibility by assessing residents' clinical skills in various core competencies while assessing hoarseness on a standardized patient (SP). Educational tool development. The OSCE checklists were developed using modified Delphi technique after obtaining feedback from faculty involved in providing care to hoarseness patients. SP-based and rest stations were created to assess clinical skills. Twelve Otolaryngology-Head and Neck Surgery residents participated in the study. Video recordings of residents' performance and their written documentation were rated by faculty members. The OSCE that we developed is a valid method of assessing residents' clinical skills for evaluating hoarseness. Senior residents performed better in all of the tasks such as obtaining history and performing a physical exam on an SP, ability to perform flexible laryngoscopy on a mannequin, and interpretation of radiologic findings. Internal consistency assessed by Cronbach's alpha as measure of inter-item reliability was 0.92 for laryngoscopic station and 0.95 for radiology station. This OSCE can be effectively used for the objective assessment of clinical competency in hoarseness. Our pilot study evaluated multiple competencies on a single occasion, including medical knowledge, patient care, professionalism, and communication and interpersonal skills. Clinical competence in history taking, physical examination, flexible fiber-optic laryngoscopy, and ability to interpret radiologic findings improved with increasing year of training. This OSCE provides targeted assessment of practice-based learning and feedback for improvement of clinical performance.

Calibration of communication skills items in OSCE checklists according to the MAAS-Global

BackgroundThe purpose of this study was to develop an objective, content-valid, and reliable assessment method for Kampo medicine using an objective structured clinical examination (OSCE) for the assessment of clinical competence in Kampo medicine.MethodsWe developed a blueprint followed by a list of 47 assessment items and three task scenarios related to clinical competence in Kampo medicine. An eight-member test committee checked the relevance of the assessment items on a Likert scale. We calculated a content validity index and content validity ratio, and used the Angoff method to set the passing threshold. We trained a total of nine simulated patients with three assigned to each scenario. We conducted an OSCE for 11 candidates with varying medical abilities, and conducted three stations per person, which were evaluated by one evaluator in one room by direct observation. We used video recordings to test the inter-rater reliability of the three raters. We used the test results to verify the reliability of the assessment chart.ResultsThe inter-rater reliability (intraclass correlation coefficient [2,1]) was 0.973. The reliability of the assessment chart for each scenario (Cronbach’s α) was 0.86, 0.89, and 0.85 for Scenarios 1, 2, and 3, respectively. The reliability of the assessment chart for the whole OSCE (Cronbach’s α) was 0.90.ConclusionsWe developed a content-valid new OSCE assessment method for Kampo medicine and obtained high inter-rater and test reliabilities. Our findings suggest that this is one of the most reliable evaluation methods for assessing clinical competence in Kampo medicine.

To determine the reliability and validity of an objective structured clinical examination (OSCE) for musculoskeletal ultrasound (MSUS). A 9-station OSCE was administered to 35 rheumatology fellows trained in MSUS and to 3 expert faculty (controls). Participants were unaware of joint health (5 diseased/4 healthy). Faculty assessors (n = 9) graded image quality with predefined checklists and a 0-5 global rating, blinded to who performed the study. Interrater reliability, correlation between a written multiple choice question examination (MCQ) and OSCE performance, and comparison of fellow OSCE results with those of the faculty were measured to determine OSCE reliability, concurrent validity, and construct validity. Assessors' interrater reliability was good (intraclass correlation coefficient [ICC] 0.7). Score reliability was good in the normal wrist and ankle stations (ICC 0.7) and moderate in the abnormal wrist and ankle stations (ICC 0.4). MCQ grades significantly correlated with OSCE grades (r = 0.52, P < 0.01). The fellows in the bottom quartile of the MCQ scored 3.07 on the OSCE, significantly worse than the top quartile fellows (3.32) and the faculty (3.29; P < 0.01). Scores also significantly discriminated bottom quartile fellows from faculty in the normal wrist and ankle stations (3.38 versus 3.78; P < 0.01), but not in the abnormal stations (3.37 versus 3.49; P = 0.08). MSUS OSCE is a reliable and valid method for evaluation of MSUS skill. Normal joint assessment stations are more reliable than abnormal joint assessment stations and better discriminate poorly performing fellows from faculty. Therefore, MSUS OSCE with normal joints can be used for the assessment of MSUS skill competency.

Standardized procedures support learning of complex psychomotoric activities. We standardized the clinical otolaryngology examination to provide an objective basis for its operationalization. The Impact of the standardization on the objective measureable learning success and the student's evaluation were determined.During their 1-week ENT internship 166 medical students learned the standardized clinical otolaryngology examination. At the end of this week they absolved an objective structured clinical examination (OSCE). The students evaluated both, the teaching (standardization) and examination method (OSCE) with 2 questionnaires.All students passed the exam with mean score of 60.3±3.5 (max. 65) points. By using the OSCE checklist 2 independent examiners appraised nearly identical examination performances. In both evaluations the students rated the teaching and examination design positive. All students greatly appreciated the standardized clinical examinations and considered it as are very important for learning practical skills.The standardization of clinical otolaryngology examination permits objective and reliable tests for learning progress and success. Furthermore transparency, process reliability and test quality are considerably improved. The student's evaluation clearly reflects the mentioned advantages.

BackgroundAlthough options counseling is a fundamental skill for medical providers, previous research has identified gaps in medical school reproductive health education.PurposeTo determine if a 1-h novel patient interaction (NPI) improves student performance when caring for a standardized patient with an unintended pregnancy.MethodsFrom September 2012 to June 2013 we randomized third-year medical students at the University of Colorado School of Medicine to the standard curriculum plus an NPI, or the standard curriculum only. The NPI consisted of a 1-h small-group session with a patient who discussed her experiences with options counseling and her decision to terminate her pregnancy. Students completed an Objective Structured Clinical Examination (OSCE) at the rotation's end, which included options counseling. The primary outcome was the proportion of participants achieving ‘excellence’ on the OSCE checklist. ‘Excellence’ was defined as a score ≥90%. Examinations were flagged as ‘unsatisfactory encounters’ if core competencies were not addressed. OSCE standardized patients and evaluators were blinded to group assignment.ResultsIn total, 135 students were eligible and randomized: 75 to NPI; 60 to control. During the OSCE, few students achieved ‘excellence’ (24% NPI vs. 28% control, p=0.57).There were no differences between scores for components of options counseling. More students in the control group ‘appeared somewhat uncomfortable’ delivering the pregnancy test results (5% NPI vs. 18% control, p=0.006). More than half (54%) of the intervention group and 67% of controls had ‘unsatisfactory encounters’ (p=0.16), almost exclusively due to omission of adoption. Most students addressed abortion (96% NPI vs. 92% control, p=0.29).ConclusionsA 1-h NPI does not improve medical students’ performance of pregnancy options counseling and the option of adoption is routinely omitted. Adoption is clearly an area that needs greater attention when designing comprehensive reproductive health curriculum for medical students.

To measure the agreement among faculty members about the importance of items on a checklist used to grade an objective structured clinical examination (OSCE) station. Six faculty members rated the importance of 47 items for an OSCE station in which students took the history of a patient with sore throat and hoarseness. Of the 47 items, the raters independently identified 15 items as very important. The reliability of each individual rater was fair (averaged value of alpha = .63.) The reliability of the mean rating of the six raters was high (alpha = .91). The results strongly suggest that when a group of faculty members cooperatively identifies the important items to be included in an OSCE checklist, the reliability of the checklist is superior to one created by a single author.

Objectives: The new educational system emphasizes acquisition of clinical competency by the time of graduation from Korean medicine colleges that allow Korean medicine doctors to immediately perform clinical tasks. This study investigated awareness of competency-based education in Korean medicine hospital residents who must simultaneously undergo training and assist medical students in clinical practice.Methods: This was a qualitative research that was conducted using focus group interviews (FGIs) to investigate the awareness of demands for improvement in competency-based Korean medicine education in Korean medicine hospital. To apply the principles and procedures of FGIs, a semi-structured questionnaire was developed. Data analysis was conducted using the five steps of framework analysis.Results: According to contents analysis, first competency-based education that reflects actual clinical practice tasks is needed. Second, sufficient basic skill mastery education must be reinforced. Third, an intermediate curriculum that mediates clinical practice and basic education is needed. Fourth, the Objective Structured Clinical Examination and Clinical Performance Examination must be expanded to prepare for the Korean medicine doctor practical test.Conclusions: Korean medicine residents reported the gap between clinical practice and use of knowledge and skills acquired in the curriculum while acting as direct observers and educations of clinical clerkship in hospitals. Based on this exploratory study it is necessary to conduct research on the educational competency of Korean medicine residents who play an important role as educational leaders in Korean medicine clinical practice training.

Background: The use of an objective structured clinical examination (OSCE) has been a powerful influence on doctor training but assessments do not always drive study behaviour in predictable ways.Aims: To investigate the impact an OSCE has on study behaviours by exploring how 5th year medical students identify what to learn for a summative OSCE and the role of the clinical environment in their preparation.Methods: A semi-structured questionnaire survey asked about strategies used by students to prepare for the OSCE. Focus group interviews explored successful methods of preparation for the OSCE. Themes were identified and classified.Results: The questionnaire response rate was 84%. Topic identification was usually from the list of examinable problems, past OSCE papers and a booklet prepared by a previous student containing a series of OSCE checklists. The study behaviours of students preparing for the OSCE exam were predominantly to practise on each other, and to rehearse routines. Strategic and efficient study habits were favoured over conscious utilization of the clinical environment.Conclusion: The expectation that an OSCE drives learning into the clinical workplace was not supported by this study. This suggests the role of clinical experience in helping students prepare for the exam may be more subliminal, or that an OSCE is more as a test of psychomotor skills than a marker of clinical experience. An unexpected benefit may be to drive more collaborative learning.

Introduction: The objective structured clinical examination (OSCE) has been adopted by many universities for the assessment of healthcare competencies and as a formative teaching tool in both undergraduate and postgraduate nursing education programs. This pilot study evaluates the validity of OSCE checklists to be used in first‐year undergraduate nurse practice education.

Surgery residents’ perception of the objective structured clinical examination (OSCE)