Clinical Skills Test Research Articles

Assessment of clinical skills in medical practice.

The introduction of a clinical skills examination (CSE) to Step 2 of the U.S. Medical Licensing Examination (USMLE) has focused attention on the design and delivery of large-scale standardized tests of clinical skills and raised the question of the appropriateness of evaluation of these competencies across the span of a physician's career. This initiative coincides with growing pressure to periodically assess the continued competence of physicians in practice. The USMLE CSE is designed to certify that candidates have the basic clinical skills required for the safe and effective practice of medicine in the supervised environment of postgraduate training. These include history taking, physical examination, effective communication with patients and other members of the health care team, and clear and accurate documentation of diagnostic impressions and plans for further assessment. The USMLE CSE does not assess procedural skills. As physicians progress through training and enter practice, both knowledge base and requisite technical skills become more diverse. A variety of indirect and direct measures are available for evaluating physicians, but, at present, no single method permits high-stake inferences about clinical skills. Systematic and standardized assessments make a contribution to comprehensive evaluations, but they retain an element of assessing capacity rather than authentic performance in practice. Much work is needed to identify the optimal combination of methods to be employed in support of programs to ensure maintenance of competence of practicing physicians.

Residency Program Director Evaluations Do Not Correlate With Performance on a Required 4th-Year Objective Structured Clinical Examination

Background: Assessment of resident performance is a complex task. Purpose: To correlate performance on a 4th-year objective structured clinical examination (OSCE) with residency program director assessment, class rank, and U.S. Medical Licensing Examination (USMLE) scores. Methods: We surveyed program directors about the performance of 50 graduates from our medical school chosen to represent the highest (OSCEHI) and lowest (OSCELO) 25 performers on our required 4th-year OSCE. Program directors were unaware of the OSCE scores of the graduates. Results: OSCE scores did not correlate with Likert scores for any survey parameter studied (r <. 23, p >. 13 for all comparisons). Similarly, program director evaluations did not correlate with class rank or USMLE scores (r <. 26, p >. 09 for all comparisons). Conclusions: We concluded that program director evaluations of resident performance do not appear to correlate with objective tests of either clinical skills or knowledge taken during medical school. These findings suggest that more structured and objective evaluative tools might improve postgraduate training program assessment of trainees.

Can internet-based continuing medical education improve physicians’ skin cancer knowledge and skills?

We sought to determine whether an Internet-based continuing medical education (CME) program could improve physician confidence, knowledge, and clinical skills in managing pigmented skin lesions. The CME program provided an interactive, customized learning experience and incorporated well-established guidelines for recognizing malignant melanoma. During a 6-week evaluation period, 354 physicians completed the online program as well as a pretest and an identical posttest. Use of the CME program was associated with significant improvements in physician confidence, correct answers to a 10-question knowledge test (52% vs 85% correct), and correct answers to a 15-question clinical skills test (81% vs 90% correct). We found that the overall improvement in clinical skills was due to a marked increase in specificity and a small decrease in sensitivity for evaluating pigmented lesions. User satisfaction was extremely high. This popular and easily distributed online CME program increased physicians’ confidence and knowledge of skin cancer. Remaining challenges include improving the program to increase physician sensitivity for evaluating pigmented lesions while preserving the enhanced specificity.

Can Internet-Based Continuing Medical Education Improve Physicians' Skin Cancer Knowledge and Skills?

We sought to determine whether an Internet-based continuing medical education (CME) program could improve physician confidence, knowledge, and clinical skills in managing pigmented skin lesions. The CME program provided an interactive, customized learning experience and incorporated well-established guidelines for recognizing malignant melanoma. During a 6-week evaluation period, 354 physicians completed the on-line program as well as a pretest and an identical posttest. Use of the CME program was associated with significant improvements in physician confidence, correct answers to a 10-question knowledge test (52% vs 85% correct), and correct answers to a 15-question clinical skills test (81% vs 90% correct). We found that the overall improvement in clinical skills was due to a marked increase in specificity and a small decrease in sensitivity for evaluating pigmented lesions. User satisfaction was extremely high. This popular and easily distributed online CME program increased physicians' confidence and knowledge of skin cancer. Remaining challenges include improving the program to increase physician sensitivity for evaluating pigmented lesions while preserving the enhanced specificity.

Differences unrelated to clinical competence in the results of repeated multiple-station tests of clinical skills.

To determine the extent to which systematic differences not related to group differences in clinical competence could be observed in the results of six administrations of the same multiple-station test of clinical skills. The same 16 stations were administered as part of the Ontario International Medical Graduates Program clinical skills screening examination on three different occasions, and on each occasion in two sessions. The interval between the first (fall 1986) and second (March 1990) administrations was four years, and that between the second and third (July 1990) administrations, four months. International medical graduates were the candidates in the first two administrations; the third administration was to a combined sample of fourth-year clinical clerks and interns. A different item functioning (DIF) approach with sessions within administration as the grouping variable was used to describe the extent of differences unrelated to clinical competence in the results of the different test administrations. For the generalizability and DIF calculations the designs were balanced by sampling down to 33 cases in each session. Station DIF values varied considerably from station to station. DIF effects between sessions with administrations were less than those between administrations, and were less between the two administrations separated by four months than between the two administrations separated by four years. Hypotheses concerning the relative magnitudes of station DIF and total-test aggregated values of DIF, as a function of the time intervals between test occasions, were substantiated, demonstrating that the greater the time interval between test administrations, the greater the magnitude of DIF.

Evaluation of a Fetal Monitoring Education Program

To evaluate the effectiveness of a fetal monitoring education program in increasing nurses' knowledge and clinical skills. Multicenter randomized control trial. Twelve hospitals in eastern Ontario, Canada. One hundred nine volunteer registered nurses randomly assigned, within each hospital, to an experimental (n = 47) or control (n = 62) group. Ninety-six nurses (40 in the experimental group and 56 in the control group) completed the 6-month follow-up (88% retention). The experimental group participated in a 1-day fetal monitoring workshop and a review session 6 months later. Performance on a 45-item knowledge test and a 25-item skills checklist. The passing score was at least 75% correct on each test. The percentage of nurses in the experimental group passing both the knowledge and the clinical skills tests after the workshop was significantly higher (p < 0.01) than that of the nurses in the control group: 68.1% versus 6.5%, respectively. A large difference between the groups remained at the 6-month follow-up (experimental, 45%; control, 6.5%). The performance of the nurses in the experimental group improved to an 85% pass rate after they attended the 6-month review session. This comprehensive, research-based program is effective in increasing fetal monitoring knowledge and clinical skills.

A comparison of short- and long-case stations in a multiple-station test of clinical skills

A comparison of short- and long-case stations in a multiple-station test of clinical skills

Medical Competence and Performance Assessment

Historically, the competence of physicians has been based on peer review of their performance. Formal assessment at the time of licensure and certification has been done by cognitive testing, a relatively easy method that is supported by a science of psychometrics. The assessment of actual performance has been less scientific, but that is changing. The National Board of Medical Examiners (NBME) and the Educational Commission for Foreign Medical Graduates (ECFMG) are developing tests of clinical skills using standardized patients for live or electronic simulations. Specialty boards are seeking standardized evaluations of residency performance as part of the certification process. Licensure boards propose to develop a final step of their examinations that will test the cognitive processes used by experienced clinicians. As recertification moves toward an evaluation of practice, there is a call for competence evaluations that will also examine practice performance. These may use a battery of techniques such as

Evaluating the clinical skills of foreign medical school graduates participating in an internship preparation program

In the fall of 1986, the Ministry of Health of the province of Ontario, Canada, implemented a special 36-week internship preparation program for graduates of foreign medical schools. The program accepts 24 candidates per year and is offered at the five Ontario medical schools. At the time of this study, two cohorts of participants had completed the program. As part of the evaluation process, each participant was administered a battery of the same Objective Structured Clinical Examination-type test stations, prior to and on completion of the pre-internship experience. Repeating the same examination permitted investigation of the predictive and construct validities of the clinical skills tests, as well as the stability of the validity and high reliability estimates that emerged from the testing of the first cohort. The results provide convincing evidence of the testing approach's predictive and construct validity and reliability.

Foreign Medical Graduates in the United States-Reply

In Reply.— Dr Khera is correct in saying that foreign physicians found it relatively easy to come to the United States a few years ago and that only a small number of foreign medical graduates can find residency opportunities today in our relatively rigidly organized education programs. The FMGEMS was a necessary development to address former inequities. The approximate pass rate fluctuates between 25% and 30%. Our clinical skills test pilot study is part of the broad issue of testing clinical skills in all medical school graduates, an issue that will require further discussion in the coming months. The principal focus of our comments concerned an expanded exchange visitor program that would provide more educational opportunities in the United States for foreign medical graduates, with a clear understanding that the individual must return to his or her home country and contribute to the improvement of health care in that country.

UJIAN OSCE REGULER BERORIENTASI ERGONOMI MENINGKATKAN KINERJA PENGUJI DI FAKULTAS KEDOKTERAN UNIVERSITAS ISLAM AL-AZHAR MATARAM

Regular Objective Structured Clinical Examination (OSCE) was a clinical skill test in the Faculty of Medicine Islamic University Al-Azhar Mataram. Examiner of OSCE in this study was a doctor who was also a lecturer trained in judging student’s clinical skills. The study aimed to know whetherergonomic-oriented regular OSCE by muscle stretching and drinking sweetened tea can improve the performance of examiners in the Faculty of Medicine Islamic University Al-Azhar Mataram. Performance improvement indicators used were decreased complaints of boredom, fatigue and musculoskeletal.Methods used in this study was experimental study with “treatment by subject design”. The study used as many as 18 sample given two treatment which was the first period without the ergonomic intervention involving stretching of the muscle and drinking sweetened tea and the second period was a group with ergonomic intervention. Boredom was measured by a boredom questionnaire.Fatigue, was measured from 30 items of fatigue questionnaire from Japan IFRC(Industrial Fatigue Research Committee). Skeletal muscle complaints was predicted from Nordic Body Map questionnaire. Data were analyzed bynormality test and t-paired test with significancy level of 5%.The results of this study showed that there was a significant difference(p&lt;0,05) in average score boredom, fatigue and musculoskeletal complaints. The mean boredom score in the first period was 39,724 and in the second period was26,415 with reduction in average boredom score was 33.5 %. Fatigue mean scorein the first period was 44,050 and the second period was 23,525 with reductionof fatigue score was 46,59 %. Mean musculoskeletal complaints score in the first period was 39,25 and the second period was 14,02 with reduction in mean musculoskeletal complaints score was 64,28 %. 19It can be concluded from this study that ergonomic intervention by muscle stretching and drinking sweetened tea in the interrupted time of examination process improved the performance of OSCE examiner in terms of boredom, fatigue and musculosceletal complaints in the Faculty of Medicine of Islamic University Al-Azhar Mataram.

May Objective Structure Clinical Examination and Clinical Skill Test Contribute to the Standardization of Clinical Clerkship in Multi-Campus Medical School?

May Objective Structure Clinical Examination and Clinical Skill Test Contribute to the Standardization of Clinical Clerkship in Multi-Campus Medical School?