Multiple-station Examination Research Articles

Predicting communication skills with a paper-and-pencil test.

This study was conducted to investigate the value of a written knowledge test of communication skills for predicting scores on a performance test of communication skills. A paper-and-pencil test of knowledge about communication skills and a performance test of communication skills, consisting of four stations with standardised patients, were administered to students of two classes of the medical schools of Maastricht and Leiden, the Netherlands. The results on these tests were compared. From the results of both instruments, the classes of the participating students could be recognised equally well: 60% correct qualifications of the classes by the knowledge test and 64% by the multiple station examination. Between the two tests an overall, disattenuated correlation of 0.60 was found (N=133, P < 0.01), suggesting moderate predictive value of the knowledge test for the performance test of communication skills. The correlation is stronger for students from Maastricht medical school than for their colleagues in Leiden. Correlation between the knowledge of communication skills test and other available test results of the participating Maastricht students is close to zero, suggesting that the test measures a distinct quality of students' competence. The paper-and-pencil test of knowledge of communication skills has predictive value for the performance of these skills, but this value seems to be less pronounced than similar findings for clinical procedural skills. The stronger relationship between 'knowing how' and 'showing' in the Maastricht student group might be indicative of an effect of the training format.

Teaching and assessing communication skills in Maastricht: the first twenty years

In order to share the issues involved in setting up a communication skills training programme in a medical school, the development of such a programme at the Skillslab of Maastricht Medical School, the Netherlands, is described and the problems encountered are addressed. A multidisciplinary working group developed teaching goals for communication skills, focused on observable behaviour to be displayed by students. These teaching goals were incorporated in a generic model for doctor-patient communication. A longitudinal training programme was created, throughout the four years of the preclinical curriculum. Students meet in small groups of 10, once every 2 weeks. In between group sessions they practise consultation skills in simulated patient contacts. Communication skills are assessed in the annual multiple station examination. In the development of this programme the following consecutive actions were taken: teaching material was produced, and an assessment tool was developed, as were instruments for programme evaluation. The programme evaluation allowed student feedback to teachers, the teachers' departments, and the administration of the medical school. Finally, teacher training was professionalized.

Assessment in general practice: the predictive value of written-knowledge tests and a multiple-station examination for actual medical performance in daily practice.

This study compares the predictive values of written-knowledge tests and a standardized multiple-station examination for the actual medical performance of general practitioners (GPs) in order to select effective assessment methods to be used in quality-improvement activities. A comprehensive assessment was performed in four phases. First, 100 GPs from the southern part of the Netherlands were assessed by a general medical knowledge test and by a knowledge test on technical skills. Second, in order to check for time-order effects, participants were randomly divided into two groups of 50 each, comparable on scores of both knowledge tests and on professional characteristics. Finally, both groups went through a multiple station examination using standardized patients and a practice video assessment of real surgery, but in opposite orders. Consultations were videotaped and assessed by well-trained peer observers. The drop-out rate was 10%. In both groups the predictive value of medical knowledge tests, ranging from 0.43 to 0.56 (Pearson correlation disattenuated), proved to be comparable with the predictive value of the multiple-station examination for actual performance (0.33-0.59). The overall explained variance of scores of the practice video assessment, measured by multiple regression analysis with performance scores as dependent variables and scores on the knowledge tests and the multiple-station examination as independent variables was moderate (19%). A time-order effect showed in only one direction: from practice video assessment to the multiple-station examination. The GP's professional characteristics did not contribute to the explanation of variation in performance. Medical knowledge tests can predict actual clinical performance to the same extent as a multiple-station examination. Compared with a station examination, a knowledge test may be a good alternative method for assessment the procedures of a large number of practising GPs.

Assessment of practicing family physicians

PURPOSE: Looking for a valid, reliable, and feasible method to collect data on the performances of practicing family physicians, the authors compare the measurement characteristics of a multiple-station examination (MSE) using standardized patients with those of a video assessment of regular consultations in daily practice (practice video assessment, PVA). METHOD: In a cross-sectional study, consultations of 90 family physicians were videotaped both in an MSE and in their daily practices. Peer-observers used a validated instrument (MAAS-Global) to assess the physicians' communication with patients and their medical performances. The physicians were randomly divided into two groups, comparable for demographic characteristics, and half underwent the assessments in reverse order to test for time-order effects. Content validity, criterion validity, reliability, and feasibility of the two methods were compared. RESULTS: Content validity of the PVA was superior to that of the MSE, since the domain of general family practice care was better covered. Observed participants judged the videotaped practice consultations to be “natural,” whereas hardly any family physician, after reviewing the videotaped consultations of the MSE, recognized his or her usual working style. Specific criteria made it possible to standardize real practice. Concerning criterion validity, only the medical-performance components of the two methods correlated. No correlation was found for the communication components. Real-practice performance proved to be less influenced by observation than was performance during the MSE. The reliabilities of the two methods, expected to be better in the controlled MSE, were comparable. The administration of the PVA was more flexible, less costly, and better accepted by the family physicians than was that of the MSE. CONCLUSION: Assessment for quality improvement of family physicians' practices by video observation in daily practice is superior to video assessment in a simulated setting using standardized patients.