Script Concordance Test Research Articles

Investing in the use of a checklist during differential diagnoses consideration: what\u2019s the trade-off?

BackgroundA key challenge clinicians face when considering differential diagnoses is whether the patient data have been adequately collected. Insufficient data may inadvertently lead to premature closure of the diagnostic process. This study aimed to test the hypothesis that the application of a mnemonic checklist helps to stimulate more patient data collection, thus leading to better diagnostic consideration.MethodsA total of 88 final year medical students were assigned to either an educational intervention group or a control group in a non-equivalent group post-test only design. Participants in the intervention group received a tutorial on the use of a mnemonic checklist aimed to minimize cognitive errors in clinical decision-making. Two weeks later, the participants in both groups were given a script concordance test consisting of 10 cases, with 3 items per case, to assess their clinical decisions when additional data are given in the case scenarios.ResultsThe Mann-Whitney U-test performed on the total scores from both groups showed no statistical significance (U = 792, z = −1.408, p = 0.159). When comparisons were made for the first half and the second half of the SCT, it was found that participants in the intervention group performed significantly better than participants in the control group in the first half of the test, with median scores of 9.15 (IQR 8.00–10.28) vs. 8.18 (IQR 7.16–9.24) respectively, U = 642.5, z = −2.661, p = 0.008. No significant difference was found in the second half of the test, with the median score of 9.58 (IQR 8.90–10.56) vs. 9.81 (IQR 8.83–11.12) for the intervention group and control group respectively (U = 897.5, z = −0.524, p = 0.60).ConclusionChecklist use in differential diagnoses consideration did show some benefit. However, this benefit seems to have been traded off by the time and effort in using it. More research is needed to determine whether this benefit could be translated into clinical practice after repetitive use.

Virtual patients in the acquisition of clinical reasoning skills: does presentation mode matter? A quasi-randomized controlled trial

BackgroundThe objective of this study is to compare two different instructional methods in the curricular use of computerized virtual patients in undergraduate medical education. We aim to investigate whether using many short and focused cases – the key feature principle – is more effective for the learning of clinical reasoning skills than using few long and systematic cases.MethodsWe conducted a quasi-randomized, non-blinded, controlled parallel-group intervention trial in a large medical school in Southwestern Germany. During two seminar sessions, fourth- and fifth-year medical students (n = 56) worked on the differential diagnosis of the acute abdomen. The educational tool – virtual patients – was the same, but the instructional method differed: In one trial arm, students worked on multiple short cases, with the instruction being focused only on important elements (“key feature arm”, n = 30). In the other trial arm, students worked on few long cases, with the instruction being comprehensive and systematic (“systematic arm”, n = 26). The overall training time was the same in both arms. The students’ clinical reasoning capacity was measured by a specifically developed instrument, a script concordance test. Their motivation and the perceived effectiveness of the instruction were assessed using a structured evaluation questionnaire.ResultsUpon completion of the script concordance test with a reference score of 80 points and a standard deviation of 5 for experts, students in the key feature arm attained a mean of 57.4 points (95% confidence interval: 50.9–63.9), and in the systematic arm, 62.7 points (57.2–68.2), with Cohen’s d at 0.337. The difference is statistically non-significant (p = 0.214). In the evaluation survey, students in the key feature arm indicated that they experienced more time pressure and perceived the material as more difficult.ConclusionsIn this study powered for a medium effect, we could not provide empirical evidence for the hypothesis that a key feature-based instruction on multiple short cases is superior to a systematic instruction on few long cases in the curricular implementation of virtual patients. The results of the evaluation survey suggest that learners should be given enough time to work through case examples, and that caution should be taken to prevent cognitive overload.

Reliability and validity of the script concordance test for postgraduate students of general practice

Background: The script concordance test (SCT) is a validated method of examining students’ clinical reasoning. Medical students’ professional skills are assessed during their postgraduate years as they study for a specialist qualification in general practice. However, no specific provision is made for assessing their clinical reasoning during their postgraduate study.Objective: The aim was to demonstrate the reliability and validity of the SCT in general practice and to determine if this tool could be used to assess medical students’ progress in acquiring clinical reasoning.Methods: A 135-question SCT was administered to postgraduate medical students at the beginning of their first year of specialized training in general practice, and then every six months throughout their three-year training, as well as to a reference panel of 20 expert general practitioners. For score calculation, we used the combined scoring method as the calculator made available by the University of Montreal’s School of Medicine in Canada. For the validity, student’ scores were compared with experts, p <.05 was considered statistically significant.Results: Ninety students completed all six assessments. The experts’ mean score (76.7/100) was significantly higher than the students’ score across all assessments (p <.001), with a Cronbach’s alpha value of over 0.65 for all assessments.Conclusion: The SCT was found to be reliable and capable of discriminating between students and experts, demonstrating that this test is a valid tool for assessing clinical reasoning skills in general practice.

Association between candidate total scores and response pattern in script concordance testing of medical students

I ntroduction: The script concordance test (SCT) aims to test clinical decision making and clinical reasoning. This study is a preliminary attempt to understand an alleged test-taking strategy where students avoid extreme response options, potentially threatening the validity of SCT scores. We investigated whether there is a significant association between the propensity to avoid the extreme response options and candidates’ overall SCT scores. Methods: The SCT scores of 660 clinical-year medical students (six cohorts from 2013–2015) were analysed for a possible association with candidates’ response pattern. The proportion of middle range response options was calculated. Propensity to avoid extreme response options is defined as a response pattern with 15% or more of middle-range responses compared to those of the expert reference panel. The distribution for candidates with propensity to avoid the extreme options was further investigated using chi-square statistics for possible association with their overall SCT results. Results: Fifty-five percent of the students from the lowest quartile, compared to 30% from the top quartile, had shown a propensity to avoid the extreme options. The differences were statistically significant ( p < 0.001) and were consistent among all six cohorts included in this study. Conclusions: Students whose SCT scores are in the lowest quartile are more likely to avoid the extreme response options in answering SCT questions. For quality assurance in high stakes summative SCTs, it may be worthwhile to select items with expert reference panel’s modal answers covering the full 5-point response options

Background noise lowers the performance of anaesthesiology residents' clinical reasoning when measured by script concordance: A randomised crossover volunteer study.

Noise, which is omnipresent in operating rooms and ICUs, may have a negative impact not only patients but also on the concentration of and communication between clinical staff. The present study attempted to evaluate the impact of noise on the performance of anaesthesiology residents' clinical reasoning. Changes in clinical reasoning were measured by script concordance tests (SCTs). This was a randomised and crossover study. Single centre at Rouen University Hospital in April 2014. All year 1 to 4 residents enrolled in the anaesthesiology training programme were included. Performance was assessed using a 56-item SCT. Two resident groups were formed, and each was exposed to both quiet and noisy atmospheres during SCT assessment. Group A did the first part of the assessment (28 SCT) in a quiet atmosphere and the second part (28 SCT) in a noisy atmosphere. Group B did the same in reverse order. The primary outcome of this study was residents' performance as measured by SCT, with and without noise (mean of 100 points 95% confidence interval). Forty-two residents were included. Residents' performance, measured by SCT, was weaker in a noisy environment than in a quiet environment [59.0 (56.0 to 62.0) vs 62.8 (60.8 to 64.9), P = 0.04]. This difference lessened as medical training advanced, as this difference in performance in noisy vs quiet environments was not observed in year 3 and 4 residents [62.9 (59.2 to 66.5) vs 64.0 (61.9 to 66.1), P = 0.60], whereas it was higher for year 1 and 2 residents [54.8 (50.6 to 59.1) vs 61.5 (57.9 to 65.1), P = 0.02]. Our study suggests that noise affects clinical reasoning of anaesthesiology residents especially junior residents when measured by SCT. This observation supports the hypothesis that noise should be prevented in operating rooms especially when junior residents are providing care.

Evolutions in clinical reasoning assessment: The Evolving Script Concordance Test

Introduction: Script concordance testing (SCT) is a method of assessment of clinical reasoning. We developed a new type of SCT case design, the evolving SCT (E-SCT), whereby the patient’s clinical story is “evolving” and with thoughtful integration of new information at each stage, decisions related to clinical decision-making become increasingly clear.Objectives: We aimed to: (1) determine whether an E-SCT could differentiate clinical reasoning ability among junior residents (JR), senior residents (SR), and pediatricians, (2) evaluate the reliability of an E-SCT, and (3) obtain qualitative feedback from participants to help inform the potential acceptability of the E-SCT.Methods: A 12-case E-SCT, embedded within a 24-case pediatric SCT (PaedSCT), was administered to 91 pediatric residents (JR: n = 50; SR: n = 41). A total of 21 pediatricians served on the panel of experts (POE). A one-way analysis of variance (ANOVA) was conducted across the levels of experience. Participants’ feedback on the E-SCT was obtained with a post-test survey and analyzed using two methods: percentage preference and thematic analysis.Results: Statistical differences existed across levels of training: F = 19.31 (df = 2); p < 0.001. The POE scored higher than SR (mean difference = 10.34; p < 0.001) and JR (mean difference = 16.00; p < 0.001). SR scored higher than JR (mean difference = 5.66; p < 0.001). Reliability (Cronbach’s α) was 0.83. Participants found the E-SCT engaging, easy to follow and true to the daily clinical decision-making process.Conclusions: The E-SCT demonstrated very good reliability and was effective in distinguishing clinical reasoning ability across three levels of experience. Participants found the E-SCT engaging and representative of real-life clinical reasoning and decision-making processes. We suggest that further refinement and utilization of the evolving style case will enhance SCT as a robust, engaging, and relevant method for the assessment of clinical reasoning.

Twelve tips for the construction of ethical dilemma case-based assessment

Ethical dilemma case-based examination (ethics Script Concordance Test, eSCT) is a written examination that can be delivered to a large group of examinees for the purpose of measuring high-level thinking. As it accommodates for diverse responses from experts, ethics SCT allows partial credits. The framework of ethics SCT includes a vignette with an ethical dilemma and a leading question, which asks the examinee to “agree” or “disagree”, plus the shifts of prior decision by adding new information. In this article, the following tips for constructing this type of examination are provided: use “true” dilemmas, select an appropriate ethical issue, target high-level cognitive tasks, list key components, keep a single central theme, device quality scoring system, be important and plausible, be clear, select quality experts, validate, know the limitation, and be familiar with test materials. The use of eSCT to measure ethical reasoning ability appears to be both viable and desirable.

Mobile learning app: A novel method to teach clinical decision making in prosthodontics.

Prosthodontics involves replacing lost dentofacial structures using artificial substitutes. Due to availability of many materials and techniques, clinician's clinical decision-making regarding appropriate selection of prosthesis requires critical thinking abilities and is demanding. Especially during graduate training years, learners do not receive the exposure to a variety of cases, thus their clinical reasoning skills are not developed optimally. Therefore, using the trend of incorporating technology in education, we developed a mobile learning app for this purpose. The aim of this study was to evaluate learners' perceptions of this app's utility and impact on their clinical decision-making skills. After taking informed consent, interns of the Department of Prosthodontics of VSPM Dental College, Nagpur, India, during the academic year May 2015-May 2016 were sent the link for the app to be installed in their Android smartphones. Their perceptions were recorded on a feedback questionnaire using 5-point Likert scale. The script concordance test (SCT) was used to check for changes in clinical reasoning abilities. Out of 120 students who were sent the link, 102 downloaded the link and 92 completed the feedback questionnaire and appeared for the SCT (response rate: 76%). The overall response to the app was positive for more than two-thirds of interns, who reported a greater confidence in their clinical decision-making around prostheses through this app and 94% of the students felt that this app should be regularly used along with conventional teaching techniques. Mean SCT scores were pretest 41.5 (±1.7) and posttest 63 (±2.4) (P < 0.005). Clinical decision-making in prosthodontics, a mobile learning app, is an effective way to improve clinical reasoning skills for planning prosthodontic rehabilitation. It is well received by students.

Justify Your Answer: The Role of Written Think Aloud in Script Concordance Testing

ABSTRACTConstruct: Clinical reasoning assessment is a growing area of interest in the medical education literature. Script concordance testing (SCT) evaluates clinical reasoning in conditions of uncertainty and has emerged as an innovative tool in the domain of clinical reasoning assessment. SCT quantifies the degree of concordance between a learner and an experienced clinician and attempts to capture the breadth of responses of expert clinicians, acknowledging the significant yet acceptable variation in practice under situations of uncertainty. Background: SCT has been shown to be a valid and reliable clinical reasoning assessment tool. However, as SCT provides only quantitative information, it may not provide a complete assessment of clinical reasoning. Approach: Think aloud (TA) is a qualitative research tool used in clinical reasoning assessment in which learners verbalize their thought process around an assigned task. This study explores the use of TA, in the form of written reflection, in SCT to assess resident clinical reasoning, hypothesizing that the information obtained from the written TA would enrich the quantitative data obtained through SCT. Ninety-one pediatric postgraduate trainees and 21 pediatricians from 4 Canadian training centers completed an online test consisting of 24 SCT cases immediately followed by retrospective written TA. Six of 24 cases were selected to gather TA data. These cases were chosen to allow all phases of clinical decision making (diagnosis, investigation, and treatment) to be represented in the TA data. Inductive thematic analysis was employed when systematically reviewing TA responses. Results: Three main benefits of adding written TA to SCT were identified: (a) uncovering instances of incorrect clinical reasoning despite a correct SCT response, (b) revealing sound clinical reasoning in the context of a suboptimal SCT response, and (c) detecting question misinterpretation. Conclusions: Written TA can optimize SCT by demonstrating when correct examinee responses are based on guessing or uncertainty rather than robust clinical rationale. TA can also enhance SCT by allowing examinees to provide justification for responses that otherwise would have been considered incorrect and by identifying questions that are frequently misinterpreted to avoid including them in future examinations. TA also has significant value in differentiating between acceptable variations in expert clinician responses and deviance associated with faulty rationale or question misinterpretation; this could improve SCT reliability. A written TA protocol appears to be a valuable tool to assess trainees' clinical reasoning and can strengthen the quantitative assessment provided by SCT.

Validation of undergraduate medical student script concordance test (SCT) scores on the clinical assessment of the acute abdomen.

BackgroundHealth professionals often manage medical problems in critical situations under time pressure and on the basis of vague information. In recent years, dual process theory has provided a framework of cognitive processes to assist students in developing clinical reasoning skills critical especially in surgery due to the high workload and the elevated stress levels. However, clinical reasoning skills can be observed only indirectly and the corresponding constructs are difficult to measure in order to assess student performance. The script concordance test has been established in this field. A number of studies suggest that the test delivers a valid assessment of clinical reasoning. However, different scoring methods have been suggested. They reflect different interpretations of the underlying construct. In this work we want to shed light on the theoretical framework of script theory and give an idea of script concordance testing. We constructed a script concordance test in the clinical context of “acute abdomen” and compared previously proposed scores with regard to their validity.MethodsA test comprising 52 items in 18 clinical scenarios was developed, revised along the guidelines and administered to 56 4th and 5th year medical students at the end of a blended-learning seminar. We scored the answers using five different scoring methods (distance (2×), aggregate (2×), single best answer) and compared the scoring keys, the resulting final scores and Cronbach’s α after normalization of the raw scores.ResultsAll scores except the single best answers calculation achieved acceptable reliability scores (>= 0.75), as measured by Cronbach’s α. Students were clearly distinguishable from the experts, whose results were set to a mean of 80 and SD of 5 by the normalization process. With the two aggregate scoring methods, the students’ means values were between 62.5 (AGGPEN) and 63.9 (AGG) equivalent to about three expert SD below the experts’ mean value (Cronbach’s α : 0.76 (AGGPEN) and 0.75 (AGG)). With the two distance scoring methods the students’ mean was between 62.8 (DMODE) and 66.8 (DMEAN) equivalent to about two expert SD below the experts’ mean value (Cronbach’s α: 0.77 (DMODE) and 0.79 (DMEAN)). In this study the single best answer (SBA) scoring key yielded the worst psychometric results (Cronbach’s α: 0.68).ConclusionAssuming the psychometric properties of the script concordance test scores are valid, then clinical reasoning skills can be measured reliably with different scoring keys in the SCT presented here. Psychometrically, the distance methods seem to be superior, wherein inherent statistical properties of the scales might play a significant role. For methodological reasons, the aggregate methods can also be used. Despite the limitations and complexity of the underlying scoring process and the calculation of reliability, we advocate for SCT because it allows a new perspective on the measurement and teaching of cognitive skills.

Introducing recent medical graduates as members of Script Concordance Test expert reference panels: what impact?

This article was migrated. The article was marked as recommended. The Script Concordance Test (SCT) is being increasingly used in professional development in clinical reasoning, with linear progression in performance in SCT's observed with increasing clinical experience. One of the limiting factors for the SCT is potential burnout in expert reference panel (ERP) members, which we have attempted to address by the introduction of recent medical graduates as panel members. We sought to evaluate the effect of introducing recent medical graduates in to our ERP's on pass/fail decisions in the final clinical reasoning examination of the 6-year undergraduate program of the University of Adelaide, Australia. We engaged an ERP comprising 50 faculty members from three collaborating universities and 13 recent medical graduates to answer on line an identical 20 case scenario, 50 question multidisciplinary SCT twice 6 months apart. The questions were used in high stakes end of year assessment of 5th year medical students (n=132). The pass mark set by the experienced, specialist members of the panel was 49.6% and this increased to 50.4% by addition of recent medical graduates to the panel. This difference would have had no effect on fail rates estimated from the data from the cohort of 132 medical student candidates. In the context of assessment of clinical reasoning in medical programs, recent medical graduates are suitable members of SCT ERP's, and their contribution can enrich the panel and might help to minimise risk of burnout of more experienced faculty.

Assessment of Clinical Reasoning in an Environment of Uncertainty: A Script Concordance Test for Neonatal-Perinatal Medicine

While clinical reasoning is a crucial aspect of graduate medical education, there are limited modalities to evaluate development of these skills. The Script Concordance assesses clinical decision making skills in situations of uncertainty. While Script Concordance Tests have been validated for other specialties, no test has been validated for neonatal-perinatal medicine.

Evaluating the scripts and thresholds of general practitioners for diagnosing heart failure in elderly people.

BackgroundMultiple diagnostic algorithms for heart failure exist. However, it is unclear whether these algorithms are incorporated in the ‘scripts’ clinicians use in every day practice. Scripts are networks of organised knowledge that are acquired and accumulated during clinical training and are refined with each clinical encounter. This study was conducted to evaluate the scripts and thresholds that GPs use to diagnose heart failure in patients aged 75 years and older.MethodsThe scripts and thresholds of 130 Belgian GPs in training and 63 experienced trainers were compared using an online questionnaire based on the same principles as the script concordance test. Two major cases with an open question and 19 minor cases with closed questions were presented. For the minor cases, all of the respondents were asked to assign a diagnostic power to individual cues. Based on these powers, a diagnostic threshold was calculated for each respondent for the two major cases.ResultsThe trainers and trainees used the same scripts to diagnose heart failure in the two major cases. Only ~50 % of the participants used natriuretic peptides in their scripts, although they judged it as the most powerful marker to demonstrate or exclude heart failure. The power that respondents gave to several cues differed significantly according to the context in which these cues were presented. In general, the average exclusive power of different cues was lower than the demonstrative power of the cues. There was no difference in diagnostic threshold between the trainers and trainees.ConclusionYoung, inexperienced GPs used the same scripts as older, more experienced GPs. In general, technical investigations were less frequently queried, compared to elements of the medical history and the clinical examination. The clinical context had a strong impact on the diagnostic power that was assigned to different factors.Electronic supplementary materialThe online version of this article (doi:10.1186/s12875-016-0481-4) contains supplementary material, which is available to authorized users.

Respective value of the traditional clinical rotation and high fidelity simulation on the acquisition of clinical reasoning skills in medical students – A Randomized Controlled Trial.

This article was migrated. The article was marked as recommended. OBJECTIVE To assess the respective value of Traditional clinical rotation and High Fidelity Simulation (HFS) on the acquisition of clinical reasoning (CR) skills in Medical Students.METHODSA randomized controlled trial was conducted. Forty medical students were exposed to a full day of HFS, either during their Paediatric Term (PT) (Experimental group = PT+, HFS+) or, after completion of their PT (Control group = PT+, HFS-). CR skills were assessed by a Script Concordance Test (SCT) prior to group allocation and at the completion of PT.RESULTS39 out of 40 students completed both SCT. Scores before (mean/SD 57.4/6.7) and after (mean/SD 61.1/7.0) the PT were significantly improved (mean increase [95% confidence interval (CI)]: 3.6 [2.0-5.2]; p&lt;0.0001; n=39). There was no added improvement from involvement in a full day of HFS in SCT scores between the experimental and the control groups (mean increase [95% CI]: 3.2 [0.5-5.9], n=18 versus 4.0 [1.9-6.1], n=21, respectively; p=0.61).CONCLUSIONExposure to PT significantly improves the medical students' CR acquisition with no further enhancement by exposure to HFS. Thus potential improvements in students' knowledge and skills due to exposure to HFS may not affect CR skills, and thus may not be assessable by SCT.

Bi-cultural, bi-national benchmarking and assessment of clinical reasoning in Obstetrics and Gynaecology

This article was migrated. The article was marked as recommended. Background: The Script Concordance Test (SCT) is being increasingly used in professional development in clinical reasoning (CR) in postgraduate medicine. On-line delivery favours multi-institutional collaboration. Objectives: To establish if: 1) SCT questions developed in the French-speaking University of Montreal were readily adaptable for use in the English-speaking University of Adelaide 2) expert reference panels (ERP) from both institutions could be used interchangeably 3) student cohorts would perform similarly in the same test. Study Design: 82 SCT questions based on 27 clinical cases in Obstetrics and Gynaecology were developed in Montreal and run in a volunteer cohort of year 3 and year 4 medical students (n=154). Local faculty translated all questions, selecting 31 based on 17 clinical cases for use in summative examinations a year 5 student cohort in Adelaide (n=123). Results: Mean (SD) percentage scores using each ERP key were: 74.2 (6.4) versus 73.3 (6.9), p&lt;0.001 for Adelaide students and 72.5 (7.8) versus 70.6 (8.8), p&lt;0.001 for Montreal students. The correlation coefficients were ≥ 0.928 (p&lt;0.001). Conclusions: Student cohorts performed similarly regardless of which ERP key was used. With appropriate editorial control, SCT's can be effectively shared between French and English-speaking institutions located in different hemispheres. Potential advantages include the creation of an international database of assessment items, benchmarking and cost sharing.

Using the Script Concordance Test to Evaluate Clinical Reasoning Skills in Psychiatry.

Although clinical reasoning is a major component of psychiatric training, most evaluating tools do not assess this skill properly. Clinicians mobilize networks of organized knowledge (scripts) to assess ambiguous or uncertain situations. The Script Concordance Test (SCT) was developed to assess clinical reasoning in an uncertainty context. The objective of this study was to test the usefulness of the SCT to assess the reasoning capacities of interns (7th year medical students) during the psychiatry training. The authors designed a SCT for psychiatry teaching, adapted to interns. The test contained 20 vignettes of five questions each. A reference panel of senior psychiatrists underwent the test, and we used their scoring as a reference for the student group. The SCT assessed the competence of students at the beginning and the end of their training in psychiatry. A panel of 10 psychiatrists and 47 interns participated to this study. As expected, the reference panel performed significantly (p<0.001) better (79.4±5.1) than the students on the SCT. Interns improved significantly (p<0.001) their scores between the beginning (58.5±6.2) and the end (65.0±5.3) of their psychiatry rotation. The students improved significantly (p<0.001) their scores between the beginning and the end of the training (6.4±4.8). This is the first study using the SCT in psychiatry. This study shows the feasibility of this procedure and its utility in the field of psychiatry for evaluating medical students in their clinical reasoning competence. It can provide a valid alternative to classical evaluation methods.

Assessing clinical reasoning in optometry using the script concordance test

BackgroundClinical reasoning is central to any health profession but its development among learners is difficult to assess. Over the last few decades, the script concordance test (SCT) has been developed to solve this dilemma and has been used in many health professions; however, no study has been published on the use of the script concordance test in optometry. The purpose of this study was to develop and validate a script concordance test for the field of optometry.MethodsA 101‐question script concordance test (27 short clinical scenarios) was developed and administered online to a convenience sample of 23-second‐year and 19 fourth‐year students of optometry. It was also administered to a reference panel of 12 experienced optometrists to develop the scoring key. An item‐total correlation was calculated for each question. Cronbach's alpha coefficient was used to evaluate the script concordance test reliability and a t‐test compared the two groups.ResultsA final 77‐question script concordance test was created by eliminating questions with low item‐total correlation. Cronbach's alpha for this optimised 77‐question script concordance test was 0.80. A group comparison revealed that the second‐year students' scores (n = 23; mean score = 66.4 ± 7.87 per cent) were statistically lower (t = ‐4.141; p < 0.001) than those of the fourth‐year students (n = 19; mean score = 75.5 ± 5.97 per cent).ConclusionThe online script concordance test developed for this study was found to be both reliable and capable of discriminating between second‐ and fourth‐year optometric students. These results demonstrate that the script concordance test may be considered as a new tool in the optometric educators' assessment arsenal. Further studies will be needed to cover additional levels of professional development.

Residents in tutored practice exchange groups have better medical reasoning as measured by script concordance test: a controlled, nonrandomized study

Study ObjectiveClinical reasoning by anesthesiology residents in emergency situations where optimal management is uncertain could be improved by setting up a tutored practice exchange group. This study attempted to evaluate the impact of a practice exchange group (PEG), tutored by a senior anesthesiologist, on anesthesiology residents in emergency situations. Changes in clinical reasoning were measured by script concordance tests (SCT). DesignWe conducted a controlled, non-randomized study. Setting and ParticipantsParticipants are residents in anesthesiology in Rouen, Caen and Amiens University Hospitals. InterventionsTwo resident groups were made up without randomization. The first group was the control group and consisted of residents from Amiens University Hospital and Caen University Hospital. The second study group (PEG group) consisted of residents from Rouen University Hospital, who followed weekly PEG sessions. Two groups had the same learning objectives except the PEG. MeasurementsIn both the control group and the study group, each resident's clinical reasoning was assessed in the same formal manner by SCT. The primary outcome measurement of this study was to compare SCT results in the study group with PEG training (PEG group) with those without (control group). Main resultsPerformance in the SCT, expressed as degree of concordance with the expert panel (95% CI), was better in the PEG group (64% [62.1%-66%]) than in control group (60% [57.5%-62.8%])) (P= .004). ConclusionOur study strongly suggests that an expert-directed, peer-conducted educational training program may improve the clinical reasoning of anesthesiology residents as measured by SCT.

Script concordance test in medical schools in Brazil: possibilities and limitations.

Routine use of the script concordance test (SCT) is not common in Brazilian universities. This study aimed to analyze application of the SCT in the medical school of a Brazilian university. Quantitative, analytical and descriptive study in the medical school of a Brazilian university. A total of 159/550 students participated. The test comprised ten clinical cases within internal medicine, with five items per case, rated on a five-point Likert scale. The test was scored in accordance with a marking key that had been validated by a reference panel. In the pre-clinical and clinical phases, the mean scores were 51.6% and 63.4% of the maximum possible scores, respectively. Comparison of the means of the responses among all the years showed that there were significant differences in 40% of the items. The panel marked all the possible answers in five items, while in one item, all the panelists marked a single answer. Cronbach's alpha was 0.64. The results indicated that the more senior students performed better. Construction of an SCT with discriminative questions was not easy. The low reliability index may have occurred due to: a) problems with the construction of the questions; b) limitations of the reference panel; and/or c) the scoring key. This instrument is very difficult to construct, apply and correct. These difficulties may make application of an SCT as an assessment method unfeasible in units with limited resources.

Development of a Method to Measure Clinical Reasoning in Pediatric Residents: The Pediatric Script Concordance Test

Introduction: The Script Concordance Test (SCT) is an assessment method of clinical reasoning skills. SCT is designed to assess a candidate’s ability to reason when faced with decisions encountered in the three phases of clinical decision-making: diagnosis, investigation and treatment. Challenges have been raised related to psychometric properties of SCT scores. Data about accept- ability of the SCT method are also needed. Objectives: 1) To examine the validity of a Pediatric Script Concordance Test (PSCT) in discriminating clinical reasoning ability between junior postgraduate year (PGY) 1 - 2 and senior PGY 3 - 4 pediatric residents, and pediatricians, 2) To determine if higher reliability could be achieved by applying specific test design strategies to the PSCT and 3) To explore trainees’/physicians’ acceptability of the PSCT. Methods: A 24-case/137 question PSCT was administered to 91 residents from four Canadian training centers. Each resident’s PSCT was scored based on the aggregate responses of 21 pediatricians (Panel of Experts (POE)). ANOVA was used to compare across the 3 levels of experience. Reliability was calculated using Cronbach’s α coefficient. Participants completed a post-test survey about the acceptability of PSCT. Results: Overall, a statistical difference in performance was noted across all levels of experience, F = 22.84 (df = 2); p α) was 0.85. Participants expressed keen interest and engagement in the PSCT. Conclusions: PSCT is a valid, reliable, feasible and acceptable method to assess the core competency of clinical reasoning. We suggest the PSCT may be effectively integrated into formative residency assessment and with increasing exposure, experience and refinement may soon be ready to pilot within summative assessments in pediatric medical education.