Objective Checklist Research Articles

A Comparison Between In-Person and Virtual Communication Skills OSCE for Medical Students.

This study investigates the effectiveness of a virtual format of an advanced communication skills observed structured clinical examination (OSCE) for senior medical students in comparison to an in-person format. The study also examines the emotional support students experience in the virtual setting. Our analysis was based on quantitative data collected through objective checklists and post-OSCE survey results. The virtual OSCE was a revision of an earlier in-person formative advanced communication skills OSCE for fourth-year medical students. Student performances were assessed by self and peers using objective checklists-the modified Master Interview Rating Scale (mMIRS) and Communication Behavior Checklist (CBC). The mMIRS measured interview process such as avoiding jargon and demonstrating empathy. The CBC examined interview content which included tasks specific to the content of the case. The OSCE was followed by a faculty-led debrief and quantitative survey. The virtual OSCE was conducted in 2021, and the results of the checklists and survey were compared with those collected from two earlier in-person OSCEs. Eighty-three students participated in the virtual OSCE. There was no difference in mMIRS scores between the virtual and in-person OSCE. Overall CBC scores were lower in the virtual OSCE compared to in-person (p < 0.05). Sixty-seven out of 83 (80.7%) students completed the post-OSCE survey. There were no differences between the virtual and in-person OSCE in terms of educational value, whether the OSCE would change the way participants talk to patients, and preparedness to have serious conversations with patients. All respondents somewhat or strongly agreed with feeling emotionally supported during the virtual OSCE. The virtual format was a suitable alternative to an in-person, formative, advanced communication skills OSCE for medical students. The virtual OSCE was educationally effective and was met with student satisfaction and a sense of emotional support. Future virtual iterations must ensure adequate instruction on interview content.

Automating medical simulations

ObjectiveThis study aims to explore speech as an alternative modality for human activity recognition (HAR) in medical settings. While current HAR technologies rely on video and sensory modalities, they are often unsuitable for the medical environment due to interference from medical personnel, privacy concerns, and environmental limitations. Therefore, we propose an end-to-end, fully automatic objective checklist validation framework that utilizes medical personnel's uttered speech to recognize and document the executed actions in a checklist format. MethodsOur framework records, processes, and analyzes medical personnel's speech to extract valuable information about performed actions. This information is then used to fill the corresponding rubrics in the checklist automatically. ResultsOur approach to activity recognition outperformed the online expert examiner, achieving an F1 score of 0.869 on verbal tasks and an ICC score of 0.822 with an offline examiner. Furthermore, the framework successfully identified communication failures and medical errors made by physicians and nurses. ConclusionImplementing a speech-based framework in medical settings, such as the emergency room and operation room, holds promise for improving care delivery and enabling the development of automated assistive technologies in various medical domains. By leveraging speech as a modality for HAR, we can overcome the limitations of existing technologies and enhance workflow efficiency and patient safety.

A novel simulator-based checklist for evaluating residents' competence in cerebral angiography in China.

Nowadays, with the fast-increasing demand for neuro-endovascular therapy, surgeons in this field are in urgent need. Unfortunately, there is still no formal skill assessment in neuro-endovascular therapy in China. We used a Delphi method to design a newly objective checklist for standards of cerebrovascular angiography in China and evaluated its validity and reliability. A total of 19 neuro-residents with no interventional experience and 19 neuro-endovascular surgeons from two centers (Guangzhou and Tianjin) were recruited; they were divided into two groups: residents and surgeons. Residents completed a simulation-based cerebrovascular angiography operation training before assessment. Assessments were under live and video record forms with two tools: the existing global rating scale (GRS) of endovascular performance and the new checklist. The average scores of residents were significantly increased after training in two centers (p < 0.05). There is good consistency between GRS and the checklist (p = 0.856). Intra-rater reliability (Spearman's rho) of the checklist was >0.9, and the same result was also observed in raters between different centers and different assessment forms (p < 0.001, rho > 0.9). The reliability of the checklist was higher than that of the GRS (Kendall's harmonious coefficient is 0.849, while GRS is 0.684). The newly developed checklist appears reliable and valid for evaluating the technical performance of cerebral angiography and differentiating between trained and untrained trainees' performance well. For its efficiency, our method has been proven to be a feasible tool for resident angiography examination in certification nationwide.

Variability in Assessment of Preclinical Prosthodontics Work of Undergraduate Student

Background and objective. Clinical professions are often concerned not just on the knowledge acquisition, but achievement of skills and their application. Assessment tools employed in preclinical prosthodontic training vary widely, the two most common being used are the glance and grading method and the objective check list criteria, and any assessment procedure employed should be transparent in manner with both the staff and students informed of the purpose and the process adopted. This study was undertaken to examine whether introduction of an objective scoring criteria can improve the validity of scoring in assessment of preclinical prosthodontics (bite block) that done by students. Methods. The study evaluated 240 undergraduate students who were in second year of study at dental school Tripoli University, performing a bite block (trial denture base with wax rim) performed on ideal stone cast. The preclinical work (bite block) were evaluated and allotted marks by four blinded independent examiners using two methods of scoring, glance and grade method and objective checklist scoring method. Results. The data were parametric and met the normal distribution. Therefore, the scores were presented by mean and standard deviation. There was a significant difference (p value less than 0.005) between two scoring methods of preclinical prosthodontics work (glance and grade and objective checklist methods). Conclusion. This study concludes by recommending that preclinical prosthodontics work of students be assessed by objective checklist criteria scoring and it should be introduced after sufficient training and calibration sessions to induce examiner reliability.

Implementing a Novel Nursing Site Manager Role in the Pediatric Emergency Department for Patient and Staff Safety During the COVID-19 Pandemic

Implementing a Novel Nursing Site Manager Role in the Pediatric Emergency Department for Patient and Staff Safety During the COVID-19 Pandemic

Comparison of Two Simulated Procedural Assessment Formats in Attending Emergency Physicians.

BackgroundEmergency physicians must be proficient at inserting central venous catheters and performing lumbar punctures to provide life-saving therapies to critically ill patients. An assessment of procedural skill is rarely performed after an emergency physician has completed residency. Current board certification exams for emergency medicine focus only on verbal descriptions of procedures to assess skill. We compared two methods of procedural skill assessment, simulated task trainer and verbal description, to assess the range of skill in central venous catheter insertion and lumbar punctures of emergency attending physicians at a large, urban, academic tertiary care institution.MethodologyThis is a prospective cohort study of simulated internal jugular central venous catheter insertion and lumbar puncture skill by emergency attending physicians on a task trainer versus verbal description. A total of 17 attending emergency medicine physicians consented to participate in the study during a yearly procedural skills session. For each subject, two expert raters used previously published checklists to assess procedural skill and give a global rating score.ResultsMore checklist items were performed correctly on the task trainer than on verbal assessment for central line (task trainer = 78.4% ± 8.32% and verbal = 68.26% ± 8.9%) and lumbar puncture (task trainer = 85.57% ± 7.6% and verbal = 73.53%4 ± 10.34%) procedures, both with significant differences (p < 0.001). Of the participants, 82% strongly preferred the task trainer format to the verbal description assessment format.ConclusionsThe higher scores on the simulated format compared to the current verbal format imply that a shift towards simulated procedural assessment techniques may benefit examinees. More work is needed to determine if objective checklist scores for practicing attending emergency physicians correlate with subjective expert assessments of their procedural skills.

Evaluating YouTube as a Source of Patient Information for Functional Endoscopic Sinus Surgery.

To evaluate the quality of information presented on YouTube regarding functional endoscopic sinus surgery (FESS) for patients. YouTube was searched using FESS-specific keywords under the setting of "relevance." The first 50 videos from each keyword were reviewed and analyzed by 2 independent physician reviewers. Videos not related to FESS and duplicates were excluded. Outcome measures included the modified DISCERN score (range 0-5), the Journal of the American Medical Association (JAMA) benchmark criteria (range: 0-4), a novel scoring checklist for FESS assessing usefulness (range: 0-16), and the Video Power Index (VPI). Intraclass correlation coefficient (ICC) was calculated. Of the 200 videos identified, 95 videos were analyzed after exclusions. Videos had an average VPI of 40.8 and SD 133.2. Average scores from the 3 objective checklists among all videos were low: modified DISCERN: 1.91, SD: 1.15; JAMA benchmark: 1.91, SD: 0.76; and FESS score: 3.54, SD: 1.77. The ICC between the 2 independent reviewers was excellent for all 3 checklists. We noted significant positive Pearson correlation between all 3 checklist scores (P < .001). In between-group comparisons of mean scores, there was significantly higher DISCERN and JAMA scores for videos from university/professional organizations, as compared to videos from medical advertising/for-profit companies and independent users. There were no significant differences in FESS scores noted between the 3 groups. There were overall low scores across the modified DISCERN, JAMA benchmark criteria, and FESS scoring checklists, reflecting the poor quality of YouTube videos as a source of patient information for FESS.

Relationship Between "Simulated Patient Scenarios and Role-Playing" Method and OSCE Performance in Senior Anesthesiology Residents: A Correlation Assessment Study.

Background“Simulated-patient scenarios and role-playing” and OSCE are among the many non-traditional education methods with variable results in different clinical settings.ObjectivesThis cross-sectional study was performed to assess the correlation between the results of these two methods in senior anesthesiology residents, with a special focus on four of the six ACGME core competencies.MethodsDuring two years, senior anesthesiology residents were subject to “simulated patient scenario and role-playing” sessions. Two faculty members took the role of the patient and one of the relatives. An objective checklist with 15 items was prepared to be rated by other department faculty members. Meanwhile, an ordered pattern of OSCE was prepared to cover four core competencies that were more related to this academic process (from a total of six core competencies). The mean and standard deviation of the score of each of the 15 items in the checklist were calculated. The correlation between cumulative checklist scoring results and OSCE exam results was assessed. A P value of less than 0.05 was considered significant.ResultsA total of 40 senior anesthesiology residents, with 344 assessments by faculty members in 40 sessions, were enrolled in the study. The questionnaire’s Cronbach’s alpha reliability was 0.74. No statistically significant disparity was detected between the results of the two assessment methods, while the results of the two assessments had a significant correlation (two-tailed correlation coefficient = 0.886; P value < 0.001).ConclusionsThere was an objective relationship between the results of “simulated patient scenario and role-playing” strategies and the results of OSCE exams using an observer-based rating method. Thus, they could be used as surrogates in the assessment of core clinical competencies of senior anesthesiology residents.

Study protocol: a randomised non-inferiority trial using interactive virtual presence to remotely assist parents with child restraint installations

BackgroundMotor vehicle crashes are the third-leading cause of death to American children aged 1–5 years. When installed correctly, child restraints (car seats) reduce risk of serious injury and death. However,...

A implantação do Núcleo Interno de Regulação em Hospital Público Terciário

Introdução: O Núcleo Interno de Regulação (NIR) é uma das estratégias de gestão criadas pela Política Nacional de Atenção Hospitalar do Ministério da Saúde em 2013. Visa a organização do acesso à consultas, serviços diagnósticos e terapêuticos e aos leitos de internação, sendo fundamental o conhecimento do perfil epidemiológico e ajuste de seus processos de trabalho de modo a garantir o fluxo no sistema de saúde. Como as normas para implantação só foram realizadas em 2018, vários serviços implantaram o NIR sem diretrizes e há necessidade de se avaliar o grau de implantação.&#x0D; Objetivo: Avaliar o nível de implantação do NIR em duas unidades de um complexo hospitalar público terciário.&#x0D; Metodologia: Estudo de avaliação, exploratório, descritivo e longitudinal, realizado em duas unidades de internação hospitalar público terciário implantadas no primeiro semestre de 2016 (Unidade de Emergência) e no primeiro semestre de 2017 (Unidade Campus).&#x0D; Resultados: Devido à natureza de sua vocação (eletivo ou urgência), observou-se perfil epidemiológico distinto nas unidades, embora tenha sido possível a construção de processo e análise de SWOT integrada para efeitos de tomada de decisão e planejamento. De acordo com os pilares propostos pelo Manual, ambas em maior ou menor grau apresentam iniciativas recomendadas. Mas, por outro lado, os entrevistados reforçam a necessidade de maior investimento para aprimorar o processo.&#x0D; Conclusão: É necessário o estabelecimento de uma política de implantação dos NIR’s com referenciais objetivos que possam ser pontuados e permitam o acompanhamento temporal do seu desenvolvimento.

NURSING STUDENTS' CLINICAL EVALUATION: OBJECTIVE STRUCTURED CLINICAL EVALUATION VERSUS OBSERVATIONAL CHECKLIST

Clinical nursing education is the heart of a nurse's professional practice. Effective and exact clinical evaluation should be the first aim for all nursing faculties and clinical instructors. Objective Structured Clinical Evaluation is one form of objective evaluation method that includes both summative and formative evaluation components. Therefore the application of objective structured clinical evaluation in nursing students' clinical evaluation increase their competencies. Aim of the Study: To compare between objective structured clinical evaluation versus observational checklist in nursing students' clinical evaluation. Methodology: A quasi-experiment design was utilized. A convenience sample of fifty students from third year, Faculty of Nursing, Neelain University, Sudan, Tools; divided into two parts; first part includes Scio-demographic characters and student's clinical evaluation score by both methods and second part included the Student's Opinion regarding advantages of both clinical evaluation methods. Results: Mean of Student's Clinical Evaluation score by Objective Structured Clinical Evaluation was (24.42%), while by Observational Checklist was (20.72). The more than half participant students (66.0%) preferred Objective Structured Clinical Evaluation than traditional observational checklist in clinical evaluation. There was a statistical significance between the Student's Opinion regarding Clinical Evaluation by Objective Structured Clinical Evaluation and Observational checklist in many questions. Conclusion: The findings of the present study concluded that clinical evaluation by Objective Structured Clinical Evaluation more objective method of assessment than the traditional method. Recommendation: Recommendation: Objective Structured Clinical Evaluation should be incorporaed with other methods for evaluating undergraduate nursing students in all studying years.

Improving Forensic Pathologic Investigation of Sudden Death in the Young: Tools, Guidance, and Methods of Cardiovascular Dissection from the Sudden Death in the Young Case Registry.

The Sudden Death in the Young (SDY) Case Registry, a prospective, population-based registry active in ten states, has developed tools to aid pathologists and death investigators in the evaluation and autopsy of unexplained, natural sudden deaths in the pediatric population. The tools were developed by a team of experts representing forensic pathology; pediatric-, cardiac-, and neuropathology; cardiology; neurology/epileptology; pediatrics; genetic counseling; and public health. These tools focus on collecting data relevant to determination of cause of death with a focus on dissection of the cardiovascular system. The tools provide an objective checklist format for ease of use and data extraction. By sharing the tools here and highlighting the examination of the cardiovascular system, the SDY Case Registry encourages a standardized approach to death investigation, autopsy, and data collection for sudden, unexpected deaths in the young towards a goal of informing prevention efforts. Acad Forensic Pathol. 2018 8(2): 347-391.

Development of a Checklist for Assessing Good Hygiene Practices of Fresh-Cut Fruits and Vegetables Using Focus Group Interviews.

This study aimed to develop a checklist for good hygiene practices (GHP) for raw material of vegetable origin using the focus groups (FGs) approach (n = 4). The final checklist for commercialization of horticultural products totaled 28 questions divided into six blocks, namely: water supply; hygiene, health, and training; waste control; control of pests; packaging and traceability; and hygiene of facilities and equipment. The FG methodology was efficient to elaborate a participatory and objective checklist, based on minimum hygiene requirements, serving as a tool for diagnosis, planning, and training in GHP of fresh vegetables, besides contributing to raise awareness of the consumers' food safety. The FG methodology provided useful information to establish the final checklist for GHP, with easy application, according to the previous participants' perception and experience.

Novel Transfer of Care Sign-out Assessment Tool in a Pediatric Emergency Department

ObjectiveTransfer of care sign-outs (TOCS) for admissions from a pediatric emergency department have unique challenges. Standardized and reliable assessment tools for TOCS remain elusive. We describe the development, reliability, and validity of a TOCS assessment tool. MethodsVideo recordings of resident TOCS were assessed to capture 4 domains: completeness, synopsis, foresight, and professionalism. In phase 1, 56 TOCS were used to modify the tool and improve reliability. In phase 2, 91 TOCS were used to examine validity. Analyses included Cronbach's alpha for internal structure, intraclass correlation and Cohen's kappa for interrater reliability, Pearson's correlation for relationships between variables, and 95% confidence interval of the mean for resident group comparisons. ResultsCronbach's alpha was 0.52 for internal structure of the tool's subjective rating scale. Intraclass correlation for the subjective rating scale items ranged from 0.70 to 0.80. Cohen's kappa for most objective checklist items ranged from 0.43 to 1. Content completeness was significantly correlated with synopsis, foresight, and professionalism (Pearson's r ranged from 0.36 to 0.62, P values were <0.001). House staff senior residents scored higher (on average) than interns and rotating senior residents in synopsis and foresight. Also, house staff interns scored higher (on average) than rotating senior residents in professionalism. House staff senior residents scored higher (on average) than rotating senior residents in content completeness. ConclusionsWe provide validity evidence to support using scores from the TOCS tool to assess higher-level transfer of care comprehension and communication by pediatric emergency department residents and to test interventions to improve TOCS.

Association between Parental Addiction and Unintentional Childhood Poisoning

Background Children that livingwith addicted parents are at risk for intentionally and unintentionally poisoning. Present study aimed to evaluate the effect of parental addiction on unintentional childhood poisoning. Materials and Methods: Totally, 140 admitted children with poisoning in Loghman hospital, Tehran-Iran, as referral center were recruited from March 2013 to July 2014. Cases were matched with 280 controls by age (within a caliper of six months), gender, and date of hospital attendance in Tehran, Iran. Parents were interviewed using an objective checklist about the risk factors of childhood poisoning. Conditional logistic regression with within-group varying weights was used to adjust for measured confounders. Vary weights within the matched set was defined by inverse probability weighting (IPW). Results: Sixty-two of cases (74.7%) were poisoned with Methadone. The odds ratio [OR] 95% confidence interval [CI] of having addicted parents in poisoned children compared to the controls in three scenarios of ordinary, un-stabilized weighted and stabilized weighted conditional regression logistic ORs (95% CI) were 17.3 (8.7, 34.6), 2.6 (1.9, 3.3) and 3.6 (2.9, 4.3) respectively. Conclusion The results indicate that child abuse and neglect have been linked to parental substance abuse. Education on preventive interventions such as safe storage of methadone and store poisoning product out of reach and sight of children are necessary in substance abusing families.

Improving ward round skills

Background: Ward rounds represent an important task in clinical settings. Despite their importance, students show substantial deficits in ward round competencies (e.g. physical examination, drug prescription, documentation). Previously reported teaching methods such as ward round training with standardized patients or educational wards, however, are time-consuming and expensive.Aims: To assess the effects of a structured on-ward supervision program for final-year students with specially trained supervising doctors focusing on ward round competencies.Method: A total of 36 medical students were randomly assigned to an intervention group (IG) and a control group (CG). During an eight-week-long training course, the intervention group (IG; n = 18) received structured ward round training while the control group (CG; n = 18) received on-ward training regarding how to take patient histories and perform physical examinations. Ward round competencies were assessed using simulated ward round scenarios with objective checklist ratings and standardized patient ratings.Results: The IG achieved significantly more predefined learning goals compared to the CG. Standardized patient ratings did not differ significantly between groups.Conclusions: Results provide evidence that the presented program is a feasible and efficient tool for teaching ward round skills at the bedside. Further research should focus on cost effectiveness compared to alternative models.

A study of examiner variability in assessment of preclinical class II amalgam preparation

Introduction: Training in typodont preclinical restorative exercises is one of the ways by which the required manual skill dexterity could be developed in students. The assessment of these exercises has been associated with inter and intra examiner variability. This study was undertaken to examine whether introduction of objective scoring criteria can reduce the examiner variability and evaluate for inter and intra examiner variability in assessing class II amalgam tooth preparation by using two different methods i.e., glance and grade and objective checklist criteria scoring. Materials and Methods: The study evaluated 41 undergraduate students performing two Class II disto-occlusal amalgam preparations performed on plastic typodont left lower and upper first molars. The preparations were evaluated by four blinded independent examiners using two methods viz., glance and grade and objective checklist scoring methods. Statistical analysis for inter and intra examiner variability was tested using Friedman test and Wilcoxon signed rank test, respectively. Results: Results of this study show that intra-examiner variability was significantly reduced with objective checklist criteria scoring. The inter-examiner variability was present both in a glance and grading method and objective checklist scoring method. Conclusion: This study concludes by recommending that preclinical operative work of students be assessed by objective checklist criteria scoring and it should be introduced after sufficient training and calibration sessions to reduce examiner variability.

Development of clinical skills in ophthalmology: Significance of objective structured clinical examinations

Introduction: Currently, skill assessment methods for medical students in formative assessment are inconsistent. Our objective was to: (1) Develop an Objective Structured Clinical Examination (OSCE) assessing performance of basic clinical ophthalmic examination (2) remedy deficiencies in knowledge, skill. Materials and Methods: Objective Structured Clinical Examination stations and checklists were developed and validated following approval of institutional ethics committee. Postgraduate student volunteers served as both simulated patients and scorers; however, one station (eye drop instillation) used a mannequin with a faculty member as an observer, scorer. Third MBBS students (n = 61) were oriented. A four station pilot examination was done. After Posting#1 spanning 2 weeks, an eight-station OSCE (7 min. duration each) assessed examination of visual acuity, color vision, ocular motility, pupillary reaction; anterior chamber depth measurement; confrontation, digital tonometry, and eyedrop instillation. Common deficiencies identified were addressed through an interactive demonstration. Six weeks later, following Posting#2, the same test was repeated. Aggregated and paired scores were compared using Student's t-test. Feedback was obtained from students, simulated patients, and faculty. Results: Mean first session score was 22.4 ± 4.66 over 40 (56%); highest being for visual acuity; lowest for eyedrop instillation. Mean score in the second was 30.2 ± 4.3 (75.5%). Paired t-test showed t = 13.73 (P < 0.0001). 77.2% students preferred change in assessment methodology; 100% voting for OSCEs. Conclusions: Feed-back from initial assessment followed by the additional focused teaching session improved students' clinical skills. All were unanimously convinced of the need to change current assessment system. OSCEs are a simple, effective way to both learn and assess clinical examination skills.

EXPERIMENTAL ANALYSIS OF PRECURSORS TO SEVERE PROBLEM BEHAVIOR

Some individuals engage in both mild and severe forms of problem behavior. Research has shown that when mild behaviors precede severe behaviors (i.e., the mild behaviors serve as precursors), they can (a) be maintained by the same source of reinforcement as severe behavior and (b) reduce rates of severe behavior observed during assessment. In Study 1, we developed an objective checklist to identify precursors via videotaped trials for 16 subjects who engaged in problem behavior and identified at least 1 precursor for every subject. In Study 2, we conducted separate functional analyses of precursor and severe problem behaviors for 8 subjects, and obtained correspondence between outcomes in 7 cases. In Study 3, we evaluated noncontingent reinforcement schedule thinning plus differential reinforcement of alternative behavior to reduce precursors, increase appropriate behavior, and maintain low rates of severe behavior during 3 treatment analyses for 2 subjects. Results showed that this treatment strategy was effective for behaviors maintained by positive and negative reinforcement.

Cataract skills: how do we judge competency?

Cataract skills: how do we judge competency?