Inter-rater Reliability Kappa Research Articles

Continuous quality evaluation of the Asanté rapid test for recent infection for robust kit lot quality verification.

The Sedia Biosciences Asanté rapid test for recent infection (RTRI) can identify HIV infections and characterize HIV-1 as recent or long-term infection via the positive verification (V) line and long-term line (LT) line, respectively. Tracking with Recency Assays to Control the Epidemic (TRACE) program uses RTRI assays. Successful implementation of TRACE requires high-quality test performance. The goal of this study is to evaluate the additional quality practices established for new kit lots prior to field use. Asanté lot quality control data from the manufacturer is reviewed by the Centers for Disease Control and Prevention International Laboratory Branch (CDC-ILB) in the Division of Global HIV and TB using. If a lot passes manufacturer quality control and CDC-ILB review, test kits are sent to CDC-ILB for further evaluation. Evaluation by CDC includes inter-rater reliability and linear regressions comparing the V and LT lines against reference data as well as V and LT line data between testers. A Bland-Altman analysis was conducted to assess bias and systematic error. Overall, CDC-ILB passed 29 (91%) out of 32 Sedia Biosciences Asanté kit lots that initially passed manufacturing quality control from July 2017 to May 2020. Regression analyses demonstrate that test kits are performing as expected with consistent R2≥0.92 for both V and LT lines. On average, inter-rater reliability kappa was 0.9, indicating a strong level of agreement. Bland-Altman analyses demonstrate high agreement with little to no systematic error and bias. Ongoing evaluation of new RTRI kit lots is important to ensure high quality test performance. Rejecting 9% of kit lots highlight the importance of continuing to work with manufacturers to ensure consistent kit production and quality assurance (QA) activities. Investing in effective QA measures, conducting both pre- and post-market performance data reviews, could help improve RTRI accuracy and outcomes in similar testing programs.

A Study of Hippocampal T2 Relaxometry Value in Temporal Lobe Epilepsy Patient

In the advancement of the current imaging technique for detection of mesial temporal sclerosis in temporal lobe epilepsy patient, there are multiple Magnetic resonance imaging (MRI) quantitative techniques. One of the long known to be valuable quantitative technique is T2 relaxometry where it has overall better accuracy for detection of mesial temporal sclerosis compared with the qualitative assessment only. A comparative cross-sectional study was conducted in Hospital Universiti Sains Malaysia from August 2021 to January 2023 which include 51 participants, 19 (36.5%) were temporal lobe epilepsy patient and 33 (63.5%) were control. The hippocampal T2 relaxometry was calculated for temporal lobe epilepsy patient and control. The mean T2 relaxometry reading for these groups were compared using Independent T-test. The correlation of T2 relaxometry value with age, duration of epilepsy and seizure frequency were analysed using Pearson correlation. Cohen Kappa inter-rater reliability was done for MR qualitative assessment. The participant age range was from 15 to 65 years old. Total of 23 samples were male (44.2%). The mean T2 relaxometry for epilepsy patient was higher than control. For correlation, moderate to good positive correlation was seen between the duration of epilepsy with T2 relaxometry of right hippocampus. Whereas poor positive correlation between the right hippocampus T2 relaxometry with age (P>0.01), as well poor positive correlation between both hippocampus T2 relaxometry with seizure frequency (P>0.01). There was one patient (5.2%) reported to be negative for mesial temporal sclerosis by qualitative assessment, but the T2 relaxometry reading however was raised. As a conclusion, T2 relaxometry sequence can be use as adjunct to the current qualitative assessment to increase sensitivity and specificity in detecting mesial temporal sclerosis.
 Keywords: Temporal lobe epilepsy, Mesial temporal sclerosis, T2 relaxometry, MRI epilepsy protocol

Effects of Applying Problem Based Learning (PBL) Strategy Use on Students’ Oral Vocabulary as One Aspect of Speaking Performance

This study’s main goal was to investigate how PBL, a learning strategy, might improve students’ oral vocabulary, which is one component of their speaking abilities in an EFL class with grade 11 students as the emphasis. The experimental (treatment) and comparative groups were the two research groups. Biftu Nekemte Secondary School was the site of the study (BNSS). The study’s population comprised all eleventh grade students enrolled in the academic year of 2014–2022. As a type of quasi-experimental research, the current study used a non-randomized pre-test, post-test comparison group design. The researcher used a multi-phase sampling strategy. 48 students from the experimental group and 49 from the comparison group took part in the study. The pre- and posttest instruments were adapted from the speaking performance rubric by Brown (2004). In fact, the researcher simply applied the rubric’s vocabulary part. The study’s quantitative portion was calculated using a number of statistical procedures. Calculations were made using descriptive statistics tools for the pretest and posttest groups to determine if the data was parametric or not. Additionally, separate t-tests were conducted. Cohen’s D (Effect Size) and Cohen’s Kappa (for inter-rater reliability) were also conducted, and they were equally significant. The usage of problem-based learning as a learning approach was found to considerably enhance EFL learners’ oral vocabulary skills, which is one facet of speaking abilities. Therefore, it is advised that various stakeholders take into account using PBL as a learning technique in EFL classes, especially for the teaching and acquisition of oral vocabulary.

Exploring the Role of Guidelines in Contributing to Medication Errors: A Descriptive Analysis of National Patient Safety Incident Data.

Clinical guidelines can contribute to medication errors but there is no overall understanding of how and where these occur. We aimed to identify guideline-related medication errors reported via a national incident reporting system, and describe types of error, stages of medication use, guidelines, drugs, specialties and clinical locations most commonly associated with such errors. Retrospective analysis of reports to the National Reporting and Learning System for England and Wales. A hierarchical task analysis (HTA) was developed, describing expected practice when using guidelines. A free-text search was conducted of medication incident reports (2016-2021) using search terms related to common guidelines. All identified reports linked to moderate-severe harm or death, and a random sample of 5100 no/low-harm reports were coded to describe deviations from the HTA. A random sample of 500 cases were independently double-coded. In total, 28,217 reports were identified, with 608 relating to moderate-severe harm or death. Fleiss' kappa for interrater reliability was 0.46. Of the 5708 reports coded, 642 described an HTA step discrepancy (including four linked to a death), suggesting over 3200 discrepancies in the entire dataset of 28,217 reports. Discrepancies related to finding guidelines (n=300 reports), finding information within guidelines (n=166) and using information (n=176). Discrepancies were most frequently identified for guidelines produced by a local organisation (n=405), and most occurred during prescribing (n=277) or medication administration (n=241). Difficulties finding and using information from clinical guidelines contribute to thousands of prescribing and medication administration incidents, some of which are associated with substantial patient harm.

Comparison of vision screening results between trained teachers and general practitioners among elementary school students in rural Bali: A cross-sectional study

Context: Visual impairment is a major healthcare burden worldwide. The vision screening program was supposed to be conducted routinely; however, the shortage of healthcare personnel posed a significant challenge. Aims: This study was conducted to compare the visual acuity examination results of elementary school students when examined by trained teachers as lay screeners and general practitioners (GPs). Settings and Design: This was a cross-sectional study involving 2056 students from 19 elementary schools under one primary healthcare center in Gianyar. Materials and Methods: All students included in the study underwent a basic visual acuity test using the Snellen’s Chart at a distance of 6 m. The test was performed two times, once by teachers and once by GPs. Statistical Analysis Used: Data analysis was performed using Cohen’s Kappa interrater reliability. Results: The total number of study participants was 2056 students. The mean age of the screened students was 9.43 ± 1.69 years, and most of them were male (52.4%). The estimated number of children needing further eye examination and referred to the ophthalmologist was 253 (12.31%). The agreement between visual acuity examinations conducted by teachers and GP was found to be almost perfect, as indicated by Kappa values of 0.859 for the right eye, 0.856 for the left eye, and 0.878 for both eyes, and these were statistically significant (P < 0.001). Conclusions: The high agreement value of visual acuity examination between teachers and GP shows the feasibility of involving teachers as lay vision screeners, which can elevate the number of children screened.

Abstract 14599: A Pilot Rheumatic Heart Disease Screening Program Using Portable, Handheld Echocardiography in Rivas, Nicaragua

Introduction: Portable, handheld echocardiography offers several advantages for rheumatic heart disease (RHD) screening in low-resource settings. Analyses of focused protocols using handheld echocardiography have yielded acceptable sensitivity and specificity for diagnosis of RHD, including in those performed by non-expert sonographers. Previous studies estimate high burden of RHD in Nicaragua. We conducted a pilot community-based RHD screening program using handheld echocardiography in Rivas, Nicaragua using contemporary RHD diagnostic criteria. Methods: Non-expert sonographers were trained using the GE Vscan Extend TM device and a simplified version of the 2012 World Heart Federation criteria for diagnosis of RHD. Referral criteria for complete echocardiogram included the presence of a mitral regurgitant jet of > 2.0 cm, aortic regurgitant jet > 1.0 cm, or any morphologic abnormality of the mitral or aortic valve, as well as left ventricular dysfunction or significant congenital heart disease. Participants were recruited by convenience-based sampling in communities throughout Rivas, Nicaragua. Participants were notified at the point-of-care of need for complete echocardiogram. All studies were overread asynchronously by expert readers for diagnostic accuracy. Results: Between March 2022 and April 2023, 494 screening echocardiographic studies were performed and evaluated for diagnostic accuracy. The median age was 36.3 years (IQR 19.2, 56.2), 22% of participants were <18 years of age, and 60% of participants were female. There were 17.6% of participants referred for complete echocardiogram at the point-of-care compared with 13.0% meeting referral criteria on expert review. Negative and positive predictive values for non-expert vs. expert assessment were 92.4% and 62.1%, respectively. Cohen’s kappa for inter-rater reliability was 0.338, suggesting fair agreement. Conclusions: This pilot program demonstrated the feasibility of a screening program for RHD in rural Nicaragua with non-expert sonographers. Lessons from this program are intended to inform future epidemiological studies to determine the population prevalence and longitudinal outcomes of RHD in this region.

The accuracy of asynchronous tele-screening for detecting dental caries in patient-captured mobile photos: A pilot study

ObjectivesUnaided visual inspection is a primary dental screening technique. Given the uneven distribution of dental services and prolonged waiting periods, an alternative screening approach is required to increase access to dental care. The purpose of this study was to evaluate the accuracy and reliability of tele-screening for detecting dental caries using mobile intra-oral photos taken by participants. MethodsDental care seekers attending Umm Al-Qura University Teaching Dental Hospital in 2022 were invited to participate in this study. The participants were initially examined by dental interns at the hospital under the supervision of faculty dentists (reference standard) before intra-oral photos were acquired by a trained sixth-year dental student using a Samsung S10 camera. Following an introduction to the photography guide, the same participants then took intra-oral photos of their teeth at home using their mobile devices, which were all uploaded to WhatsApp for later review. Two trained dental reviewers (sixth-year dental students) independently reviewed the intra-oral photos. Sensitivity, specificity, and Kappa scores were estimated to assess the performance of the tele-screening approach relative to the reference unaided dental examination. ResultsTwenty-three participants, with a mean age of 30 ± 12 years, were enrolled. The mean decayed, missing, and filled teeth (DMFT) was 13.43 ± 5.48. Patient-delivered tele-screening demonstrated a sensitivity, specificity, and inter-rater reliability kappa of 94 %, 90 %, and 0.81, respectively, when compared to unaided dental examination. Dentist-delivered tele-screening approach demonstrated a sensitivity of 88–89 %, specificity of 88–91 %, and kappa score of 0.75–0.79 relative to unaided dental examination. ConclusionsThis study demonstrated that the tele-screening approach based on reviewing intra-oral photos taken by participants can be a valid and reliable alternative to unaided dental examination. This is important for ensuring sustainable access to dental care.

JudgED: Comparison between Kickboxing Referee Performance at a Novel Serious Game for Judging Improvement and at World Championships

The particular responsibility of referees in combat sports lies in their decision-making to enforce the rules of the sport, which requires considerable experience and a multitude of skills, including perception, categorization, memory processing, and information integration. As a cost-effective alternative to in-tournament training, this research aims to evaluate the novel video-based serious game called “JudgED” to train martial arts referees’ decision-making processes through immediate feedback. The effectiveness of the JudgED game was assessed by (a) measuring decision accuracy and specific reaction time, (b) calculating a theoretical probability of correct scoring, and (c) comparing these results with real competition judging agreement data. A field study was conducted to analyze the performance of 16 kickboxing referees. The study involved two video-based tests in the serious game. The performance data for JudgED were obtained via a procedure that compares the players’ inputs in the serious game with expert-defined decisions. The results were compared to real-competition data gathered through qualitative analysis of kickboxing fights (n = 400 fights/1200 bouts) at the WAKO World Championships 2021. The findings showed an average decision accuracy of 43.011% and an average reaction time of 1.022 s. For further comparison, binominal distribution for the probability of correct final decisions (between 15.3% and 67.2%) in JudgED and Fleiss’ Kappa interrater reliability for JudgED (Ring: κ = 0.371; Tatami: κ = 0.398; p < 0.001) and tournament decisions (by bout: κ = 0.114; by fight κ = 0.063; by outcome κ = 0.166; p < 0.001) were calculated. The results suggest that more training is required to improve referee decision accuracy, and JudgED bears the potential to work as a suitable supporting system.

A Pilot Study of the Striatal Dopamine Transporter Levels in Kratom-Dependent and Normal Subjects Using 99mTc-TRODAT-1 Single Photon Emission Computed Tomography-Computed Tomography (SPECT-CT).

The study aims to elucidate the effects of kratom addiction on dopamine transporter (DAT) using[2-[[2-[[[3-(4-chlorophenyl)-8-methyl-8-azabicyclo[3.2.1]oct-2-yl]methyl](2-mercaptoethyl)amino]ethyl]amino]ethanethiolato(3-)-N2,N20,S2,S20]oxo-[1R-(exo-exo)]-[99mTc] technetium (99mTc-TRODAT-1) brain single photon emission computed tomography-computed tomography (SPECT-CT) in kratom-dependent and healthy subjects. We recruited 12 kratom-dependent subjects and 13 healthy men to participate in this study. Addiction, craving, depression, and cognitive scores were assessed. All subjects received a single bolus injection of 99mTc-TRODAT-1 with 914.1 MBq ± 65.5 of activity (mean ± SD). The brain SPECT-CT images were reconstructed using 3D ordered subset expectation maximization (3D-OSEM) along with attenuation correction (AC), scatter correction (SC), and resolution recovery (RR) with an iteration number of four and a subset of 10. The Cohen's Kappa interrater-reliability between two raters, the standardized uptake value of body weight (SUVBW), and the asymmetrical index percentage (AI%) were evaluated. Kappa statistics showed a fine agreement of abnormal 99mTc-TRODAT-1 uptake in the striatum region for the kratom-dependent group with the κ value of 0.69 (p = 0.0001), and the percentage of agreement for rater 1 and rater 2 was 56% and 64%, respectively. There was a reduction in average SUV in kratom-dependent subjects compared to healthy control subjects in the left caudate and left striatum (0.938 vs. 1.251, p = 0.014, and 1.055 vs. 1.29, p = 0.036, respectively). There was a significant difference in the AI% of the caudate region between the kratom-dependent group and the normal group (33% vs. 14%, p = 0.019). Our findings signify that kratom addiction, may cause a change in DAT level and the results can be confirmed using 99mTc-TRODAT-1 SPECT-CT.

Development and Validation of the Osteoporosis Knowledge, Attitude, and Behaviors Questionnaire for Female Osteoporosis Patients in Taiwan: A Mediation Model.

This research develops the Osteoporosis Knowledge, Attitude and Behaviors Questionnaire (OKABQ) with the intent to evaluate the levels of osteoporosis knowledge, attitude and behavior change by developing and validating the OKABQ while establishing a mediation model of the research. A quantitative, descriptive and instrumental study was conducted in two phases: Phase I-development of the scale through Delphi Method by osteoporosis experts; and Phase II-evaluation of the validity and reliability of the scale and construction of a mediation model using SmartPLS. In Phase I, the content validity index (CVI) of the questionnaire was higher than 0.96 and the inter-rater reliability (IRR) kappa was 1.00. In Phase II, exploratory factor analysis showed that two predominating factors of attitude as a mediation were addressed by the 26-item OKABQ. The indirect effect results from the estimated model indicate that attitude mediates the relationship between knowledge and behaviors (β = 0.114, t = 2.627, p < 0.001), which is positive and statistically significant. We concluded that the OKABQ is a valid measure of osteoporosis knowledge, attitudes and behaviors for women with osteoporosis. These assessment results could assist healthcare providers in mitigating insufficiency in health education and help patients better adapt to a more active bone-healthy lifestyle.

Fracture Resistance Evaluation and Failure Modes Rating Agreement for Two Endocrown Designs: An In Vitro Study

This in vitro study aimed to evaluate the fracture resistance and mode of failure of endocrowns with proximal extension design compared to the conventional design, and to assess the agreement of evaluators on the failure mode. Twenty mandibular third molars free of caries with approximately similar root lengths and crown dimensions were sectioned parallel to the occlusal plane 2 mm above the cementoenamel junction (CEJ). Then, pulp chambers and canals were accessed, cleaned, and smoothed for the path of insertion. To achieve a symmetrical pulp chamber with an average depth of 3 mm, chambers were filled with flowable resin composite. Then, teeth were randomly divided into two groups (n = 10). The control group has all the walls at the same level. The test group had a box extension on the proximal wall 2 mm apical to the buccal and lingual wall level. Endocrowns (n = 20) of two different designs (conventional and proximal extensions) were fabricated using lithium disilicate (IPS e.max CAD, Ivoclar Vivadent, Schaan, Liechtenstein). After cementation, specimens were loaded with a round-end vertical loading tip at a rate of 0.5 mm per minute until fracture (INSTRON, Norwood, MA, USA). Then, the fractured specimens were collected for evaluation and assessment. Statistical analyses were performed using the Mann-Whitney U-test (α = 0.05) for fracture test analysis and Cohen’s Kappa for inter-rater reliability. The Mann-Whitney U-test showed a non-significant difference between the two groups (p = 0.142). However, the mode of failure of the endocrown with proximal extension showed more catastrophic failures than the conventional design. Adding proximal boxes to the endocrown design did not negatively impact the fracture resistance of the restoration. Visualization of the fracture line and tracing their apical end by direct vision or other diagnostic tools is an essential part of the evaluation of failures of endocrowns. Endodontically treated molar teeth with proximal caries extension might be restored with an endocrown with proximal extension. Clinicians should take extra care when visualizing the fracture lines and tracing their apical end as it might be very misleading.

The Impact of Dental Care Programs on Individuals and Their Families: A Scoping Review.

Despite significant global improvements in oral health, inequities persist. Targeted dental care programs are perceived as a viable approach to both improving oral health and to address inequities. However, the impacts of dental care programs on individual and family oral health outcomes remain unclear. The purpose of this scoping review is to map the evidence on impacts of existing dental programs, specifically on individual and family level outcomes. We systematically searched four scientific databases, MEDLINE, EMBASE, CINAHL, and Sociological Abstracts for studies published in the English language between December 1999 and November 2021. Search terms were kept broad to capture a range of programs. Four reviewers (AG, VD, AE, and KKP) independently screened the abstracts and reviewed full-text articles and extracted the data. Cohen's kappa inter-rater reliability score was 0.875, indicating excellent agreement between the reviewers. Data were summarized according to the PRISMA statement. The search yielded 65,887 studies, of which 76 were included in the data synthesis. All but one study assessed various individual-level outcomes (n = 75) and only five investigated family outcomes. The most common program interventions are diagnostic and preventive (n = 35, 46%) care, targeted children (n = 42, 55%), and delivered in school-based settings (n = 28, 37%). The majority of studies (n = 43, 57%) reported a significant improvement in one or more of their reported outcomes; the most assessed outcome was change in dental decay (n = 35). Dental care programs demonstrated effectiveness in addressing individual oral health outcomes. However, evidence to show the impact on family-related outcomes remains limited and requires attention in future research.

Reliability of fast-spin echo T2-weighted three-dimensional sequences to predict endoscopic third ventriculocisternostomy patency in children.

Clinical and radiological assessment of endoscopic third ventriculocisternostomy (ETV) patency can be challenging in children. The objective of our study was thus to test the accuracy and interrater reliability of 3D fast-spin echo (FSE) T2-weighted sequences to assess the patency of ETV. We included all the consecutive children who underwent surgery for ETV over a two-year period and selected the children who presented ETV dysfunction and matched them with children without dysfunction. We evaluated the Kappa interrater reliability of three experienced physicians for prediction of ETV patency using solely the flow void sign in 3D FSE T2-weighted sequences. Nineteen children underwent surgery for ETV dysfunction and 12 children without dysfunction were matched. Sensitivity was 0.79, 0.89 and 0.84 and specificity was 1 for all raters. None of the patent ETV was wrongly considered to be dysfunctional. Fleiss' kappa was 0.871 (p < 0.001). The interrater reliability was excellent with respect to the patency or not of the ETV. FSE T2-weighted sequence is a simple and reproducible tool that can be widely used in daily practice to assess the patency of ETV. Interrater reliability of this sequence is high and accessibility in outpatient setting is acceptable.

Multidisciplinary perceptions on robotic surgical training: the robot is a stimulus for surgical education change.

It is unclear how to best establish successful robotic training programs or if subspecialty robotic program principles can be adapted for general surgery practice. The objective of this study is to understand the perspectives of high-volume robotic surgical educators on best practices in robotic surgery training and to provide recommendations transferable across surgical disciplines. This multi-institutional qualitative analysis involved semi-structured interviews with high-volume robotic educators from academic general surgery (AGS), community general surgery (CGS), urology (URO), and gynecology (GYN). Purposeful sampling and snowballing ensured high-volume status and geographically balanced representation across four strata. Interviews were transcribed, deidentified, and independently, inductively coded. A codebook was developed and refined using constant comparative method until interrater reliability kappa reached 0.95. A qualitative thematic, framework analysis was completed. Thirty-four interviews were completed: AGS (n = 9), CGS (n = 8), URO (n = 9), and GYN (n = 8) resulting in 40 codes and four themes. Theme 1: intangibles of culture, resident engagement, and faculty and administrative buy-in are as important as tangibles of robot and simulator access, online modules, and case volumes. Theme 2: robotic OR integration stresses the trainee-autonomy versus patient-safety balance. Theme 3: trainees acquire robotic skills along individual learning curves; benchmark assessments track progress. Theme 4: AGS can learn from URO and GYN through multidisciplinary collaboration but must balance pre-existing training program use with context-specific curricular needs. Robotic surgical experts emphasize the importance of universal training paradigms, such as a strong educational culture that balances autonomy and patient safety, collaboration between disciplines, and routine assessments for continuous growth. Often, introduction and acceptance of the robot serves as a stimulus to discuss broader surgical education change.

Evidence on the social, economic, and environmental impact of interventions that facilitate bamboo industry development for sustainable livelihoods: a systematic map protocol

BackgroundBamboo has been identified as a potential instrument for socioeconomic development due to its fast growth, perceived environmental benefits, promising material properties, myriad applications, and relative underdevelopment as a global industrial product. Many projects and interventions have been carried out that aim to utilize bamboo’s social and environmental potential in development. However, critical evaluations that demonstrate this effect using real-world evidence and outcomes are rare, and existing case studies have not been collated and analyzed in a systematic way. The proposed systematic map aims to summarize and evaluate evidence on the social, economic, and environmental impact of bamboo industry development initiatives on beneficiary communities and ecosystems, and to identify priority areas for future funding and research.MethodsIn the proposed systematic map, we will collect and thematically categorize evidence on the social, environmental, and economic impact of bamboo development implementations, identifying themes, research gaps, and critical success factors. Literature discussing this type of intervention is published by researchers, organizations, and governments in academic journals, institutional reports, and program evaluations describing various socio-economic and environmental outcomes, impacts and metrics for success. Search sources for this systematic map therefore include bibliographic databases, institutional websites, web-based search engines, and expert consultation. Targeted search strings will be used to identify relevant texts in a two-step review process comprising an abstract and a full-text screening process. Sources describing interventions with a primary aim of bamboo industry development for social benefit that concluded between 1990 and 2021 will be sought. Metadata coded from these texts will be reviewed, categorized, and checked by two reviewers. Reviewers will be checked for consistency on batches of 30 articles using the Kappa interrater reliability test with a goal of a Kappa coefficient of 0.9. Metadata will be coded into different categories including outcomes and impacts using NVivo. Results of both quantitative and qualitative data analysis will be summarized in a searchable online database. Themes will be synthesized and explored in a narrative review and using simple logic models demonstrating theories of change for eligible case studies.

Translation, cultural adaptation, and validation of the Integrated Palliative Outcome Scale-renal (IPOS-r) to Czech

BackgroundPatients with advanced kidney disease suffer from burdensome symptoms, which should be assessed by valid and reliable patient-reported outcome measures.This study aimed to provide a translation, cultural adaptation, and validation of the Czech version of the IPOS-r.MethodsThe IPOS-r was translated to Czech and culturally adapted using cognitive interviews. During the validation phase, patients and staff in dialysis centres and outpatient renal clinics completed the IPOS-r. Internal consistency was tested with Cronbach’s alpha, its reliability via intraclass correlation coefficient for total IPOS-r score, and weighted Kappa (for test-retest and interrater reliability of individual items). Convergent validity was tested with Spearman correlation to Kidney Disease Quality of Life Survey-Short Form 1.2 (KDQOL-SF 1.2). We assessed sensitivity to change using a distribution-based approach.ResultsTwo sets of translators independently performed forward and backward translations of the IPOS-r. Ten patients and ten health care professionals participated in cognitive pre-testing. The sample size for validation included 88 patients (mean age 66 ± SD13.8; 58% men) who were treated with haemodialysis (70.5%), home haemodialysis (5.5%), peritoneal dialysis (3%), and conservative management (21%). Cronbach’s alpha was 0.72, and the intraclass correlation was 0.84 for test-retest reliability and 0.73 for interrater reliability. The IPOS-r correlated with KDQOL-SF 1.2 had a rho between 0.4–0.8 for most of the IPOS-r items, showing good convergent validity. The IPOS-r measure is feasible and takes 9 minutes to complete. Patients who reported a change in health status after 1 month demonstrated a total IPOS-r score change of eight points in both positive and negative directions.ConclusionsThe process of translation and cultural adaptation of the IPOS-r was successful, and the Czech IPOS-r measure is a valid and reliable tool. The Czech IPOS-r can be used to assess symptoms in patients with advanced chronic kidney disease.Trial registrationGAUK [82121].

Quality Evaluation of a Checklist for Intubation Preparation in Graduate Medical Education.

BackgroundRapid sequence intubation (RSI) is a multistep process that emergency physicians commonly perform. Unfortunately, there is little published in the graduate medical education literature regarding the use of checklists for RSI education.MethodsWe developed a pre-intubation checklist for RSI preparation and evaluated emergency medicine residents’ use of it. We developed the checklist using a three-round modified Delphi process among a group of emergency medicine faculty physicians within our institution. Over a three-year period, residents were randomized into two groups: a “checklist group” and a “without-checklist group.” Residents were then evaluated for RSI critical step completion in a simulated critically ill patient by two independent study investigators. Inter-rater reliability kappa scores were calculated. Following completion of the scenario, residents in both groups were asked to complete an anonymous survey. Both groups had access to the checklist at the time of the survey. The survey was used to determine if they found the checklist helpful. Odds ratios with p-values, at an alpha of 0.05 for significance, were computed for checklist items comparing the checklist and without-checklist groups. Data analysis was performed using SAS software (SAS, Cary, NC v 9.4). This study was approved by the authors’ Institutional Review Board.ResultsEach assessment was completed by two investigators. Inter-rater reliability was substantial (κ=0.79). Residents having access to the checklist were more likely to verbalize a critical step with a p-value of < 0.0001 and an odds ratio of 2.17 (95% CI: 1.48, 3.19). The checklist group normalized vital signs prior to intubation in 25/28 (89%, 95% CI: 72.81, 96.29) versus only 6/29 (21%, 95% CI: 9.85, 38.39) with a p-value of <0.0001 in the without-checklist group. The checklist group evaluated for difficult laryngoscopy 26/28 (93%, 95% CI: 77.36, 98.02) versus only 21/29 (72%, CI 95% 54.28, 85.30) with p=0.0223 in the without-checklist group. All of the surveyed residents indicated that the checklist would be helpful for future use in the ED.ConclusionThis RSI checklist improved adherence to preparatory steps of RSI. Utilizing a checklist increased evaluation for a difficult airway and normalizing vital signs. Residents found the checklist helpful for ED use.

Validation of a High-Fidelity Fracture Fixation Model for Skill Acquisition in Orthopedic Surgery Residents

Simulation has become a widely accepted part of training and credentialing processes due to its ability to supplement technical skill acquisition outside of the operating room (OR). This project explores implementation of a bench-top simulation of open reduction with internal fixation (ORIF) as a cost-effective method for practicing and evaluating surgical skill. Participants ranging from intern to attending surgeon performed ORIF using a standard fixation set and a bovine or porcine tibia/radius model. Performance was recorded and scored by blinded reviewers based on a modified global rating scale (GRS), objective structured assessment of technical skills (OSATS) procedure-specific checklist, and critical-mistakes (CM) model. We calculated Fleiss' kappa for inter-rater reliability, Cronbach's alpha for internal consistency of scoring systems, and used univariate analysis to determine the ability of this model to discriminate between training levels. We also performed a normalized performance-versus-cost analysis to characterize perceived value of this simulation compared to other modalities. Twenty subjects completed the fracture fixation exercise. Fleiss' kappa for all scoring systems indicated substantial inter-rater agreement (k = 0.81, 0.80, and 0.74 for GRS, OSATS, and CM, respectively). Internal consistency reliability for GRS and OSATS were high with Cronbach's alpha 0.96(95%CI 0.94-0.97) and 0.94(95%CI 0.91-0.96), respectively. Using a Kuskal-Wallis rank sum test, GRS, OSATS, and CM were found effective for measuring differences between resident levels (p < 0.001, p < 0.001, and p = 0.002, respectively). Qualitative valuation of the exercise indicated similar value for education compared to time spent in the OR and surgical skills labs. This benchtop surgical simulation provides quantitative measurement of operative skills progression, increases trainee familiarity with ORIF principles, and permits targeted education by senior surgeons with the goal of training safe graduates. Procedure-specific checklist grading tools reliably differentiated between training levels with high internal validity. Implementing this model may decrease training costs and accelerate skill acquisition.

Conventional MRI Criteria to Differentiate Progressive Disease From Treatment-Induced Effects in High-Grade (WHO Grade 3-4) Gliomas.

Background and ObjectivesPosttreatment radiologic deterioration of an irradiated high-grade (WHO grade 3–4) glioma (HGG) may be the result of true progressive disease or treatment-induced effects (TIE). Differentiation between these entities is of great importance but remains a diagnostic challenge. This study assesses the diagnostic value of conventional MRI characteristics to differentiate progressive disease from TIE in HGGs.MethodsIn this single-center, retrospective, consecutive cohort study, we included adults with a HGG who were treated with (chemo-)radiotherapy and subsequently developed a new or increasing contrast-enhancing lesion on conventional follow-up MRI. TIE and progressive disease were defined radiologically as stable/decreased for ≥6 weeks or Response Assessment in Neuro-Oncology progression and histologically as TIE without viable tumor or progressive disease. Two neuroradiologists assessed 21 preselected MRI characteristics of the progressive lesions. The statistical analysis included logistic regression to develop a full multivariable model, a diagnostic model with model reduction, and a Cohen kappa interrater reliability (IRR) coefficient.ResultsA total of 210 patients (median age 61 years, interquartile range 54–68, 189 male) with 284 lesions were included, of whom 141 (50%) had progressive disease. Median time to progressive disease was 2 (0.7–6.1) and to TIE 0.9 (0.7–3.5) months after radiotherapy. After multivariable modeling and model reduction, the following determinants prevailed: radiation dose (odds ratio [OR] 0.68, 95% CI 0.49–0.93), longer time to progression (TTP; OR 3.56, 95% CI 1.84–6.88), marginal enhancement (OR 2.04, 95% CI 1.09–3.83), soap bubble enhancement (OR 2.63, 95% CI 1.39–4.98), and isointense apparent diffusion coefficient (ADC) signal (OR 2.11, 95% CI 1.05–4.24). ORs >1 indicate higher odds of progressive disease. The Hosmer & Lemeshow test showed good calibration (p = 0.947) and the area under the receiver operating characteristic curve was 0.722 (95% CI 0.66–0.78). In the glioblastoma subgroup, TTP, marginal enhancement, and ADC signal were significant. IRR analysis between neuroradiologists revealed moderate to near perfect agreement for the predictive items but poor agreement for others.DiscussionSeveral characteristics from conventional MRI are significant predictors for the discrimination between progressive disease and TIE. However, IRR was variable. Conventional MRI characteristics from this study should be incorporated into a multimodal diagnostic model with advanced imaging techniques.Classification of EvidenceThis study provides Class II evidence that in patients with irradiated HGGs, radiation dose, longer TTP, marginal enhancement, soap bubble enhancement, and isointense ADC signal distinguish progressive disease from TIE.

Student pharmacist opioid risk consultations: a pre-post educational intervention study.

The primary aim was to assess third year student pharmacists' communication skills about opioid risks and safety before and after an educational intervention. This assessment was utilized to identify gaps that skills training programmes need to address for students and pharmacists. Seventy-one students in 2018 (pre-intervention/baseline) and 133 students in 2019 (post-intervention) were videotaped during consultation with standardized patients receiving opioid medications for low back pain. The consults were quantitatively coded for what topics students discussed, terms used, eye contact and filler words. Coding of video-recording had high inter-rater reliability (kappa = 0.90). A significant increase was seen in the post-intervention phase compared with baseline data in the number of students who mentioned the term opioid and initiated conversations about opioid risks. The majority of student pharmacists discussed common opioid side effects and performed teach-back with patients. In both of the phases, students used more filler words when discussing dependence, addiction or overdose risk when compared with the rest of the consult. At baseline, students in the expressed discomfort and desired additional training and resources for communicating about opioids, and students in the post-intervention phase reported increased confidence. This educational intervention demonstrated improved opioid risk communication skills among student pharmacists. This study warrants national evaluation of student pharmacist preparedness and provision of structured education and training as necessary to help empower student pharmacists as opioid risk and safety educators.