Inter-rater Research Articles

Interobserver and intraobserver reliability of skull base angles measured on magnetic resonance images

BACKGROUND Determination of platybasia and basilar kyphosis are significant parts of routine cranial magnetic resonance (MR) interpretations. These situations may explain a patient’s symptoms or may be associated with other anomalies. AIM To indicate the interobserver and intraobserver reliability of the skull base angles (SBA) (Koenigsberg standard) and modified SBA (mSBA) measurement techniques. METHODS In total, 391 patients who had undergone cranial MR imaging were re-assessed regarding the SBA measurements. The SBA and mSBA techniques were used on MR images. Two reviewers independently measured the same angles twice within a 15-day interval, using different monitors. Intraclass correlation coefficient (ICC) was calculated to reveal the intraobserver and interobserver agreements. RESULTS There was an excellent agreement between reviewers regarding both angle measurements (ICC was 0.998 for SBA and mSBA). Excellent agreement levels were also observed for intraobserver measurements. ICC was 0.998 for SBA and 0.999 for mSBA for reviewer 1. ICC was 0.997 for SBA and 0.999 for mSBA according to the measurement results of reviewer 2. Higher SBA and mSBA values were observed for females compared to males. There was no correlation between SBA and age for SBA. However, a negative and low-level correlation was observed between mSBA values and age for both reviewers. CONCLUSION SBA and mSBA measurements indicated excellent agreement regarding interobserver and intraobserver differences. The study results showed that SBA angles were reliable measurement techniques to be used on MR images.

Accuracy and reliability of large language models in assessing learning outcomes achievement across cognitive domains.

The advent of artificial intelligence (AI), particularly large language models (LLMs) like ChatGPT and Gemini, has significantly impacted the educational landscape, offering unique opportunities for learning and assessment. In the realm of written assessment grading, traditionally viewed as a laborious and subjective process, this study sought to evaluate the accuracy and reliability of these LLMs in evaluating the achievement of learning outcomes across different cognitive domains in a scientific inquiry course on sports physiology. Human graders and three LLMs, GPT-3.5, GPT-4o, and Gemini, were tasked with scoring submitted student assignments according to a set of rubrics aligned with various cognitive domains, namely "Understand," "Analyze," and "Evaluate" from the revised Bloom's taxonomy and "Scientific Inquiry Competency." Our findings revealed that while LLMs demonstrated some level of competency, they do not yet meet the assessment standards of human graders. Specifically, interrater reliability (percentage agreement and correlation analysis) between human graders was superior as compared to between two grading rounds for each LLM, respectively. Furthermore, concordance and correlation between human and LLM graders were mostly moderate to poor in terms of overall scores and across the pre-specified cognitive domains. The results suggest a future where AI could complement human expertise in educational assessment but underscore the importance of adaptive learning by educators and continuous improvement in current AI technologies to fully realize this potential.NEW & NOTEWORTHY The advent of large language models (LLMs) such as ChatGPT and Gemini has offered new learning and assessment opportunities to integrate artificial intelligence (AI) with education. This study evaluated the accuracy of LLMs in assessing an assignment from a course on sports physiology. Concordance and correlation between human graders and LLMs were mostly moderate to poor. The findings suggest AI's potential to complement human expertise in educational assessment alongside the need for adaptive learning by educators.

Investigation of Study Procedures to Estimate Sensitivity and Reliability of a Virtual Physical Assessment Developed for Workplace Concussions: Method-Comparison Feasibility Study

Abstract Background Remote approaches to workplace concussion assessment have demonstrated value to end users. The feasibility of administering physical concussion assessment measures in a remote context has been minimally explored, and there is limited information on important psychometric properties of physical assessment measures used in remote contexts. Objective The objectives of this feasibility study were to determine recruitment capability for a future larger-scale study aimed at determining sensitivity and reliability of the remote assessment, time required to complete study assessments, and acceptability of remote assessment to people with brain injuries and clinicians; document preliminary results of the sensitivity of the remote assessment when compared to the in-person assessment; and estimate the preliminary interrater and intrarater reliability of the remote assessments to inform procedures of a future larger-scale study that is adequately powered to reliably estimate these parameters of interest. Methods People living with acquired brain injury attended 2 assessments (1 in-person and 1 remote) in a randomized order. The measures administered in these assessments included the finger-to-nose test; balance testing; and the Vestibular/Ocular Motor Screening (VOMS) tool, including documentation of change in symptoms and distance for near point convergence, saccades, cervical spine range of motion, and evaluation of effort. Both assessments occurred at the Ottawa Hospital Rehabilitation Center. After the assessments, a clinician different from the person who completed the original assessments then viewed and documented findings independently on the recordings of the remote assessment. The same second clinician viewed the recording again approximately 1 month following the initial observation. Results The rate of recruitment was 61% (20/33) of people approached, with a total of 20 patient-participants included in the feasibility study. A total of 3 clinicians participated as assessors. The length of time required to complete the in-person and remote assessment procedures averaged 9 and 13 minutes, respectively. The majority of clinicians and patient-participants agreed or strongly agreed that they were confident in the findings on both in-person and remote assessments. Feedback obtained revolved around technology (eg, screen size), lighting, and fatigue of participants in the second assessment. Preliminary estimates of sensitivity of the remote assessment ranged from poor (finger-to-nose testing: 0.0) to excellent (near point convergence: 1.0). Preliminary estimates of reliability of the remote assessment ranged from poor (balance testing, saccades, and range of motion: κ=0.38‐0.49) to excellent (VOMS change in symptoms: κ=1.0). Conclusions The results of this feasibility study indicate that our study procedures are feasible and acceptable to participants. Certain measures show promising psychometric properties (reliability and sensitivity); however, wide CIs due to the small sample size limit the ability to draw definitive conclusions. A planned follow-up study will expand on this work and include a sufficiently large sample to estimate these important properties with acceptable precision.

Rapid and motion-robust pediatric brain imaging: T2-weighted turbo-spin-echo PROPELLER acquisition with compressed sensing.

In pediatric magnetic resonance imaging (MRI), reducing the rate of non-diagnostic scans due to artifacts and shortening acquisition time are crucial not only for economic reasons but also to minimize sedation or general anesthesia. Enabling faster and motion-robust MRI of the brain in infants and children using a novel, enhanced compressed sensing (CS) algorithm in combination with a turbo-spin-echo T2-weighted sequence utilizing the PROPELLER-technique (periodically rotated overlapping parallel lines with enhanced reconstruction; T2PROPELLER CS). This prospective study included 31 patients (8.0 ± 4.7years, 15 males) undergoing a clinically indicated MRI examination of the brain on a 3-T scanner. The T2PROPELLER CS sequence was compared to a conventional, CS-accelerated Cartesian turbo-spin-echo T2-weighted sequence (T2Cartesian CS). Apparent contrast-to-noise ratio (aCNR) and signal-to-noise ratio (aSNR) were calculated. Three blinded radiologists independently rated both sequences twice qualitatively on a 5-point Likert-scale from 1-5 (non-diagnostic-excellent) for artifacts, image sharpness, basal ganglia delineation, lesion conspicuity, and overall image quality. Statistical analysis was performed using the Wilcoxon signed-rank test and paired sample t test. Intra- and interrater reliability of qualitative image assessment was evaluated by computing Krippendorff's reliability estimates. The average acquisition time of the T2PROPELLER CS (189 ± 27s) was 31% shorter than that of the T2Cartesian CS sequence (273 ± 21s; P < 0.001). aCNR (7.7 ± 4.6 vs. 6.2 ± 2.8; P = 0.004) and aSNR (24.8 ± 9.7 vs. 18.8 ± 5.5; P < 0.001) were higher for the T2Cartesian CS compared to the T2PROPELLER CS sequence. The T2PROPELLER CS sequence significantly reduced (motion-)artifacts (P < 0.001) and increased image sharpness (P < 0.001), basal ganglia delineation (P<0.001), lesion conspicuity (raters 1 and 2, P < 0.001; rater 3, P = 0.004), and overall image quality (P < 0.001). Metal artifacts were prominent in both sequences, though slightly more pronounced in the T2PROPELLER CS sequence. The T2PROPELLER CS sequence enables faster and motion-robust imaging of the brain in infants and children, reducing the rate of non-diagnostic scans and potentially allowing sedation or general anesthesia to be minimized in the future.

Piloting an Educational Approach to Assess eHealth Literacy and Evidence-Based Medicine in Integrative Health: A Feasibility and Validation Study.

Introduction: Low rates of online health literacy put consumers at risk of misinformation, but this could be mitigated through practitioner engagement. Integrative health (IH) crosses health care disciplines, so it is well-positioned to improve health information sharing. IH practitioners require evidence-based medicine (EBM) and electronic health literacy (eHL) competencies to make such impact. Several EBM assessments exist, but none are IH-specific. The Fresno Test of EBM FEBM is a validated, performance-based assessment used in medical education. We sought to assess feasibility of incorporating eHL and EBM assessments into graduate coursework while adapting and validating the FEBM for an IH audience (FEBM-IH). Methods: A pilot observational design was used to adapt, evaluate, administer, and validate the FEBM-IH. Revalidation of the FEBM-IH began with a discipline-focused adaptation, which was reviewed by an expert panel. The FEBM-IH was then administered to IH students and faculty. Independently scored assessments determined inter-rater reliability, internal consistency, item discrimination, and item difficulty. eHL assessments (eHEALS and General Health Numeracy Test-6) were also embedded in the online course. Results: Outcome completion rates suggest the FEBM-IH and eHL assessment tools are feasible to include in online courses, with 68.9% (102/148) eligible participants joining and 76.5% (78/102) completing all questions in all measures. The FEBM-IH demonstrated excellent assessor agreement (kappa = 0.97, p < 0.001), high internal consistency (α = 0.799), and acceptable item discrimination (0.26-0.68). Median self-perceived eHL scores increased from 30/40 to 33/40 points by course's end, suggesting some increase in eHL. Conclusions: Tools were feasible to integrate; FEBM-IH maintains acceptable validity; and further exploration of the relationship between EBM and eHL is warranted.

A Comparative Analysis of Twelve Research Paradigms Across Six Knowledge Acquisitions and Twelve Sources of Knowledge: A Philosophical Discourse

This study investigates whether a single research paradigm can dominate knowledge acquisition across empirical and interpretive domains or if a multi-paradigm approach is essential for achieving a holistic understanding. Twelve paradigms, including Positivism, Pragmatism, Hermeneutics, and Feminism, were evaluated across six domains: Science, Philosophy, History, Art, Religion, and Culture. The methodology employed comparative analysis, factor analysis, sensitivity testing, and inter-rater reliability assessment to quantify paradigm performance. The findings confirm Positivism's strength in empirical fields like Science and Philosophy but reveal its limitations in addressing ethical, cultural, and interpretive dimensions, where paradigms like Hermeneutics and Feminism excel. Pragmatism and Critical Realism bridge empirical rigor with interpretive depth, offering integrative solutions for interdisciplinary challenges. By applying statistical tools within a philosophical inquiry, this research highlights the necessity of humility and open-mindedness in selecting paradigms to address complex questions. The study concludes that no single paradigm suffices across all domains, advocating for a multi-paradigm approach to foster collaboration, inclusivity, and a more comprehensive understanding of knowledge production. These insights have implications for interdisciplinary research, education, and policymaking, emphasizing the importance of integrating diverse epistemological perspectives.

Comparing three-dimensional zero echo time (3D-ZTE) lung MRI and chest CT in the evaluation of systemic sclerosis-related interstitial lung disease.

Systemic sclerosis (SSc) is a chronic disease that can cause interstitial lung disease (ILD), a poor prognostic factor in SSc patients. Given the concerns over radiation exposure from repeated CT scans, there is a growing interest in exploring radiation-free imaging alternatives like MRI for ILD evaluation. The aim of this study is to assess the efficacy of three-dimensional zero echo time (3D-ZTE) MRI in assessing SSc-related ILD compared to the thin-slice chest CT. This prospective single-center study investigated 65 SSc patients. SSc patients underwent CT, 3D-ZTE lung MRI, and pulmonary function tests (PFTs) within a week. Three independent reviews visually quantified ILD extent on ZTE and CT imaging and the correlation of ILD extent with PFTs was analyzed. Statistical analyses were performed, including the intraclass correlation coefficient (ICC), Kruskal-Wallis tests, Bland-Altmann analysis, and correlation analyses between imaging results and PFTs. ILD was detected in 45 patients via CT. 3D-ZTE MRI identified ILD in 41 (91.1%) of these cases, demonstrating a strong correlation with CT in assessing ILD severity (r = 0.986, p < 0.001). The median ILD extent scores were 5% for CT and 6% for 3D-ZTE MRI. Interobserver reliability for 3D-ZTE MRI was excellent, with ICC values ranging from 0.853 to 0.969. The analysis also revealed significant negative correlations between ILD extent on ZTE MRI and lung function, particularly FVC. 3D-ZTE lung MRI is a reliable and radiation-free alternative to chest CT for evaluating SSc-related ILD, with a strong correlation in assessing total fibrosis and ground-glass opacities, though limitations remain in detecting fine reticulations and coarseness. Question Can 3D-ZTE MRI replace thin-slice chest CT as a radiation-free method for assessing SSc-related ILD? Findings 3D-ZTE lung MRI showed an excellent agreement with thin-slice CT in evaluating ILD extent in SSc patients (r = 0.986, p < 0.001). Clinical relevance 3D-ZTE lung MRI provides a reliable, radiation-free alternative to CT for assessing ILD extent in SSc patients, ensuring safer longitudinal monitoring and management.

Development and validation of an observational tool to measure engagement in telepractice early intervention

Purpose To develop and validate an evidence-based observational tool to measure engagement in telepractice early intervention (EI). Materials and Methods The Engagement in Telepractice Observational Tool (eTOT) was developed based on a previous e-Delphi study. Two independent raters used a set of 50 items to score 45 videorecorded telepractice EI sessions with young children with communication difficulties and their family. Principal component analysis (PCA) and reliability measures were conducted. Results PCA identified six key components related to the measurement of engagement in telepractice. The components representing professional behaviours were (1) interacting directly with the parent; (2) getting to know the child and family to keep them motivated; (3) maximising the telepractice environment; and (4) demonstrating and practising the parent’s role as agent of therapy. The two remaining components represented family behaviours: (5) parent interacting and supporting their child; and (6) family collaborating directly with the professional. The eTOT was found to have good reliability (α = .86), with very good test-retest and inter-rater reliability scores (κw = .85 and .94 respectively). Conclusion There is preliminary evidence that the eTOT is a valid and reliable tool to measure engagement during telepractice EI. The eTOT can be used as a self-reflection tool to identify and provide engagement-building opportunities in family-centred telepractice EI.

Development and validation of a self-perception instrument for critical media literacy in Chilean pre-service teachers

IntroductionThe influence of mass media and social media has profoundly impacted the way individuals access, interpret, and engage with information. It is becoming increasingly important to gain an understanding of media literacy practices, particularly among those who influence children and young people. In Spanish, there is little knowledge about critical literacy practices related to media content. This study aims to develop and validate a self-perception instrument designed to measure critical media literacy practices among Chilean pre-service teachers.MethodA structured written questionnaire with closed-ended questions was designed and validated. The instrument encompasses three dimensions: situating discourse within sociocultural contexts, recognizing and participating in discursive practices, and assessing the impact of media on the community and the individual. It comprises 28 first-person statements, rated on a five-point Likert scale.ResultsThe content was validated by a panel of experts in education, linguistics, and journalism in two phases, resulting in a high inter-rater agreement (κ = 0.78) and a content validity coefficient (CVC) of 0.97. A pilot study was conducted with 92 undergraduate students enrolled in teacher education programs. Exploratory factor analysis confirmed a three-factor structure that aligns with the proposed dimensions, explaining 51% of the total variance. Internal consistency analyses showed strong reliability, with a Cronbach’s alpha of 0.84 and McDonald’s omega of 0.85.DiscussionThe results indicate that the instrument is valid and reliable for assessing self-perceived critical media literacy practices. The findings underscore the need for educational interventions to enhance students’ ability to critically engage with media content.

Otolaryngologists Demonstrate Low Agreement on Pediatric Airway Inflammation Assessment.

Microlaryngoscopy and bronchoscopy (MLB) are essential tools for evaluating airway inflammation, but the reliability of endoscopic assessments for this purpose remains unclear. The aim of this study was to assess surgeons' interrater reliability during endoscopic assessment of airway inflammation. The endoscopic factors used to determine overall airway inflammation were also assessed. We conducted a cross-sectional study involving 24 pediatric patients who underwent MLB and arytenoid biopsy. Surgeons rated airway inflammation based on endoscopic images, and interrater reliability was assessed using Fleiss' Kappa. Fleiss's Kappa demonstrated poor interrater reliability among all surgeons (0.111) and experienced surgeons (0.117). Surgeons varied in prioritizing visual features for assessing inflammation. Current subjective assessments of airway inflammation during MLB exhibit poor interrater reliability, necessitating further research for improved diagnostic accuracy and informed treatment decisions in pediatric airway interventions. Level 3 Laryngoscope, 2024.

Diagnostic evaluation of open knee injuries: a comparison of arthrography and CT scans.

Currently, traumatic arthrotomy of the knee could be diagnosed by saline loading test and computed tomography (CT) which reported excellent sensitivity and specificity. However, there are drawbacks associated with CT, such as cost and radiation hazards. This study aims to evaluate a new diagnostic tool using knee arthrography for individuals with suspected open knee injury and reported sensitivity and specificity using CT scan as the gold standard. A prospective diagnostic study, involving 43 patients who sustained laceration wounds over the knee, was conducted. The physical examination and conventional radiographic imaging were inconclusive in determining the presence of open joint injury. The established protocol involved directing the patients to undergo a CT scan, which is considered the gold standard for detecting open knee injuries. Subsequently, arthrography of the affected knee was performed. The diagnostic value was subsequently determined based on the outcomes derived from these procedures. There were 5/43 cases (11.6%) that diagnosed open knee injuries as determined by CT scans. The calculated diagnostic parameters of arthrography derived from these findings are as follows: The sensitivity was 83.3%, the specificity was 96.5%, the negative predictive value was 96.5%, the positive predictive value was 83.3%, the accuracy is 94.3%, the false positive rate is 16.7%, and the false negative rate was 3.4%. Interobserver reliability was substantial (Kappa 0.617). No complication was observed during the arthrography procedure. Knee arthrography demonstrated high sensitivity, specificity and accuracy for diagnosing open knee injuries, offering a viable alternative to CT scans in certain situations, such as when minimizing radiation exposure is a priority and cost considerations are paramount. Ultimately, careful clinical judgement should guide the choice of diagnostic modality based on individual patient circumstances.

Fracture Line Morphology and a Novel Classification ofPilon Fractures.

Currently, there is no research that includes a comprehensive three-dimensional fracture mapping encompassing all types of Pilon fractures. Moreover, the existing classification systems for Pilon fractures exhibit only moderate to fair consistency and reproducibility. Additionally, some of these classification systems fail to accurately depict the morphological characteristics of the fractures. This study aimed to create a fracture map encompassing all types of Pilon fractures by three-dimensional fracture mapping. In addition, this study conducted a finite element analysis of the normal ankle joint, and based on the distribution of fracture lines and the stress distribution at the distal tibia, proposed a new classification for Pilon fractures. A retrospective analysis of Pilon fractures in our hospital from January 2018 to January 2024 was performed. A total of two hundred forty-four Pilon fractures were included, and their fracture lines were transcribed onto the tibia and fibula templates, and fracture maps and heat maps were created. A nonhomogeneous model of the ankle joint was constructed and verified, and the stress distribution on the distal tibia articular surface was measured and analyzed in three models (neutral, dorsiflexed, and plantarflexed model). Based on the fracture map and stress distribution, a five-column classification system for Pilon fractures was proposed, and the intraobserver and interobserver reliability was calculated using Cohen and Fleiss k statistics. The fracture line on the distal tibia articular surface showed a V-shaped distribution. One branch extended from the junction of the medial malleolar articular surface and the inferior tibial articular surface toward the medial malleolus. The other branch extended from the middle of the fibular notch to the posterior part of the medial ankle, toward the tibial shaft. The fibula fracture line mainly extended from the anterior and lower part of the lateral malleolus to the posterior and upper part. As evidenced by the neutral, dorsiflexed, and plantar flexion models, the stress on the posterolateral articular surface (posterolateral column) was low, while the majority of the stress was concentrated in the center. Three-column fractures were the most common, followed by two-column fractures. Using the five-column classification, the K-weighted values of interobserver and intraobserver analysis were 0.653 (p < 0.001) and 0.708 (p < 0.001), respectively. In this study, the fracture line and morphological characteristics of Pilon fractures were analyzed in detail by three-dimensional mapping. In addition, this study conducted a finite element analysis of the stress distribution on the distal tibial joint surface of the normal ankle joint. Moreover, a novel classification system was proposed to reflect these findings. The new classification not only exhibits greater consistency, facilitating accurate communication of fracture characteristics among surgeons, but also aids in understanding the mechanisms of injury and formulating surgical strategies.

Using Deep Learning to Suggest Treatment for Proximal Humerus Fractures.

Proximal humeral fractures are among the most common fractures seen in emergency departments. Accurately diagnosing and selecting the most appropriate treatment for these fractures can be challenging, and consultation with a senior orthopedic surgeon can be time-consuming for both the patient and the emergency unit. We developed a machine learning model for predicting the type of treatment based on injury radiographic images. The model distinguishes between nonoperative and operative treatment options, achieving an accuracy of 86% and an interobserver reliability (kappa) of 0.722 for test-dataset, which is more than the interobserver agreement between shoulder surgeons. This model has the potential to serve as a therapeutic decision support system for the practitioners in the emergency departments to expedite treatment decisions and to reduce patients' waiting time.

Subtle bradykinesia features are easier to identify and more prevalent than questionable dystonia in essential tremor.

Essential tremor (ET) is characterized by upper limbs action tremor, sometimes extending to other body parts. However, ET can present with additional neurological features known as "soft signs." These signs of uncertain clinical significance are not sufficient to suggest an alternative neurological diagnosis, and include, among others, questionable dystonia and subtle voluntary movement alterations, i.e., bradykinesia and related features. This study aimed to explore the prevalence and relationship between questionable dystonia and subtle bradykinesia features in ET. Forty ET patients were video-recorded during clinical examination. Five movement disorder experts reviewed the videos to identify soft motor signs, i.e., dystonia and movement alterations during finger-tapping namely, (i) bradykinesia (reduced velocity), (ii) dysrhythmia, and (iii) sequence effect. Inter-rater agreement was quantified using the Fleiss' Kappa index. Data analysis was performed using nonparametric tests. We found a fair inter-rater agreement for upper limb dystonia (Fleiss' K = 0.27). Inter-rater agreement was higher (moderate) for head dystonia (Fleiss' K = 0.49) and finger-tapping assessment (Fleiss' K = 0.45). Upper limb dystonia was identified in 70% of patients, head dystonia in 35%, and finger-tapping alterations (in variable combinations) were observed in 95% of individuals (P < 0.001 by Fisher's exact test), including subtle bradykinesia and related features. No significant concordance or correlation was found between the soft signs. Subtle bradykinesia and related features are the most easily identifiable and frequent soft signs in ET, appearing in a higher percentage of patients than subtle dystonia. These findings provide insights into the clinical and pathophysiological understanding of ET.

Osteochondritis Dissecans of the Capitellum: Influence of Activity on Lesion Location Within a Zone of Propensity.

There may be an association between repetitive stress on the elbow and capitellar osteochondritis dissecans (COCD). The influence of activity on the characteristics or specific location of the lesion within a capitellar zone of propensity remains unclear. The purpose of this study was to evaluate COCD in gymnasts and baseball players compared with a non-upper extremity (non-UE) sport COCD cohort. It was hypothesized that activity would influence presentation and lesion location and characteristics. Cross-sectional study, Level of evidence, 3. Consecutive patients presenting with COCD treated in a tertiary sports medicine practice between March 2006 and March 2021 were reviewed. Demographic and imaging lesion characteristics were recorded. The sagittal inclination angle position of the lesion center relative to the humeral shaft was recorded. Intra- and interrater reliability was confirmed (intraclass correlation coefficient, 0.808; intraclass correlation coefficient, 0.824, respectively). Participants were analyzed by activity. A total of 126 elbows were identified: 68 in gymnasts, 31 in baseball players, and 27 in patients with no history of UE sports. The mean age of the participants was 12.5 years (range, 8-18 years). All (100%) unilateral cases of osteochondritis dissecans (OCD) in baseball players occurred in the dominant throwing arm, compared with less hand-dominance correlation in gymnasts (64%) and non-UE athletes (67%). Significantly, 14 of the 20 (70%) patients with bilateral OCD were gymnasts (P = .036). In non-UE athletes, lesions occurred at a mean sagittal inclination angle of 52.7° (±13.8°), with a mean lateral containment of 4.7 (±2.1) mm. In gymnasts, lesions occurred more posteriorly at 41.2° (±14.9°), with a lateral containment of 3.6 (±1.9) mm. In baseball players, COCDs occurred at 55.1° (±11.9°), with a lateral containment of 2.5 (±1.6) mm. These more posterior lesion positions in gymnasts (P = .001) and lesser lesion containment noted in gymnasts (P = .015) and baseball players (P < .001) were significant. Compared with COCD lesions in adolescents with no history of UE sports, gymnastics participation (impact weightbearing) correlated with a higher rate of bilateral lesion formation and a more posterior (extended) lesion position. Baseball participation (resulting in valgus shear forces in the elbow) resulted in a sagittal position similar to non-UE lesions, but more lateral and with less containment.

Evaluation of post-extraction healing after atraumatic extraction in gutka chewers and non-gutka chewers using the modified inflammation proliferation remodeling scale

BackgroundGutka, a smokeless tobacco mixture containing tobacco and areca nut, is widely consumed in South Asia and impairs wound healing due to vasoconstrictive and cytotoxic effects. Wound healing assessment in oral surgery lacks standardisation, and phase-specific evaluation tools are seldom used. This study aimed to compare post-extraction healing after atraumatic tooth extraction in gutka chewers and non-chewers using the modified Inflammation Proliferation Remodeling (IPR) scale to validate this scale for clinical use.MethodsA prospective cohort study at a dental college in Karachi, Pakistan, included 200 participants aged 18–45 undergoing atraumatic extraction of premolars or molars, divided into gutka chewers (n = 100) and non-chewers (n = 100). The modified IPR scale was developed and validated using content validity assessment by experts, pilot testing, inter-rater reliability (Cohen’s kappa), and internal consistency (Cronbach’s alpha). Wound healing was assessed using the validated modified IPR scale immediately post-extraction (Day 0), at the proliferative phase (Day 7), and the remodelling phase (6 weeks). Statistical analyses included Mann-Whitney U tests, multivariate linear regression, and Spearman’s correlation.ResultsThe modified IPR scale demonstrated good content validity, substantial inter-rater reliability (Cohen’s kappa 0.80–0.86), and good internal consistency (Cronbach’s alpha 0.84). No significant difference in IPR scores was observed between groups during the inflammatory phase (Day 0). Gutka chewers had significantly higher IPR scores during the proliferative phase (mean 5.9 ± 1.6 versus 4.2 ± 1.2; p < 0.001) and the remodelling phase (mean 3.6 ± 1.2 vs. 2.1 ± 0.9; p < 0.001), indicating delayed healing. Gutka use was a significant predictor of delayed healing (B = 1.2, p < 0.001) after adjusting for age and gender. Duration of gutka use positively correlated with higher IPR scores during the proliferative (rho = 0.46, p < 0.001) and remodelling phases (rho = 0.51, p < 0.001), reflecting a dose-dependent effect. The questionnaire was effectively validated with an 80% response rate and a 90.9% completion rate.ConclusionsGutka chewing is significantly associated with delayed post-extraction wound healing, with longer duration correlating with poorer outcomes. The modified IPR scale was effectively validated and proved to be a reliable tool for assessing wound healing progression, demonstrating its utility in clinical practice and research settings. Dental professionals should consider gutka use when planning patient care, and public health initiatives should aim to eliminate gutka consumption to improve oral health outcomes.

Evaluation Framework of Large Language Models in Medical Documentation: Development and Usability Study.

The advancement of large language models (LLMs) offers significant opportunities for health care, particularly in the generation of medical documentation. However, challenges related to ensuring the accuracy and reliability of LLM outputs, coupled with the absence of established quality standards, have raised concerns about their clinical application. This study aimed to develop and validate an evaluation framework for assessing the accuracy and clinical applicability of LLM-generated emergency department (ED) records, aiming to enhance artificial intelligence integration in health care documentation. We organized the Healthcare Prompt-a-thon, a competitive event designed to explore the capabilities of LLMs in generating accurate medical records. The event involved 52 participants who generated 33 initial ED records using HyperCLOVA X, a Korean-specialized LLM. We applied a dual evaluation approach. First, clinical evaluation: 4 medical professionals evaluated the records using a 5-point Likert scale across 5 criteria-appropriateness, accuracy, structure/format, conciseness, and clinical validity. Second, quantitative evaluation: We developed a framework to categorize and count errors in the LLM outputs, identifying 7 key error types. Statistical methods, including Pearson correlation and intraclass correlation coefficients (ICC), were used to assess consistency and agreement among evaluators. The clinical evaluation demonstrated strong interrater reliability, with ICC values ranging from 0.653 to 0.887 (P<.001), and a test-retest reliability Pearson correlation coefficient of 0.776 (P<.001). Quantitative analysis revealed that invalid generation errors were the most common, constituting 35.38% of total errors, while structural malformation errors had the most significant negative impact on the clinical evaluation score (Pearson r=-0.654; P<.001). A strong negative correlation was found between the number of quantitative errors and clinical evaluation scores (Pearson r=-0.633; P<.001), indicating that higher error rates corresponded to lower clinical acceptability. Our research provides robust support for the reliability and clinical acceptability of the proposed evaluation framework. It underscores the framework's potential to mitigate clinical burdens and foster the responsible integration of artificial intelligence technologies in health care, suggesting a promising direction for future research and practical applications in the field.

Thermographic and ultrasound assessment in patients with rheumatoid arthritis: can thermography detect subclinical synovitis at the wrist?

BackgroundThermography is an emerging imaging modality which allows for a quick and objective measure of joint surface temperature in patients with rheumatoid arthritis (RA). To date, there are no published studies comparing thermography with ultrasonography in the subclinical assessment of joint inflammation at the wrist of patients with RA, and no published data on inter-rater reliability for multiple raters for thermographic assessment at the RA wrist. In our study comparing thermography and ultrasonography at the RA wrist, we aim to determine if thermography can detect subclinical synovitis. Additionally, we performed inter-reliability testing (multiple raters) for both thermography and ultrasonography.MethodsThermographic (average (Tavg), maximum (Tmax) and minimum (Tmin) temperatures) and ultrasound (total grey-scale (TGS) score and total power Doppler (TPD) scores) parameters were compared between two patient groups: Asymptomatic Group (with non-swollen and non-tender wrists) and Symptomatic Group (with swollen and/or tender wrists). Among Asymptomatic Group patients, thermographic parameters were compared between those with and without wrist joint recess(es) having ultrasound synovitis (PD ≥ 1 or GS ≥ 2); Spearman’s correlation and simple linear regression were used to study the relationship between thermographic and ultrasound parameters. Intra-class correlation coefficient (ICC) was used for inter-rater reliability calculation.ResultsEighty-seven RA patients’ right wrists were imaged in this cross-sectional study. Thermographic temperatures, TPD and TGS scores were all significantly (p < 0.05) greater among Symptomatic Group versus Asymptomatic Group patients. Among Asymptomatic Group patients, thermographic temperatures were all significantly higher (P < 0.01) in wrists having joint recess(es) with ultrasound PD ≥ 1 or GS ≥ 2, while all thermographic parameters correlated significantly with TPD (correlation coefficient ranging from 0.43 to 0.48, p < 0.001) and TGS (correlation coefficient ranging from 0.33 to 0.37, p < 0.01). The ICC values based on a subset of images obtained for inter-reliability testing were high for thermography (0.994 to 0.998) and ultrasonography (0.933 to 0.952).ConclusionsSwollen and/or tender RA wrists displayed greater thermographic and ultrasound-detected joint inflammation. At clinically quiescent (non-swollen; non-tender) wrists, thermographic temperatures significantly correlated with ultrasound-detected joint inflammation.Clinical trial numberNot applicable.

Assessing the Reliability of the Bleeding Assessment Scale in Critically Ill Children (BASIC) Definition: A Prospective Cohort Study.

To determine the reliability of the Bleeding Assessment Scale in critically Ill Children (BASIC) definition of bleeding severity in a diverse cohort of critically ill children. Prospective cohort study. Eight mixed PICUs in the Netherlands, Israel, and the United States. Children ages 0-18 years admitted to participating PICUs from January 1, 2020, to December 31, 2022, with bleeding noted by bedside nurse. None. The bleeding events were classified as minimal, moderate, or severe, according to the BASIC definition, by two independent physicians at two different time points. Patient demographic data, laboratory values, and clinical outcomes were collected. Three hundred twenty-eight patients were enrolled. The overall inter-rater reliability was substantial (weighted kappa coefficient, 0.736; 95% CI, 0.683-0.789), and the intra-rater reliability was "almost-perfect" (weighted kappa coefficient, 0.816; 95% CI, 0.769-0.863). The platelet count (p = 0.008), prothrombin time (p = 0.004), activated partial thromboplastin time (p = 0.025), and fibrinogen levels (p = 0.035) were associated with the bleeding severity, but the international normalized ratio was not (p = 0.195). Patients were transfused blood components in response to any bleeding in 31% of cases and received hemostatic medications in 9% of cases. More severe bleeding was associated with increased 28-day mortality, longer hospital length of stay, and more days receiving inotropic support. The BASIC definition is a reliable tool for identifying and classifying bleeding in critically ill children. Implementing this definition into clinical and research practice may provide a consistent and reliable evaluation of bleeding.

Evaluating Inter-Rater Reliability: Transitioning to a Single Rater for Marking Modified Essay Questions in Undergraduate Medical Education

Modified Essay Questions (MEQs) are often included in high-stakes examinations to assess higher-order cognitive skills. If the marking guides for MEQs are inadequate, this can lead to inconsistencies in marking. To ensure the reliability of MEQs as a subjective assessment tool, candidates’ responses are typically evaluated by two or more assessors. Previous studies have examined the impact of marker variance. Current study explores the possibility of assigning a single assessor to mark the students' performances in MEQ based on statistically drawn evidence in the clinical phase of the MBBS program at a private medical school in Malaysia. A robust evaluation method was employed to determine whether to continue with two raters or shift to a single-rater scheme for MEQs, using the Discrepancy-Agreement Grading (DAG) System for evaluation. A low standard deviation was observed across all 11 pairs of scores, with insignificant t-statistics (P&gt;0.05) in 2 pairs (18.18%) and significant t-statistics (P&lt;0.05) in 9 pairs (81.81%). The Intraclass Correlation Coefficient (ICC) results were excellent, ranging from .815 to .997, all with P&lt;0.001. Regarding practical effect size (Cohen’s d), 1 pair (9.09%) was categorized as having a strong effect size (&gt;0.8), 7 pairs (63.63%) as having a moderate effect size (0.5-&lt;0.8), and 3 pairs (27.27%) as having a weak effect size (0.2-&lt;0.5). The data analysis suggests that it is feasible to consider marking MEQ items by a single assessor without negatively impacting the reliability of the MEQ as an assessment tool.