Evaluate Interrater Reliability Research Articles

Radiographs demonstrate poor accuracy for evaluation of postoperative osseous integration of osteochondral allografts in the knee

PurposeTo evaluate the interrater reliability and accuracy of radiographs in assessing healing of osteochondral allografts (OCA) in the knee compared to computed tomography (CT) imaging. MethodsA retrospective review patients who underwent OCA transplantation of the knee was conducted. Postoperatively, knee radiographs were obtained at three and six months, and CT scans were obtained at six months. Three board-eligible orthopaedic surgeons independently graded postoperative radiographs, utilizing the grading scale established by Oladeji et al 2017, and the CT scans utilizing the grading scale established by Gelber et al 2021. Their consensus measurements served as a comparative gold standard. ResultsAmong 44 prospectively imaged grafts, 13 (29.5%) had radiographs at three months and 28 (63.6%) had radiographs available for evaluation at six months. Assessment of radiographic integration demonstrated low reliability (ICC: -0.02 to 0.14). On CT scan, this improved to fair reliability (k=0.29, p<0.001). However, Spearman rho between CT and XR grading demonstrated poor correlation (ρ = -0.23, p=0.04). Among six-month postoperative radiographs, accuracy of cyst identification was negatively correlated to greater degree of cystic changes from 90.9% (30/33 correct; no cystic changes) to 19.4% (7/36 correct; small cystic changes) and 33.3% (5/15 correct; large cystic changes). Overall true positive cyst detection rates on radiographs were 27.8% to 46.7%. ConclusionRadiographs demonstrate poor interrater reliability and accuracy in evaluation of healing after OCA transplantation of the knee. Notably, there was a high rate of missed cystic changes on radiographic analysis, as well as poor correlation of integration assessment with CT scan findings.

Diaphragmatic Ultrasound in Children With Asthma Exacerbations.

Asthma is a leading cause of pediatric emergency department visits, yet few tools exist to objectively measure asthma severity. Diaphragmatic ultrasound (DUS) is a novel method of assessing respiratory distress; however, data are lacking for children.Our primary aim was to determine if diaphragmatic excursion (DE), diaphragmatic thickening (DT), or diaphragmatic thickening fraction (TF) correlated with asthma severity as determined by the Pediatric Respiratory Assessment Measure (PRAM) score. Secondarily, we examined if these parameters correlated with need for respiratory support and evaluated interrater reliability. We conducted a prospective study of children 5-18 years presenting to a pediatric emergency department with an asthma exacerbation. Diaphragmatic ultrasound was performed by a trained pediatric emergency medicine sonologist in subcostal (DE) and midaxillary (DT). Thickening fraction was calculated from DT values as previously described in literature. To evaluate interrater reliability, a subset of subjects had DUS performed by a second sonologist. We enrolled 47 subjects for a total of 51 encounters. The mean age was 9.1 ± 3.7 years. Twenty-five (49%) had mild, 24 (47%) had moderate, and 2 (4%) had severe asthma. There was a significant difference in midaxillary DT and TF between children with mild and moderate asthma (P = 0.02; mean difference, 0.2 mm; 95% confidence interval [CI], 0.03-0.4 and P = 0.02; mean difference, 0.11 mm; 95% CI, 0.02-0.2, respectively). No difference was found in subcostal DE (P = 0.43; mean difference, 1.4 mm; 95% CI, -2.1 to 4.8). No association was found between use of positive pressure and DUS parameters. Fourteen encounters had 2 sonologists perform DUS, with strong interrater reliability found for midaxillary DT (Pearson correlation, 0.56) and poor association for subcostal DE (Pearson correlation, 0.18). In this pilot study, we conclude that DUS may be helpful in assessing severity of asthma. The midaxillary view assessment for DT and TF had the best correlation with asthma severity and the best interrater reliability. Future studies may benefit from focusing on the midaxillary view for DT and TF.

Navigating Intimate Partner Violence: A Comparative Analysis Between ACOG and ChatGPT on Patient Education [ID 2683552

INTRODUCTION: With the surge in artificial intelligence usage, there is an increasing need to assess its applicability in conveying public health information. We compare the readability and quality of responses between ChatGPT and the American College of Obstetricians and Gynecologists (ACOG) in answering frequently asked questions (FAQs) on intimate partner violence (IPV). METHODS: Twelve questions from ACOG's “Intimate Partner Violence” FAQs were posed to ChatGPT-3.5 (July 19 Version). Readability and grade-level scores were determined. The quality of responses were also graded by two obstetrician–gynecologists using a 1–4 scale, where 1 represents a comprehensive response and 4 indicates an incorrect response. Statistical analysis utilized a two-tailed t-test. A weighted Cohen’s kappa coefficient evaluated interrater reliability. RESULTS: Mean readability favored ACOG over ChatGPT, but only the Coleman Liau Index was statistically significant (ACOG: 12.71; ChatGPT: 15.76; P=.003). Other readability measures demonstrated no significant differences. The ACOG responses were graded with an average of 1.29, alongside a Cohen’s kappa coefficient of 0.375, implying fair agreement between graders. All ChatGPT’s responses were graded a 1 with a Cohen’s kappa coefficient of 1, indicating perfect agreement. The difference in grades between ACOG and ChatGPT was statistically significant (P=.013). CONCLUSION: The ACOG FAQ responses are relatively equivalent in readability when compared to ChatGPT-generated responses with only one statistically significant difference across the indices. However, ChatGPT's responses were more comprehensive and accurate. Although patients can use ACOG as a source, ChatGPT provides equivalently clear and more comprehensive information on IPV.

Pregnancy Loss: A Comparative Analysis of Information Delivery Between ACOG FAQs and ChatGPT [ID 2683594

INTRODUCTION: Pregnancy loss is a sensitive topic, and clear, reliable information is crucial. This study compares the readability and accuracy of ChatGPT's responses to ACOG's frequently asked questions (FAQ) sheets on the subject. METHODS: Within the five FAQ sheets under “Pregnancy Loss,” 66 questions were assessed using ChatGPT-3.5. Readability scores, ie, Flesch–Kincaid Reading Ease, Flesch–Kincaid Grade Level, Gunning Fog Score, Smog Index, Coleman Liau Index, and Automated Readability Index were computed for each response. The quality of responses were also graded by two maternal–fetal medicine specialists using a 1–4 scale, where 1 represents a comprehensive response and 4 indicates an incorrect response. Statistical analysis utilized a two-tailed t-test. A weighted Cohen’s kappa coefficient evaluated interrater reliability. RESULTS: ACOG attained better readability scores over ChatGPT across all six metrics with statistical significance (P&lt;.001). Among grading, ACOG also outperformed ChatGPT in quality of information with a summative mean of 1.53 (SD 0.60) versus 1.85 (SD 0.90) with statistical significance (P&lt;.001). Grading for ACOG possessed a Cohen's kappa coefficient of 0.493, implying moderate agreement between graders. Grading for ChatGPT possessed a Cohen's kappa coefficient of 0.744, implying substantial agreement between graders. CONCLUSION: The responses by ACOG to frequently asked questions on pregnancy loss were both clearer and more comprehensive than those of ChatGPT with consistent agreement between graders. Although ChatGPT can be a convenient source of medical queries, further improvements ought to be made to these large-language models to provide clear and accurate information regarding pregnancy loss to patients.

Evaluation of the Interrater Reliability of Sonographic Measurements of Muscle Thickness of 38 Piriformis Muscles in 19 Patients with Piriformis Syndrome.

BACKGROUND The piriformis muscle is a flat superficial muscle of the deep gluteal muscles that externally rotates the hip. Ultrasound is widely used to identify the piriformis muscle, especially for guidance of the needle during injections; however, its diagnostic use has recently gained popularity. The operator-dependent nature of ultrasound requires demonstration of reliability between operators. This study aimed to evaluate interrater reliability of sonographic measurements of muscle thickness of 38 piriformis muscles in 19 patients with piriformis syndrome. MATERIAL AND METHODS An ultrasound transducer was placed transversely on the sacral spinous process and moved caudo-laterally until the piriformis muscle was visualized under the gluteus maximus while patients were lying in prone position. The thickness of piriformis muscle was measured with a 2 to 5-MHz broadband curvilinear transducer in 3 regions (thickest regions of muscle over the ilium, near the greater trochanter, and near the sacrum). The interrater reliability of measurements of 2 examiners who were blinded to each other's measurements was assessed by intraclass correlation coefficient. RESULTS In total, 114 samples from 38 piriformis muscles of 19 patients with a diagnosis of piriformis syndrome were evaluated by 2 raters in this study. The median (interquartile range) patient age was 41 (15) years. Intraclass correlation coefficient value for overall thickness measurements of piriformis muscle was 0.836. Intraclass correlation coefficient values for 3 different regions were over the ilium, near the greater trochanter, and near the sacrum were 0.777, 0.883, and 0.811, respectively. CONCLUSIONS Ultrasound measurement of piriformis muscle thickness has good interrater reliability.

Interrater reliability of the ovarian–adnexal reporting and data system magnetic resonance imaging (O‑RADS MRI)

Abstract Aim The current study aims to evaluate interrater reliability of ovarian–adnexal reporting and data system (O-RADS) magnetic resonance imaging (MRI) in interpretation of adnexal and ovarian lesions. Material and methods Retrospective analysis of 131 ovarian lesions was as done for 106 consecutive female patients with adnexal and/or ovarian lesions that underwent MR imaging of the pelvis. Images interpretation was accomplished by two-blinded independent raters for cystic and solid parts of ovarian lesions. The score was 5 types classified pursuant to O-RADS. Results A perfect interrater agreement regarding overall O-RADS [Kappa: 0.874, P: 0.001]. There was a perfect interrater agreement of the solid component (Kappa: 0.979, 95% confidence interval (CI) 0.938–1.0, P: 0.001), enhancement degree relative to myometrium (Kappa: 0.876, 95% CI 0.781–0.971, P: 0.001) and entirely solid lesions (Kappa: 1.0, 95% CI 1.0–1.0, P: 0.001). A perfect interrater agreement for ORADS 1 (Kappa: 0.937, P: 0.001), ORADS 2 (Kappa: 0.983, P: 0.001), ORADS 3 (Kappa: 0.834, P: 0.001), ORADS 4 (Kappa: 0.827, P: 0.001) and ORADS 5 (Kappa: 0.963, P: 0.001) was present. Conclusions The O-RADS MRI scoring system has better characterization of adnexal masses with high interrater agreement. Overcoming limitations of this study, O-RADS, may be suggested as a basic system in assessment of adnexal masses.

A Novel Method to Determine Patient Skin Type: The Skin Analyzer.

Measuring skin color for medical research in an objective and nonbiased manner usually requires expensive equipment such as spectrophotometry and requires the subject to be present in person. We present a novel method to measure skin color from photographs using the Skin Analyzer application as a more effective, accessible, and efficient alternative. A desktop application, the Skin Analyzer, was developed to convert skin samples collected from digital images to the L*a*b color space and uses those values to calculate an individual typology angle that correlates to a Fitzpatrick skin type. To assess accuracy in variable lighting, six known colors representing the six Fitzpatrick skin types were printed and photographed in 15 separate locations within the hospital. To account for user variability in sample selection, interrater reliability was calculated with data generated by 13 untrained users testing the app on six subjects. The accuracy of measuring known values, which is the classification accuracy, was calculated to be 80%. Krippendorff alpha test was used to evaluate interrater reliability. The obtained alpha of 0.84 indicates a high interrater reliability. The high accuracy and reliability make the Skin Analyzer a suitable method of objectively determining Fitzpatrick skin type from images. The app may be used to investigate the effects of skin tone in various areas of interest, especially in retrospective studies where skin colorimeters cannot be used.

Reliability of the Single-Arm and Double-Arm Jobe Test for the Diagnosis of Full-Thickness Supraspinatus Tendon Tear.

The Jobe test is commonly used to diagnose full-thickness (FT) supraspinatus (SSP) tendon tear. The original Jobe test used single-arm testing, although the double-arm Jobe test has also been used in clinical practice. To evaluate the reliability, accuracy, and diagnostic value of the single-arm and double-arm Jobe test for diagnosis of FT SSP tear. Cohort study (diagnosis); Level of evidence, 2. Patients with shoulder pain requiring magnetic resonance imaging (MRI) of the shoulder between March 1, 2021, and March 31, 2022, were enrolled. Each patient underwent both single-arm and double-arm Jobe tests by 2 orthopaedic surgeons independently, and the presence of pain, weakness, or both during the test was documented. Diagnosis of FT SSP tear on MRI scan was used as the gold standard to compare the diagnostic value of the Jobe test. The interrater reliability of the Jobe test, and the inter- and intrarater reliability of the MRI evaluation (patients with vs without FT SSP tear) was performed using the kappa (κ) coefficient. A total of 80 patients (57 females [71%]; mean age, 61.46 ± 9.61 years) were included. MRI scans revealed FT SSP tears in 32 (40%) of the patients. Both single- and double-arm Jobe tests had low diagnostic values (accuracy, 46.25% to 60%; sensitivity, 46.9% to 84.4%; specificity, 25% to 66.7%). The single-arm test with weakness revealed the highest sensitivity (84.4%). The double-arm test with weakness plus pain revealed the highest specificity (66.7%). Double-arm testing with pain had the highest accuracy (60%), with the highest positive likelihood ratio (1.5). The interrater reliability of the Jobe test indicated substantial agreement (double-arm vs single-arm, κ = 0.771 and 0.716, respectively, agreement 85%; P < .05). The interrater reliability of MRI scan evaluation of the FT SSP tear indicated substantial agreement (κ = 0.750, agreement 85%; P < .05), while the intrarater reliability indicated almost perfect agreement (κ = 0.917, agreement 96%; P < .05). The Jobe test, either single- or double-arm, had low accuracy and diagnostic value in diagnosing FT SSP tear. The concern with a single-arm examination for weakness is that it may be an inappropriate diagnostic test for ruling out FT SSP with 84% sensitivity, while a double-arm examination provides a higher specificity.

An Application of Theril Indexes for the Interrater Reliability: A Comparison with Intraclass Correlations

This study proposes to apply the Theil-index ratios for the interrater reliability. We discuss the theoretical foundations and examine its function empirically using real data. Our analyses show that Theil-index rations and intraclass correlation (ICC) estimates are highly correlated. However, ICC may underestimate the interrater reliability by some extreme disagreement among raters and be more likely to be influenced by the extreme disagreement. As Theil-index ratios overcome the limitations of ICC to some degree, it seems that Theil-index ratios provide an alternative to evaluating interrater reliability, at least under certain conditions, e.g., when outliers exist in the data, it is difficult to obtain the variance component estimates, or ICC underestimates interrater reliability.

Interrater reliability of the Standing and Walking Assessment Tool for spinal cord injury.

Psychometric study. The Standing and Walking Assessment Tool (SWAT) is a standardized approach to the evaluation of standing and walking capacity following traumatic spinal cord injury (tSCI) in Canada. The SWAT classifies individuals with a tSCI into 12 stages of standing and walking capacity that are paired with well-established outcome measures, such as the Berg Balance Scale and 10-m Walk Test. Prior research has demonstrated the validity and responsiveness of the SWAT stages; however, the reliability remains unknown. The objective of this study was to evaluate the interrater reliability of the SWAT stages. Inpatient units of two Canadian rehabilitation hospitals. Adults with sub-acute tSCI were recruited. SWAT stage was evaluated for each participant by two physical therapists separately. The two therapists aimed to complete the evaluations within one day of each other. To evaluate interrater reliability, the percentage agreement between the SWAT stages rated by the two physical therapists was calculated, along with a linear weighted kappa statistic with a 95% CI. Forty-five individuals with sub-acute tSCI (36 males, 9 females, mean (SD) age of 54.8 (17.9) years) participated. The percentage agreement in SWAT stages between the two physical therapists was 75.6%. A kappa statistic of 0.93 with a 95% CI, 0.81-1.05 was obtained. In cases where therapists disagreed (18% of participants), therapists differed by 1-2 stages only. The SWAT stages have high interrater reliability, providing further support for the use of the SWAT in rehabilitation practice in Canada.

Poster 303: Radiographs Demonstrate Poor Interrater Reliability and Accuracy for Evaluation of Postoperative Healing for Osteochondral Allograft Transplants of the Knee

Objectives: To evaluate the reliability of radiographs to evaluate healing of osteochondral allografts in the knee compared to CT scans as gold standard imaging. Methods: A retrospective review of prospectively enrolled patients who underwent osteochondral allograft transplantation for focal chondral defects of the knee was performed. At 6 months, all patients prospectively underwent CT scan of the knee. A retrospective review was performed to assess knee radiographs obtained in clinic at 3 and 6 months postoperatively. Three board-eligible orthopaedic sports fellows independently graded available radiographs while remaining blinded to the CT scan results. Separately, two board-eligible orthopaedic sports fellows graded CT scans utilizing the ACTOCA system, and their consensus measurements served as the gold standard for this study. Intraclass correlation (ICC) with a two-way random effects model and kappa coefficients were calculated for grades between the three raters. Results: Among the 44 grafts prospectively imaged, 13 (29.5%) grafts had radiographs at 3 months and 28 (63.6%) grafts had radiographs available for evaluation at 6 months. Among three graders, assessment of integration demonstrated low to moderate reliability (ICC: 0.25-0.41). Contrary to this, assessment of qualities such as graft sclerosis and cyst formation on radiographs demonstrated poor interrater reliability with inability to reject the null hypothesis that all grades were assigned randomly (p&gt;0.05) (Table 1). At 6 months, in patients with large cysts (&gt; 3 mm) on CT, only 2/18 (11%) were accurately detected on radiographs and 5/18 (27.8%) were underestimated as small cysts. In patients with small cysts (&lt; 3 mm) on CT, only 4/42 grafts (9.5%) were accurately graded, and 3/42 (7.1%) evaluated grafts were overestimated to have large cysts (Figure 1). Among patients who had small or large cysts at 6 months, only 1/21 (4.8%) and 1/3 (33.3%) radiographic assessments detected this cystic change at 3 months, respectively. Patients with crossing trabeculae on CT were associated with a mean radiographic integration score of 81.0 ± 15.7, which was significantly greater than those with a discernible cleft &lt; 3 mm (mean 68.7 ± 19.3, p=0.01). However, these groups were not significantly different than radiographic scores for grafts that had discernible clefts &gt; 3 mm (80.0 ± 19.7, p&gt;0.05). Among the 23 patients with a 6-month radiograph available, 4 required subsequent surgery for symptomatic failure. Three (75%) of these patients demonstrated large cyst formation on CT scan at 6 months. Conclusions: Evaluation of osteochondral allograft healing by radiographs alone may be inaccurate and associated with missed abnormalities such as cyst formation. [Table: see text]

CT-based analysis of oblique lateral interbody fusion from L1 to L5: location of incision, feasibility of safe corridor approach, and influencing factors.

The primary aim was to measure the safe corridor (SC), surgical incision anterior margin (AM), and posterior margin (PM) for OLIF bilaterally from L1 to L5. The secondary aim was to determine the feasibility of approach via the SC. The tertiary aim was to analyze the influence of demographic and anthropometric factors on OLIF parameters. We performed a radiographic analysis of 100 subjects who received an abdominal CT. Measurements of the AM, PM, and SC were obtained as well as patient age, sex, height, weight, and BMI. The intraclass correlation coefficient was used to evaluate interrater reliability. To assess associations among variables, Pearson's correlation tests and multivariate linear regression models were constructed. Sex differences were analyzed using Student's t tests. At L1-2, L2-3, L3-4, and L4-5, the PM was 6.6, 8.2, 9.4, and 10.2cm on the left side and 7.2, 7.7, 8.8, and 9.5cm on the right side in relation to the disk space center. The SC was less than 1 cm 1%, 3%,3%, and 18% of the time on the left side, and 15%, 12%,29%, and 60% on the right side. None of the anthropometric factors demonstrated a strong correlation with incision location. SC was larger on the left side. Interrater ICC was .934. This study is the first to provide guidelines on the appropriate location of the incision line during OLIF based on SC from L1 to L5. SC measurements do not vary by sex. OLIF is more feasible via a left-sided approach.

Interrater reliability of chart-based assessment of functional impairment after pediatric injury using the functional status scale.

Functional impairment has been proposed as an alternative outcome for quality improvement in pediatric trauma. The functional status scale (FSS) has been used in studies of injured children, but has only been validated with resource-intensive in-person assessment. Implementation with retrospective chart-based FSS assessment would offer a simplified and scalable alternative. The purpose of this study was to evaluate interrater reliability of retrospective FSS assessment and to identify factors associated with unreliable assessment. A retrospective cohort of admissions to a Level I pediatric trauma center between July 2020 and June 2021 was analyzed. Two physicians and two nurse registrars reviewed charts to obtain measures of six FSS domains (mental status, sensory functioning, communication, motor functioning, feeding, and respiratory status) at discharge. Functional impairment was categorized by total FSS scores as good (6,7), mild impairment (8,9), moderate impairment (10-15), severe impairment (16-21), or very severe impairment (>21). Interrater reliability was assessed using intraclass correlation (ICC). Predictors of rater disagreement were evaluated using multivariable logistic regression. The cohort included 443 children with a mean age of 7.4 years (standard deviation, 5.4 years) and median Injury Severity Score of 9 (interquartile range, 5-12). The median time per chart to assess FSS was 2 minutes (interquartile range, 1-2). Thirty-seven patients (8%) had functional impairment at discharge. Interrater reliability was excellent for total FSS score (ICC = 0.87) and good for FSS impairment categorization (ICC = 0.80). Rater disagreement of functional impairment categorization occurred in 14% of cases overall. Higher level of functional impairment and use of therapies (occupational and speech language therapy) were independently associated with more frequent rater disagreement. Chart-based FSS assessment is feasible and reliable, but may require more detailed review for patients with higher level of impairment that require allied health therapy. Validation of chart-based assessment is needed before widespread implementation. Prognostic/Epidemiological, Level III.

Using appropriate Kappa statistic in evaluating inter-rater reliability. Short communication on “Groundwater vulnerability and contamination risk mapping of semi-arid Totko river basin, India using GIS-based DRASTIC model and AHP techniques”

In this article, some misuses of Kappa statistic in the original paper [Chemosphere, 307, 135831] are discussed. By using DRASTIC and Analytic Hierarchy Process (AHP) models, the authors have assessed the groundwater vulnerability of Totko, India. High nitrate concentrations in groundwater have been found in highly vulnerable areas, and the accuracy of the models has been assessed through Pearson's correlation coefficient and Kappa coefficient. However, using Cohen's Kappa to estimate the intra-rater reliabilities (IRRs) of the two models is not appropriate on the condition of ordinal categorical variables in five categories in the original paper. We briefly introduce the Kappa statistic and propose to use weighted Kappa to compute IRRs under such conditions. To conclude, we recognize that this does not significantly alter the conclusions of the original paper, but it is necessary to ensure that the appropriate statistical tools are used.

Reliability of the 44-question Home Fall Hazard Assessment Tool and personal characteristics associated with home hazards among the Thai elderly

Background: The 44-question Thai Home Fall Hazard Assessment Tool (Thai-HFHAT) was developed to assist healthcare professionals in identifying the risk of falls among community-dwelling elderly from their home environment. However, the reliability of this tool has not been studied. This study aimed to examine the reliability of the 44-question Thai-HFHAT and determine the personal characteristics associated with home hazards. Methods: A descriptive cross-sectional study design was used to evaluate interrater reliability. The participants in this study were 51 elderly people from various types of Thai houses: a one-story elevated house, a one-story non-elevated house, and a house with two or more floors, 51 caregivers of elderly patients, and 5 village health volunteers (VHV). A prospective design was used to evaluate test-retest reliability with older people at different times in their homes. All participants answered 44 Thai-HFHAT questions to determine inter-rater and test-retest reliabilities. The reliabilities were analyzed using an intra-class correlation coefficient (ICC). Personal characteristics including sex, occupation, and education were used to identify the factors affecting home hazards, and linear regression was used to analyze. Results: The ICC of inter-rater reliability of the 44-question Thai-HFHAT was 0.74 (95% CI: 0.57-0.84) and the test-retest reliability was 0.80 (95% CI: 0.64-0.88) for the elderly, 0.80 (95% CI: 0.65-0.89) for the caregivers and 0.70 (95% CI: 0.477-0.83) for the VHV. Personal business career and education level grades 1-3 are less than 0.05, which showed these factors had a significant relationship with the 44-question Thai-HFHAT score as the dependent variable. Conclusions: The 44-question Thai-HFHAT is suitable for home hazard assessment among the elderly in Thailand. Further studies are needed to investigate changes in the house environment after using the 44-question Thai-HFHAT to determine which changes can reduce fall risk.

No correlation found between palpation and ultrasound for evaluation of effusion in the medial femorotibial and femoropatellar joint compartments of horses.

To compare palpation and ultrasound scores of effusion of the medial femorotibial and femoropatellar joints of horses. 40 horses (80 stifles) were evaluated over a 12-week period. Horses > 1 year of age without history of stifle disease were enrolled from September to December 2022. Palpation of right and left medial femorotibial and femoropatellar joint compartments was performed. Amount of effusion was scored by a board-certified large animal surgeon, a third-year large animal surgery resident, and an equine sports medicine intern. Effusion of right and left medial femorotibial and femoropatellar joints was quantified with ultrasound by a board-certified equine sports medicine and rehabilitation clinician. Amount of effusion on palpation and ultrasound was graded as none-mild (1), moderate (2), or severe (3). A 2-way intraclass correlation coefficient evaluated interrater reliability of palpation scores. The Spearman rank correlation determined association between palpation and ultrasound scores. Interrater reliability for palpation of effusion was poor between all observers for all joint compartments. No significant correlation was identified between palpation and ultrasound scores for any joint compartment for any observer. Clinicians often rely on palpation of joint effusion as an indication of stifle pathology. We found interrater reliability to be poor for palpation scores, indicating low agreement for palpation of joint effusion between clinicians within our group. No correlation was found between palpation and ultrasound scores for joint effusion, indicating that clinicians should not rely on palpation alone to quantify joint effusion of the medial femorotibial and femoropatellar joints.

INTERRATER RELIABILITY OF AN ADVANCED NONCONTACT WOUND MEASUREMENT SYSTEM USING ARTIFICIAL INTELLIGENCE

Abstract Venous leg ulcers (VLU) are recurring, disabling wounds that develop primarily in older adults with comorbidities. In an ongoing clinical trial testing a nutritional intervention researchers use advanced technology (Swift Skin and Wound) to measure VLU healing over time. This noncontact wound measurement system uses artificial intelligence to automatically calculate wound measurements and track healing progress. However, a subjective component of the Swift system involves delineating wound perimeters on photographs captured with a tablet or smartphone. To evaluate interrater reliability of the system in the current study, measurements of 11 wounds by two independent raters were assessed using an intraclass correlation coefficient (ICC). ICC estimates and their 95% confidence intervals were calculated using SPSS statistical package version 27 based on a mean-rating (k = 2), consistency, 2-way mixed-effects model. For area measures, ICC = 0.99 with 95% confidence interval = 0.998-1.0, indicating the Swift measurement system has excellent interrater reliability.

Validation of a Clinical Decision Rule for Ultrasound Identification of MRSA Skin Abscesses in Children.

The aim of this study was to validate an adult-derived clinical decision rule for ultrasound identification of methicillin-resistant Staphylococcus aureus (MRSA) skin abscesses in a pediatric cohort. We conducted a retrospective study of skin and soft tissue infections in patients <21 years presenting to the emergency department who had radiology performed ultrasounds completed and wound cultures obtained. Ultrasound scans were reviewed for edge definition, volume, and shape by 2 pediatric emergency physicians with expertise in point-of-care ultrasound, with approximately 25% of scans reviewed by both experts to evaluate interrater reliability. A third, blinded expert weighed in for discrepancies before analysis. Test performance characteristics were calculated for the clinical decision rule in children. Two hundred nine patients were enrolled, with mean age of 9.8 (±6.7) years; 87 (42%) were male. Sixty-nine (33%) patients had a wound culture positive for MRSA. The clinical decision rule had a sensitivity of 86% (95% confidence interval [CI], 75%-93%), specificity of 32% (95% CI, 25%-41%), positive predictive value of 38% (95% CI, 35%-42%), negative predictive value of 82% (95% CI, 71%-89%), positive likelihood ratio of 1.26 (95% CI, 1.08-1.46), negative likelihood ratio of 0.45 (95% CI, 0.24-0.84), and an odds ratio of 2.8 (95% CI, 1.31-5.97). This clinical decision rule for ultrasound identification of MRSA abscesses had moderately high sensitivity and negative predictive value in pediatric patients, with similar sensitivity compared with the original adult validation group. Ultrasound may help identify MRSA abscesses, allowing for improved antibiotic choices and outcomes for children with MRSA abscesses.

T2 Turbo Spin Echo With Compressed Sensing and Propeller Acquisition (Sampling k-Space by Utilizing Rotating Blades) for Fast and Motion Robust Prostate MRI: Comparison With Conventional Acquisition.

The aim of this study was to compare a new compressed sensing (CS) method for T2-weighted propeller acquisitions (T2 CS ) with conventional T2-weighted propeller sequences (T2 conv ) in terms of achieving a higher image quality, while reducing the acquisition time. Male participants with a clinical suspicion of prostate cancer were prospectively enrolled and underwent prostate magnetic resonance imaging at 3 T. Axial and sagittal images of the T2 conv sequence and the T2 CS sequence were acquired. Sequences were qualitatively assessed by 2 blinded radiologists concerning artifacts, image-sharpness, lesion conspicuity, capsule delineation, and overall image quality using 5-point Likert items ranging from 1 (nondiagnostic) to 5 (excellent). The apparent signal-to-noise ratio and apparent contrast-to-noise ratio were evaluated. PI-RADS scores were assessed for both sequences. Statistical analysis was performed by using Wilcoxon signed rank test and paired samples t test. Intrarater and interrater reliability of qualitative image evaluation was assessed using intraclass correlation coefficient (ICC) estimates. A total of 29 male participants were included (mean age, 66 ± 8 years). The acquisition time of the T2 CS sequence was respectively 26% (axial plane) and 24% (sagittal plane) shorter compared with the T2 conv sequence (eg, axial: 171 vs 232 seconds; P < 0.001). In the axial plane, the T2 CS sequence had fewer artifacts (4 [4-4.5] vs 4 [3-4]; P < 0.001), better image-sharpness (4 [4-4.5] vs 3 [3-3.5]; P < 0.001), better capsule delineation (4 [3-4] vs 3 [3-3.5]; P < 0.001), and better overall image quality (4 [4-4] vs 4 [3-4]; P < 0.001) compared with the T2 conv sequence. The ratings of lesion conspicuity were similar (4 [4-4] vs 4 [3-4]; P = 0.166). In the sagittal plane, the T2 CS sequence outperformed the T2 conv sequence in the categories artifacts (4 [4-4] vs 3 [3-4]; P < 0.001), image sharpness (4 [4-5] vs 4 [3-4]; P < 0.001), lesion conspicuity (4 [4-4] vs 4 [3-4]; P = 0.002), and overall image quality (4 [4-4] vs 4 [3-4]; P = 0.002). Capsule delineation was similar between both sequences (3 [3-4] vs 3 [3-3]; P = 0.07). Intraobserver and interobserver reliability for qualitative scoring were good (ICC intra: 0.92; ICC inter: 0.86). Quantitative analysis revealed a higher apparent signal-to-noise ratio (eg, axial: 52.2 ± 9.7 vs 22.8 ± 3.6; P < 0.001) and a higher apparent contrast-to-noise ratio (eg, axial: 44.0 ± 9.6 vs 18.6 ± 3.7; P ≤ 0.001) of the T2 CS sequence. PI-RADS scores were the same for both sequences in all participants. CS-accelerated T2-weighted propeller acquisition had a superior image quality compared with conventional T2-weighted propeller sequences while significantly reducing the acquisition time.

The First Tarsometatarsal Joint in Lisfranc Injuries.

Lisfranc injuries are among the most debilitating injuries to the foot. Characterization of first tarsometatarsal (TMT) joint involvement in Lisfranc injuries is limited. Multiple studies have indicated that this joint is damaged in a variety of Lisfranc injury patterns, but there is sparse information regarding how often and in what form. A retrospective review was performed of operative Lisfranc fractures from 2010 to 2020 with patients identified by Combined Procedural Terminology codes. Hardcastle and Myerson Lisfranc injury classifications and computed tomography and radiograph characterizations of the first TMT joint were evaluated by 3 foot and ankle fellowship-trained orthopaedic surgeons. Radiographic characteristics were collected. Light's kappa coefficient evaluated interrater reliability for injury classification. Injury mechanism and Lisfranc classification effects on the first TMT joint were further assessed using inferential statistics. Of 71 patients with a Lisfranc injury of which 37 (52%) were high energy, 61 (86%) showed radiographic evidence of first TMT joint injury. A fragment was present in the TMT articular surface in 33 (47%) with median size = 8.7 mm and medial capsular avulsion in n = 25 (35%). Forty-eight patients (68%) had medial/lateral TMT joint incongruence ≥2 mm (median overhang = 4 mm), 21 (30%) had dorsal/plantar incongruence (median overhang = 6 mm). Angulation of TMT articular surfaces ≥5 degrees on the transverse/anteroposterior plane occurred in n = 32 (45%) and in n = 12 (17%) on the sagittal/lateral plane, which significantly differed between classifications (P = .020). The overwhelming majority of Lisfranc midfoot injuries seen at our tertiary referral center had imaging evidence of damage to the first TMT joint (86%), and the incidence may be higher. The most common patterns of first TMT joint involvement we found were joint incongruity, articular surface fractures, angulation of the articular surfaces, and medial capsular ligament avulsion fractures. A better understanding of injuries to the first TMT joint can help orthopaedic surgeons with diagnosis.