Poor Interobserver Reliability Research Articles

Comparison and interobserver reliability of three different methods for measuring the amount of the tibial tuberosity advancement in the preoperative planning of TTAT.

The goal of preoperative planning techniques for advancement of the tibial tuberosity is to determine the amount of advancement required to achieve a postoperative patellar tendon angle of 90° and to select the optimal wedge size to achieve this target. Three radiographic methods for determining the advancement distance for the tibial tuberosity were evaluated for comparability and interobserver reliability. Among the methods developed, we decided to include the common tangent method, the tibial anatomy-based method, and the Bielecki method. For all techniques, radiographs were taken in mediolateral projection with the knee joint flexed at 135°. Three observers with different levels of experience independently evaluated and scored the degree of osteoarthritis for each stifle, as previously described, and performed measurements of the amount of advancement of the tibial tuberosity on 33 stifles using common tangent method, tibial anatomy-based method, and Bielecki method. According to the results, the overall score for osteoarthritis in the mediolateral view was influenced by the experience of the observers, which contradicts the results from a previous study. Regarding the measurement methods used to assess advancement, poor interobserver reliability was found for common tangent method and Bielecki method, while only a slightly moderate interobserver agreement was found for tibial anatomy-based method. These results are inconsistent with data collected by Bielecki and colleagues. Moreover, measurements from common tangent method and tibial anatomy-based method were overlapping, as confirmed previously. Conversely, Bielecki method showed no agreement with the other methods included in the present study, with a significantly higher mean rank, probably due to its correction formula. Based on the results of the present study, tibial anatomy-based method has better interobserver reliability and is easier to perform according to the observers.

Proximal humerus fractures: A review of current practice

The management of proximal humeral fractures (PHF) remains controversial. Its incidence is increasing. Patients should be meticulously assessed clinically for co-morbidities and neuro-vascular injuries. Radiological investigation helps provide information on the fracture configuration and dislocations. Enhanced by 3-dimensional CT scanning, these further help in decision making and operative planning. PHF classifications have been demonstrated to have poor intra-observer and inter-observer reliability. Research has identified some radiographic predictive factors for humeral head ischaemia and likely failure of surgical fixation. The range of management options include non-operative treatment, operative fixation, intramedullary nailing and arthroplasty (hemiarthroplasty, reverse shoulder replacement).The majority of PHFs are stable injuries and non-operative management is usually successful. Some degree of malunion is readily tolerated especially by elderly patients. Surgical management of significantly displaced, unstable proximal humerus fractures should aim to stabilise the fracture adequately and provide satisfactory function for the long term. Management of the greater tuberosity is pivotal for the eventual outcome. When fixation may appear to be compromised by poor bone quality, likely poor function, age related rotator cuff degeneration or likely humeral head ischaemia clinicians may opt for arthroplasty. Successful hemiarthroplasty outcomes are dependent on sufficient healing of the tuberosity and recovery of the rotator cuff integrity. Reverse shoulder replacement can predictably deliver good functional outcomes for the shoulder in elderly patients, where rotator cuff dysfunction is suspected or as a revision procedure following failure of other surgical interventions. As opposed to hemiarthroplasty, which has shown a downward trend, there has been an increasing trend towards the use of reverse shoulder replacement in proximal humeral fractures. The management of PHFs should be patient specific, fracture specific and meet the functional demands and needs of the individual patient. The surgeon's skill set and clinical experience also plays an important role in the options of management available.

The Reliability of Measurements for Tibial Torsion: A Comparison of CT, MRI, Biplanar Radiography, and 3D Reconstructions With and Without Standardized Measurement Training

Background: Accurate and reliable assessment of tibial torsion is important for the identification and treatment of tibial rotation malalignment; however, the ideal rotational measurement modality and technique are controversial. This study compares rotational measurements between computed tomography (CT), magnetic resonance imaging (MRI), biplanar erect radiograph (BER) reconstructions, and three-dimensional (3D) reconstructed CT before and after standardized training of observers to evaluate the reliability of each assessment method.Methods: Eight adult cadaveric specimens underwent CT, MRI, and BER imaging. Tibial torsion was measured by three independent observers (one resident and two experienced orthopaedic surgeons) both before and after standardized measurement instruction. Reliability for inter-observer measurement was evaluated using the Intraclass Correlation Coefficient (ICC). Measurement values for CT, MRI, and BER reconstructions were compared to 3D CT reconstructions analyzed using custom software to align and measure tibial torsion (used as the reference standard).Results: Before training, there was poor inter-observer reliability for CT (ICC=0.492, p=0.014) and moderate interobserver reliability for MRI (ICC=0.633, p=0.002). There was no inter-method reliability between 3D CT and MRI for two of the three surgeons and moderate to good reliability between 3D CT and CT. After training, the interobserver reliability for CT improved to 0.536 and the inter-observer reliability for MRI improved to 0.701. The BER measurements (no observer involvement) had moderate reliability compared to the 3D CT reconstructions (ICC=0.69, p=0.026). Measurement error was 4 degrees for CT pre-training and 7 degrees post-training, and 7 degrees for MRI pre-training and 8 degrees post-training.Conclusions: A standardized training regimen for MRI measurements improved both inter-observer and intraobserver reliability. Inter-method reliability between CT, MRI, and BER compared to reference 3D CT reconstructions demonstrated that all imaging modalities are a valid means to measure tibial rotation but differ in reliability from moderate to good. When assessing tibial torsional deformities, it is important to consider these variations from true rotation and feel comfortable using them for pre-operative planning purposes.Level of Evidence: N/A for a cadaveric imaging study Key Concepts•A standardized training regimen for MRI measurements of tibial torsion significantly improved inter-observer and intra-observer reliability.•CT, MRI, and BER measurements are valid imaging modalities compared to 3D CT reference standards for measuring tibial rotation.•Established manual measurements of tibial torsion from MRI may be improved with adequate training.•Measurement error of up to 8 degrees can be seen from true tibial torsion during the imaging assessment.

Automated Society of Fetal Urology (SFU) grading of hydronephrosis on ultrasound imaging using a convolutional neural network

Grading of hydronephrosis severity on postnatal renal ultrasound guides management decisions in antenatal hydronephrosis (ANH). Multiple systems exist to help standardize hydronephrosis grading, yet poor inter-observer reliability persists. Machine learning methods may provide tools to improve the efficiency and accuracy of hydronephrosis grading. To develop an automated convolutional neural network (CNN) model to classify hydronephrosis on renal ultrasound imaging according to the Society of Fetal Urology (SFU) system as potential clinical adjunct. A cross-sectional, single-institution cohort of postnatal renal ultrasounds with radiologist SFU grading from pediatric patients with and without hydronephrosis of stable severity was obtained. Imaging labels were used to automatedly select sagittal and transverse grey-scale renal images from all available studies from each patient. A VGG16 pre-trained ImageNet CNN model analyzed these preprocessed images. Three-fold stratified cross-validation was used to build and evaluate the model that was used to classify renal ultrasounds on a per patient basis into five classes based on the SFU system (normal, SFU I, SFU II, SFU III, or SFU IV). These predictions were compared to radiologist grading. Confusion matrices evaluated model performance. Gradient class activation mapping demonstrated imaging features driving model predictions. We identified 710 patients with 4659 postnatal renal ultrasound series. Per radiologist grading, 183 were normal, 157 were SFU I, 132 were SFU II, 100 were SFU III, and 138 were SFU IV. The machine learning model predicted hydronephrosis grade with 82.0% (95% CI: 75-83%) overall accuracy and classified 97.6% (95% CI: 95-98%) of the patients correctly or within one grade of the radiologist grade. The model classified 92.3% (95% CI: 86-95%) normal, 73.2% (95% CI: 69-76%) SFU I, 73.5% (95% CI: 67-75%) SFU II, 79.0% (95% CI: 73-82%) SFU III, and 88.4% (95% CI: 85-92%) SFU IV patients accurately. Gradient class activation mapping demonstrated that the ultrasound appearance of the renal collecting system drove the model's predictions. The CNN-based model classified hydronephrosis on renal ultrasounds automatically and accurately based on the expected imaging features in the SFU system. Compared to prior studies, the model functioned more automatically with greater accuracy. Limitations include the retrospective, relatively small cohort, and averaging across multiple imaging studies per patient. An automated CNN-based system classified hydronephrosis on renal ultrasounds according to the SFU system with promising accuracy based on appropriate imaging features. These findings suggest a possible adjunctive role for machine learning systems in the grading of ANH.

Reliability of MRI Interpretation of Discoid Lateral Meniscus: A Multicenter Study.

Discoid lateral meniscus (DLM) has a varied and complex morphology that can be challenging to assess and treat. Preoperative magnetic resonance imaging (MRI) is frequently used for diagnosis and surgical planning; however, it is not known whether surgeons are reliable and accurate in their interpretation of MRI findings when defining the pathomorphology of DLM. Surgeons experienced in treating DLM are able to reliably interpret DLM pathology using MRI. Cohort study (diagnosis); Level of evidence, 3. Knee MRI scans from 44 patients (45 knees) were selected from a pool of surgically treated patients with DLM. Five reviewers (fellowship-trained pediatric sports medicine surgeons) performed independent review of each MRI scan using the PRiSM Discoid Meniscus Classification. Inter- and intraobserver reliability of the rating factors-primary (width, height, presence of peripheral instability or tear) and secondary (location of instability or tear, tear type)-was assessed using the Fleiss κ coefficient, designed for multiple readers with nominal variables (fair reliability, 0.21-0.40; moderate, 0.41-0.60; substantial, 0.61-0.80; excellent, 0.81-1.00). Reliability is reported as κ (95% CI). Interobserver reliability in assessing most primary and secondary characteristics ranged from substantial (meniscal width) to moderate (peripheral instability, anterior instability, posterior instability, and posterior tear). Intraobserver reliability for most characteristics ranged from substantial (peripheral instability, presence of tear, anterior instability, posterior instability, and posterior tear) to moderate (meniscal width, anterior tear, and tear type). Notable exceptions were presence of tear, anterior tear, and tear type-all with fair interobserver reliability. Height had poor interobserver reliability and fair intraobserver reliability. Orthopaedic surgeons reliably interpret MRI scans using the PRiSM Discoid Meniscus Classification for the majority of DLM characteristics but vary in their assessment of height and presence and type of tear. MRI evaluation may be helpful to diagnose discoid by width and identify the presence of instability: 2 major factors in the decision to proceed with surgery. Arthroscopic evaluation should be used in conjunction with MRI findings for complete DLM diagnosis.

MP18-20 AUTOMATED URINARY TRACT DILATION GRADING OF HYDRONEPHROSIS ON ULTRASOUND IMAGING USING A CONVOLUTIONAL NEURAL NETWORK

You have accessJournal of UrologyCME1 Apr 2023MP18-20 AUTOMATED URINARY TRACT DILATION GRADING OF HYDRONEPHROSIS ON ULTRASOUND IMAGING USING A CONVOLUTIONAL NEURAL NETWORK David Ostrowski, Joseph Logan, Maria Antony, Reilly Broms, Dana Weiss, Jason van Batavia, Christopher Long, Ariana Smith, Stephen Zderic, Rebecca Edwins, Raymond Pominville, Jessica Hannick, Lynn Woo, Yong Fan, Gregory Tasian, and John Weaver David OstrowskiDavid Ostrowski More articles by this author , Joseph LoganJoseph Logan More articles by this author , Maria AntonyMaria Antony More articles by this author , Reilly BromsReilly Broms More articles by this author , Dana WeissDana Weiss More articles by this author , Jason van BataviaJason van Batavia More articles by this author , Christopher LongChristopher Long More articles by this author , Ariana SmithAriana Smith More articles by this author , Stephen ZdericStephen Zderic More articles by this author , Rebecca EdwinsRebecca Edwins More articles by this author , Raymond PominvilleRaymond Pominville More articles by this author , Jessica HannickJessica Hannick More articles by this author , Lynn WooLynn Woo More articles by this author , Yong FanYong Fan More articles by this author , Gregory TasianGregory Tasian More articles by this author , and John WeaverJohn Weaver More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000003238.20AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: The grading of antenatal hydronephrosis (ANH) on postnatal renal ultrasounds dictates management of patients with ANH. Multiple grading systems attempt to standardize hydronephrosis grading, yet poor inter-observer reliability persists. Machine learning methods may provide tools to improve accuracy and efficiency of hydronephrosis grading. We sought to develop an automated convolutional neural network (CNN) model to classify renal ultrasounds according to the Urinary Tract Dilation (UTD) system as an adjunct for hydronephrosis grading. METHODS: A retrospective, single-institution cohort of postnatal renal ultrasounds from patients with and without hydronephrosis with radiologist UTD grading was obtained. An algorithm used image labels to select sagittal renal images from each study. These preprocessed images were analyzed by a VGG16 pre-trained ImageNet CNN model. 3K-fold stratified cross-validation was used to predict on the dataset. The model classified images into five UTD classes (normal, mild hydronephrosis not meeting criteria for P1, P1, P2, or P3). These predictions were compared to radiologist grading. Confusion matrices evaluated model performance. Gradient class activation mapping demonstrated image features driving model predictions. RESULTS: We identified 610 patients with 2,828 renal ultrasound series. Per radiologist grading, 847 were normal, 601 were mild hydronephrosis, 336 were P1, 662 were P2, and 442 were P3. The model predicted hydronephrosis grade with 79% overall accuracy and classified 97% of the studies correctly or within one grade of the radiologist grade. The model classified 94% of normal studies, 74% of mild hydronephrosis studies, 68% of P1 ultrasounds, 74% of P2 ultrasounds, and 90% of P3 ultrasounds accurately. Gradient class activation mapping demonstrated the renal collecting system drove the model’s predictions. CONCLUSIONS: An automated CNN model classified hydronephrosis on renal ultrasounds by the UTD system with promising accuracy compared to prior studies. The model graded images based on the UTD system using appropriate imaging features. These findings suggest a possible adjunctive role for machine learning systems in the grading of ANH. Source of Funding: Pennsylvania Department of Health Tobacco Grant Funding Agency Number SAP #4100085749 © 2023 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 209Issue Supplement 4April 2023Page: e231 Advertisement Copyright & Permissions© 2023 by American Urological Association Education and Research, Inc.MetricsAuthor Information David Ostrowski More articles by this author Joseph Logan More articles by this author Maria Antony More articles by this author Reilly Broms More articles by this author Dana Weiss More articles by this author Jason van Batavia More articles by this author Christopher Long More articles by this author Ariana Smith More articles by this author Stephen Zderic More articles by this author Rebecca Edwins More articles by this author Raymond Pominville More articles by this author Jessica Hannick More articles by this author Lynn Woo More articles by this author Yong Fan More articles by this author Gregory Tasian More articles by this author John Weaver More articles by this author Expand All Advertisement PDF downloadLoading ...

Recall patterns and risk of primary liver cancer for subcentimeter ultrasound liver observations: a multicenter study

Patients with cirrhosis and subcentimeter lesions on liver ultrasound are recommended to undergo short-interval follow-up ultrasound because of the presumed low risk of primary liver cancer (PLC). The aim of this study is to characterize recall patterns and risk of PLC in patients with subcentimeter liver lesions on ultrasound. We conducted a multicenter retrospective cohort study among patients with cirrhosis or chronic hepatitis B infection who had subcentimeter ultrasound lesions between January 2017 and December 2019. We excluded patients with a history of PLC or concomitant lesions ≥1cm in diameter. We used Kaplan Meier and multivariable Cox regression analyses to characterize time-to-PLC and factors associated with PLC, respectively. Of 746 eligible patients, most (66.0%) had a single observation, and the median diameter was 0.7cm (interquartile range: 0.5-0.8cm). Recall strategies varied, with only 27.8% of patients undergoing guideline-concordant ultrasound within 3-6 months. Over a median follow-up of 26 months, 42 patients developed PLC (39 HCC and 3 cholangiocarcinoma), yielding an incidence of 25.7 cases (95% CI, 6.2-47.0) per 1000 person-years, with 3.9% and 6.7% developing PLC at 2 and 3 years, respectively. Factors associated with time-to-PLC were baseline alpha-fetoprotein >10ng/mL (HR: 4.01, 95% CI, 1.85-8.71), platelet count ≤150 (HR: 4.90, 95% CI, 1.95-12.28), and Child-Pugh B cirrhosis (vs. Child-Pugh A: HR: 2.54, 95% CI, 1.27-5.08). Recall patterns for patients with subcentimeter liver lesions on ultrasound varied widely. The low risk of PLC in these patients supports short-interval ultrasound in 3-6 months, although diagnostic CT/MRI may be warranted for high-risk subgroups such as those with elevated alpha-fetoprotein levels.

What is the angle of a banana? The difficulty in reliable assessment of hypospadias chordee

The degree of chordee associated with hypospadias impacts operative management. Unfortunately, poor inter-observer reliability in assessing chordee by multiple methods invitro has been demonstrated. This variability may be related to the fact that chordee is not a discrete angle, but rather an arc-like curvature similar to that of a banana. On an attempt to improve this variability, we assessed the inter-rater reliability of a novel method of chordee measurement and compared it to measurements with a goniometer both invitro and invivo. Invitro assessment of curvature was performed using 5 bananas. Invivo chordee measurement was performed during 43 hypospadias repairs. On invitro and invivo cases, chordee was assessed independently by faculty and resident physicians. Angle assessment was performed in a standard manner with a goniometer and with a smartphone app using ruler measurements of the length and width of the arc (Summary Figure). The proximal and distal aspect of the arc to be measured was marked on the bananas, whereas the penile measurements were taken from the penoscrotal to the sub-coronal junctions. Invitro banana assessment demonstrated strong intra- and inter-rater reliability for length (0.89 and 0.88, respectively) and width measurements (0.97 and 0.96). The calculated angle demonstrated an intra- and inter-rater reliability of 0.67 and 0.67. The banana goniometer/protractor measurements were weak with an intra-rater and inter-rater reliability of 0.33 and 0.21. With hypospadias chordee, the inter-rater reliability was strong for length and width measurements (0.95 and 0.94) and 0.48 for calculated angle. The inter-rater reliability of the goniometer angle was 0.96. Further assessment of inter-rater goniometer reliability was performed relative to degree of chordee as characterized by faculty. The inter-rater reliability for ≤15°, 16-30, and ≥30° was 0.68 (n=20), 0.34 (n=14), and 0.90 (n=9), respectively. When the goniometer angle was classified as ≤15, 16-30, or ≥30° by one physician, it was classified outside of this range by the other physician 23%, 47%, and 25% of the time, respectively. Our data demonstrate significant limitations of the goniometer for assessing chordee invitro and invivo. We were unable to demonstrate significant improvement in chordee assessment using arc length and width measurements to calculate radians. Reliable and precise techniques for measuring hypospadias chordee remain elusive and draw into question the validity and usability of management algorithms employing discrete values.

Computed tomography improves the diagnostic accuracy but not the interobserver reliability of the Boileau classification of proximal humerus fracture sequelae.

The aim of this study was to investigate the impact of computed tomography on the reliability of Boileau classification for proximal humerus fracture sequelae. A prospective study was designed using STARD guidelines. We included all patients diagnosed with proximal humerus fracture sequelae who underwent surgery at our institution between 2017 and 2021. Preoperative radiographs and computed tomography scans were reviewed by three independent observers. Intra- and inter-observer reliability and the diagnostic accuracy of radiographs and computed tomography scans in detecting chronic dislocation, nonunion, and severe greater tuberosity dislocation were assessed. Fifty-two patients were included in the study. The overall interobserver agreement was low on both radiographs and computed tomography scans. On radiographic images, we found a sensitivity of 97%, 88.9%, and 84.1%, and a specificity of 58.3%, 40%, and 53.3% to detect chronic dislocation, nonunion, and greater tuberosity dislocation, respectively. On computed tomography scans we reported a sensitivity of 100%, 96.8%, and 93.7%, and a specificity of 91.7%, 86.7%, and 93.3% to detect chronic dislocation, nonunion, and greater tuberosity dislocation, respectively. Computed tomography scan was more specific than radiographs in the assessment of proximal humerus fracture sequelae. However, even using a three-dimensional evaluation of the deformity, the Boileau classification had a poor interobserver reliability. I. Testing previously developed diagnostic criteria in a consecutive series of patients and a universally applied "gold standard."

Clinical effectiveness of the Ganga Hospital Open Injury Severity Score for limb salvage versus amputation in patients with complex limb injuries : a systematic review and meta-analysis.

Clinical management of open fractures is challenging and frequently requires complex reconstruction procedures. The Gustilo-Anderson classification lacks uniform interpretation, has poor interobserver reliability, and fails to account for injuries to musculotendinous units and bone. The Ganga Hospital Open Injury Severity Score (GHOISS) was designed to address these concerns. The major aim of this review was to ascertain the evidence available on accuracy of the GHOISS in predicting successful limb salvage in patients with mangled limbs. We searched electronic data bases including PubMed, CENTRAL, EMBASE, CINAHL, Scopus, and Web of Science to identify studies that employed the GHOISS risk tool in managing complex limb injuries published from April 2006, when the score was introduced, until April 2021. Primary outcome was the measured sensitivity and specificity of the GHOISS risk tool for predicting amputation at a specified threshold score. Secondary outcomes included length of stay, need for plastic surgery, deep infection rate, time to fracture union, and functional outcome measures. Diagnostic test accuracy meta-analysis was performed using a random effects bivariate binomial model. We identified 1,304 records, of which six prospective cohort studies and two retrospective cohort studies evaluating a total of 788 patients were deemed eligible for inclusion. A diagnostic test meta-analysis conducted on five cohort studies, with 474 participants, showed that GHOISS at a threshold score of 14 has a pooled sensitivity of 93.4% (95% confidence interval (CI) 78.4 to 98.2) and a specificity of 95% (95% CI 88.7 to 97.9) for predicting primary or secondary amputations in people with complex lower limb injuries. GHOISS is highly accurate in predicting success of limb salvage, and can inform management and predict secondary outcomes. However, there is a need for high-quality multicentre trials to confirm these findings and investigate the effectiveness of the score in children, and in predicting secondary amputations.Cite this article: Bone Joint J2023;105-B(1):21-28.

Computed tomography for ventral hernia: Need for a standardised reporting format.

Computed tomography for ventral hernia: Need for a standardised reporting format.

Reliability of Bigliani's Classification using Magnetic Resonance Imaging for Determination of Acromial Morphology.

Bigliani classification is used for determination of acromial morphology, but poor inter-observer reliability has been reported on conventional radiographs. This study aims to assess inter- and intra-observer reliability using magnetic resonance imaging (MRI). Forty consecutive patients diagnosed with subacromial impingement syndrome were included to study. All subjects underwent standard shoulder MRI scan and acromial shape was evaluated by nine observers of different level of expertise (three attending surgeons, three senior orthopaedic residents and three radiologists). A second set of evaluation was performed in order to assess intra-observer reproducibility. Kappa (κ) coefficient analyses both for interobserver reliability and intra-observer reproducibility were then performed. Overall inter-observer agreement among nine observers was fair (κ=0.323). κ values for all 4 individual types ranged from 0.234 to 0.720 with highest agreement for type 4 and lowest agreement for type 3. Second evaluation did not result with an increase of inter-observer agreement (κ=0.338, fair). The κ coefficients for intra-observer reproducibility of nine observers ranged from 0.496 to 0.867. Overall intra-observer reproducibility was substantial. Comparison of inter- and intra-observer reliability among three groups showed no significant difference (p=0.92 and 0.22, respectively). Results showed that MRI did not show superior reliability compared to conventional radiographs. Moreover, inter- and intra-observer agreement did not differ between observers of different level of expertise. Findings of present study suggest that despite a sophisticated imaging modality like MRI, Bigliani's classification apparently lacks accuracy and additional criteria, or different assessment methods are required to assess acromial morphology for clinical guidance.

Improvement of an interobserver agreement of ARDS diagnosis by adding additional imaging and a confidence scale.

Acute respiratory distress syndrome (ARDS) often is not recognized in clinical practice, largely due to variation in the interpretation of chest x-ray (CXR) leading to poor interobserver reliability. We hypothesized that the agreement in the interpretation of chest imaging for the diagnosis of ARDS in invasively ventilated intensive care unit patients between experts improves when using an 8-grade confidence scale compared to using a dichotomous assessment and that the agreement increases after adding chest computed tomography (CT) or lung ultrasound (LUS) to CXR. Three experts scored ARDS according to the Berlin definition based on case records from an observational cohort study using a dichotomous assessment and an 8-grade confidence scale. The intraclass correlation (ICC), imaging modality, and the scoring method were calculated per day and compared using bootstrapping. A consensus judgement on the presence of ARDS was based on the combined confidence grades of the experts, followed by a consensus meeting for conflicting scores. In total, 401 patients were included in the analysis. The best ICC was found using an 8-grade confidence scale for LUS (ICC: 0.49; 95%-CI: 0.29–0.63) and CT evaluation (ICC: 0.49; 95%-CI: 0.34–0.61). The ICC of CXR increased by 0.022 and of CT by 0.065 when 8-grade scoring was used instead of the dichotomous assessment. Adding information from LUS or chest CT increased the ICC by 0.25 when using the 8-grade confidence assessment. An agreement on the diagnosis of ARDS can increase substantially by adapting the scoring system from a dichotomous assessment to an 8-grade confidence scale and by adding additional imaging modalities such as LUS or chest CT. This suggests that a simple assessment of the diagnosis of ARDS with a chart review by one assessor is insufficient to define ARDS in future studies.Clinical trial registrationTrialregister.nl (identifier NL8226).

Poster 111: Poor Inter-observer Reliability of TT-TG Measurement Among Orthopaedic Sports Medicine Surgeons Regardless of Measurement Technique

Objectives:Tibial tubercle-trochlear groove distance (TT-TG) is an important radiologic parameter in the diagnosis and treatment of patellar instability. However, there is substantial heterogeneity in the literature regarding the most appropriate measurement protocol for assessing TT-TG distance. The objective of this study was to evaluate the inter- and intra-observer reliability of TT-TG distance measurement in healthy knees performed by fellowship-trained orthopaedic sports medicine surgeons.Methods:A convenience sample of 29 magnetic resonance imaging scans (MRIs) of the knee was assembled after screening to ensure there was no ligamentous rupture, meniscal pathology, cartilage injury, subchondral edema, or abnormal masses in the bone or muscle. An a priori power analysis determined that this number of MRIs would provide greater than 90% power to detect a statistically significant difference in intraclass correlation of 0.95 (a < 0.05). Three fellowship-trained orthopaedic sports medicine surgeons read the MRIs and recorded the TT-TG distance using their method of choice. After a minimum of 5 months, the MRIs were read again, this time using a standardized evidence-based method to measure the TT-TG distance. Kappa statistics were calculated to evaluate inter- and intra-observer reliability.Results:Among the initial TT-TG distance measurements, kappa was equal to -0.006 (P = 0.690) when assessing inter-observer reliability. Introduction of the standardized measurement protocol was associated with no significant change in inter-observer reliability (kappa = 0.124, P = 0.070). The individual TT slice that was selected for TT-TG distance measurement demonstrated a kappa statistic of 0.192 (P < 0.001) for the surgeons’ method of choice and a kappa statistic of 0.037 (P = 0.455) for the standardized method. Similarly, TG slice selection using the surgeons’ method of choice yielded a kappa statistic of -0.005 (P = 0.949) and a kappa statistic of -0.102 (P = 0.140) for the standardized method.Conclusions:Our findings demonstrate poor inter-observer reliability of TT-TG distance measurement in healthy knees that was not improved with the application of a standardized evidence-based measurement protocol. TT and TG slices selected by surgeons varied widely. Future research should focus on comparing TT-TG distance measurements across a wider variety of knees (ranging from healthy to those with patellar instability), as this may improve the inter-observer reliability of TT-TG measurement.Table I.Variation in TT-TG Distance Measurement and MRI Slice Selection Between Surgeons

Reliability and validity of Vancouver Scar Scale and Withey score after syndactyly release.

This study aimed to analyze the reliability and validity of the Vancouver Scar Scale (VSS) and the Withey score after syndactyly release. Over a 3-year period, 13 patients who underwent syndactyly release were evaluated. The mean age at the time of syndactyly release was 12 months (range, 8-18 months), and the mean follow-up period was 29 months (range, 17-52 months). We obtained hand photographs and finger motion videos and collected the satisfaction scores for hand function and cosmesis. Three clinicians evaluated the hand photographs and finger motion video of each patient twice using the VSS and the Withey score. The interobserver and intraobserver reliabilities of the VSS and Withey score were determined using intraclass correlation coefficients (ICCs). The validity of the VSS and Withey score was determined using Spearman's correlation test with the functional and cosmetic satisfaction score. The ICCs for the interobserver reliability of VSS were 0.31 and 0.39 for each measurement, and ICCs for the intraobserver reliability of VSS were 0.46, 0.51, and 0.54 for each observer. The ICCs for the interobserver reliability of the Withey score were 0.74 and 0.70, and the ICCs for the intraobserver reliability of the Withey score were 0.91, 0.74, and 0.96. The Withey score was significantly correlated with the satisfaction score for hand function and hand cosmesis, but the VSS was not. The VSS had poor interobserver reliability and fair intraobserver reliability, whereas the Withey score had good interobserver reliability and excellent intraobserver reliability based on photographic evaluation after syndactyly release.

Need for syndesmotic fixation and assessment of reduction during ankle fracture fixation, with and without contralateral fluoroscopic images, has poor interobserver reliability.

To determine the interobserver reliability of syndesmosis assessment using intraoperative ankle mortise fluoroscopic images, with and without contralateral images. A survey of 19 operative ankle fracture cases was administered to 17 orthopedic surgeons. Respondents were presented with fluoroscopic mortise and stress images of the ankle after fracture fixation and asked if they would fix the syndesmosis. Final fluoroscopic mortise images were then shown, and respondents were asked to assess the reduction of the syndesmosis. Six weeks later, the survey was administered again with the addition of contralateral fluoroscopic ankle mortise images. Responses were compared to a standard response agreed upon by fellowship-trained orthopedic trauma surgeons. Interobserver reliability for syndesmosis fixation and reduction, with and without contralateral images, was considered weak (kappa 0.48 and 0.43; mean difference 0.05, 95% confidence interval (CI) 0.01 to 0.1) and minimal (kappa 0.25 and 0.22; mean difference 0.02, CI - 0.02 to 0.08). With the addition of contralateral mortise images, the number of surgeons who changed their response for syndesmosis fixation and reduction quality ranged from 0% to 41% and 0% to 88%; with the number of responses matching the standard increasing for both fixation (proportional difference (PD) 7%, CI 1% to 14%) and reduction (PD 14%, CI 7% to 21%); CONCLUSIONS: Interobserver reliability of syndesmosis fixation and reduction remained weak to minimal between surgeons, with and without contralateral images. Future studies are necessary to understand the variability in surgeon responses in order to improve the intraoperative assessment and fixation of syndesmotic injuries.

Objective Assessment of Chronic Pain in Horses Using the Horse Chronic Pain Scale (HCPS): A Scale-Construction Study.

Simple SummaryThe objective assessment of acute pain in horses can be performed by means of different types of pain scales. To date, no studies have been conducted to develop a structured pain measuring tool for the assessment of chronic pain in horses. In donkeys, a pain scale incorporating behavioural and facial expression-based parameters has been shown to be of added value in assessing chronic pain. In this study, we used the same concept to develop a pain scale (the Horse Chronic Pain Scale; HCPS) to measure chronic pain in horses, based on behavioural parameters and facial expressions. This pain scale was tested in 26 horses with different types of chronic pain (osteoarthritis, chronic laminitis, chronic back and neck problems, chronic dental disorders) and 27 healthy control animals. The authors found the HCPS to be a reproducible pain scale and useful to assess chronic pain in horses. Further studies are needed to confirm these findings in other horses and to validate its use in subsets of horses with specific chronic pain states.The objective assessment of chronic pain is of utmost importance for improving welfare and quality of life in horses. Freedom from disease and pain is one of the ‘five freedoms’ that are necessary for animal welfare. The aim of this study was to develop a pain scale for the assessment of chronic pain in horses (Horse Chronic Pain Scale; HCPS), which is based on behavioural and facial expressions. The scale was used to assess 53 horses (26 horses diagnosed with chronic painful conditions by means of clinical examination and additional diagnostic procedures (consisting of osteoarthritis, chronic laminitis, chronic back and neck problems, chronic dental disorders) and 27 healthy control animals). Animals were assessed once daily for three consecutive days by two observers that were blinded to the condition of the animals and were unaware of any analgesic treatment regimens. The HCPS consists of two parts, the Horse Chronic Pain Composite Pain Scale (HCP CPS, with behavioural parameters) and the EQUUS-FAP (Equine Utrecht University Scale for Facial Assessment of Pain). The HCP CPS had good inter-observer reliability (intraclass correlation coefficient (ICC) = 0.84, p < 0.001), while the EQUUS-FAP component (with facial expression-based parameters) had poor inter-observer reliability (ICC = 0.45, p < 0.05). The inter-observer reliability of the combined HCPS was good (ICC = 0.78, p < 0.001). The HCPS revealed significant differences between horses with chronic painful conditions and control horses on 2 out of 3 days (p < 0.05). In conclusion, we tested a composite pain scale for the assessment of chronic pain in horses based on behavioural and facial expression-based parameters. Further studies are needed to validate this pain scale before it can be used in practice.

Accuracy of the Radiographic Assessment of Lung Edema Score for the Diagnosis of ARDS.

Background: Bilateral opacities on chest radiographs are part of the Berlin Definition for Acute Respiratory Distress Syndrome (ARDS) but have poor interobserver reliability. The “Radiographic Assessment of Lung Edema” (RALE) score was recently proposed for evaluation of the extent and density of alveolar opacities on chest radiographs of ARDS patients. The current study determined the accuracy of the RALE score for the diagnosis and the prognosis of ARDS.Methods: Post-hoc analysis of a cohort of invasively ventilated intensive care unit (ICU) patients expected to need invasive ventilation for >24 h. The Berlin Definition was used as the gold standard. The RALE score was calculated for the first available chest radiograph after start of ventilation in the ICU. The primary endpoint was the diagnostic accuracy for ARDS of the RALE score. Secondary endpoints included the prognostic value of the RALE score for ICU and hospital mortality, and the association with ARDS severity, and the PaO2/FiO2. Receiver operating characteristic (ROC) curves were constructed, and the optimal cutoff was used to determine sensitivity, specificity and the negative and positive predictive value of the RALE score for ARDS.Results: The study included 131 patients, of whom 30 had ARDS (11 mild, 15 moderate, and 4 severe ARDS). The first available chest radiograph was obtained median 0 [0 to 1] days after start of invasive ventilation in ICU. Compared to patients without ARDS, a higher RALE score was found in patients with ARDS (24 [interquartile range (IQR) 16–30] vs. 6 [IQR 3–11]; P < 0.001), with RALE scores of 20 [IQR 14–24], 26 [IQR 16–32], and 32 [IQR 19–36] for mild, moderate and severe ARDS, respectively, (P = 0.166). The area under the ROC for ARDS was excellent (0.91 [0.86–0.96]). The best cutoff for ARDS diagnosis was 10 with 100% sensitivity, 71% specificity, 51% positive predictive value and 100% negative predictive value. The RALE score was not associated with ICU or hospital mortality, and weakly correlated with the PaO2/FiO2.Conclusion: In this cohort of invasively ventilated ICU patients, the RALE score had excellent diagnostic accuracy for ARDS.

A Comparison of Interobserver Reliability Between Orthopedic Surgeons Using the Centers for Disease Control Surgical Wound Class Definitions.

The Centers for Disease Control (CDC) created a classification to help stratify surgical wounds based on contamination and risk of developing a surgical site infection. The classification includes four options (I to IV) depending on the level of contamination present. Although universally applied to a variety of surgical specialties, it is unknown whether the current system is reliable when considering orthopaedic surgeries. The purpose of this study was to compare the degree of interobserver reliability between orthopaedic surgeons using the current CDC wound class definitions. A questionnaire containing 30 clinical vignettes was completed by 39 orthopaedic surgeons at our institution. After each vignette, respondents were asked to determine the appropriate wound class based on information provided in the vignette. The overall interobserver agreement among all participants was analyzed. In addition, respondents were queried about the adequacy of the current classification system in describing orthopaedic surgical wound class. Interobserver agreement was poor at 66%, with a coefficient of concordance of 0.48. Only six physicians (15.4%) thought that the current wound classification system adequately covered orthopaedic surgery. There is poor interobserver reliability using the CDC surgical wound class definitions for orthopaedic surgeries. Alternate definitions are needed to improve the validity of the system for subspecialty procedures.

Reduction techniques for intramedullary nailing of tibial shaft fractures: a comparative study.

Objectives:To assess the impact of various reduction techniques on postoperative alignment following intramedullary nail (IMN) fixation of tibial shaft fractures.Design:Retrospective comparative study.Setting:Level I trauma center.Patients:Four hundred twenty-eight adult patients who underwent IMN fixation of a tibial shaft fracture between 2008 and 2017.Intervention:IMN fixation with use of one or more of the following reduction techniques: manual reduction, traveling traction, percutaneous clamps, provisional plating, or blocking screws.Main outcome measures:Immediate postoperative coronal and sagittal plane alignment, measured as deviation from anatomic axis (DFAA); coronal and sagittal plane malalignment (defined as DFAA >5° in either plane).Results:Four hundred twenty-eight patients met inclusion criteria. Manual reduction (MR) alone was used in 11% of fractures, and adjunctive reduction aids were used for the remaining 89%. After controlling for age, BMI, and fracture location, the use of traveling traction (TT) with or without percutaneous clamping (PC) resulted in significantly improved coronal plane alignment compared to MR alone (TT: 3.4°, TT+PC: 3.2°, MR: 4.5°, P = .007 and P = .01, respectively). Using TT+PC resulted in the lowest rate of coronal plane malalignment (13% vs 39% with MR alone, P = .01), and using any adjunctive reduction technique resulted in decreased malalignment rates compared to MR (24% vs 39%, P = .02). No difference was observed in sagittal plane alignment between reduction techniques. Intraclass correlation coefficient (ICC) results indicated excellent intraobserver reliability on both planes (both ICC>0.85), good inter-observer reliability in the coronal plane (ICC = 0.7), and poor inter-observer reliability in the sagittal plane (ICC = 0.05).Conclusions:The use of adjunctive reduction techniques during IMN fixation of tibia fractures is associated with a lower incidence of coronal plane malalignment when compared to manual reduction alone.Level of evidence:Therapeutic Level III.