Inter-reader Intraclass Correlation Coefficients Research Articles

RELIABILITY AND VALIDITY OF SINGLE AXIAL SLICE VS. MULTIPLE SLICE QUANTITATIVE MEASUREMENT OF KNEE EFFUSION-SYNOVITIS ON 3T MRI

Inflammation is increasingly recognized as critical in the pathogenesis of KOA, and effusion-synovitis (ES) on knee MRI has been identified as an important biomarker of OA. Semi-automated software methods offer objective measurements of ES volume on MRI, but efficiency is crucial, especially for studies involving large numbers of scans. We have developed a single-slice quantitative measurement of ES using the single axial slice with the largest area of ES, and compared this to previously validated multi-slice measurements. We evaluated the reliability of a new single-slice quantitative measurement of ES in knees with radiographic OA, and correlated this measurement of ES with a validated multi-slice quantitative measurement of ES, and with a validated semi-quantitative scoring of ES in order to demonstrate that the more efficient single-slice ES method is a valid substitute for the multi-slice approach. OAI participants with radiographic OA (i.e., Kellgren-Lawrence [KL] 2,3) in at least one knee at baseline with prior semi-quantitative assessment of ES by a MSK radiologist were randomly selected (n=50 knees; 1 knee per participant). Four non-expert readers were trained to use custom-designed semi-automated software to measure ES volume on 3T MRI, under the direction of a MSK radiologist, and then read scans from the KL 2,3 sample blinded to prior assessments (twice for 3 readers and once for 1 reader). Intra- and inter-reader reliability was evaluated based on intraclass correlation coefficients (ICC) from a linear mixed model with random effects for knee, reader, and interaction between knee and reader. We assessed agreement with the mean absolute difference (MAD). In a subset of the FNIH study (n=301), one reader measured single-slice ES volume, blinded to prior multi-slice quantitative measurement of ES. We estimated the Spearman correlation coefficient between single-slice and multi-slice volume of ES in the FNIH subset. Validity was evaluated based on Spearman correlation between single-slice ES volume and semi-quantitative ES MRI Osteoarthritis Knee Score (MOAKS) in the KL 2,3 sample and the FNIH subset. 95% confidence intervals (CI) were generated from bias-corrected and acceleration-adjusted bootstrap. In the sample of KL 2,3 knees, the mean single-slice ES volume was 312.8 mm 3 with standard deviation 218.4 mm 3 . The intra-reader ICC was 0.96 (95%CI: 0.93, 0.97), and inter-reader ICC was 0.90 (95%CI: 0.87, 0.95); the intra-reader MAD was 36 mm 3 (95%CI: 28, 44), and inter-reader MAD was 61 mm 3 (95%CI: 48, 75). The estimated correlation between single-slice and multi-slice ES volume was 0.75 (95%CI: 0.68, 0.81; Figure 1). Correlation between single-slice ES volume and MOAKS ES was 0.62 (95%CI: 0.39, 0.79) in the KL 2,3 sample and 0.67 (95%CI: 0.59, 0.73) in the FNIH subset (Figure 2A), while correlation between multi-slice ES volume and MOAKS ES was 0.74 (95%CI: 0.68, 0.79) in the FNIH subset (Figure 2B). Single-slice quantitative measurement of ES had excellent intra-and inter-reader reliability with non-expert readers, and good correlation with both multi-slice quantitative ES measurement and with MOAKS. Single-slice quantitative measurement is an efficient surrogate for multi-slice assessment, having shown similar validity based on comparison to semi-quantitative scoring by MSK radiologists. None CKK is consultant to Regeneron, LG Chem, and Express Scripts. He is principal investigator for pharma sponsored clinical trials to Abbvie, Cumberland, and GSK and DSMB to Kolon TissueGene and Avalor Therapeutics. CORRESPONDENCE ADDRESS: kwoh@arthritis.arizona.edu

Comparing ARIA-E severity scales and effects of treatment management thresholds.

Amyloid-related imaging abnormalities-edema (ARIA-E) is associated with anti-amyloid beta monoclonal antibody treatment. ARIA-E severity may be assessed using the Barkhof Grand Total Scale (BGTS) or the 3- or 5-point Severity Scales of ARIA-E (SSAE-3/SSAE-5). We assessed inter- and intra-reader correlations between SSAE-3/5 and BGTS. Magnetic resonance imaging scans were collected from 75 participants in the SCarlet RoAD and Marguerite RoAD studies. Three neuroradiologists reviewed scans at baseline and at follow-up. Concordance in dichotomized ARIA-E ratings was assessed for a range of BGTS thresholds. SSAE-3/5 scores correlated with BGTS scores, with high inter-reader intraclass correlation coefficients across all scales. There was high agreement in dichotomized ratings for SSAE-3>1versus BGTS>3 for all readers (accuracy 0.85-0.93) and between pairs of readers. SSAE-3/5 showed high degrees of correlation with BGTS, potentially allowing seamless transition from the BGTS to SSAE-3/5 for ARIA-E management.

Reliability and Reproducibility of Sacro-Femoro-Pubic Angle Measurements on Anteroposterior Pelvis Radiographs.

BackgroundThe sacro-femoro-pubic (SFP) angle is a surrogate measurement for pelvic tilt obtained on anteroposterior (AP) pelvis radiographs. It is unknown whether the SFP angle can be measured reliably by senior surgeons and trainees alike.MethodsWe conducted an intrareader and interreader reliability study using preoperative AP pelvis radiographs. Using our hospital electronic database, we randomly selected 31 subjects undergoing primary total hip arthroplasty. An attending orthopedic surgeon and two orthopedic surgery residents performed two separate SFP angle measurements for each subject. Intrareader and interreader agreement were measured using intraclass correlation coefficients (ICC).ResultsPairwise intrareader and interreader ICCs ranged from 0.91 to 0.99. Overall ICC was 0.94. Interreader ICCs between the attending surgeon and each orthopedic resident were 0.93 and 0.98, and the ICC between orthopedic residents was 0.91. Interreader mean differences for SFP angle measurements were 2.32 degrees or less: -2.03 (standard deviation: 2.23) and 0.29 (1.64) between the attending surgeon and each orthopedic resident, and 2.32 (2.44) between orthopedic residents.ConclusionSFP angle may be measured reliably on AP pelvis radiographs by senior clinicians and trainees. SFP angle measurement may prove helpful as an alternative to pelvic tilt measurements when lateral pelvic radiographs are not available.

Consistency of 3D femoral torsion measurement from MRI compared to CT gold standard

BackgroundSeveral hip and knee pathologies are associated with aberrant femoral torsion. Diagnostic workup includes computed tomography (CT) and magnetic resonance imaging (MRI). For three-dimensional (3D) analysis of complex deformities it would be desirable to measure femoral torsion from MRI data to avoid ionizing radiation of CT in a young patient population. 3D measurement of femoral torsion from MRI has not yet been compared to measurements from CT images. We hypothesize that agreement will exist between MRI and CT 3D measurements of femoral torsion.MethodsCT and MRI data from 29 hips of 15 patients with routine diagnostic workup for suspected femoroacetabular impingement (FAI) were used to generate 3D bone models. 3D measurement of femoral torsion was performed by two independent readers using the method of Kim et al. which is validated for CT. Inter-modalitiy and inter-reader intraclass correlation coefficients (ICC) were calculated.ResultsBetween MRI and CT 3D measurements an ICC of 0.950 (0.898; 0.976) (reader 1) respectively 0.950 (0.897; 0.976) (Reader 2) was found. The ICC (95% CI) expressing the inter-reader reliability for both modalities was 0.945 (0.886; 0.973) for MRI and 0.957 (0.910; 0.979) for CT, respectively. Mean difference between CT and MRI measurement was 0.42° (MRI – CT, SD: 2.77°, p = 0.253).ConclusionsThere was consistency between 3D measurements of femoral torsion between computer rendered MRI images compared to measurements with the “gold standard” of CT images. ICC for inter-modality and inter-reader consistency indicate excellent reliability. Accurate, reliable and reproducible 3D measurement of femoral torsion is possible from MRI images.

Comparison between 1.5T and 3.0T MRI: both field strengths sensitively detect subclinical inflammation of hand and forefoot in patients with arthralgia

Objective: Magnetic resonance imaging (MRI) of small joints sensitively detects inflammation. This inflammation, and tenosynovitis in particular, has been shown to predict rheumatoid arthritis (RA) development in arthralgia patients. These data have predominantly been acquired on 1.0–1.5 T MRI. However, 3.0 T is now commonly used in practice. Evidence on the comparability of these field strengths is scarce and has never included subtle inflammation in arthralgia patients or tenosynovitis. Therefore, we assessed the comparability of 1.5 T and 3.0 T in detecting subclinical inflammation in arthralgia patients.Method: A total of 2968 locations (joints, bones, tendon sheaths) in the hands and forefeet of 28 patients with small-joint arthralgia, at risk for RA, were imaged on both 1.5 and 3.0 T MRI. Two blinded readers independently scored erosions, osteitis, synovitis, and tenosynovitis, in line with the Rheumatoid Arthritis Magnetic Resonance Imaging Score (RAMRIS). Features were summed into inflammation (osteitis, synovitis, tenosynovitis) and RAMRIS (inflammation and erosions). Agreement was assessed with intraclass correlation coefficients (ICCs) for continuous scores and after dichotomization into presence or absence of inflammation, on patient and location levels.Results: Interreader ICCs were excellent (> 0.90). Comparing 1.5 and 3.0 T revealed an ICC of 0.90 for inflammation and RAMRIS. ICCs for individual inflammation features were: tenosynovitis 0.87 (95% confidence interval 0.74–0.94), synovitis 0.65 (0.24–0.84), and osteitis 0.96 (0.91–0.98). Agreement was 83% for inflammation and 89% for RAMRIS. Analyses on the location level showed similar results.Conclusion: Agreement on subclinical inflammation between 1.5 T and 3.0 T was excellent. Although synovitis scores were slightly different, synovitis often occurs simultaneously with other inflammatory signs, suggesting that scientific results on the predictive value of MRI-detected inflammation for RA, obtained on 1.5 T MRI, can be generalized to 3.0 T MRI.

Feasibility of CT-derived myocardial strain measurement in patients with advanced cardiac valve disease

To explore the feasibility of CT-derived myocardial strain measurement in patients with advanced cardiac valve disease and to compare it to strain measurements derived from transthoracic echocardiography (TTE). 43 consecutive patients with advanced cardiac valve disease and clinically indicated retrospectively gated cardiac CTs were retrospectively analyzed. The longitudinal, circumferential as well as radial systolic strain were determined in all patients utilizing a commercially available CT strain software. In 36/43 (84%) patients, CT-derived longitudinal strain was compared to speckle-tracking TTE. Pearson’s correlation coefficients as well as Bland–Altman analysis were used to compare the CT-derived strain measurements to TTE. The intra- and inter-reader-reliability of the CT-derived strain measurements were assessed by intra-class correlation coefficients (ICCs). Strain measurements were feasible in all patients. CT-derived global longitudinal strain (GLS) correlated moderately with TTE-derived GLS (r = 0.6, p < 0.001). A moderate correlation between CT-derived GLS and CT-derived left ventricular ejection fraction was found (LVEF, r = − 0.66, p = 0.036). Bland–Altman analysis showed a systematic underestimation of myocardial strain by cardiac CT compared to TTE (mean difference: − 5.8%, 95% limit of agreement between − 13.3 and 1.8%). Strain measurements showed an excellent intra- and inter-reader-reliability with an intra-reader ICC of 1.0 and an inter-reader ICC of 0.99 for GLS measurements. CT-derived myocardial strain measurements are feasible in patients with advanced cardiac valve disease. They are highly reproducible and correlate with established parameters of strain measurements. Our results encourage the implementation of CT-derived strain measurement into clinical routine.

Effect of radiographic disease severity in high-resolution quantitative computed tomography assessment of metacarpophalangeal joint erosion and cysts.

Bone erosions are the hallmark of rheumatoid arthritis (RA). High-resolution peripheral quantitative computed tomography (HR-pQCT) enables 3-dimensional visualization of arthritic bone erosions at a high resolution. However, the degree of erosive disease could influence the reliability of HR-pQCT evaluation. We aim to assess the intra- and inter-reader variability of identification of erosions in the metacarpophalangeal (MCP) joints using HR-pQCT in healthy controls and patients with RA, stratified according to van der Heijde-modified Sharp Score (HSS) of radiographic erosions. We analyzed HR-pQCT images from 78 patients with RA and 25 healthy controls. Patients were allocated to one of three groups of mild, moderate or severe disease according to HSS of MCP joints 2 and 3. Total HR-pQCT scans were analyzed twice in random order by three experienced readers, blinded to group distribution. The number of cortical interruptions and their classification as either erosions or cysts according to predefined criteria were recorded. Intraclass correlation coefficients (ICC) for cortical interruptions, erosions and cysts were calculated for each group using a 2-way random-effects model for inter-reader ICC and a 2-way mixed-effects model for intra-reader ICC. The intra- and inter-reader ICC were good to moderate for cortical interruptions and moderate for erosions throughout disease severity groups. The ICCs for the identification of cysts decreased with increasing degree of erosive disease. The detection of cortical interruptions is only minimally affected by the degree of erosive damage, whereas the distinction between erosions and cysts is more complex in patients with extensive erosive disease.

Reliability of Magnetic Resonance Imaging (MRI) Scoring of the Metatarsophalangeal Joints of the Foot according to the Rheumatoid Arthritis MRI Score.

The Rheumatoid Arthritis Magnetic Resonance Imaging Score (RAMRIS) is validated for hand MRI. Its reliability applied to metatarsophalangeal (MTP 1-5) joints is unknown and was studied in early arthritis and clinically suspect arthralgia. Patients underwent 1.5 Tesla MRI of MTP, metacarpophalangeal (MCP 2-5), and wrist joints. Two paired readers scored bone marrow edema (BME), synovitis, tenosynovitis, and erosions. Interreader reliability was assessed of 441 consecutive early arthritis patients at baseline, 215 by 2 readers, and the remaining 226 by 2 different readers. Two readers scored baseline MRI of 82 consecutive patients with clinically suspect arthralgia, and 40 randomly selected patients by 9 readers. Intrareader reliability was determined on a random set of 15 early arthritis patients, scored twice by 2 readers. For change scores, 30 early arthritis patients with baseline and 1-year followup MRI were scored by 2 readers. Intraclass correlation coefficients (ICC), Bland-Altman (BA) plots, and smallest detectable change (SDC) were determined. MRI data of MTP joints were compared to wrist and MCP joints. Interreader ICC and mean scores in early arthritis were BME ICC 0.91-0.92 (mean 1.5 ± SD 2.6), synovitis 0.90-0.92 (1.3 ± 1.7), tenosynovitis 0.80-0.85 (1.1 ± 1.8), and erosions 0.88-0.89 (0.7 ± 1.0). In patients with clinically suspect arthralgia, ICC were comparable. Intrareader ICC for inflammatory MRI features were 0.84-0.98, for erosions 0.71 (reader 1), and 0.92 (reader 2). Change score ICC were ≥ 0.90, except erosions (0.77). SDC were ≤ 1.0. BA plots showed no systematic bias. Reliability scores of MTP joints were similar to MCP and wrist joints. Status and change MRI scores of BME, synovitis, tenosynovitis, and erosions of MTP joints can be assessed reliably by RAMRIS.

Variability and Reliability of Normal Liver 18F-FDG Standardized Uptake Value in Relation to Placement of Volume of Interest

Background: The impact of volume-of-interest (VOI) placement on liver standardized-uptake-value (SUV) measurement remains unclear. Objectives: To investigate variability and reliability of liver SUV in relation to placement of VOI within the right lobe of the liver. Patients and Methods: Positron emission tomography-computed tomography (PET/CT) of 343 patients with normal liver were retrieved. Liver mean SUV (SUVmean) were measured by placing spherical VOIs at upper, portal vein and lower level of the right-lobe liver. The general linear model, Bland-Altman analysis, one-way ANOVA were performed and the intraclass correlation coefficients (ICC) were calculated to identify impacts of reader and VOI position on variations of liver SUVmean. Results: The liver SUVmean at the upper, lower and portal vein level were 1.85 ± 0.435, 1.88 ± 0.409, and 1.89 ± 0.414, respectively, with no significant inter-location difference (P = 0.449). The coefficients of variation were close among locations (21.9% - 23.5%). The inter-location ICCs controlling for reader were 0.955 for reader 1 and 0.980 for reader 2, while the inter-reader ICCs controlling for location were between 0.930 and 0.937, indicating great reliability and reproducibility. However, inter-reader bias of SUVmean were significantly different among locations (P < 0.001). Bias at portal vein level (0.069 ± 0.112) was significantly larger than those at the upper level (0.004 ± 0.144, P < 0.001) and lower level (0.002 ± 0.136, P < 0.001). Conclusion: Liver SUVmean measured by placing VOI at the portal vein level exhibited a relatively larger variation and less precision between readers, which was not recommended.

Measuring spinal mobility in early axial spondyloarthritis: does it matter?

To investigate the frequency and order of impairment of spinal mobility measures (SMMs) and their cross-sectional and longitudinal usefulness in early axial spondyloarthritis. SMMs measurements of patients from the DESIR (5-year data) and SPACE (2.6 (1.9) years of follow-up) cohorts were analysed. Cross-sectional (group level) and longitudinal (individual level) analyses were performed comparing SMMs to pre-defined cut-offs derived from healthy individuals. Subgroup analyses were used to study patient and disease characteristics potentially influencing spinal mobility. Reliability was analysed using intraclass correlation coefficients and the smallest detectable change. In 328 DESIR and 148 SPACE patients, lateral spinal flexion (LSF) and mSchober were the most impaired SMMs. If both (LSF and mSchober) were measured, 84% (DESIR) and 74% (SPACE) of the patients with impairment in ≥1 SMM would be captured. LSF and Bath AS Metrology Index best discriminated between subgroups of patients (higher impairment in patients ever treated with biologics, with higher disease activity and presence of baseline syndesmophytes): e.g. 31% of LSF impairment in patients with Ankylosing Spondylitis Disease Activity Score (ASDAS) < 2.1 in ≥2/3 visits vs 49% in those with ASDS ≥ 2.1. A high variability in SMMs within the same patient over time was observed, even when restricting the analysis to patients with low disease activity. Reliability of SMMs was 'fair' to 'good' (inter-reader intraclass correlation coefficients (2, 1): 0.55-0.84; intrareader intraclass correlation coefficients (2, 1): 0.49-0.72). Smallest detectable changes were in general high, e.g. 5.1 cm for LSF. Cross-sectional use of SMMs, at the group level, is informative in patients with early axial spondyloarthritis. However, the high variation of SMMs over time impairs their use, at the individual patient level.

Detection of Sacroiliitis by Short-tau Inversion Recovery and T2-weighted Turbo Spin Echo Sequences: Results from the SIMACT Study.

To compare proton density-weighted short-tau inversion recovery (PD-STIR) and T2-weighted fat-suppressed turbo spin echo (T2-FS) sequences for detecting osteitis lesions of the sacroiliac joints (SIJ) in patients with chronic low back pain (CLBP). This prospective study included 110 patients with CLBP and suspected spondyloarthritis and 18 healthy controls. All 128 participants (age range: 19-57 yrs) underwent 3.0 Tesla magnetic resonance imaging (MRI) of the SIJ including PD-STIR and T2-FS. Two readers independently scored PD-STIR and T2-FS images for osteitis in separate sessions. Sum scores and signal-to-noise (SNR) and contrast-to-noise (CNR) ratios were calculated. Images were further analyzed as to whether they fulfilled the Assessment of SpondyloArthritis international Society (ASAS) criterion of a positive MRI (MRI+). Interreader agreement was calculated using intraclass correlation coefficients. Average osteitis sum scores were higher for T2-FS images (mean sum score of 4.10 in T2-FS vs 2.55 in PD-STIR, p = 0.017). Mean SNR was 16.54 for PD-STIR and 37.30 for T2-FS (p = 0.0289). Mean CNR was 4.14 for PD-STIR and 20.20 for T2-FS (p = 0.0212). For both readers, the ASAS MRI+ definition was more often fulfilled by T2-FS than by PD-STIR images, resulting in more patients being classified as having axial spondyloarthritis (axSpA): 68 patients using T2-FS versus 58 patients using PD-STIR. Interreader intraclass correlation coefficients were very good for both PD-STIR (0.91) and T2-FS (0.86). T2-FS sequences improve image quality and hence the detection of osteitis compared to the PD-STIR sequence. More patients were classified as axSpA based on a positive MRI by T2-FS.

Reader agreement and accuracy of ultrasound features for hepatic steatosis.

The purpose of the study is to assess the reader agreement and accuracy of eight ultrasound imaging features for classifying hepatic steatosis in adults with known or suspected hepatic steatosis. This was an IRB-approved, HIPAA-compliant prospective study of adult patients with known or suspected hepatic steatosis. All patients signed written informed consent. Ultrasound images (Siemens S3000, 6C1HD, and 4C1 transducers) were acquired by experienced sonographers following a standard protocol. Eight readers independently graded eight features and their overall impression of hepatic steatosis on ordinal scales using an electronic case report form. Duplicated images from the 6C1HD transducer were read twice to assess intra-reader agreement. Intra-reader, inter-transducer, and inter-reader agreement were assessed using intraclass correlation coefficients (ICC). Features with the highest intra-reader agreement were selected as predictors for dichotomized histological steatosis using Classification and Regression Tree (CART) analysis, and the accuracy of the decision rule was compared to the accuracy of the radiologists' overall impression. 45 patients (18 males, 27 females; mean age 56 ± 12years) scanned from September 2015 to July 2016 were included. Mean intra-reader ICCs ranged from 0.430 to 0.777, inter-transducer ICCs ranged from 0.228 to 0.640, and inter-reader ICCs ranged from 0.014 to 0.561. The CART decision rule selected only large hepatic vein blurring and achieved similar accuracy to the overall impression (74% to 75% and 68% to 72%, respectively). Large hepatic vein blurring, liver-kidney contrast, and overall impression provided the highest reader agreement. Large hepatic vein blurring may provide the highest classification accuracy for dichotomized grading of hepatic steatosis.

Development and Validation of the OMERACT Rheumatoid Arthritis Magnetic Resonance Tenosynovitis Scoring System in a Multireader Exercise.

To develop and validate a magnetic resonance imaging (MRI) tenosynovitis (TS) score for tendons at the wrist and metacarpophalangeal (MCP) joint levels in patients with rheumatoid arthritis (RA). Axial T1-weighted precontrast and postcontrast fat-saturated MR image sets of the hands of 43 patients with RA initiating rituximab therapy were obtained at baseline and after 14, 26, 38, or 52 weeks. The MR images were scored twice by 4 readers. Nine tendon compartments of the wrist and 4 flexor tendon compartments at the MCP joints were assessed. Tenosynovitis was scored as follows: 0: No; 1: < 1.5 mm; 2: ≥ 1.5 mm but < 3 mm; 3: ≥ 3 mm peritendinous effusion and/or postcontrast enhancement. Intrareader and interreader intraclass correlation coefficients (ICC), smallest detectable change (SDC), percentage of exact and close agreement (PEA/PCA), and standardized response mean (SRM) were calculated. Intrareader and interreader ICC for status and change scores were very good (≥ 0.80) for total scores for all readers. Intrareader SDC was ≤ 3.0 and interreader SDC was < 2.0. The overall PEA/PCA intrareader and interreader agreements for change scores in all tendons were 73.8%/97.6% and 47.9%/85.0%, respectively. Average SRM was moderate for total scores and 60.5% of the patients had a tenosynovitis change score ≥ SDC. The TS score showed high intrareader and interreader agreement for wrist and finger tendons, with moderate responsiveness, and the majority of the patients showed a change above the SDC. This scoring system may be included as a component of the RAMRIS.

Validation of a Knowledge Transfer Tool for the Knee Inflammation MRI Scoring System for Bone Marrow Lesions According to the OMERACT Filter: Data from the Osteoarthritis Initiative.

To assess feasibility and reliability of scoring bone marrow lesions (BML) on knee magnetic resonance imaging (MRI) in osteoarthritis using the Outcome Measures in Rheumatology Knee Inflammation MRI Scoring System (KIMRISS), with a Web-based interface and online training with real-time iterative calibration. Six readers new to the KIMRISS (3 radiologists, 3 rheumatologists) scored sagittal T2-weighted fat-saturated MRI in 20 subjects randomly selected from the Osteoarthritis Initiative data, at baseline and 1-year followup. In the KIMRISS, the reader moves a transparent overlay grid within a Web-based interface to fit bones, then clicks or touches each region containing BML per slice, to score 1 if BML is present. Regional and total scores are automatically calculated. Outcomes include the interreader intraclass correlation coefficients (ICC) and the smallest detectable change (SDC). Scoring took 3-12 min per scan and all readers rated the process as moderately to very user friendly. Despite a low BML burden (average score 2.8% of maximum possible) and small changes, interobserver reliability was moderate to high for BML status and change in the femur and tibia (ICC 0.78-0.88). Four readers also scored the patella reliably, whereas 2 readers were outliers, likely because of image artifacts. SDC of 1.5-5.6 represented 0.7% of the maximum possible score. We confirmed feasibility of knee BML scoring by new readers using interactive training and a Web-based touch-sensitive overlay system, finding high reliability and sensitivity to change. Further work will include adjustments to training materials regarding patellar scoring, and study in therapeutic trial datasets with higher burden of BML and larger changes.

Bowel Sonography and MR Enterography in Children.

The objective of this study was to assess the diagnostic agreement between bowel sonography and MR enterography in children with suspected or confirmed inflammatory bowel disease (IBD). We prospectively studied 33 children undergoing clinically indicated MR enterography. Bowel sonography was performed within 2 hours before MR enterography. The reliability of each modality was assessed by computing the interreader kappa agreement on nine inflammatory markers in eight bowel segments and the interreader intraclass correlation coefficients (ICC) in each bowel segment. The agreement between both modalities in each bowel segment was also calculated using the Kendall coefficient of concordance (KCC) in addition to the sensitivity and specificity of sonography using those of MR enterography as the reference standard. Sonographic readings are substantially reliable in the colon and terminal and distal ileum (ICC = 0.79-0.88) but are highly unreliable in the mid and proximal small bowel. MR enterography has moderate to almost perfect reliability for all bowel segments (ICC = 0.43-0.96). Sonography was in substantial agreement with MR enterography in most of the large bowel (KCC = 0.67-0.83) and terminal and distal ileum (KCC = 0.75) and in moderate agreement in the remaining bowel segments (KCC = 0.45-0.48). Sonographic findings are specific for disease using MR enterography as the reference standard in all bowel segments (0.83-1.00), but sensitivity markedly varies according to disease location. Sonography is highly specific and shows substantial agreement with MR enterography in detecting inflammatory and damage changes in the colon and terminal and distal ileum, the segments most likely to be affected by IBD. Therefore, sonography can be a good complement to MR enterography in evaluations of patients with known or suspected IBD, particularly if MR enterography access is limited.

The Longitudinal Reliability and Responsiveness of the OMERACT Hand Osteoarthritis Magnetic Resonance Imaging Scoring System (HOAMRIS).

To evaluate the interreader reliability of change scores and the responsiveness of the OMERACT Hand Osteoarthritis (OA) Magnetic Resonance Image (MRI) Scoring System (HOAMRIS). Paired MRI (baseline and 5-yr followup) from 20 patients with hand OA were scored with known time sequence by 3 readers according to the HOAMRIS: Synovitis, erosive damage, cysts, osteophytes, cartilage space loss, malalignment, and bone marrow lesions (BML; 0-3 scales with 0.5 increments for synovitis, erosive damage, and BML). Interreader reliability for status and change scores were assessed by intraclass correlation coefficients (ICC), percentage exact agreement and percentage close agreement (PEA/PCA), and smallest detectable change (SDC). Responsiveness was assessed by standardized response means (SRM). Cross-sectional interreader ICC were good to very good (≥ 0.74) for all features except synovitis, cysts, and malalignment (ICC 0.50-0.58). The range of change values was small, leading to low ICC for change scores. The SDC values for sum scores (total range 0-24) varied between 1.97-3.05 (except 1.08 for malalignment). For status scores, PEA/PCA on scores in individual joints across the readers were 8.1-50.0 and 43.8-78.1, respectively. Similarly, PEA/PCA for change scores were 20.6-63.8 and 66.3-93.1, respectively. All features except cysts and BML demonstrated good responsiveness with higher SRM for sum scores (range 0.46-1.62) than for scores in individual joints (range 0.24-0.73). Good to very good interreader ICC values were found for cross-sectional readings, whereas the longitudinal reliability was lower because of a smaller range of change scores. All features, except cysts and BML, showed good responsiveness.

Validity and responsiveness of a new measure of knee osteophytes for osteoarthritis studies: data from the osteoarthritis initiative

To validate a novel quantitative MRI method to measure osteophyte volume. 90 subjects were selected from the Progression Cohort of the Osteoarthritis Initiative (OAI) at baseline and 48 months, and analyzed using a semi-automated software tool. Marginal osteophyte volume was calculated for four compartments of the central weight-bearing region of the tibiofemoral joint. Standardized response mean (SRM) for change in volume was used to quantify responsiveness. Concurrent validity was assessed via a comparison with MRI Osteoarthritis Knee Score (MOAKS) using Kruskal-Wallis analysis and Spearman's correlation coefficient. Intra- and inter-reader reliability was assessed on a subset of 20 knees using intra-class correlation coefficients (ICCs) and the root mean square standard deviation (RMSSD). The average change in osteophyte volume (ΔV) was 196mm(3) (SD=272mm(3)), and the baseline to 48-month SRM was 0.72. An increase in osteophyte volume was observed for 84% (76/90) of the subjects. Kruskal-Wallis analysis across the four MOAKS osteophyte categories was significant for medial and lateral compartments of both the tibia and femur (P<0.001 for all). The intra-reader ICC was 0.98, and RMSSD was 82mm(3), while inter-reader ICC was 0.97 and RMSSD was 91mm(3). A statistically significant positive correlation was observed between osteophyte volume and several MOAKS cartilage and BML scores. The reader time was approximately 10min per knee. The method is responsive, efficient, and precise, making it practical for use in large cohort studies and observational research.

Assessment and quantification of sources of variability in breast apparent diffusion coefficient (ADC) measurements at diffusion weighted imaging

PurposeApparent Diffusion Coefficient (ADC) measurements are increasingly used for assessing breast cancer response to neoadjuvant chemotherapy although little data exists on ADC measurement reproducibility. The purpose of this work was to investigate and characterise the magnitude of errors in ADC measures that may be encountered in such follow-up studies- namely scanner stability, scan–scan reproducibility, inter- and intra- observer measures and the most reproducible measurement of ADC. MethodsInstitutional Review Board approval was obtained for the prospective study of healthy volunteers and written consent acquired for the retrospective study of patient images. All scanning was performed on a 3.0-T MRI scanner. Scanner stability was assessed using an ice-water phantom weekly for 12 weeks. Inter-scan repeatability was assessed across two scans of 10 healthy volunteers (26–61 years; mean: 44.7 years). Inter- and intra-reader analysis repeatability was measured in 52 carcinomas from clinical patients (29–70 years; mean: 50.0 years) by measuring the whole tumor ADC value on a single slice with maximum tumor diameter (ADCS) and the ADC value of a small region of interest (ROI) on the same slice (ADCmin).Repeatability was assessed using intraclass correlation coefficients (ICC) and coefficients of repeatability (CoR). ResultsScanner stability contributed 6% error to phantom ADC measurements (0.071×10−3mm2/s; mean ADC=1.089×10−3mm2/s). The measured scan-scan CoR in the volunteers was 0.122×10−3mm2/s, contributing an error of 8% to the mean measured values (ADCscan1=1.529×10−3mm2/s; ADCscan2=1.507×10−3mm2/s).Technical and clinical observers demonstrated excellent intra-observer repeatability (ICC>0.9). Clinical observer CoR values were marginally better than technical observer measures (ADCS=0.035×10−3mm2/s vs. 0.097×10−3mm2/s; ADCmin=0.09×10−3mm2/s vs. 0.114×10−3mm2/s). Inter-reader ICC values were good 0.864 (ADCS) and fair 0.677 (ADCmin). Corresponding CoR values were 0.202×10−3mm2/s and 0.264×10−3mm2/s, respectively. ConclusionsBoth scanner stability and scan–scan variation have minimal influence on breast ADC measurements, contributing less than 10% error of average measured ADC values. Measurement of ADC values from a small ROI contributes a greater variability in measurements compared with measurement of ADC across the whole visible tumor on one slice. The greatest source of error in follow-up studies is likely to be associated with measures made by multiple observers, and this should be considered where multiple measures are required to assess response to treatment.

Validation of the OMERACT Magnetic Resonance Imaging Joint Space Narrowing Score for the Wrist in a Multireader Longitudinal Trial.

To assess the intrareader and interreader agreement and sensitivity to change of the Outcome Measures in Rheumatology (OMERACT) Rheumatoid Arthritis Magnetic Resonance Imaging Joint Space Narrowing (RAMRIS-JSN) score in the rheumatoid arthritis (RA) wrist in a longitudinal multireader exercise. Coronal T1-weighted MR image sets of 1 wrist from 20 patients with early RA were assessed twice for JSN at 17 sites at baseline and after 36 or 60 months by 4 readers blinded to patient data but not time order. The joints were scored 0-4 according to the OMERACT RAMRIS-JSN score. Intraclass correlation coefficients (ICC), smallest detectable change (SDC), percentage exact/close agreement (PEA/PCA), and standardized response mean (SRM) were calculated. Median baseline and change score was 10.3 and 1.9, respectively. Intrareader ICC for baseline and change scores was good (≥ 0.50) to very good (≥ 0.80) for all and 3 of 4 readers, respectively. Interreader ICC was very good for change (0.93), while poor for baseline score if all 4 readers were included (0.36), but very good if 1 reader was excluded (0.87). Intrareader and interreader SDC was low (2.34-3.18), except for the intrareader SDC for 1 reader (6.75). The mean PEA/PCA was high for baseline and change scores both within and between the readers (51.5-99.2), except for interreader baseline PEA (14.4). SRM was moderate for all readers (0.55-0.77). The OMERACT RAMRIS-JSN score showed high overall intrareader and interreader reliability, and moderate sensitivity to change, supporting inclusion of the measure as part of the OMERACT RAMRIS system.

Development and validation of a new radiographic scoring system to evaluate bone and cartilage destruction and healing of large joints with rheumatoid arthritis: ARASHI (Assessment of rheumatoid arthritis by scoring of large joint destruction and healing in radiographic imaging) study

Objective: To assess the reliability and sensitivity of a novel scoring method to evaluate the radiographic appearance of and longitudinal changes including joint remodeling in large joints with early and established rheumatoid arthritis (RA).Methods: The ARASHI study group devised new radiographic scoring systems (Status score; range 0–16 points, and Change score; range −11 to 12 points) for evaluation of large joints with RA. Radiographs showing anterior/posterior views of large joints (shoulder, elbow, hip, knee, and ankle joints) taken at two time points (mean interval 2.3 years) were collected from 25 patients with established RA (5 patients for each of the 5 joints, 50 films in total), and an additional 5 films of each joint with severe joint destruction were collected from 5 different sets of RA patients. After consensus on the definition of each component and reader training, images were evaluated using the Larsen’s grading system and the ARASHI Status and Change score by 9 independent senior orthopedic surgeons. The reliability was estimated by intra-class correlation coefficients (ICCs) and measurement error by 95 % confidence intervals of minimum detectable change (MDC95).Results: ARASHI Status score and Change score significantly correlated with Larsen’s grade (r = 0.89, P < 0.0001) and follow-up-baseline differences in Larsen’s grade (r = 0.83, P < 0.0001), respectively. Inter-reader ICCs were very high for both Status score (0.88, 95 % confidence interval [CI], 0.83–0.92, P < 0.001) and Change score (0.92, 95 % CI, 0.87–0.96, P < 0.001). Intra-reader ICCs were also very high for both Status score (0.92, 95 % CI, 0.71–0.98, P < 0.001) and Change score (0.97, 95 % CI, 0.91–0.99, P < 0.001). The MDC95 for inter-reader agreement were 4.18 (25 % of maximum obtainable score, MOS) and 4.99 (21 % of MOS) for Status score and Change score, respectively. The MDC95 for intra-reader agreement was acceptable with 2.82 (17 % of MOS) and 3.02 (13 % of MOS) for Status score and Change score, respectively.Conclusion: The ARASHI scoring method showed good inter-/intra-reader reliability with high ICCs and acceptable MDC95 with respect to each large joint and the components of both Status and Change scores. The results suggest that the ARASHI scoring method might be useful for the assessment of status, as well as longitudinal monitoring of destruction and remodeling of large joints with RA.