Substantial Interreader Agreement Research Articles

Assessment for Carotid Atherosclerotic Plaque Using Vessel Wall Magnetic Resonance Imaging: A Multireader ROC Study to Determine Optimal Sequence for Detecting Vessel Wall Calcification

Introduction: We aimed to compare conventional vessel wall MR imaging techniques and quantitative susceptibility mapping (QSM) to determine the optimal sequence for detecting carotid artery calcification. Methods: Twenty-two patients who underwent carotid vessel wall MR imaging and neck CT were enrolled. Four slices of 6-mm sections from the bilateral internal carotid bifurcation were subdivided into 4 segments according to clock position (0–3, 3–6, 6–9, and 9–12) and assessed for calcification. Two blinded radiologists independently reviewed a total of 704 segments and scored the likelihood of calcification using a 5-point scale on spin-echo imaging, FLASH, and QSM. The observer performance for detecting calcification was evaluated by a multireader, multiple-case receiver operating characteristic study. Weighted κ statistics were calculated to assess interobserver agreement. Results: QSM had a mean area under the receiver operating characteristic curve of 0.85, which was significantly higher than that of any other sequence (p < 0.01) and showed substantial interreader agreement (κ = 0.68). A segment with a score of 3–5 was defined as positive, and a segment with a score of 1–2 was defined as negative; the sensitivity and specificity of QSM were 0.75 and 0.87, respectively. Conclusion: QSM was the most reliable MR sequence for the detection of plaque calcification.

Diagnostic validity and reliability of BT-RADS in the management of recurrent high-grade glioma

Background and purposeBT-RADS is a new framework system for reporting the treatment response of brain tumors. The aim of the study was to assess the diagnostic performance and reliability of the BT-RADS in predicting the recurrence of high-grade glioma (HGG). Materials and MethodsThis prospective single-center study recruited 81 cases with previously operated and pathologically proven HGG. The patients underwent baseline and follow-up contrast-enhanced MRI (CE-MRI). Two neuro-radiologists with ten years-experience in neuroimaging independently analyzed and interpreted the MRI images and assigned a BT-RADS category for each case. To assess the diagnostic accuracy of the BT-RADS for detecting recurrent HGG, the reference standard was the histopathology for BT-RADS categories 3 and 4, while neurological clinical examination and clinical follow up were used as a reference for BT-RADS categories 1 and 2. The inter-reader agreement was assessed using the Cohen's Kappa test. ResultsThe study included 81 cases of HGG, of which 42 were recurrent and 39 were non-recurrent HGG cases based on the reference test. BT-RADS 3B was the best cutoff for predicting recurrent HGG with a sensitivity of 90.5 % to 92.9 %, specificity of 76.9 % to 84.6 %, and accuracy of 83.9 % to 88.9 %, based on both readers. The BT-RADS showed a substantial inter-reader agreement with a K of 0.710 (P = 0.001). ConclusionsThe BT-RADS is a valid and reliable framework for predicting recurrent HGG. Moreover, BT-RADS can help neuro-oncologists make clinical decisions that can potentially improve the patient's outcome.

Application of Artificial Intelligence (AI) System in Opportunistic Screening and Diagnostic Population in a Middle-income Nation.

This study evaluates the effectiveness of artificial intelligence (AI) in mammography in a diverse population from a middle-income nation and compares it to traditional methods. A retrospective study was conducted on 543 mammograms of 467 Malays, 48 Chinese, and 28 Indians in a middle-income nation. Three breast radiologists interpreted the examinations independently in two reading sessions (with and without AI support). Breast density and BI-RADS categories were assessed, comparing the accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) results. Of 543 mammograms, 69.2% had lesions detected. Biopsies were performed on 25%(n=136), with 66(48.5%) benign and 70(51.5%) malignant. Substantial agreement in density assessment between the radiologist and AI software (κ =0.606, p < 0.001) and the BI-RADS category with and without AI (κ =0.74, p < 0.001). The performance of the AI software was comparable to the traditional methods. The sensitivity, specificity, PPV, and NPV or radiologists alone, radiologist + AI, and AI alone were 81.9%,90.4%,56.0%, and 97.1%; 81.0%, 93.1%,55.5%, and 97.0%; and 90.0%,76.5%,36.2%, and 98.1%, respectively. AI software enhances the accuracy of lesion diagnosis and reduces unnecessary biopsies, particularly for BI-RADS 4 lesions. The AI software results for synthetic were almost similar to the original 2D mammography, with AUC of 0.925 and 0.871, respectively. AI software may assist in the accurate diagnosis of breast lesions, enhancing the efficiency of breast lesion diagnosis in a mixed population of opportunistic screening and diagnostic patients. • The use of artificial intelligence (AI) in mammography for population-based breast cancer screening has been validated in high-income nations, with reported improved diagnostic performance. Our study evaluated the usage of an AI tool in an opportunistic screening setting in a multi-ethnic and middle-income nation. • The application of AI in mammography enhances diagnostic accuracy, potentially leading to reduced unnecessary biopsies. • AI integration into the workflow did not disrupt the performance of trained breast radiologists, as there is a substantial inter-reader agreement for BI-RADS category assessment and breast density.

Breast lesion morphology assessment with high and standard b values in diffusion-weighted imaging at 3 Tesla

IntroductionWith increasing spatial resolution, diffusion-weighted imaging (DWI) may be suitable for morphologic lesion characterization in breast MRI – an area that has traditionally been occupied by dynamic contrast-enhanced imaging (DCE). This investigation compared DWI with b values of 800 and 1600 s/mm2 to DCE for lesion morphology assessment in high-resolution breast MRI at 3 Tesla. Material and methodsMultiparametric breast MRI was performed in 91 patients with 93 histopathologically proven lesions (31 benign, 62 malignant). Two radiologists independently evaluated three datasets per patient (DWIb800; DWIb1600; DCE) and assessed lesion visibility and BIRADS morphology criteria. Diagnostic accuracy was compared among readers and datasets using Cochran's Q test and pairwise post-hoc McNemar tests. Bland-Altman analyses were conducted for lesion size comparisons. ResultsDiscrimination of carcinomas was superior compared to benign findings in both DWIb800 and DWIb1600 (p < 0.001) with no b value-dependent difference. Similarly, assessability of mass lesions was better than of non-mass lesions, irrespective of b value (p < 0.001). Intra-reader reliability for the analysis of morphologic BIRADS criteria among DCE and DWI datasets was at least moderate (Fleiss κ≥0.557), while at least substantial inter-reader agreement was ascertained over all assessed categories (κ≥0.776). In pairwise Bland-Altman analyses, the measurement bias between DCE and DWIb800 was 0.7 mm, whereas the difference between DCE and DWIb1600 was 2.8 mm. DWIb1600 allowed for higher specificity than DCE (p = 0.007/0.062). ConclusionsDWI can be employed for reliable morphologic lesion characterization in high-resolution breast MRI. High b values increase diagnostic specificity, while lesion size assessment is more precise with standard 800 s/mm2 images.

Diagnostic performance of Node Reporting and Data System (Node-RADS) for regional lymph node staging of gastric cancer by CT

ObjectivesDiagnostic performance of imaging for regional lymph node assessment in gastric cancer is still limited, and there is a lack of consensus on radiological evaluation. At the same time, there is an increasing demand for structured reporting using Reporting and Data Systems (RADS) to standardize oncological imaging. We aimed at investigating the diagnostic performance of Node-RADS compared to the use of various individual criteria for assessing regional lymph nodes in gastric cancer using histopathology as reference.MethodsIn this retrospective single-center study, consecutive 91 patients (median age, 66 years, range 33–91 years, 54 men) with CT scans and histologically proven gastric adenocarcinoma were assessed using Node-RADS assigning scores from 1 to 5 for the likelihood of regional lymph node metastases. Additionally, different Node-RADS criteria as well as subcategories of altered border contour (lobulated, spiculated, indistinct) were assessed individually. Sensitivity, specificity, and Youden’s index were calculated for Node-RADS scores, and all criteria investigated. Interreader agreement was calculated using Cohen’s kappa.ResultsAmong all criteria, best performance was found for Node-RADS scores ≥ 3 and ≥ 4 with a sensitivity/specificity/Youden’s index of 56.8%/90.7%/0.48 and 48.6%/98.1%/0.47, respectively, both with substantial interreader agreement (κ = 0.73 and 0.67, p < 0.01). Among individual criteria, the best performance was found for short-axis diameter of 10 mm with sensitivity/specificity/Youden’s index of 56.8%/87.0%/0.44 (κ = 0.65, p < 0.01).ConclusionThis study shows that structured reporting of combined size and configuration criteria of regional lymph nodes in gastric cancer slightly improves overall diagnostic performance compared to individual criteria including short-axis diameter alone. The results show an increase in specificity and unchanged sensitivity.Clinical relevance statementThe results of this study suggest that Node-RADS may be a suitable tool for structured reporting of regional lymph nodes in gastric cancer.Key Points• Assessment of lymph nodes in gastric cancer is still limited, and there is a lack of consensus on radiological evaluation.• Node-RADS in gastric cancer improves overall diagnostic performance compared to individual criteria including short-axis diameter.• Node-RADS may be a suitable tool for structured reporting of regional lymph nodes in gastric cancer.

Intraoperative anteroposterior and oblique fluoroscopic views for detection of mediolateral pedicle screw misplacement in the lumbar spine: a randomized cadaveric study

Backgroud contextPedicle screws are commonly used for posterior fixation of the lumbar spine. Inaccuracy of screw placement can lead to disastrous complications. PurposeAs fluoroscopic assisted pedicle screw instrumentation is the most frequently used technique, the aim of this study was to assess the specificity, sensitivity and accuracy of intraoperative fluoroscopy to detect mediolateral screw malpositioning. We also analyzed whether the addition of an oblique view could improve these parameters. Study designOn 12 human cadavers, 138 pedicle screws were placed intentionally either with 0 to 2 mm (75 screws), with 2 to 4 mm (six medial and 12 lateral screws) and with >4 mm (22 medial and 23 lateral screws) breach of the pedicle from Th12 to L5. MethodsThree experienced spine surgeons evaluated the screw positioning in fluoroscopic AP views and 4 weeks later in AP views and additional oblique views. The surgeons' interpretation was compared with the effective screw position on postoperative CT scans. ResultsPedicle breaches greater than 2 mm were detected in 68% with AP views and in 67% with additional oblique views (p=.742). The specificity of AP views was 0.86 and 0.93 with additional oblique views (p=<.01). The accuracy was 0.78 with AP views and 0.81 with AP + oblique views (p=.114). There was a substantial inter-reader agreement (Fleiss's kappa: 0.632). ConclusionsFluoroscopic screening of pedicle screw misplacement has a limited sensitivity. Adding an oblique view improves specificity but not sensitivity and accuracy in detecting screw malpositions. Clinical significanceWhen in doubt of a screw malpositioning, other modalities than a fluoroscopic assisted pedicle screw instrumentation such as intraoperative CT imaging or an intraoperative exploration of the screw trajectory must be evaluated.

Improved diagnostic performance of susceptibility-weighted imaging with compressed sensing-sensitivity encoding and neuromelanin-sensitive MRI for Parkinson's disease and atypical Parkinsonism

To verify the diagnostic performance of the loss of nigrosome-1 on susceptibility-weighted imaging (SWI) with compressed sensing-sensitivity encoding (CS-SENSE) and neuromelanin on neuromelanin-sensitive (NM) magnetic resonance imaging (MRI) for the diagnosis of Parkinson's disease (PD) and atypical Parkinsonism. A total of 195 patients who underwent MRI between October 2019 and February 2020, including SWI, with or without CS-SENSE, and NM-MRI, were reviewed retrospectively. Two neuroradiologists assessed the loss of nigrosome-1 on SWI and neuromelanin on the NM-MRI. The result of N-3-fluoropropyl-2-beta-carbomethoxy-3-beta-(4-iodophenyl) nortropane positron-emission tomography (PET) was set as the reference standard. When CS-SENSE was applied for nigrosome-1 imaging on SWI, the non-diagnostic scan rate was lowered significantly from 19.3% (17/88) to 5.6% (6/107; p=0.004). Diagnosis of PD and atypical Parkinsonism based on the loss of nigrosome-1 on SWI and based on NM-MRI showed good diagnostic value (area under the curve [AUC] 0.821, 95% confidence interval [CI]=0.755-0.875: AUC 0.832, 95% CI=0.771-0.882, respectively) with a substantial inter-reader agreement (κ=0.791 and 0.681, respectively). Combined SWI and neuromelanin had a similar discriminatory ability (AUC 0.830, 95% CI=0.770-0.880). Similarly, the diagnosis of PD was excellent. CS-SENSE may add value to the diagnostic capability of nigrosome-1 on SWI to reduce the nondiagnostic scan rates. Furthermore, loss of nigrosome-1 on SWI or volume loss of neuromelanin on NM-MRI may be helpful for diagnosing PD.

An MRI-based scoring system for pretreatment risk stratification in locally advanced rectal cancer.

Accurately assessing the risk of recurrence in patients with locally advanced rectal cancer (LARC) before treatment is important for the development of treatment strategies. The purpose of this study is to develop an MRI-based scoring system to predict the risk of recurrence in patients with LARC. This was a multicenter observational study that enrolled participants who underwent neoadjuvant chemoradiotherapy. To evaluate the risk of recurrence in these patients, we developed the mrDEC scoring system and assessed inter-reader agreement. Additionally, we plotted Kaplan-Meier curves to compare the 3-year disease-free survival (DFS) and 5-year overall survival (OS) rates among patients with different mrDEC scores. A total of 1287 patients with LARC were included in this study. We observed substantial inter-reader agreement for mrDEC. Based on the mrDEC scores ranging from 0 to 3, the patients were categorized into four groups. The 3-year DFS rates for the groups were 91.0%, 79.5%, 65.5%, and 44.0% (P < 0.0001), respectively, and the 5-year OS rates were 92.9%, 87.1%, 74.8%, and 44.5%, respectively (P < 0.0001). The mrDEC scoring system proved to be an effective tool for predicting the prognosis of patients with LARC and can assist clinicians in clinical decision-making.

Diagnostic Performance of the Darth Vader Sign for the Diagnosis of Lumbar Spondylolysis in Routinely Acquired Abdominal CT.

Spondylolysis is underdiagnosed and often missed in non-musculoskeletal abdominal CT imaging. Our aim was to assess the inter-reader agreement and diagnostic performance of a novel "Darth Vader sign" for the detection of spondylolysis in routine axial images. We performed a retrospective search in the institutional report archives through keyword strings for lumbar spondylolysis and spondylolisthesis. Abdominal CTs from 53 spondylolysis cases (41% female) and from controls (n = 6) without spine abnormalities were identified. A total of 139 single axial slices covering the lumbar spine (86 normal images, 40 with spondylolysis, 13 with degenerative spondylolisthesis without spondylolysis) were exported. Two radiology residents rated all images for the presence or absence of the "Darth Vader sign". The diagnostic accuracy for both readers, as well as the inter-reader agreement, was calculated. The "Darth Vader sign" showed an inter-reader agreement of 0.77. Using the "Darth Vader sign", spondylolysis was detected with a sensitivity and specificity of 65.0-88.2% and 96.2-99.0%, respectively. The "Darth Vader sign" shows excellent diagnostic performance at a substantial inter-reader agreement for the detection of spondylolysis. Using the "Darth Vader sign" in the CT reading routine may be an easy yet effective tool to improve the detection rate of spondylolysis in non-musculoskeletal cases and hence improve patient care.

Quality assessment of prostate MRI by PI-QUAL score: Inter-reader agreement and impact on prostate cancer local staging at 3 Tesla

PurposeTo evaluate whether the Prostate Imaging Quality (PI-QUAL) score impacts prostate cancer (PCa) staging on MRI. The secondary goal was to test inter-reader agreement among radiologists experienced in prostate imaging. MethodA retrospective, single-center study with patients who underwent 3 Tesla prostate MRI scans and were submitted to radical prostatectomy (RP) between January 2018 and November 2021 and were eligible for our study. Extraprostatic extension (EPE) data were collected from original MR reports (EPEm) and pathological reports of RP specimens (EPEp). Three expert prostate radiologists (ESUR/ESUI criteria R1, R2, R3) independently evaluated all MRI exams according to PI-QUAL score for image quality (1 to 5; 1: poor, 5: excellent), blinded to original imaging reports and clinical data. We studied the diagnostic performance of MRI using pooled data from PI-QUAL scores (≤3 vs. ≥4). We also performed univariate and multivariate analyses to assess the PI-QUAL score impact on local PCa staging. Cohen’s K and Tau-b Kendall tests were used to assess the inter-reader agreement for PI-QUAL score, T2WI, DWI, and DCE. ResultsOur final cohort included 146 patients, of which 27.4% presented EPE on pathology. We observed no impact of imaging quality on accuracy for EPE prediction: AUC of 0.750 (95% CI 0.26-1) for PI-QUAL ≤ 3 and 0.705 (95% CI 0.618–0.793) for PI-QUAL ≥ 4. The multivariate analysis demonstrated a correlation of EPEm (OR 3.25, p 0.001) and ISUP grade group (OR 1.89, p 0.012) to predict EPEp. The inter-reader agreement was moderate to substantial (0.539 for R1-R2, 0.522 for R2-R3, and 0.694 for R1-R3). ConclusionOur clinical impact evaluation showed no direct correlation between MRI quality by PI-QUAL score and accuracy in detecting EPE in patients undergoing RP. Additionally, we had moderate to a substantial inter-reader agreement for the PI-QUAL score.

Diagnostic value of multiparametric MRI-based models in the assessment of extra-prostatic extension of prostate cancer

Objective: To evaluate the diagnostic value of multiparametric magnetic resonance imaging (mpMRI) based models in the assessment of extra-prostatic extension (EPE) of prostate cancer. Methods: This retrospective study included 168 consecutive men with prostate cancers [aged 48 to 82 (66.6±6.8) years] who underwent radical prostatectomy and preoperative mpMRI examinations at the First Medical Center of the PLA General Hospital from January 2021 to February 2022. According to European Society of Urogenital Radiology (ESUR) score, EPE grade and mEPE score, all cases were independently evaluated by two radiologists, with disagreement reviewed by a senior radiologist as the final result. The diagnostic performance of each MRI-based model for pathologic EPE prediction was assessed using receiver operating characteristic curve (ROC), and the differences between the corresponding area under the curve (AUC) were compared using the DeLong test. The weighted Kappa test was used to evaluate the inter-reader agreement of each MRI-based model. Results: A total of 62 (36.9%) prostate cancer patients had pathologic confirmed EPE after radical prostatectomy. The AUC of ESUR score, EPE grade and mEPE score for predicting pathologic EPE were 0.836 (95%CI: 0.771-0.888), 0.834 (95%CI: 0.769-0.887) and 0.785 (95%CI: 0.715-0.844), respectively. The AUC of ESUR score and EPE grade were both superior to that of mEPE score with significant differences (all P<0.05), while there was no significant difference between the ESUR score and EPE grade models (P=0.900). EPE grading and mEPE score had good inter-reader consistency, with weighted Kappa values of 0.65 (95%CI: 0.56-0.74) and 0.74 (95%CI: 0.64-0.84), respectively. The inter-reader consistency of ESUR score was moderate, and the weighted Kappa value was 0.52 (95%CI: 0.40-0.63). Conclusion: All MRI-based models showed good preoperative diagnostic value in predicting EPE, among which the EPE grade resulted in more reliable performance with substantial inter-reader agreement.

Reticular Pseudodrusen: Interreader Agreement of Evaluation on OCT Imaging in Age-Related Macular Degeneration

PurposeTo determine the interreader agreement for reticular pseudodrusen (RPD) assessment on combined infrared reflectance (IR) and OCT imaging in the early stages of age-related macular degeneration across a range of different criteria to define their presence.DesignInterreader agreement study.ParticipantsTwelve readers from 6 reading centers.MethodsAll readers evaluated 100 eyes from individuals with bilateral large drusen for the following: (1) the presence of RPD across a range of different criteria and (2) the number of Stage 2 or 3 RPD lesions (from 0 to ≥ 5 lesions) on an entire OCT volume scan and on a selected OCT B-scan. Supportive information was available from the corresponding IR image.Main Outcome MeasuresInterreader agreement, as assessed by Gwet’s first-order agreement coefficient (AC1).ResultsWhen evaluating an entire OCT volume scan, there was substantial interreader agreement for the presence of any RPD, any or ≥ 5 Stage 2 or 3 lesions, and ≥ 5 definite lesions on en face IR images corresponding to Stage 2 or 3 lesions (AC1 = 0.60–0.72). On selected OCT B-scans, there was also moderate-to-substantial agreement for the presence of any RPD, any or ≥ 5 Stage 2 or 3 lesions (AC1 = 0.58–0.65) and increasing levels of agreement with increasing RPD stage (AC1 = 0.08, 0.56, 0.78, and 0.99 for the presence of any Stage 1, 2, 3, and 4 lesions, respectively). There was substantial agreement regarding the number of Stage 2 or 3 lesions on an entire OCT volume scan (AC1 = 0.68), but only fair agreement for this evaluation on selected B-scans (AC1 = 0.30).ConclusionsThere was generally substantial or near-substantial—but not near-perfect—agreement for assessing the presence of RPD on entire OCT volume scans or selected B-scans across a range of differing RPD criteria. These findings underscore how interreader variability would likely contribute to the variability of findings related to the clinical associations of RPD. The low levels of agreement for assessing RPD number on OCT B-scans underscore the likely challenges of quantifying RPD extent with manual grading.Financial Disclosure(s)Proprietary or commercial disclosure may be found after the references. To determine the interreader agreement for reticular pseudodrusen (RPD) assessment on combined infrared reflectance (IR) and OCT imaging in the early stages of age-related macular degeneration across a range of different criteria to define their presence. Interreader agreement study. Twelve readers from 6 reading centers. All readers evaluated 100 eyes from individuals with bilateral large drusen for the following: (1) the presence of RPD across a range of different criteria and (2) the number of Stage 2 or 3 RPD lesions (from 0 to ≥ 5 lesions) on an entire OCT volume scan and on a selected OCT B-scan. Supportive information was available from the corresponding IR image. Interreader agreement, as assessed by Gwet’s first-order agreement coefficient (AC1). When evaluating an entire OCT volume scan, there was substantial interreader agreement for the presence of any RPD, any or ≥ 5 Stage 2 or 3 lesions, and ≥ 5 definite lesions on en face IR images corresponding to Stage 2 or 3 lesions (AC1 = 0.60–0.72). On selected OCT B-scans, there was also moderate-to-substantial agreement for the presence of any RPD, any or ≥ 5 Stage 2 or 3 lesions (AC1 = 0.58–0.65) and increasing levels of agreement with increasing RPD stage (AC1 = 0.08, 0.56, 0.78, and 0.99 for the presence of any Stage 1, 2, 3, and 4 lesions, respectively). There was substantial agreement regarding the number of Stage 2 or 3 lesions on an entire OCT volume scan (AC1 = 0.68), but only fair agreement for this evaluation on selected B-scans (AC1 = 0.30). There was generally substantial or near-substantial—but not near-perfect—agreement for assessing the presence of RPD on entire OCT volume scans or selected B-scans across a range of differing RPD criteria. These findings underscore how interreader variability would likely contribute to the variability of findings related to the clinical associations of RPD. The low levels of agreement for assessing RPD number on OCT B-scans underscore the likely challenges of quantifying RPD extent with manual grading.

Development and Validation of Diagnostic Interpretation Criteria for Evaluation of F-18 Fluoromisonidazole Hypoxia PET/CT – Inter-Reader Reliability and Accuracy

<h3>Purpose/Objective(s)</h3> F-18 Fluoromisonidazole (FMISO) is a robust imaging biomarker for tumor hypoxia. Hypoxia limits effective cancer treatment, specifically radiotherapy in head and neck cancers. We have completed and have ongoing trials using hypoxia imaging to guide radiation dose de-escalation. To ensure wide applicability, we sought to develop and validate standardized qualitative and quantitative criteria to accurately and reliably interpret clinical FMISO PET/CT images. <h3>Materials/Methods</h3> All HPV+ oropharyngeal cancer patients who had FMISO PET/CT imaging before and about 2 weeks into chemoradiation therapy and pre-treatment F-18 FDG PET/CT were included. 4 independent nuclear medicine (NM) physicians with varying clinical experience of 1, 3, 5, and 15 years with no FMISO experience interpreted images. Training and reference interpretations were performed by a 5^th NM physician with 30 years of clinical and 15 years of FMISO experience. Suspicious cervical lymph nodes (LN) > 1 cm in short axis with abnormal FDG avidity on FDG PET/CT were categorized as positive or negative for FMISO uptake. A LN was considered positive if it demonstrated SUVmax visually greater than floor of mouth (FOM). FMISO SUVmax of the LN and FOM were obtained using a 3D VOI. The training and validation process consisted of: (1) initial instruction; (2) practice interpretation; (3) one on one re-training on contentious group reads; (4) final group training; (5) validation interpretation of new scans. Qualitative, visual comparison of FMISO uptake in LN to FOM, and quantitative, ratio of FMISO SUVmax of LN to FOM, were analyzed. The Fleiss κ coefficient (κ) was calculated to measure inter-reader agreement. <h3>Results</h3> 192 FMISO scans were included. Qualitative criteria mean sensitivity and specificity (SE/SP) after initial instruction were 0.773 and 0.809. κ was 0.338. Mean SE/SP after additional training were 0.976 and 0.869. κ was 0.864 [95%CI: 0.782 to 0.944]. For the quantitative criteria, ROC curve analysis was used to estimate the best threshold by maximizing the Youden index, which was determined to be SUVmax LN/SUVmax FOM > 1.2. Quantitative criteria mean SE/SP after additional training were 0.896 and 0.953. κ was 0.859 (95%CI: 0.761 to 0.944). The quantitative criteria alone demonstrated a lower sensitivity than the qualitative, 0.896 vs. 0.976. Therefore, we suggest using a hybrid method with quantitative criteria as a guide with the ultimate decision based on qualitative assessment to minimize the number of patients with false negatives and subsequent undertreatment. <h3>Conclusion</h3> The hybrid qualitative/quantitative diagnostic interpretation criteria developed for evaluating FMISO PET/CT in combination with dedicated training has substantial inter-reader agreement and excellent diagnostic performance. This suggests that multi-institutional studies that employ FMISO PET for patient stratification in prospective trials are feasible and can be implemented.

The Simplified MR Index of Activity Score in Pediatric Small-Bowel Crohn Disease: An Interreader Agreement and Responsiveness Study.

BACKGROUND. The simplified MR index of activity (MaRIA) score is used to assess the severity of small-bowel inflammation without use of IV contrast material. OBJECTIVE. The purposes of this study were to assess interreader agreement on the use of simplified MaRIA scores for evaluation of the inflammatory activity of terminal ileal Crohn disease in children and young adults and to assess whether simplified MaRIA scores change after biologic medical therapy. METHODS. This analysis was ancillary to a previously reported primary prospective research investigation. The study included 20 children and young adults with newly diagnosed ileal Crohn disease and 15 healthy control participants who underwent research small-bowel MRI examinations between December 2018 and October 2021. The participants with Crohn disease underwent baseline MRI and MRI 6 weeks and 6 months after beginning anti-tumor necrosis factor α-treatment as well as weighted pediatric Crohn disease activity index (wPCDAI) and C-reactive protein (CRP) assessment on the day of each examination. Control participants underwent one MRI examination. Four pediatric radiologists independently assigned simplified MaRIA scores using axial and coronal T2-weighted SSFSE images. Median simplified MaRIA score among readers was computed. Interreader agreement was assessed with Fleiss kappa coefficients and intra-class correlation coefficient (ICC). Analysis included the Mann-Whitney U test, Friedman test, and Spearman rank correlation. RESULTS. Simplified MaRIA scores (across time points and study groups) had substantial interreader agreement (κ = 0.65 [95% CI, 0.56-0.74]; ICC, 0.71 [95% CI, 0.63-0.78]). Median scores were higher in participants with Crohn disease at baseline than in healthy control participants (3.5 [IQR, 2.5-4.9] vs 0.5 [IQR, 0-2.0]; p < .001). Scores decreased after medical treatment in participants with Crohn disease (p = .005). The median score was 3.5 (IQR, 2.5-4.9) at baseline, 2.3 (IQR, 1.6-3.9) at 6 weeks, and 2.0 (IQR, 0.5-2.5) at 6 months. In participants with Crohn disease, median scores had significant correlations with wPCDAI (ρ = 0.46 [95% CI, 0.18-0.64]; p < .001) and CRP level (ρ = 0.48 [95% CI, 0.27-0.65]; p < .001). CONCLUSION. Radiologists had substantial agreement in use of simplified MaRIA scores to assess intestinal inflammation in ileal Crohn disease. Scores changed over time after medical therapy. CLINICAL IMPACT. The results support the simplified MaRIA score as an objective MRI-based clinical measure of intestinal inflammation in children and young adults with Crohn disease.

68Ga-PSMA-11 PET/MRI versus multiparametric MRI in men referred for prostate biopsy: primary tumour localization and interreader agreement

BackgroundMagnetic resonance imaging (MRI) is recommended by the European Urology Association guidelines as the standard modality for imaging-guided biopsy. Recently positron emission tomography with prostate-specific membrane antigen (PSMA PET) has shown promising results as a tool for this purpose. The aim of this study was to compare the accuracy of positron emission tomography with prostate-specific membrane antigen/magnetic resonance imaging (PET/MRI) using the gallium-labeled prostate-specific membrane antigen (68Ga-PSMA-11) and multiparametric MRI (mpMRI) for pre-biopsy tumour localization and interreader agreement for visual and semiquantitative analysis. Semiquantitative parameters included apparent diffusion coefficient (ADC) and maximum lesion diameter for mpMRI and standardized uptake value (SUVmax) and PSMA-positive volume (PSMAvol) for PSMA PET/MRI.ResultsSensitivity and specificity were 61.4% and 92.9% for mpMRI and 66.7% and 92.9% for PSMA PET/MRI for reader one, respectively. RPE was available in 23 patients and 41 of 47 quadrants with discrepant findings. Based on RPE results, the specificity for both imaging modalities increased to 98% and 99%, and the sensitivity improved to 63.9% and 72.1% for mpMRI and PSMA PET/MRI, respectively. Both modalities yielded a substantial interreader agreement for primary tumour localization (mpMRI kappa = 0.65 (0.52–0.79), PSMA PET/MRI kappa = 0.73 (0.61–0.84)). ICC for SUVmax, PSMAvol and lesion diameter were almost perfect (≥ 0.90) while for ADC it was only moderate (ICC = 0.54 (0.04–0.78)). ADC and lesion diameter did not correlate significantly with Gleason score (ρ = 0.26 and ρ = 0.16) while SUVmax and PSMAvol did (ρ = − 0.474 and ρ = − 0.468).ConclusionsPSMA PET/MRI has similar accuracy and reliability to mpMRI regarding primary prostate cancer (PCa) localization. In our cohort, semiquantitative parameters from PSMA PET/MRI correlated with tumour grade and were more reliable than the ones from mpMRI.

Inter-reader agreement of multi-variable MR evaluation of Placenta Accreta Spectrum (PAS) and association with cesarean hysterectomy

• Multiple PAS-MR characteristics were significantly associated with hysterectomy . . • Cesarean hysterectomy was significantly correlated to FIGO classifications. . • Substantial inter-reader agreement between an expert and an intermediate reader. .

Comparison of muscle fat fraction measurements in the lower spine musculature with non-contrast-enhanced CT and different MR imaging sequences

PurposeTo assess whether two-point Dixon (TPD) MRI, true fast imaging with steady-state free precession (TRUFI) MRI and non-contrast-enhanced CT (NECT) can accurately measure muscle fat fraction (FF) in the autochthonous back muscles (AM) and the psoas muscle (PM) compared to multi-point Dixon (MPD) MRI. Method29 oncological patients who received MRI including MPD, TPD and NECT imaging in a period of three months were analyzed retrospectively. A sub-cohort of 16 patients additionally underwent TRUFI MRI and were included in a sub-analysis. Region of interest (ROI) measurements for each muscle compartment of the AM and PM were conducted by two examiners. Additionally, the Goutallier classification was used to quantify the amount of fatty infiltration of each muscle. Intermodality correlations were assessed with the Pearson correlation coefficient (r), and interreader and intrareader agreements with the intraclass correlation coefficient (ICC). ResultsGood intermodality correlations were found for NECT (r = 0.969), TPD (r = 0.942) and TRUFI (r = 0.904, all P < 0.001) when assessing FF in the AM and slightly lower in the PM. Interreader agreement showed good correlations and low median deviations (1.1 – 4.1 %, depending on the modality). The Goutallier classification of the AM showed good separation between grades with substantial interreader agreement (κ = 0.627, P < 0.001). ConclusionsROI measurements of the AM in NECT, TPD and TRUFI highly correlate with muscle FF measurements in MPD MRI and may be used to assess sarcopenia in oncological patients.

Accelerated Two-Point Dixon MR Angiography Improves Diagnostic Performance for Cervical Artery Diseases.

Nonenhanced MR angiography (MRA) studies are often used to manage acute and chronic large cervical artery disease, but lengthy scan times limit their clinical usefulness. To develop an accelerated cervical MRA and test its diagnostic performance. Prospective. Patients with cervical artery disease (n= 32, 17 males). 3.0 T; accelerated two-point Dixon three-dimensional Cartesian spoiled gradient-echo (FLEXA) and conventional time-of-flight MRA (cMRA) sequences. All patients underwent FLEXA (1'28″) and cMRA (6'47″) acquisitions. Quantitative evaluation (artery-to-background signal ratio and a blur metric) and qualitative evaluation using diagnostic performance measured by the sensitivity, specificity, and positive/negative predictive values (PPV/NPV), and vessel and plaque visualization scores from three board-certified radiologists' (with 10, 11, and 12 years of experience) independent readings using maximum intensity projection (MIP) for luminal diseases and axial images for plaque. The reference standards were contrast-enhanced angiography and fat-saturated T1-weighted images, respectively. All measures were compared between FLEXA and cMRA using the paired t, Wilcoxon signed-rank, McNemar's, or chi-squared test, as appropriate. Interreader agreement was assessed using Cohen's κ. P< 0.05 was considered statistically significant. The artery-to-background signal ratio was significantly higher for FLEXA (FLEXA: 7.20 ± 1.63 [fat]; 4.26 ± 0.52 [muscle]; cMRA: 2.57 ± 0.49 [fat]), while image blurring was significantly less (FLEXA: 0.24 ± 0.016; cMRA: 0.30 ± 0.029). In luminal disease detection, sensitivity (FLEXA: 0.97/0.91/0.91; cMRA:0.71/0.69/0.63), specificity (FLEXA: 0.98/0.93/0.98; cMRA:0.93/0.85/0.92), PPV (FLEXA: 0.92/0.86/0.86; cMRA: 0.64/0.5/0.58), and NPV (FLEXA: 0.99/0.98/0.98; cMRA: 0.92/0.91/0.9) were significantly higher for FLEXA. interreader agreement was substantial to almost perfect for FLEXA (κ= 0.82/0.86/0.78) and moderate to substantial for cMRA (κ= 0.67/0.56/0.57). MIP visualization scores were significantly higher for FLEXA, with substantial to almost perfect interreader agreement (FLEXA: κ= 0.83/0.86/0.82; cMRA: κ= 0.89/0.79/0.79). In plaque detection, sensitivity (FLEXA: 0.9/0.9/0.7; cMRA: 0.3/0.6/0.2) and specificity (FLEXA: 1/0.87/1; cMRA: 0.93/0.63/0.97) were significantly higher for FLEXA in two of three readers. The interreader plaque detection agreement was fair to substantial (FLEXA: κ= 0.63/0.69/0.48; cMRA: κ= 0.21/0.45/0.20). Side-by-side plaque and vessel wall visualization was superior for FLEXA in all readers, with moderate to substantial interreader agreement (plaque: κ= 0.73/0.73/0.77; vessel wall: κ= 0.57/0.40/0.39). FLEXA enhanced visualization of the cervical arterial system and improved diagnostic performance for luminal abnormalities and plaques in patients with cervical artery diseases. 1 TECHNICAL EFFICACY STAGE: 2.

PET/MR Imaging in Evaluating Treatment Failure of Head and Neck Malignancies: A Neck Imaging Reporting and Data System-Based Study.

PET/MR imaging is a relatively new hybrid technology that holds great promise for the evaluation of head and neck cancer. The aim of this study was to assess the performance of simultaneous PET/MR imaging versus MR imaging in the evaluation of posttreatment head and neck malignancies, as determined by its ability to predict locoregional recurrence or progression after imaging. The electronic medical records of patients who had posttreatment PET/MR imaging studies were reviewed, and after applying the exclusion criteria, we retrospectively included 46 studies. PET/MR imaging studies were independently reviewed by 2 neuroradiologists, who recorded scores based on the Neck Imaging Reporting and Data System (using CT/PET-CT criteria) for the diagnostic MR imaging sequences alone and the combined PET/MR imaging. Treatment failure was determined with either biopsy pathology or initiation of new treatment. Statistical analyses including univariate association, interobserver agreement, and receiver operating characteristic analysis were performed. There was substantial interreader agreement among PET/MR imaging scores (κ = 0.634; 95% CI, 0.605-0.663). PET/MR imaging scores showed a strong association with treatment failure by univariate association analysis, with P < .001 for the primary site, neck lymph nodes, and combined sites. Receiver operating characteristic curves of PET/MR imaging scores versus treatment failure indicated statistically significant diagnostic accuracy (area under curve range, 0.864-0.987; P < .001). Simultaneous PET/MR imaging has excellent discriminatory performance for treatment outcomes of head and neck malignancy when the Neck Imaging Reporting and Data System is applied. PET/MR imaging could play an important role in surveillance imaging for head and neck cancer.

Substantial interreader agreement for biopsy with reduction in biopsy rate: A multireader diagnostic performance study of ACR TI-RADS

PurposeTo assess diagnostic performance of ACR TI-RADS in thyroid cancer detection and evaluate interobserver agreement among radiologists with lower interpreting experience. MethodsFour radiologists retrospectively assessed 295 biopsied thyroid nodules from ultrasound studies performed between 2009 and 2019, blinded to histopathology. Diagnostic performance for cancer detection was determined individually, and interobserver agreement among four readers was evaluated with Fleiss kappa coefficient (ⱪ). Results245 (83.1%) benign and 50 (16.9%) malignant nodules were evaluated. Diagnostic performance based on final TR level was consistent and without significant difference among four readers, with excellent sensitivity (≥98.0%) and negative predictive value (NPV) [≥94.4%] for TR levels 3 to 5. Diagnostic performance based on recommendation to biopsy has moderate sensitivity (≥62%) and high NPV (≥84.7%). Retrospective scoring with established ACR TI-RADS criteria would have substantially reduced the number of biopsies performed, with 63.2% of malignancy not biopsied meeting criteria for sonographic surveillance. Interobserver agreement on TI-RADS scoring for final TR level was fair (ⱪ = 0.39, 95% CI 0.32, 0.47), with substantial agreement for recommendation to biopsy (ⱪ = 0.64, 95% CI 0.58, 0.70). ConclusionsSubstantial reduction in biopsy rate (up to 48%) would have been achieved using the ACR TI-RADS criteria, with 63% of malignancy not biopsied meeting criteria for sonographic surveillance. Interobserver agreement was fair for TI-RADS level scoring and substantial for recommendation to biopsy.