Interobserver Reproducibility Research Articles

World Health Organization Reporting System for Soft Tissue Cytopathology: Risk of malignancy and reproducibility of categories among observers.

In 2024, the World Health Organization (WHO) is scheduled to publish the WHO Reporting System for Soft Tissue Cytopathology (WHORSSTC). This system establishes categories with well-defined definitions, criteria, and estimated risks of malignancy (ROMs) for soft tissue tumors. The estimates of ROM are based on a relatively small number of published studies. Interobserver reproducibility is not addressed in the reporting system even though reproducibility of a reporting system is highly important. A manual search of one authors personal consultation files and teaching set (L.J.L.) was conducted for all cytologic specimens of soft tissue tumors accessioned between January 1, 1985 and December 31, 2022. Only cases with documented surgical pathology follow-up were included in the study. Slides from each case were evaluated independently by three cytopathologists with each case assigned to one of the WHORSSTC categories. A ROM for each of the WHORSSTC categories was calculated. Interobserver agreement was evaluated by the kappa and weighted kappa statistics. Risk for malignancy by category were: Category 1: 0%, Category 2: 28%, Category 3: 57%, Category 4: 47%, Category 5: 63%, and Category 6: 88%. Kappa statistics for agreement between raters varied from 0.2183 to 0.3465 and weighted kappa varied from 0.3778 to 0.5217. The WHORSSTC showed a progression of malignancy risk from the category "benign" (28%) to the category "malignant" (88%). Interobserver agreement was only fair.

Low irisin levels predict acutely decompensated heart failure in type 2 diabetes mellitus patients

Abstract Funding Acknowledgements None. Background Acutely decompensated heart failure (ADHF) continues to be associated with unacceptably increasing in-hospital mortality rates and one-year mortality rates. Distinct scenarios of the natural course of ADHF relate to clinical heterogeneity among patients admitted to hospitals, cardiac dysfunction etiology, precipitating factors contributing to heart failure (HF) decompensation including type 2 diabetes mellitus (T2DM). The aim of this study was to determine the discriminative value of irisin for ADHF in T2DM patients with hemodynamically stable chronic HF. Methods A total of 738 patients with T2DM were prescreened using the local database of "Vita Center" (Ukraine). Using criteria of inclusion (male/female with age of ≥18 years, established T2DM with hemodynamically stable chronic HF, glycosylated hemoglobin < 6.9%, informed consent to participate in the study), we enrolled 489 patients with T2DM with concomitant chronic HF I-IV New York Heart Association (NYHA) functional classes. We enrolled 480 T2DM patients with any phenotype of HF and followed them for 52 weeks. Hemodynamic performances and the serum levels of biomarkers were detected at the study entry. The primary clinical end-point was ADHF that led to urgent hospitalization. Results We found that the serum levels of N-terminal natriuretic pro-peptide (NT-proBNP) were higher (1719 [980-2457] pmol/mL vs. 1057 [570-2607] pmol/mL, respectively) and the levels of irisin were lower (4.96 [3.14-6.85] ng/mL vs. 7.95 [5.73-9.16] ng/mL) in ADHF patients than in those without ADHF. The ROC curve analysis showed that the estimated cut-off point for serum irisin levels (ADHF versus non-ADHF) was 7.85 ng/mL (area under curve [AUC] = 0.869 (95% CI = 0.800-0.937), sensitivity = 82.7%, specificity = 73.5%; p = 0.0001). The multivariate logistic regression yielded that the serum levels of irisin < 7.85 ng/mL (OR = 1.20; p = 0.001) and NT-proBNP > 1215 pmol/mL (OR = 1.18; p = 0.001) retained the predictors for ADHF. Kaplan-Meier plots (Figure) showed a significant difference of clinical end-point accumulations in patients with HF depending on irisin levels (<7.85 ng/mL versus ≥7.85 ng/mL). The intra-class correlation coefficient for inter-observer reproducibility of LV dimensions was 0.88 (95% CI = 0.83–0.92), of LVEF was 0.93 (95% CI = 0.90–0.97), of LAVI was 0.92 (95% CI = 0.89–0.94), and of E/e’ was 0.90 (95% CI = 0.87–0.94). Along with it, the intra-class correlation coefficient for the intra-observer reproducibility of LV dimensions was 0.91 (95% CI = 0.88–0.95), of LVEF was 0.94 (95% CI = 0.90–0.98), of LAVI was 0.95 (95% CI = 0.93–0.97), and of E/e’ was 0.92 (95% CI = 0.90–0.95). In conclusion, we established that decreased levels of irisin were associated with ADHF presentation in chronic HF patients with T2DM independently from NT-proBNP.Flow chart of the study designThe Kaplan–Meier analysis of ADHF

Assessing the reproducibility and predictive value of objective cough measurement for successful withdrawal of invasive ventilatory support in adult patients

BackgroundUtilizing clinical tests, such as objective cough measurement, can assist in predicting the success of the weaning process in critically ill patients.MethodsA multicenter observational analytical study was conducted within a prospective cohort of patients recruited to participate in COBRE-US. We assessed the capability of objective cough measurement to predict the success of the spontaneous breathing trial (SBT) and extubation. Intra- and inter-observer reproducibility of the cough test and was evaluated using the intraclass correlation coefficient (ICC) and Cohen’s weighted kappa. We used receiver operating characteristic curves (ROC-curve) to evaluate the predictive ability of objective cough measurement.ResultsWe recruited 367 subjects who were receiving invasive mechanical ventilation. A total of 451 objective cough measurements and 456 SBTs were conducted. A significant association was found between objective cough measurement and successful SBT (OR: 1.68; 95% CI 1.48–1.90; p = 0.001). The predictive capability of the objective cough test for SBT success had a ROC-curve of 0.58 (95% CI: 0.56–0.61). Objective cough measurement to predict successful extubation had a ROC-curve of 0.61 (95% CI: 0.56–0.66). The intraobserver reproducibility exhibited an ICC of 0.94 (95% CI: 0.89–0.96; p < 0.001), while the interobserver reproducibility demonstrated an ICC of 0.72 (95% CI: 0.51–0.85; p < 0.001). The intraobserver agreement, assessed using Cohen’s weighted kappa was 0.94 (95% CI: 0.93–0.99; p < 0.001), whereas the interobserver agreement was 0.84 (95% CI: 0.67 − 0.10; p < 0.001).ConclusionsThe objective measurement of cough using the method employed in our study demonstrates nearly perfect intra-observer reproducibility and agreement. However, its ability to predict success or failure in the weaning process is limited.

Abstract PO5-14-08: External Reproducibility of PD-L1 IHC 22C3 pharmDx on Breast Cancer Specimens at CPS ≥ 1 and CPS ≥ 10 Cutoffs

Abstract Background: PD-L1 IHC 22C3 pharmDx is an approved companion diagnostic for pembrolizumab treatment in triple-negative breast cancer (TNBC). Clinical evidence for pembrolizumab approval in TNBC was demonstrated by KEYNOTE-355 (NCT02819518), which investigated the clinical validity of PD-L1 IHC 22C3 pharmDx in identifying patients with PD-L1 expressing Combined Positive Score (CPS) ≥ 10 in previously untreated locally recurrent unresectable or metastatic TNBC.1 The role of immune checkpoint inhibitors for breast cancer as a whole is not as well defined as it is for TNBC. In this study, we show evidence of the reproducibility of PD-L1 IHC 22C3 pharmDx in PD-L1 expression determination in breast cancer specimens at the CPS ≥ 1 and CPS ≥ 10 cutoffs documenting robust assay performance in breast cancer. Methods: A blinded, randomized study was conducted at three external sites using formalin-fixed, paraffin-embedded breast cancer specimens. Reproducibility of PD-L1 IHC 22C3 pharmDx was assessed at the inter-site, intra-site/inter-day, inter-observer, and intra-observer endpoints at the CPS ≥ 1 and CPS ≥ 10 cutoffs. To assess inter-site and intra-site reproducibility, five replicate sets of pre-qualified unstained specimens were stained and evaluated at each of the three external sites. To assess inter-observer and intra-observer reproducibility, one set of pre-stained specimens was evaluated three times by each of three external pathologists. Negative percent agreement (NPA), positive percent agreement (PPA), and overall percent agreement (OA), based on comparisons to consensus diagnostic outcome, were computed with corresponding two-sided 95% percentile bootstrap confidence intervals. Acceptance criteria required at least 85.0% for the lower-bound (LB) of the two-sided 95% confidence interval (CI) on NPA, PPA, and OA for all endpoints. Results: All but one endpoint in the external reproducibility study met pre-determined acceptance criteria. Both the inter- and intra-site reproducibility endpoints had 95% CI LB values of 98.8% for NPA, 95.9% for PPA, and 97.9% OA at the CPS ≥ 1 cutoff. At the CPS ≥ 10 cutoff, the inter-site reproducibility endpoint had 95% CI LB values of 88.0% for NPA, 87.1% for PPA, and 89.5% OA, and the intra-site reproducibility endpoint had 95% CI LB values of 96.7% for NPA, 92.2% for PPA, and 95.6% OA. The inter-observer reproducibility endpoint had 95% CI LB values of 95.3% for NPA, 92.3% for PPA, and 94.7% OA at the CPS ≥ 1 cutoff. At the CPS ≥ 10 cutoff, the inter-observer reproducibility endpoint had 95% CI LB values of 84.9% for NPA, 86.8% for PPA, and 87.3% OA. The intra-observer reproducibility endpoint had 95% CI LB values of 97.0% for NPA, 95.0% for PPA, and 96.6% OA at the CPS ≥ 1 cutoff. At the CPS ≥ 10 cutoff, the intra-observer reproducibility endpoint had 95% CI LB values of 92.2% for NPA, 89.2% for PPA, and 91.7% OA. Conclusions: Acceptance criteria were met for all endpoints in the breast cancer external reproducibility study of PD-L1 IHC 22C3 pharmDx at the CPS ≥ 1 and CPS ≥ 10 cutoffs, except for inter-observer at the CPS ≥ 10 cutoff, which had a 95% CI LB value of 84.9% for NPA.

Feasibility and reproducibility of cardiovascular magnetic resonance-feature tracking for quantitative right atrial function in dilated cardiomyopathy patients.

The importance of right heart assessment in dilated cardiomyopathy (DCM) is increasingly recognized. The development of cardiovascular magnetic resonance-feature tracking (CMR-FT) has provided a novel approach to quantify myocardial deformation and evaluate cardiac function. In this study, we aimed to evaluate the feasibility and reproducibility of CMR-FT for the quantitative derivation of right atrial (RA) strain and strain rate (SR) in patients with DCM. A total of 68 DCM patients (84% male; aged 50.6±13.2 years) and 58 healthy controls (81% male; aged 48.4±11.2 years) were retrospectively enrolled from September 2018 to August 2022 at the First Affiliated Hospital of Zhejiang Chinese Medical University and Shenzhen Clinical Medical College of Guangzhou University of Chinese Medicine. RA reservoir, conduit, and booster strain (εs, εe, and εa) and peak positive, peak early negative, and peak late negative SR (SRs, SRe, and SRa) were measured using CMR-FT and compared between 2 groups using Student's t-test. Intra- and inter-observer reproducibility was evaluated using intraclass correlation coefficients (ICC) and Bland-Altman plots. Compared to healthy controls, DCM patients showed significantly lower RA strain (εs: 19.7%±9.0% vs. 44.4%±9.7%; εe: 7.9%±5.3% vs. 25.8%±8.6%; εa: 11.8%±6.2% vs. 18.6%±5.1%, all P<0.001) and SR (SRs: 1.17±0.48 vs. 1.92±0.62 s-1; SRe: -0.85±0.56 vs. -1.94±0.63 s-1; SRa: -1.39±0.71 vs. -2.01±0.65 s-1, all P<0.001). There was no significant difference in RA maximum volume index between the 2 groups. Simple linear regression analysis demonstrated a significant correlation between N-terminal B-type natriuretic peptide (NT-proBNP), RA emptying fraction passive (RAEF passive), and RA εe [(NT-proBNP and εe): r=-0.48, P<0.001, 95% confidence interval (CI): -0.64 to -0.26; and (RAEF passive and εe): r=0.41, P=0.001, 95% CI: 0.22 to 0.56, respectively] in DCM patients. Intra- and inter-observer reproducibility was excellent (all ICCs >0.85) for RA deformation measurements. CMR-FT is a promising, noninvasive approach for the quantitative assessment of RA phasic function in patients with DCM. DCM patients exhibit impaired RA reservoir, conduit, and booster pump function prior to visible RA enlargement.

Improving the efficiency of single-isocenter multiple metastases stereotactic radiosurgery treatment

PurposeMultiple brain metastases can be treated efficiently with Stereotactic Radio Surgery (SRS) using a Single Isocenter Dynamic Conformal Arc (SIDCA) technique. Currently plans are manually optimized, which may lead to unnecessary table angles and arcs being used. This study aimed to evaluate an automatic 4π-optimization SIDCA algorithm for treatment efficiency and plan quality.Methods and MaterialsAutomatic 4π-optimized SIDCA plans were created and compared to the manually optimized clinical plans for 54 patients who received single-fraction SRS to 2-10 metastases. The number of table angles and arcs were compared with a paired t-test using a Bonferroni-corrected significance level of p<0.05/4=0.0125. The reduction in treatment time was estimated from the difference in the number of table angles and arcs. Plan quality was assessed through the volume-averaged inverse Paddick Conformity Index (CI) and Gradient Index (GI), and the volume of normal brain surrounding each metastasis receiving 12 Gy (local V12 Gy). For a five-patient subset the automatic plans were manually adjusted further. CI and GI were assessed for non-inferiority using a one-sided t-test with the non-inferiority limit equal to the 95% inter-observer reproducibility limit from a separate planning study (corrected significance level p<0.05/(4−1)=0.017).ResultsThe automatic plans significantly improved treatment efficiency with a mean reduction in the number of table angles and arcs of −0.5±0.1 and −1.3±0.2 respectively (± standard error, both p<0.001). Estimated treatment time saving was −2.7±0.5minutes, 14% of the total. The volume-averaged CI and GI were non-inferior to the clinical plans (both p<0.001), although there was a small systematic shift in CI of 0.07±0.01. The resulting difference in local V12 Gy, 0.25±0.04cm3, was not clinically significant. Minor manual adjustment of the automatic plans removed these slight differences whilst preserving the improved treatment efficiency.ConclusionsAutomatic 4π-optimization can generate SIDCA SRS plans with improved treatment efficiency and non-inferior plan quality.

Establishment and preliminary evaluation of CT-based classification for distal radius fracture

Establish a new classification system of distal radius fracture based on computed tomographic (CT), and evaluate its reliability and reproducibility preliminarily, and provide a new theoretical reference for clinicians to use the clinical classification system. The imaging data and clinical data of 204 inpatients with distal radius fracture during 6 years from January 1, 2014 to January 1, 2019 in orthopaedic department were analyzed retrospectively and classified based on CT. Three observers evaluated the image data of 48 randomly selected cases based on CT at different time nodes of T1 and T2. Cohen's kappa was used to calculate the consistency. At the last follow-up, patients' Disabilities of the Arm, Shoulder and Hand (DASH), Patient Rated Wrist Evaluation (PRWE), and VAS scores were collected. Among 204 cases, there were 12 cases of type 1, including 6 cases of type 1-D, 4 cases of type 1-V and 2 cases of type 1-R. There were 6 cases of type 2, including 2 cases of type 2-DV, 2 cases of type 2-DR and 2 cases of type 2-VR. There were 186 cases of type 3, including 32 cases of type 3–0, 127 cases of type 3–1 and 27 cases of type 3–2. There was no significant difference in DASH, PRWE and VAS scores among all types (P > 0.05). The results of interobserver reproducibility were kappa = 0.985, ICC = 0.984 in the first evaluation, kappa = 0.986, ICC = 0.986 in the second evaluation. The results of intraobserver reproducibility were O1 = 0.991, O2 = 0.991, O3 = 0.989 respectively. The new classification system of distal radius fracture based on CT has theoretical and practical significance for incision selection, fracture reduction and internal fixation. 123 classification system is clear, comprehensive, easy to understand and remember. Moreover, it has higher interobserver reliability and intraobserver reproducibility than other systems reported at present.

Pulmonary Transit Time Assessment by CEUS in Healthy Rabbits: Feasibility, and the Effects of UCAs Dilution Concentration.

To evaluate the inter-observer variability and the intra-observer repeatability of pulmonary transit time (PTT) measurement using contrast-enhanced ultrasound (CEUS) in healthy rabbits, and assess the effects of dilution concentration of ultrasound contrast agents (UCAs) on PTT. Thirteen healthy rabbits were selected, and five concentrations UCAs of 1:200, 1:100, 1:50, 1:10, and 1:1 were injected into the right ear vein. Five digital loops were obtained from the apical 4-chamber view. Four sonographers obtained PTT by plotting the TIC of right atrium (RA) and left atrium (LA) at two time points (T1 and T2). The frame counts of the first appearance of UCAs in RA and LA had excellent inter-observer agreement, with intra-class correlations (ICC) of 0.996, 0.988, respectively. The agreement of PTT among four observers was all good at five different concentrations, with an ICC of 0.758-0.873. The reproducibility of PTT obtained by four observers at T1 and T2 was performed well, with ICC of 0.888-0.961. The median inter-observer variability across 13 rabbits was 6.5% and the median variability within 14 days for 4 observers was 1.9%, 1.7%, 2.2%, 1.9%, respectively; The PTT of 13 healthy rabbits is 1.01 ± 0.18 second. The difference of PTT between five concentrations is statistically significant. The PTT obtained by a concentration of 1:200 and 1:100 were higher than that of 1:1, while there were no significantly differences in PTT of a concentration of 1:1, 1:10, and 1:50. PTT measured by CEUS in rabbits is feasible, with excellent inter-observer and intra-observer reliability and reproducibility, and dilution concentration of UCAs influences PTT results.

A Study on Interobserver and Intraobserver Reliability of the Huashan Radiologic Classification System for Cervical Spinal Cord Injury Without Fracture and Dislocation.

Observational study. To assess the reproducibility and reliability of the system. The Huashan radiologic classification system for cervical spinal cord injury without fracture and dislocation (CSCIWFD) was recently proposed and found useful for clinical practice. Patients diagnosed with CSCIWFD between 2015 and 2021 were recruited. Six spine surgeons from different institutions, three experienced and other inexperienced respectively, were trained as observers of the system, and these surgeons classified the recruited patients using the system. Then, 8 weeks later, they repeated the classification on the same patients in a different order. The interobserver and intraobserver agreement between the results was analyzed using percentage agreement, weighted kappa, and Cohen kappa (κ) statistics. A total of 60 patients were included in the analysis. Type I was the most frequent type (29 cases, 48.3%), followed by type II (13 cases, 21.7%), type III (12 cases, 20%), and type IV (6 cases, 10%). For all the observers, experienced observers, and inexperienced observers, the overall agreement percentages were 77.6% (κ = 0.78), 84.4% (κ = 0.84), and 72.8% (κ = 0.74), respectively, indicating substantial to nearly perfect interobserver reproducibility. A higher level of agreement was found for differentiating type I from other types, with the percentage agreement ranging from 87.8% to 94.4% (κ= 0.74-0.88). For distinguishing compression on the spinal cord (types I and II vs types III and IV) among the different groups of observers, the percentage agreement was 97.8% (κ = 0.94), indicating nearly perfect reproducibility. As for intraobserver agreement, the percentage agreement ranged from 86.7% to 96.7% (κ = 0.78-0.95), indicating at least substantial reliability. The Huashan radiologic classification system for CSCIWFD was easy to learn and apply in a clinical environment, showing excellent reproducibility and reliability. Therefore, it would be promising to apply and promote this system for the precise evaluation and personalized treatment strategy.

Performance evaluation of commercial and non-commercial shear wave elastography implementations for vascular applications

BackgroundShear wave elastography (SWE) is mainly used for stiffness estimation of large, homogeneous tissues, such as the liver and breasts. However, little is known about its accuracy and applicability in thin (∼0.5–2 mm) vessel walls. To identify possible performance differences among vendors, we quantified differences in measured wave velocities obtained by commercial SWE implementations of various vendors over different imaging depths in a vessel-mimicking phantom. For reference, we measured SWE values in the cylindrical inclusions and homogeneous background of a commercial SWE phantom. Additionally, we compared the accuracy between a research implementation and the commercially available clinical SWE on an Aixplorer ultrasound system in phantoms and in vivo in patients. MethodsSWE measurements were performed over varying depths (0–35 mm) using three ultrasound machines with four ultrasound probes in the homogeneous 20 kPa background and cylindrical targets of 10, 40, and 60 kPa of a multi-purpose phantom (CIRS-040GSE) and in the anterior and posterior wall of a homogeneous polyvinyl alcohol vessel-mimicking phantom. These phantom data, along with in vivo SWE data of carotid arteries in 23 patients with a (prior) head and neck neoplasm, were also acquired in the research and clinical mode of the Aixplorer ultrasound machine. Machine-specific estimated phantom stiffness values (CIRS phantom) or wave velocities (vessel phantom) over all depths were visualized, and the relative error to the reference values and inter-frame variability (interquartile range/median) were calculated. Correlations between SWE values and target/vessel wall depth were explored in phantoms and in vivo using Spearman’s correlations. Differences in wave velocities between the anterior and posterior arterial wall were assessed with Wilcoxon signed-rank tests. Intra-class correlation coefficients were calculated for a sample of ten patients as a measure of intra- and interobserver reproducibility of SWE analyses in research and clinical mode. ResultsThere was a high variability in obtained SWE values among ultrasound machines, probes, and, in some cases, with depth. Compared to the homogeneous CIRS-background, this variation was more pronounced for the inclusions and the vessel-mimicking phantom. Furthermore, higher stiffnesses were generally underestimated. In the vessel-mimicking phantom, anterior wave velocities were (incorrectly) higher than posterior wave velocities (3.4–5.6 m/s versus 2.9–5.9 m/s, p ≤ 0.005 for 3/4 probes) and remarkably correlated with measurement depth for most machines (Spearman’s ρ = -0.873–0.969, p < 0.001 for 3/4 probes). In the Aixplorer’s research mode, this difference was smaller (3.3–3.9 m/s versus 3.2–3.6 m/s, p = 0.005) and values did not correlate with measurement depth (Spearman’s ρ = 0.039–0.659, p ≥ 0.002). In vivo, wave velocities were higher in the posterior than the anterior vessel wall in research (left p = 0.001, right p < 0.001) but not in clinical mode (left: p = 0.114, right: p = 0.483). Yet, wave velocities correlated with vessel wall depth in clinical (Spearman’s ρ = 0.574–0.698, p < 0.001) but not in research mode (Spearman’s ρ = -0.080–0.466, p ≥ 0.003). ConclusionsWe observed more variation in SWE values among ultrasound machines and probes in tissue with high stiffness and thin-walled geometry than in low stiffness, homogeneous tissue. Together with a depth-correlation in some machines, where carotid arteries have a fixed location, this calls for caution in interpreting SWE results in clinical practice for vascular applications.

Effect of anti-VEGF on retinal blood flow in diabetic mice using laser speckle flowgraphy.

To assess intra- (repeatability) and inter-observer (reproducibility) variability of laser speckle flowgraphy (LSFG) for retinal blood flow (RBF) measurement in 20 eyes of wild type (C57BL/6J) mice and effect of intravitreal Aflibercept on RBF in optic nerve head (ONH) region of 10 eyes of Ins2 (Akita) diabetic mice. 'Mean blur rate (MBR)' was measured for all quadrants of tissue area (MT), vessel (MV) and total area (MA) of ONH region. Changes in MT were analysed at each timepoint. Repeatability was evaluated by measuring MBR variability without changing mouse head position, and reproducibility after resetting mouse head position by another operator. Coefficient of repeatability (CR) through Bland-Altman plot method coefficient of variation (COV) and Intraclass correlation coefficient (ICC) was calculated. Intravitreal Aflibercept (1 μg) was administered to Akita eyes and intraocular pressure (IOP) was measured using a tonometer at baseline, day 7, 14, 21 and 28 post-injection. Hurvich and Tsai's criterion was used. Coefficient of repeatability values of repeatability and reproducibility for all quadrants were within limits of agreement. Reliability was excellent (ICC 0.98-0.99) and reproducibility was moderate to excellent (ICC 0.64-0.96). There was a non-significant IOP increase in all Akita eyes at Day 28 (p > 0.05), and significant increase in MT in all quadrants at Day 21 and superior, inferior and temporal quadrants at Day 28 (p < 0.05). Laser speckle flowgraphy demonstrates excellent repeatability and moderate to excellent reproducibility in measuring RBF. Intravitreal Aflibercept injection results in a significant increase in MT up to 28 days post-injection without significant increase in IOP.

Interobserver Reliability of Milan and Modified Milan System for Reporting Salivary Gland Cytopathology: An Institutional Experience

Background: In 2018, the Milan System for Reporting Salivary Gland Cytopathology (MSRSGC) was published, defining a diagnostic categorization scheme. However, this system of classification was criticized due to its suboptimal diagnostic accuracy and low interobserver reliability. For these reasons, the modified Milan system was recently proposed by a few authors claiming it to be more clinically reliable. The present study aimed to analyze the interobserver reliability of MSRSGC and modified MSRSGC. Methods: A total of 100 salivary gland fine-needle aspirations were reviewed over a 1-year period and classified by two independent observers according to MSRSGC and modified MSRSGC. Interobserver reproducibility was estimated using observed agreement and chance-corrected agreement (Cohen’s kappa). Results: An overall chance corrected agreement of 0.37 (fair) was obtained for the original Milan system and 0.41 (moderate) for the modified one. In addition, subcategories of modified MSRSGC performed better than the MSRSGC in terms of interobserver reliability. Conclusion: The present study suggests that modified MSRSGC should be accepted worldwide as the clinical relevance of any categorization scheme requires diagnostic accuracy along with excellent interobserver reproducibility.

Reproducibility and prognostic ability of chronicity parameters in kidney biopsy – Comprehensive evaluation comparing microscopy and artificial intelligence in digital pathology

IntroductionSemi-quantitative scoring of various parameters in renal biopsy is accepted as an important tool to assess disease activity and prognostication. There are concerns on the impact of interobserver variability in its prognostic utility, generating a need for computerized quantification. MethodsWe studied 94 patients with renal biopsies, 45 with native diseases and 49 transplant patients with index biopsies for Polyomavirus nephropathy. Chronicity scores were evaluated using two methods. A standard definition diagram was agreed after international consultation and four renal pathologists scored each parameter in a double-blinded manner. Interstitial fibrosis (IF) score was assessed with five different computerized and AI-based algorithms on trichrome and PAS stains. ResultsThere was strong prognostic correlation with renal function and graft outcome at a median follow-up ranging from 24 to 42 months respectively, independent of moderate concordance for pathologists scores. IF scores with two of the computerized algorithms showed significant correlation with estimated glomerular filtration rate (eGFR) at biopsy but not at the end of follow-up. There was poor concordance for AI based platforms. ConclusionChronicity scores are robust prognostic tools despite interobserver reproducibility. AI-algorithms have absolute precision but are limited by significant variation when different hardware and software algorithms are used for quantification.

Mitotic activity: A systematic literature review of the assessment methodology and prognostic value in canine tumors.

One of the most relevant prognostic indices for tumors is cellular proliferation, which is most commonly measured by the mitotic activity in routine tumor sections. The goal of this systematic review was to analyze the methods and prognostic relevance of histologically measuring mitotic activity that have been reported for canine tumors in the literature. A total of 137 articles that correlated the mitotic activity in canine tumors with patient outcome were identified through a systematic (PubMed and Scopus) and nonsystematic (Google Scholar) literature search and eligibility screening process. Mitotic activity methods encompassed the mitotic count (MC, number of mitotic figures per tumor area) in 126 studies, presumably the MC (method not specified) in 6 studies, and the mitotic index (MI, number of mitotic figures per number of tumor cells) in 5 studies. A particularly high risk of bias was identified based on the available details of the MC methods and statistical analyses, which often did not quantify the prognostic discriminative ability of the MC and only reported P values. A significant association of the MC with survival was found in 72 of 109 (66%) studies. However, survival was evaluated by at least 3 studies in only 7 tumor types/groups, of which a prognostic relevance is apparent for mast cell tumors of the skin, cutaneous melanoma, and soft tissue tumor of the skin and subcutis. None of the studies using the MI found a prognostic relevance. This review highlights the need for more studies with standardized methods and appropriate analysis of the discriminative ability to prove the prognostic value of the MC and MI in various tumor types. Future studies are needed to evaluate the influence of the performance of individual pathologists on the appropriateness of prognostic thresholds and investigate methods to improve interobserver reproducibility.

Interobserver reproducibility of cervical histology interpretation with and without p16 immunohistochemistry.

Histopathological diagnosis of colposcopically identified cervical lesions is a critical step for the recognition of cervical cancer precursors requiring treatment. Although there have been efforts to standardize the histologic diagnosis of cervical biopsy specimens, in terms of terminology and use of biomarkers, there is no uniform approach in the pathology community. Adjunctive p16 immunohistochemistry (IHC) can highlight precancer diagnoses, with use recommendations outlined by the Lower Anogenital Squamous Terminology project. We assessed the diagnostic reproducibility of cervical histopathological biopsy specimens with and without p16 staining among 2 expert pathologists. Interpretation of p16 IHC as positive vs negative was highly reproducible (92.5% agreement, κ = 0.85); greater variation was seen in the choice of which biopsy specimens required adjunctive p16 staining (78.0% agreement, κ = 0.43). Adjunctive p16 IHC did not significantly increase diagnostic agreement under multitiered grading systems (benign vs cervical intraepithelial neoplasia [CIN] 1/low-grade squamous intraepithelial lesion vs atypical squamous metaplasia vs CIN2/high-grade squamous intraepithelial lesion [HSIL] vs CIN3/HSIL-CIN3 vs cancer) (65.5% agreement, κ = 0.56 without p16; 70.0% agreement, κ = 0.58 with p16). However, when dichotomizing diagnoses based on clinical management (less than HSIL vs HSIL+), diagnostic agreement increased with p16 IHC (90.5% agreement, κ = 0.79 without p16; 92.0% agreement, κ = 0.84 with p16). For biopsy specimens taken from women positive for human papillomavirus (HPV) type 16, agreement was similar with or without adjunctive p16 (κ = 0.80 without p16; κ = 0.78-0.80 with p16). In contrast, p16 IHC substantially improved diagnostic agreement for cervical biopsy specimens taken from women positive for other high-risk HPV strains, producing improvements in κ from 0.03 to 0.24. Adjunctive p16 immunostaining provides useful information in the evaluation of cervical biopsies for precancer. In our study, we have demonstrated that it is highly reproducible between 2 pathologists, although the decision of which biopsies warrant its use is less so. Furthermore, although p16 IHC showed a limited increase in diagnostic reproducibility for all biopsies included in our study, it did demonstrate a more sizable gain in biopsies negative for HPV 16 but positive for other high-risk genotypes. Further studies are needed to clarify the role of p16 IHC and how it can be optimized for the detection of cervical precancer, particularly in HPV-vaccinated populations where types other than HPV 16 are relatively more important.

Quantitative MRI assessment of joint effusion using T2-relaxometry at 3 Tesla: a feasibility and reproducibility study.

T2-relaxometry could differentiate between physiological and haemorrhagic joint effusion (≥ 5% blood) in vitro. Are quantitative T2-relaxation time measurements of synovial fluid feasible and reproducible in vivo in clinically bleed-free joints of men with haemophilia? In this cross-sectional study, we measured T2-relaxation times of synovial fluid in clinically bleed-free ankles, knees or elbows of men with severe haemophilia A using a T2-mapping sequence (duration ≤ 7min) at 3 Tesla MRI. Manual and circular regions of interest (ROI) were drawn in the synovial fluid of each joint by two independent observers to measure T2-relaxation times. Measurement feasibility was expressed as the success rate of the measurements by both observers. The interobserver and intraobserver reproducibility of the measurements were evaluated by the intraclass correlation coefficient of absolute agreement (ICC) and the limits of agreement (LoA) from Bland Altman analysis. We evaluated 39 clinically bleed-free joints (11 ankles, 12 knees, 16 elbows) of 39 men (median age, 24years; range 17-33) with severe haemophilia A. The success rate of the T2-measurements was ≥ 90%. Interobserver reliability was good to excellent (manual ROI: ICC = 0.92, 95% CI 0.76-0.97; circular ROI: ICC = 0.82, 95% CI 0.66-0.91) and interobserver agreement was adequate (manual ROI: LoA = 71ms; circular ROI: LoA = 146ms). Intraobserver reliability was good to excellent (manual ROI: ICC = 0.78, 95% CI - 0.06-0.94; circular RO:ICC = 0.99, 95% CI 0.98-0.99) and intraobserver agreement was good (manual ROI: LoA = 63ms; circular ROI: LoA = 41ms). T2-relaxometry of synovial fluid in haemophilia patients is feasible with good interobserver and intraobserver reproducibility.

Age- and Sex-Specific MR-Feature Tracking Reference Values of Right Atrial Deformation in Healthy Adults.

Although right atrial (RA) myocardial deformation has important implications for patient diagnosis, prognosis, and risk stratification, its implementation in clinical practice has been hampered by limited normal reference values, especially in Asian populations. To establish age- and sex-specific reference values for RA strain, strain rate (SR), and displacement based on a large sample of healthy Chinese adults using MR-feature tracking (MR-FT). Retrospective. 524 healthy Chinese adults (287 male; mean age 43.7 ± 11.9 years). 1.5T/balanced steady-state free precession. RA deformation parameters, including reservoir, conduit, and booster strain (εs, εe, and εa), peak positive, early negative, and late negative SR (SRs, SRe, and SRa), and total, passive, and active displacement (Ds, De, and Da), were assessed using MR-FT. Student's t-test, one-way ANOVA, coefficients of determination (r2 ), intraclass correlation coefficients (ICC), and Bland-Altman plots. A P value <0.05 was considered significant. Women demonstrated significantly greater magnitudes of RA deformation parameters than men: εs (57.4% ± 15.1% vs. 44.3% ± 12.6%), εe (37.5% ± 13.4% vs. 27.4% ± 10.9%), εa (19.9% ± 5.7% vs. 16.9% ± 5.0%), SRs (2.62 ± 0.88 sec-1 vs. 2.00 ± 0.63 sec-1 ), SRe (-2.98 ± 1.26 sec-1 vs. -2.16 ± 0.92 sec-1 ), SRa (-2.28 ± 0.75 sec-1 vs. -1.84 ± 0.62 sec-1 ), Ds (-7.80 ± 1.90 mm vs. -7.46 ± 1.70 mm), and De (-4.84 ± 1.31 mm vs. -4.49 ± 1.21 mm). For both sexes, aging was significantly associated with decreased RA reservoir and conduit function (εs, SRs, Ds, εe, SRe, and De), and with increased εa and Da. RA deformation measurements had good to excellent intraobserver and interobserver reproducibility, with ICCs ranging from to 0.790 to 0.972. This study provides age- and sex-specific reference values of RA strain, SR, and displacement based on a large cohort of healthy Chinese adults using MR-FT. 3 TECHNICAL EFFICACY: Stage 2.

Improvement in the estimation of perfusable tissue fraction and myocardial flow reserve in the ischemic myocardial lesions using ECG-gated dynamic myocardial PET with 15O-water.

Perfusable tissue fraction (PTF) and myocardial flow reserve (MFR) from 15O-water dynamic positron emission tomography (PET) are parameters of myocardial viability. However, myocardial motion causes errors in these values. We aimed to develop accurate estimation of PTF and MFR in ischemic lesions using an electro-cardiogram (ECG)-gated dynamic myocardial PET with 15O-water. Twenty-seven patients with ischemic heart disease were enrolled. All patients underwent percutaneous coronary intervention (PCI) or coronary artery bypass grafting (CABG). List mode 3D PET data and ECG signals were acquired using Philips Gemini TF64 instrument. For each scan, 500MBq of 15O-water was infused slowly for 2min, and the dynamic data were scanned for 6min. Both non-gated dynamic images and ECG-gated diastolic dynamic images were reconstructed. On the myocardial PET images of each patient, the entire myocardial region of interest (ROI) was set and divided into 17 segments. Myocardial blood flow in the resting state (rest MBF), hyperemic state (stress MBF), PTF, and MFR in each segment were estimated from both non-gated and ECG-gated dynamic PET images. Coronary arteriograms were obtained for all patients. In total, 128 normal segments without stenosis and 50 ischemic segments with > 90% stenosis were evaluated. In the ischemic myocardial segments, the PTF with ECG-gated PET was estimated as significantly lower than that with non-gated PET (0.63 ± 0.09 vs. 0.72 ± 0.08 [mL/mL], p < 0.001). The ECG-gated PET estimated a significantly lower PTF in the ischemic segments than in the normal segments (0.63 ± 0.09 vs. 0.67 ± 0.07 [mL/mL], p < 0.01). In the normal segments, the ECG-gated PET detected no significant difference in MFR compared with those from the non-gated PET (2.15 ± 0.76 vs. 2.24 ± 0.79, p = 0.28). However, in the ischemic myocardial segments, the MFR with ECG-gated PET was estimated as significantly lower than that with the non-gated PET (1.23 ± 0.29 vs. 1.69 ± 0.71, p < 0.001). The ECG-gated PET presented a significantly higher inter-observer reproducibility of PTF and rest MBF than the non-gated PET (p < 0.01). Neither stress MBF nor MFR yielded significant differences in inter-observer reproducibility between the ECG-gated and non-gated PET. The ECG-gated dynamic 15O-water PET suppressed the myocardial motion effect and resulted in a lower PTF and MFR in ischemic myocardial lesions than the non-gated PET. The ECG-gated PET seemed to be better than the conventional non-gated dynamic PET for the detection of ischemic myocardial lesion.

Characterization of renal masses with MRI-based radiomics: assessment of inter-package and inter-observer reproducibility in a prospective pilot study.

To evaluate radiomics features' reproducibility using inter-package/inter-observer measurement analysis in renal masses (RMs) based on MRI and to employ machine learning (ML) models for RM characterization. 32 Patients (23M/9F; age 61.8 ± 10.6years) with RMs (25 renal cell carcinomas (RCC)/7 benign masses; mean size, 3.43 ± 1.73cm) undergoing resection were prospectively recruited. All patients underwent 1.5T MRI with T2-weighted (T2-WI), diffusion-weighted (DWI)/apparent diffusion coefficient (ADC), and pre-/post-contrast-enhanced T1-weighted imaging (T1-WI). RMs were manually segmented using volume of interest (VOI) on T2-WI, DWI/ADC, and T1-WI pre-/post-contrast imaging (1-min, 3-min post-injection) by two independent observers using two radiomics software packages for inter-package and inter-observer assessments of shape/histogram/texture features common to both packages (104 features; n = 26 patients). Intra-class correlation coefficients (ICCs) were calculated to assess inter-observer and inter-package reproducibility of radiomics measurements [good (ICC ≥ 0.8)/moderate (ICC = 0.5-0.8)/poor (ICC < 0.5)]. ML models were employed using reproducible features (between observers and packages, ICC > 0.8) to distinguish RCC from benign RM. Inter-package comparisons demonstrated that radiomics features from T1-WI-post-contrast had the highest proportion of good/moderate ICCs (54.8-58.6% for T1-WI-1min), while most features extracted from T2-WI, T1-WI-pre-contrast, and ADC exhibited poor ICCs. Inter-observer comparisons found that radiomics measurements from T1-WI pre/post-contrast and T2-WI had the greatest proportion of features with good/moderate ICCs (95.3-99.1% T1-WI-post-contrast 1-min), while ADC measurements yielded mostly poor ICCs. ML models generated an AUC of 0.71 [95% confidence interval = 0.67-0.75] for diagnosis of RCC vs. benign RM. Radiomics features extracted from T1-WI-post-contrast demonstrated greater inter-package and inter-observer reproducibility compared to ADC, with fair accuracy for distinguishing RCC from benign RM. Knowledge of reproducibility of MRI radiomics features obtained on renal masses will aid in future study design and may enhance the diagnostic utility of radiomics models for renal mass characterization.

A Comparative Study of Three Systems for Liver Magnetic Resonance Elastography.

Different MR elastography (MRE) systems may produce different stiffness measurements, making direct comparison difficult in multi-center investigations. To assess the repeatability and reproducibility of liver stiffness measured by three typical MRE systems. Prospective. Thirty volunteers without liver disease history (20 males, aged 21-28)/5 gel phantoms. 3.0 T United Imaging Healthcare (UIH), 1.5 T Siemens Healthcare, 3.0 T General Electric Healthcare (GE)/Echo planar imaging-based MRE sequence. Wave images of volunteers and phantoms were acquired by three MRE systems. Tissue stiffness was evaluated by two observers, while phantom stiffness was assessed automatically by code. The reproducibility across three MRE systems was quantified based on the mean stiffness of each volunteer and phantom. Intraclass correlation coefficients (ICC), coefficients of variation (CV), and Bland-Altman analyses were used to assess the interobserver reproducibility, the interscan repeatability, and the intersystem reproducibility. Paired t-tests were performed to assess the interobserver and interscan variation. Friedman tests with Dunn's multiple comparison correction were performed to assess the intersystem variation. P values less than 0.05 indicated significant difference. The reproducibility of stiffness measured by the two observers demonstrated consistency with ICC > 0.92, CV < 4.32%, Mean bias < 2.23%, and P > 0.06. The repeatability of measurements obtained using the electromagnetic system for the liver revealed ICC > 0.96, CV < 3.86%, Mean bias < 0.19%, P > 0.90. When considering the range of reproducibility across the three systems for liver evaluations, results ranged with ICCs from 0.70 to 0.87, CVs from 6.46% to 10.99%, and Mean biases between 1.89% and 6.30%. Phantom studies showed similar results. The values of measured stiffness differed across all three systems significantly. Liver stiffness values measured from different MRE systems can be different, but the measurements across the three MRE systems produced consistent results with excellent reproducibility. 1 TECHNICAL EFFICACY: Stage 2.