Inter-pathologist Agreement Research Articles

Object-based feedback attention in convolutional neural networks improves tumour detection in digital pathology

Human visual attention allows prior knowledge or expectations to influence visual processing, allocating limited computational resources to only that part of the image that are likely to behaviourally important. Here, we present an image recognition system based on biological vision that guides attention to more informative locations within a larger parent image, using a sequence of saccade-like motions. We demonstrate that at the end of the saccade sequence the system has an improved classification ability compared to the convolutional neural network (CNN) that represents the feedforward part of the model. Feedback activations highlight salient image features supporting the explainability of the classification. Our attention model deviates substantially from more common feedforward attention mechanisms, which linearly reweight part of the input. This model uses several passes of feedforward and backward activation, which interact non-linearly. We apply our feedback architecture to histopathology patch images, demonstrating a 3.5% improvement in accuracy (p < 0.001) when retrospectively processing 59,057 9-class patches from 689 colorectal cancer WSIs. In the saccade implementation, overall agreement between expert-labelled patches and model prediction reached 93.23% for tumour tissue, surpassing inter-pathologist agreement. Our method is adaptable to other areas of science which rely on the analysis of extremely large-scale images.

A feasibility study using quantitative and interpretable histological analyses of celiac disease for automated cell type and tissue area classification

Histological assessment is essential for the diagnosis and management of celiac disease. Current scoring systems, including modified Marsh (Marsh–Oberhuber) score, lack inter-pathologist agreement. To address this unmet need, we aimed to develop a fully automated, quantitative approach for histology characterisation of celiac disease. Convolutional neural network models were trained using pathologist annotations of hematoxylin and eosin-stained biopsies of celiac disease mucosa and normal duodenum to identify cells, tissue and artifact regions. Biopsies of duodenal mucosa of varying celiac disease severity, and normal duodenum were collected from a large central laboratory. Celiac disease slides (N = 318) were split into training (n = 230; 72.3%), validation (n = 60; 18.9%) and test (n = 28; 8.8%) datasets. Normal duodenum slides (N = 58) were similarly divided into training (n = 40; 69.0%), validation (n = 12; 20.7%) and test (n = 6; 10.3%) datasets. Human interpretable features were extracted and the strength of their correlation with Marsh scores were calculated using Spearman rank correlations. Our model identified cells, tissue regions and artifacts, including distinguishing intraepithelial lymphocytes and differentiating villous epithelium from crypt epithelium. Proportional area measurements representing villous atrophy negatively correlated with Marsh scores (r = − 0.79), while measurements indicative of crypt hyperplasia positively correlated (r = 0.71). Furthermore, features distinguishing celiac disease from normal duodenum were identified. Our novel model provides an explainable and fully automated approach for histology characterisation of celiac disease that correlates with modified Marsh scores, potentially facilitating diagnosis, prognosis, clinical trials and treatment response monitoring.

Brightfield Multiplex Immunohistochemistry Assay for PD-L1 Evaluation in Challenging Melanoma Samples.

Targeting the PD1/PD-L1 immune checkpoint pathway has rapidly become a therapeutic strategy for melanoma patients. Indeed, the quantification of PD-L1 expression by immunohistochemistry (IHC) in melanoma samples is already required, in some contexts, to allow access to anti-PD-1/PD-L1 immunotherapy. Despite a rising demand for PD-L1 testing, paralleling increasing cumulative experience in its assessment and quantification, it is fair to recognize that PD-L1 evaluation in melanoma samples still presents some critical issues. The aim of this technical report is to develop and validate a multiplex double staining protocol for PD-L1/SOX10 in Ventana Benchmark Ultra for routine practice. Our results show that double labeling provides the necessary tools to identify PD-L1 + melanoma cells clearly. The simultaneous visualization of 2 different proteins targets allows the topographical relationship between the 2 labeling to be evaluated within the context of the tissue morphology. Future studies are needed to test this technique's real-world applicability and effectiveness in implementing interpathologist agreement.

Integration of Pathological Criteria and Immunohistochemical Evaluation for Invasive Lobular Carcinoma Diagnosis: Recommendations From the European Lobular Breast Cancer Consortium

Invasive lobular carcinoma (ILC) is the second most frequent type of breast cancer (BC) and its peculiar morphology is mainly driven by inactivation of CDH1, the gene coding for E-cadherin cell adhesion protein. ILC-specific therapeutic and disease-monitoring approaches are gaining momentum in the clinic, increasing the importance of accurate ILC diagnosis. Several essential and desirable morphologic diagnostic criteria are currently defined by the World Health Organization, the routine use of immunohistochemistry (IHC) for E-cadherin is not recommended. Disagreement in the diagnosis of ILC has been repeatedly reported, but interpathologist agreement increases with the use of E-cadherin IHC. In this study, we aimed to harmonize the pathological diagnosis of ILC by comparing 5 commonly used E-cadherin antibody clones (NCH-38, EP700Y, Clone 36, NCL-L-E-cad [Clone 36B5], and ECH-6). We determined their biochemical specificity for the E-cadherin protein and IHC staining performance according to type and location of mutation on the CDH1 gene. Western blot analysis on mouse cell lines with conditional E-cadherin expression revealed a reduced specificity of EP700Y and NCL-L-E-cad for E-cadherin, with cross-reactivity of Clone 36 to P-cadherin. The use of IHC improved interpathologist agreement for ILC, lobular carcinoma in situ, and atypical lobular hyperplasia. The E-cadherin IHC staining pattern was associated with variant allele frequency and likelihood of nonsense-mediated RNA decay but not with the type or position of CDH1 mutations. Based on these results, we recommend the indication for E-cadherin staining, choice of antibodies, and their interpretation to standardize ILC diagnosis in current pathology practice.

Clinical criteria accurately diagnose severe but not moderate alcohol-associated hepatitis: A systematic review and meta-analysis.

The precision of clinical criteria and the utility of liver biopsy for diagnosis or prognosis remain unclear in patients with alcohol-associated hepatitis (AH). We systematically reviewed the literature to answer these questions. Four databases were searched for studies describing the precision of clinical criteria (National Institute on Alcohol Abuse and Alcoholism, European Association for Study of Liver, or classical) and the role of histology in AH. The precision(positive predictive value) of criteria was pooled through random-effects meta-analysis, and its variation was investigated through subgroups and meta-regression of study-level factors with their percent contribution to variation (R2). The risk of bias among studies was evaluated through the QUADAS2 tool (PROSPERO-ID-CRD4203457250). Of 4320 studies, 18 in the systematic review and 15 (10/5: low/high risk of bias, N=1639) were included in the meta-analysis. The pooled precision of clinical criteria was 80.2% (95% CI: 69.7-89.7, I2:93%, p < 0.01), higher in studies with severe AH (mean-Model for End-Stage Liver Disease > 20) versus moderate AH (mean-Model for End-Stage Liver Disease < 20): 92% versus 67.1%, p < 0.01, and in studies with serum bilirubin cutoff 5 versus 3 mg/dL (88.5% vs.78.8%, p = 0.01). The factors contributing to variation in precision were Model for End-Stage Liver Disease (R2:72.7%), upper gastrointestinal bleed (R2:56.3%), aspartate aminotransferase:aspartate aminotransferase ratio (R2:100%), clinical criteria (R2:40.9%), bilirubin (R2:22.5%), and Mallory body on histology (R2:19.1%).The net inter-pathologist agreement for histologic findings of AH was variable (0.33-0.97), best among 2 studies describing AH through simple and uniform criteria, including steatosis, ballooning, and neutrophilic inflammation. Few studies reported the utility of histology in estimating steroid responsiveness (N = 1) and patient prognosis (N = 4); however, very broad septa, pericellular fibrosis, and cholestasis were associated with mortality. Bilirubinostasis was associated with infection in 1 study. Clinical criteria are reasonably precise for diagnosing severe AH, while there is an unmet need for better criteria for diagnosing moderate AH. Histologic diagnosis of AH should be simple and uniform.

Concordance between pathologists and between specimen types in detection of HER2-low breast carcinoma by immunohistochemistry

ContextRecent clinical trials indicate that HER2-targeted therapy may benefit HER2-low breast cancer patients including HER2 score 1+ or 2+ and no gene amplification.Concordance between pathologists and between core biopsy and surgical excision in establishing HER2-low status was evaluated. Design57 patients with HER2 negative breast cancer (IHC 0, 1+, or 2+, no gene amplification) by core biopsy were included. Core biopsy and representative tumor from corresponding surgical excision was immunostained for HER2. Original HER2 IHC scores were interpreted using 2018 guidelines. Three pathologists independently interpreted again under 2023 guidelines. Kappa statistic evaluated agreement of HER2 IHC scores. ResultsApplying 2023 guidelines, HER2 IHC scores were concordant among study pathologists in 46 of 57 (81 %) core biopsy and 50 of 57 (88 %) surgical resections. Kappa statistics were 0.78 and 0.85 (substantial agreement), for inter-pathologist agreement of core biopsy and surgical resections under 2023 guidelines; 0.55 (moderate agreement) for agreement between first interpretation by 2018 guidelines and second interpretation by 2023 guidelines; and 0.13 (slight agreement) for agreement in HER2 consensus scores between outside core and surgical resection and 0.49 (moderate agreement) for inside core and surgical resection. Low HER2 expression was found in 28 of 57 (49 %) core biopsy and in 25 of 57 (44 %) surgical excisions. ConclusionsInterobserver agreement among study pathologists was good in core biopsy and surgical excisions, applying updated 2023 guidelines. Intratumoral heterogeneity in protein expression and preanalytical factors may result in variable identification of HER2-low status in core biopsy and surgical excision specimens.

Automated detection of microscopic placental features indicative of maternal vascular malperfusion using machine learning

IntroductionHypertensive disorders of pregnancy (HDP) and fetal growth restriction (FGR) are common obstetrical complications, often with pathological features of maternal vascular malperfusion (MVM) in the placenta. Currently, clinical placental pathology methods involve a manual visual examination of histology sections, a practice that can be resource-intensive and demonstrates moderate-to-poor inter-pathologist agreement on diagnostic outcomes, dependant on the degree of pathologist sub-specialty training. MethodsThis study aims to apply machine learning (ML) feature extraction methods to classify digital images of placental histopathology specimens, collected from cases of HDP [pregnancy induced hypertension (PIH), preeclampsia (PE), PE + FGR], normotensive FGR, and healthy pregnancies, according to the presence or absence of MVM lesions. 159 digital images were captured from histological placental specimens, manually scored for MVM lesions (MVM- or MVM+) and used to develop a support vector machine (SVM) classifier model, using features extracted from pre-trained ResNet18. The model was trained with data augmentation and shuffling, with the performance assessed for patch-level and image-level classification through measurements of accuracy, precision, and recall using confusion matrices. ResultsThe SVM model demonstrated accuracies of 70 % and 79 % for patch-level and image-level MVM classification, respectively, with poorest performance observed on images with borderline MVM presence, as determined through post hoc observation. DiscussionThe results are promising for the integration of ML methods into the placental histopathological examination process. Using this study as a proof-of-concept will lead our group and others to carry ML models further in placental histopathology.

Assessing the role of phosphorylated S6 ribosomal protein in the pathological diagnosis of pulmonary antibody-mediated rejection

BackgroundPulmonary antibody-mediated rejection is still a challenging diagnosis as C4d has poor sensitivity. Previous studies have indicated that the phosphorylated S6 ribosomal protein, a component of the mammalian target of rapamycin (mTOR) pathway, is correlated with de novo donor-specific antibodies in lung transplantation. The objective of this study was to evaluate the phosphorylation of S6 ribosomal protein as a surrogate for AMR diagnosis in lung transplant patients. MethodsThis multicentre retrospective study analyzed transbronchial biopsies from 216 lung transplanted patients, 114 with antibody-mediated rejection and 102 without (19 with acute cellular rejection, 17 with ischemia/reperfusion injury, 18 with infection, and 48 without post-transplant complications). Immunohistochemistry was used to quantify phosphorylated S6 ribosomal protein expression in macrophages, endothelium, epithelium, and inter-pathologist agreement was assessed. ResultsMedian phosphorylated S6 ribosomal protein expression values were higher in antibody-mediated rejection cases than in controls for all cell components, with the highest sensitivity in macrophages (0.9) and the highest specificity in endothelial expression (0.8). The difference was mainly significant in macrophages compared to other post-lung transplantation complications. Inter-pathologist agreement was moderate for macrophages and endothelium, with higher agreement when phosphorylated S6 ribosomal protein expression was dichotomized into positive/negative. The inclusion of phosphorylated S6 ribosomal protein in the diagnostic algorithm could have increased antibody-mediated rejection certainty levels by 25%. ConclusionsThe study supports the role of the mTOR pathway in antibody-mediated rejection-related graft injury and suggests that tissue phosphorylation of S6 ribosomal protein could be a useful surrogate for a more accurate pathological diagnosis of lung antibody-mediated rejection.

Pathologist Computer-Aided Diagnostic Scoring of Tumor Cell Fraction: A Swiss National Study

Tumor cell fraction (TCF) estimation is a common clinical task with well-established large interobserver variability. It thus provides an ideal test bed to evaluate potential impacts of employing a tumor cell fraction computer-aided diagnostic (TCFCAD) tool to support pathologists’ evaluation. During a National Slide Seminar event, pathologists (n = 69) were asked to visually estimate TCF in 10 regions of interest (ROIs) from hematoxylin and eosin colorectal cancer images intentionally curated for diverse tissue compositions, cellularity, and stain intensities. Next, they re-evaluated the same ROIs while being provided a TCFCAD-created overlay highlighting predicted tumor vs nontumor cells, together with the corresponding TCF percentage. Participants also reported confidence levels in their assessments using a 5-tier scale, indicating no confidence to high confidence, respectively. The TCF ground truth (GT) was defined by manual cell-counting by experts. When assisted, interobserver variability significantly decreased, showing estimates converging to the GT. This improvement remained even when TCFCAD predictions deviated slightly from the GT. The standard deviation (SD) of the estimated TCF to the GT across ROIs was 9.9% vs 5.8% with TCFCAD (P < .0001). The intraclass correlation coefficient increased from 0.8 to 0.93 (95% CI, 0.65-0.93 vs 0.86-0.98), and pathologists stated feeling more confident when aided (3.67 ± 0.81 vs 4.17 ± 0.82 with the computer-aided diagnostic [CAD] tool). TCFCAD estimation support demonstrated improved scoring accuracy, interpathologist agreement, and scoring confidence. Interestingly, pathologists also expressed more willingness to use such a CAD tool at the end of the survey, highlighting the importance of training/education to increase adoption of CAD systems.

PD27-11 COMPARISON OF PROSTATE BIOPSY GRADE OBTAINED BY HUMAN PATHOLOGISTS VERSUS AN ARTIFICIAL INTELLIGENCE ALGORITHM FOR PREDICTING BIOCHEMICAL RECURRENCE AFTER RADICAL PROSTATECTOMY

You have accessJournal of UrologyCME1 Apr 2023PD27-11 COMPARISON OF PROSTATE BIOPSY GRADE OBTAINED BY HUMAN PATHOLOGISTS VERSUS AN ARTIFICIAL INTELLIGENCE ALGORITHM FOR PREDICTING BIOCHEMICAL RECURRENCE AFTER RADICAL PROSTATECTOMY Claire de la Calle, Thiago Vidotto, Eric Erak, Daniela Salles, Oluwademilade Dairo, Christian Pavlovich, Misop Han, Joonyoung Cho, In Hye Suh, Hye Yoon Chang, Sun Woo Kim, Michael Gorin, and Tamara Lotan Claire de la CalleClaire de la Calle More articles by this author , Thiago VidottoThiago Vidotto More articles by this author , Eric ErakEric Erak More articles by this author , Daniela SallesDaniela Salles More articles by this author , Oluwademilade DairoOluwademilade Dairo More articles by this author , Christian PavlovichChristian Pavlovich More articles by this author , Misop HanMisop Han More articles by this author , Joonyoung ChoJoonyoung Cho More articles by this author , In Hye SuhIn Hye Suh More articles by this author , Hye Yoon ChangHye Yoon Chang More articles by this author , Sun Woo KimSun Woo Kim More articles by this author , Michael GorinMichael Gorin More articles by this author , and Tamara LotanTamara Lotan More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000003305.11AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Several artificial intelligence (AI) algorithms have been developed to streamline biopsy Gleason grading, but most have not been challenged with predicting oncologic outcomes. In this study, we leveraged a cohort of patients with biopsy-detected Grade Group (GG) 2 prostate cancer (PCa) and examined the association of biopsy grade by contemporary pathologist consensus grade or AI algorithm with time to biochemical recurrence (BCR) after radical prostatectomy (RP). METHODS: We identified 286 patients originally diagnosed with GG2 PCa on biopsy who underwent RP at our institution from 2000 to 2014. All biopsies were re-graded by two expert genitourinary pathologists (pathologist 1 and 2). For cases with discrepant reads, a third expert genitourinary pathologist determined the consensus pathology read. GG was also obtained using a previously validated and published AI algorithm. Concordance between pathologist and AI GG was quantified using the quadratic kappa. BCR was defined as two consecutive PSAs ≥0.2 ng/mL. Kaplan-Meier BCR free survival estimates by GG were compared using the log-rank test. RESULTS: After RP median follow-up was 4 years (range 1-14). To date, 16% of the men have had BCR and median time to BCR was 2 years (range 1-10). Biopsy grading between pathologist 1 and 2 generated a kappa of 0.17. Grading between the consensus pathology read and the AI algorithm generated a kappa of 0.33. Grading by consensus pathology read was associated with time to BCR-free survival (log-rank p=0.003) as was the grading by the AI algorithm (log-rank p=0.004; Figure 1). CONCLUSIONS: The AI algorithm Gleason grading demonstrated poor agreement with the consensus contemporary pathology read, but the agreement was also poor between the two pathologists. Yet, our cohort was only comprised of men originally diagnosed with GG2 PCa, a GG usually associated with poor inter-pathologist agreement. Remarkably, both the consensus pathology and the AI algorithm grades on the biopsy stratified patients well for subsequent BCR after RP, suggesting that AI algorithms could be developed to stratify for oncologic outcomes instead of attempting to replicate somewhat subjective human grading. Source of Funding: Research funding from Deep Bio © 2023 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 209Issue Supplement 4April 2023Page: e746 Advertisement Copyright & Permissions© 2023 by American Urological Association Education and Research, Inc.MetricsAuthor Information Claire de la Calle More articles by this author Thiago Vidotto More articles by this author Eric Erak More articles by this author Daniela Salles More articles by this author Oluwademilade Dairo More articles by this author Christian Pavlovich More articles by this author Misop Han More articles by this author Joonyoung Cho More articles by this author In Hye Suh More articles by this author Hye Yoon Chang More articles by this author Sun Woo Kim More articles by this author Michael Gorin More articles by this author Tamara Lotan More articles by this author Expand All Advertisement PDF downloadLoading ...

High Interobserver Variability Among Pathologists Using Combined Positive Score to Evaluate PD-L1 Expression in Gastric, Gastroesophageal Junction, and Esophageal Adenocarcinoma

Reliable, reproducible methods to interpret programmed death ligand-1 (PD-L1) expression on tumor cells (TC) and immune cells (IC) are needed for pathologists to inform decisions associated with checkpoint inhibitor therapies. Our international study compared interpathologist agreement of PD-L1 expression using the combined positive score (CPS) under standardized conditions on samples from patients with gastric/gastroesophageal junction/esophageal adenocarcinoma. Tissue sections from 100 adenocarcinoma pretreatment biopsies were stained in a single laboratory using the PD-L1 immunohistochemistry 28-8 and 22C3 (Agilent) pharmDx immunohistochemical assays. PD-L1 CPS was evaluated by 12 pathologists on scanned whole slide images of these biopsies before and after a 2-hour CPS training session by Agilent. Additionally, pathologists determined PD-L1–positive TC, IC, and total viable TC on a single tissue fragment from 35 of 100 biopsy samples. Scoring agreement among pathologists was assessed using the intraclass correlation coefficient (ICC). Interobserver variability for CPS for 100 biopsies was high, with only fair agreement among pathologists both pre- (range, 0.45-0.55) and posttraining (range, 0.56-0.57) for both assays. For the 35 single biopsy samples, poor/fair agreement was also observed for the total number of viable TC (ICC, 0.09), number of PD-L1–positive IC (ICC, 0.19), number of PD-L1–positive TC (ICC, 0.54), and calculated CPS (ICC, 0.14), whereas calculated TC score (positive TC/total TC) showed excellent agreement (ICC, 0.82). Retrospective histologic review of samples with the poorest interpathologist agreement revealed the following as possible confounding factors: (1) ambiguous identification of positively staining stromal cells, (2) faint or variable intensity of staining, (3) difficulty in distinguishing membranous from cytoplasmic tumor staining, and (4) cautery and crush artifacts. These results emphasize the need for objective techniques to standardize the interpretation of PD-L1 expression when using the CPS methodology on gastric/gastroesophageal junction cancer biopsies to accurately identify patients most likely to benefit from immune checkpoint inhibitor therapy.

Pathologist Validation of a Machine Learning–Derived Feature for Colon Cancer Risk Stratification

Identifying new prognostic features in colon cancer has the potential to refine histopathologic review and inform patient care. Although prognostic artificial intelligence systems have recently demonstrated significant risk stratification for several cancer types, studies have not yet shown that the machine learning-derived features associated with these prognostic artificial intelligence systems are both interpretable and usable by pathologists. To evaluate whether pathologist scoring of a histopathologic feature previously identified by machine learning is associated with survival among patients with colon cancer. This prognostic study used deidentified, archived colorectal cancer cases from January 2013 to December 2015 from the University of Milano-Bicocca. All available histologic slides from 258 consecutive colon adenocarcinoma cases were reviewed from December 2021 to February 2022 by 2 pathologists, who conducted semiquantitative scoring for tumor adipose feature (TAF), which was previously identified via a prognostic deep learning model developed with an independent colorectal cancer cohort. Prognostic value of TAF for overall survival and disease-specific survival as measured by univariable and multivariable regression analyses. Interpathologist agreement in TAF scoring was also evaluated. A total of 258 colon adenocarcinoma histopathologic cases from 258 patients (138 men [53%]; median age, 67 years [IQR, 65-81 years]) with stage II (n = 119) or stage III (n = 139) cancer were included. Tumor adipose feature was identified in 120 cases (widespread in 63 cases, multifocal in 31, and unifocal in 26). For overall survival analysis after adjustment for tumor stage, TAF was independently prognostic in 2 ways: TAF as a binary feature (presence vs absence: hazard ratio [HR] for presence of TAF, 1.55 [95% CI, 1.07-2.25]; P = .02) and TAF as a semiquantitative categorical feature (HR for widespread TAF, 1.87 [95% CI, 1.23-2.85]; P = .004). Interpathologist agreement for widespread TAF vs lower categories (absent, unifocal, or multifocal) was 90%, corresponding to a κ metric at this threshold of 0.69 (95% CI, 0.58-0.80). In this prognostic study, pathologists were able to learn and reproducibly score for TAF, providing significant risk stratification on this independent data set. Although additional work is warranted to understand the biological significance of this feature and to establish broadly reproducible TAF scoring, this work represents the first validation to date of human expert learning from machine learning in pathology. Specifically, this validation demonstrates that a computationally identified histologic feature can represent a human-identifiable, prognostic feature with the potential for integration into pathology practice.

The influence of papillary features on the risk of malignancy in thyroid nodules diagnosed as atypia of undetermined significance or follicular lesion of undetermined significance

BackgroundThe objective of this study was to the assess the risk of malignancy in thyroid lesions that were diagnosed as AUS/FLUS when using a novel cytology subclassification system that is based on the presence or absence of papillary features. MethodsAUS/FLUS case cytology was re-reviewed and subclassified into minor or major concern groups based upon the absence or presence of papillary features, respectively. The risk of malignancy (ROM) was calculated and compared between the two groups. Inter-pathologist agreement in case subclassification was also measured. ResultsThe minor concern group had a 12.6% associated ROM, while the major concern group had a significantly higher ROM (58.4%), (P < 0.001). Based on 108 cases, the inter-pathologist agreement in case subclassification was 79%, and the κ value was 0.47. ConclusionsThe identification of papillary features significantly increases the ROM in thyroid lesions with an AUS/FLUS diagnosis.

PD-L1 Assessment in Needle Core Biopsies of Non-Small Cell Lung Cancer: Interpathologist Agreement and Potential Associated Histopathological Features.

Immune checkpoint inhibitors are used in the treatment of non-small cell lung cancer (NSCLC). Programmed cell death-ligand 1 (PD-L1) immunohistochemistry (IHC) assessed by pathologists is subject to interobserver variability. In advanced/metastatic disease and inoperable patients, PD-L1 assessment relies on biopsy specimens, commonly needle core biopsies (NCB). We aimed to determine the interobserver agreement for PD-L1 tumor proportion score (TPS) in NSCLC NCBs and identify histopathological features that may be related to interobserver variability. Sixty NSCLC NCBs with PD-L1 IHC were evaluated independently by four pathologists from different institutions. PD-L1 TPS was evaluated in three categories: no/low expression ( < 1%), intermediate expression (1%49%), and high expression (≥50%). Histological tumor type, necrosis, tumor-infiltrating lymphocytes, tumor length/percentage in the biopsy, and crush/squeeze artifact was evaluated. The statistical analysis of the three PD-L1 TPS categories demonstrated moderate agreement (Fleiss Kappa 0.477) in the no/low category, fair agreement (Fleiss Kappa 0.390) in the intermediate category, and almost perfect agreement (Fleiss Kappa 0.952) in the high category. A significant correlation (p=0.003) was found between the crush/squeeze artifact in NCB and rate of discordant TPS categories. There was no significant correlation between pathologists' agreement in the TPS categories and histological tumor type, tumor length, tumor ratio, necrosis, and tumor-infiltrating lymphocytes. Our results demonstrated moderate agreement among pathologists for the PD-L1 TPS 1% cut-off in NSCLC NCB, which is lower than that reported in resection materials. The presence of crush/squeeze artifact in NCBs is significantly related to the rate of discordant TPS categories, suggesting that PD-L1 assessment of pulmonary NCBs requires an awareness of this artifact.

Evaluation of large airway specimens obtained by transbronchial lung cryobiopsy in diffuse parenchymal lung diseases

BackgroundThe difference in diagnostic yield between surgical lung biopsy and transbronchial lung cryobiopsy (TBLC) in diffuse parenchymal lung diseases (DPLD) has been reported to be due to differences in the rate of interpathologist agreement, specimen size, and specimen adequacy. In TBLC, the specimens containing large airway components are generally believed as inadequate specimens for histological evaluation, but the detailed characteristics of TBLC specimens including the large airway and the impact on histological diagnostic rates of DPLD have not been investigated.MethodsWe retrospectively reviewed the specimen characteristics of patients with DPLD who underwent TBLC.ResultsBetween February 2018 and January 2020, 74 patients and 177 specimens were included. There were 85 (48.0%) large airway specimens (LAS) that contained bronchial gland or bronchial cartilage. The ideal specimen ratio was significantly lower in the LAS-positive group than that in the LAS-negative group (5.8% vs. 45.6%), and the proportion of bronchioles, alveoli, and perilobular area were similarly lower in the LAS-positive group. The presence of traction bronchiectasis and diaphragm overlap sign on high-resolution computed tomography (HRCT) were also significantly higher in the LAS-positive group than those in the LAS-negative group. We observed a statistically significant trend in histological diagnostic yield (40.7% in LAS positive group; 60.8% in LAS positive and negative group; 91.6% in LAS negative group) (Cochran-Armitage trend test).ConclusionLAS is a specimen often collected in TBLC and contains a low percentage of bronchioles, alveoli, and perilobular area. Since the histological diagnostic yield tends to be higher in cases that do not contain LAS, it may be important to determine the biopsy site that reduces the frequency of LAS collection by referring to the HRCT findings in TBLC.

Prediction of malignant transformation and recurrence of oral epithelial dysplasia using architectural and cytological feature specific prognostic models

AbstractOral epithelial dysplasia (OED) is a precursor state usually preceding oral squamous cell carcinoma (OSCC). Histological grading is the current gold standard for OED prognostication but is subjective and variable with unreliable outcome prediction. We explore if individual OED histological features can be used to develop and evaluate prognostic models for malignant transformation and recurrence prediction. Digitised tissue slides for a cohort of 109 OED cases were reviewed by three expert pathologists, where the prevalence and agreement of architectural and cytological histological features was assessed and association with clinical outcomes analysed using Cox proportional hazards regression and Kaplan–Meier curves. Within the cohort, the most prevalent features were basal cell hyperplasia (72%) and irregular surface keratin (60%), and least common were verrucous surface (26%), loss of epithelial cohesion (30%), lymphocytic band and dyskeratosis (34%). Several features were significant for transformation (p < 0.036) and recurrence (p < 0.015) including bulbous rete pegs, hyperchromatism, loss of epithelial cohesion, loss of stratification, suprabasal mitoses and nuclear pleomorphism. This led us to propose two prognostic scoring systems including a ‘6-point model' using the six features showing a greater statistical association with transformation and recurrence (bulbous rete pegs, hyperchromatism, loss of epithelial cohesion, loss of stratification, suprabasal mitoses, nuclear pleomorphism) and a ‘two-point model' using the two features with highest inter-pathologist agreement (loss of epithelial cohesion and bulbous rete pegs). Both the ‘six point' and ‘two point' models showed good predictive ability (AUROC ≥ 0.774 for transformation and 0.726 for recurrence) with further improvement when age, gender and histological grade were added. These results demonstrate a correlation between individual OED histological features and prognosis for the first time. The proposed models have the potential to simplify OED grading and aid patient management. Validation on larger multicentre cohorts with prospective analysis is needed to establish their usefulness in clinical practice.

Inter-pathologist agreement on diagnosis, classification and grading of canine glioma.

Histopathological evaluation of tumours is a subjective process, but studies of inter-pathologist agreement are uncommon in veterinary medicine. The Comparative Brain Tumour Consortium (CBTC) recently published diagnostic criteria for canine gliomas. Our objective was to assess the degree of inter-pathologist agreement on intracranial canine gliomas, utilising the CBTC diagnostic criteria in a cohort of eighty-five samples from dogs with an archival diagnosis of intracranial glioma. Five pathologists independently reviewed H&E and immunohistochemistry sections and provided a diagnosis and grade. Percentage agreement and kappa statistics were calculated to measure inter-pathologist agreement between pairs and amongst the entire group. A consensus diagnosis of glioma subtype and grade was achieved for 71/85 (84%) cases. For these cases, percentage agreement on combined diagnosis (subtype and grade), subtype only and grade only were 66%, 80% and 82%, respectively. Kappa statistics for the same were 0.466, 0.542 and 0.516, respectively. Kappa statistics for oligodendroglioma, astrocytoma and undefined glioma were 0.585, 0.566 and 0.280 and were 0.516 for both low-grade and high-grade tumours. Kappa statistics amongst pairs of pathologists for combined diagnosis varied from 0.352 to 0.839. 8 % of archival oligodendrogliomas and 61% of archival astrocytomas were reclassified as another entity after review. Inter-pathologist agreement utilising CBTC guidelines for canine glioma was moderate overall but varied from fair to almost perfect between pairs of pathologists. Agreement was similar for oligodendrogliomas and astrocytomas but lower for undefined gliomas. These results are similar to pathologist agreement in human glioma studies and with other tumour entities in veterinary medicine.

Evaluation of a Probability-Based Predictive Tool on Pathologist Agreement Using Urinary Bladder as a Pilot Tissue.

Simple SummaryThere is a common joke in pathology—put three pathologists in a room and you will obtain three different answers. This saying comes from the fact that pathology can be subjective; pathologists’ diagnoses can be influenced by many different biases, and pathologists are also influenced by the presence or absence of animal information and medical history. Compared to pathology, statistics is a much more objective field. This study aimed to develop a probability-based tool using statistics obtained by analyzing 338 histopathology slides of canine and feline urinary bladders, then see if the tool affected agreement between the test pathologists. Four pathologists diagnosed 25 canine and feline bladder slides and they conducted this three times: without animal and clinical information, then with this information, and finally using the probability tool. Results showed large differences in the pathologists’ interpretation of bladder slides, with kappa agreement values (low value for digital slide images, high value for glass slides) of 7–37% without any animal or clinical information, 23–37% with animal signalment and history, and 31–42% when our probability tool was used. This study provides a starting point for the use of probability-based tools in standardizing pathologist agreement in veterinary pathology.Inter-pathologist variation is widely recognized across human and veterinary pathology and is often compounded by missing animal or clinical information on pathology submission forms. Variation in pathologist threshold levels of resident inflammatory cells in the tissue of interest can further decrease inter-pathologist agreement. This study applied a predictive modeling tool to bladder histology slides that were assessed by four pathologists: first without animal and clinical information, then with this information, and finally using the predictive tool. All three assessments were performed twice, using digital whole-slide images (WSI) and then glass slides. Results showed marked variation in pathologists’ interpretation of bladder slides, with kappa agreement values of 7–37% without any animal or clinical information, 23–37% with animal signalment and history, and 31–42% when our predictive tool was applied, for digital WSI and glass slides. The concurrence of test pathologists to the reference diagnosis was 60% overall. This study provides a starting point for the use of predictive modeling in standardizing pathologist agreement in veterinary pathology. It also highlights the importance of high-quality whole-slide imaging to limit the effect of digitization on inter-pathologist agreement and the benefit of continued standardization of tissue assessment in veterinary pathology.

Abstract 460: Tools for collecting pathologist annotations and understanding interobserver variability

Abstract Background: The High Throughput Truthing (HTT) project is assessing pathologist agreement estimates of stromal tumor-infiltrating lymphocytes (sTILs) density in hematoxylin and eosin (H&amp;E) stained breast cancer biopsy slides. The HTT project will create a validation dataset for artificial intelligence and machine learning (AI/ML) algorithms in digital pathology fit for a training, proficiency testing, and regulatory purpose. Methods: The pilot study crowdsourced pathologists to estimate sTIL density in 640 regions of interest (ROIs) across 64 slides via two modalities: an optical microscope (eeDAP) and two digital platforms (caMicroscope and PathPresenter). eeDAP is a hardware-software interface that presents the observer with pre-defined fields of view on H&amp;E slides that corresponds to the ROI on a whole slide image. The PathPresenter and caMicroscope web-applications replicate the eeDAP workflow on the whole slide image without microscope hardware. In the workflow, pathologists evaluated the eligibility of an ROI for sTILs content then estimated the densities of tumor-associated stroma and sTILs in the ROI. Inter-pathologist agreement within ROIs was characterized with the root mean-squared difference. Using 72 of the highest variability ROIs selected from the pilot study, seven practicing pathologists participated in a subsequent focus group to improve the clinical training and data-collection workflows. Results: The pilot study collected 7,373 sTIL density estimates from 35 pathologists between February 2020 and May 2021. The focus group provided an additional 411 evaluations on 72 ROIs and in-depth discussions to identify pitfalls, gaps in training, and workflow improvements. Installation of eeDAP for physical data collection guided improvements in documentation and operation capabilities. Updated training materials refine the definition of tumor-associated stroma, provide reference images to differentiate sTILs from other cell types, and provide feedback during training. Digital and microscope platforms benefitted from enforcing registration and training, standardizing workflows, and accelerating eeDAP slide-image registration. Conclusions: The slides, images, and annotations provided by volunteer collaborators and participants for our pilot study led to improvements in data collection tools and crowdsourcing workflows to ensure consistency and minimize annotation variability. Our pilot dataset and analysis methods are available on a public HTT Github repository to allow open access to our methodology and feedback from the digital pathology and statistics communities. These data-collection and analysis methods are applicable to other quantitative biomarkers for validation of AI/ML algorithms. The lessons learned from this work will be applied to the HTT pivotal study and inform future quality data-collection methods of pathologist annotations. Citation Format: Katherine N. Elfer, Kim Blenman, Sarah N. Dudgeon, Victor Garcia, Anna Ehinger, Xiaoxian Li, Amy Ly, Dieter Peeters, Bruce Werness, Matthew Hanna, Roberto Salgado. Tools for collecting pathologist annotations and understanding interobserver variability [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr 460.

Deep Learning-Based Artificial Intelligence System for Automatic Assessment of Glomerular Pathological Findings in Lupus Nephritis.

Accurate assessment of renal histopathology is crucial for the clinical management of patients with lupus nephritis (LN). However, the current classification system has poor interpathologist agreement. This paper proposes a deep convolutional neural network (CNN)-based system that detects and classifies glomerular pathological findings in LN. A dataset of 349 renal biopsy whole-slide images (WSIs) (163 patients with LN, periodic acid-Schiff stain, 3906 glomeruli) annotated by three expert nephropathologists was used. The CNN models YOLOv4 and VGG16 were employed to localise the glomeruli and classify glomerular lesions (slight/severe impairments or sclerotic lesions). An additional 321 unannotated WSIs from 161 patients were used for performance evaluation at the per-patient kidney level. The proposed model achieved an accuracy of 0.951 and Cohen’s kappa of 0.932 (95% CI 0.915–0.949) for the entire test set for classifying the glomerular lesions. For multiclass detection at the glomerular level, the mean average precision of the CNN was 0.807, with ‘slight’ and ‘severe’ glomerular lesions being easily identified (F1: 0.924 and 0.952, respectively). At the per-patient kidney level, the model achieved a high agreement with nephropathologist (linear weighted kappa: 0.855, 95% CI: 0.795–0.916, p < 0.001; quadratic weighted kappa: 0.906, 95% CI: 0.873–0.938, p < 0.001). The results suggest that deep learning is a feasible assistive tool for the objective and automatic assessment of pathological LN lesions.