Interobserver Variability In Diagnosis Research Articles

Atypical Squamous Verrucous Lesions of the Oral Cavity: Challenges in Interpretation of Small Incisional Biopsies.

Squamous verrucous proliferative lesions of oral cavity can pose a diagnostic challenge for the general pathologist, especially on small biopsies. The superficial nature of incisional biopsies and inconsistent histologic terminologies used for these lesions contribute to often-discrepant clinical diagnosis, resulting in delayed treatment.This study aims to explore the proliferative squamous lesions of oral cavity, correlate biopsy & resection diagnoses, and evaluate possible reasons for discrepant diagnosis (if any). A retrospective review of oral verrucous squamous lesions was undertaken. Pathology database was searched for oral cavity biopsies from January2018 through August2022 with the keywords: atypical, verrucous, squamous, and proliferative. Cases with follow-up were included in this study. A blinded review of the biopsy slides was performed and documented by a single head and neck pathologist. Demographic data, biopsy and final diagnosis were recorded. Twenty-three cases met criteria for inclusion. The mean patient age was 61.1years with a male: female ratio of 1.09. Most frequent site was lateral border of tongue (36%) followed by buccal mucosa and retromolar trigone. The most common biopsy diagnosis was "Atypical squamoproliferative lesion, excision recommended" (n = 16/23, 69%) with 13/16 showing conventional squamous cell carcinoma (SCC) on follow-up resection. 2/16 atypical cases underwent repeat biopsy for confirmation of diagnosis. Overall, conventional SCC was the most prevalent final diagnosis (73%, n = 17), followed by verrucous carcinoma (17%, n = 4). On slide review, six initial biopsies were reclassified as SCC, while one final diagnosis was reclassified as a hybrid carcinoma (on resection specimen). Diagnostic concordance (biopsy and resection) was observed in three cases, all three were recurrences. The primary reasons for discrepant diagnosis on initial biopsies were found to be 1. Obscuring inflammation, 2. Superficial biopsies, and 3. Under recognition of morphologic features (e.g., tear shaped rete, loss of polarity, dyskeratotic cells, paradoxical maturation) that help differentiate dysplasia from reactive atypia. This study highlights the rampant interobserver variability in diagnosis of oral cavity squamous lesions and emphasizes importance of identifying morphologic clues that can aid in correct diagnosis, thereby helping in adequate clinical management.

Reliability of histopathologic diagnosis of fibrotic interstitial lung disease:\xa0an international collaborative standardization project

BackgroundCurrent interstitial lung disease (ILD) diagnostic guidelines assess criteria across clinical, radiologic and pathologic domains. Significant interobserver variation in histopathologic evaluation has previously been shown but the specific source of these discrepancies is poorly documented. We sought to document specific areas of difficulty and develop improved criteria that would reduce overall interobserver variation.MethodsUsing an internet-based approach, we reviewed selected images of specific diagnostic features of ILD histopathology and whole slide images of fibrotic ILD. After an initial round of review, we confirmed the presence of interobserver variation among our group. We then developed refined criteria and reviewed a second set of cases.ResultsThe initial round reproduced the existing literature on interobserver variation in diagnosis of ILD. Cases which were pre-selected as inconsistent with usual interstitial pneumonia/idiopathic pulmonary fibrosis (UIP/IPF) were confirmed as such by multi-observer review. Cases which were thought to be in the spectrum of chronic fibrotic ILD for which UIP/IPF were in the differential showed marked variation in nearly all aspects of ILD evaluation including extent of inflammation and extent and pattern of fibrosis. A proposed set of more explicit criteria had only modest effects on this outcome. While we were only modestly successful in reducing interobserver variation, we did identify specific reasons that current histopathologic criteria of fibrotic ILD are not well defined in practice.ConclusionsAny additional classification scheme must address interobserver variation in histopathologic diagnosis of fibrotic ILD order to remain clinically relevant. Improvements to tissue-based diagnostics may require substantial resources such as larger datasets or novel technologies to improve reproducibility. Benchmarks should be established for expected outcomes among clinically defined subgroups as a quality metric.

Abstract No. 507 Quantifying the accuracy of magnetic resonance venography assessments in the diagnosis of May-Thurner syndrome

May-Thurner syndrome (MTS) is a venous anatomical abnormality where the right iliac artery compresses the left iliac vein. Magnetic resonance venography (MRV) is often used to diagnose stenosis associated with MTS. While previous studies have hypothesized that MRV may inaccurately estimate the extent of compression in MTS, there is need to clarify the role of MRV in the diagnosis of this disease.1,2 The purpose of this study is to evaluate interobserver variability in diagnosis and classification of MTS by MRV. An IRB-approved retrospective analysis of 33 MRVs performed at The George Washington University Hospital between January 2012 and December 2015 for “left leg swelling” was performed. These studies were evaluated by three board certified interventional radiologists for iliac vein compression on a simple grading scale (“none,” “mild” i.e., 25% stenosis, “moderate” i.e., 50% stenosis, or “severe” i.e., 75% stenosis) in a blinded fashion. Physician ratings were compared to assess provider variation in MRV evaluation. An analysis was then performed to determine the interrater reliability for assessing the degree of compression. 33 MRVs were reviewed in total. On average, the providers agreed 85.86% of the time on the presence of any stenosis. However, the interrater consensus dropped to 44.44% when reporting the degree of stenosis. Fleiss’ Kappa scores were calculated to quantify the interrater reliability on assessing the degree of stenosis. Interestingly, the grading of compression severity had a low interrater reliability (Fleiss’ k = 0.24) demonstrating only fair agreement between providers. While MRV appears reliable in assessing the presence of any stenosis, Fleiss’ Kappa analysis indicates substantial variability in clinical judgement of the degree of stenosis of the left common iliac vein when diagnosing or ruling out MTS. Low interrater reliability of reading MRVs therefore suggests that more definitive diagnostic methods such as catheter venography supplemented with Intravascular Ultrasound (IVUS) may be more accurate to diagnose MTS.

Forefront of AI Applications for COVID-19 Imaging Diagnosis

The intra- and inter-observer variability in diagnosis of thoracic CT images may affect the diagnosis of COVID-19. Therefore, several studies have been reported to develop artificial intelligence (AI) approaches using deep learning (DL) and radiomics technologies. The difference between them is automatic feature extraction (DL) and hand-crafted one (radiomics). The advantages of the AI-based imaging approaches for the COVID-19 are fast throughput, non-invasion, quantification, and integration of PCR results, CT findings, and clinical information. To the best of my knowledge, three types of the AI approaches have been studied: detection, severity differentiation, and prognosis prediction of COVID-19. AI technologies on assessment of severity/prediction of prognosis for COVID-19 may be more crucial than detection of COVID-19 pneumonia after COVID-19 becomes one of common diseases.

Challenges and Future Directions in the Detection and Treatment of Retinopathy of Prematurity

Multicenter studies addressing screening and intervention for retinopathy of prematurity (ROP) inform the management guidelines jointly recommended by American pediatric and ophthalmology academies. Current research focuses on improvements in the identification of ROP and in the treatment of high-risk disease. The development of digital image technology may address various challenges in the diagnosis of ROP, including availability of pediatric ophthalmologic expertise, interobserver variation in diagnosis, and inherent limitations in visual diagnosis. Improved clinical prediction models based on nonophthalmologic data may complement examination-based ROP diagnosis. Alternatives to retinal ablation therapy are being studied to decrease the associated morbidities of such therapy.

Abstract 2974: Comprehensive and integrative genomic characterization of diffuse lower grade gliomas

Abstract Background: Diffuse low and intermediate grade gliomas (LGG; WHO grades II and III) have highly variable clinical behaviors that are not optimally predicted by histology; some are indolent, while others quickly progress to glioblastoma. Uncertainty is compounded by inter-observer variability in diagnosis, in part due to modest correlation between histology and biologic classes. Mutations in isocitrate dehydrogenase (IDH), TP53, ATRX, and co-deletion of chromosomes 1p and 19q are implicated as clinically relevant LGG markers. Methods: We performed a comprehensive genome-wide analysis of 293 LGG, incorporating exome sequence, DNA copy number, DNA methylation, messenger RNA and microRNA expression and targeted protein expression. Molecular data was integrated with clinical outcomes to identify biologically robust LGG subtypes. Results: We uncovered 19 mutated genes, including recognized drivers and novel mutations. Unsupervised clustering of genome-wide data from RNA-, DNA- and DNA methylation platforms uncovered concordant classification of three robust, non-overlapping LGG subtypes that were captured with highest correlation by IDH and 1p/19q status. The large majority of IDH wild-type LGG showed strikingly similar genomic aberrations and clinical behavior to primary glioblastoma. Nearly all IDH-mutant LGG lacking 1p/19q co-deletion had mutations in TP53 (94%) and ATRX inactivation (86%). IDH-mutant, 1p/19q-co-deleted LGG had most favorable clinical outcomes and harbored mutations in CIC, FUBP1, NOTCH1 and the TERT promoter. Conclusions: Integration of genome-wide data from multiple platforms cleanly delineates three LGG molecular subtypes concordant with IDH and 1p/19q status: IDH-mutant LGG are either 1p/19 co-deleted or TP53 mutant, whereas most IDH wild-type LGG are molecularly and clinically similar to glioblastoma. Overall, results support a molecular-based, clinically relevant classification for implementation into routine clinical practice. Citation Format: Daniel J. Brat, The Cancer Genome Atlas (TCGA) Research Network. Comprehensive and integrative genomic characterization of diffuse lower grade gliomas. [abstract]. In: Proceedings of the 106th Annual Meeting of the American Association for Cancer Research; 2015 Apr 18-22; Philadelphia, PA. Philadelphia (PA): AACR; Cancer Res 2015;75(15 Suppl):Abstract nr 2974. doi:10.1158/1538-7445.AM2015-2974

A rare presentation of a necrotic undifferentiated ovarian carcinoma in an adolescent girl causing a diagnostic dilemma

Undifferentiated carcinoma is rare tumors with the worst prognosis than any type of surface epithelial carcinoma. They are common in fourth and fifth decade. In most cases, these spread beyond the ovary at the time of diagnosis and cannot be optimally debulked. A 17 years unmarried girl presented with an ovarian tumor, which on ultrasonography measured 9.9 cm Χ 6.7 cm Χ 6.6 cm arising from the left ovary and extending into the pelvis. Right ovary and other organs were unremarkable. Lab investigations: S. alpha fetoprotein (S.AFP) - 0.85 (0.5-2 ng/ml), B - human chorionic gonadotropin - 1.2 (<5 mIU/ml), cancer antigen - 125-127 (35 units/ml), carcinoembryonic antigen - 3.26 (<2.5 ng/ml). At histology, the tumor had undergone extensive areas of necrosis. The focal viable areas showed a poorly differentiated aggressive tumor. Extensive inter-observer variability in diagnosis existed. S.AFP levels were normal. AE1/AE3, EMA were positive while the rest IHC markers did not contribute to any specific diagnosis. The tumor was then diagnosed as undifferentiated carcinoma of the ovary. She received 3 cycles of cisplatin-based chemotherapy postoperatively. Two months later, she developed a metastatic nodule near the umbilicus. She succumbed to her illness within 3 months of diagnosis.

Inter-observer variation in the diagnosis of coronal articular fracture lines in the lunate facet of the distal radius

Several studies support the use of CT for diagnosing coronal fractures of the distal radius but the inter-observer reliability of these observations is less well studied. We tested the null hypothesis that radiographs alone and the combination of radiographs and two-dimensional computed tomography scans (2DCT) have the same inter-observer variation for the diagnosis of coronal articular fracture lines in the distal radius. Using a web-based survey, 63 surgeons were randomized to evaluate 16 fractures of the distal radius on radiographs alone or radiographs and 2DCT for the presence or absence of a coronal fracture line of the lunate facet and, if present, the stability of the fracture. The kappa multirater measure was calculated to estimate agreement between observers. The inter-observer variation in diagnosis of a coronal fracture line was fair with both radiographs and 2DCT, as was the diagnosis of instability of the volar lunate facet fracture when present. Two-dimensional computed tomography does not improve observer agreement on the diagnosis of coronal plane articular fracture lines in the lunate facet of the distal radius.

Variabilidade interobservadores no diagnóstico de lesões precursoras do câncer anal: estudo do cenário habitual

To assess interobserver variability in the diagnosis of anal cancer precursor lesions in the usual scenario of a service consisting of pathologists without previous experience in the diagnosis of these lesions. Five hundred and two anal specimens taken from 372 HIV-positive and HIV-negative patients were analyzed at the Pathology Department of the Tropical Medicine Foundation of Amazonas by three pathologists with extensive experience in the diagnosis of infectious and tropical diseases, but without significant prior experience in the diagnosis of anal cancer precursor lesions. The individual readings of each pathologist were compared to the one following the consensus diagnosis in shared optical microscope by kappa statistics. The absolute agreement between each individual diagnosis and corresponding consensus was poor (kappa = -0.002). Considering only the positive or negative results for anal squamous intraepithelial lesions, we obtained a fair agreement between observers (kappa = 0.35), while the agreement was moderate when the histopathological findings were considered positive or negative for high-grade squamous intraepithelial lesion or cancer (kappa = 0.52). The interobserver variability in histopathologic diagnosis of anal cancer and its precursor lesions among pathologists with little experience in the area is such that the diagnoses in this field and this scenario should always be a consensus.

Risk Factors for Progression of Low-Grade Dysplasia in Patients With Barrett's Esophagus

Data vary on the progression of low-grade dysplasia (LGD) in patients with Barrett's esophagus (BE); in patients with LGD, we investigated the incidence of high-grade dysplasia (HGD) and esophageal adenocarcinoma (EAC) and compared progression in patients with different forms of LGD (prevalent vs incident and multifocal vs unifocal). We assessed the effects of consensus diagnosis of LGD on progression rates to HGD and EAC among expert pathologists. In a multicenter outcomes project, 210 patients with BE and LGD (classified as incident, prevalent, or persistent) were included. Patients were followed up for an average of 6.2 years (959.6 patient-years). Persistent LGD was defined as detection of LGD on ≥2 consecutive occasions during the follow-up period and extent as either unifocal (LGD at one level of BE segment) or multifocal (>1 level). Histology specimens were reviewed by 2 blinded pathologists. Six patients developed EAC (incidence of 0.44%/year), and 21 developed HGD (incidence of 1.6%/year). The incidence of the combination of HGD and EAC was 1.83%/year. There were no associations between presence of prevalent, incident, or persistent LGD and the extent of LGD with progression rates. Based on consensus diagnosis of 88 reviewed specimens, there was no difference in the progression of LGD to either EAC (the incidence based on analyses by the local pathologist was 0.18%/year, the incidence when there was agreement between the local and one central pathologist was 0.21%/year, and the incidence when all 3 pathologists were in agreement was 0.39%/year) or combined HGD and EAC (0.94%/year, 0.87%/year, and 0.84%/year, respectively). Overall, patients with BE and LGD have a low annual incidence of EAC, similar to nondysplastic BE. There are no risk factors for progression and there is significant interobserver variation in diagnosis, even among expert pathologists.

Interobserver variability of teledermoscopy: an international study

Teledermoscopy is a rapidly developing field of dermatology with studies demonstrating excellent agreement with face-to-face diagnosis. However, we are unaware of studies evaluating interobserver variability in diagnosis between dermatologists from different continents. This evaluation is important to determine the robustness of teledermoscopy and allow comparisons to be made between different studies. To assess the interobserver diagnostic variability between five independent experienced dermatologists (A-E) in New Zealand, Australia and the U.S.A. Images from 979 lesions from 206 patients were distributed to five dermatologists. The lesions were viewed and diagnoses recorded using MoleMap Diagnose (MoleMap, Auckland, New Zealand) software. The diagnoses were analysed for interobserver variability. There was excellent agreement between four of five dermatologists (A-D) for lesions that were agreed upon as melanoma (κ = 0·81-0·97) and benign naevus (κ = 0·77-0·82).The fifth dermatologist (E) made a more frequent diagnosis of atypical naevus and melanoma than the others. For nonmelanocytic lesions, there was moderate to very good agreement for seborrhoeic keratosis (κ = 0·64-0·80) and basal cell carcinoma (κ = 0·55-0·67), but poor agreement for invasive squamous cell carcinoma (SCC) (κ = 0·05-0·15). Agreement for actinic keratosis (κ = 0·32-0·67) and SCC in situ (κ = 0·15-0·32) was only moderate. When atypical and benign naevi were grouped together and actinic keratosis and SCC in situ grouped together, there was better agreement among all dermatologists. There was good ability to distinguish malignant from benign lesions (κ = 0·57-0·93). There was good agreement among dermatologists A-D but dermatologist E varied from the group with more frequent diagnosis of melanoma and atypical naevus. This difference could be due to different definition of terms with lack of consensus guidelines in definition of atypical naevus, lack of familiarity with the specific patient population and/or diagnostic drift.

Lymphovascular Invasion in Colorectal Cancer

Lymphovascular invasion (LVI) in colorectal cancer (CRC) is considered a strong stage-independent prognostic factor and influences decisions regarding adjuvant chemotherapy in patients with stage II tumors. However, the degree of interobserver agreement among pathologists for LVI in CRC is largely unknown. This study was undertaken to examine such interobserver variability, and we hypothesized that the use of immunohistochemical markers for vascular and lymphatic channels could improve interobserver agreement. Fifty cases of American Joint Committee on Cancer stage II moderately differentiated CRC from 1990 to 2005 from the pathology archives were selected; mucinous, medullary, and other recognized special subtypes were excluded. Fifty hematoxylin and eosin (H&E) slides (1 from each case) were circulated to 6 gastrointestinal pathologists, who independently assessed small and large vessel invasion. No diagnostic guidelines were given to the participating pathologists; each was instructed to apply the criteria for LVI that he or she used in daily practice. Immunohistochemistry (IHC) for D2-40 and CD31 was performed on corresponding paraffin blocks. The IHC slides were randomized, recirculated, and rescored for LVI. Results were analyzed by kappa (kappa) statistics, which correct for agreement by chance, and for percentage agreement. The average kappa values were determined for the H&E slides (large and small vessel), CD31 (small vessel), and D2-40 (small vessel) (Fig. 1). Agreement was fair for H&E small vessel invasion [kappa=0.28; 95% confidence interval (CI): 0.22-0.34]. The least agreement was seen in interpretation of H&E large vessel invasion (kappa=0.18; 95%CI: 0.11-0.26). Agreement was not improved by use of immunohistochemical stains: CD31 (large vessel, kappa=0.42, 95%CI: 0.20-0.63, small vessel, kappa=0.26, 95%CI: 0.10-0.42) and D2-40 (kappa=0.32, 95%CI: 0.21-0.42). Interobserver variability in diagnosis of LVI was substantial on H&E slides and did not improve upon use of IHC. Agreement in evaluation of large vessel invasion was only slightly higher than would be seen by chance alone. This study highlights the need for criteria in evaluation of LVI, as this assessment may impact patient prognosis and thus change the course of clinical treatment.

Evaluación de la displasia en las enfermedades digestivas

La displasia, o neoplasia intraepitelial, es una proliferación celular neoplásica de carácter no invasivo, que puede preceder o acompañar a una neoplasia invasiva. Su diagnóstico se basa fundamentalmente en criterios histológicos, que incluyen alteraciones citológicas y estructurales, ya que frecuentemente no da lugar a lesiones reconocibles macroscópicamente. En todas las clasificaciones actuales la displasia se divide en dos categorías, bajo y alto grado, con el propósito de intentar evaluar el riesgo y orientar la actitud terapéutica. La clasificación de consenso de Viena pretende aunar criterios y disminuir la variabilidad interobservador en el diagnóstico. En el aparato digestivo, la evaluación de la displasia epitelial tiene especial relevancia en cuatro circunstancias: esófago de Barrett, gastritis crónica, enfermedad inflamatoria intestinal y adenomas colorrectales. Los criterios de diagnóstico y gradación de la displasia son iguales en todas ellas, pero la orientación terapéutica puede variar en función del órgano y el contexto clinicopatológico. Dysplasia, or intraepithelial neoplasia, consists of noninvasive neoplastic cellular proliferation that may precede or accompany invasive neoplasia. Diagnosis is mainly based on histological criteria, which include cytological and structural alterations, since macroscopically identifiable lesions often do not occur. In all current classifications, dysplasia is divided in two categories, low–and high-grade, with the aim of attempting to evaluate risk and guide the therapeutic approach. The classification of the Vienna consensus aims to unity criteria and decrease interobserver variability in diagnosis. In the digestive tract, evaluation of epithelial dysplasia is especially important in four entities: Barrett's esophagus, chronic gastritis, inflammatory bowel disease, and colorectal adenomas. The criteria for diagnosis and dysplasia staging are the same in all these entities, but the therapeutic approach may vary according to the affected organ and the clinico-pathological context.

Inter-observer variation in cytological and histological diagnoses of cervical neoplasia and its epidemiologic implication

Inter-observer variation in cytohistological diagnosis was assessed for 1506 cervical smears and 883 histological slides from four case-control studies on cervical neoplasia. The kappa statistic among a panel of three cytopathologists was highest for diagnosis of invasive cancer (0.70 for cytology and 0.74 for histology), followed by normal/inflammatory in cytology (0.68) and CIN III in histology (0.58). There was also nearly perfect agreement between the final panel diagnoses and the original diagnoses made by local cytopathologists, except for those of CIN III. Inter-observer variation in diagnosis for CIN III was inversely associated with age, number of children (in histology) and sexual activity (in cytology). However, the odds ratios for CIN III calculated by each cytopathologist's diagnosis were not different from each other for any etiologic factor. These results indicate that the diagnoses of invasive cancer and of normal/inflammatory changes are highly reproducible and that the inter-observer variation does not have much impact on the etiologic risk estimates.

Agreement among multiple observers on endoscopic diagnosis of esophageal varices before bleeding

The interobserver variation in diagnosis and grading of esophageal varices may be ascribed by characteristics of the observers as well as to the patients. Assessment of this variation therefore requires the contributions of multiple observers and patients. Twenty-eight patients with cirrhosis without previous bleeding or known presence of varices were subjected to upper gastrointestinal endoscopy. Each endoscopy was videotaped and shown to 22 endoscopists. The varices were graded on a scale of 0 to 3 according to size. Each endoscopist diagnosed varices in 8 to 20 patients (mean = 15.9). Overall agreement on the presence (grades 1 to 3) or absence (grade 0) of varices was 70%. The average kappa value was 0.38 (standard deviation = 0.16). Discrimination between varices graded 0 to 1 and varices graded 2 to 3 gave a higher kappa value (p less than 0.01) of 0.52 (standard deviation = 0.17). There was a large variation in kappa values (range = -0.025 to 0.975). No significant correlation was observed between kappa values for the two dichotomies (range = 0.16). The kappa values were not related to the experience of the endoscopist. Considerable variation in the agreement on diagnosis and grading of esophageal varices was found. These results must be taken into account in the assessment of trials of prophylaxis of first-time variceal bleeding.