Interobserver Variation Research Articles

Analytical aspects of the antinuclear antibody test by HEp-2 indirect immunofluorescence: EFLM report on an international survey.

Detection of antinuclear antibodies (ANA) by indirect immunofluorescence assay using HEp-2cells (HEp-2 IFA) is used to screen for various autoimmune diseases. HEp-2 IFA suffers from variability, which hampers harmonization. A questionnaire was developed to collect information on HEp-2 IFA methodology, computer-assisted diagnosis (CAD) systems, training, inter-observer variability, quality assessment, reagent lot change control, and method verification. The questionnaire was distributed to laboratories by Sciensano (Belgium), national EASI groups (Italy, Croatia, Portugal, Estonia, Greece) and ICAP (worldwide). Answers were obtained by 414 laboratories. The results were analysed in the framework of the recent EFLM/EASI/ICAP ANA recommendations (companion paper). Laboratories used either HEp-2, HEp-2000, or HEp-20-10cells and most laboratories (80%) applied the same screening dilution for children and adults. The conjugate used varied between laboratories [IgG-specific (in 57% of laboratories) vs. polyvalent]. Sixty-nine percent of CAD users reviewed the automatic nuclear pattern and 53% of CAD users did not fully exploit the fluorescence intensity for quality assurance. Internal quality control was performed by 96% of the laboratories, in 52% of the laboratories only with strongly positive samples. Interobserver variation was controlled by 79% of the laboratories. Limited lot-to-lot evaluation was performed by 68% of the laboratories. Method verification was done by 80% of the respondents. Even though many laboratories embrace high-quality HEp-2 IFA, substantial differences in how HEp-2 IFA is performed and controlled remain. Acting according to the EFLM/EASI/ICAP ANA recommendations can improve the global performance and quality of HEp-2 IFA and nurture harmonization.

Recent Advancements in Computed Tomography Assessment of Fibrotic Interstitial Lung Diseases.

Interstitial lung disease (ILD) is a heterogeneous group of disorders with complex and varied imaging manifestations and prognosis. High-resolution computed tomography (HRCT) is the current standard-of-care imaging tool for ILD assessment. However, visual evaluation of HRCT is limited by interobserver variation and poor sensitivity for subtle changes. Such challenges have led to tremendous recent research interest in objective and reproducible methods to examine ILDs. Computer-aided CT analysis to include texture analysis and machine learning methods have recently been shown to be viable supplements to traditional visual assessment through improved characterization and quantification of ILDs. These quantitative tools have not only been shown to correlate well with pulmonary function tests and patient outcomes but are also useful in disease diagnosis, surveillance and management. In this review, we provide an overview of recent computer-aided tools in diagnosis, prognosis, and longitudinal evaluation of fibrotic ILDs, while outlining some of the pitfalls and challenges that have precluded further advancement of these tools as well as potential solutions and further endeavors.

Adverse neonatal outcome and veno-arterial differences in umbilical cord blood pH (ΔpH) at birth: a population-based study of 108,629 newborns

BackgroundUmbilical cord blood gases are routinely used by midwives and obstetricians for quality assurance of birth management and in clinical research. They can form the basis for solving medicolegal issues in the identification of severe intrapartum hypoxia at birth. However, the scientific value of veno-arterial differences in cord blood pH, also known as ΔpH, is largely unknown. By tradition, the Apgar score is frequently used to predict perinatal morbidity and mortality, however significant inter-observer and regional variations decrease its reliability and there is a need to identify more accurate markers of perinatal asphyxia. The aim of our study was to investigate the association of small and large veno-arterial differences in umbilical cord pH, ΔpH, with adverse neonatal outcome.MethodsThis retrospective, population-based study collected obstetric and neonatal data from women giving birth in nine maternity units from Southern Sweden from 1995 to 2015. Data was extracted from the Perinatal South Revision Register, a quality regional health database. Newborns at ≥37 gestational weeks with a complete and validated set of umbilical cord blood samples from both cord artery and vein were included. Outcome measures included: ΔpH percentiles, ‘Small ΔpH’ (10th percentile), ‘Large ΔpH’ (90th percentile), Apgar score (0–6), need for continuous positive airway pressure (CPAP) and admission to neonatal intensive care unit (NICU). Relative risks (RR) were calculated with modified Poisson regression model.ResultsThe study population comprised of 108,629 newborns with complete and validated data. Mean and median ΔpH was 0.08 ± 0.05. Analyses of RR showed that ‘Large ΔpH’ was associated with a decreased RR of adverse perinatal outcome with increasing UApH (at UApH ≥7.20: RR for low Apgar 0.29, P = 0.01; CPAP 0.55, P = 0.02; NICU admission 0.81, P = 0.01). ‘Small ΔpH’ was associated with an increased RR for low Apgar score and NICU admission only at higher UApH values (at UApH 7.15–7.199: RR for low Apgar 1.96, P = 0.01; at UApH ≥7.20: RR for low Apgar 1.65, P = 0.00, RR for NICU admission 1.13, P = 0.01).ConclusionLarge differences between cord venous and arterial pH (ΔpH) at birth were associated with a lower risk for perinatal morbidity including low 5-minute Apgar Score, the need for continuous positive airway pressure and NICU admission when UApH was above 7.15. Clinically, ΔpH may be a useful tool in the assessment of the newborn’s metabolic condition at birth. Our findings may stem from the ability of the placenta to adequately replenish acid-base balance in fetal blood. ‘Large ΔpH’ may therefore be a marker of effective gas exchange in the placenta during birth.

Clinical assessment of a novel machine-learning automated contouring tool for radiotherapy planning.

Contouring has become an increasingly important aspect of radiotherapy due to inverse planning. Several studies have suggested that the clinical implementation of automated contouring tools can reduce inter-observer variation while increasing contouring efficiency, thereby improving the quality of radiotherapy treatment and reducing the time between simulation and treatment. In this study, a novel, commercial automated contouring tool based on machine learning, the AI-Rad Companion Organs RT™ (AI-Rad) software (Version VA31) (Siemens Healthineers, Munich, Germany), was assessed against both manually delineated contours and another commercially available automated contouring software, Varian Smart Segmentation™ (SS) (Version 16.0) (Varian, Palo Alto, CA, United States). The quality of contours generated by AI-Rad in Head and Neck (H&N), Thorax, Breast, Male Pelvis (Pelvis_M), and Female Pelvis (Pevis_F) anatomical areas was evaluated both quantitatively and qualitatively using several metrics. A timing analysis was subsequently performed to explore potential time savings achieved by AI-Rad. Results showed that most automated contours generated by AI-Rad were not only clinically acceptable and required minimal editing, but also superior in quality to contours generated by SS in multiple structures. In addition, timing analysis favored AI-Rad over manual contouring, indicating the largest time saving (753s per patient) in the Thorax area. AI-Rad was concluded to be a promising automated contouring solution that generated clinically acceptable contours and achieved time savings, thereby greatly benefiting the radiotherapy process.

Interobserver variation of clinical oncologists compared to therapeutic radiographers (RTT) prostate contours on T2 weighted MRI.

The implementation of MRI-guided online adaptive radiotherapy has enabled extension of therapeutic radiographers' roles to include contouring. An offline interobserver variability study compared five radiographers' and five clinicians' contours on 10 MRIs acquired on a MR-Linac from 10 patients. All contours were compared to a "gold standard" created from an average of clinicians' contours. The median (range) DSC of radiographers' and clinicians' contours compared to the "gold standard" was 0.91 (0.86-0.96), and 0.93 (0.88-0.97) respectively illustrating non-inferiority of the radiographers' contours to the clinicians. There was no significant difference in HD, MDA or volume size between the groups.

Noninferiority of Artificial Intelligence–Assisted Analysis of Ki-67 and Estrogen/Progesterone Receptor in Breast Cancer Routine Diagnostics

Image analysis assistance with artificial intelligence (AI) has become one of the great promises over recent years in pathology, with many scientific studies being published each year. Nonetheless, and perhaps surprisingly, only few image AI systems are already in routine clinical use. A major reason for this is the missing validation of the robustness of many AI systems: beyond a narrow context, the large variability in digital images due to differences in preanalytical laboratory procedures, staining procedures, and scanners can be challenging for the subsequent image analysis. Resulting faulty AI analysis may bias the pathologist and contribute to incorrect diagnoses and, therefore, may lead to inappropriate therapy or prognosis. In this study, a pretrained AI assistance tool for the quantification of Ki-67, estrogen receptor (ER), and progesterone receptor (PR) in breast cancer was evaluated within a realistic study set representative of clinical routine on a total of 204 slides (72 Ki-67, 66 ER, and 66 PR slides). This represents the cohort with the largest image variance for AI tool evaluation to date, including 3 staining systems, 5 whole-slide scanners, and 1 microscope camera. These routine cases were collected without manual preselection and analyzed by 10 participant pathologists from 8 sites. Agreement rates for individual pathologists were found to be 87.6% for Ki-67 and 89.4% for ER/PR, respectively, between scoring with and without the assistance of the AI tool regarding clinical categories. Individual AI analysis results were confirmed by the majority of pathologists in 95.8% of Ki-67 cases and 93.2% of ER/PR cases. The statistical analysis provides evidence for high interobserver variance between pathologists (Krippendorff's α, 0.69) in conventional immunohistochemical quantification. Pathologist agreement increased slightly when using AI support (Krippendorff α, 0.72). Agreement rates of pathologist scores with and without AI assistance provide evidence for the reliability of immunohistochemical scoring with the support of the investigated AI tool under a large number of environmental variables that influence the quality of the diagnosed tissue images.

Interobserver and Interantibody Reproducibility of HER2 Immunohistochemical Scoring in an Enriched HER2-Low-Expressing Breast Cancer Cohort.

We assessed the interobserver and interantibody reproducibility of HER2 immunohistochemical scoring in an enriched HER2-low-expressing breast cancer cohort. A total of 114 breast cancer specimens were stained by HercepTest (Agilent Dako) and PATHWAY anti-HER2 (4B5) (Ventana) antibody assays and scored by 6 breast pathologists independently using current HER2 guidelines. Level of agreement was evaluated by Cohen κ analysis. Although the interobserver agreement rate for both antibodies achieved substantial agreement, the average rate of agreement for HercepTest was significantly higher than that for the 4B5 clone (74.3% vs 65.1%; P = .002). The overall interantibody agreement rate between the 2 antibodies was 57.8%. Complete interobserver concordance was achieved in 44.7% of cases by HercepTest and 45.6% of cases by 4B5. Absolute agreement rates increased from HER2 0-1+ cases (78.1% by HercepTest and 72.2% by 4B5; moderate agreement) to 2-3+ cases (91.9% by HercepTest and 86.3% by 4B5; almost perfect agreement). Our results demonstrated notable interobserver and interantibody variation on evaluating HER2 immunohistochemistry, especially in cases with scores of 0-1+, although the performance was much more improved among breast-specialized pathologists with the awareness of HER2-low concept. More accurate and reproducible methods are needed for selecting patients who may benefit from the newly approved HER2-targeting agent on HER2-low breast cancers.

Magnetic resonance imaging organ at risk delineation for nasopharyngeal radiotherapy: Measuring the effectiveness of an educational intervention.

Magnetic resonance imaging (MRI) demonstrates superior soft tissue contrast and is increasingly being used in radiotherapy planning. This study evaluated the impact of an education workshop in minimising inter-observer variation (IOV) for nasopharyngeal organs at risk (OAR) delineation on MRI. Ten observers delineated 14 OARs on 4 retrospective nasopharyngeal MRI data sets. Standard contouring guidelines were provided pre-workshop. Following an education workshop on MRI OAR delineation, observers blinded to their original contours repeated the 14 OAR delineations. For comparison, reference volumes were delineated by two head and neck radiation oncologists. IOV was evaluated using dice similarity coefficient (DSC), Hausdorff distance (HD) and relative volume. Location of largest deviations was evaluated with centroid values. Observer confidence pre- and post-workshop was also recorded using a 6-point Likert scale. The workshop was deemed beneficial for an OAR if ≥50% of observers mean scores improved in any metric and ≥50% of observers' confidence improved. All OARs had ≥50% of observers improve in at least one metric. Base of tongue, larynx, spinal cord and right temporal lobe were the only OARs achieving a mean DSC score of ≥0.7. Base of tongue, left and right lacrimal glands, larynx, left optic nerve and right parotid gland all exhibited statistically significant HD improvements post-workshop (P < 0.05). Brainstem and left and right temporal lobes all had statistically significant relative volume improvements post-workshop (P < 0.05). Post-workshop observer confidence improvement was observed for all OARs (P < 0.001). The educational workshop reduced IOV and improved observers' confidence when delineating nasopharyngeal OARs on MRI.

Deep reinforcement learning for cerebral anterior vessel tree extraction from 3D CTA images.

Extracting the cerebral anterior vessel tree of patients with an intracranial large vessel occlusion(LVO) is relevant to investigate potential biomarkers that can contribute to treatment decision making. The purpose of our work is to develop a method that can achieve this from routinely acquired computed tomography angiography(CTA)and computed tomography perfusion(CTP)images. To this end, we regard the anterior vessel tree as a set of bifurcations and connected centerlines. The method consists of a proximal policy optimization(PPO) based deep reinforcement learning (DRL) approach for tracking centerlines, a convolutional neural network based bifurcation detector, and a breadth-first vessel tree construction approach taking the tracking and bifurcation detection results as input. We experimentally determine the added values of various components of the tracker. Both DRL vessel tracking and CNN bifurcation detection were assessed in a cross validation experiment using 115 subjects. The anterior vessel tree formation was evaluated on an independent test set of 25 subjects, and compared to interobserver variation on a small subset of images. The DRL tracking result achieves a median overlapping rate until the first error (1.8mm off the reference standard) of 100, [46, 100]% on 8032 vessels over 115 subjects. The bifurcation detector reaches an average recall and precision of 76% and 87% respectively during the vessel tree formation process. The final vessel tree formation achieves a median recall of 68% and precision of 70%, which is in line with the interobserver agreement.

P254 Deep Learning Model for Distinguishing MES 0 and MES 1 in Patients with Ulcerative Colitis

Abstract Background Endoscopic remission has recently been defined as a Mayo endoscopic sub-score (MES) of 0. Therefore, patients with MES 1 need to step up to achieve an MES of 0 on the endoscopic score. In discriminating between MES 0 and 1, the inter-observer variation was very severe among the endoscopists. This study aimed to narrow the gap in distinguishing between MES 0 and MES1 using a deep learning model. Methods From the endoscopic images of 492 ulcerative colitis (UC) patients with MES improvement (MES 0 or MES 1) from January 2018 to December 2019 at this center, two representative images of the colon and rectum were selected, and a total of 984 images were analyzed. Our model was composed of a convolutional neural network (CNN)-based encoder, two auxiliary classifiers for the colon and rectum, and a final MES classifier for the combined image features of the two inputs. The following three experiments were conducted to validate and test the proposed model: 12-fold cross-validation for representative model selection, performance comparison with a novice group through an internal test dataset, and an external test with Hyperkvasir, a public gastrointestinal endoscopic image dataset. Results In the internal test, our model showed an F1-score of 0.92. This was, on average, 0.11 higher than each of the seven novices and 0.02 higher than their consensus. Considering MES 1 as positive, the area under the receiver operating characteristic curve (AUROC) was 0.97, and the area under the precision-recall curve (AUPRC) was 0.98. In the external test with Hyperkvasir, our model showed an F1-score of 0.89, AUROC of 0.86, and AUPRC of 0.97. Conclusion We found that the proposed CNN-based model that integrates the image features of the colon and rectum has superior performance in discriminating between MES 0 and MES 1 in patients with UC. Further prospective studies are required to prove the clinical utility of our model in clinical practice.

Visual interstitial fibrosis assessment as continuous variable in protocol renal transplant biopsies.

In current renal transplant pathology practice interstitial fibrosis is visually assessed in categories according to the Banff classification. As this has a moderate reproducibility, which is little ameliorated by morphometric analysis, we investigated whether visual renal fibrosis assessment is feasible on a continuous scale, i.e. as percentage affected area of the cortex. Protocol renal biopsies taken at transplantation (n=125), three (n=73) and twelve months (n=88) after transplantation were visually scored in categories (Banff), and percentages for interstitial fibrosis (ci). Interobserver variation (ICC and weighted κ) was assessed, and morphometric analysis on Sirus Red stained sections was performed. Correlations between the different methods and their association with donor age and eGFR 1 year and 5 year post transplant were analyzed using Pearson's or Spearman's rho. Interobserver agreement was equivalent for Banffci and %ci (κ0.713 vs. ICC 0.792), and for BanffIF/TA and %IF/TA (κ0.615 vs. ICC 0.743). Both Banffci and %ci were associated with Sirius Red morphometry in three and twelve month. With all three methods, a significant correlation was found between donor age and fibrosis in the implantation biopsy, and between fibrosis in the 12 month biopsy and eGFR at 1 and 5 years (eGFR at 1 year: Sirius Red ρ-0.487, %ci ρ-0.393, Banffci ρ-0.413, all p<0.01, eGFR at 5 years: Sirius Red ρ-0.392, %ci ρ-0.333, Banffci ρ-0.435, all p<0.01. Interstitial fibrosis assessment on a continuous scale can be used next to scoring in categories according to the Banff classification in protocol renal transplant biopsies.

Effect of Three-Month Weight Training Program on Resting Heart Rate and Blood Pressure in Healthy Young Adult Males.

Background A planned and structured physical activity is the cornerstone of improving and sustaining body fitness. The underlying reason for exercise is personal interest, maintaining good health, or improving endurance for sports. Furthermore, exercise may be either isotonic or isometric. In weight training, different types of weight are being used and are lifted against gravity, and this type of exercise is of isotonic type. Objective The objective of this study was to observe the changes in heart rate (HR) and blood pressure (BP) after a three-month weight training intervention in healthy young adult males and to compare them with age-matched healthy control. Materials and methods We initially recruited a total of 25 healthy male volunteers for the study and 25 age-matched participants in the control group. Research participants were screened for any existing diseases and suitability for participation by the Physical Activity Readiness Questionnaire. We lost one participant from the study group and three participants from the control group in the follow-up. A structured weight training program (five days a week for three months) was applied for the study group with direct instruction and supervision in a controlled environment. A single expert clinician measured baseline and post-program (after three months) HR and BP (measured after 15 minutes, 30 minutes, and 24 hours of rest after exercise) to reduce any possible inter-observer variation. For comparing the pre-exercise and post-exercise parameters, we considered the post-exercise measurement, which was done after 24 hours of exercise. Mann-Whitney Utest, Wilcoxon signed-rank test, and Friedman test compared the parameters. Result A total of 24 males with a median age of 19 years (Q1-Q3: 18-20) participated as the study group and 22 males with the same median age were the control group. At the end of the three-month weight training exercise program, there was no significant change in the HR (median 82 versus 81 bpm, p = 0.27) in the study group. The systolic BP was increased (median 116 versus 126 mmHg, p <0.0001) after three months of the weight training program. In addition, pulse pressure and mean arterial BP was also increased. However, diastolic (median 76 versus 80 mmHg, p = 0.11) BP was not significantly increased. There was no change in HR, systolic and diastolic BP in the control group. Conclusion A structured weight training program (used in this study) for three months may sustain an increase in systolic BP at rest in young adult males while diastolic BP remains the same. The HR remains unchanged before and after the exercise program. Hence, those enrolling in such an exercise program should be monitored frequently for changes in BP over time for any timely intervention appropriate for the candidate. However, being a small-scale study, the result of this study would be validated by further observing the underlying causes of the increment of systolic blood pressure.

ENDOSCOPIC SCORE INTER-READER CONCORDANCE ANALYSIS FROM A PHASE 2B STUDY IN PATIENTS WITH MILD-TO-MODERATE UC

Abstract BACKGROUND Endoscopic assessment of disease severity is a key component of ulcerative colitis (UC) clinical studies. However, interobserver variability may contribute to bias in selecting study candidates and evaluating response rates. Scoring by blinded central readers has become the recommended standard method of assessment to minimize bias and interobserver variance. We present concordance rates between central and local readings from a Phase 2b study of investigational microbiome therapeutic SER-287 in patients with mild-to-moderate UC. METHODS This double-blind, placebo-controlled study screened 494 and randomized 203 adults (51.7% male; mean age 45.6 years; 83.7% white) 1:1:1 to 3 arms (Figure 1). Active UC was defined as ≥15 cm disease from the anal verge, a Three-Component Modified Mayo Score of 3-7, endoscopic subscore (ES) ≥1, and stool frequency subscore ≥1. Endoscopy was performed at screening, after induction, after exploratory maintenance or open-label, and for flare assessments. Endoscopic videos were scored by each site’s gastroenterologist (local ES), and by any 1 of 4 blinded central readers (Alimentiv; Ontario), with adjudication by a 2nd blinded central reader (Alimentiv), if necessary. If all 3 scores were discordant, the median score was designated as the final ES. Concordance between local and central readers were analyzed by timepoint and in total and quantified using weighted kappa statistics. RESULTS Concordance analyses were conducted for 608 ES comparisons between local and the 1st central reader (Local vs Central1 [LvC1]), and 337 comparisons where adjudication was necessary (Local vs Central2 [LvC2]; Central1 vs Central2 [C1vC2]). Observed concordance rates were: -LvC1: 44% (271/608, kappa = 0.61, Figure 2); -LvC2: 26% (86/337, kappa = 0.46); -C1vC2: 64% (216/337, kappa = 0.74). Concordance rates were comparable across study periods: -Screening: -LvC1: 42% (131/310, kappa = 0.59) -LvC2: 27% (48/179, kappa = 0.48) -C1vC2: 66% (118/179, kappa = 0.76) -Week 11: -LvC1: 46% (79/170, kappa = 0.61) -LvC2: 25% (22/91, kappa = 0.46) -C1vC2: 62% (56/91, kappa = 0.75) Local readers scored lower than C1 readers in 274/337 (81%) discordant cases (Figure 2), and lower than C2 in 221/251 (88%) discordant cases. The most frequent sources of discordance were L=ES2 vs C1=ES3 (153 videos) and L=ES1 vs C1=ES2 (77 videos). CONCLUSIONS In this Phase 2b study, low ES concordance highlights marked interobserver variability between local and blinded central readers. Local scores were consistently lower than central scores across all study time points. Despite low concordance rates, efficacy conclusions remain unchanged if only local ES scores were used for endpoint assessments. Objective, standardized, and consistent methods for evaluating ES are needed to better quantify disease severity and the impact of UC therapies.

QUANTITATIVE AND EXPLAINABLE ARTIFICIAL INTELLIGENCE (AI)-POWERED APPROACHES TO PREDICT ULCERATIVE COLITIS DISEASE ACTIVITY FROM HEMATOXYLIN AND EOSIN (H&amp;E)-STAINED WHOLE SLIDE IMAGES (WSI)

Abstract BACKGROUND Microscopic inflammation has been shown to be an important indicator of disease activity in ulcerative colitis (UC). However, manual histologic scoring is semi-quantitative and subject to interobserver variation, and AI-based solutions often lack interpretability. Here we report two distinct quantitative approaches to predict disease activity scores and histological remission using AI-powered digital pathology. Both the random forest classifier (RFC) and graph neural network (GNN) further provide explainability and biological insight by identifying histological features informing model predictions. METHODS Convolutional neural networks (CNNs) were developed using &amp;gt;162k annotations on 820 WSI of H&amp;E-stained colorectal biopsies for pixel-level identification of tissue regions (e.g. crypt abscesses, erosion/ulceration) and cell types (e.g. neutrophils, plasma cells). All WSI were scored by 5 board-certified pathologists using the Nancy Histological Index (NHI) to establish consensus ground truth. A rich, quantitative set of human interpretable features that capture CNN predictions of the tissue region and cell type across each WSI was extracted and used to train a RFC to predict slide-level NHI score. To test the hypothesis that tissue region spatial relationships and cellular composition can inform AI-based predictions of disease activity, a separate GNN was trained, using nodes defined by spatially-resolved CNN model-generated outputs, to predict NHI score. The RFC and GNN also predicted histologic remission (NHI&amp;lt;2). Feature importance was calculated for all combinations of RFC (Fig. 1), and the GNNExplainer was applied to locate important interactions between regions in the tissue and identify features significantly contributing to GNN predictions (Fig. 2). RESULTS The RFC and GNN both predicted histologic remission with high accuracy (weighted kappa 0.87 and 0.85, respectively). Both models also identified histologic features relevant to disease activity predictions. Some features are well established, e.g. infiltrated epithelium or neutrophil cell features distinguish cases with histologic remission. The models also identified features beyond those assessed by the NHI, e.g. area proportion of basal plasmacytosis associated with predictions of NHI 2 and 3. Other features not previously implicated in UC disease activity were also identified, e.g. intraepithelial lymphocytes differentiate cases with NHI 3. CONCLUSIONS We report quantitative and interpretable AI-powered approaches for UC histological assessment. CNN identification of UC histology was used as input to two distinct disease activity classifiers that showed strong concordance with consensus pathologist scoring. Both approaches provide interpretable features that explain model predictions and that may be used to inform biomarker selection and clinical development efforts.

Considerable interobserver variation calls for unambiguous definitions of thyroid nodule ultrasound characteristics.

Thyroid nodule ultrasound characteristics are used as an indication for fine-needle aspiration cytology, usually as the basis for Thyroid Imaging Reporting and Data System (TIRADS) score calculation. Few studies on interobserver variation are available, all of which are based on analysis of preselected still ultrasound images and often lack surgical confirmation. After the blinded online evaluation of video recordings of the ultrasound examinations of 47 consecutive malignant and 76 consecutive benign thyroid lesions, 7 experts from 7 thyroid centers answered 17 TIRADS-related questions. Surgical histology was the reference standard. Interobserver variations of each ultrasound characteristic were compared using Gwet's AC1 inter-rater coefficients; higher values mean better concordance, the maximum being 1.0. On a scale from 0.0 to 1.0, the Gwet's AC1 values were 0.34, 0.53, 0.72, and 0.79 for the four most important features in decision-making, i.e. irregular margins, microcalcifications, echogenicity, and extrathyroidal extension, respectively. The concordance in the discrimination between mildly/moderately and very hypoechogenic nodules was 0.17. The smaller the nodule size the better the agreement in echogenicity, and the larger the nodule size the better the agreement on the presence of microcalcifications. Extrathyroidal extension was correctly identified in just 45.8% of the cases. Examination of video recordings, closely simulating the real-world situation, revealed substantial interobserver variation in the interpretation of each of the four most important ultrasound characteristics. In view of the importance for the management of thyroid nodules, unambiguous and widely accepted definitions of each nodule characteristic are warranted, although it remains to be investigated whether this diminishes observer variation.

Objective hand eczema severity assessment with automated lesion anatomical stratification.

Hand eczema (HE) is one of the most frequent dermatoses, known to be both relapsing and remitting. Regular and precise evaluation of the disease severity is key for treatment management. Current scoring systems such as the hand eczema severity index (HECSI) suffer from intra- and inter-observer variance. We propose an automated system based on deep learning models (DLM) to quantify HE lesions' surface and determine their anatomical stratification. In this retrospective study, a team of 11 experienced dermatologists annotated eczema lesions in 312 HE pictures, and a medical student created anatomical maps of 215 hands pictures based on 37 anatomical subregions. Each data set was split into training and test pictures and used to train and evaluate two DLMs, one for anatomical mapping, the other for HE lesions segmentation. On the respective test sets, the anatomy DLM achieved average precision and sensitivity of 83% (95% confidence interval [CI] 80-85) and 85% (CI 82-88), while the HE DLM achieved precision and sensitivity of 75% (CI 64-82) and 69% (CI 55-81). The intraclass correlation of the predicted HE surface with dermatologists' estimated surface was 0.94 (CI 0.90-0.96). The proposed method automatically predicts the anatomical stratification of HE lesions' surface and can serve as support to evaluate hand eczema severity, improving reliability, precision and efficiency over manual assessment. Furthermore, the anatomical DLM is not limited to HE and can be applied to any other skin disease occurring on the hands such as lentigo or psoriasis.

Impact of MRI resolution for Linac-based stereotactic radiosurgery.

Magnetic resonance imaging (MRI) is a standard imaging modality in intracranial stereotactic radiosurgery (SRS) for defining target volumes. However, wide disparities in MRI resolution exist, which could directly impact accuracy of target delineation. Here, sequences with various MRI resolution were acquired on phantoms to evaluate the effect on volume definition and dosimetric consequence for cranial SRS. Four T1-weighted MR sequences with increasing 3D resolution were compared, including two Spin Echo (SE) 2D acquisitions with 5mm and 3mm slice thickness (SE5mm, SE3mm) and two gradient echo 3D acquisitions (TFE, BRAVO). The voxel sizes were 0.4×0.4×5.0, 0.5×0.5×3.0, 0.9×0.9×1.25, and 0.4×0.4×0.5 mm3, respectively. Four phantoms with simulated lesions of different shape and volume (range, 0.53-25.0 cm3) were imaged, resulting in 16 total sets of MRIs. Four radiation oncologists provided contours on individual MR image set. All observer contours were compared with ground truth, defined on CT image according to the absolute dimensions of the target structure, using Dice similarity coefficient (DSC), Hausdorff distance (HD), mean distance-to-agreement (MDA), and the ratio between reconstructed and true volume (Ratiovol ). For dosimetric consequence, SRS plans targeting observer volumes were created. The true Paddick conformity index ( ), calculated with true target volume, was correlated with quality of observer volume. All measures of observer contours improved as increasingly higher MRI resolution was provided from SE5mm to BRAVO. The improvement in DSC, HD and MDA was statistically significant (p<0.01). Dosimetrically, strongly correlated with DSC of the planning observer volume (Pearson's r=0.94, p<0.00001). Significant improvement in target definition and reduced inter-observer variation was observed as the MRI resolution improved, which also improved the quality of SRS plans. Results imply that high resolution 3D MR sequences should be used to minimize potential errors in target definition, and multi-slice 2D sequences should be avoided.

Identification of Highly Sensitive Pleural Effusion Protein Biomarkers for Malignant Pleural Mesothelioma by Affinity-Based Quantitative Proteomics.

Malignant pleural mesothelioma (MPM) is an asbestos-associated, highly aggressive cancer characterized by late-stage diagnosis and poor prognosis. Gold standards for diagnosis are pleural biopsy and cytology of pleural effusion (PE), both of which are limited by low sensitivity and markedly inter-observer variations. Therefore, the assessment of PE biomarkers is considered a viable and objective diagnostic tool for MPM diagnosis. We applied a novel affinity-enrichment mass spectrometry-based proteomics method for explorative analysis of pleural effusions from a prospective cohort of 84 patients referred for thoracoscopy due to clinical suspicion of MPM. Protein biomarkers with a high capability to discriminate MPM from non-MPM patients were identified, and a Random Forest algorithm was applied for building classification models. Immunohistology of pleural biopsies confirmed MPM in 40 patients and ruled out MPM in 44 patients. Proteomic analysis of pleural effusions identified panels of proteins with excellent diagnostic properties (90-100% sensitivities, 89-98% specificities, and AUC 0.97-0.99) depending on the specific protein combination. Diagnostic proteins associated with cancer growth included galactin-3 binding protein, testican-2, haptoglobin, Beta ig-h3, and protein AMBP. Moreover, we also confirmed previously reported diagnostic accuracies of the MPM markers fibulin-3 and mesothelin measured by two complementary mass spectrometry-based methods. In conclusion, a novel affinity-enrichment mass spectrometry-based proteomics identified panels of proteins in pleural effusion with extraordinary diagnostic accuracies, which are described here for the first time as biomarkers for MPM.

Anastomotic perfusion assessment with indocyanine green in robot-assisted low-anterior resection, a multicenter study of interobserver variation

BackgroundSecuring sufficient blood perfusion to the anastomotic area after low-anterior resection is a crucial factor in preventing anastomotic leakage (AL). Intra-operative indocyanine green fluorescent imaging (ICG-FI) has been suggested as a tool to assess perfusion. However, knowledge of inter-observer variation among surgeons in the interpretation of ICG-FI is sparse. Our primary objective was to evaluate inter-observer variation among surgeons in the interpretation of bowel blood-perfusion assessed visually by ICG-FI. Our secondary objective was to compare the results both from the visual assessment of ICG and from computer-based quantitative analyses of ICG-FI between patients with and without the development of AL.MethodA multicenter study, including patients undergoing robot-assisted low anterior resection with stapled anastomosis. ICG-FI was evaluated visually by the surgeon intra-operatively. Postoperatively, recorded videos were anonymized and exchanged between centers for inter-observer evaluation. Time to visibility (TTV), time to maximum visibility (TMV), and time to wash-out (TWO) were visually assessed. In addition, the ICG-FI video-recordings were analyzed using validated pixel analysis software to quantify blood perfusion.ResultsFifty-five patients were included, and five developed clinical AL. Bland–Altman plots (BA plots) demonstrated wide inter-observer variation for visually assessed fluorescence on all parameters (TTV, TMV, and TWO). Comparing leak-group with no-leak group, we found no significant differences for TTV: Hazard Ratio; HR = 0.82 (CI 0.32; 2.08), TMV: HR = 0.62 (CI 0.24; 1.59), or TWO: HR = 1.11 (CI 0.40; 3.11). In the quantitative pixel analysis, a lower slope of the fluorescence time-curve was found in patients with a subsequent leak: median 0.08 (0.07;0.10) compared with non-leak patients: median 0.13 (0.10;0.17) (p = 0.04).ConclusionThe surgeon’s visual assessment of the ICG-FI demonstrated wide inter-observer variation, there were no differences between patients with and without AL. However, quantitative pixel analysis showed a significant difference between groups.Trial RegistrationClinicalTrials.gov Identifier: NCT04766060.

Interobserver variation in clinical target volume (CTV) delineation for stereotactic radiotherapy to non-spinal bone metastases in prostate cancer: CT, MRI and PET/CT fusion

Background and purposeThe use of SBRT for the treatment of oligometastatic prostate cancer is increasing rapidly. While consensus guidelines are available for non-spinal bone metastases practice continues to vary widely. The aim of this study is to look at inter-observer variability in the contouring of prostate cancer non-spinal bone metastases with different imaging modalities. Materials and methods15 metastases from 13 patients treated at our centre were selected. 4 observers independently contoured clinical target volumes (CTV) on planning CT alone, planning CT with MRI fusion, planning CT with PET-CT fusion and planning CT with both MRI and PET-CT fusion combined. The mean inter-observer agreement on each modality was compared by measuring the delineated volume, generalized conformity index (CIgen), and the distance of the centre of mass (dCOM), calculated per metastasis and imaging modality. ResultsMean CTV volume delineated on planning CT with MRI and PET-CT fusion combined was significantly larger compared to other imaging modalities (p = 0.0001). CIgen showed marked variation between modalities with the highest agreement between planning CT + PET-CT (mean CIgen 0.55, range 0.32–0.73) and planning CT + MRI + PET-CT (mean CIgen 0.59, range 0.34–0.73). dCOM showed small variations between imaging modalities but a significantly shorter distance found on planning CT + PET-CT when compared with planning CT + PET-CT + MRI combined (p = 0.03). ConclusionsHighest consistency in CTV delineation between observers was seen with planning CT + PET-CT and planning CT + PET-CT + MRI combined.