Lesion Annotations Research Articles

Microtomography to traditional dental radiograph: Projecting 3D initial proximal caries lesion annotations for enhanced radiographic delineation.

This study was undertaken to generate high-quality radiographic annotations of initial proximal carious lesions based on micro-CT scans. Specifically, we projected manually and automatically acquired annotations of micro-CT scans onto corresponding traditional dental radiographs. We utilized the Diagnostic Insights for Radiographic Early-caries with micro-CT (ACTA-DIRECT) dataset of manually annotated initial proximal carious lesions in micro-CT scans and radiographs, the former serving as reference-standard. Production of high-quality radiographic annotations entailed the following: (1) acquiring a reference-standard (for a semi-automated approach) or generating a fully automated micro-CT-based annotation (for a fully automated approach); (2) simulating the corresponding radiograph by projecting the micro-CT scan to find the suitable projection parameters; and (3) superimposing micro-CT-based caries annotations onto radiographs, using identical projection parameters. To evaluate subsequent accuracy of the annotations on radiograph, we assessed the sensitivity, specificity, and International Caries Classification and Management System (ICCMS) staging of micro-CT-based automated annotations. Projection accuracy was qualitatively gauged. Micro-CT-based automated annotations outperformed conventional annotations achieving a sensitivity of 50% (95% confidence interval [CI]: 42-59%) compared to 42% (95% CI: 34-51%) and specificity of 99% (95% CI: 96-100%) compared to 92% (95% CI: 87-94%]). Among correctly identified micro-CT-based automated annotations, 94% (61/65) were also accurately classified; and 80% of micro-CT projections were ranked as suitably similar to corresponding radiographs. Micro-CT imaging offers resource-rich depictions, enabling more accurate annotations than those achievable through conventional means. By projecting micro-CT-based annotations of initial proximal caries onto radiographs, some limitations of the conventional radiograph annotation process may be overcome.

Real-World and Clinical Trial Validation of a Deep Learning Radiomic Biomarker for PD-(L)1 Immune Checkpoint Inhibitor Response in Advanced Non-Small Cell Lung Cancer.

This study developed and validated a novel deep learning radiomic biomarker to estimate response to immune checkpoint inhibitor (ICI) therapy in advanced non-small cell lung cancer (NSCLC) using real-world data (RWD) and clinical trial data. Retrospective RWD of 1,829 patients with advanced NSCLC treated with PD-(L)1 ICIs were collected from 10 academic and community institutions in the United States and Europe. The RWD included data sets for discovery (Data Set A-Discovery, n = 1,173) and independent test (Data Set B, n = 458). A radiomic pipeline, containing a deep learning feature extractor and a survival model, generated the computed tomography (CT) response score (CTRS) applied to the pretreatment routine CT/positron emission tomography (PET)-CT scan. An enhanced CTRS (eCTRS) also incorporated age, sex, treatment line, and lesion annotations. Performance was evaluated against progression-free survival (PFS) and overall survival (OS). Biomarker generalizability was further evaluated using a secondary analysis of a prospective clinical trial (ClinicalTrials.gov identifier: NCT02573259) evaluating the PD-1 inhibitor sasanlimab in second or later line of treatment (Data Set C, n = 54). In RWD Test Data Set B, the CTRS identified patients with a high probability of response to ICI with a PFS hazard ratio (HR) of 0.46 (95% CI, 0.26 to 0.82) and an OS HR of 0.50 (95% CI, 0.28 to 0.92) in the first-line ICI monotherapy cohort, after adjustment for baseline covariates including the PD-L1 tumor proportion score. In Clinical Trial Data Set C, the CTRS demonstrated an adjusted PFS HR of 1.03 (95% CI, 0.43 to 2.47) and an OS HR of 0.33 (95% CI, 0.14 to 0.91). The CTRS and eCTRS outperformed traditional imaging biomarkers of lesion size in PFS and OS for RWD Test Data Set B and in OS for the Clinical Trial Data Set. The study developed and validated a deep learning radiomic biomarker using pretreatment routine CT/PET-CT scans to identify ICI benefit in advanced NSCLC.

Cross-shaped windows transformer with self-supervised pretraining for clinically significant prostate cancer detection in bi-parametric MRI.

Bi-parametric magnetic resonance imaging (bpMRI) has demonstrated promising results in prostate cancer (PCa) detection. Vision transformers have achieved competitive performance compared to convolutional neural network (CNN) in deep learning, but they need abundant annotated data for training. Self-supervised learning can effectively leverage unlabeled data to extract useful semantic representations without annotation and its associated costs. This study proposes a novel self-supervised learning framework and a transformer model to enhance PCa detection using prostate bpMRI. We introduce a novel end-to-end Cross-Shaped windows (CSwin) transformer UNet model, CSwin UNet, to detect clinically significant prostate cancer (csPCa) in prostate bpMRI. We also propose a multitask self-supervised learning framework to leverage unlabeled data and improve network generalizability. Using a large prostate bpMRI dataset (PI-CAI) with 1476 patients, we first pretrain CSwin transformer using multitask self-supervised learning to improve data-efficiency and network generalizability. We then finetune using lesion annotations to perform csPCa detection. We also test the network generalization using a separate bpMRI dataset with 158 patients (Prostate158). Five-fold cross validation shows that self-supervised CSwin UNet achieves 0.888±0.010 aread under receiver operating characterstics curve (AUC) and 0.545±0.060 Average Precision (AP) on PI-CAI dataset, significantly outperforming four comparable models (nnFormer, Swin UNETR, DynUNet, Attention UNet, UNet). On model generalizability, self-supervised CSwin UNet achieves 0.79 AUC and 0.45 AP, still outperforming all other comparable methods and demonstrating good generalization to external data. This study proposes CSwin UNet, a new transformer-based model for end-to-end detection of csPCa, enhanced by self-supervised pretraining to enhance network generalizability. We employ an automatic weighted loss (AWL) to unify pretext tasks, improving representation learning. Evaluated on two multi-institutional public datasets, our method surpasses existing methods in detection metrics and demonstrates good generalization to external data.

RDAG U-Net: An Advanced AI Model for Efficient and Accurate CT Scan Analysis of SARS-CoV-2 Pneumonia Lesions.

Background/Objective: This study aims to utilize advanced artificial intelligence (AI) image recog-nition technologies to establish a robust system for identifying features in lung computed tomog-raphy (CT) scans, thereby detecting respiratory infections such as SARS-CoV-2 pneumonia. Spe-cifically, the research focuses on developing a new model called Residual-Dense-Attention Gates U-Net (RDAG U-Net) to improve accuracy and efficiency in identification. Methods: This study employed Attention U-Net, Attention Res U-Net, and the newly developed RDAG U-Net model. RDAG U-Net extends the U-Net architecture by incorporating ResBlock and DenseBlock modules in the encoder to retain training parameters and reduce computation time. The training dataset in-cludes 3,520 CT scans from an open database, augmented to 10,560 samples through data en-hancement techniques. The research also focused on optimizing convolutional architectures, image preprocessing, interpolation methods, data management, and extensive fine-tuning of training parameters and neural network modules. Result: The RDAG U-Net model achieved an outstanding accuracy of 93.29% in identifying pulmonary lesions, with a 45% reduction in computation time compared to other models. The study demonstrated that RDAG U-Net performed stably during training and exhibited good generalization capability by evaluating loss values, model-predicted lesion annotations, and validation-epoch curves. Furthermore, using ITK-Snap to convert 2D pre-dictions into 3D lung and lesion segmentation models, the results delineated lesion contours, en-hancing interpretability. Conclusion: The RDAG U-Net model showed significant improvements in accuracy and efficiency in the analysis of CT images for SARS-CoV-2 pneumonia, achieving a 93.29% recognition accuracy and reducing computation time by 45% compared to other models. These results indicate the potential of the RDAG U-Net model in clinical applications, as it can accelerate the detection of pulmonary lesions and effectively enhance diagnostic accuracy. Additionally, the 2D and 3D visualization results allow physicians to understand lesions' morphology and distribution better, strengthening decision support capabilities and providing valuable medical diagnosis and treatment planning tools.

Noise Resilience in Dermoscopic Image Segmentation: Comparing Deep Learning Architectures for Enhanced Accuracy

Skin diseases and lesions can be ambiguous to recognize due to the similarity of lesions and enhanced imaging features. In this study, we compared three cutting-edge deep learning frameworks for dermoscopic segmentation: U-Net, SegAN, and MultiResUNet. We used a dermoscopic dataset including detailed lesion annotations with segmentation masks to help train and evaluate models on the precise localization of melanomas. SegAN is a special type of Generative Adversarial Network (GAN) that introduces a new architecture by adding generator and discriminator steps. U-Net has become a common strategy in segmentation to encode and decode image features for limited data. MultiResUNet is a U-Net-based architecture that overcomes the insufficient data problem in medical imaging by extracting contextual details. We trained the three frameworks on colored images after preprocessing. We added incremental Gaussian noise to measure the robustness of segmentation performance. We evaluated the frameworks using the following parameters: accuracy, sensitivity, specificity, Dice and Jaccard coefficients. Our accuracy results show that SegAN (92%) and MultiResUNet (92%) both outperform U-Net (86%), which is a well-known segmentation framework for skin lesion analysis. MultiResUNet sensitivity (96%) outperforms the methods in the challenge leaderboard. These results suggest that SegAN and MultiResUNet are more resistant techniques against noise in dermoscopic segmentation.

A graph-theoretic approach for the analysis of lesion changes and lesions detection review in longitudinal oncological imaging

Radiological follow-up of oncology patients requires the detection of lesions and the quantitative analysis of lesion changes in longitudinal imaging studies of patients, which is time-consuming and requires expertise.We present here a new method and workflow for the analysis and review of lesions and volumetric lesion changes in longitudinal scans of a patient. The generic graph-based method consists of lesion matching, classification of changes in individual lesions, and detection of patterns of lesion changes computed from the properties of the graph and its connected components. The workflow guides clinicians in the detection of missed lesions and wrongly identified lesions in manual and computed lesion annotations using the analysis of lesion changes. It serves as a heuristic method for the automatic revision of ground truth lesion annotations in longitudinal scans.The methods were evaluated on longitudinal studies of patients with three or more examinations of metastatic lesions in the lung (19 patients, 83 CT scans, 1178 lesions), the liver (18 patients, 77 CECT scans, 800 lesions) and the brain (30 patients, 102 T1W-Gad MRI scans, 317 lesions) with ground-truth lesion annotations. Lesion matching yielded a precision of 0.92–1.0 and recall of 0.91–0.99. The classification of changes in individual lesions yielded an accuracy of 0.87–0.97. The classification of patterns of lesion changes yielded an accuracy of 0.80–0.94. The lesion detection review workflow applied to manual and computed lesion annotations yielded 120 and 55 missed lesions and 20 and 164 wrongly identified lesions for all longitudinal studies of patients, respectively.The automatic analysis of lesion changes and review of lesion detection in longitudinal studies of oncological patients helps detect missed lesions and wrongly identified lesions. This method may help improve the accuracy of radiological interpretation and the disease status evaluation.

Three scans are better than two for follow-up: An automatic method for finding missed and misidentified lesions in cross-sectional follow-up of oncology patients

PurposeMissed and misidentified neoplastic lesions in longitudinal studies of oncology patients are pervasive and may affect the evaluation of the disease status. Two newly identified patterns of lesion changes, lone lesions and non-consecutive lesion changes, may help radiologists to detect these lesions. This study evaluated a new interpretation revision workflow of lesion annotations in three or more consecutive scans based on these suspicious patterns. MethodsThe interpretation revision workflow was evaluated on manual and computed lesion annotations in longitudinal oncology patient studies. For the manual revision, a senior radiologist and a senior neurosurgeon (the readers) manually annotated the lesions in each scan and later revised their annotations to identify missed and misidentified lesions with the workflow using the automatically detected patterns. For the computerized revision, lesion annotations were first computed with a previously trained nnU-Net and were then automatically revised with an AI-based method that automates the workflow readers’ decisions. The evaluation included 67 patient studies with 2295 metastatic lesions in lung (19 patients, 83 CT scans, 1178 lesions), liver (18 patients, 77 CECT scans, 800 lesions) and brain (30 patients, 102 T1W-Gad MRI scans, 317 lesions). ResultsRevision of the manual lesion annotations revealed 120 missed lesions and 20 misidentified lesions in 31 out of 67 (46%) studies. The automatic revision reduced the number of computed missed lesions by 55 and computed misidentified lesions by 164 in 51 out of 67 (76%) studies. ConclusionAutomatic analysis of three or more consecutive volumetric scans helps find missed and misidentified lesions and may improve the evaluation of temporal changes of oncological lesions.

Learning Without Real Data Annotations to Detect Hepatic Lesions in PET Images.

Deep neural networks have been recently applied to lesion identification in fluorodeoxyglucose (FDG) positron emission tomography (PET) images, but they typically rely on a large amount of well-annotated data for model training. This is extremely difficult to achieve for neuroendocrine tumors (NETs), because of low incidence of NETs and expensive lesion annotation in PET images. The objective of this study is to design a novel, adaptable deep learning method, which uses no real lesion annotations but instead low-cost, list mode-simulated data, for hepatic lesion detection in real-world clinical NET PET images. We first propose a region-guided generative adversarial network (RG-GAN) for lesion-preserved image-to-image translation. Then, we design a specific data augmentation module for our list-mode simulated data and incorporate this module into the RG-GAN to improve model training. Finally, we combine the RG-GAN, the data augmentation module and a lesion detection neural network into a unified framework for joint-task learning to adaptatively identify lesions in real-world PET data. The proposed method outperforms recent state-of-the-art lesion detection methods in real clinical 68Ga-DOTATATE PET images, and produces very competitive performance with the target model that is trained with real lesion annotations. With RG-GAN modeling and specific data augmentation, we can obtain good lesion detection performance without using any real data annotations. This study introduces an adaptable deep learning method for hepatic lesion identification in NETs, which can significantly reduce human effort for data annotation and improve model generalizability for lesion detection with PET imaging.

Supervised training models with or without manual lesion delineation outperform clinicians in distinguishing pulmonary cryptococcosis from lung adenocarcinoma on chest CT.

The role of artificial intelligence (AI) in the discrimination between pulmonary cryptococcosis (PC) and lung adenocarcinoma (LA) warrants further research. To compare the performances of AI models with clinicians in distinguishing PC from LA on chest CT. Patients diagnosed with confirmed PC or LA were retrospectively recruited from three tertiary hospitals in Guangzhou. A deep learning framework was employed to develop two models: an undelineated supervised training (UST) model utilising original CT images, and a delineated supervised training (DST) model utilising CT images with manual lesion annotations provided by physicians. A subset of 20 cases was randomly selected from the entire dataset and reviewed by clinicians through a network questionnaire. The sensitivity, specificity and accuracy of the models and the clinicians were calculated. A total of 395 PC cases and 249 LA cases were included in the final analysis. The internal validation results for the UST model showed a sensitivity of 85.3%, specificity of 81.0%, accuracy of 83.6% and an area under the curve (AUC) of 0.93. Similarly, the DST model exhibited a sensitivity of 88.2%, specificity of 88.1%, accuracy of 88.2% and an AUC of 0.94. The external validation of the two models yielded AUC values of 0.74 and 0.77, respectively. The average sensitivity, specificity and accuracy of 102 clinicians were determined to be 63.1%, 53.7% and 59.3%, respectively. Both models outperformed the clinicians in distinguishing between PC and LA on chest CT, with the UST model exhibiting comparable performance to the DST model.

A brain MRI dataset and baseline evaluations for tumor recurrence prediction after Gamma Knife radiotherapy

Prediction and identification of tumor recurrence are critical for brain cancer treatment design and planning. Stereotactic radiation therapy delivered with Gamma Knife has been developed as one of the common treatment approaches combined with others by delivering radiation that targets accurately on the tumor while not affecting nearby healthy tissues. In this paper, we release a fully publicly available brain cancer MRI dataset and the companion Gamma Knife treatment planning and follow-up data for the purpose of tumor recurrence prediction. The dataset contains original patient MRI images, radiation therapy data, and clinical information. Lesion annotations are provided, and inclusive preprocessing steps have been specified to simplify the usage of this dataset. A baseline framework based on a convolutional neural network is proposed companionably with basic evaluations. The release of this dataset will contribute to the future development of automated brain tumor recurrence prediction algorithms and promote the clinical implementations associated with the computer vision field. The dataset is made publicly available on The Cancer Imaging Archive (TCIA) (https://doi.org/10.7937/xb6d-py67).

Multi-center real-world data curation and assessment of tumor growth rate and overall survival in advanced NSCLC treated with PD-(L)1 immune checkpoint inhibitor therapy.

9124 Background: While Immune Checkpoint Inhibitors (ICI) have become the standard-of-care for advanced non-small cell lung cancer (NSCLC) for certain patient populations, durable responses only occur in a small subset of patients. Therefore, there is an unmet need to identify patients that will respond before and during treatment to optimize treatment strategies. Multiple studies have shown promise that quantitative imaging biomarkers can predict response to ICI therapy. Studies have also shown that tumor growth parameters derived from time series data can provide early indications of long-term response. However, most studies to date have investigated single institution or clinical trial datasets, which are inadequate to capture the broad spectrum of real-world diversity of ICI treatments. In this study, we aimed to build a generalizable, well-validated imaging biomarker for treatment response. Methods: We have curated a retrospective dataset including serial imaging data of 1,215 advanced NSCLC patients who received ICIs from nine institutions across the US and Europe, capturing the diversity of the imaging data, such as CT scanner models, image acquisition parameters, reconstruction algorithms, as well as diversity in patient population, performance status, and treatment settings. Expert lesion annotations and manual volumetric segmentations were performed on all visible lesions (ranging from 1-18 lesions per subject, totaling 6,441 lesions) across the serial imaging data (ranging from 1-18 follow-up scans per subject), which enables lesion and organ-specific tumor burden assessment. We stratified overall survival based on volumetric response and tumor growth rate using a model defined by the sum of exponential growth ( g) and decay ( d), vol(t) = vol(0)*(egt + e-dt -1). Results: Survival analysis showed that the 3-month volumetric response (N = 875) was significantly associated with overall survival (OS) for the lowest quartile (Q1; log-rank p-value, P = 2.1e-4) and highest quartile (Q4; P = 9.8e-8) compared to the middle quartiles (Q2+Q3). The median overall survival (OS) stratified by the volumetric response was 547.7, 450.9, and 332.0 days in the Q1, Q2+Q3, Q4 quadrants, respectively. Among patients having a minimum of 2 follow-up scans in the first 18 weeks (N = 264), we found that patients with the tumor growth parameter ( g) in the highest quartile had significantly lower OS compared to the patients in the Q2+Q3 quadrants (P = 7.2e-5). Conclusions: Our results demonstrate the potential of large multi-center real-world imaging datasets in investigating novel early response assessment methodologies, such as volumetric tumor burden quantification and tumor growth modeling. In our further work, we will investigate response patterns at individual lesion levels and correlations with overall patient-level response.

Multi-arm U-Net with dense input and skip connectivity for T2 lesion segmentation in clinical trials of multiple sclerosis

T2 lesion quantification plays a crucial role in monitoring disease progression and evaluating treatment response in multiple sclerosis (MS). We developed a 3D, multi-arm U-Net for T2 lesion segmentation, which was trained on a large, multicenter clinical trial dataset of relapsing MS. We investigated its generalization to other relapsing and primary progressive MS clinical trial datasets, and to an external dataset from the MICCAI 2016 MS lesion segmentation challenge. Additionally, we assessed the model’s ability to reproduce the separation of T2 lesion volumes between treatment and control arms; and the association of baseline T2 lesion volumes with clinical disability scores compared with manual lesion annotations. The trained model achieved a mean dice coefficient of ≥ 0.66 and a lesion detection sensitivity of ≥ 0.72 across the internal test datasets. On the external test dataset, the model achieved a mean dice coefficient of 0.62, which is comparable to 0.59 from the best model in the challenge, and a lesion detection sensitivity of 0.68. Lesion detection performance was reduced for smaller lesions (≤ 30 μL, 3–10 voxels). The model successfully maintained the separation of the longitudinal changes in T2 lesion volumes between the treatment and control arms. Such tools could facilitate semi-automated MS lesion quantification; and reduce rater burden in clinical trials.

The EMory BrEast imaging Dataset (EMBED): A Racially Diverse, Granular Dataset of 3.4 Million Screening and Diagnostic Mammographic Images.

Supplemental material is available for this article. Keywords: Mammography, Breast, Machine Learning © RSNA, 2023.

Attention-Driven Cascaded Network for Diabetic Retinopathy Grading from Fundus Images

Recently, academic and clinical communities have paid increasing attention to design computer-aided diagnosis methods for automatic and accurate diabetic retinopathy (DR) grading from fundus images. However, most existing methods either possess limited capability in extracting lesion-aware information or require manual lesion annotations, resulting in ordinary grading performance or additional undesirable labor. To address these issues, this paper proposes an end-to-end Attention-Driven Cascaded Network (ADCNet) for DR grading. Specifically, we first propose a hybrid attention module at the shallow layer by incorporating a multi-branch spatial attention and a loss-based attention to extract rich lesion-aware information without any manual lesion annotations. Then, we orderly cascade the lesion-aware information from shallow to high layers through an attention-driven aggregation strategy to obtain and integrate plentiful DR-related features. Finally, the grading score is generated by fusing DR-related features of all layers. Experimental results on two publicly available datasets demonstrate that the proposed ADCNet is competent for accurate DR grading, and outperforms the state-of-the-art methods on seven widely used evaluation criteria.

Lesion segmentation in lung CT scans using unsupervised adversarial learning

Lesion segmentation in medical images is difficult yet crucial for proper diagnosis and treatment. Identifying lesions in medical images is costly and time-consuming and requires highly specialized knowledge. For this reason, supervised and semi-supervised learning techniques have been developed. Nevertheless, the lack of annotated data, which is common in medical imaging, is an issue; in this context, interesting approaches can use unsupervised learning to accurately distinguish between healthy tissues and lesions, training the network without using the annotations. In this work, an unsupervised learning technique is proposed to automatically segment coronavirus disease 2019 (COVID-19) lesions on 2D axial CT lung slices. The proposed approach uses the technique of image translation to generate healthy lung images based on the infected lung image without the need for lesion annotations. Attention masks are used to improve the quality of the segmentation further. Experiments showed the capability of the proposed approaches to segment the lesions, and it outperforms a range of unsupervised lesion detection approaches. The average reported results for the test dataset based on the metrics: Dice Score, Sensitivity, Specificity, Structure Measure, Enhanced-Alignment Measure, and Mean Absolute Error are 0.695, 0.694, 0.961, 0.791, 0.875, and 0.082 respectively. The achieved results are promising compared with the state-of-the-art and could constitute a valuable tool for future developments.Graphical abstract

COVID-19-Associated Lung Lesion Detection by Annotating Medical Image with Semi Self-Supervised Technique

Diagnosing COVID-19 infection through the classification of chest images using machine learning techniques faces many controversial problems owing to the intrinsic nature of medical image data and classification architectures. The detection of lesions caused by COVID-19 in the human lung with properties such as location, size, and distribution is more practical and meaningful to medical workers for severity assessment, progress monitoring, and treatment, thus improving patients’ recovery. We proposed a COVID-19-associated lung lesion detector based on an object detection architecture. It correctly learns disease-relevant features by focusing on lung lesion annotation data of medical images. An annotated COVID-19 image dataset is currently nonexistent. We designed our semi-self-supervised method, which can extract knowledge from available annotated pneumonia image data and guide a novice in annotating lesions on COVID-19 images in the absence of a medical specialist. We prepared a sufficient dataset with nearly 8000 lung lesion annotations to train our deep learning model. We comprehensively evaluated our model on a test dataset with nearly 1500 annotations. The results demonstrated that the COVID-19 images annotated by our method significantly enhanced the model’s accuracy by as much as 1.68 times, and our model competes with commercialized solutions. Finally, all experimental data from multiple sources with different annotation data formats are standardized into a unified COCO format and publicly available to the research community to accelerate research on the detection of COVID-19 using deep learning.

Unsupervised Deep Learning for Stroke Lesion Segmentation on Follow-up CT Based on Generative Adversarial Networks.

Supervised deep learning is the state-of-the-art method for stroke lesion segmentation on NCCT. Supervised methods require manual lesion annotations for model development, while unsupervised deep learning methods such as generative adversarial networks do not. The aim of this study was to develop and evaluate a generative adversarial network to segment infarct and hemorrhagic stroke lesions on follow-up NCCT scans. Training data consisted of 820 patients with baseline and follow-up NCCT from 3 Dutch acute ischemic stroke trials. A generative adversarial network was optimized to transform a follow-up scan with a lesion to a generated baseline scan without a lesion by generating a difference map that was subtracted from the follow-up scan. The generated difference map was used to automatically extract lesion segmentations. Segmentation of primary hemorrhagic lesions, hemorrhagic transformation of ischemic stroke, and 24-hour and 1-week follow-up infarct lesions were evaluated relative to expert annotations with the Dice similarity coefficient, Bland-Altman analysis, and intraclass correlation coefficient. The median Dice similarity coefficient was 0.31 (interquartile range, 0.08-0.59) and 0.59 (interquartile range, 0.29-0.74) for the 24-hour and 1-week infarct lesions, respectively. A much lower Dice similarity coefficient was measured for hemorrhagic transformation (median, 0.02; interquartile range, 0-0.14) and primary hemorrhage lesions (median, 0.08; interquartile range, 0.01-0.35). Predicted lesion volume and the intraclass correlation coefficient were good for the 24-hour (bias, 3 mL; limits of agreement, -64-59 mL; intraclass correlation coefficient, 0.83; 95% CI, 0.78-0.88) and excellent for the 1-week (bias, -4 m; limits of agreement,-66-58 mL; intraclass correlation coefficient, 0.90; 95% CI, 0.83-0.93) follow-up infarct lesions. An unsupervised generative adversarial network can be used to obtain automated infarct lesion segmentations with a moderate Dice similarity coefficient and good volumetric correspondence.

Quantitative analysis of spatial distribution of lymphocytes in hepatocellular carcinoma: A biomarker correlated with survival and gene expression in cancer immune system.

4119 Background: Tumor microenvironment (TME) is known to impact prognosis in hepatocellular carcinoma (HCC). Although digital pathology and artificial intelligence have been adopted in modern medicine and oncology, few quantitative biomarkers have been identified to predict the prognosis and guide treatment for HCC via an automated analysis of TME at the cellular level. Methods: Histopathological images and clinical data of 365 cases with HCC were obtained from TCGA (The Cancer Genome Atlas), and 60 of HCC pathology images and cancer lesion annotations were collected from PAIP2019 [1]. DenseNet-based HCC segmentation model (F1-score, 0.904) and Hover-Net-based cell detection model (F1-score, 0.914) were developed using PAIP2019 and MoNuSac datasets, respectively [2,3,4]. Each histopathology image of TME was segmented via the segmentation model into two areas: 1) non-tumoral regions that include the stroma; 2) tumoral regions where HCC cells are concentrated. The cell detection model recognized individual cells on images, specified lymphocytes, and calculated ratios of lymphocyte to total cell count (RLTCC) in segmented regions. RLTCC was then correlated with clinical survival outcomes, HCC primary risk factors, and RNA expression profiles. Results: RLTCC in tumoral regions was not significantly associated with prognosis. Patient groups with higher RLTCC in non-tumoral regions (RLTCC in NT) showed better overall survival (OS) than those with lower RLTCC in NT regardless of HCC risk factors (median OS 45.7 vs 18.6 months; log hazard ratio of -1.6 ± 1.1, p=0.006). These patients had significantly enriched expression of genes (p<0.05) related to cancer antigen presentation (higher gene expression by +33.7%), recognition of cancer cells by T-cell (+32.0%), T-cell priming and activation (+32.2%), immune cell localization to tumors (+31.9%), and killing of cancer cells (+24.7%). Those with HCC etiology of hepatitis B and C had more patients in the higher RLTCC in NT (17/21 patients, 81.0%; 23/29, 79.3%, respectively). In comparison, those with alcohol consumption showed equal distribution (26/53, 49.1%). The RLTCC in NT in hepatitis B/C groups was statistically higher than alcohol consumption group (p<0.05). Conclusions: A digital prognostic biomarker, RLTCC in NT of TME was identified as a significant prognostic indicator, and it was shown to correlate with RNA gene expression related to T-cell mediated cancer immunity. A retrospective analysis of clinical response from systemic therapy in relation to digital biomarkers is underway and will be reported.

Automatic Segmentation of Novel Coronavirus Pneumonia Lesions in CT Images Utilizing Deep-Supervised Ensemble Learning Network.

Background: The novel coronavirus disease 2019 (COVID-19) has been spread widely in the world, causing a huge threat to the living environment of people.Objective: Under CT imaging, the structure features of COVID-19 lesions are complicated and varied greatly in different cases. To accurately locate COVID-19 lesions and assist doctors to make the best diagnosis and treatment plan, a deep-supervised ensemble learning network is presented for COVID-19 lesion segmentation in CT images.Methods: Since a large number of COVID-19 CT images and the corresponding lesion annotations are difficult to obtain, a transfer learning strategy is employed to make up for the shortcoming and alleviate the overfitting problem. Based on the reality that traditional single deep learning framework is difficult to extract complicated and varied COVID-19 lesion features effectively that may cause some lesions to be undetected. To overcome the problem, a deep-supervised ensemble learning network is presented to combine with local and global features for COVID-19 lesion segmentation.Results: The performance of the proposed method was validated in experiments with a publicly available dataset. Compared with manual annotations, the proposed method acquired a high intersection over union (IoU) of 0.7279 and a low Hausdorff distance (H) of 92.4604.Conclusion: A deep-supervised ensemble learning network was presented for coronavirus pneumonia lesion segmentation in CT images. The effectiveness of the proposed method was verified by visual inspection and quantitative evaluation. Experimental results indicated that the proposed method has a good performance in COVID-19 lesion segmentation.

Quantification of pulmonary involvement in COVID-19 pneumonia by means of a cascade of two U-nets: training and assessment on multiple datasets using different annotation criteria

PurposeThis study aims at exploiting artificial intelligence (AI) for the identification, segmentation and quantification of COVID-19 pulmonary lesions. The limited data availability and the annotation quality are relevant factors in training AI-methods. We investigated the effects of using multiple datasets, heterogeneously populated and annotated according to different criteria.MethodsWe developed an automated analysis pipeline, the LungQuant system, based on a cascade of two U-nets. The first one (U-net_1) is devoted to the identification of the lung parenchyma; the second one (U-net_2) acts on a bounding box enclosing the segmented lungs to identify the areas affected by COVID-19 lesions. Different public datasets were used to train the U-nets and to evaluate their segmentation performances, which have been quantified in terms of the Dice Similarity Coefficients. The accuracy in predicting the CT-Severity Score (CT-SS) of the LungQuant system has been also evaluated.ResultsBoth the volumetric DSC (vDSC) and the accuracy showed a dependency on the annotation quality of the released data samples. On an independent dataset (COVID-19-CT-Seg), both the vDSC and the surface DSC (sDSC) were measured between the masks predicted by LungQuant system and the reference ones. The vDSC (sDSC) values of 0.95±0.01 and 0.66±0.13 (0.95±0.02 and 0.76±0.18, with 5 mm tolerance) were obtained for the segmentation of lungs and COVID-19 lesions, respectively. The system achieved an accuracy of 90% in CT-SS identification on this benchmark dataset.ConclusionWe analysed the impact of using data samples with different annotation criteria in training an AI-based quantification system for pulmonary involvement in COVID-19 pneumonia. In terms of vDSC measures, the U-net segmentation strongly depends on the quality of the lesion annotations. Nevertheless, the CT-SS can be accurately predicted on independent test sets, demonstrating the satisfactory generalization ability of the LungQuant.