Ground Truth Annotations Research Articles

NnU-Net Deep Learning Method for Segmenting Parenchyma and Determining Liver Volume From Computed Tomography Images.

Background:Recipient donor matching in liver transplantation can require precise estimations of liver volume. Currently utilized demographic-based organ volume estimates are imprecise and nonspecific. Manual image organ annotation from medical imaging is effective; however, this process is cumbersome, often taking an undesirable length of time to complete. Additionally, manual organ segmentation and volume measurement incurs additional direct costs to payers for either a clinician or trained technician to complete. Deep learning-based image automatic segmentation tools are well positioned to address this clinical need.Objectives:To build a deep learning model that could accurately estimate liver volumes and create 3D organ renderings from computed tomography (CT) medical images.Methods:We trained a nnU-Net deep learning model to identify liver borders in images of the abdominal cavity. We used 151 publicly available CT scans. For each CT scan, a board-certified radiologist annotated the liver margins (ground truth annotations). We split our image dataset into training, validation, and test sets. We trained our nnU-Net model on these data to identify liver borders in 3D voxels and integrated these to reconstruct a total organ volume estimate.Results:The nnU-Net model accurately identified the border of the liver with a mean overlap accuracy of 97.5% compared with ground truth annotations. Our calculated volume estimates achieved a mean percent error of 1.92% + 1.54% on the test set.Conclusions:Precise volume estimation of livers from CT scans is accurate using a nnU-Net deep learning architecture. Appropriately deployed, a nnU-Net algorithm is accurate and quick, making it suitable for incorporation into the pretransplant clinical decision-making workflow.

A Data-Adaptive Loss Function for Incomplete Data and Incremental Learning in Semantic Image Segmentation.

In the last years, deep learning has dramatically improved the performances in a variety of medical image analysis applications. Among different types of deep learning models, convolutional neural networks have been among the most successful and they have been used in many applications in medical imaging. Training deep convolutional neural networks often requires large amounts of image data to generalize well to new unseen images. It is often time-consuming and expensive to collect large amounts of data in the medical image domain due to expensive imaging systems, and the need for experts to manually make ground truth annotations. A potential problem arises if new structures are added when a decision support system is already deployed and in use. Since the field of radiation therapy is constantly developing, the new structures would also have to be covered by the decision support system. In the present work, we propose a novel loss function to solve multiple problems: imbalanced datasets, partially-labeled data, and incremental learning. The proposed loss function adapts to the available data in order to utilize all available data, even when some have missing annotations. We demonstrate that the proposed loss function also works well in an incremental learning setting, where an existing model is easily adapted to semi-automatically incorporate delineations of new organs when they appear. Experiments on a large in-house dataset show that the proposed method performs on par with baseline models, while greatly reducing the training time and eliminating the hassle of maintaining multiple models in practice.

EfficientLPS: Efficient LiDAR Panoptic Segmentation

Panoptic segmentation of point clouds is a crucial task that enables autonomous vehicles to comprehend their vicinity using their highly accurate and reliable LiDAR sensors. Existing top–down approaches tackle this problem by either combining independent task-specific networks or translating methods from the image domain ignoring the intricacies of LiDAR data and thus often resulting in suboptimal performance. In this article, we present the novel top–down efficient LiDAR panoptic segmentation (EfficientLPS) architecture that addresses multiple challenges in segmenting LiDAR point clouds, including distance-dependent sparsity, severe occlusions, large scale-variations, and reprojection errors. EfficientLPS comprises of a novel shared backbone that encodes with strengthened geometric transformation modeling capacity and aggregates semantically rich range-aware multiscale features. It incorporates new scale-invariant semantic and instance segmentation heads along with the panoptic fusion module which is supervised by our proposed panoptic periphery loss function. Additionally, we formulate a regularized pseudolabeling framework to further improve the performance of EfficientLPS by training on unlabeled data. We benchmark our proposed model on two large-scale LiDAR datasets: nuScenes, for which we also provide ground truth annotations, and SemanticKITTI. Notably, EfficientLPS sets the new state-of-the-art on both these datasets.

The effect of machine learning explanations on user trust for automated diagnosis of COVID-19

Recent years have seen deep neural networks (DNN) gain widespread acceptance for a range of computer vision tasks that include medical imaging. Motivated by their performance, multiple studies have focused on designing deep convolutional neural network architectures tailored to detect COVID-19 cases from chest computerized tomography (CT) images. However, a fundamental challenge of DNN models is their inability to explain the reasoning for a diagnosis. Explainability is essential for medical diagnosis, where understanding the reason for a decision is as important as the decision itself. A variety of algorithms have been proposed that generate explanations and strive to enhance users' trust in DNN models. Yet, the influence of the generated machine learning explanations on clinicians' trust for complex decision tasks in healthcare has not been understood. This study evaluates the quality of explanations generated for a deep learning model that detects COVID-19 based on CT images and examines the influence of the quality of these explanations on clinicians’ trust. First, we collect radiologist-annotated explanations of the CT images for the diagnosis of COVID-19 to create the ground truth. We then compare ground truth explanations with machine learning explanations. Our evaluation shows that the explanations produced.by different algorithms were often correct (high precision) when compared to the radiologist annotated ground truth but a significant number of explanations were missed (significantly lower recall). We further conduct a controlled experiment to study the influence of machine learning explanations on clinicians' trust for the diagnosis of COVID-19. Our findings show that while the clinicians’ trust in automated diagnosis increases with the explanations, their reliance on the diagnosis reduces as clinicians are less likely to rely on algorithms that are not close to human judgement. Clinicians want higher recall of the explanations for a better understanding of an automated diagnosis system.

Abstract 349: Segmentation Of Atherosclerotic Plaque Features From Histopathology Images Using Novel Deep Learning Techniques

Objective: Atherosclerotic plaques have a complex composition, consisting of inflammation, fibrosis, cholesterol crystals, hemorrhage, and/or calcification. The segmentation and quantification of plaque features in histopathology images form the foundation for studies evaluating plaque instability and the mechanisms that underlie the atherosclerotic process. Manual segmentation of plaque features from histology images is a tedious, time-consuming, and subjective visual recognition task. Herein, we present a fully automatic approach using state-of-the-art deep learning techniques to identify three major features of the atherosclerotic plaque: calcification, lipid core, and fibrosis. Methods: Plaques (n=70) were collected from patients who underwent a carotid endarterectomy at McGill University-affiliated hospitals. Hematoxylin and Eosin-stained sections were obtained from the region with the largest plaque burden. The “ground truth annotations” for lipid core, calcification, and fibrosis were performed manually by three blinded cardiovascular pathologists, using Sedeen Viewer. A total of 23,000 patches with 512x512 pixel size were extracted from our image dataset, and divided into train, validate, and test sets. Using Transfer Learning, multi-class U-Net models for semantic segmentation were trained on the patches to extract fibrosis, lipid, and calcification plaque features. Evaluation of model performance was based on the mean value of the Intersection over Union (Mean-IOU) between the prediction results and the “ground truth annotations”. Results: Our models resulted in an overall performance of 77% for test images, and a per-class performance for the three plaque features: fibrosis = 0.77±0.2, lipid core = 0.80±0.3, calcification = 0.75±0.25. However, a qualitative evaluation by the pathologists confirmed that the prediction results in fact outperformed the “ground truth annotations”, and detected non-annotated regions. Conclusion: To our knowledge, this is a first attempt at developing a fully automatic approach for atherosclerotic plaque feature segmentation from histology images. Our models can accelerate atherosclerosis research, by improving the speed, quality, and reproducibility of plaque analysis.

Semantic Segmentation in Art Paintings

AbstractSemantic segmentation is a difficult task even when trained in a supervised manner on photographs. In this paper, we tackle the problem of semantic segmentation of artistic paintings, an even more challenging task because of a much larger diversity in colors, textures, and shapes and because there are no ground truth annotations available for segmentation. We propose an unsupervised method for semantic segmentation of paintings using domain adaptation. Our approach creates a training set of pseudo‐paintings in specific artistic styles by using style‐transfer on the PASCAL VOC 2012 dataset, and then applies domain confusion between PASCAL VOC 2012 and real paintings. These two steps build on a new dataset we gathered called DRAM (Diverse Realism in Art Movements) composed of figurative art paintings from four movements, which are highly diverse in pattern, color, and geometry. To segment new paintings, we present a composite multi‐domain adaptation method that trains on each sub‐domain separately and composes their solutions during inference time. Our method provides better segmentation results not only on the specific artistic movements of DRAM, but also on other, unseen ones. We compare our approach to alternative methods and show applications of semantic segmentation in art paintings. The code and models for our approach are publicly available at: https://github.com/Nadavc220/SemanticSegmentationInArtPaintings.

Endoscopy Artefact Detection by Deep Transfer Learning of Baseline Models.

To visualise the tumours inside the body on a screen, a long and thin tube is inserted with a light source and a camera at the tip to obtain video frames inside organs in endoscopy. However, multiple artefacts exist in these video frames that cause difficulty during the diagnosis of cancers. In this research, deep learning was applied to detect eight kinds of artefacts: specularity, bubbles, saturation, contrast, blood, instrument, blur, and imaging artefacts. Based on transfer learning with pre-trained parameters and fine-tuning, two state-of-the-art methods were applied for detection: faster region-based convolutional neural networks (Faster R-CNN) and EfficientDet. Experiments were implemented on the grand challenge dataset, Endoscopy Artefact Detection and Segmentation (EAD2020). To validate our approach in this study, we used phase I of 2,200 frames and phase II of 331 frames in the original training dataset with ground-truth annotations as training and testing dataset, respectively. Among the tested methods, EfficientDet-D2 achieves a score of 0.2008 (mAPd[Formula: see text]0.6+mIoUd[Formula: see text]0.4) on the dataset that is better than three other baselines: Faster-RCNN, YOLOv3, and RetinaNet, and competitive to the best non-baseline result scored 0.25123 on the leaderboard although our testing was on phase II of 331 frames instead of the original 200 testing frames. Without extra improvement techniques beyond basic neural networks such as test-time augmentation, we showed that a simple baseline could achieve state-of-the-art performance in detecting artefacts in endoscopy. In conclusion, we proposed the combination of EfficientDet-D2 with suitable data augmentation and pre-trained parameters during fine-tuning training to detect the artefacts in endoscopy.

Sensitivity examination of YOLOv4 regarding test image distortion and training dataset attribute for apple flower bud classification

ABSTRACT Applications of convolutional neural network (CNN)-based object detectors in agriculture have been a popular research topic in recent years. However, complicated agricultural environments bring many difficulties for ground truth annotation as well as potential uncertainties for image data quality. Using YOLOv4 as a representation of state-of-the-art object detectors, this study quantified YOLOv4’s sensitivity against artificial image distortions including white noise, motion blur, hue shift, saturation change, and intensity change, and examined the importance of various training dataset attributes based on model classification accuracies, including dataset size, label quality, negative sample presence, image sequence, and image distortion levels. The YOLOv4 model trained and validated on the original datasets failed at 31.91% white noise, 22.05-pixel motion blur, 77.38° hue clockwise shift, 64.81° hue counterclockwise shift, 89.98% saturation decrease, 895.35% saturation increase, 79.80% intensity decrease, and 162.71% intensity increase with 30% mean average precisions (mAPs) for four apple flower bud growth stages. The performance of YOLOv4 decreased with both declining training dataset size and training image label quality. Negative samples and training image sequence did not make a substantial difference in model performance. Incorporating distorted images during training improved the classification accuracies of YOLOv4 models on noisy test datasets by 13 to 390%. In the context of apple flower bud growth-stage classification, except for motion blur, YOLOv4 is sufficiently robust for potential image distortions by white noise, hue shift, saturation change, and intensity change in real life. Training image label quality and training instance number are more important factors than training dataset size. Exposing models to test-image-alike training images is crucial for optimal model classification accuracies. The study enhances understanding of implementing object detectors in agricultural research.

Panoptic Nuscenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking

Panoptic scene understanding and tracking of dynamic agents are essential for robots and automated vehicles to navigate in urban environments. As LiDARs provide accurate illumination-independent geometric depictions of the scene, performing these tasks using LiDAR point clouds provides reliable predictions. However, existing datasets lack diversity in the type of urban scenes and have a limited number of dynamic object instances which hinders both learning of these tasks as well as credible benchmarking of the developed methods. In this letter, we introduce the large-scale Panoptic nuScenes benchmark dataset that extends our popular nuScenes dataset with point-wise groundtruth annotations for semantic segmentation, panoptic segmentation, and panoptic tracking tasks. To facilitate comparison, we provide several strong baselines for each of these tasks on our proposed dataset. Moreover, we analyze the drawbacks of the existing metrics for panoptic tracking and propose the novel instance-centric PAT metric that addresses the concerns. We present exhaustive experiments that demonstrate the utility of Panoptic nuScenes compared to existing datasets and make the online evaluation server available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nuScenes.org</uri> . We believe that this extension will accelerate the research of novel methods for scene understanding of dynamic urban environments.

Automated Femoral Landmark Detection from Pelvic X-Rays

Surgical planning requires the identification of anatomical landmarks on medical images pre, intra, and post-procedures. Landmark detection on X-rays is thus an important, though time intensive, step to ensure good surgical outcomes. The operative workflow can be made more efficient by using machine learning approaches to automate the detection of landmarks. These models could also assist with the development of robotic surgical procedures, as robotic approaches to surgery require the robotic device to orient itself to the patient's anatomy. Previously described models are effective, but complex, making it difficult for landmark identification to occur in real-time in a robotic system. We propose a streamlined approach to landmark detection on pelvis radiographs that achieves similar accuracy to gold-standard manual annotations. To train and evaluate the network, 902 pelvic radiographs (382 Outlet view, 520 AP view) were annotated with 22 landmarks and split into training (n = 700), validation (n = 99), and testing (n = 103) set. A U-Net architecture with five encoding layers and five decoding layers was used. Landmark labels were converted into 128 × 128 × 22 heatmap array by placing a Gaussian blur filter (5% of image size) over each landmark in order to generate a single 128 × 128 layer of the array, thus generating the 22 layers of the overall heatmap array with one landmark heatmap on each layer. Each layer was an array of 0 except a circular Gaussian around the area of the landmark. A 256 × 256 array representing the image was fed into the U-Net architecture, which outputted a 128 × 128 × 22 heatmap array. Intersection over union loss was used to train the network. Following training for 100 epochs, the algorithm was able to predict the position of 22 clinically relevant landmarks with an average error of 2.36±.5 mm compared to ground-truth annotations made by trained experts and musculoskeletal radiologists with experience in pelvic scan analysis. The network was able to accurately determine the position of relevant landmarks. It was able to do this despite highly variable contrast and brightness in the dataset. Prediction accuracy was invariant to view (AP: 2.27±.51 mm, Outlet: 2.45±.64 mm). A prediction took 1.13±.07 seconds. The network is able to rapidly provide accurate femoral landmarks on a radiograph of the pelvis. This will allow for the automation of numerous clinically relevant measurements. Additionally, an important avenue of further research is exploring whether automated use of these measurements can aid in the prediction of femur fractures in machine learning models.

A framework for synthetic image generation and augmentation for improving automatic sewer pipe defect detection

Sewer pipes are essential infrastructure for discharging wastewater. Regular pipe inspection is necessary to prevent malfunction of sewer systems, for which closed-circuit television (CCTV) crawlers are commonly used to capture images of the pipe interior. As manual assessment of pipe condition is labor-intensive and time-consuming, automated defect detection using computer vision and deep learning has been increasingly studied in recent years. However, deep learning approaches require large amount of annotated data for model training. Data collection in underground sewer pipes is expensive and difficult since they are inaccessible without the use of an inspection robot. Meanwhile, ground-truth annotation needs to be accurate and consistent, requiring massive time and expertise. This paper proposes a framework for synthetic data generation and augmentation to address the data shortage problem for sewer pipe defect detection. First, synthetic images of sewer pipes are generated by 3D modeling and simulation in virtual environment. The quality of the generated images is then enhanced using style transfer with reference to real inspection images. In addition, a contrastive learning module is developed to further improve the deep learning process for defect detection. Experiment results show that the average precision (AP) of the defect detection model is improved by 2.7% and 4.8% respectively after adding style-transferred synthetic images and applying the contrastive module. When both methods are applied, the AP of the model is boosted by 7.7%, from 22.22% to 23.92%, indicating the effectiveness of our proposed approaches. This study is expected to alleviate the burden on data collection and annotation for applying deep learning models in defect detection.

Deep Learning and Domain-Specific Knowledge to Segment the Liver from Synthetic Dual Energy CT Iodine Scans.

We map single energy CT (SECT) scans to synthetic dual-energy CT (synth-DECT) material density iodine (MDI) scans using deep learning (DL) and demonstrate their value for liver segmentation. A 2D pix2pix (P2P) network was trained on 100 abdominal DECT scans to infer synth-DECT MDI scans from SECT scans. The source and target domain were paired with DECT monochromatic 70 keV and MDI scans. The trained P2P algorithm then transformed 140 public SECT scans to synth-DECT scans. We split 131 scans into 60% train, 20% tune, and 20% held-out test to train four existing liver segmentation frameworks. The remaining nine low-dose SECT scans tested system generalization. Segmentation accuracy was measured with the dice coefficient (DSC). The DSC per slice was computed to identify sources of error. With synth-DECT (and SECT) scans, an average DSC score of () and () was achieved on the held-out and generalization test sets. Synth-DECT-trained systems required less data to perform as well as SECT-trained systems. Low DSC scores were primarily observed around the scan margin or due to non-liver tissue or distortions within ground-truth annotations. In general, training with synth-DECT scans resulted in improved segmentation performance with less data.

A multi-spectral myelin annotation tool for machine learning based myelin quantification

Myelin is an essential component of the nervous system and myelin damage causes demyelination diseases. Myelin is a sheet of oligodendrocyte membrane wrapped around the neuronal axon. In the fluorescent images, experts manually identify myelin by co-localization of oligodendrocyte and axonal membranes that fit certain shape and size criteria. Because myelin wriggles along x-y-z axes, machine learning is ideal for its segmentation. However, machine-learning methods, especially convolutional neural networks (CNNs), require a high number of annotated images, which necessitate expert labor. To facilitate myelin annotation, we developed a workflow and software for myelin ground truth extraction from multi-spectral fluorescent images. Additionally, to the best of our knowledge, for the first time, a set of annotated myelin ground truths for machine learning applications were shared with the community.

AI‐driven user aesthetics preference prediction for UI layouts via deep convolutional neural networks

Leveraging the power of computational methods, AI can perform effective strategies in intelligent design. Researchers are pushing the boundaries of AI, developing computational systems to solve complex questions. The authors investigate the association of user preference for UI and deep image features, aiming to predict user preference level using deep convolutional neural networks (DCNNs) trained on a UI design image dataset. A total of 12,186 UI design images were collected from UI.cn and DOOOOR.com. Users' views and likes can help understand the implicit user preference level, which is set as the ground-truth annotation for the dataset. Six DCNNs, including VGG-19, InceptionNet-V3, MobileNet, EfficientNet, ResNet-50 and NASNetLarge were trained to learn the user preference of UI images. The experiment achieves an optimal result with a mean-squared error of 0.000214 and a mean absolute error of 0.0103 based on EfficientNet, which indicates that the proposed method provides the possibility in learning the pattern of user aesthetics preference for UI design. On the basis of the prediction model, a mobile application named ‘HotUI’ was developed for UI design recommendations.

Segmentation of Glottal Images from High-Speed Videoendoscopy Optimized by Synchronous Acoustic Recordings.

Laryngeal high-speed videoendoscopy (LHSV) is an imaging technique offering novel visualization quality of the vibratory activity of the vocal folds. However, in most image analysis methods, the interaction of the medical personnel and access to ground truth annotations are required to achieve accurate detection of vocal folds edges. In our fully automatic method, we combine video and acoustic data that are synchronously recorded during the laryngeal endoscopy. We show that the image segmentation algorithm of the glottal area can be optimized by matching the Fourier spectra of the pre-processed video and the spectra of the acoustic recording during the phonation of sustained vowel /i:/. We verify our method on a set of LHSV recordings taken from subjects with normophonic voice and patients with voice disorders due to glottal insufficiency. We show that the computed geometric indices of the glottal area make it possible to discriminate between normal and pathologic voices. The median of the Open Quotient and Minimal Relative Glottal Area values for healthy subjects were 0.69 and 0.06, respectively, while for dysphonic subjects were 1 and 0.35, respectively. We also validate these results using independent phoniatrician experts.

Fully automated quantification of cardiac chamber and function assessment in 2-D echocardiography: clinical feasibility of deep learning-based algorithms

We aimed to compare the segmentation performance of the current prominent deep learning (DL) algorithms with ground-truth segmentations and to validate the reproducibility of the manually created 2D echocardiographic four cardiac chamber ground-truth annotation. Recently emerged DL based fully-automated chamber segmentation and function assessment methods have shown great potential for future application in aiding image acquisition, quantification, and suggestion for diagnosis. However, the performance of current DL algorithms have not previously been compared with each other. In addition, the reproducibility of ground-truth annotations which are the basis of these algorithms have not yet been fully validated. We retrospectively enrolled 500 consecutive patients who underwent transthoracic echocardiogram (TTE) from December 2019 to December 2020. Simple U-net, Res-U-net, and Dense-U-net algorithms were compared for the segmentation performances and clinical indices such as left atrial volume (LAV), left ventricular end diastolic volume (LVEDV), left ventricular end systolic volume (LVESV), LV mass, and ejection fraction (EF) were evaluated. The inter- and intra-observer variability analysis was performed by two expert sonographers for a randomly selected echocardiographic view in 100 patients (apical 2-chamber, apical 4-chamber, and parasternal short axis views). The overall performance of all DL methods was excellent [average dice similarity coefficient (DSC) 0.91 to 0.95 and average Intersection over union (IOU) 0.83 to 0.90], with the exception of LV wall area on PSAX view (average DSC of 0.83, IOU 0.72). In addition, there were no significant difference in clinical indices between ground truth and automated DL measurements. For inter- and intra-observer variability analysis, the overall intra observer reproducibility was excellent: LAV (ICC = 0.995), LVEDV (ICC = 0.996), LVESV (ICC = 0.997), LV mass (ICC = 0.991) and EF (ICC = 0.984). The inter-observer reproducibility was slightly lower as compared to intraobserver agreement: LAV (ICC = 0.976), LVEDV (ICC = 0.982), LVESV (ICC = 0.970), LV mass (ICC = 0.971), and EF (ICC = 0.899). The three current prominent DL-based fully automated methods are able to reliably perform four-chamber segmentation and quantification of clinical indices. Furthermore, we were able to validate the four cardiac chamber ground-truth annotation and demonstrate an overall excellent reproducibility, but still with some degree of inter-observer variability.

FSOCO: The Formula Student Objects in Context Dataset

This paper presents the FSOCO dataset, a collaborative dataset for vision-based cone detection systems in Formula Student Driverless competitions. It contains human annotated ground truth labels for both bounding boxes and instance-wise segmentation masks. The data buy-in philosophy of FSOCO asks student teams to contribute to the database first before being granted access ensuring continuous growth. By providing clear labeling guidelines and tools for a sophisticated raw image selection, new annotations are guaranteed to meet the desired quality. The effectiveness of the approach is shown by comparing prediction results of a network trained on FSOCO and its unregulated predecessor. The FSOCO dataset can be found at fsoco-dataset.com.

Multicenter Study on COVID-19 Lung Computed Tomography Segmentation with varying Glass Ground Opacities using Unseen Deep Learning Artificial Intelligence Paradigms: COVLIAS 1.0 Validation.

Variations in COVID-19 lesions such as glass ground opacities (GGO), consolidations, and crazy paving can compromise the ability of solo-deep learning (SDL) or hybrid-deep learning (HDL) artificial intelligence (AI) models in predicting automated COVID-19 lung segmentation in Computed Tomography (CT) from unseen data leading to poor clinical manifestations. As the first study of its kind, “COVLIAS 1.0-Unseen” proves two hypotheses, (i) contrast adjustment is vital for AI, and (ii) HDL is superior to SDL. In a multicenter study, 10,000 CT slices were collected from 72 Italian (ITA) patients with low-GGO, and 80 Croatian (CRO) patients with high-GGO. Hounsfield Units (HU) were automatically adjusted to train the AI models and predict from test data, leading to four combinations—two Unseen sets: (i) train-CRO:test-ITA, (ii) train-ITA:test-CRO, and two Seen sets: (iii) train-CRO:test-CRO, (iv) train-ITA:test-ITA. COVILAS used three SDL models: PSPNet, SegNet, UNet and six HDL models: VGG-PSPNet, VGG-SegNet, VGG-UNet, ResNet-PSPNet, ResNet-SegNet, and ResNet-UNet. Two trained, blinded senior radiologists conducted ground truth annotations. Five types of performance metrics were used to validate COVLIAS 1.0-Unseen which was further benchmarked against MedSeg, an open-source web-based system. After HU adjustment for DS and JI, HDL (Unseen AI) > SDL (Unseen AI) by 4% and 5%, respectively. For CC, HDL (Unseen AI) > SDL (Unseen AI) by 6%. The COVLIAS-MedSeg difference was < 5%, meeting regulatory guidelines.Unseen AI was successfully demonstrated using automated HU adjustment. HDL was found to be superior to SDL. Supplementary InformationThe online version contains supplementary material available at 10.1007/s10916-022-01850-y.

Structure-Aware Weakly Supervised Network for Building Extraction From Remote Sensing Images

The use of fully supervised deep learning methods to extract buildings from remote sensing images has shown excellent performance, which requires large amounts of training data with laborious per-pixel labeling. Compared with pixelated intensive labeling, it is much easier to label data using scribbles, which only takes few seconds for one image. In this paper, a one-stage structure-aware weakly supervised network (SAWSN) for building extraction is proposed, and it learns from easily accessible scribbles rather than from densely annotated ground truth. Firstly, to solve the problem that direct training with scribble labels will lead to poor building structures, an auxiliary edge detection task is introduced to localize building edges explicitly. Secondly, a structure aware scribble extension module (SASEM) is designed to recover building structures from scribbles through effective utilization of edge features. Finally, an edge-structure-aware loss is proposed to limit the scope of the restored structure. We perform extensive experiments on three newly labeled benchmark building extraction datasets (WHU, ISPRS Potsdam, and Vaihingen). Experimental results show that our method achieved 91.72%, 92.83%, and 92.22% of F1 using the ISPRS Vaihingen, Potsdam, and WHU datasets, respectively, and outperformed the state-of-the-art scribble-based weakly supervised methods by 3.27% of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IoU</i> .

Adaptive Segmentation for Discontinuity Detection on Karstified Carbonate Outcrop Images From UAV-SfM Acquisition and Detection Bias Analysis

The identification of fractures and discontinuities has great importance on the fluid flow estimation in hydrocarbon reservoirs since they influence the properties of porosity and permeability. Due to the inaccessibility and sparsity of reservoir data, the fracture characterization is generally assessed through the study of outcrop analogues using remote sensing or in <i>situ</i> observations by a specialist. Considering the remote sensing methods, the unmanned Aerial Vehicle (UAV) acquisition combined with Structure from Motion (SfM) photogrammetry is a low-cost way to generate products like orthorectified images, allowing manual and automated methods of fracture trace detection. Automatic approaches, commonly used to address this problem, present some known limitations and disadvantages due to the nature of the outcrops and weather conditions during UAV acquisitions. In this work, we focus on fracture detection over karstic regions that are highly fractured. For this, we evaluated a series of adaptive segmentation methods based on thresholding. The Sauvola local adaptive segmentation presented the best result when compared to a manually annotated ground truth. The segmentation results were further improved by the use of the binary denoising method Non-Local means. We also carried an evaluation of the influence of the sun position in the fracture detection, and to reduce this inherent bias we combined three UAV acquisitions done over the karstic carbonate outcrop, namely Ros&#x00E1;rio pavement in the Janda&#x00ED;ra formation northeast Brazil. With the proposed methodology we acquired more accurate fracture data over the study area, which follows the directional statistics of previous works carried out in the region.