Ground Truth Annotations Research Articles

Abstract P6-04-16: ART: Automated Region segmentation of Tumor on HER2-stained breast cancer tissue

Abstract Background Computational pathology-based methods, eg, Quantitative Continuous Scoring (QCS) [Gustavson, SABCS 2020], are built to provide objective and quantitative methods to assess HER2 expression in breast cancer (BC). For accurate HER2 quantification, it is important to exclude non-invasive epithelium from analysis since HER2 overexpression could be more frequent in ductal carcinoma in situ (DCIS) and pleomorphic lobular carcinoma in situ (PLCIS) than in invasive BC [Lari, J Cancer 2011]. Generally, computational pathology-based approaches require experts to delineate the invasive BC regions of interest for analysis and exclude all benign/non-invasive epithelium. Developing a tool that delineates the invasive BC regions automatically without human intervention to avoid subjectivity of manual annotation by pathologists is ideal. We developed a novel, deep-learning–based system, called DualScaleNet, to perform Automated Region segmentation of Tumor (ART) by automatically identifying the invasive BC regions and excluding benign/non-invasive epithelium on HER2-stained digitized images. Identification and diagnosis of these regions, especially the in situ tumors are a challenge as they can mimic benign and invasive lesions causing wrong HER2 evaluation. Additional stains (eg, p63 for myoepithelial cells or laminin for basement membrane) are often required for diagnosis [Pinder, Mod Pathol 2010] but were not available for this study. Methods DualScaleNet works simultaneously on HER2-stained immunohistochemistry (IHC) image patches at 2 different resolutions. The target branch uses a higher resolution RGB image (0.5 μm/pixel) to learn accurate local details; the context branch uses a lower resolution image (4.0 μm/pixel) to incorporate more context in visual learning. The algorithm generates 4 output image layers representing probabilities of 4 classes: invasive tumor, ductal/lobular carcinoma in situ, benign epithelium, and other tissue. The final segmentation result is generated by assigning each image pixel the class with the largest probability value. The algorithm was trained using ground truth (GT) annotations generated by 5 pathologists using 6157 square field of views (FOVs), 200-500 μm in size. These FOVs were collected from 850 whole slide images (WSI), spanning 9 commercial BC sample cohorts stained with different HER2 assays and scanned by several versions of the Aperio AT2 scanner. The samples included a mixture of biopsies and resections and covered different BC histologies and HER2 staining intensities. To evaluate the reproducibility of tumor area detection by human pathologists, an interpathologist comparison in detection of invasive tumor regions was performed using 225 FOVs annotated by multiple pathologists. Results Analysis generated an average Dice/F1 Score of 81.6% among different pathologists for invasive cancer. On the same sample set (independent of the ART training set), the invasive cancer detection by the ART algorithm was on par with human pathologists, achieving a similar average Dice/F1 score of 80.7%. Conclusions Novel deep learning-based ART algorithm provides accurate segmentation of invasive cancer on HER2-stained IHC images. The performance was verified against the GT annotations provided by multiple pathologists. Since the algorithm is trained using annotations from multiple pathologists, it is not possible to generate higher accuracy with computational pathology than is achievable between independent pathologists. Importantly, the same WSI read by the ART will consistently output the exact same tumor region identification result thus removing the inherent human subjectivity and variability, while improving the turnaround time for analysis. This development serves as the necessary foundation upon which a computational pathology-based diagnostic can be built. Citation Format: Ansh Kapil, Anatoliy Shumilov, Philipp Wortmann, Sihem Khelifa, Jessica Chan, Michel Vandenberghe, Craig Barker, Mark Gustavson, Danielle Carroll, Hadassah Sade, Günter Schmidt. ART: Automated Region segmentation of Tumor on HER2-stained breast cancer tissue [abstract]. In: Proceedings of the 2022 San Antonio Breast Cancer Symposium; 2022 Dec 6-10; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2023;83(5 Suppl):Abstract nr P6-04-16.

Choroidal vascularity index (CVI)-Net-based automatic assessment of diabetic retinopathy severity using CVI in optical coherence tomography images.

A deep learning model called choroidal vascularity index (CVI)-Net is proposed to automatically segment the choroid layer and its vessels in overall optical coherence tomography (OCT) scans. Clinical parameters are then automatically quantified to determine structural and vascular changes in the choroid with the progression of diabetic retinopathy (DR) severity. The study includes 65 eyes consisting of 34 with proliferative DR (PDR), 17 with nonproliferative DR (NPDR), and 14 healthy controls from two OCT systems. On a dataset of 396 OCT B-scan images with manually annotated ground truths, overall Dice coefficients of 96.6 ± 1.5 and 89.1 ± 3.1 are obtained by CVI-Net for the choroid layer and vessel segmentation, respectively. The mean CVI values among the normal, NPDR, and PDR groups are consistent with reported outcomes. Statistical results indicate that CVI shows a significant negative correlation with DR severity level, and this correlation is independent of changes in other physiological parameters.

LiDAR-based simultaneous multi-object tracking and static mapping in nearshore scenario

Reliable object tracking is a key ingredient for maritime autonomous surface ship (MASS), which can perceive surrounding objects and predict their motion state. Existing tracking algorithms mainly rely on visual sensor, and their application background is predominantly concentrated in the field of unmanned vehicles. In this paper, a scheme of directly processing LiDAR point cloud is proposed to achieve accurate and stable multi-object tracking in nearshore scenario. Firstly, based on the point cloud clustering results, an object position and volume estimation method is investigated. Then, the geometric distribution and 3D shape descriptors are conceived to construct the time-varying affinity matrix. The data association results are forwarded into a Kalman Filter based tracker life management mechanism. Finally, the remaining scattered points and unqualified trackers are composed into a point set, which is then modeled by convex polygons. The real-world experiment results confirm the effectiveness of the proposed maritime perception framework in tracking objects and mapping the ambience in a bustling lake. Compared with annotated ground-truth, the average multi-object tracking precision and accuracy is 0.69 m and 85%, respectively. And the average computational frequency achieves 29Hz. The proposed scheme successfully applies LiDAR to the field of sea surface object tracking, which provides a practical and meaningful reference for the application of LiDAR in MASS.

Design of an automotive platform for computer vision research

The goal of our work is to design an automotive platform for AD/ADAS data acquisition and to demonstrate its application to behavior analysis of vulnerable road users. We present a novel data capture platform mounted on a Mercedes GLC vehicle. The car is equipped with an array of sensors and recording hardware including multiple RGB cameras, Lidar, GPS and IMU. For subsequent research on human behavior analysis in traffic scenes, we have conducted two kinds of data recordings. Firstly, we have designed a range of artificial test cases which we recorded on a safety regulated proving ground with stunt persons to capture rare events in traffic scenes in a predictable and structured way. Secondly, we have recorded data on public streets of Vienna, Austria, showing unconstrained pedestrian behavior in an urban setting, while also considering European General Data Protection Regulation (GDPR) requirements. We describe the overall framework including data acquisition and ground truth annotation, and demonstrate its applicability for the implementation and evaluation of selected deep learning models for pedestrian behavior prediction.

A Surface Defect Inspection Model via Rich Feature Extraction and Residual-Based Progressive Integration CNN

Surface defect inspection is vital for the quality control of products and the fault diagnosis of equipment. Defect inspection remains challenging due to the low level of automation in some manufacturing plants and the difficulty in identifying defects. To improve the automation and intelligence levels of defect inspection, a CNN model is proposed for the high-precision defect inspection of USB components in the actual demands of factories. First, the defect inspection system was built, and a dataset named USB-SG, which contained five types of defects—dents, scratches, spots, stains, and normal—was established. The pixel-level defect ground-truth annotations were manually marked. This paper puts forward a CNN model for solving the problem of defect inspection tasks, and three strategies are proposed to improve the model’s performance. The proposed model is built based on the lightweight SqueezeNet network, and a rich feature extraction block is designed to capture semantic and detailed information. Residual-based progressive feature integration is proposed to fuse the extracted features, which can reduce the difficulty of model fine-tuning and improve the generalization ability. Finally, a multi-step deep supervision scheme is proposed to supervise the feature integration process. The experiments on the USB-SG dataset prove that the model proposed in this paper has better performance than that of other methods, and the running speed can meet the real-time demand, which has broad application prospects in the industrial inspection scene.

RGBT Salient Object Detection: A Large-Scale Dataset and Benchmark

Salient object detection in complex scenes and environments is a challenging research topic. Most works focus on RGB-based salient object detection, which limits its performance of real-life applications when confronted with adverse conditions such as dark environments and complex backgrounds. Taking advantage of RGB and thermal infrared(RGBT) images becomes a new research direction for detecting salient objects in complex scenes, since the thermal infrared spectrum provides the complementary information and has been used in many computer vision tasks. However, current research for RGBT salient object detection is limited by the lack of a large-scale dataset and comprehensive benchmark. This work contributes such a RGBT image dataset named VT5000, including 5000 spatially aligned RGBT image pairs with ground truth annotations. VT5000 has 11 challenges collected in different scenes and environments for exploring the robustness of algorithms. With this dataset, we propose a powerful baseline approach, which extracts multilevel features of each modality and aggregates these features of all modalities with the attention mechanism for accurate RGBT salient object detection. To further solve the problem of blur boundaries of salient objects, we also use an edge loss to refine the boundaries. Extensive experiments show that the proposed baseline approach outperforms the state-of-the-art methods on VT5000 dataset and other two public datasets. In addition, we carry out a comprehensive analysis of different algorithms of RGBT salient object detection on VT5000 dataset, and then make several valuable conclusions and provide some potential research directions for RGBT salient object detection. Our new VT5000 dataset is made publicly available at https://github.com/lz118/RGBT-Salient-Object-Detection.

Quantifying the Effects of Ground Truth Annotation Quality on Object Detection and Instance Segmentation Performance

Fully-supervised object detection and instance segmentation models have accomplished notable results on large-scale computer vision benchmark datasets. However, fully-supervised machine learning algorithms’ performances are immensely dependent on the quality of the training data. Preparing computer vision datasets for object detection and instance segmentation is a labor-intensive task requiring each instance in an image to be annotated. In practice, this often results in the quality of bounding box and polygon mask annotations being suboptimal. This paper quantifies empirically the ground truth annotation quality and COCO’s mean average precision (mAP) performance by introducing two separate noise measures, uniform and radial, into the ground truth bounding box and polygon mask annotations for the COCO and Cityscapes datasets. Mask-RCNN models are trained on various levels of noise measures to investigate the performance of each level of noise. The results showed degradation of mAP as the level of both noise measures increased. For object detection and instance segmentation respectively, using the highest level of noise measure resulted in a mAP degradation of 0.185 & 0.208 for uniform noise with reductions of 0.118 & 0.064 for radial noise on the COCO dataset. As for the Cityscapes datasets, reductions of mAP performance of 0.147 & 0.142 for uniform noise and 0.101 & 0.033 for radial noise were recorded. Furthermore, a decrease in average precision is seen across all classes, with the exception of the class motorcycle. The reductions between classes vary, indicating the effects of annotation uncertainty are class-dependent.

PREVENTING POTENTIAL ROBBERY CRIMES USING DEEP LEARNING ALGORITHM OF DATA PROCESSING

Recently, deep learning technologies, namely Neural Networks [1], are attracting more and more attention from businesses and the scientific community, as they help optimize processes and find real solutions to problems much more efficiently and economically than many other approaches. In particular, Neural Networks are well suited for situations when you need to detect objects or look for similar patterns in videos and images, making them relevant in the field of information and measurement technologies in mechatronics and robotics. With the increasing number of robbed apartments and houses every year, addressing this issue has become one of the highest priorities in today's society. By leveraging deep learning techniques, such as Neural Networks, in mechatronics and robotics, innovative solutions can be developed to enhance security systems, enabling more effective detection and prevention of apartment crimes. To evaluate the performance of our trained network, we conducted extensive experiments on a separate test dataset that was distinct from the training data. We meticulously labeled this dataset to obtain accurate ground truth annotations for comparison. By measuring precision scores, we determined the effectiveness of our model in detecting potential crimes. Our experiments yielded an accuracy rate of 97% in the detection of potential crimes. This achievement demonstrates the capability of YOLO and the effectiveness of our trained network in accurately identifying criminal activities. The high accuracy rate indicates that our system can effectively assist in property protection efforts, providing a valuable tool for security personnel and law enforcement agencies.

Fast Ocean Front Detection Using Deep Learning Edge Detection Models

Small-scale ocean fronts play a significant role in absorbing the excess heat and CO2 generated by climate change, yet their dynamics are not well understood. Existing in-situ and remote sensing measurements of the ocean have inadequate spatial and temporal coverage to map small-scale ocean fronts globally. Additionally, conventional algorithms to generate ocean front maps are computationally intensive and require data with long lead times. We propose machine learning (ML) models to detect temperature and chlorophyll ocean fronts from unprocessed and radiometrically uncorrected satellite imagery by transfer learning from existing models for edge detection. We use two separate datasets: one based on conventional approaches to ocean front detection, and a second based on human annotated ground truth <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> . The deep learning front detection approach significantly reduces the resources and overall lead times needed for detecting ocean fronts. The deep learning models are developed with resource-constrained edge compute platforms like CubeSats in mind, as such platforms can address the spatial and temporal coverage challenges. The highest performing models achieve accuracies of 96% and make predictions in milliseconds using unoptimized desktop CPUs and less than 100 MB of storage; these capabilities are well-suited for CubeSat deployment.

A hand motion capture method based on infrared thermography for measuring fine motor skills in biomedicine

Many biomedical applications require fine motor skill assessments; however, real-time and contactless fine motor skill assessments are not typically implemented. In this study, we followed the 2D-to-3D pipeline principle and proposed a transformer-based spatial–temporal network to accurately regress 3D hand joint locations by inputting infrared thermal video for eliminating need of multiple cameras or RGB-D devices. We also developed a dataset composed of infrared thermal videos and ground truth annotations for training. The label represents a set of 3D joint locations from infrared optical trackers, which is considered the gold standard for clinical applications. To demonstrate their potential, the proposed method was used to measure the finger motion angle, and we investigated its accuracy by comparing the proposal with the Azure Kinect system and Leap Motion system. On the proposed dataset, the proposed method achieved a 3D hand pose mean error of less than 14 mm and outperforms the other deep learning methods. When the error thresholds were larger than approximately 35 mm, our method first to achieved excellent performance (>80%) in terms of the fraction of good frames. For the finger motion angle calculation task, the proposed and commercial systems had comparable inter-system reliability (ICC2,1 ranging from 0.81 to 0.83) and excellent validity (Pearson’s r-values ranging from 0.82 to 0.86). We believe that the proposed approaches can capture hand motion and measure finger motion angles and can be used in different biomedicine scenarios as an effective evaluation tool for fine motor skills.

Project to Adapt: Domain Adaptation for Depth Completion from Noisy and Sparse Sensor Data

Depth completion aims to predict a dense depth map from a sparse depth input. The acquisition of dense ground-truth annotations for depth completion settings can be difficult and, at the same time, a significant domain gap between real LiDAR measurements and synthetic data has prevented from successful training of models in virtual settings. We propose a domain adaptation approach for sparse-to-dense depth completion that is trained from synthetic data, without annotations in the real domain or additional sensors. Our approach simulates the real sensor noise in an RGB + LiDAR set-up, and consists of three modules: simulating the real LiDAR input in the synthetic domain via projections, filtering the real noisy LiDAR for supervision and adapting the synthetic RGB image using a CycleGAN approach. We extensively evaluate these modules in the KITTI depth completion benchmark.

LapTrack: linear assignment particle tracking with tunable metrics.

Particle tracking is an important step of analysis in a variety of scientific fields and is particularly indispensable for the construction of cellular lineages from live images. Although various supervised machine learning methods have been developed for cell tracking, the diversity of the data still necessitates heuristic methods that require parameter estimations from small amounts of data. For this, solving tracking as a linear assignment problem (LAP) has been widely applied and demonstrated to be efficient. However, there has been no implementation that allows custom connection costs, parallel parameter tuning with ground truth annotations, and the functionality to preserve ground truth connections, limiting the application to datasets with partial annotations. We developed LapTrack, a LAP-based tracker which allows including arbitrary cost functions and inputs, parallel parameter tuning and ground-truth track preservation. Analysis of real and artificial datasets demonstrates the advantage of custom metric functions for tracking score improvement from distance-only cases. The tracker can be easily combined with other Python-based tools for particle detection, segmentation and visualization. LapTrack is available as a Python package on PyPi, and the notebook examples are shared at https://github.com/yfukai/laptrack. The data and code for this publication are hosted at https://github.com/NoneqPhysLivingMatterLab/laptrack-optimisation. Supplementary data are available at Bioinformatics online.

Freddie: annotation-independent detection and discovery of transcriptomic alternative splicing isoforms using long-read sequencing.

Alternative splicing (AS) is an important mechanism in the development of many cancers, as novel or aberrant AS patterns play an important role as an independent onco-driver. In addition, cancer-specific AS is potentially an effective target of personalized cancer therapeutics. However, detecting AS events remains a challenging task, especially if these AS events are novel. This is exacerbated by the fact that existing transcriptome annotation databases are far from being comprehensive, especially with regard to cancer-specific AS. Additionally, traditional sequencing technologies are severely limited by the short length of the generated reads, which rarely spans more than a single splice junction site. Given these challenges, transcriptomic long-read (LR) sequencing presents a promising potential for the detection and discovery of AS. We present Freddie, a computational annotation-independent isoform discovery and detection tool. Freddie takes as input transcriptomic LR sequencing of a sample alongside its genomic split alignment and computes a set of isoforms for the given sample. It then partitions the input reads into sets that can be processed independently and in parallel. For each partition, Freddie segments the genomic alignment of the reads into canonical exon segments. The goal of this segmentation is to be able to represent any potential isoform as a subset of these canonical exons. This segmentation is formulated as an optimization problem and is solved with a dynamic programming algorithm. Then, Freddie reconstructs the isoforms by jointly clustering and error-correcting the reads using the canonical segmentation as a succinct representation. The clustering and error-correcting step is formulated as an optimization problem-the Minimum Error Clustering into Isoforms (MErCi) problem-and is solved using integer linear programming (ILP). We compare the performance of Freddie on simulated datasets with other isoform detection tools with varying dependence on annotation databases. We show that Freddie outperforms the other tools in its accuracy, including those given the complete ground truth annotation. We also run Freddie on a transcriptomic LR dataset generated in-house from a prostate cancer cell line with a matched short-read RNA-seq dataset. Freddie results in isoforms with a higher short-read cross-validation rate than the other tested tools. Freddie is open source and available at https://github.com/vpc-ccg/freddie/.

Deep neural network automated segmentation of cellular structures in volume electron microscopy.

Volume electron microscopy is an important imaging modality in contemporary cell biology. Identification of intracellular structures is a laborious process limiting the effective use of this potentially powerful tool. We resolved this bottleneck with automated segmentation of intracellular substructures in electron microscopy (ASEM), a new pipeline to train a convolutional neural network to detect structures of a wide range in size and complexity. We obtained dedicated models for each structure based on a small number of sparsely annotated ground truth images from only one or two cells. Model generalization was improved with a rapid, computationally effective strategy to refine a trained model by including a few additional annotations. We identified mitochondria, Golgi apparatus, endoplasmic reticulum, nuclear pore complexes, caveolae, clathrin-coated pits, and vesicles imaged by focused ion beam scanning electron microscopy. We uncovered a wide range of membrane-nuclear pore diameters within a single cell and derived morphological metrics from clathrin-coated pits and vesicles, consistent with the classical constant-growth assembly model.

An objective method for pedestrian occlusion level classification

Pedestrian detection is among the most safety-critical features of driver assistance systems for autonomous vehicles. One of the most complex detection challenges is that of partial occlusion, where a target object is only partially available to the sensor due to obstruction by another foreground object.A number of current pedestrian detection benchmarks provide annotation for partial occlusion to assess algorithm performance in these scenarios, however each benchmark varies greatly in their definition of the occurrence and severity of occlusion. In addition, current occlusion level annotation methods contain a high degree of subjectivity by the human annotator. This can lead to inaccurate or inconsistent reporting of an algorithm’s detection performance for partially occluded pedestrians, depending on which benchmark is used. This research presents a novel, objective method for pedestrian occlusion level classification for ground truth annotation. Occlusion level classification is achieved through the identification of visible pedestrian keypoints and through the use of a novel, effective method of 2D body surface area estimation. Experimental results demonstrate that the proposed method reflects the pixel-wise occlusion level of pedestrians in images and is effective for all forms of occlusion, including challenging edge cases such as self-occlusion, truncation and inter-occluding pedestrians.

Adjusting the Ground Truth Annotations for Connectivity-Based Learning to Delineate.

Deep learning-based approaches to delineating 3D structure depend on accurate annotations to train the networks. Yet in practice, people, no matter how conscientious, have trouble precisely delineating in 3D and on a large scale, in part because the data is often hard to interpret visually and in part because the 3D interfaces are awkward to use. In this paper, we introduce a method that explicitly accounts for annotation inaccuracies. To this end, we treat the annotations as active contour models that can deform themselves while preserving their topology. This enables us to jointly train the network and correct potential errors in the original annotations. The result is an approach that boosts performance of deep networks trained with potentially inaccurate annotations.

SHRED

We present SHRED, a method for 3D SHape REgion Decomposition. SHRED takes a 3D point cloud as input and uses learned local operations to produce a segmentation that approximates fine-grained part instances. We endow SHRED with three decomposition operations: splitting regions, fixing the boundaries between regions, and merging regions together. Modules are trained independently and locally, allowing SHRED to generate high-quality segmentations for categories not seen during training. We train and evaluate SHRED with fine-grained segmentations from PartNet; using its merge-threshold hyperparameter, we show that SHRED produces segmentations that better respect ground-truth annotations compared with baseline methods, at any desired decomposition granularity. Finally, we demonstrate that SHRED is useful for downstream applications, out-performing all baselines on zero-shot fine-grained part instance segmentation and few-shot finegrained semantic segmentation when combined with methods that learn to label shape regions.

Multiclass Land Cover Mapping from Historical Orthophotos Using Domain Adaptation and Spatio-Temporal Transfer Learning

Historical land cover (LC) maps are an essential instrument for studying long-term spatio-temporal changes of the landscape. However, manual labelling on low-quality monochromatic historical orthophotos for semantic segmentation (pixel-level classification) is particularly challenging and time consuming. Therefore, this paper proposes a methodology for the automated extraction of very-high-resolution (VHR) multi-class LC maps from historical orthophotos under the absence of target-specific ground truth annotations. The methodology builds on recent evolutions in deep learning, leveraging domain adaptation and transfer learning. First, an unpaired image-to-image (I2I) translation between a source domain (recent RGB image of high quality, annotations available) and the target domain (historical monochromatic image of low quality, no annotations available) is learned using a conditional generative adversarial network (GAN). Second, a state-of-the-art fully convolutional network (FCN) for semantic segmentation is pre-trained on a large annotated RGB earth observation (EO) dataset that is converted to the target domain using the I2I function. Third, the FCN is fine-tuned using self-annotated data on a recent RGB orthophoto of the study area under consideration, after conversion using again the I2I function. The methodology is tested on a new custom dataset: the ‘Sagalassos historical land cover dataset’, which consists of three historical monochromatic orthophotos (1971, 1981, 1992) and one recent RGB orthophoto (2015) of VHR (0.3–0.84 m GSD) all capturing the same greater area around Sagalassos archaeological site (Turkey), and corresponding manually created annotations (2.7 km² per orthophoto) distinguishing 14 different LC classes. Furthermore, a comprehensive overview of open-source annotated EO datasets for multiclass semantic segmentation is provided, based on which an appropriate pretraining dataset can be selected. Results indicate that the proposed methodology is effective, increasing the mean intersection over union by 27.2% when using domain adaptation, and by 13.0% when using domain pretraining, and that transferring weights from a model pretrained on a dataset closer to the target domain is preferred.

Motion-region annotation for complex videos via label propagation across occluders

Motion cue is pivotal in moving object analysis, which is the root for motion segmentation and detection. These preprocessing tasks are building blocks for several applications such as recognition, matching and estimation. To devise a robust algorithm for motion analysis, it is imperative to have a comprehensive dataset to evaluate an algorithm’s performance. The main limitation in making these kind of datasets is the creation of ground-truth annotation of motion, as each moving object might span over multiple frames with changes in size, illumination and angle of view. Besides the optical changes, the object can undergo occlusion by static or moving occluders. The challenge increases when the video is captured by a moving camera. In this paper, we tackle the task of providing ground-truth annotation on motion regions in videos captured from a moving camera. With minimal manual annotation of an object mask, we are able to propagate the label mask in all the frames. Object label correction based on static and moving occluder is also performed by applying occluder mask tracking for a given depth ordering. A motion annotation dataset is also proposed to evaluate algorithm performance. The results show that our cascaded-naive approach provides successful results. All the resources of the annotation tool are publicly available at http://dixie.udg.edu/anntool/.

Deep learning model for automatic segmentation of lungs and pulmonary metastasis in small animal MR images.

Lungs are the most frequent site of metastases growth. The amount and size of pulmonary metastases acquired from MRI imaging data are the important criteria to assess the efficacy of new drugs in preclinical models. While efficient solutions both for MR imaging and the downstream automatic segmentation have been proposed for human patients, both MRI lung imaging and segmentation in preclinical animal models remains challenging due to the physiological motion (respiratory and cardiac movements), to the low amount of protons in this organ and to the particular challenge of precise segmentation of metastases. As a consequence post-mortem analysis is currently required to obtain information on metastatic volume. In this work, we have developed a complete methodological pipeline for automated analysis of lungs and metastases in mice, consisting of an MR sequence for image acquisition and a deep learning method for automatic segmentation of both lungs and metastases. On one hand, we optimized an MR sequence for mouse lung imaging with high contrast for high detection sensitivity. On the other hand we developed DeepMeta, a multiclass U-Net 3+ deep learning model to automatically segment the images. To assess if the proposed deep learning pipeline is able to provide an accurate segmentation of both lungs and pulmonary metastases, we have longitudinally imaged mice with fast- and slow-growing metastasis. Fifty-five balb/c mice were injected with two different derivatives of renal carcinoma cells. Mice were imaged with a SG-bSSFP (self-gated balanced steady state free precession) sequence at different time points after the injection of cancer cells. Both lung and metastases segmentations were manually performed by experts. DeepMeta was trained to perform lung and metastases segmentation based on the resulting ground truth annotations. Volumes of lungs and of pulmonary metastases as well as the number of metastases per mouse were measured on a separate test dataset of MR images. Thanks to the SG method, the 3D bSSFP images of lungs were artifact-free, enabling the downstream detection and serial follow-up of metastases. Moreover, both lungs and metastases segmentation was accurately performed by DeepMeta as soon as they reached the volume of . Thus we were able to distinguish two groups of mice in terms of number and volume of pulmonary metastases as well as in terms of the slow versus fast patterns of growth of metastases. We have shown that our methodology combining SG-bSSFP with deep learning, enables processing of the whole animal lungs and is thus a viable alternative to histology alone.