Deep Learning Segmentation Research Articles

Abstract 7386: Multi-plane rectal tumor segmentation on pre-treatment MRI via human-in-the-loop deep learning

Abstract Introduction: Precise segmentation of rectal tumors on pre-treatment MRI is critical for in vivo clinical staging and treatment selection. While automated deep learning models can help alleviate laborious manual annotation, these require significant expertise and time commitment. Towards addressing these issues, we present initial results of an integrated human-in-the-loop refinement learning model for semi-automated segmentation of rectal tumors across multiple acquisition planes (axial, coronal) on pre-treatment MRI scans. Methods: This retrospective study accrued baseline MRI scans from rectal cancer patients at four different institutions. A deep learning (DL) nnUNet tumor segmentation model was first trained on both axial plane and coronal plane T2w MRIs from two institutions, with segregated training and internal validation cohorts based on available radiologist annotations. The DL model outputs tumor delineations on both axial and coronal MRI scans in a separate human-in-the-loop (HITL) optimization cohort. Two rounds of HITL refinement learning were then conducted, where in each round model-generated segmentations were refined by two novice radiology readers. Refined annotations were then used to re-optimize the nnUnet model to yield final tumor segmentations. Model-generated segmentations were evaluated before HITL as well as after each round of HITL against “ground truth” radiologist annotations obtained for an external validation cohort, with performance quantified via dice similarity coefficient (DSC) as well as time taken for human annotation of MRIs. Statistical significance was determined via Wilcoxon rank-sum testing. Results: A total of 381 MRI scans from 231 rectal cancer patients were considered. After one round of HITL refinement, DL-based segmentations yielded the best overall performance on the external validation cohort (N=20, two institutions) for both axial (DSC = 0.76) and coronal scans (DSC=0.75). A second round of HITL refinement did not yield significantly different model performance. By comparison, the pre-HITL DL segmentation model yielded significantly worse performance on both axial (DSC=0.60, p=0.0362) and coronal (DSC=0.62, p=0.0186) MRI scans. HITL refinement for rectal tumor annotation was also found to significantly improve reader efficiency by requiring only 18 minutes/scan compared to manual “ground truth” de novo annotations which took up to 38 minutes/scan (p<0.001). Conclusion: Human-in-the-loop deep learning models can help generate accurate rectal tumor segmentations on pre-treatment MRI scans across both axial and coronal acquisition planes. Improvements to model generalizability, model efficacy, and reader efficiency suggest the potential for HITL-based semi-automated segmentation schemes for other medical imaging tasks. Citation Format: Michael Kong, Thomas DeSilvio, Leo Bao, Brennan Flannery, Benjamin N. Parker, Stephen Tang, Gregory O'Connor, Amit Gupta, Emily Steinhagen, Andrei S. Purysko, Eric L. Marderstein, Aaron Carroll, Marka Crittenden, Michael Gough, Kristina Young, Satish E. Viswanath. Multi-plane rectal tumor segmentation on pre-treatment MRI via human-in-the-loop deep learning [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 7386.

Abstract MP76: Street View Greenspace Exposures and Cardiovascular Health Among Children - Evidence From Project Viva

Introduction: Research on greenspace and cardiovascular health (CVH) in children is limited, and most previous studies rely on coarse satellite-based measurements of greenspace, which do not capture exposure as individuals experience it. Street View imagery (SVI) enables measurement of greenspace from a ground-level perspective and captures specific elements of greenspace that may be most relevant to health. This study aimed to examine associations of SVI-based greenspace with CVH in children in Project Viva, an eastern Massachusetts-based cohort. Methods: Using data from 1,163 children, we applied deep learning segmentation algorithms to SVI from 2007-2010 to derive metrics of greenspace (%of trees, or grass) within a 500m radius of participants’ residential addresses obtained during mid-childhood (mean age 7.9y). At mid-childhood and early adolescence (mean age 13.2y), we calculated CVH scores (range 0-100 units; higher scores represent better CVH) using the American Heart Association’s Life’s Essential 8 algorithm, based on 4 biomedical measures (body mass index, blood lipids, blood glucose, and blood pressure) and 4 behavioral measures (heart healthy levels of diet, physical activity, avoidance of nicotine, and sleep). Using linear regression, we examined associations of mid-childhood SVI-based greenspace metrics with mid-childhood (cross-sectional) and early adolescence (longitudinal) CVH scores, adjusting for child’s sex, race and ethnicity, and age at outcome, as well as an index score of socioeconomic status comprising individual and neighborhood metrics. Results: Among the 1163 participants, 65% were non-Hispanic White and 49% were female. The mean (SD) LE8 score was 83.3 (8.0) in mid-childhood and 81.7 (8.9) in early adolescence. In the unadjusted models, a higher %trees and grass per SD, as measured by SVI within children’s residence in mid-childhood, was associated with a higher overall CVH score (β 1.63 [95%CI: 0.93, 2.33] for trees; β 1.10 [95%CI: 0.36, 1.83] for grass). In longitudinal analyses, the results were similar (β 1.69 [95%CI: 0.87, 2.52] for trees; β 0.82 [95%CI: -0.01, 1.64] for grass). In fully-adjusted analyses, the cross-sectional and longitudinal results for %trees and grass were attenuated and not statistically significant. However, when CVH scores were disaggregated by behavioral and biomedical sub-scores, %trees was positively associated with behavioral sub-scores in cross-sectional fully-adjusted analyses at mid-childhood (β 0.89 [95%CI: 0.32, 1.46]). Conclusion: Among participants in Project Viva, SVI-based metrics of greenspace exposure in mid-childhood were cross-sectionally associated with behavioral factors promoting higher CVH. Greenspace may hold promise for improving children’s CVH through the behavioral pathway and should be considered in future health-promotion efforts.

Liver tumor segmentation method based on U-Net architecture: a review

Liver cancer is a disease with a high incidence and high probability of deterioration, and for the rapid diagnosis of liver disease, CT scans must be used to segment the liver tumors. For the past few years, with the rapid development of deep learning, many deep learning methods for liver tumor segmentation using abdominal computed tomography (CT) images have appeared, and the clinical application of these methods is of important significance for computer-aided diagnosis of liver tumors. The U-Net, with its unique U-shape network structure, exhibits excellent performance in medical image segmentation field and has been extensively utilized in various medical image segmentation applications. In this paper, we summarize the researches of U-Net and its improved networks in CT image segmentation of liver tumors by deep learning methods and classify various U-Net-based convolutional neural networks (CNNs) into 2D (two-dimensional), 3D (three-dimensional), and 2.5D (2.5-dimensional). In this paper, 2D, 3D, and 2.5D convolutional neural networks are summarized. In addition, this paper summarizes the advantages and disadvantages as well as the improvement methods of each type of network, which provides a useful reference for the studies of deep learning based on liver tumor segmentation field. Finally, this paper envisions future research trends for deep learning segmentation methods in the context of liver tumors.

SASAN: ground truth for the effective segmentation and classification of skin cancer using biopsy images.

Early skin cancer diagnosis can save lives; however, traditional methods rely on expert knowledge and can be time-consuming. This calls for automated systems using machine learning and deep learning. However, existing datasets often focus on flat skin surfaces, neglecting more complex cases on organs or with nearby lesions. This work addresses this gap by proposing a skin cancer diagnosis methodology using a dataset named ASAN that covers diverse skin cancer cases but suffers from noisy features. To overcome the noisy feature problem, a segmentation dataset named SASAN is introduced, focusing on Region of Interest (ROI) extraction-based classification. This allows models to concentrate on critical areas within the images while ignoring learning the noisy features. Various deep learning segmentation models such as UNet, LinkNet, PSPNet, and FPN were trained on the SASAN dataset to perform segmentation-based ROI extraction. Classification was then performed using the dataset with and without ROI extraction. The results demonstrate that ROI extraction significantly improves the performance of these models in classification. This implies that SASAN is effective in evaluating performance metrics for complex skin cancer cases. This study highlights the importance of expanding datasets to include challenging scenarios and developing better segmentation methods to enhance automated skin cancer diagnosis. The SASAN dataset serves as a valuable tool for researchers aiming to improve such systems and ultimately contribute to better diagnostic outcomes.

Evaluation of pore-fracture microstructure of gypsum rock fragments using micro-CT

This study utilized X-ray micro-computed tomography (micro-CT) to investigate weathered gypsum rocks which can or do serve as a rock substrate for endolithic organisms, focusing on their internal pore-fracture microstructure, estimating porosity, and quantitative comparison between various samples. Examining sections and reconstructed 3D models provides a more detailed insight into the overall structural conditions within rock fragments and the interconnectivity in pore networks, surpassing the limitations of analyzing individual 2D images. Results revealed diverse gypsum forms, cavities, fractures, and secondary features influenced by weathering. Using deep learning segmentation based on the U-Net models within the Dragonfly software enabled to identify and visualize the porous systems and determinate void space which was used to calculate porosity. This approach allowed to describe what type of microstructures and cavities is responsible for the porous spaces in different gypsum samples. A set of quantitative analysis of the detected void and modeled networks provided a needed information about the development of the pore system, connectivity, and pore size distribution. Comparison with mercury intrusion porosimetry showed that both methods consider different populations of pores. In our case, micro-CT typically detects larger pores (> 10 μm) which is related to the effective resolution of the scanned images. Still, micro-CT demonstrated to be an efficient tool in examining the internal microstructures of weathered gypsum rocks, with promising implications particularly in geobiology and microbiology for the characterization of lithic habitats.

A rapid, low-cost wheat spike grain segmentation and counting system based on deep learning and image processing

Spike grain number is a crucial parameter when estimating wheat yield. However, most methods predominantly focus on a single period, lacking universality. To achieve rapid and intelligent wheat spike grain counting from the filling stage to the mature stage, this study used different smartphones to obtain wheat ear images of different varieties and constructs the spike grain dataset required for deep learning segmentation model through image normalization and data enhancement. Then we propose a segmentation model called Multi-Attention TransUNet (MATransUNet) and a spike grain counting model (SGCountM) to achieve wheat spike grain counting. Our results indicate that, comparing with other models, MATransUNet achieves the best segmentation performance with strong robustness and generalization capabilities, achieving a mean intersection over union(mIoU) of 85.93%. Through the comparative analysis with manual counting results, our Method I, doubling the number of grains on one side of the ears, results in a root mean squared error (RMSE) of 4.38 and a coefficient of determination(R2) of 0.85; while the Method II, adding the grains on both sides of the ears, yields a RMSE of 3.13 and a R2 of 0.93. The counting accuracy significantly improves compared to not using the SGCountM. Moreover, transfer learning notably enhances the accuracy of the segmentation and counting model, enabling accurate spike grain counting from the filling stage to the mature stage. Finally, we integrate MATransUNet and SGCountM to design and implement a wheat spike grain segmentation and counting system (WeChat mini program). In practical tests, the mini program can effectively, accurately segments and counts wheat spike grain, demonstrating its feasibility and effectiveness. This research provides valuable insights into the intelligent application of wheat spike grain counting.

Dense Multi-Scale Graph Convolutional Network for Knee Joint Cartilage Segmentation.

In this paper, we propose a dense multi-scale adaptive graph convolutional network (DMA-GCN) method for automatic segmentation of the knee joint cartilage from MR images. Under the multi-atlas setting, the suggested approach exhibits several novelties, as described in the following. First, our models integrate both local-level and global-level learning simultaneously. The local learning task aggregates spatial contextual information from aligned spatial neighborhoods of nodes, at multiple scales, while global learning explores pairwise affinities between nodes, located globally at different positions in the image. We propose two different structures of building models, whereby the local and global convolutional units are combined by following an alternating or a sequential manner. Secondly, based on the previous models, we develop the DMA-GCN network, by utilizing a densely connected architecture with residual skip connections. This is a deeper GCN structure, expanded over different block layers, thus being capable of providing more expressive node feature representations. Third, all units pertaining to the overall network are equipped with their individual adaptive graph learning mechanism, which allows the graph structures to be automatically learned during training. The proposed cartilage segmentation method is evaluated on the entire publicly available Osteoarthritis Initiative (OAI) cohort. To this end, we have devised a thorough experimental setup, with the goal of investigating the effect of several factors of our approach on the classification rates. Furthermore, we present exhaustive comparative results, considering traditional existing methods, six deep learning segmentation methods, and seven graph-based convolution methods, including the currently most representative models from this field. The obtained results demonstrate that the DMA-GCN outperforms all competing methods across all evaluation measures, providing DSC=95.71% and DSC=94.02% for the segmentation of femoral and tibial cartilage, respectively.

Improved deep learning segmentation of outdoor point clouds with different sampling strategies and using intensities

Abstract The rapid growth of outdoor digital twin data sets and advancements in 3D data acquisition technology have sparked interest in improving segmentation performance using deep learning. This research aims to analyze and evaluate different sampling strategies and optimization techniques while exploring the intensity information of outdoor point cloud data. Two sampling strategies, random and stratified sampling, are employed to divide a limited data set. Additionally, the data set is divided into point cloud data with and without intensity. The PointNet++ model is used to segment the point cloud data into two classes, vegetation and structure. The results indicate that stratified sampling outperforms random sampling, yielding a considerable improvement in mean intersection over union scores of up to 10%. Interestingly, the inclusion of intensity information in the data set does not universally enhance performance. Although the use of intensity improves the performance of random sampling, it does not benefit stratified sampling. This research provides insights into the effectiveness of different sampling strategies for outdoor point cloud data segmentation. The findings can contribute to the development of optimized approaches to improving segmentation accuracy in outdoor digital twin applications using deep learning techniques.

OrgaSegment: deep-learning based organoid segmentation to quantify CFTR dependent fluid secretion

Epithelial ion and fluid transport studies in patient-derived organoids (PDOs) are increasingly being used for preclinical studies, drug development and precision medicine applications. Epithelial fluid transport properties in PDOs can be measured through visual changes in organoid (lumen) size. Such organoid phenotypes have been highly instrumental for the studying of diseases, including cystic fibrosis (CF), which is characterized by genetic mutations of the CF transmembrane conductance regulator (CFTR) ion channel. Here we present OrgaSegment, a MASK-RCNN based deep-learning segmentation model allowing for the segmentation of individual intestinal PDO structures from bright-field images. OrgaSegment recognizes spherical structures in addition to the oddly-shaped organoids that are a hallmark of CF organoids and can be used in organoid swelling assays, including the new drug-induced swelling assay that we show here. OrgaSegment enabled easy quantification of organoid swelling and could discriminate between organoids with different CFTR mutations, as well as measure responses to CFTR modulating drugs. The easy-to-apply label-free segmentation tool can help to study CFTR-based fluid secretion and possibly other epithelial ion transport mechanisms in organoids.

Automatic segmentation of hepatocellular carcinoma on dynamic contrast-enhanced MRI based on deep learning

Objective. Precise hepatocellular carcinoma (HCC) detection is crucial for clinical management. While studies focus on computed tomography-based automatic algorithms, there is a rareness of research on automatic detection based on dynamic contrast enhanced (DCE) magnetic resonance imaging. This study is to develop an automatic detection and segmentation deep learning model for HCC using DCE. Approach: DCE images acquired from 2016 to 2021 were retrospectively collected. Then, 382 patients (301 male; 81 female) with 466 lesions pathologically confirmed were included and divided into an 80% training-validation set and a 20% independent test set. For external validation, 51 patients (42 male; 9 female) in another hospital from 2018 to 2021 were included. The U-net architecture was modified to accommodate multi-phasic DCE input. The model was trained with the training-validation set using five-fold cross-validation, and furtherly evaluated with the independent test set using comprehensive metrics for segmentation and detection performance. The proposed automatic segmentation model consisted of five main steps: phase registration, automatic liver region extraction using a pre-trained model, automatic HCC lesion segmentation using the multi-phasic deep learning model, ensemble of five-fold predictions, and post-processing using connected component analysis to enhance the performance to refine predictions and eliminate false positives. Main results. The proposed model achieved a mean dice similarity coefficient (DSC) of 0.81 ± 0.11, a sensitivity of 94.41 ± 15.50%, a precision of 94.19 ± 17.32%, and 0.14 ± 0.48 false positive lesions per patient in the independent test set. The model detected 88% (80/91) HCC lesions in the condition of DSC > 0.5, and the DSC per tumor was 0.80 ± 0.13. In the external set, the model detected 92% (58/62) lesions with 0.12 ± 0.33 false positives per patient, and the DSC per tumor was 0.75 ± 0.10. Significance. This study developed an automatic detection and segmentation deep learning model for HCC using DCE, which yielded promising post-processed results in accurately identifying and delineating HCC lesions.

Annotator bias and its effect on deep learning segmentation of uncured composite micrographs

This study demonstrated substantial agreement between groups of experts and non-experts in labelling image regions required for semantic segmentation of X-ray micrographs of uncured composite prepregs. Twelve participants, six experts and six non-experts, were given a 1-h training session covering the three different image regions that are present in an uncured composite micrograph: voids, dry fibre areas, and filled fibre areas. High consensus was observed in the centre of objects, but disagreement in labelling between the groups was observed at the interphase regions where the grey level intensity becomes ambiguous in these low-contrast images, such as at the edges of the dry fibre areas and voids. Also, labelling small interlaminar voids caused disagreement. The participants highlighted the role of software by reporting a preference to defining the vertices of a polygon over colouring-in regions of image segments. The resulting Deep Learning segmentation with expert and non-expert group labels were in almost perfect agreement, as measured by the Fleiss Kappa coefficient, and was able to segment voids and dry fibre areas better than thresholding.

Temporary growth cessation of wheat roots following defoliation

Background and aimsDefoliation triggers the remobilisation of root reserves to generate new leaves which can affect root growth until the shoot resumes net assimilation. However, the duration of root growth cessation and its impact on resource uptake potential is uncertain.MethodsWinter wheat was established in a 4 m high outdoor rhizobox facility equipped with imaging panels, sensors, and access points for tracer-labelling. The wheat was defoliated in autumn at early tillering and roots were imaged at a high-time resolution and analyzed by deep learning segmentation. The water and nitrogen (N) uptake were measured using time-domain reflectometer (TDR) sensors and 2H and 15N isotopes.ResultsRoot penetration of wheat paused for 269 °C days (20 days) following defoliation after which it resumed at a similar rate to un-defoliated plants (1.8 mm °C days−1). This caused a substantial decrease in root density with an associated reduction in water and N uptake at maturity, especially from deeper soil layers (>2 m).ConclusionsOur results have significant implications for managing the grazing of dual-purpose crops to balance the interplay between canopy removal and the capacity of deep roots to provide water and N for yield recovery.

A Novel Grading System for Diffuse Chorioretinal Atrophy in Pathologic Myopia.

This study aims to quantitatively assess diffuse chorioretinal atrophy (DCA) in pathologic myopia and establish a standardized classification system utilizing artificial intelligence. A total of 202 patients underwent comprehensive examinations, and 338 eyes were included in the study. The methodology involved image preprocessing, sample labeling, employing deep learning segmentation models, measuring and calculating the area and density of DCA lesions. Lesion severity of DCA was graded using statistical methods, and grades were assigned to describe the morphology of corresponding fundus photographs. Hierarchical clustering was employed to categorize diffuse atrophy fundus into three groups based on the area and density of diffuse atrophy (G1, G2, G3), while high myopic fundus without diffuse atrophy was designated as G0. One-way analysis of variance (ANOVA) and nonparametric tests were conducted to assess the statistical association with different grades of DCA. On the basis of the area and density of DCA, the condition was classified into four grades: G0, G1 (0 < density ≤ 0.093), G2 (0.093 < density ≤ 0.245), and G3 (0.245 < density ≤ 0.712). Fundus photographs depicted a progressive enlargement of atrophic lesions, evolving from punctate-shaped to patchy with indistinct boundaries. DCA atrophy lesions exhibited a gradual shift in color from brown-yellow to yellow-white, originating from the temporal side of the optic disc and extending towards the macula, with severe cases exhibiting widespread distribution throughout the posterior pole. Patients with DCA were significantly older [34.00 (27.00, 48.00) vs 29.00 (26.00, 34.00) years], possessed a longer axial length (28.85 ± 1.57 vs 27.11 ± 1.01 mm), and exhibited a more myopic spherical equivalent [- 13.00 (- 16.00, - 10.50) vs - 9.09 ± 2.41 D] compared to those without DCA (G0) (all P < 0.001). In eyes with DCA, a trend emerged as grades increased from G1 to G3, showing associations with older age, longer axial length, deeper myopic spherical equivalent, larger area of parapapillary atrophy, and increased fundus tessellated density (all P < 0.001). The novel grading system for DCA, based on assessments of area and density, serves as a reliable measure for evaluating the severity of this condition, making it suitable for widespread application in the screening of pathologic myopia.

Computational pathology model to assess acute and chronic transformations of the tubulointerstitial compartment in renal allograft biopsies

Managing patients with kidney allografts largely depends on biopsy diagnosis which is based on semiquantitative assessments of rejection features and extent of acute and chronic changes within the renal parenchyma. Current methods lack reproducibility while digital image data-driven computational models enable comprehensive and quantitative assays. In this study we aimed to develop a computational method for automated assessment of histopathology transformations within the tubulointerstitial compartment of the renal cortex. Whole slide images of modified Picrosirius red-stained biopsy slides were used for the training (n = 852) and both internal (n = 172) and external (n = 94) tests datasets. The pipeline utilizes deep learning segmentations of renal tubules, interstitium, and peritubular capillaries from which morphometry features were extracted. Seven indicators were selected for exploring the intrinsic spatial interactions within the tubulointerstitial compartment. A principal component analysis revealed two independent factors which can be interpreted as representing chronic and acute tubulointerstitial injury. A K-means clustering classified biopsies according to potential phenotypes of combined acute and chronic transformations of various degrees. We conclude that multivariate analyses of tubulointerstitial morphometry transformations enable extraction of and quantification of acute and chronic components of injury. The method is developed for renal allograft biopsies; however, the principle can be applied more broadly for kidney pathology assessment.

Pan-cancer image segmentation based on feature pyramids and Mask R-CNN framework.

Cancer, a disease with a high mortality rate, poses a great threat to patients' physical and mental health and can lead to huge medical costs and emotional damage. With the continuous development of artificial intelligence technologies, deep learning-based cancer image segmentation techniques are becoming increasingly important in cancer detection and accurate diagnosis. However, in segmentation tasks, there are differences in efficiency between large and small objects and limited segmentation effects on objects of individual sizes. The previous segmentation frameworks still have room for improvement in multi-scale collaboration when segmenting objects. This paper proposes a method to train a deep learning segmentation framework using a feature pyramid processing dataset to improve the average precision (AP) index, and realizes multi-scale cooperation in target segmentation. Pan-Cancer Histology Dataset for Nuclei Instance Segmentation and Classification (PanNuke) dataset was selected to include approximately 7500 pathology images with cells from 19 different types of tissues, including five classifications of cancer, non-cancer, inflammation, death, and connective tissue. First, the method uses whole-slide images in the pan-cancer histology dataset for nuclei instance segmentation and classification (PanNuke) dataset, combined with the mask region convolutional neural network (Mask R-CNN) segmentation framework and improved loss function to segment and detect each cellular tissue in cancerous sections. Second, to address the problem of non-synergistic object segmentation at different scales in cancerous tissue segmentation, a scheme using feature pyramids to process the dataset was adopted as part of the feature extraction module. Extensive experimental results on this dataset show that the method in this paper yields 0.269 AP and a boost of about 4% compared to the original Mask R-CNN framework. It is effective and feasible to use feature pyramid to process data set to improve the effect of medical image segmentation.

Adaptive Tensor-Based Feature Extraction for Pupil Segmentation in Cataract Surgery.

Cataract surgery remains the only definitive treatment for visually significant cataracts, which are a major cause of preventable blindness worldwide. Successful performance of cataract surgery relies on stable dilation of the pupil. Automated pupil segmentation from surgical videos can assist surgeons in detecting risk factors for pupillary instability prior to the development of surgical complications. However, surgical illumination variations, surgical instrument obstruction, and lens material hydration during cataract surgery can limit pupil segmentation accuracy. To address these problems, we propose a novel method named adaptive wavelet tensor feature extraction (AWTFE). AWTFE is designed to enhance the accuracy of deep learning-powered pupil recognition systems. First, we represent the correlations among spatial information, color channels, and wavelet subbands by constructing a third-order tensor. We then utilize higher-order singular value decomposition to eliminate redundant information adaptively and estimate pupil feature information. We evaluated the proposed method by conducting experiments with state-of-the-art deep learning segmentation models on our BigCat dataset consisting of 5,700 annotated intraoperative images from 190 cataract surgeries and a public CaDIS dataset. The experimental results reveal that the AWTFE method effectively identifies features relevant to the pupil region and improved the overall performance of segmentation models by up to 2.26% (BigCat) and 3.31% (CaDIS). Incorporation of the AWTFE method led to statistically significant improvements in segmentation performance (P < 1.29 × 10-10 for each model) and yielded the highest-performing model overall (Dice coefficients of 94.74% and 96.71% for the BigCat and CaDIS datasets, respectively). In performance comparisons, the AWTFE consistently outperformed other feature extraction methods in enhancing model performance. In addition, the proposed AWTFE method significantly improved pupil recognition performance by up to 2.87% in particularly challenging phases of cataract surgery.

A New Multi-Atlas Based Deep Learning Segmentation Framework With Differentiable Atlas Feature Warping.

Deep learning based multi-atlas segmentation (DL-MA) has achieved the state-of-the-art performance in many medical image segmentation tasks, e.g., brain parcellation. In DL-MA methods, atlas-target correspondence is the key for accurate segmentation. In most existing DL-MA methods, such correspondence is usually established using traditional or deep learning based registration methods at image level with no further feature level adaption. This could cause possible atlas-target feature inconsistency. As a result, the information from atlases often has limited positive and even counteractive impact on the final segmentation results. To tackle this issue, in this paper, we propose a new DL-MA framework, where a novel differentiable atlas feature warping module with a new smooth regularization term is presented to establish feature level atlas-target correspondence. Comparing with the existing DL-MA methods, in our framework, atlas features containing anatomical prior knowledge are more relevant to the target image feature, leading the final segmentation results to a high accuracy level. We evaluate our framework in the context of brain parcellation using two public MR brain image datasets: LPBA40 and NIREP-NA0. The experimental results demonstrate that our framework outperforms both traditional multi-atlas segmentation (MAS) and state-of-the-art DL-MA methods with statistical significance. Further ablation studies confirm the effectiveness of the proposed differentiable atlas feature warping module.

Deep learning segmentation and registration-driven lung parenchymal volume and movement CT analysis in prone positioning.

To conduct a volumetric and movement analysis of lung parenchyma in prone positioning using deep neural networks (DNNs). We included patients with suspected interstitial lung abnormalities or disease who underwent full-inspiratory supine and prone chest CT at a single institution between June 2021 and March 2022. A thoracic radiologist visually assessed the fibrosis extent in the total lung (using units of 10%) on supine CT. After preprocessing the images into 192×192×192 resolution, a DNN automatically segmented the whole lung and pulmonary lobes in prone and supine CT images. Affine registration matched the patient's center and location, and the DNN deformably registered prone and supine CT images to calculate the x-, y-, z-axis, and 3D pixel movements. In total, 108 CT pairs had successful registration. Prone positioning significantly increased the left lower (90.2±69.5 mL, P = 0.000) and right lower lobar volumes (52.5±74.2 mL, P = 0.000). During deformable registration, the average maximum whole-lung pixel movements between the two positions were 1.5, 1.9, 1.6, and 2.8 cm in each axis and 3D plane. Compared to patients with <30% fibrosis, those with ≥30% fibrosis had smaller volume changes (P<0.001) and smaller pixel movements in all axes between the positions (P = 0.000-0.007). Forced vital capacity (FVC) correlated with the left lower lobar volume increase (Spearman correlation coefficient, 0.238) and the maximum whole-lung pixel movements in all axes (coefficients, 0.311 to 0.357). Prone positioning led to the preferential expansion of the lower lobes, correlated with FVC, and lung fibrosis limited lung expansion during prone positioning.

Modular segmentation, spatial analysis and visualization of volume electron microscopy datasets.

Volume electron microscopy is the method of choice for the in situ interrogation of cellular ultrastructure at the nanometer scale, and with the increase in large raw image datasets generated, improving computational strategies for image segmentation and spatial analysis is necessary. Here we describe a practical and annotation-efficient pipeline for organelle-specific segmentation, spatial analysis and visualization of large volume electron microscopy datasets using freely available, user-friendly software tools that can be run on a single standard workstation. The procedures are aimed at researchers in the life sciences with modest computational expertise, who use volume electron microscopy and need to generate three-dimensional (3D) segmentation labels for different types of cell organelles while minimizing manual annotation efforts, to analyze the spatial interactions between organelle instances and to visualize the 3D segmentation results. We provide detailed guidelines for choosing well-suited segmentation tools for specific cell organelles, and to bridge compatibility issues between freely available open-source tools, we distribute the critical steps as easily installable Album solutions for deep learning segmentation, spatial analysis and 3D rendering. Our detailed description can serve as a reference for similar projects requiring particular strategies for single- or multiple-organelle analysis, which can be achieved with computational resources commonly available to single-user setups.

Deep learning segmentation of the choroid plexus from structural magnetic resonance imaging (MRI): validation and normative ranges across the adult lifespan

BackgroundThe choroid plexus functions as the blood-cerebrospinal fluid (CSF) barrier, plays an important role in CSF production and circulation, and has gained increased attention in light of the recent elucidation of CSF circulation dysfunction in neurodegenerative conditions. However, methods for routinely quantifying choroid plexus volume are suboptimal and require technical improvements and validation. Here, we propose three deep learning models that can segment the choroid plexus from commonly-acquired anatomical MRI data and report performance metrics and changes across the adult lifespan.MethodsFully convolutional neural networks were trained from 3D T1-weighted, 3D T2-weighted, and 2D T2-weighted FLAIR MRI using gold-standard manual segmentations in control and neurodegenerative participants across the lifespan (n = 50; age = 21–85 years). Dice coefficients, 95% Hausdorff distances, and area-under-curve (AUCs) were calculated for each model and compared to segmentations from FreeSurfer using two-tailed Wilcoxon tests (significance criteria: p < 0.05 after false discovery rate multiple comparisons correction). Metrics were regressed against lateral ventricular volume using generalized linear models to assess model performance for varying levels of atrophy. Finally, models were applied to an expanded cohort of adult controls (n = 98; age = 21–89 years) to provide an exemplar of choroid plexus volumetry values across the lifespan.ResultsDeep learning results yielded Dice coefficient = 0.72, Hausdorff distance = 1.97 mm, AUC = 0.87 for T1-weighted MRI, Dice coefficient = 0.72, Hausdorff distance = 2.22 mm, AUC = 0.87 for T2-weighted MRI, and Dice coefficient = 0.74, Hausdorff distance = 1.69 mm, AUC = 0.87 for T2-weighted FLAIR MRI; values did not differ significantly between MRI sequences and were statistically improved compared to current commercially-available algorithms (p < 0.001). The intraclass coefficients were 0.95, 0.95, and 0.96 between T1-weighted and T2-weighted FLAIR, T1-weighted and T2-weighted, and T2-weighted and T2-weighted FLAIR models, respectively. Mean lateral ventricle choroid plexus volume across all participants was 3.20 ± 1.4 cm3; a significant, positive relationship (R2 = 0.54-0.60) was observed between participant age and choroid plexus volume for all MRI sequences (p < 0.001).ConclusionsFindings support comparable performance in choroid plexus delineation between standard, clinically available, non-contrasted anatomical MRI sequences. The software embedding the evaluated models is freely available online and should provide a useful tool for the growing number of studies that desire to quantitatively evaluate choroid plexus structure and function (https://github.com/hettk/chp_seg).