Segmentation Of Anatomical Structures Research Articles

Segmenting functional tissue units across human organs using community-driven development of generalizable machine learning algorithms

The development of a reference atlas of the healthy human body requires automated image segmentation of major anatomical structures across multiple organs based on spatial bioimages generated from various sources with differences in sample preparation. We present the setup and results of the Hacking the Human Body machine learning algorithm development competition hosted by the Human Biomolecular Atlas (HuBMAP) and the Human Protein Atlas (HPA) teams on the Kaggle platform. We create a dataset containing 880 histology images with 12,901 segmented structures, engaging 1175 teams from 78 countries in community-driven, open-science development of machine learning models. Tissue variations in the dataset pose a major challenge to the teams which they overcome by using color normalization techniques and combining vision transformers with convolutional models. The best model will be productized in the HuBMAP portal to process tissue image datasets at scale in support of Human Reference Atlas construction.

Roadmaps for Guiding Chest Computed Tomography Interpretation involving Pneumonia.

CT scanning of the chest is one the most important imaging modalities available for pulmonary disease diagnosis. Lung segmentation plays a crucial step in the pipeline of computer-aided analysis and diagnosis. As deep learning models have achieved human-level accuracy in semantic segmentation of anatomical structures, we propose to use trained deep learning models to predict both healthy and infectious areas in chest CT slices. The semantic segmentation results are summarized and visualized using volume rendering technology in the form of roadmaps. The roadmaps consist of both location and volume information that can be used as a location guidance for inspecting suspected pulmonary lesions of chest CT and can possibly be combined into a rapid triage algorithm for treating acute pulmonary diseases.Clinical Relevance- This research applied trained semantic segmentation models in identifying normal lung and pneumonic infection areas to generate a roadmap for assisting medical doctors in browsing chest CT and prognostication.

Current and Emerging Trends in Medical Image Segmentation With Deep Learning

In recent years, the segmentation of anatomical or pathological structures using deep learning has experienced a widespread interest in medical image analysis. Remarkably successful performance has been reported in many imaging modalities and for a variety of clinical contexts to support clinicians in computer-assisted diagnosis, therapy or surgical planning purposes. However, despite the increasing amount of medical image segmentation challenges, there remains little consensus on which methodology perform best. Therefore, we examine in this paper the numerous developments and breakthroughs brought since the rise of U-Net inspired architectures. Especially, we focus on the technical challenges and emerging trends that the community is now focusing on, including conditional generative adversarial and cascaded networks, medical Transformers, contrastive learning, knowledge distillation, active learning, prior knowledge embedding, cross-modality learning, multi-structure analysis, federated learning or semi-supervised and self-supervised paradigms. We also suggest possible avenues to be further investigated in future research efforts.

3D capturing anatomic complex biomodel in medicine veterinary: how to prepare, digitize and replicate the male canine urogenital system

Medical images combined with 3D printing techniques have been a differential for clinical surgical and didactic approaches. The segmentation of anatomical structures in medical images for their 3D printing of biomodels has been showing promise in medical applications. The use of iodine contrast in solution on anatomical specimens promotes an increasing the density and better caption resolution of CT, expanding the range of possibilities for 3D medicine. With the capture of a complex anatomical arrangement, there is an advantage over the use of biomodels both in a digital and physical interactive way. In this article, we explain how to digitize a set of anatomical structures of the canine urogenital system in topographic position and how to create 3D digital and printed replicas. The biomodel can be applied in the most diverse areas of veterinary medicine and related fields.

Research status and outlook of deep learning in oral and maxillofacial medical imaging

Artificial intelligence, represented by deep learning, has received increasing attention in the field of oral and maxillofacial medical imaging, which has been widely studied in image analysis and image quality improvement. This narrative review provides an insight into the following applications of deep learning in oral and maxillofacial imaging: detection, recognition and segmentation of teeth and other anatomical structures, detection and diagnosis of oral and maxillofacial diseases, and forensic personal identification. In addition, the limitations of the studies and the directions for future development are summarized.

Segmentation of Anatomical Structures of the Left Heart from Echocardiographic Images Using Deep Learning.

Knowledge about the anatomical structures of the left heart, specifically the atrium (LA) and ventricle (i.e., endocardium-Vendo-and epicardium-LVepi) is essential for the evaluation of cardiac functionality. Manual segmentation of cardiac structures from echocardiography is the baseline reference, but results are user-dependent and time-consuming. With the aim of supporting clinical practice, this paper presents a new deep-learning (DL)-based tool for segmenting anatomical structures of the left heart from echocardiographic images. Specifically, it was designed as a combination of two convolutional neural networks, the YOLOv7 algorithm and a U-Net, and it aims to automatically segment an echocardiographic image into LVendo, LVepi and LA. The DL-based tool was trained and tested on the Cardiac Acquisitions for Multi-Structure Ultrasound Segmentation (CAMUS) dataset of the University Hospital of St. Etienne, which consists of echocardiographic images from 450 patients. For each patient, apical two- and four-chamber views at end-systole and end-diastole were acquired and annotated by clinicians. Globally, our DL-based tool was able to segment LVendo, LVepi and LA, providing Dice similarity coefficients equal to 92.63%, 85.59%, and 87.57%, respectively. In conclusion, the presented DL-based tool proved to be reliable in automatically segmenting the anatomical structures of the left heart and supporting the cardiological clinical practice.

ACU[formula omitted]E-Net: A novel predict–refine attention network for segmentation of soft-tissue structures in ultrasound images

Segmentation of anatomical structures in ultrasound images is a challenging task due to existence of artifacts inherit to the modality such as speckle noise, attenuation, shadowing, uneven textures and blurred boundaries. This paper presents a novel attention-based predict–refine network, called ACU2E-Net, for segmentation of soft-tissue structures in ultrasound images. The network consists of two modules: a predict module, which is built upon our newly proposed attentive coordinate convolution; and a novel multi-head residual refinement module, which consists of three parallel residual refinement modules. The attentive coordinate convolution is designed to improve the segmentation accuracy by perceiving the shape and positional information of the target anatomy. The proposed multi-head residual refinement module reduces both segmentation biases and variances by integrating residual refinement and ensemble strategies. Moreover, it avoids multi-pass training and inference commonly seen in ensemble methods. To show the effectiveness of our method, we collect a comprehensive dataset of thyroid ultrasound scans from 12 different imaging centers, and evaluate our proposed network against state-of-the-art segmentation methods. Comparisons against state-of-the-art models demonstrate the competitive performance of our newly designed network on both the transverse and sagittal thyroid images. Ablation studies show that proposed modules improve the segmentation Dice score of the baseline model from 79.62% to 80.97% and 82.92% while reducing the variance from 6.12% to 4.67% and 3.21% in transverse and sagittal views, respectively.

Vision Transformers in medical computer vision—A contemplative retrospection

Vision Transformers (ViTs), with the magnificent potential to unravel the information contained within images, have evolved as one of the most contemporary and dominant architectures that are being used in the field of computer vision. These are immensely utilized by plenty of researchers to perform new as well as former experiments. Here, in this article, we investigate the intersection of vision transformers and medical images. We proffered an overview of various ViT based frameworks that are being used by different researchers to decipher the obstacles in medical computer vision. We surveyed the applications of Vision Transformers in different areas of medical computer vision such as image-based disease classification, anatomical structure segmentation, registration, region-based lesion detection, captioning, report generation, and reconstruction using multiple medical imaging modalities that greatly assist in medical diagnosis and hence treatment process. Along with this, we also demystify several imaging modalities used in medical computer vision. Moreover, to get more insight and deeper understanding, the self-attention mechanism of transformers is also explained briefly. Conclusively, the ViT based solutions for each image analytics task are critically analyzed, open challenges are discussed and the pointers to possible solutions for future direction are deliberated. We hope this review article will open future research directions for medical computer vision researchers.

A deep learning based dual encoder–decoder framework for anatomical structure segmentation in chest X-ray images

Automated multi-organ segmentation plays an essential part in the computer-aided diagnostic (CAD) of chest X-ray fluoroscopy. However, developing a CAD system for the anatomical structure segmentation remains challenging due to several indistinct structures, variations in the anatomical structure shape among different individuals, the presence of medical tools, such as pacemakers and catheters, and various artifacts in the chest radiographic images. In this paper, we propose a robust deep learning segmentation framework for the anatomical structure in chest radiographs that utilizes a dual encoder–decoder convolutional neural network (CNN). The first network in the dual encoder–decoder structure effectively utilizes a pre-trained VGG19 as an encoder for the segmentation task. The pre-trained encoder output is fed into the squeeze-and-excitation (SE) to boost the network’s representation power, which enables it to perform dynamic channel-wise feature calibrations. The calibrated features are efficiently passed into the first decoder to generate the mask. We integrated the generated mask with the input image and passed it through a second encoder–decoder network with the recurrent residual blocks and an attention the gate module to capture the additional contextual features and improve the segmentation of the smaller regions. Three public chest X-ray datasets are used to evaluate the proposed method for multi-organs segmentation, such as the heart, lungs, and clavicles, and single-organ segmentation, which include only lungs. The results from the experiment show that our proposed technique outperformed the existing multi-class and single-class segmentation methods.

Contour interpolation by deep learning approach.

Contour interpolation is an important tool for expediting manual segmentation of anatomical structures. The process allows users to manually contour on discontinuous slices and then automatically fill in the gaps, therefore saving time and efforts. The most used conventional shape-based interpolation (SBI) algorithm, which operates on shape information, often performs suboptimally near the superior and inferior borders of organs and for the gastrointestinal structures. In this study, we present a generic deep learning solution to improve the robustness and accuracy for contour interpolation, especially for these historically difficult cases. A generic deep contour interpolation model was developed and trained using 16,796 publicly available cases from 5 different data libraries, covering 15 organs. The network inputs were a image patch and the two-dimensional contour masks for the top and bottom slices of the patch. The outputs were the organ masks for the three middle slices. The performance was evaluated on both dice scores and distance-to-agreement (DTA) values. The deep contour interpolation model achieved a dice score of and a mean DTA value of , averaged on 3167 testing cases of all 15 organs. In a comparison, the results by the conventional SBI method were and , respectively. For the difficult cases, the dice score and DTA value were and by the deep interpolator, compared with and by SBI. The t-test results confirmed that the performance improvements were statistically significant ( ) for all cases in dice scores and for small organs and difficult cases in DTA values. Ablation studies were also performed. A deep learning method was developed to enhance the process of contour interpolation. It could be useful for expediting the tasks of manual segmentation of organs and structures in the medical images.

DBA-UNet: a double U-shaped boundary attention network for maxillary sinus anatomical structure segmentation in CBCT images

DBA-UNet: a double U-shaped boundary attention network for maxillary sinus anatomical structure segmentation in CBCT images

MobileUNet-FPN: A Semantic Segmentation Model for Fetal Ultrasound Four-Chamber Segmentation in Edge Computing Environments.

The apical four-chamber (A4C) view in fetal echocardiography is a prenatal examination widely used for the early diagnosis of congenital heart disease (CHD). Accurate segmentation of A4C key anatomical structures is the basis for automatic measurement of growth parameters and necessary disease diagnosis. However, due to the ultrasound imaging arising from artefacts and scattered noise, the variability of anatomical structures in different gestational weeks, and the discontinuity of anatomical structure boundaries, accurately segmenting the fetal heart organ in the A4C view is a very challenging task. To this end, we propose to combine an explicit Feature Pyramid Network (FPN), MobileNet and UNet, i.e., MobileUNet-FPN, for the segmentation of 13 key heart structures. To our knowledge, this is the first AI-based method that can segment so many anatomical structures in fetal A4C view. We split the MobileNet backbone network into four stages and use the features of these four phases as the encoder and the upsampling operation as the decoder. We build an explicit FPN network to enhance multi-scale semantic information and ultimately generate segmentation masks of key anatomical structures. In addition, we design a multi-level edge computing system and deploy the distributed edge nodes in different hospitals and city servers, respectively. Then, we train the MobileUNet-FPN model in parallel at each edge node to effectively reduce the network communication overhead. Extensive experiments are conducted and the results show the superior performance of the proposed model on the fetal A4C and femoral-length images.

IFT-Net: Interactive Fusion Transformer Network for Quantitative Analysis of Pediatric Echocardiography.

The task of automatic segmentation and measurement of key anatomical structures in echocardiography is critical for subsequent extraction of clinical parameters. However, the influence of boundary blur, speckle noise, and other factors increase the difficulty of fully automatically segmenting 2D ultrasound images. The previous research has addressed this challenge using convolutional neural networks (CNN), which fails to consider global contextual information and long-range dependency. To further improve the quantitative analysis of pediatric echocardiography, this paper proposes an interactive fusion transformer network (IFT-Net) for quantitative analysis of pediatric echocardiography, which achieves the bidirectional fusion between local features and global context information by constructing interactive learning between the convolution branch and the transformer branch. First, we construct a dual-attention pyramid transformer (DPT) branch to model the long-range dependency from spatial and channels and enhance the learning of global context information. Second, we design a bidirectional interactive fusion (BIF) unit that fuses the local and global features interactively, maximizes their preservation and refines the segmentation. Finally, we measure the clinical anatomical parameters through key point positioning. Based on the parasternal short-axis (PSAX) view of the heart base from pediatric echocardiography, we segment and quantify the right ventricular outflow tract (RVOT) and aorta (AO) with promising results, indicating the potential clinical application. The code is publicly available at: https://github.com/Zhaocheng1/IFT-Net.

Deep Learning for Multi-Tissue Segmentation and Fully Automatic Personalized Biomechanical Models from BACPAC Clinical Lumbar Spine MRI.

In vivo retrospective study of fully automatic quantitative imaging feature extraction from clinically acquired lumbar spine magnetic resonance imaging (MRI). To demonstrate the feasibility of substituting automatic for human-demarcated segmentation of major anatomic structures in clinical lumbar spine MRI to generate quantitative image-based features and biomechanical models. Previous studies have demonstrated the viability of automatic segmentation applied to medical images; however, the feasibility of these networks to segment clinically acquired images has not yet been demonstrated, as they largely rely on specialized sequences or strict quality of imaging data to achieve good performance. Convolutional neural networks were trained to demarcate vertebral bodies, intervertebral disc, and paraspinous muscles from sagittal and axial T1-weighted MRIs. Intervertebral disc height, muscle cross-sectional area, and subject-specific musculoskeletal models of tissue loading in the lumbar spine were then computed from these segmentations and compared against those computed from human-demarcated masks. Segmentation masks, as well as the morphological metrics and biomechanical models computed from those masks, were highly similar between human- and computer-generated methods. Segmentations were similar, with Dice similarity coefficients of 0.77 or greater across networks, and morphological metrics and biomechanical models were similar, with Pearson R correlation coefficients of 0.69 or greater when significant. This study demonstrates the feasibility of substituting computer-generated for human-generated segmentations of major anatomic structures in lumbar spine MRI to compute quantitative image-based morphological metrics and subject-specific musculoskeletal models of tissue loading quickly, efficiently, and at scale without interrupting routine clinical care.

Analysis of Current Deep Learning Networks for Semantic Segmentation of Anatomical Structures in Laparoscopic Surgery.

Semantic segmentation of anatomical structures in laparoscopic videos is a crucial task to enable the development of new computer-assisted systems that can assist surgeons during surgery. However, this is a difficult task due to artifacts and similar visual characteristics of anatomical structures on the laparoscopic videos. Recently, deep learning algorithms have been showed promising results on the segmentation of laparoscopic instruments. However, due to the lack of large public datasets for semantic segmentation of anatomical structures, there are only a few studies on this task. In this work, we evaluate the performance of five networks, namely U-Net, U-Net++, DynUNet, UNETR and DeepLabV3+, for segmentation of laparoscopic cholecystectomy images from the recently released CholecSeg8k dataset. To the best of our knowledge, this is the first benchmark performed on this dataset. Training was performed with dice loss. The networks were evaluated on segmentation of 8 anatomical structures and instruments, performance was quantified through the dice coefficient, intersection over union, recall, and precision. Apart from the U-Net, all networks obtained scores similar to each other, with the U-Net++ being the network with the best overall score with a mean Dice value of 0.62. Overall, the results show that there is still room for improvement in the segmentation of anatomical structures from laparoscopic videos. Clinical Relevance- The results of this study show the potential of deep neural networks for the segmentation of anatomical structures in laparoscopic images which can later be incorporated into computer-aided systems for surgeons.

Artificial Intelligence Radiotherapy Planning: Automatic Segmentation of Human Organs in CT Images Based on a Modified Convolutional Neural Network.

Objective:Precise segmentation of human organs and anatomic structures (especially organs at risk, OARs) is the basis and prerequisite for the treatment planning of radiation therapy. In order to ensure rapid and accurate design of radiotherapy treatment planning, an automatic organ segmentation technique was investigated based on deep learning convolutional neural network.MethodA deep learning convolutional neural network (CNN) algorithm called BCDU-Net has been modified and developed further by us. Twenty two thousand CT images and the corresponding organ contours of 17 types delineated manually by experienced physicians from 329 patients were used to train and validate the algorithm. The CT images randomly selected were employed to test the modified BCDU-Net algorithm. The weight parameters of the algorithm model were acquired from the training of the convolutional neural network.ResultThe average Dice similarity coefficient (DSC) of the automatic segmentation and manual segmentation of the human organs of 17 types reached 0.8376, and the best coefficient reached up to 0.9676. It took 1.5–2 s and about 1 h to automatically segment the contours of an organ in an image of the CT dataset for a patient and the 17 organs for the CT dataset with the method developed by us, respectively.ConclusionThe modified deep neural network algorithm could be used to automatically segment human organs of 17 types quickly and accurately. The accuracy and speed of the method meet the requirements of its application in radiotherapy.

Variational Inference for Quantifying Inter-observer Variability in Segmentation of Anatomical Structures.

Lesions or organ boundaries visible through medical imaging data are often ambiguous, thus resulting in significant variations in multi-reader delineations, i.e., the source of aleatoric uncertainty. In particular, quantifying the inter-observer variability of manual annotations with Magnetic Resonance (MR) Imaging data plays a crucial role in establishing a reference standard for various diagnosis and treatment tasks. Most segmentation methods, however, simply model a mapping from an image to its single segmentation map and do not take the disagreement of annotators into consideration. In order to account for inter-observer variability, without sacrificing accuracy, we propose a novel variational inference framework to model the distribution of plausible segmentation maps, given a specific MR image, which explicitly represents the multi-reader variability. Specifically, we resort to a latent vector to encode the multi-reader variability and counteract the inherent information loss in the imaging data. Then, we apply a variational autoencoder network and optimize its evidence lower bound (ELBO) to efficiently approximate the distribution of the segmentation map, given an MR image. Experimental results, carried out with the QUBIQ brain growth MRI segmentation datasets with seven annotators, demonstrate the effectiveness of our approach.

금속 음영이 포함된 CBCT 영상에서 딥러닝을 이용한 해부학적 구조물의 다중 클래스 분할 방법

In order to perform preoperative surgical planning, accurate segmentation of anatomical structures in cone-beam computed tomography (CBCT) images is required. However, this image segmentation is often impeded by metal artifacts, and it takes a lot of time due to morphological variability in patients. In this paper, we proposed a deep learning based automatic multi-class segmentation method for anatomical structures in CBCT images containing metal artifacts. Four U-Net based deep learning models were used for anatomical structure segmentation. Each deep learning model was constructed by changing the encoder of U-Net architecture to the backbones (DenseNet121, VGGNet16, ResNet101, and EfficienNetB4). For training and testing our method, we used 20744 CBCT images containing metal artifacts from 30 patient datasets. Experimental results show that the segmentation performances of the mandible, midfacial bone, mandibular canal, and maxillary sinus were achieved F1 scores of 0.912±0.070, 0.880±0.080, 0.687±0.265, and 0.954±0.063 using DenseNet121 with Tversky loss, respectively. Furthermore, our method was able to perform robust and accurate segmentation of anatomical structures in CBCT images containing metal artifacts.

Deep-Learning-Based Cerebral Artery Semantic Segmentation in Neurosurgical Operating Microscope Vision Using Indocyanine Green Fluorescence Videoangiography

There have been few anatomical structure segmentation studies using deep learning. Numbers of training and ground truth images applied were small and the accuracies of which were low or inconsistent. For a surgical video anatomy analysis, various obstacles, including a variable fast-changing view, large deformations, occlusions, low illumination, and inadequate focus occur. In addition, it is difficult and costly to obtain a large and accurate dataset on operational video anatomical structures, including arteries. In this study, we investigated cerebral artery segmentation using an automatic ground-truth generation method. Indocyanine green (ICG) fluorescence intraoperative cerebral videoangiography was used to create a ground-truth dataset mainly for cerebral arteries and partly for cerebral blood vessels, including veins. Four different neural network models were trained using the dataset and compared. Before augmentation, 35,975 training images and 11,266 validation images were used. After augmentation, 260,499 training and 90,129 validation images were used. A Dice score of 79% for cerebral artery segmentation was achieved using the DeepLabv3+ model trained using an automatically generated dataset. Strict validation in different patient groups was conducted. Arteries were also discerned from the veins using the ICG videoangiography phase. We achieved fair accuracy, which demonstrated the appropriateness of the methodology. This study proved the feasibility of operating field view of the cerebral artery segmentation using deep learning, and the effectiveness of the automatic blood vessel ground truth generation method using ICG fluorescence videoangiography. Using this method, computer vision can discern blood vessels and arteries from veins in a neurosurgical microscope field of view. Thus, this technique is essential for neurosurgical field vessel anatomy-based navigation. In addition, surgical assistance, safety, and autonomous surgery neurorobotics that can detect or manipulate cerebral vessels would require computer vision to identify blood vessels and arteries.

Automatic segmentation of the clinical target volume and organs at risk for rectal cancer radiotherapy using structure-contextual representations based on 3D high-resolution network

Radiotherapy is a cancer treatment that uses high doses of radiation to kill cancer cells. Segmentation of the clinical target volumes (CTVs) and organs at risk (OARs) is an essential step in rectal cancer radiotherapy treatment planning. However, the manual segmentation of CTVs and OARs is labor-intensive and prone to intra- and inter-rater variations. The challenge of this task lies in large shape variations and low-contrast structure boundary for some structures like intestine. The contextual information is very important to handle this challenge. Recently, deep learning has greatly improved the state-of-the-art results in automatic segmentation of anatomical structures including OARs and CTVs. However, existing approaches barely take advantages of relations among anatomical structures as contextual information, which limits the segmentation accuracy. In this study, we propose a novel structure-contextual representations approach based on 3D high-resolution network (3D HRNet-SCR) for segmentation of OARs and CTV of rectal cancer. First, we design a structure-contextual representation module (SCR) which could compute the representation for each structure of interest by aggregating the representations of voxels inside the structure and enhance the feature representation of each voxel with learned structure-contextual representations. Second, we extend the powerful 2D HRNet for semantic segmentation to 3D paradigm to better capture the contextual information across slices of volumetric data. Our propose approach integrates the proposed SCR module on top of the 3D HRNet to form a high-performance segmentation framework. Finally, we collect a large-scale rectal cancer dataset of 536 CT scans (total 64320 slices) for evaluation. Our proposed framework is extensively tested on this self-collected dataset, showing superiority compared with state-of-the-art OARs and CTV segmentation methods for rectal cancer treatment.