Voxel Class Research Articles

A Hybrid Capsule Network for Automatic 3D Mandible Segmentation applied in Virtual Surgical Planning.

Automatic mandible segmentation of CT images is an essential step to achieve an accurate preoperative prediction of an intended target in three-dimensional (3D) virtual surgical planning. Segmentation of the mandible is a challenging task due to the complexity of the mandible structure, imaging artifacts, and metal implants or dental filling materials. In recent years, utilizing convolutional neural networks (CNNs) have made significant improvements in mandible segmentation. However, aggregating data at pooling layers in addition to collecting and labeling a large volume of data for training CNNs are significant issues in medical practice. We have optimized data-efficient 3D-UCaps to achieve the advantages of both the capsule network and the CNN, for accurate mandible segmentation on volumetric CT images. A novel hybrid loss function based on a weighted combination of the focal and margin loss functions is also proposed to handle the problem of voxel class imbalance. To evaluate the performance of our proposed method, a similar experiment was conducted with the 3D-UNet. All experiments are performed on the public domain database for computational anatomy (PDDCA). The proposed method and 3D-UNet achieved an average dice coefficient of 90% and 88% on the PDDCA, respectively. The results indicate that the proposed method leads to accurate mandible segmentation and outperforms the popular 3D-UNet model. It is concluded that the proposed approach is very effective as it requires more than 50% fewer parameters than the 3D-UNet.

DiSegNet: A deep dilated convolutional encoder-decoder architecture for lymph node segmentation on PET/CT images.

Automated lymph node (LN) recognition and segmentation from cross-sectional medical images is an important step for the automated diagnostic assessment of patients with cancer. Yet, it is still a difficult task owing to the low contrast of LNs and surrounding soft tissues as well as due to the variation in nodal size and shape. In this paper, we present a novel LN segmentation method based on a newly designed neural network for positron emission tomography/computed tomography (PET/CT) images. This work communicates two problems involved in LN segmentation task. Firstly, an efficient loss function named cosine-sine (CS) is proposed for the voxel class imbalance problem in the convolution network training process. Second, a multi-stage and multi-scale Atrous (Dilated) spatial pyramid pooling sub-module, named MS-ASPP, is introduced to the encoder-decoder architecture (SegNet), which aims to make use of multi-scale information to improve the performance of LN segmentation. The new architecture is named DiSegNet (Dilated SegNet). Four-fold cross-validation is performed on 63 PET/CT data sets. In each experiment, 10 data sets are selected randomly for testing and the other 53 for training. The results show that we reach an average 77 % Dice similarity coefficient score with CS loss function by trained DiSegNet, compared to a baseline method SegNet by cross-entropy (CE) with 71 % Dice similarity coefficient. The performance of the proposed DiSegNet with CS loss function suggests its potential clinical value for disease quantification.

Objective and Automated Detection of Diffuse White Matter Abnormality in Preterm Infants Using Deep Convolutional Neural Networks.

Diffuse white matter abnormality (DWMA), or diffuse excessive high signal intensity is observed in 50–80% of very preterm infants at term-equivalent age. It is subjectively defined as higher than normal signal intensity in periventricular and subcortical white matter in comparison to normal unmyelinated white matter on T2-weighted MRI images. Despite the well-documented presence of DWMA, it remains debatable whether DWMA represents pathological tissue injury or a transient developmental phenomenon. Manual tracing of DWMA exhibits poor reliability and reproducibility and unduly increases image processing time. Thus, objective and ideally automatic assessment is critical to accurately elucidate the biologic nature of DWMA. We propose a deep learning approach to automatically identify DWMA regions on T2-weighted MRI images. Specifically, we formulated DWMA detection as an image voxel classification task; that is, the voxels on T2-weighted images are treated as samples and exclusively assigned as DWMA or normal white matter voxel classes. To utilize the spatial information of individual voxels, small image patches centered on the given voxels are retrieved. A deep convolutional neural networks (CNN) model was developed to differentiate DWMA and normal voxels. We tested our deep CNN in multiple validation experiments. First, we examined DWMA detection accuracy of our CNN model using computer simulations. This was followed by in vivo assessments in a cohort of very preterm infants (N = 95) using cross-validation and holdout validation. Finally, we tested our approach on an independent preterm cohort (N = 28) to externally validate our model. Our deep CNN model achieved Dice similarity index values ranging from 0.85 to 0.99 for DWMA detection in the aforementioned validation experiments. Our proposed deep CNN model exhibited significantly better performance than other popular machine learning models. We present an objective and automated approach for accurately identifying DWMA that may facilitate the clinical diagnosis of DWMA in very preterm infants.

Abstract WMP29: Detection and Segmentation of Subarachnoid Hemorrhages With Deep Learning

Introduction: About 5% of all strokes are subarachnoid hemorrhages (SAHs). Accurate segmentation and detection of SAH in CT scans is important because the SAH volume is linked to delayed cerebral ischemia and poor patient outcome. SAH segmentation is a difficult task with high interobserver variability. Previous studies reported a limited average Dice of 0.66 between expert radiologists. We studied the use of deep learning for fast and accurate detection and segmentation of SAH in CT scans. Methodology: A convolutional neural network (CNN) was designed for segmentation and detection of SAH in CT scans. This CNN was trained to classify each voxel as “SAH” or “no SAH”. The input of this CNN was the local information around the voxel and its output was the probability that this voxel is “SAH” or “no SAH”. We used active learning for optimal selection of training samples. By doing so, we boosted the learning performance while keeping a balanced representation of both classes of voxels. Results: The CNN segmented 469 scans from SAH patients and achieved an average Dice of 0.62 ± 0.16 [0.19, 0.92] (avg. ± std. [min., max.]). The CNN also segmented 383 scans from patients with repeated bleeding after an initial SAH. Because most of these patients were treated with endovascular coiling, strong metal artefacts were present in these 383 scans. Regardless of these metal artefacts, an average Dice of 0.66 ± 0.19 [0.05, 0.92] was achieved. Finally, the CNN was used for detection of SAH in 35 scans from SAH patients and 35 images from ischemic stroke patients. The proposed CNN achieved an accuracy of 96% for detection of SAH in these 70 images. The CNN took less than 40 seconds to analyze a CT scan. Conclusion: The proposed CNN can produce SAH segmentations with a similar accuracy as expert radiologists. This CNN was able to reach state-of-the-art SAH segmentation results and SAH detection accuracy.

A Novel Image-Centric Approach Toward Direct Volume Rendering

Transfer function (TF) generation is a fundamental problem in direct volume rendering (DVR). A TF maps voxels to color and opacity values to reveal inner structures. Existing TF tools are complex and unintuitive for the users who are more likely to be medical professionals than computer scientists. In this article, we propose a novel image-centric method for TF generation where instead of complex tools, the user directly manipulates volume data to generate DVR. The user’s work is further simplified by presenting only the most informative volume slices for selection. Based on the selected parts, the voxels are classified using our novel sparse nonparametric support vector machine classifier, which combines both local and near-global distributional information of the training data. The voxel classes are mapped to aesthetically pleasing and distinguishable color and opacity values using harmonic colors. Experimental results on several benchmark datasets and a detailed user survey show the effectiveness of the proposed method.

Regression Guided Deformable Models for Segmentation of Multiple Brain ROIs.

This paper proposes a novel method of using regression-guided deformable models for brain regions of interest (ROIs) segmentation. Different from conventional deformable segmentation, which often deforms shape model locally and thus sensitive to initialization, we propose to learn a regressor to explicitly guide the shape deformation, thus eventually improves the performance of ROI segmentation. The regressor is learned via two steps, (1) a joint classification and regression random forest (CRRF) and (2) an auto-context model. The CRRF predicts each voxel's deformation to the nearest point on the ROI boundary as well as each voxel's class label (e.g., ROI versus background). The auto-context model further refines all voxel's deformations (i.e., deformation field) and class labels (i.e., label maps) by considering the neighboring structures. Compared to the conventional random forest regressor, the proposed regressor provides more accurate deformation field estimation and thus more robust in guiding deformation of the shape model. Validated in segmentation of 14 midbrain ROIs from the IXI dataset, our method outperforms the state-of-art multi-atlas label fusion and classification methods, and also significantly reduces the computation cost.

Intensity Nonuniformity Correction for Brain MR Images with Known Voxel Classes

Intensity nonuniformity in magnetic resonance (MR) images, represented by a smooth and slowly varying function, is a typical artifact that is a nuisance for many image processing methods. To eliminate the artifact, we have to estimate the nonuniformity as a smooth and slowly varying function and factor it out from the given data. We reformulate the problem as a problem of finding a unique smooth function in a particular set of piecewise smooth functions and propose a variational method for finding it. We deliver the main idea using a simple one-dimensional example first and provide a thorough analysis of the problem in a three-phase scenario in three dimensions whose application can be found in the brain MR images. Experiments with synthetic and real MR images and a comparison with a state-of-the-art method, N3, show our algorithm's satisfactory performance in estimating the nonuniformity with and without noise. An automated procedure is also proposed for practical use.

Improving MR brain image segmentation using self-organising maps and entropy-gradient clustering

The primary brain image segmentation goal is to partition a given brain image into different regions representing anatomical structures. Magnetic resonance image (MRI) segmentation is especially interesting because the accurate representation of white matter, grey matter and cerebrospinal fluid provides a way to identify many brain disorders, such as dementia, schizophrenia or Alzheimer’s disease (AD). This paper presents a fully unsupervised method for MRI segmentation based on Self-Organising Maps (SOMs) and Genetic Algorithms (GAs). In particular, the proposed method is based on five stages consisting of image acquisition, feature extraction, feature selection using evolutionary computation, voxel classification using SOM, and sharp map clustering. Moreover, a novel SOM clustering mechanism is presented that defines cluster borders by considering the output space and the relationship with the input space. This clustering mechanism accommodates spatial relationships using an entropy-gradient (EG) function calculation to group the units on the output space (SOM). The complete procedure does not use any a priori knowledge regarding voxel class assignment, but reveals a fully unsupervised, automated method for MRI segmentation to directly identify different tissue classes. Our algorithm was tested using images from the Internet Brain Image Repository (IBSR), outperforming the CGMM method in WM and CSF delineation, and was found to be the most effective among other techniques. Furthermore, this method provided excellent results using high-resolution MR images provided by the Nuclear Medicine Service of the “Virgen de las Nieves” Hospital (Granada, Spain).

Automatic segmentation of cerebral white matter hyperintensities using only 3D FLAIR images

Magnetic Resonance (MR) white matter hyperintensities have been shown to predict an increased risk of developing cognitive decline. However, their actual role in the conversion to dementia is still not fully understood. Automatic segmentation methods can help in the screening and monitoring of Mild Cognitive Impairment patients who take part in large population-based studies. Most existing segmentation approaches use multimodal MR images. However, multiple acquisitions represent a limitation in terms of both patient comfort and computational complexity of the algorithms. In this work, we propose an automatic lesion segmentation method that uses only three-dimensional fluid-attenuation inversion recovery (FLAIR) images. We use a modified context-sensitive Gaussian mixture model to determine voxel class probabilities, followed by correction of FLAIR artifacts. We evaluate the method against the manual segmentation performed by an experienced neuroradiologist and compare the results with other unimodal segmentation approaches. Finally, we apply our method to the segmentation of multiple sclerosis lesions by using a publicly available benchmark dataset. Results show a similar performance to other state-of-the-art multimodal methods, as well as to the human rater.

Unsupervised Neural Techniques Applied to MR Brain Image Segmentation

The primary goal of brain image segmentation is to partition a given brain image into different regions representing anatomical structures. Magnetic resonance image (MRI) segmentation is especially interesting, since accurate segmentation in white matter, grey matter and cerebrospinal fluid provides a way to identify many brain disorders such as dementia, schizophrenia or Alzheimer’s disease (AD). Then, image segmentation results in a very interesting tool for neuroanatomical analyses. In this paper we show three alternatives to MR brain image segmentation algorithms, with the Self-Organizing Map (SOM) as the core of the algorithms. The procedures devised do not use any a priori knowledge about voxel class assignment, and results in fully-unsupervised methods for MRI segmentation, making it possible to automatically discover different tissue classes. Our algorithm has been tested using the images from the Internet Brain Image Repository (IBSR) outperforming existing methods, providing values for the average overlap metric of 0.7 for the white and grey matter and 0.45 for the cerebrospinal fluid. Furthermore, it also provides good results for high-resolution MR images provided by the Nuclear Medicine Service of the “Virgen de las Nieves” Hospital (Granada, Spain).

Multi-Scale hierarchical generation of PET parametric maps: Application and testing on a [11C]DPN study

We propose a general approach to generate parametric maps. It consists in a multi-stage hierarchical scheme where, starting from the kinetic analysis of the whole brain, we then cascade the kinetic information to anatomical systems that are akin in terms of receptor densities, and then down to the voxel level. A-priori classes of voxels are generated either by anatomical atlas segmentation or by functional segmentation using unsupervised clustering. Kinetic properties are transmitted to the voxels in each class using maximum a posteriori (MAP) estimation method. We validate the novel method on a [11C]diprenorphine (DPN) test-retest data-set that represents a challenge to estimation given [11C]DPN's slow equilibration in tissue. The estimated parametric maps of volume of distribution (VT) reflect the opioid receptor distributions known from previous [11C]DPN studies. When priors are derived from the anatomical atlas, there is an excellent agreement and strong correlation among voxel MAP and ROI results and excellent test-retest reliability for all subjects but one. Voxel level results did not change when priors were defined through unsupervised clustering. This new method is fast (i.e. 15 min per subject) and applied to [11C]DPN data achieves accurate quantification of VT as well as high quality VT images. Moreover, the way the priors are defined (i.e. using an anatomical atlas or unsupervised clustering) does not affect the estimates.

Changes in NAA and lactate following ischemic stroke

Although much tissue damage may occur within the first few hours of ischemic stroke, the duration of tissue injury is not well defined. We assessed the temporal pattern of neuronal loss and ischemia after ischemic stroke using magnetic resonance spectroscopic imaging (MRSI) and diffusion-weighted imaging (DWI). We measured N-acetylaspartate (NAA) and lactate in 51 patients with acute ischemic stroke at five time points, from admission to 3 months, in voxels classified as normal, possibly or definitely abnormal (ischemic) according to the appearance of the stroke lesion on the admission DWI. We compared changes in NAA and lactate in different voxel classes using linear mixed models. NAA was significantly reduced from admission in definitely and possibly abnormal (p < 0.01) compared to contralateral normal voxels, reaching a nadir by 2 weeks and remaining reduced at 3 months. Lactate was significantly increased in definitely and possibly abnormal voxels (p < 0.01) during the first 5 days, falling to normal at 2 weeks, rising again later in these voxels. The progressive fall in N-acetylaspartate suggests that some additional neuronal death may continue beyond the first few hours for up to 2 weeks or longer. The mechanism is unclear but, if correct, then it is possible that interventions to limit this ongoing subacute tissue damage might add to the benefit of hyperacute treatment, making further improvements in outcome possible.

Brain Reading Using Full Brain Support Vector Machines for Object Recognition: There Is No “Face” Identification Area

Over the past decade, object recognition work has confounded voxel response detection with potential voxel class identification. Consequently, the claim that there are areas of the brain that are necessary and sufficient for object identification cannot be resolved with existing associative methods (e.g., the general linear model) that are dominant in brain imaging methods. In order to explore this controversy we trained full brain (40,000 voxels) single TR (repetition time) classifiers on data from 10 subjects in two different recognition tasks on the most controversial classes of stimuli (house and face) and show 97.4% median out-of-sample (unseen TRs) generalization. This performance allowed us to reliably and uniquely assay the classifier's voxel diagnosticity in all individual subjects' brains. In this two-class case, there may be specific areas diagnostic for house stimuli (e.g., LO) or for face stimuli (e.g., STS); however, in contrast to the detection results common in this literature, neither the fusiform face area nor parahippocampal place area is shown to be uniquely diagnostic for faces or places, respectively.

An automated procedure for the assessment of white matter hyperintensities by multispectral (T1, T2, PD) MRI and an evaluation of its between-centre reproducibility based on two large community databases

An automated procedure for the detection, quantification, localization and statistical mapping of white matter hyperintensities (WMH) on T2-weighted magnetic resonance (MR) images is presented and validated based on the results of a between-centre reproducibility study. The first step is the identification of white matter (WM) tissue using a multispectral (T1, T2, PD) segmentation. In a second step, WMH are identified within the WM tissue by segmenting T2 images, isolating two different classes of WMH voxels - low- and high-contrast WMH voxels, respectively. The reliability of the whole procedure was assessed by applying it to the analysis of two large MR imaging databases (n = 650 and n= 710, respectively) of healthy elderly subjects matched for demographic characteristics. Average overall WMH load and spatial distribution were found to be similar in the two samples, (1.81 and 1.79% of the WM volume, respectively). White matter hyperintensity load was found to be significantly associated with both age and high blood pressure, with similar effects in both samples. With specific reference to the 650 subject cohort, we also found that WMH load provided by this automated procedure was significantly associated with visual grading of the severity of WMH, as assessed by a trained neurologist. The results show that this method is sensitive, well correlated with semi-quantitative visual rating and highly reproducible.

Statistical Segmentation of Surgical Instruments in 3-D Ultrasound Images

The recent development of real-time 3-D ultrasound (US) enables intracardiac beating-heart procedures, but the distorted appearance of surgical instruments is a major challenge to surgeons. In addition, tissue and instruments have similar gray levels in US images and the interface between instruments and tissue is poorly defined. We present an algorithm that automatically estimates instrument location in intracardiac procedures. Expert-segmented images are used to initialize the statistical distributions of blood, tissue and instruments. Voxels are labeled through an iterative expectation-maximization algorithm using information from the neighboring voxels through a smoothing kernel. Once the three classes of voxels are separated, additional neighboring information is combined with the known shape characteristics of instruments to correct for misclassifications. We analyze the major axis of segmented data through their principal components and refine the results by a watershed transform, which corrects the results at the contact between instrument and tissue. We present results on 3-D in-vitro data from a tank trial and 3-D in-vivo data from cardiac interventions on porcine beating hearts, using instruments of four types of materials. The comparison of algorithm results to expert-annotated images shows the correct segmentation and position of the instrument shaft. (E-mail: lingurarum@mail.nih.gov)

An information theoretic approach for non-rigid image registration using voxel class probabilities

We propose two information theoretic similarity measures that allow to incorporate tissue class information in non-rigid image registration. The first measure assumes that tissue class probabilities have been assigned to each of the images to be registered by prior segmentation of both of them. One image is then non-rigidly deformed to match the other such that the fuzzy overlap of corresponding voxel object labels becomes similar to the ideal case whereby the tissue probability maps of both images are identical. Image similarity is assessed during registration by the divergence between the ideal and actual joint class probability distributions of both images. A second registration measure is proposed that applies in case a segmentation is available for only one of the images, for instance an atlas image that is to be matched to a study image to guide the segmentation thereof. Intensities in one image are matched to the fuzzy class labels in the other image by minimizing the conditional entropy of the intensities in the first image given the class labels in the second image. We derive analytic expressions for the gradient of each measure with respect to individual voxel displacements to derive a force field that drives the registration process, which is regularized by a viscous fluid model. The performance of the class-based measures is evaluated in the context of non-rigid inter-subject registration and atlas-based segmentation of MR brain images and compared with maximization of mutual information using only intensity information. Our results demonstrate that incorporation of class information in the registration measure significantly improves the overlap between corresponding tissue classes after non-rigid matching. The methods proposed here open new perspectives for integrating segmentation and registration in a single process, whereby the output of one is used to guide the other.

An algorithm for automatic segmentation and classification of magnetic resonance brain images.

In this article, we describe the development and validation of an automatic algorithm to segment brain from extracranial tissues, and to classify intracranial tissues as cerebrospinal fluid (CSF), gray matter (GM), white matter (WM) or pathology. T1 weighted spin echo, dual echo fast spin echo (T2 weighted and proton density (PD) weighted images) and fast Fluid Attenuated Inversion Recovery (FLAIR) magnetic resonance (MR) images were acquired in 100 normal patients and 9 multiple sclerosis (MS) patients. One of the normal studies had synthesized MS-like lesions superimposed. This allowed precise measurement of the accuracy of the classification. The 9 MS patients were imaged twice in one week. The algorithm was applied to these data sets to measure reproducibility. The accuracy was measured based on the synthetic lesion images, where the true voxel class was known. Ninety-six percent of normal intradural tissue voxels (GM, WM, and CSF) were labeled correctly, and 94% of pathological tissues were labeled correctly. A low coefficient of variation (COV) was found (mean, 4.1%) for measurement of brain tissues and pathology when comparing MRI scans on the 9 patients. A totally automatic segmentation algorithm has been described which accurately and reproducibly segments and classifies intradural tissues based on both synthetic and actual images.

A voxel visualization and analysis system based on AutoCAD

A collection of AutoLISP programs is presented which enable the visualization and analysis of voxel models by AutoCAD rel. 12/rel. 13. The programs serve as an interactive, graphical front end for manipulating the results of three-dimensional modeling software producing block estimation data. ASCII data files describing geometry and attributes per estimation block are imported and stored as a voxel array. Each voxel may contain multiple attributes, therefore different parameters may be incorporated in one voxel array. Voxel classification is implemented on a layer basis providing flexible treatment of voxel classes such as recoloring, peeling, or volumetry. A versatile clipping tool enables slicing voxel arrays according to combinations of three perpendicular clipping planes. The programs feature an up-to-date, graphical user interface for user-friendly operation by non AutoCAD specialists.