Segmentation Of Cardiac Magnetic Resonance Images Research Articles

From Accuracy to Reliability and Robustness in Cardiac Magnetic Resonance Image Segmentation: A Review

Since the rise of deep learning (DL) in the mid-2010s, cardiac magnetic resonance (CMR) image segmentation has achieved state-of-the-art performance. Despite achieving inter-observer variability in terms of different accuracy performance measures, visual inspections reveal errors in most segmentation results, indicating a lack of reliability and robustness of DL segmentation models, which can be critical if a model was to be deployed into clinical practice. In this work, we aim to bring attention to reliability and robustness, two unmet needs of cardiac image segmentation methods, which are hampering their translation into practice. To this end, we first study the performance accuracy evolution of CMR segmentation, illustrate the improvements brought by DL algorithms and highlight the symptoms of performance stagnation. Afterwards, we provide formal definitions of reliability and robustness. Based on the two definitions, we identify the factors that limit the reliability and robustness of state-of-the-art deep learning CMR segmentation techniques. Finally, we give an overview of the current set of works that focus on improving the reliability and robustness of CMR segmentation, and we categorize them into two families of methods: quality control methods and model improvement techniques. The first category corresponds to simpler strategies that only aim to flag situations where a model may be incurring poor reliability or robustness. The second one, instead, directly tackles the problem by bringing improvements into different aspects of the CMR segmentation model development process. We aim to bring the attention of more researchers towards these emerging trends regarding the development of reliable and robust CMR segmentation frameworks, which can guarantee the safe use of DL in clinical routines and studies.

From Accuracy to Reliability and Robustness in Cardiac Magnetic Resonance Image Segmentation: A Review

From Accuracy to Reliability and Robustness in Cardiac Magnetic Resonance Image Segmentation: A Review

Ensemble of 2D Residual Neural Networks Integrated with Atrous Spatial Pyramid Pooling Module for Myocardium Segmentation of Left Ventricle Cardiac MRI

Cardiac disease diagnosis and identification is problematic mostly by inaccurate segmentation of the cardiac left ventricle (LV). Besides, LV segmentation is challenging since it involves complex and variable cardiac structures in terms of components and the intricacy of time-based crescendos. In addition, full segmentation and quantification of the LV myocardium border is even more challenging because of different shapes and sizes of the myocardium border zone. The foremost purpose of this research is to design a precise automatic segmentation technique employing deep learning models for the myocardium border using cardiac magnetic resonance imaging (MRI). The ASPP module (Atrous Spatial Pyramid Pooling) was integrated with a proposed 2D-residual neural network for segmentation of the myocardium border using a cardiac MRI dataset. Further, the ensemble technique based on a majority voting ensemble method was used to blend the results of recent deep learning models on different set of hyperparameters. The proposed model produced an 85.43% dice score on validation samples and 98.23% on training samples and provided excellent performance compared to recent deep learning models. The myocardium border was successfully segmented across diverse subject slices with different shapes, sizes and contrast using the proposed deep learning ensemble models. The proposed model can be employed for automatic detection and segmentation of the myocardium border for precise quantification of reflow, myocardial infarction, myocarditis, and h cardiomyopathy (HCM) for clinical applications.

Cardiac MRI segmentation with focal loss constrained deep residual networks

Delineating anatomical structures for cardiac magnetic resonance imaging (CMRI) is crucial for various medical applications such as medical diagnoses, treatment, and pathological studies. CMRI segmentation, which aims to automatically and accurately segment the heart structures, is highly beneficial for cardiologists. However, it is non-trivial to perfectly segment the ventricles, especially for the heart apex slices, considering their small sizes compared to the input images. For example, the endocardium in the Sunnybrook dataset only occupies 4% of the entire image by average. During the training process, these target pixels, or other hard samples, are buried by the massive backgrounds that make the model mostly receive optimization signals from easy samples. In this paper, we propose a focal loss constrained residual network (FR-Net) to tackle the problem. In order to mitigate the fact that the gradients of the hard samples can be easily overwhelmed by the easy samples, we use a pixel-wise re-weighting strategy to balance the gradients. Furthermore, considering focal loss constraints for each pixel independently, we propose an alternative training fashion that trains the model with focal loss and dice loss alternatively. The segmentation model can not only benefit from the pixel-wise focal loss but also from the region-wise dice loss to comprehensively optimize the model. We conducted thorough experiments on the Sunnybrook dataset, CMRI dataset, right ventricle dataset, and ACDC dataset to verify the effectiveness of the proposed method.

DBAN: Adversarial Network With Multi-Scale Features for Cardiac MRI Segmentation.

With the development of medical artificial intelligence, automatic magnetic resonance image (MRI) segmentation method is quite desirable. Inspired by the power of deep neural networks, a novel deep adversarial network, dilated block adversarial network (DBAN), is proposed to perform left ventricle, right ventricle, and myocardium segmentation in short-axis cardiac MRI. DBAN contains a segmentor along with a discriminator. In the segmentor, the dilated block (DB) is proposed to capture, and aggregate multi-scale features. The segmentor can produce segmentation probability maps while the discriminator can differentiate the segmentation probability map, and the ground truth at the pixel level. In addition, confidence probability maps generated by the discriminator can guide the segmentor to modify segmentation probability maps. Extensive experiments demonstrate that DBAN has achieved the state-of-the-art performance on the ACDC dataset. Quantitative analyses indicate that cardiac function indices from DBAN are similar to those from clinical experts. Therefore, DBAN can be a potential candidate for short-axis cardiac MRI segmentation in clinical applications.

LCC-Net: A Lightweight Cross-Consistency Network for Semisupervised Cardiac MR Image Segmentation.

Semantic segmentation plays a crucial role in cardiac magnetic resonance (MR) image analysis. Although supervised deep learning methods have made significant performance improvements, they highly rely on a large amount of pixel-wise annotated data, which are often unavailable in clinical practices. Besides, top-performing methods usually have a vast number of parameters, which result in high computation complexity for model training and testing. This study addresses cardiac image segmentation in scenarios where few labeled data are available with a lightweight cross-consistency network named LCC-Net. Specifically, to reduce the risk of overfitting on small labeled datasets, we substitute computationally intensive standard convolutions with a lightweight module. To leverage plenty of unlabeled data, we introduce extreme consistency learning, which enforces equivariant constraints on the predictions of different perturbed versions of the input image. Cutting and mixing different training images, as an extreme perturbation on both the labeled and unlabeled data, are utilized to enhance the robust representation learning. Extensive comparisons demonstrate that the proposed model shows promising performance with high annotation- and computation-efficiency. With only two annotated subjects for model training, the LCC-Net obtains a performance gain of 14.4% in the mean Dice over the baseline U-Net trained from scratch.

Automatic segmentation of the cardiac MR images based on nested fully convolutional dense network with dilated convolution

Cardiac Magnetic Resonance Image (MRI) segmentation plays a helpful role in diagnosing cardiac disease. It is the preliminary step to estimate the functional indices such as ejection fraction (EF) and stroke volume. In this paper, we propose an automatic method for cardiac MRI segmentation based on deep learning. A nested U-shape network with Compressed Dense Blocks (CDBlocks) called BLU-Net is introduced. The Fully Convolutional Dense Network (FCD) is employed as the backbone. Compared with common dense blocks, the CDBlock reduces the connection between the input and the inner layers, and a 1 × 1 convolution is employed to compress the generated feature maps obtained by inner layers. Dilated convolution is employed in the CDBlock to obtain a larger receptive field without losing spatial resolution and reducing the loss of feature information. To learn more semantic information, an additional up-sampling path is adopted, and it makes our model more robust. Our method is evaluated on four cardiac MRI datasets, and the DSC and the HD metrics are employed in the experiment. The experimental results show that BLU-Net outperforms FCD and also outperforms some mainstream networks.

A Multi-Task Network for Cardiac Magnetic Resonance Image Segmentation and Classification

Cardiomyopathy is a group of diseases that affect the heart and can cause serious health problems. Segmentation and classification are important for automating the clinical diagnosis and treatment planning for cardiomyopathy. However, this automation is difficult because of the poor quality of cardiac magnetic resonance (CMR) imaging data and varying dimensions caused by movement of the ventricle. To address these problems, a deep multi-task framework based on a convolutional neural network (CNN) is proposed to segment the left ventricle (LV) myocardium and classify cardiopathy simultaneously. The proposed model consists of a longitudinal encoder–decoder structure that obtains high- and low-level features at the same time. The encoder employs a feature pyramid module (FPM) and dense atrous convolution (DAC) to extract features from images with variable scales for classification. Meanwhile, the decoder leverages the subpixel layer to recover spatial information caused by downsampling in the encoder for segmentation. The approach was verified using 654 magnetic resonance images. It achieved a Dice similarity coefficient (DSC) metric of 82.14% on segmentation and a classification accuracy of 95.72%, with an area under the receiver operating characteristic curve (AUC) of 97.88%. The proposed method can aid in the segmentation of cardiac magnetic images and improve the classification accuracy of cardiopathy.

Cardiac magnetic resonance image segmentation based on convolutional neural network

ObjectiveIn cardiac medical imaging, the extraction and segmentation of the part of interest is the key to the diagnosis of heart disease. Due to irregular diastole and contraction, magnetic resonance imaging (MRI) images have poorly defined boundaries, and traditional segmentation algorithms have poor performance. In this paper, a cardiac MRI segmentation technique using convolutional neural network and image saliency is suggested. MethodsThe convolutional neural network is used for detecting target area, filter out the ribs, muscles and the other parts of the anatomy where the contrast is not clearly defined. It can also be used to extract the region of interest (ROI), and compute the contrast of the ROI in order to improve clarity of the heart tissue within the ROI. The cardiac image diagnosis is performed using the obtained saliency image and compared with the segmentation result of the region growth algorithm. Finally, the images of 85 patients were used to train and test the algorithm model. Here, 46 patients were randomly selected for training, and the remaining 39 were harnessed for further tests. ResultsSegmentation accuracy of our algorithm model in ventricles, septum and the apex of the heart segment reaches 93.14%, 92.58% and 96.21% respectively, which are better than the segmentation method based on the regional growth technique. ConclusionsThe segmentation method using convolutional neural network and image saliency can meet the needs of automatic heart segmentation tasks based on cardiac MRI image sequences. The segmented image is able to assist the doctor to observe the patient's heart health more effectively. As such, our proposed technique has strong potential in clinical applications.

Left ventricle automatic segmentation in cardiac MRI using a combined CNN and U-net approach

Cardiovascular diseases can be effectively prevented from worsening through early diagnosis. To this end, various methods have been proposed to detect the disease source by analyzing cardiac magnetic resonance images (MRI), wherein left ventricular segmentation plays an indispensable role. However, since the left ventricle (LV) is easily confused with other regions in cardiac MRI, segmentation of the LV is a challenging problem. To address this issue, we propose a composite model combining CNN and U-net to accurately segment the LV. In our model, CNN is used to locate the region of interest (ROI) and the U-net network achieve segmentation of LV. We used the cardiac MRI datasets of the MICCAI 2009 left ventricular segmentation challenge to train and test our model and demonstrated the accuracy and robustness of the proposed model. The proposed model achieved state-of-the-art results. The metrics are Dice metric (DM), volumetric overlap error (VOE) and Hausdorff distance (HD), in which DM reaches 0.951, VOE reaches 0.053 and HD reaches 3.641.

Improving cardiac MRI convolutional neural network segmentation on small training datasets and dataset shift: A continuous kernel cut approach.

Cardiac magnetic resonance imaging (MRI) provides a wealth of imaging biomarkers for cardiovascular disease care and segmentation of cardiac structures is required as a first step in enumerating these biomarkers. Deep convolutional neural networks (CNNs) have demonstrated remarkable success in image segmentation but typically require large training datasets and provide suboptimal results that require further improvements. Here, we developed a way to enhance cardiac MRI multi-class segmentation by combining the strengths of CNN and interpretable machine learning algorithms. We developed a continuous kernel cut segmentation algorithm by integrating normalized cuts and continuous regularization in a unified framework. The high-order formulation was solved through upper bound relaxation and a continuous max-flow algorithm in an iterative manner using CNN predictions as inputs. We applied our approach to two representative cardiac MRI datasets across a wide range of cardiovascular pathologies. We comprehensively evaluated the performance of our approach for two CNNs trained with various small numbers of training cases, tested on the same and different datasets. Experimental results showed that our approach improved baseline CNN segmentation by a large margin, reduced CNN segmentation variability substantially, and achieved excellent segmentation accuracy with minimal extra computational cost. These results suggest that our approach provides a way to enhance the applicability of CNN by enabling the use of smaller training datasets and improving the segmentation accuracy and reproducibility for cardiac MRI segmentation in research and clinical patient care.

Inhomogeneity correction and hybrid-based segmentation in cardiac MRI

Image segmentation is an important step in medical image analysis and segmentation of ventricles in Cardiac Magnetic Resonance (CMR) images is challenging due to an in-built artefact called intensity-inhomogeneity. The short-axis cine Magnetic Resonance Images (MRI) recorded under a steady-state free precision protocol were corrected for intensity-inhomogeneity using Bias Corrected Fuzzy C-Means (BCFCM) method, Level Set (LS) and Multiplicative Intrinsic Component Optimisation (MICO) methods. The statistical measures show that bias correction by BCFCM has better performance than MICO and LS. In addition, the original and bias corrected images are validated by Multifractal Analysis (MFA). The results show that in bias corrected images, the low frequency components are removed thereby enhancing the sharpness of the ventricular boundaries. Further, ventricular segmentation is performed using the proposed automatic hybrid Sobel edge detector with optimised level set method. The validation parameters of segmented results show that the ventricular detection in bias corrected images matches better with ground truth.

Automatic initialization and quality control of large-scale cardiac MRI segmentations.

Continuous advances in imaging technologies enable ever more comprehensive phenotyping of human anatomy and physiology. Concomitant reduction of imaging costs has resulted in widespread use of imaging in large clinical trials and population imaging studies. Magnetic Resonance Imaging (MRI), in particular, offers one-stop-shop multidimensional biomarkers of cardiovascular physiology and pathology. A wide range of analysis methods offer sophisticated cardiac image assessment and quantification for clinical and research studies. However, most methods have only been evaluated on relatively small databases often not accessible for open and fair benchmarking. Consequently, published performance indices are not directly comparable across studies and their translation and scalability to large clinical trials or population imaging cohorts is uncertain. Most existing techniques still rely on considerable manual intervention for the initialization and quality control of the segmentation process, becoming prohibitive when dealing with thousands of images. The contributions of this paper are three-fold. First, we propose a fully automatic method for initializing cardiac MRI segmentation, by using image features and random forests regression to predict an initial position of the heart and key anatomical landmarks in an MRI volume. In processing a full imaging database, the technique predicts the optimal corrective displacements and positions in relation to the initial rough intersections of the long and short axis images. Second, we introduce for the first time a quality control measure capable of identifying incorrect cardiac segmentations with no visual assessment. The method uses statistical, pattern and fractal descriptors in a random forest classifier to detect failures to be corrected or removed from subsequent statistical analysis. Finally, we validate these new techniques within a full pipeline for cardiac segmentation applicable to large-scale cardiac MRI databases. The results obtained based on over 1200 cases from the Cardiac Atlas Project show the promise of fully automatic initialization and quality control for population studies.

Active contour model and nonlinear shape priors with application to left ventricle segmentation in cardiac MR images

This paper presents a new active contour model for left ventricle segmentation in cardiac magnetic resonance (MR) images. In this study, the shape prior is coupled with intensity information in the proposed energy functional which includes a data term and a shape term. The data term, inspired from a region-based active contour model, is used to guide the motion of the initial curve to desired object boundaries. Meanwhile, the shape term is utilized to constrain the evolving contour with respect to the reference shape, which helps the model deal with images in the presence of background clutter and object occlusion. Especially, in the paper, to reconstruct the shape prior, we utilize the kernel principal component analysis (KPCA) that allows the obtained prior shape to be faithful to the shape of the desired object. In addition, the pose variations between the shapes are handled by employing shape normalization procedure instead of solving a set of Euler–Lagrange equations as in conventional approaches. The proposed model is first presented as a two-phase level set formulation and then extended to a multi-phase formulation. Comparative experiments on a set of cardiac MR images show the advantages of the proposed model.

A geodesic-active-contour-based variational model for short-axis cardiac MR image segmentation

We propose a novel geodesic-active-contour-based (GAC-based) variational model that uses two level-set functions to segment the right and left ventricles and the epicardium in short-axis magnetic resonance (MR) images. For the right ventricle, the myocardial wall is typically very thin and hard to identify using the resolution of existing MR scanners. We propose to use two level sets to identify both the endocardial wall by pushing away one level-set function from another, in the setting of the edge-driven GAC model with a new edge detection function. Existing edge detection functions have strict restrictions on the location of initial contours. We develop a new edge detection function that relaxes this restriction and propose an iterative method that uses a sequence of edge detection functions to minimize the energy of our model successively. Experimental results are presented to validate the effectiveness of the proposed model.

Automatic Left Ventricle Segmentation in Cardiac MRI Using Topological Stable-State Thresholding and Region Restricted Dynamic Programming

Segmentation of the left ventricle (LV) is very important in the assessment of cardiac functional parameters. The aim of this study is to develop a novel and robust algorithm which can improve the accuracy of automatic LV segmentation on short-axis cardiac magnetic resonance images (MRI). The database used in this study consists of 45 cases obtained from the Sunnybrook Health Sciences Centre. The 45 cases contain 12 ischemic heart failures, 12 non-ischemic heart failures, 12 LV hypertrophies, and 9 normal cases. Three key techniques are developed in this segmentation algorithm: 1) topological stable-state thresholding method is proposed to refine the endocardial contour, 2) an edge map with non-maxima gradient suppression approach, and 3) a region-restricted technique that is proposed to improve the dynamic programming to derive the epicardial boundary. The validation experiments were performed on a pool of data sets of 45 cases. For both endo- and epicardial contours of our results, percentage of good contours is about 91%, the average perpendicular distance is about 2 mm, and the overlapping dice metric is about 0.91. The regression and determination coefficient for the experts and our proposed method on the ejection fraction is 1.05 and 0.9048, respectively; they are 0.98 and 0.8221 for LV mass. An automatic method using topological stable-state thresholding and region restricted dynamic programming has been proposed to segment left ventricle in short-axis cardiac MRI. Evaluation results indicate that the proposed segmentation method can improve the accuracy and robust of left ventricle segmentation. The proposed segmentation approach shows the better performance and has great potential in improving the accuracy of computer-aided diagnosis systems in cardiovascular diseases.

Multiview Active Appearance Models for Simultaneous Segmentation of Cardiac 2- and 4-Chamber Long-Axis Magnetic Resonance Images

Long-axis cardiac magnetic resonance (MR) views enable a rapid, online evaluation of cardiac function from only 2 views. In this article, we aimed to evaluate a model-based method for the simultaneous detection of 2- and 4-chamber endocardial and epicardial contours in end-diastolic and end-systolic phases of MR images. We introduce multiview Active Appearance Models for the automated segmentation of long-axis cardiac MR images of the left ventricle. Two modes of initialization were used to test the accuracy of the model with minimal user interaction and the best-obtainable accuracy with this model. The segmentation was initialized by annotating 2 points in the base and one in the apex. We tested the method's performance by comparing the point-to-curve errors, ejection fractions, and biplane area-length volumes calculated with the automatically detected contours to those calculated from contours that were annotated manually by experts. Leave-one-out experiments were performed with 2- and 4-chamber long axis MR images of 59 subjects in end-diastolic and end-systolic phases. When initializing in all 4 frames, 97% of the segmentations were successful, and the standard deviation in the volume-errors with respect to the average manually identified volume was 9.0% for the end-diastolic volumes and 15% for the end-systolic volumes. When the method was initialized in the end-systolic frames only, 92% of the segmentations were successful, and the standard deviation in the errors in the volumes with respect to the average manually identified volume was 13.3% for the end-diastolic volumes and 16.7% for the end-systolic volumes. Bland-Altman plots showed that the errors were distributed randomly around 0, and by using a paired t test comparing manual and computer-determined volumes, we were able to detect that the volume differences were not significant. Simultaneous detection of the endocardial and epicardial contours in 2- and 4-chamber views and end-diastolic and end-systolic phases for one subject takes approximately 3 seconds. The accuracy of the reported method is comparable with the interobserver variability for initialization in all frames and slightly worse than the interobserver variability with initialization in the end-systolic frames only. However, in both cases the errors were not significant. Initialization in end-systolic frames only leads to a statistically insignificantly lower model accuracy; however, it requires only half the user interaction. Therefore, we can conclude that this method enables rapid analysis of the cardiac left ventricular function with little user interaction.

3-D active appearance models: segmentation of cardiac MR and ultrasound images

A model-based method for three-dimensional image segmentation was developed and its performance assessed in segmentation of volumetric cardiac magnetic resonance (MR) images and echocardiographic temporal image sequences. Comprehensive design of a three-dimensional (3-D) active appearance model (AAM) is reported for the first time as an involved extension of the AAM framework introduced by Cootes et al. The model's behavior is learned from manually traced segmentation examples during an automated training stage. Information about shape and image appearance of the cardiac structures is contained in a single model. This ensures a spatially and/or temporally consistent segmentation of three-dimensional cardiac images. The clinical potential of the 3-D AAM is demonstrated in short-axis cardiac MR images and four-chamber echocardiographic sequences. The method's performance was assessed by comparison with manually identified independent standards in 56 clinical MR and 64 clinical echo image sequences. The AAM method showed good agreement with the independent standard using quantitative indexes of border positioning errors, endo- and epicardial volumes, and left ventricular mass. In MR, the endocardial volumes, epicardial volumes, and left ventricular wall mass correlation coefficients between manual and AAM were R2 = 0.94, 0.97, 0.82, respectively. For echocardiographic analysis, the area correlation was R2 = 0.79. The AAM method shows high promise for successful application to MR and echocardiographic image analysis in a clinical setting.