Segmentation Of Echocardiographic Images Research Articles

Automatic segmentation of echocardiographic images using a shifted windows vision transformer architecture

Echocardiography is one the most commonly used imaging modalities for the diagnosis of congenital heart disease. Echocardiographic image analysis is crucial to obtaining accurate cardiac anatomy information. Semantic segmentation models can be used to precisely delimit the borders of the left ventricle, and allow an accurate and automatic identification of the region of interest, which can be extremely useful for cardiologists. In the field of computer vision, convolutional neural network (CNN) architectures remain dominant. Existing CNN approaches have proved highly efficient for the segmentation of various medical images over the past decade. However, these solutions usually struggle to capture long-range dependencies, especially when it comes to images with objects of different scales and complex structures. In this study, we present an efficient method for semantic segmentation of echocardiographic images that overcomes these challenges by leveraging the self-attention mechanism of the Transformer architecture. The proposed solution extracts long-range dependencies and efficiently processes objects at different scales, improving performance in a variety of tasks. We introduce Shifted Windows Transformer models (Swin Transformers), which encode both the content of anatomical structures and the relationship between them. Our solution combines the Swin Transformer and U-Net architectures, producing a U-shaped variant. The validation of the proposed method is performed with the EchoNet-Dynamic dataset used to train our model. The results show an accuracy of 0.97, a Dice coefficient of 0.87, and an Intersection over union (IoU) of 0.78. Swin Transformer models are promising for semantically segmenting echocardiographic images and may help assist cardiologists in automatically analyzing and measuring complex echocardiographic images.

Semi-Supervised Gan for medical image segmentation

Echocardiography is a popular ultrasound imaging method used for the diagnosis of heart conditions. With the advent of numerous image processing algorithms, echocardiographic image segmentation has become more significant. This is a crucial stage since it offers a framework for evaluating numerous cardiac parameters, including LV volume and heart wall, valve motion, ejection fraction, thickness, etc. All these factors are crucial for determining a heart's health. The task of manual segmentation requires skilled operators and takes a lot of time. By requiring the discriminator network to output class labels, we extend Generative Adversarial Networks to the semi-supervised type. This paper examines image segmentation techniques for echocardiography to find the borders of the left ventricle. In this paper, we introduce a new convolution neural network model for the auto-segmentation of the left ventricle in echo images. The division of a picture into regions is known as image segmentation. Segments, that computer vision can use to automatically understand. This method makes it easier to simultaneously evaluate and diagnose echo pictures. The segmentation of echocardiographic images can be utilized to measure cardiac characteristics like heart wall thickness.

An efficient annotated data generation method for echocardiographic image segmentation

Background:In recent years, deep learning techniques have demonstrated promising performances in echocardiography (echo) data segmentation, which constitutes a critical step in the diagnosis and prognosis of cardiovascular diseases (CVDs). However, their successful implementation requires large number and high-quality annotated samples, whose acquisition is arduous and expertise-demanding. To this end, this study aims at circumventing the tedious, time-consuming and expertise-demanding data annotation involved in deep learning-based echo data segmentation. Methods:We propose a two-phase framework for fast generation of annotated echo data needed for implementing intelligent cardiac structure segmentation systems. First, multi-size and multi-orientation cardiac structures are simulated leveraging polynomial fitting method. Second, the obtained cardiac structures are embedded onto curated endoscopic ultrasound images using Fourier Transform algorithm, resulting in pairs of annotated samples. The practical significance of the proposed framework is validated through using the generated realistic annotated images as auxiliary dataset to pretrain deep learning models for automatic segmentation of left ventricle and left ventricle wall in real echo data, respectively. Results:Extensive experimental analyses indicate that compared with training from scratch, fine-tuning after pretraining with the generated dataset always results in significant performance improvement whereby the improvement margins in terms of Dice and IoU can reach 12.9% and 7.74%, respectively. Conclusion:The proposed framework has great potential to overcome the shortage of labeled data hampering the deployment of deep learning approaches in echo data analysis.

Echocardiographic Image Segmentation for Diagnosing Fetal Cardiac Rhabdomyoma During Pregnancy Using Deep Learning

Automated interpretation of cardiac images has the potential to change clinical practice in many ways. For example, it could make it possible for non-experts in primary care and rural settings to test the heart’s function over time. In this paper, we tested the research hypothesis that recent developments in computer vision would make it possible to create a fully automated, scalable analysis for echocardiogram interpretation, covering all of the steps from view identification and Medical Image Segmentation (MIS) to structure and function quantification and Fetal Cardiac RhabDomyomas (FCRD) detection. Even though they are rare, FCRDs are the most frequent cause of Fetal Cardiac Tumor (FCT). When it comes to diagnosing and monitoring fetuses with an injured circulatory system, imaging (particularly echocardiography (ECG)) has proven helpful in the field of fetal cardiology. Because of the severe lack of qualified and experienced sonographers, it is very challenging to diagnose Cardiac RhabDomyomas (CRD). Prior to delivery, accurate segmentation of the FC to identify structural cardiac defects is critical for minimizing the illness among newborns. To automate the process of segmenting the cardiac chamber for the CRD, we propose a novel Attention-Residual Network-based V-Net architecture (ARVNet). In this study, examinations were performed on Fetal Rhabdomyomas noted in the Right Ventricle (FRRV), Fetal Rhabdomyomas noted in the Left Ventricle (FRLV), Fetal Rhabdomyomas noted in the Right Atrium (FRRA), Fetal Rhabdomyomas noted in the Left Atrium (FRLA), Fetal Rhabdomyomas noted in the Tricuspid Valve (FRTV). Images without Rhabdomyoma mean “Normal Condition (NC)” at Selvam Hospital in Melapalayam, Tirunelveli, Tamil Nadu, India. Even with a relatively small number of datasets, the proposed technique possesses high CRD detection performance, as evidenced by the results. The results showed that the proposed model did a good job segmenting all the views, with a specificity of 99.7% and a Dice coefficient similarity of 99.8%. It also did well at finding CRDs, with an average mean accuracy of around 99.85%.

Echocardiographic image multi-structure segmentation using Cardiac-SegNet.

Cardiac boundary segmentation of echocardiographic images is important for cardiac function assessment and disease diagnosis. However, it is challenging to segment cardiac ventricles due to the low contrast-to-noise ratio and speckle noise of the echocardiographic images. Manual segmentation is subject to interobserver variability and is too slow for real-time image-guided interventions. We aim to develop a deep learning-based method for automated multi-structure segmentation of echocardiographic images. We developed an anchor-free mask convolutional neural network (CNN), termed Cardiac-SegNet, which consists of three subnetworks, that is, a backbone, a fully convolutional one-state object detector (FCOS) head, and a mask head. The backbone extracts multi-level and multi-scale features from endocardium image. The FOCS head utilizes these features to detect and label the region-of-interests (ROIs) of the segmentation targets. Unlike the traditional mask regional CNN (Mask R-CNN) method, the FCOS head is anchor-free and can model the spatial relationship of the targets. The mask head utilizes a spatial attention strategy, which allows the network to highlight salient features to perform segmentation on each detected ROI. For evaluation, we investigated 450 patient datasets by a five-fold cross-validation and a hold-out test. The endocardium (LVEndo ) and epicardium (LVEpi ) of the left ventricle and left atrium (LA) were segmented and compared with manual contours using the Dice similarity coefficient (DSC), Hausdorff distance (HD), mean absolute distance (MAD), and center-of-mass distance (CMD). Compared to U-Net and Mask R-CNN, our method achieved higher segmentation accuracy and fewer erroneous speckles. When our method was evaluated on a separate hold-out dataset at the end diastole (ED) and the end systole (ES) phases, the average DSC were 0.952 and 0.939 at ED and ES for the LVEndo , 0.965 and 0.959 at ED and ES for the LVEpi , and 0.924 and 0.926 at ED and ES for the LA. For patients with a typical image size of 549×788 pixels, the proposed method can perform the segmentation within 0.5s. We proposed a fast and accurate method to segment echocardiographic images using an anchor-free mask CNN.

Site-specific automated contouring model generalisibiliy enhancement

Abstract Funding Acknowledgements Type of funding sources: None. Background Segmentation of cardiac structures in echocardiography is a pre-requisite for accurately assessing cardiac morphology and function. Manual or semi-automated segmentation are both routinely used in clinical practice, although these can be time-consuming, and can introduce high inter- and intra- operator variability resulting in decreased reproducibility. Effective contouring with no manual input has proven to be challenging due to variations in image quality, image noise, motion during the acquisition and the lack of a well-defined geometry. Methods This work proposes a coordinate regression method for automated left ventricle (LV) segmentation, presented in Figure 1 (a). The proposed method is based on a modified U-net architecture that outputs the likelihood of coordinates of landmark points. The obtained likelihood heatmaps are converted to 2D coordinates using a differentiable spatial to numerical transform. The model was trained and validated on UK multisite data (1383 subjects) comprising apical 2 and 4 chamber views for both contrast and non-contrast echocardiographic images. The Cardiac Acquisitions for Multi-structure Ultrasound Segmentation (CAMUS) echocardiographic image segmentation database was used to assess the performance of the proposed method acting as data from a new clinical site. The CAMUS dataset comprises apical 2 and 4 chamber views acquired from 500 patients with manually annotated cardiac structures for end-diastole and end-systole frames. The original CAMUS dataset was split into training (450 patients) and testing (50 patients), with manual contours being available only for the training dataset. Therefore, we used the CAMUS training dataset to both test and improve our model, by using a random sample of 100 studies as an independent testing dataset and the remaining 350 studies were used for retraining the initial model to improve performance for this dataset. Results The results obtained on the testing images are presented in Figure 1 (b). When the model was trained using no CAMUS data for the LV segmentation, a mean Dice coefficient of 0.890 and a median of 0.911 was obtained. Including 350 studies with the original 1383 UK dataset and retraining the same model improved the average Dice coefficient to 0.930 and the median to 0.939. The CAMUS dataset authors reported the best average Dice coefficient of 0.924 on the 50 CAMUS testing images, therefore the proposed points regression method introduces a promising alternative to mask-based segmentation models. Conclusions In conclusion, the auto-contouring framework has proven to be effective in terms of its performance and ability to generalise to new data. Furthermore, this work highlights the importance of both evaluating model performance on data from new clinical sites and also enhancing model performance. Abstract Figure.

ResDUnet: A Deep Learning-Based Left Ventricle Segmentation Method for Echocardiography

Segmentation of echocardiographic images is an essential step for assessing the cardiac functionality and providing indicative clinical measures, and all further heart analysis relies on the accuracy of this process. However, the fuzzy nature of echocardiographic images degraded by distortion and speckle noise poses some challenges on the manual segmentation task. In this paper, we propose a fully automated left ventricle segmentation method that can overcome those challenges. Our method performs accurate delineation for the ventricle boundaries despite the ill-defined borders and shape variability of the left ventricle. The well-known deep learning segmentation model, known as the U-net, has addressed some of these challenges with outstanding performance. However, it still ignores the contribution of all semantic information through the segmentation process. Here we propose a novel deep learning segmentation method based on U-net, named ResDUnet. It incorporates feature extraction at different scales through the integration of cascaded dilated convolution. To ease the training process, residual blocks are deployed instead of the basic U-net blocks. Each residual block is enriched with a squeeze and excitation unit for channel-wise attention and adaptive feature re-calibration. The performance of the method is evaluated on a dataset of 2000 images acquired from 500 patients with large variability in quality and patient pathology. ResDUnet outperforms state-of-the-art methods with a Dice similarity increase of 8.4% and 1.2% compared to deeplabv3 and U-net, respectively. Furthermore, to demonstrate the impact of each proposed sub-module, several experiments have been carried out with different designs and variations of the integrated sub-modules. We also describe and discuss all technical elements of a deep-learning model via a step-by-step explanation of parameters and methods, while using our left ventricle segmentation as a case study, to explain the application of AI to echocardiographic imaging.

Local maximum likelihood segmentation of echocardiographic images with Rayleigh distribution

In order to interpret ultrasound images, it is important to understand their formation and the properties that affect them, especially speckle noise. This image texture, or speckle, is a correlated and multiplicative noise that inherently occurs in all types of coherent imaging systems. Indeed, its statistics depend on the density and on the type of scatterers in the tissues. This paper presents a new method for echocardiographic images segmentation in a variational level set framework. A partial differential equation-based flow is designed locally in order to achieve a maximum likelihood segmentation of the region of interest. A Rayleigh probability distribution is considered to model the local B-mode ultrasound images intensities. In order to confront more the speckle noise and local changes of intensity, the proposed local region term is combined with a local phase-based geodesic active contours term. Comparison results on natural and simulated images show that the proposed model is robust to attenuations and captures well the low-contrast boundaries.

PSnakes: A new radial active contour model and its application in the segmentation of the left ventricle from echocardiographic images

Active contours are image segmentation methods that minimize the total energy of the contour to be segmented. Among the active contour methods, the radial methods have lower computational complexity and can be applied in real time. This work aims to present a new radial active contour technique, called pSnakes, using the 1D Hilbert transform as external energy. The pSnakes method is based on the fact that the beams in ultrasound equipment diverge from a single point of the probe, thus enabling the use of polar coordinates in the segmentation. The control points or nodes of the active contour are obtained in pairs and are called twin nodes. The internal energies as well as the external one, Hilbertian energy, are redefined. The results showed that pSnakes can be used in image segmentation of short-axis echocardiogram images and that they were effective in image segmentation of the left ventricle. The echo-cardiologist's golden standard showed that the pSnakes was the best method when compared with other methods. The main contributions of this work are the use of pSnakes and Hilbertian energy, as the external energy, in image segmentation. The Hilbertian energy is calculated by the 1D Hilbert transform. Compared with traditional methods, the pSnakes method is more suitable for ultrasound images because it is not affected by variations in image contrast, such as noise. The experimental results obtained by the left ventricle segmentation of echocardiographic images demonstrated the advantages of the proposed model. The results presented in this paper are justified due to an improved performance of the Hilbert energy in the presence of speckle noise.

A Novel Approach to Medical Image Segmentation

Problem statement: Segmentation is a vital aspect of medical imaging. It aids in the visualization of medical data and diagnostics of various diseases. Ultrasound image segmentation, in particular echocardiographic image segmentation, is required to identify the regions of interest such as Left Ventricle (LV) and other cardiac cavities. Existing methods do not address the drawbacks of speed and quality of segmentation. A faster method is required for effective, accurate and scalable clinical analysis and diagnosis. Approach: In this research, a novel approach is used to segment the 2D echo images of various views. A modified K-Means clustering algorithm, called “Fast SQL K-Means” is proposed using the power of SQL in DBMS environment. In K-Means, Euclidean distance computation is the most time consuming process. However, here it computed with a single database table and no joins. This method takes less than 10 sec to cluster an image size of 400×250 (100K pixels), whereas the running time of direct K-Means is around 900 sec. Since the entire processing is done with database, additional overhead of import and export of data is not required. The 2D echo images are acquired from the local Cardiology Hospital for conducting the experiments. Results: The proposed algorithm was tested by considering a number of echo images in apical four chamber, long-axis and short axis views. We have compared the direct K-Means implementation with the proposed algorithm by varying the data size from 10-100K and found that the results outperformed compared to the results of other authors. The pattern of the data and the number of clusters had almost no impact on the clustering time. Conclusion: An efficient and non-traditional model for echo image segmentation is presented by using the SQL. Fast algorithms are required for immediate analysis of echo images within ICUs, remote places, telemedicine. The challenge is that ultrasound images are prone to speckle noise, segmented echo images carry gaps in the cardiac regions which in turn causes difficulties in boundary tracing and selection of seed values for the K-Means. Future research can enhance the speed by partitioning the database tables and use of parallel SQL statements.

Phase Symmetry Approach Applied to Children Heart Chambers Segmentation: A Comparative Study

Segmentation of echocardiographic images presents a great challenge because these images contain strong speckle noise and artifacts. Besides, most ultrasound segmentation methods are semi-automatic, requiring initial contour to be manually identified in the images. In this work, we propose an algorithm based on the phase symmetry approach and level set evolution, in order to extract simultaneously all heart cavities in a fully automatic way. The level set evolution uses a new logarithmic based stopping function, which demonstrates to perform well in the boundary extraction. We compared our method with other level set approaches, the watershed technique, and the manual segmentation made by two physicians. The experimental work was based on echocardiography images of children. Similarity metrics, namely Pratt Function, Pixel Mean Error, and Similarity Angle have been used for the performance evaluation of the different methods. The results indicate that our method has a performance at least 4% superior to the other methods able to segment the four chambers. Even for the two worst boundary extraction cases (right ventricle and left atrium) the performance of the proposed method still is better than the other techniques.

Decision support in heart failure through processing of electro- and echocardiograms

Signal and imaging investigations are currently key components in the diagnosis, prognosis and follow up of heart diseases. Nowadays, the need for more efficient, cost-effective and personalised care has led to a renaissance of clinical decision support systems (CDSSs). The purpose of this paper is to present an effective way of achieving a high-level integration of signal and image processing methods in the general process of care, by means of a clinical decision support system, and to discuss the advantages of such an approach. From the wide range of heart diseases, heart failure, whose complexity best highlights the benefits of this integration, has been selected. After an analysis of users' needs and expectations, significant and suitably designed image and signal processing algorithms are introduced to objectively and reliably evaluate important features involved in decisional problems in the heart failure domain. Then, a CDSS is conceived so as to combine the domain knowledge with advanced analytical tools for data processing. In particular, the relevant and significant medical knowledge and experts' knowhow are formalised according to an ontological formalism, suitably augmented with a base of rules for inferential reasoning. The proposed methods were tested and evaluated in the daily practice of the physicians operating at the Department of Cardiology, University Magna Graecia, Catanzaro, Italy, on a population of 79 patients. Different scenarios, involving decisional problems based on the analysis of biomedical signals and images, were considered. In these scenarios, after some training and 3 months of use, the CDSS was able to provide important and useful suggestions in routine workflows, by integrating the clinical parameters computed through the developed methods for echocardiographic image segmentation and the algorithms for electrocardiography processing. The CDSS allows the integration of signal and image processing algorithms into the general process of care. Feedback from end-users has been positive.

Endocardial boundary extraction in left ventricular echocardiographic images using fast and adaptive B-spline snake algorithm

A fast and robust algorithm was developed for automatic segmentation of the left ventricular endocardial boundary in echocardiographic images. The method was applied to calculate left ventricular volume and ejection fraction estimation. A fast adaptive B-spline snake algorithm that resolves the computational concerns of conventional active contours and avoids computationally expensive optimizations was developed. A combination of external forces, adaptive node insertion, and multiresolution strategy was incorporated in the proposed algorithm. Boundary extraction with area and volume estimation in left ventricular echocardiographic images was implemented using the B-spline snake algorithm. The method was implemented in MATLAB and 50 medical images were used to evaluate the algorithm performance. Experimental validation was done using a database of echocardiographic images that had been manually evaluated by experts. Comparison of methods demonstrates significant improvement over conventional algorithms using the adaptive B-spline technique. Moreover, our method reached a reasonable agreement with the results obtained manually by experts. The accuracy of boundary detection was calculated with Dice's coefficient equation (91.13%), and the average computational time was 1.24 s in a PC implementation. In sum, the proposed method achieves satisfactory results with low computational complexity. This algorithm provides a robust and feasible technique for echocardiographic image segmentation. Suggestions for future improvements of the method are provided.

Modeling Envelope Statistics of Blood and Myocardium for Segmentation of Echocardiographic Images

The objective of this study was to investigate the use of speckle statistics as a preprocessing step for segmentation of the myocardium in echocardiographic images. Three-dimensional (3D) and biplane image sequences of the left ventricle of two healthy children and one dog (beagle) were acquired. Pixel-based speckle statistics of manually segmented blood and myocardial regions were investigated by fitting various probability density functions (pdf). The statistics of heart muscle and blood could both be optimally modeled by a K-pdf or Gamma-pdf (Kolmogorov-Smirnov goodness-of-fit test). Scale and shape parameters of both distributions could differentiate between blood and myocardium. Local estimation of these parameters was used to obtain parametric images, where window size was related to speckle size (5 × 2 speckles). Moment-based and maximum-likelihood estimators were used. Scale parameters were still able to differentiate blood from myocardium; however, smoothing of edges of anatomical structures occurred. Estimation of the shape parameter required a larger window size, leading to unacceptable blurring. Using these parameters as an input for segmentation resulted in unreliable segmentation. Adaptive mean squares filtering was then introduced using the moment-based scale parameter (σ 2/μ) of the Gamma-pdf to automatically steer the two-dimensional (2D) local filtering process. This method adequately preserved sharpness of the edges. In conclusion, a trade-off between preservation of sharpness of edges and goodness-of-fit when estimating local shape and scale parameters is evident for parametric images. For this reason, adaptive filtering outperforms parametric imaging for the segmentation of echocardiographic images. (E-mail: m.m.nillesen@cukz.umcn.nl)

Computer Modeling Technology to Assess Extracapsular Tissue Coverage of Whole Mount Sections After Retropubic and Laparoscopic Radical Prostatectomy

The introduction of new surgical approaches to radical prostatectomy requires methodologies that permit valid comparison that are more expedient than long-term outcomes of biochemical local and distant failure and survival. We used a computer modeling program to assess the percent of extracapsular tissue coverage of prostate glands removed by the open retropubic and laparoscopic approaches. Specimens were available for 15 and 17 patients who underwent open and laparoscopic radical prostatectomy, respectively. Serial whole mount sections were taken at 5 mm intervals. A genitourinary pathologist drew the contours of the prostate capsule on each tissue section. The whole mount was scanned to produce digital images. A software program was used to create a file with capsule information and a file with extraprostatic fibroadipose tissue information. Two separate point cloud files were generated to represent the capsule and extraprostatic models, and software algorithms were used to generate differences in the point clouds to quantify the extent of extracapsular tissue coverage. When separated into sides dissected by a nerve or nonnerve sparing technique, the overall percent of gland surface coverage by extracapsular fibroadipose tissue was statistically greater with laparoscopic dissection than with the open approach. When a segmental analysis of gland coverage was evaluated, a statistically greater percent of fibroadipose coverage was associated with laparoscopic dissection in the apical and inferolateral segments with nonnerve sparing, and in the apical segment with nerve sparing. This small radical prostatectomy series, analyzed by computer reconstruction as described, provides information suggesting that overall extracapsular tissue coverage is at least equal if not superior using the laparoscopic vs the open approach. This was specifically the case in areas of inferolateral and apical dissection with nonnerve sparing procedures and in areas of the apical dissection with nerve sparing procedures.

Combinative multi-scale level set framework for echocardiographic image segmentation

In the automatic segmentation of echocardiographic images, a priori shape knowledge has been used to compensate for poor features in ultrasound images. This shape knowledge is often learned via an off-line training process, which requires tedious human effort and is highly expertise-dependent. More importantly, a learned shape template can only be used to segment a specific class of images with similar boundary shape. In this paper, we present a multi-scale level set framework for segmentation of endocardial boundaries at each frame in a multiframe echocardiographic image sequence. We point out that the intensity distribution of an ultrasound image at a very coarse scale can be approximately modeled by Gaussian. Then we combine region homogeneity and edge features in a level set approach to extract boundaries automatically at this coarse scale. At finer scale levels, these coarse boundaries are used to both initialize boundary detection and serve as an external constraint to guide contour evolution. This constraint functions similar to a traditional shape prior. Experimental results validate this combinative framework.