Real Ultrasound Images Research Articles

An accurate paradigm for denoising degraded ultrasound images based on artificial intelligence systems.

Ultrasound images are susceptible to various forms of quality degradation that negatively impact diagnosis. Common degradations include speckle noise, Gaussian noise, salt and pepper noise, and blurring. This research proposes an accurate ultrasound image denoising strategy based on firstly detecting the noise type, then, suitable denoising methods can be applied for each corruption. The technique depends on convolutional neural networks to categorize the type of noise affecting an input ultrasound image. Pre-trained convolutional neural network models including GoogleNet, VGG-19, AlexNet and AlexNet-support vector machine (SVM) are developed and trained to perform this classification. A dataset of 782 numerically generated ultrasound images across different diseases and noise types is utilized for model training and evaluation. Results show AlexNet-SVM achieves the highest accuracy of 99.2% in classifying noise types. The results indicate that, the present technique is considered one of the top-performing models is then applied to real ultrasound images with different noise corruptions to demonstrate efficacy of the proposed detect-then-denoise system. RESEARCH HIGHLIGHTS: Proposes an accurate ultrasound image denoising strategy based on detecting noise type first. Uses pre-trained convolutional neural networks to categorize noise type in input images. Evaluates GoogleNet, VGG-19, AlexNet, and AlexNet-support vector machine (SVM) models on a dataset of 782 synthetic ultrasound images. AlexNet-SVM achieves highest accuracy of 99.2% in classifying noise types. Demonstrates efficacy of the proposed detect-then-denoise system on real ultrasound images.

Ultrasound-guided needle tracking with deep learning: A novel approach with photoacoustic ground truth

Accurate needle guidance is crucial for safe and effective clinical diagnosis and treatment procedures. Conventional ultrasound (US)-guided needle insertion often encounters challenges in consistency and precisely visualizing the needle, necessitating the development of reliable methods to track the needle. As a powerful tool in image processing, deep learning has shown promise for enhancing needle visibility in US images, although its dependence on manual annotation or simulated data as ground truth can lead to potential bias or difficulties in generalizing to real US images. Photoacoustic (PA) imaging has demonstrated its capability for high-contrast needle visualization. In this study, we explore the potential of PA imaging as a reliable ground truth for deep learning network training without the need for expert annotation. Our network (UIU-Net), trained on ex vivo tissue image datasets, has shown remarkable precision in localizing needles within US images. The evaluation of needle segmentation performance extends across previously unseen ex vivo data and in vivo human data (collected from an open-source data repository). Specifically, for human data, the Modified Hausdorff Distance (MHD) value stands at approximately 3.73, and the targeting error value is around 2.03, indicating the strong similarity and small needle orientation deviation between the predicted needle and actual needle location. A key advantage of our method is its applicability beyond US images captured from specific imaging systems, extending to images from other US imaging systems.

Sequential Active Contour Based on Morphological-Driven Thresholding for Ultrasound Image Segmentation of Ascites.

Paracentesis is a high-demanding and routine operation, which has great potentials and benefits if semi-autonomous procedures can be developed. One of the most important techniques that facilitate semi-autonomous paracentesis is to segment the ascites from ultrasound images accurately and efficiently. The ascites, however, is usually with significantly different shapes and noise among different patients, and its shape/size changes dynamically during the paracentesis. This makes most of existing image segmentation methods either time consuming or inaccurate for segmenting ascites from its background. In this article, we propose a two-stage active contour method to facilitate accurate and efficient segmentation of ascites. First, a morphological-driven thresholding method is developed to locate the initial contour of the ascites automatically. Then, the identified initial contour is fed into a novel sequential active contour algorithm to segment the ascites from background accurately. The proposed method is tested and compared with state-of-the-art active contour methods on over 100 real ultrasound images of ascites, and the results show the superiority of our method in both accuracy and time efficiency.

Advanced framework for enhancing ultrasound images through an optimized hybrid search algorithm and a novel motion compounding processing chain

Ultrasound imaging is a fast, widespread, and essential diagnostic technique for examining the body’s internal anatomy to find abnormal tissues or diseases. However, speckle noise in ultrasound imaging corrupts fine details and edges and degrades the image’s resolution and contrast, making diagnosing more difficult. In this work, a speckle reduction method using motion compounding is proposed. The objective of this work is ultrasound image enhancement keeping all the diagnostics details and edges. The proposed method uses the pre-locations frames that the sonographer generates before locating the required diagnostic frame. These frames are applied to the proposed optimized Modified Three-Step Search (O-MTSS) algorithm to enhance the final processed frame. The O-MTSS algorithm is a hybrid between the Three Step Search algorithm (TSS) and the New Diamond Search Algorithm (NDS). The method requires a scoring layer for storing additional frames to select optimal frames that will be used for enhancement. The proposed methodology is tested on synthetic and real ultrasound images. The outcomes produce considerable speckle reduction while maintaining the image edges with good computational time for real-time scanning. The result is evaluated by subjective physicians, radiologists, and image evaluation metrics. According to the percentages of the subjective analysis, there is a remarkable improvement in the processed image. The proposed algorithm result is compared with existing algorithms; It is observed that the improvement in terms of signal to noise ratio, peak signal to noise ratio, mean square error, root mean square error and structural similarity index values of the proposed method are 4.6%, 3.32%, 12.02%, 6.2% and 1.57% respectively over Non-Local Low-Rank (NLLR) method. According to qualitative and quantitative analysis, the suggested method outperforms existing speckle reduction techniques regarding edge and fine detail preservation.

Ultrasound renal stone diagnosis based on convolutional neural network and VGG16 features

This paper deals with the classification of the kidneys for renal stones on ultrasound images. Convolutional neural network (CNN) and pre-trained CNN (VGG16) models are used to extract features from ultrasound images. Extreme gradient boosting (XGBoost) classifiers and random forests are used for classification. The features extracted from CNN and VGG16 are used to compare the performance of XGBoost and random forest. An image with normal and renal stones was classified. This work uses 630 real ultrasound images from Al-Diwaniyah General Teaching Hospital (a lithotripsy center) in Iraq. Classifier performance is evaluated using its accuracy, recall, and F1 score. With an accuracy of 99.47%, CNN-XGBoost is the most accurate model.

Performance evaluation of 67 denoising filters in ultrasound images: A systematic comparison analysis

AbstractNoise corrupts ultrasound images and degrades spatial and contrast resolutions. Hence, it is challenging to characterize the lesions precisely using ultrasound images. The present study aims to evaluate 67 denoising filters and select the best one for ultrasound image denoising. Seven test images were synthesized to evaluate the performance of filters at three different noise levels. Eleven full‐reference quantitative image quality metrics (IQMs) were employed to evaluate the performance of the filters. A new filter evaluation method, Rank Analysis, was introduced and utilized at each noise level. The ten best filters with the smallest mean rank in all noise levels were defined for further analysis on real ultrasound images. The Rank Analysis was also employed for real ultrasound images, and filters were evaluated based on 14 IQMs (11 full‐reference and three no‐reference). Finally, the best filter was defined using the repeated measures analysis statistical test. According to the Rank Analysis results, the Spatial correlation (SCorr) filter obtained the best results with the mean rank scores±SD of 1 ± 0, which was significantly better than the other nine filters (p < 0.001). The second‐best results were achieved by three filters, Bitonic, most homogeneous neighborhood, and Lee diffusion (p < 0.05). We concluded that SCorr is the best filter for ultrasound image denoising. It can be used in the pre‐processing step before segmentation and diagnostic procedures. In addition, a new filter evaluation method, Rank Analysis, was introduced in this study, which is easy to use, fast, and provides reliable results. So, it can be used to evaluate newly developed filters in the future studies.

Style transfer-enabled Sim2Real framework for efficient learning of robotic transesophageal echocardiography using simulation

Recently, various machine learning techniques have been proposed to realize automatic ultrasound (US) image analysis for robotic US acquisition tasks. However, obtaining large amounts of real US images for training is usually expensive or even infeasible in some clinical applications, such as transesophageal echocardiography (TEE). An alternative is to build a simulator to generate synthetic US data for training, but the differences between simulated and real US images may result in poor model performance. This work presents a novel Sim2Real framework to learn robotic US image analysis based on simulated data to address the challenges of limited real US data for robotic TEE. A style transfer model is proposed based on unsupervised contrastive learning to convert real US images into the simulation style. Moreover, a task-relevant model is designed to combine CNNs with vision transformers and trained only on the simulated data to generate task-dependent predictions. We demonstrate the effectiveness of our method in an image regression task to predict the probe position based on US images in robotic TEE. Our results show that using only simulated US data and a small amount of unlabelled real data for training, our method can achieve comparable performance to semi-supervised and fully supervised learning methods. Moreover, the effectiveness of our previously proposed CT-based US image simulation method is also indirectly confirmed.

US-Net: A lightweight network for simultaneous speckle suppression and texture enhancement in ultrasound images

Background:Numerous traditional filtering approaches and deep learning-based methods have been proposed to improve the quality of ultrasound (US) image data. However, their results tend to suffer from over-smoothing and loss of texture and fine details. Moreover, they perform poorly on images with different degradation levels and mainly focus on speckle reduction, even though texture and fine detail enhancement are of crucial importance in clinical diagnosis. Methods:We propose an end-to-end framework termed US-Net for simultaneous speckle suppression and texture enhancement in US images. The architecture of US-Net is inspired by U-Net, whereby a feature refinement attention block (FRAB) is introduced to enable an effective learning of multi-level and multi-contextual representative features. Specifically, FRAB aims to emphasize high-frequency image information, which helps boost the restoration and preservation of fine-grained and textural details. Furthermore, our proposed US-Net is trained essentially with real US image data, whereby real US images embedded with simulated multi-level speckle noise are used as an auxiliary training set. Results:Extensive quantitative and qualitative experiments indicate that although trained with only one US image data type, our proposed US-Net is capable of restoring images acquired from different body parts and scanning settings with different degradation levels, while exhibiting favorable performance against state-of-the-art image enhancement approaches. Furthermore, utilizing our proposed US-Net as a pre-processing stage for COVID-19 diagnosis results in a gain of 3.6% in diagnostic accuracy. Conclusions:The proposed framework can help improve the accuracy of ultrasound diagnosis.

A New Technique Based on AMID Using Adaptive Thresholding for Ultrasound Speckle Reduction

A new technique based on adaptive multi-resolution image decomposition (AMID) using adaptive thresholding for ultrasound speckle reduction has been proposed in this paper. The noisy image is first filtered to reduce large multiplicative noise. The image is then log-transformed before decomposing it to its approximation and detailed coefficients using AMID algorithm. The approximation coefficients are appropriately filtered while adaptive thresholding is applied to the detail coefficients. The image is reconstructed back from all the modified coefficients to obtain the denoised image. This method combining adaptive decomposition with adaptive thresholding makes it adaptive to both signal and noise components, thereby retaining the edge details in the denoised image effectively. The proposed technique performance is tested on synthetic as well as real ultrasound images with the noise of different variances. The experimental results show that the proposed algorithm gives better performance than the other state-of-the-art methods in terms of edge keeping index (EKI), correlation coefficient (CC), figure of merit (FOM), peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and signal-to-noise ratio (SNR) for synthetic images. The algorithm gives better performance in terms of equivalent number of looks (ENL) and mean-to-variance ratio (MVR) for real ultrasound images.

Sketch guided and progressive growing GAN for realistic and editable ultrasound image synthesis.

Ultrasound (US) imaging is widely used for anatomical structure inspection in clinical diagnosis. The training of new sonographers and deep learning based algorithms for US image analysis usually requires a large amount of data. However, obtaining and labeling large-scale US imaging data are not easy tasks, especially for diseases with low incidence. Realistic US image synthesis can alleviate this problem to a great extent. In this paper, we propose a generative adversarial network (GAN) based image synthesis framework. Our main contributions include: (1) we present the first work that can synthesize realistic B-mode US images with high-resolution and customized texture editing features; (2) to enhance structural details of generated images, we propose to introduce auxiliary sketch guidance into a conditional GAN. We superpose the edge sketch onto the object mask and use the composite mask as the network input; (3) to generate high-resolution US images, we adopt a progressive training strategy to gradually generate high-resolution images from low-resolution images. In addition, a feature loss is proposed to minimize the difference of high-level features between the generated and real images, which further improves the quality of generated images; (4) the proposed US image synthesis method is quite universal and can also be generalized to the US images of other anatomical structures besides the three ones tested in our study (lung, hip joint, and ovary); (5) extensive experiments on three large US image datasets are conducted to validate our method. Ablation studies, customized texture editing, user studies, and segmentation tests demonstrate promising results of our method in synthesizing realistic US images.

Ultrasound Speckle Reduction Using Wavelet-Based Generative Adversarial Network.

The visual quality of ultrasound (US) images is crucial for clinical diagnosis and treatment. The main source of image quality degradation is the inherent speckle noise generated during US image acquisition. Current deep learning-based methods cannot preserve the maximum boundary contrast when removing noise and speckle. In this paper, we address the issue by proposing a novel wavelet-based generative adversarial network (GAN) for real-time high-quality US image reconstruction, viz. WGAN-DUS. First, we propose a batch normalization module (BNM) to balance the importance of each sub-band image and fuse sub-band features simultaneously. Then, a wavelet reconstruction module (WRM) integrated with a cascade of wavelet residual channel attention block (WRCAB) is proposed to extract distinctive sub-band features used to reconstruct denoised images. A gradual tuning strategy is proposed to fine-tune our generator for better despeckling performance. We further propose a wavelet-based discriminator and a comprehensive loss function to effectively suppress speckle noise and preserve the image features. Besides, we have designed an algorithm to estimate the noise levels during despeckling of real US images. The performance of our network was then evaluated on natural, synthetic, simulated and clinical US images and compared against various despeckling methods. To verify the feasibility of WGAN-DUS, we further extend our work to uterine fibroid segmentation with the denoised US image of the proposed approach. Experimental result demonstrates that our proposed method is feasible and can be generalized to clinical applications for despeckling of US images in real-time without losing its fine details.

GANs for generation of synthetic ultrasound images from small datasets

Abstract The task of medical image classification is increasingly supported by algorithms. Deep learning methods like convolutional neural networks (CNNs) show superior performance in medical image analysis but need a high-quality training dataset with a large number of annotated samples. Particularly in the medical domain, the availability of such datasets is rare due to data privacy or the lack of data sharing practices among institutes. Generative adversarial networks (GANs) are able to generate high quality synthetic images. This work investigates the capabilities of different state-of-the-art GAN architectures in generating realistic breast ultrasound images if only a small amount of training data is available. In a second step, these synthetic images are used to augment the real ultrasound image dataset utilized for training CNNs. The training of both GANs and CNNs is conducted with systematically reduced dataset sizes. The GAN architectures are capable of generating realistic ultrasound images. GANs using data augmentation techniques outperform the baseline Style- GAN2 with respect to the Frechet Inception distance by up to 64.2%. CNN models trained with additional synthetic data outperform the baseline CNN model using only real data for training by up to 15.3% with respect to the F1 score, especially for datasets containing less than 100 images. As a conclusion, GANs can successfully be used to generate synthetic ultrasound images of high quality and diversity, improve classification performance of CNNs and thus provide a benefit to computer-aided diagnostics.

BEMD Based Cross Bilateral Filtering Technique for Speckle Reduction in Ultrasound Images

In this paper, Bidimensional Empirical Mode Decomposition (BEMD) based Cross Bilateral Filter (CBF) technique for speckle reduction in ultrasound images has been proposed. The reference image is obtained by denoising the noisy image using pixel-wise Wiener filtering. Then, both the noisy image and the reference image are decomposed into a set of Intrinsic Mode Functions (IMFs) and the residue image using BEMD technique. CBF is applied between noisy image IMFs and the corresponding reference image IMFs. The image is reconstructed back with these modified IMFs and the residue. The proposed method exploits the edge information in the reference image for improving the quality of the denoised image. The performance of the proposed method has been tested for real ultrasound images and simulated images having noise of different variance. The experimental results show that the proposed algorithm performs better than other state-of-art methods in terms of Edge Keeping Index (EKI), Correlation Coefficient (CC), Figure of Merit (FOM), Structural Similarity (SSIM), Peak Signal to Noise Ratio (PSNR) and Signal to Noise Ratio (SNR) for synthetic images. The algorithm gives better performance for real ultrasound images in terms of Mean to Variance Ratio (MVR) and Equivalent Number of Looks (ENL).

Cost-Sensitive Broad Learning System for Imbalanced Classification and Its Medical Application

As an effective and efficient discriminative learning method, the broad learning system (BLS) has received increasing attention due to its outstanding performance without large computational resources. The standard BLS is derived under the minimum mean square error (MMSE) criterion, while MMSE is with poor performance when dealing with imbalanced data. However, imbalanced data are widely encountered in real-world applications. To address this issue, a novel cost-sensitive BLS algorithm (CS-BLS) is proposed. In the CS-BLS, many variations can be adopted, and CS-BLS with weighted cross-entropy is analyzed in this paper. Weighted penalty factors are used in CS-BLS to constrain the contribution of each sample in different classes. The samples in minor classes are allocated higher weights to increase their contributions. Four different weight calculation methods are adopted to the CS-BLS, and thus, four CS-BLS methods are proposed: Log-CS-BLS, Lin-CS-BLS, Sqr-CS-BLS, and EN-CS-BLS. Experiments based on artificially imbalanced datasets of MNIST and small NORB are firstly conducted and compared with the standard BLS. The results show that the proposed CS-BLS methods have better generalization and robustness than the standard BLS. Then, experiments on a real ultrasound breast image dataset are conducted, and the results demonstrate that the proposed CS-BLS methods are effective in actual medical diagnosis.

CT2US: Cross-modal transfer learning for kidney segmentation in ultrasound images with synthesized data

Accurate segmentation of kidney in ultrasound images is a vital procedure in clinical diagnosis and interventional operation. In recent years, deep learning technology has demonstrated promising prospects in medical image analysis. However, due to the inherent problems of ultrasound images, data with annotations are scarce and arduous to acquire, hampering the application of data-hungry deep learning methods. In this paper, we propose cross-modal transfer learning from computerized tomography (CT) to ultrasound (US) by leveraging annotated data in the CT modality. In particular, we adopt cycle generative adversarial network (CycleGAN) to synthesize US images from CT data and construct a transition dataset to mitigate the immense domain discrepancy between US and CT. Mainstream convolutional neural networks such as U-Net, U-Res, PSPNet, and DeepLab v3+ are pretrained on the transition dataset and then transferred to real US images. We first trained CNN models on a data set composed of 50 ultrasound images and validated them on a validation set composed of 30 ultrasound images. In addition, we selected 82 ultrasound images from another hospital to construct a cross-site data set to verify the generalization performance of the models. The experimental results show that with our proposed transfer learning strategy, the segmentation accuracy in dice similarity coefficient (DSC) reaches 0.853 for U-Net, 0.850 for U-Res, 0.826 for PSPNet and 0.827 for DeepLab v3+ on the cross-site test set. Compared with training from scratch, the accuracy improvement was 0.127, 0.097, 0.105 and 0.036 respectively. Our transfer learning strategy effectively improves the accuracy and generalization ability of ultrasound image segmentation model with limited training data.

A fuzzy edge detector driven telegraph total variation model for image despeckling

<p style='text-indent:20px;'>Speckle noise suppression is a challenging and crucial pre-processing stage for higher-level image analysis. In this work, a new attempt has been made using telegraph total variation equation and fuzzy set theory for image despeckling. The intuitionistic fuzzy divergence function has been used to distinguish between edges and noise. To the best of the authors' knowledge, most of the studies on the multiplicative speckle noise removal process focus only on diffusion-based filters, and little attention has been paid to the study of fuzzy set theory. The proposed approach enjoys the benefits of both telegraph total variation equation and fuzzy edge detector, which is robust to noise and preserves image structural details. Moreover, we establish the existence and uniqueness of weak solutions of a regularized version of the present system using the Schauder fixed point theorem. With the proposed technique, despeckling is carried out on natural, real synthetic aperture radar, and real ultrasound images. The experimental results computed by the suggested method are reported, which are found better in terms of noise elimination and detail/edge preservation, concerning the existing approaches.</p>

A Weibull‐distribution‐based hybrid total variation method for speckle reduction in ultrasound images

Speckle reduction is still an intractable task in ultrasound imaging field. Ultrasound speckle is usually described as multiplicative noise with its statistics following a Rayleigh or Gaussian distribution. To employ these two distributions effectively, the authors attempt to describe ultrasound speckle using a Weibull distribution, because it can include the Rayleigh distribution as a special case and also approximate a Gaussian distribution by varying its shape and scale parameters. The authors’ contribution in this paper is to propose a Weibull-distribution-based hybrid total variation (WHTV) method to reduce ultrasound speckle. The WHTV energy functional is convex and consists of a new data fidelity term and a new regularization term. The former is derived from the multiplicative Weibull model of ultrasound speckle based on the maximum likelihood criterion. The latter is a new edge-weighted combination of the first- and second-order total variation, with the advantage of preserving edges while alleviating the staircase effects. The minimization of the WHTV energy functional is implemented by the split Bregman algorithm. Experimental results on synthetic and real ultrasound images have demonstrated not only that the Weibull distribution is a better fitting model for the statistics of ultrasound speckle than other distributions such as Rayleigh, Gaussian, Gamma, and Nakagami, but also that the proposed WHTV method can achieve better despeckling performance than several state-of-the-art variational methods.

Restoration and enhancement of breast ultrasound images using extended complex diffusion based unsharp masking.

Ultrasound is a well-known imaging modality for the interpretation of breast cancer. It is playing very important role for breast cancer detection that are missed by mammograms. The image acquisition is usually affected by the presence of noise, artifacts, and distortion. To overcome such type of issues, there is a need of image restoration and enhancement to improve the quality of image. This paper proposes a single framework for denoising and enhancement of ultrasound images, where a smoothing filter is replaced with an extended complex diffusion-based filter in an unsharp masking technique. The performance evaluation of the proposed method is tested on real ultrasound breast cancer images database and synthetic ultrasound image. The performance evaluation comprises qualitative and quantitative evaluation along with comparative analysis of pre-existing and proposed method. The quantitative evaluation metrics are mean squared error, peak-signal-to-noise ratio, correlation parameter, normalized absolute error, universal quality index, similarity structure index, edge preservation index, a measure of enhancement, a measure of enhancement by entropy, and second derivative like measurement. The result specifies that the proposed method is better suited approach for the removal of speckle noise which follows Rayleigh distribution, restoration of information, enhancement of abnormalities, and proper edge preservation.

BEMD Based Ultrasound Image Speckle Reduction Technique Using Pixel-Wise Wiener Filtering

In this paper, an improved Bidimensional Empirical Mode Decomposition (BEMD) based speckle reduction technique for ultrasound images has been proposed. The noisy image has been decomposed into its Intrinsic Mode Functions (IMFs) and a~residue. The noise component of the low order IMFs is removed with the pixel-wise Wiener filtering. The image is reconstructed with these filtered low order IMFs, high order IMFs and the residue. The performance of the proposed method has been tested on synthetic as well as real ultrasound images having noise components of different variance. The experimental results show that the proposed algorithm performs better than other existing methods for synthetic images as well as real ultrasound images in terms of various image quality matrices.

An efficient transfer learning-based Super-Resolution model for Medical Ultrasound Image

There are many imaging modalities for clinical diagnosis, but we heavily use ultrasound imaging; some of the ultrasound images are low resolution because of the body’s internal weakness. This paper presents a novel unsupervised super-resolution framework for ultrasound image enhancement without considering any large sets of training samples which is the major concern in many single image super-resolution techniques. Here, the most powerful nonlinear mapping is introduced within convolutional neural networks to restore potentially useful and prominent spatial information generated from the model sets considered for image enhancements. Here, two techniques are used: dilated convolution and residual learning to increase the convergence and reconstruction accuracy in terms of quality measures. The potential metrics of dilated convolution in extract spatial information and residual learning are used to narrow the convergence time by learning only the difference between the test input and distorted input image. By evaluating the proposed framework on real ultrasound image sets, the performance metrics are validated. The proposed model outperformed the other competitive image enhancement models in the USSR field.