Residual Encoder-Decoder Research Articles

Residual encoder-decoder based architecture for medical image denoising

AbstractHigh-resolution computed tomography (CT) scans require high doses of X-rays, posing potential health risks to patients, including genetic damage and cancer. Conversely, low doses of X-rays result in noise and artifacts in the reconstructed CT scans. Consequently, the problem of denoising low-dose CT (LDCT) images has become a critical yet challenging issue in the field of CT imaging. However, existing deep learning-based LDCT image denoising methods frequently result in the loss of high-frequency features, such as edges and textures, due to the use of mean squared error loss. To address this issue, we propose a method based on high-frequency feature learning to enhance the denoising performance of existing models. Our method is designed to simultaneously learn the primary task of LDCT image denoising and the auxiliary task of LDCT edge detection, thereby improving the denoising performance without increasing the number of model parameters and the inference time. Our method significantly improves the denoising performance of the RED-CNN model, achieving competitive results compared to state-of-the-art denoising models on the AAPM and Qin-LUNG-CT datasets.

Innovative Noise Extraction and Denoising in Low-Dose CT Using a Supervised Deep Learning Framework

Low-dose computed tomography (LDCT) imaging is a critical tool in medical diagnostics due to its reduced radiation exposure. However, this reduction often results in increased noise levels, compromising image quality and diagnostic accuracy. Despite advancements in denoising techniques, a robust method that effectively balances noise reduction and detail preservation remains a significant need. Current denoising algorithms frequently fail to maintain the necessary balance between suppressing noise and preserving crucial diagnostic details. Addressing this gap, our study focuses on developing a deep learning-based denoising algorithm that enhances LDCT image quality without losing essential diagnostic information. Here we present a novel supervised learning-based LDCT denoising algorithm that employs innovative noise extraction and denoising techniques. Our method significantly enhances LDCT image quality by incorporating multiple attention mechanisms within a U-Net-like architecture. Our approach includes a noise extraction network designed to capture diverse noise patterns precisely. This network is integrated into a comprehensive denoising system consisting of a generator network, a discriminator network, and a feature extraction AutoEncoder network. The generator network removes noise and produces high-quality CT images, while the discriminator network differentiates real images from denoised ones, improving the realism of the outputs. The AutoEncoder network ensures the preservation of image details and diagnostic integrity. Our method improves the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) by 7.777 and 0.128 compared to LDCT, by 0.483 and 0.064 compared to residual encoder–decoder convolutional neural network (RED-CNN), by 4.101 and 0.017 compared to Wasserstein generative adversarial network–visual geometry group (WGAN-VGG), and by 3.895 and 0.011 compared to Wasserstein generative adversarial network–autoencoder (WGAN-AE). This demonstrates that our method has a significant advantage in enhancing the signal-to-noise ratio of images. Extensive experiments on multiple standard datasets demonstrate our method’s superior performance in noise suppression and image quality enhancement compared to existing techniques. Our findings significantly impact medical imaging, particularly improving LDCT scan diagnostic accuracy. The enhanced image clarity and detail preservation offered by our method open new avenues for clinical applications and research. This improvement in LDCT image quality promises substantial contributions to clinical diagnostics, disease detection, and treatment planning, ensuring high-quality diagnostic outcomes while minimizing patient radiation exposure.

High-quality AFM image acquisition of living cells by modified residual encoder-decoder network

Atomic force microscope enables ultra-precision imaging of living cells. However, atomic force microscope imaging is a complex and time-consuming process. The obtained images of living cells usually have low resolution and are easily influenced by noise leading to unsatisfactory imaging quality, obstructing the research and analysis based on cell images. Herein, an adaptive attention image reconstruction network based on residual encoder-decoder was proposed, through the combination of deep learning technology and atomic force microscope imaging supporting high-quality cell image acquisition. Compared with other learning-based methods, the proposed network showed higher peak signal-to-noise ratio, higher structural similarity and better image reconstruction performances. In addition, the cell images reconstructed by each method were used for cell recognition, and the cell images reconstructed by the proposed network had the highest cell recognition rate. The proposed network has brought insights into the atomic force microscope-based imaging of living cells and cell image reconstruction, which is of great significance in biological and medical research.

DON6D: a decoupled one-stage network for 6D pose estimation

The six-dimensional (6D) pose object estimation is a key task in robotic manipulation and grasping scenes. Many existing two-stage solutions with a slow inference speed require extra refinement to handle the challenges of variations in lighting, sensor noise, object occlusion, and truncation. To address these challenges, this work proposes a decoupled one-stage network (DON6D) model for 6D pose estimation that improves inference speed on the premise of maintaining accuracy. Particularly, since the RGB images are aligned with the RGB-D images, the proposed DON6D first uses a two-dimensional detection network to locate the interested objects in RGB-D images. Then, a module of feature extraction and fusion is used to extract color and geometric features fully. Further, dual data augmentation is performed to enhance the generalization ability of the proposed model. Finally, the features are fused, and an attention residual encoder–decoder, which can improve the pose estimation performance to obtain an accurate 6D pose, is introduced. The proposed DON6D model is evaluated on the LINEMOD and YCB-Video datasets. The results demonstrate that the proposed DON6D is superior to several state-of-the-art methods regarding the ADD(-S) and ADD(-S) AUC metrics.

Structure-preserving low-dose computed tomography image denoising using a deep residual adaptive global context attention network.

Low-dose computed tomography (LDCT) scans can effectively reduce the radiation damage to patients, but this is highly detrimental to CT image quality. Deep convolutional neural networks (CNNs) have shown their potential in improving LDCT image quality. However, the conventional CNN-based approaches rely fundamentally on the convolution operations, which are ineffective for modeling the correlations among nonlocal similar structures and the regionally distinct statistical properties in CT images. This modeling deficiency hampers the denoising performance for CT images derived in this manner. In this paper, we propose an adaptive global context (AGC) modeling scheme to describe the nonlocal correlations and the regionally distinct statistics in CT images with negligible computation load. We further propose an AGC-based long-short residual encoder-decoder (AGC-LSRED) network for efficient LDCT image noise artifact-suppression tasks. Specifically, stacks of residual AGC attention blocks (RAGCBs) with long and short skip connections are constructed in the AGC-LSRED network, which allows valuable structural and positional information to be bypassed through these identity-based skip connections and thus eases the training of the deep denoising network. For training the AGC-LSRED network, we propose a compound loss that combines the L1 loss, adversarial loss, and self-supervised multi-scale perceptual loss. Quantitative and qualitative experimental studies were performed to verify and validate the effectiveness of the proposed method. The simulation experiments demonstrated the proposed method exhibits the best result in terms of noise suppression [root-mean-square error (RMSE) =9.02; peak signal-to-noise ratio (PSNR) =33.17] and fine structure preservation [structural similarity index (SSIM) =0.925] compared with other competitive CNN-based methods. The experiments on real data illustrated that the proposed method has advantages over other methods in terms of radiologists' subjective assessment scores (averaged scores =4.34). With the use of the AGC modeling scheme to characterize the structural information in CT images and of residual AGC-attention blocks with long and short skip connections to ease the network training, the proposed AGC-LSRED method achieves satisfactory results in preserving fine anatomical structures and suppressing noise in LDCT images.

Adaptive multi-scale TF-net for high-resolution time–frequency representations

A novel adaptive multi-scale time–frequency network (AMTFN) is proposed to provide high-resolution time–frequency representations for nonstationary signals. AMTFN is an end-to-end deep network, which firstly adaptively learns the comprehensive basis functions to produce time–frequency (TF) feature maps through multi-scale 1D convolutional kernels. Then, the channel attention mechanism is embedded into AMTFN to rescale the TF feature maps selectively. Thus, the subsequent residual encoder–decoder block’s energy concentration performance is greatly improved with these rescaled TF feature maps. Besides, this paper designs a new training strategy to elegantly enable the model to pay more attention to the intersections of instantaneous frequency trajectories. In the end, a series of simulations as well as real-world cases, are studied to demonstrate the effectiveness and advantages of the proposed method.

Learned spatiotemporal correlation priors for CEST image denoising using incorporated global-spectral convolution neural network.

To develop a deep learning-based method, dubbed Denoising CEST Network (DECENT), to fully exploit the spatiotemporal correlation prior to CEST image denoising. DECENT is composed of two parallel pathways with different convolution kernel sizes aiming to extract the global and spectral features embedded in CEST images. Each pathway consists of a modified U-Net with residual Encoder-Decoder network and 3D convolution. Fusion pathway with 1 × 1 × 1 convolution kernel is utilized to concatenate two parallel pathways, and the output of DECENT is noise-reduced CEST images. The performance of DECENT was validated in numerical simulations, egg white phantom experiments, and ischemic mouse brain and human skeletal muscle experiments in comparison with existing state-of-the-art denoising methods. Rician noise was added to CEST images to mimic a low SNR situation for numerical simulation, egg white phantom experiment, and mouse brain experiments, while human skeletal muscle experiments were of inherently low SNR. From the denoising results evaluated by peak SNR (PSNR) and structural similarity index (SSIM), the proposed deep learning-based denoising method (DECENT) can achieve better performance compared to existing CEST denoising methods such as NLmCED, MLSVD, and BM4D, avoiding complicated parameter tuning or time-consuming iterative processes. DECENT can well exploit the prior spatiotemporal correlation knowledge of CEST images and restore the noise-free images from their noisy observations, outperforming state-of-the-art denoising methods.

Full-Resolution Lung Nodule Localization From Chest X-Ray Images Using Residual Encoder-Decoder Networks

Full-Resolution Lung Nodule Localization From Chest X-Ray Images Using Residual Encoder-Decoder Networks

Bandwidth Improvement in Ultrasound Image Reconstruction Using Deep Learning Techniques.

Ultrasound (US) imaging is a medical imaging modality that uses the reflection of sound in the range of 2-18 MHz to image internal body structures. In US, the frequency bandwidth (BW) is directly associated with image resolution. BW is a property of the transducer and more bandwidth comes at a higher cost. Thus, methods that can transform strongly bandlimited ultrasound data into broadband data are essential. In this work, we propose a deep learning (DL) technique to improve the image quality for a given bandwidth by learning features provided by broadband data of the same field of view. Therefore, the performance of several DL architectures and conventional state-of-the-art techniques for image quality improvement and artifact removal have been compared on in vitro US datasets. Two training losses have been utilized on three different architectures: a super resolution convolutional neural network (SRCNN), U-Net, and a residual encoder decoder network (REDNet) architecture. The models have been trained to transform low-bandwidth image reconstructions to high-bandwidth image reconstructions, to reduce the artifacts, and make the reconstructions visually more attractive. Experiments were performed for 20%, 40%, and 60% fractional bandwidth on the original images and showed that the improvements obtained are as high as 45.5% in RMSE, and 3.85 dB in PSNR, in datasets with a 20% bandwidth limitation.

Noise Suppression of Computed Tomography (CT) Images Using Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)

In this study, an in-house residual encoder-decoder convolutional neural network (RED-CNN)-based algorithm was composed and trained using images of cylindrical polymethyl-methacrylate (PMMA) phantom with a diameter of 26 cm at different simulated noise levels. The model was tested on 21 × 26 cm elliptical PMMA computed tomography (CT) phantom images with simulated noise to evaluate its denoising capability using signal to noise ratio (SNR), comparative peak signal-to-noise ratio (cPSNR), structural similarity (SSIM) index, modulation transfer function frequencies (MTF 10 %) and noise power spectra (NPS) values as parameters. Evaluation of a possible decrease of image quality was also performed by testing the model using homogenous water phantom and wire phantom images acquired using different mAs values. Results show that the model was able to consistently increase SNR, cPSNR, SSIM values, and decrease the integral noise power spectra (NPS). However, the noise level on either training or testing data affects the model’s final denoising performance. The lower noise level on testing data images tends to result in over-smoothed images, as indicated by the shift of the NPS curves. In contrast, higher simulated noise level tends to result in less satisfactory denoising performance, as indicated by lower SNR, cPSNR, and SSIM values. Meanwhile, the higher noise level on training data images tends to produce denoised images with reduced sharpness, as indicated by the decrease of the MTF 10 % values. Further studies are required to better understand the character of RED-CNN for CT noise suppression regarding the optimum parameters for best results.

Deep Cascade Residual Networks (DCRNs): Optimizing an Encoder–Decoder Convolutional Neural Network for Low-Dose CT Imaging

To suppress noise and artifacts caused by the reduced radiation exposure in low-dose computed tomography, several deep learning (DL)-based image restoration methods have been proposed over the past few years. Many of these popular DL-based methods adopt an encoder–decoder framework, for instance, the residual encoder–decoder convolutional neural network. However, this popular framework may suffer from information loss for continual downsampling operations. In this article, deep cascaded residual networks (DCRNs) are proposed to optimize the popular encoder–decoder network. First, cross up- and downsampling operations as well as attention extraction are substitutes for the strict “downsampling and then upping” principle. What is more, four hybrid loss functions, namely, mean absolute error, edge loss, perceptual loss and adversarial loss, are engaged to achieve better visual effects and suppress noise. The experiments are conducted on three individual clinical CT datasets: dental CT data collected with a scanner manufactured by Zhongke Tianyue Company (ZTC), data from the American Association of Physicists in Medicine (AAPM) Challenge, and data collected with a commercial CT scanner from United Imaging Healthcare (UIH). The experimental results indicate the effective noise reduction and detail preservation capabilities of the proposed methods under different radiation dose-reduction strategies.

Deep Shape-from-Template: Single-image quasi-isometric deformable registration and reconstruction

Shape-from-Template (SfT) solves 3D vision from a single image and a deformable 3D object model, called a template. Concretely, SfT computes registration (the correspondence between the template and the image) and reconstruction (the depth in camera frame). It constrains the object deformation to quasi-isometry. Real-time and automatic SfT represents an open problem for complex objects and imaging conditions. We present four contributions to address core unmet challenges to realise SfT with a Deep Neural Network (DNN). First, we propose a novel DNN called DeepSfT, which encodes the template in its weights and hence copes with highly complex templates. Second, we propose a semi-supervised training procedure to exploit real data. This is a practical solution to overcome the render gap that occurs when training only with simulated data. Third, we propose a geometry adaptation module to deal with different cameras at training and inference. Fourth, we combine statistical learning with physics-based reasoning. DeepSfT runs automatically and in real-time and we show with numerous experiments and an ablation study that it consistently achieves a lower 3D error than previous work. It outperforms in generalisation and achieves great performance in terms of reconstruction and registration error with wide-baseline, occlusions, illumination changes, weak texture and blur. • DeepSfT is a DNN fully-convolutional based on residual encoder-decoder for SfT. • We propose a semi-supervised end-to-end training procedure with synthetic and real data. • DeepSfT is a solution to cope with multiple imaging geometries, at training and inference. • We propose to combine DeepSfT with a physics-based estimation procedure.

Speckle denoising of optical coherence tomography image using residual encoder–decoder CycleGAN

Speckle denoising of optical coherence tomography image using residual encoder–decoder CycleGAN

RED-MAM: A residual encoder-decoder network based on multi-attention fusion for ultrasound image denoising

The dependence of speckle noise in ultrasound images on image data makes the research of ultrasound image denoising tasks a great challenge. Several deep learning-based ultrasound image denoising methods have been proposed, but almost all of them suffer from the inability to focus on the importance of feature information on multiple domains at the same time. Therefore, this paper proposes a residual encoder-decoder based on multi-attention fusion attention module (RED-MAM) for ultrasound image denoising, which consists of five convolution layers, five deconvolution layers and two multi-attention fusion attention blocks. In order to make the denoising model better adapted to the properties of speckle noise, a multi-attention fusion attention block is constructed by several different attention modules. The multi-attention fusion attention block is introduced into the residual encoder-decoder denoising network to make the model focus on the ultrasound image texture structure information on multiple domains, thus enhancing the denoising effect of the model on ultrasound images. The performance of our proposed model is evaluated on three ultrasound image datasets. In terms of quantitative metrics, RED-MAM shows substantial improvement in Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM) and Root Mean Square Error (RMSE). The qualitative results show that RED-MAM has significantly improved denoising performance and has good results in noise suppression as well as structure retention. Our method achieves state-of-the-art performance on all three ultrasound datasets.

A selective kernel-based cycle-consistent generative adversarial network for unpaired low-dose CT denoising.

Low-dose computed tomography (LDCT) denoising is an indispensable procedure in the medical imaging field, which not only improves image quality, but can mitigate the potential hazard to patients caused by routine doses. Despite the improvement in performance of the cycle-consistent generative adversarial network (CycleGAN) due to the well-paired CT images shortage, there is still a need to further reduce image noise while retaining detailed features. Inspired by the residual encoder–decoder convolutional neural network (RED-CNN) and U-Net, we propose a novel unsupervised model using CycleGAN for LDCT imaging, which injects a two-sided network into selective kernel networks (SK-NET) to adaptively select features, and uses the patchGAN discriminator to generate CT images with more detail maintenance, aided by added perceptual loss. Based on patch-based training, the experimental results demonstrated that the proposed SKFCycleGAN outperforms competing methods in both a clinical dataset and the Mayo dataset. The main advantages of our method lie in noise suppression and edge preservation.

Detecting Outliers with Foreign Patch Interpolation

In medical imaging, outliers can contain hypo/hyper-intensities, minor deformations, or completely altered anatomy. To detect these irregularities it is helpful to learn the features present in both normal and abnormal images. However this is difficult because of the wide range of possible abnormalities and also the number of ways that normal anatomy can vary naturally. As such, we leverage the natural variations in normal anatomy to create a range of synthetic abnormalities. Specifically, the same patch region is extracted from two independent samples and replaced with an interpolation between both patches. The interpolation factor, patch size, and patch location are randomly sampled from uniform distributions. A wide residual encoder decoder is trained to give a pixel-wise prediction of the patch and its interpolation factor. This encourages the network to learn what features to expect normally and to identify where foreign patterns have been introduced. The estimate of the interpolation factor lends itself nicely to the derivation of an outlier score. Meanwhile the pixel-wise output allows for pixel- and subject- level predictions using the same model.<br>Our code is available at <a href='https://github.com/jemtan/FPI'>https://github.com/jemtan/FPI</a>

Multi-domain residual encoder–decoder networks for generalized compression artifact reduction

A fundamental requirement for designing compression artifact reduction techniques is to restore the artifact free image from its compressed version regardless of the compression level. Most existing algorithms require the prior knowledge of JPEG encoding parameters to operate effectively. Although there are works that attempt to train universal models to deal with different compression levels, some JPEG quality factors (QF) are still missing. To overcome these potential limitations, in this paper, we present a generalized JPEG-compression artifact reduction framework that relies on improved QF estimator and rectified networks to take into account all possible QF values. Our method, called a generalized compression artifact reducer (G-CAR), first predicts QF by analyzing luminance patches with high activity. Then, based on the estimated QF, images are adaptively restored by the cascaded residual encoder–decoder networks learned in multiple domains. Results tested on six benchmark datasets demonstrate the effectiveness of our proposed model.

RCEN: A Deep-Learning-Based Background Noise Suppression Method for DAS-VSP Records

Recently, distributed optical fiber acoustic sensing (DAS) is regarded as a transformative technology in seismic exploration. However, both various complex background noise and weak desired signals significantly limit its practical application. To explore an effective denoising method for the vertical seismic profile (VSP) record received by DAS, we propose an improved residual encoder–decoder deep neural network (RED-Net) enhanced by deep iterative memory block (DMB) and channel aggregation block (CAB), called residual channel aggregation encoder–decoder network (RCEN). Here, DMB uses the weight accumulation theory to improve the feature extraction ability and achieve accurate noise elimination. Meanwhile, CAB, using the multi-channel analysis architecture, enhances the weak signal retention performance. In addition, we leverage both the synthetic data obtained by forward modeling and real DAS noise data to construct a sufficient training dataset with high authenticity, thereby meeting the requirement of network training. Both the synthetic and field DAS-VSP data processing results demonstrate the advantage of RCEN compared with competing algorithms, including singular value decomposition (SVD), conventional RED-Net, and feed-forward denoising convolutional neural network (DnCNN).

A Multi-Scale Fusion Residual Encoder-Decoder Approach for Low Illumination Image Enhancement

In a low-light environment, due to the extremely small number of photons and high noise, the sensory light source of the line-scan camera cannot be fully exposed, resulting in a decrease in image quality. To this end, a low illumination image enhancement approach based on multi-scale fusion residual encoder- decoder is proposed, which directly learn the end-to-end mapping between the original sensor RAW light and dark images, and effectively enhance the brightness of the image while completely restoring the original image details and colors. In order to increase the feature diversity and speed up the network training, the network structure is added with residual block. To aggregate the global multi-scale features of the context, a dense context feature aggregation module is designed to make up for the lack of spatial information in the network. Based on the SID dataset, a comparative experiment with other 10 methods shows that proposed method is significantly better than most other methods in visual effects and quantitative evaluation of PSNR and SSIM. While restoring the brightness of the image, the edges and colors of the image are effectively represented, and finally a satisfactory visual quality is obtained under low light enhancement.

Red-Mam: A Residual Encoder-Decoder Network Based on Multi-Attention Fusion for Ultrasound Image Denoising

Red-Mam: A Residual Encoder-Decoder Network Based on Multi-Attention Fusion for Ultrasound Image Denoising