Medical Image Super-resolution Research Articles

Deep local-to-global feature learning for medical image super-resolution

Medical images play a vital role in medical analysis by providing crucial information about patients’ pathological conditions. However, the quality of these images can be compromised by many factors, such as limited resolution of the instruments, artifacts caused by movements, and the complexity of the scanned areas. As a result, low-resolution (LR) images cannot provide sufficient information for diagnosis. To address this issue, researchers have attempted to apply image super-resolution (SR) techniques to restore the high-resolution (HR) images from their LR counterparts. However, these techniques are designed for generic images, and thus suffer from many challenges unique to medical images. An obvious one is the diversity of the scanned objects; for example, the organs, tissues, and vessels typically appear in different sizes and shapes, and are thus hard to restore with standard convolution neural networks (CNNs). In this paper, we develop a dynamic-local learning framework to capture the details of these diverse areas, consisting of deformable convolutions with adjustable kernel shapes. Moreover, the global information between the tissues and organs is vital for medical diagnosis. To preserve global information, we propose pixel–pixel and patch–patch global learning using a non-local mechanism and a vision transformer (ViT), respectively. The result is a novel CNN-ViT neural network with Local-to-Global feature learning for medical image SR, referred to as LGSR, which can accurately restore both local details and global information. We evaluate our method on six public datasets and one large-scale private dataset, which include five different types of medical images (i.e., Ultrasound, OCT, Endoscope, CT, and MRI images). Experiments show that the proposed method achieves superior PSNR/SSIM and visual performance than the state of the arts with competitive computational costs, measured in network parameters, runtime, and FLOPs. What is more, the experiment conducted on OCT image segmentation for the downstream task demonstrates a significantly positive performance effect of LGSR.

CMISR: Circular medical image super-resolution

Classical methods of medical image super-resolution (MISR) utilize open-loop architecture with implicit under-resolution (UR) unit and explicit super-resolution (SR) unit. The UR unit can always be given, assumed, or estimated, while the SR unit is elaborately designed according to various SR algorithms. The closed-loop feedback mechanism is widely employed in current MISR approaches and can efficiently improve their performance. The feedback mechanism may be divided into two categories: local feedback and global feedback. Therefore, this paper proposes a global feedback-based closed-cycle framework, circular MISR (CMISR), with unambiguous UR and advanced SR elements. Mathematical model and closed-loop equation of CMISR are built. Mathematical proof with Taylor-series approximation indicates that CMISR has zero recovery error in steady-state. In addition, CMISR holds plug-and-play characteristic that fuses model-based and learning-based approaches and can be established on any existing MISR algorithms. Five CMISR algorithms are respectively proposed based on the state-of-the-art open-loop MISR algorithms. Experimental results with three scale factors and on three open medical image datasets show that CMISR is superior to MISR in reconstruction performance and is particularly suited to medical images with strong edges or intense contrast.

Texture Pattern-based Bi-directional Projections for Medical Image Super-resolution

Texture Pattern-based Bi-directional Projections for Medical Image Super-resolution

Asymmetric convolution Swin transformer for medical image super-resolution

Medical Image Super-Resolution plays a pivotal role in enhancing diagnostic accuracy. Transformer-based methods, such as Image Restoration Using Swin Transformer (SwinIR) and Swin transformer for fast Magnetic Resonance Imaging (SwinMR), have shown prowess in this area but also exhibit limitations. Specifically, LayerNorm channel normalization diminishes high-frequency detail, while the Multilayer Perceptron prioritizes global information over local information. Moreover, low-resolution inputs contain substantial low-frequency information, treated uniformly in self-attention, hampering the effectiveness of shifted window-based self-attention. To address these challenges, this study proposes a novel Asymmetric convolution Swin Transformer Layer that leverages global and local information within adjacent windows or pixels. Furthermore, this study presents an innovative joint attention mechanism that integrates channel attention into the Swin Transformer architecture, allowing for the simultaneous capture of local and global information and enhancing the network's ability to adapt and represent information effectively. Based on these components, this study develops a joint Channel Attention and Swin Transformer Residual Network (CSRNet) for medical image super-resolution. To evaluate its performance, this study conducts comprehensive experiments on established datasets consisting of high-resolution medical images. Compared to the state-of-the-art medical image super-resolution methods, the proposed method outperforms in reconstructing medical images across qualitative and quantitative metrics.

Enhancing the Super-Resolution of Medical Images: Introducing the Deep Residual Feature Distillation Channel Attention Network for Optimized Performance and Efficiency.

In the advancement of medical image super-resolution (SR), the Deep Residual Feature Distillation Channel Attention Network (DRFDCAN) marks a significant step forward. This work presents DRFDCAN, a model that innovates traditional SR approaches by introducing a channel attention block that is tailored for high-frequency features-crucial for the nuanced details in medical diagnostics-while streamlining the network structure for enhanced computational efficiency. DRFDCAN's architecture adopts a residual-within-residual design to facilitate faster inference and reduce memory demands without compromising the integrity of the image reconstruction. This design strategy, combined with an innovative feature extraction method that emphasizes the utility of the initial layer features, allows for improved image clarity and is particularly effective in optimizing the peak signal-to-noise ratio (PSNR). The proposed work redefines efficiency in SR models, outperforming established frameworks like RFDN by improving model compactness and accelerating inference. The meticulous crafting of a feature extractor that effectively captures edge and texture information exemplifies the model's capacity to render detailed images, necessary for accurate medical analysis. The implications of this study are two-fold: it presents a viable solution for deploying SR technology in real-time medical applications, and it sets a precedent for future models that address the delicate balance between computational efficiency and high-fidelity image reconstruction. This balance is paramount in medical applications where the clarity of images can significantly influence diagnostic outcomes. The DRFDCAN model thus stands as a transformative contribution to the field of medical image super-resolution.

Medical image super-resolution for smart healthcare applications: A comprehensive survey

The digital transformation in healthcare, propelled by the integration of deep learning models and the Internet of Things (IoT), is creating unprecedented opportunities for improving patient care. However, the utilization of low-resolution images, often generated by IoT devices, introduces biases in the deep learning models, thereby affecting the overall clinical decision-making process. While super-resolution techniques have been extensively employed to transform low-resolution images into high-resolution counterparts, the challenge of achieving highly accurate image restoration remains unresolved. This is especially critical in the medical imaging domain, where even minor inaccuracies can lead to significant biases in model training and, consequently, impact clinical outcomes. Although existing surveys have explored various super-resolution methods and their applications across different fields, a comprehensive review emphasizing the accuracy of image restoration in medical imaging and its subsequent influence on deep learning models is notably lacking. This survey seeks to bridge this gap by offering a systematic review of current state-of-the-art models, highlighting the limitations of existing surveys, and underscoring open questions that merit further research. Specifically, we delve into the intricacies of medical image restoration, identify research gaps and unmet challenges in achieving optimal restoration of medical images, and emphasize the crucial role of developing more precise and resilient super-resolution methods to enhance the quality of medical images and, consequently, the performance of deep learning models in healthcare applications. Ultimately, this survey fosters a deeper comprehension of the prevailing challenges and unresolved questions in the field, thus setting the stage for future research efforts focused on refining medical image restoration and, subsequently, boosting the efficacy of deep learning models in healthcare.

Improved sparse domain super-resolution reconstruction algorithm based on CMUT.

A novel breast ultrasound tomography system based on a circular array of capacitive micromechanical ultrasound transducers (CMUT) has broad application prospects. However, the images produced by this system are not suitable as input for the training phase of the super-resolution (SR) reconstruction algorithm. To solve the problem, this paper proposes an improved medical image super-resolution (MeSR) method based on the sparse domain. First, we use the simultaneous algebraic reconstruction technique (SART) with high imaging accuracy to reconstruct the image into a training image in a sparse domain model. Secondly, we denoise and enhance the contrast of the SART images to obtain improved detail images before training the dictionary. Then, we use the original detail image as the guide image to further process the improved detail image. Therefore, a high-precision dictionary was obtained during the testing phase and applied to filtered back projection SR reconstruction. We compared the proposed algorithm with previously reported algorithms in the Shepp Logan model and the model based on the CMUT background. The results showed significant improvements in peak signal-to-noise ratio, entropy, and average gradient compared to previously reported algorithms. The experimental results demonstrated that the proposed MeSR method can use noisy reconstructed images as input for the training phase of the SR algorithm and produce excellent visual effects.

Medical Image Super-Resolution Based on Semantic Perception Transfer Learning.

Medical images are an important basis for doctors to diagnose diseases, but some medical images have low resolution due to hardware technology and cost constraints. Super-resolution technology can reconstruct low-resolution medical images into high-resolution images and enhance the quality of low-resolution images, thus assisting doctors in diagnosing diseases. However, traditional super-resolution methods mainly learn the mapping relationships among modal pixels from low resolution to high resolution, lacking the learning of high-level semantic features, resulting in a lack of understanding and utilization of semantic information, such as reconstructed objects, object attributes, and spatial relationships between two objects. In this paper, we propose a medical image super-resolution method based on semantic perception transfer learning. First, we propose a novel semantic perception super-resolution method that empowers super-resolution models to perceive high-level semantics by transferring features of the image description generation network in natural language processing. Second, we construct a semantic feature extraction network and an image description generation network and comprehensively utilized image and text modal data to learn transferable, high-level semantic features. Third, we train an end-to-end, semantic perception super-resolution model by fusing dynamic perceptual convolution, a semantic extraction network, and distillation polarization self-attention. Experiments show that semantic perception transfer learning can effectively improve the quality of super-resolution reconstruction.

Learning the degradation distribution for medical image superresolution via sparse swin transformer

High-resolution (HR) medical images can provide rich details, which are important for discovering subtle lesions to make diagnoses. Convolutional neural networks (CNNs) are widely used in this field, but struggle to model long-range dependencies. Although transformer-based methods have improved in this respect, this method requires large quantities of data. Unfortunately, large quantities of low-resolution (LR) and HR medical image pairs may not always be available. In addition, most medical image superresolution (SR) methods are deterministic, while the degradation in real scenarios is stochastic. To address these problems, we introduce a probabilistic degradation model that combines natural and medical images for training. This design alleviates the problem of insufficient medical image pairs and learns the degradation process of the natural scene. In addition, we propose a new medical image SR model that consists of CNNs and the Swin Transformer structure to excavate both local and global semantic features. Moreover, to reduce computational stress, the spherical locality-sensitive hashing (SLSH) module is employed in the nonlocal attention (NLA) mechanism to form the ENLA module. This design enables the proposed Sparse Swin Transformer (SSFormer) model to generate HR medical images without extensive training images. Experiments on diverse datasets (natural images and medical images) demonstrate that the proposed method is robust and effective, qualitatively and quantitatively outperforming other medical image SR methods. Code is available at https://github.com/codehxj/SSFormer.

A Review of Image Super Resolution using Deep Learning

The image processing methods collectively known as super-resolution have proven useful in creating high-quality images from a group of low-resolution photographic images. Single image super resolution (SISR) has been applied in a variety of fields. The paper offers an in-depth analysis of a few current picture super resolution works created in various domains. In order to comprehend the most current developments in the development of Image super resolution systems, these recent publications have been examined with particular emphasis paid to the domain for which these systems have been designed, image enhancement used or not, among other factors. To improve the accuracy of the image super resolution, a different deep learning techniques might be explored. In fact, greater research into the image super resolution in medical imaging is possible to improve the data's suitability for future analysis. In light of this, it can be said that there is a lot of scope for research in the field of medical imaging.

Deep learning in medical image super resolution: a review

Deep learning in medical image super resolution: a review

Medical image super-resolution: A review, recent trends and techniques

The quality and resolution of medical images significantly impact the accuracy of clinical decisions. Preprocessing of medical images plays a crucial role in the performance of computer-aided diagnosis (CAD) systems, which helps to improve the quality of input images, remove noise, and enhance contrast. Medical images are often characterized by low resolution due to different constraints such as hardware limits and image acquisition time. To address this issue, researchers have proposed various super-resolution approaches to improve the resolution of medical imagesand therefore better image quality. Hence, the super-resolution (SR) process aims to reconstruct a high-resolution (HR) image from its low-resolution (LR) counterpart. Several techniques are suggested in the literature including interpolation-based methods, reconstruction-based methods and learning-based methods. In recent years, deep learning-based methods have shownpromising results in various medical image applications while preserving important features and structures. In this article, we present a review of different studies that have proven to be useful in medical image super-resolution including classical computer vision methods and deep learning based-methods.

Unsupervised MRI Super Resolution Using Deep External Learning and Guided Residual Dense Network With Multimodal Image Priors

Deep learning techniques have led to state-of-the-art image super resolution with natural images. Normally, pairs of high-resolution and low-resolution images are used to train the deep learning models. These techniques have also been applied to medical image super-resolution. The characteristics of medical images differ significantly from natural images in several ways. First, it is difficult to obtain high-resolution images for training in real clinical applications due to the limitations of imaging systems and clinical requirements. Second, other modal high-resolution images are available (e.g., high-resolution T1-weighted images are available for enhancing low-resolution T2-weighted images). In this paper, we propose an unsupervised image super-resolution technique based on simple prior knowledge of the human anatomy. This technique does not require target T2WI high-resolution images for training. Furthermore, we present a guided residual dense network, which incorporates a residual dense network with a guided deep convolutional neural network for enhancing the resolution of low-resolution images by referring to different modal high-resolution images of the same subject. Experiments on a publicly available brain MRI database showed that our proposed method achieves better performance than the state-of-the-art methods.

IRMIRS: Inception-ResNet-Based Network for MRI Image Super-Resolution

Medical image super-resolution is a fundamental challenge due to absorption and scattering in tissues. These challenges are increasing the interest in the quality of medical images. Recent research has proven that the rapid progress in convolutional neural networks (CNNs) has achieved superior performance in the area of medical image super-resolution. However, the traditional CNN approaches use interpolation techniques as a preprocessing stage to enlarge low-resolution magnetic resonance (MR) images, adding extra noise in the models and more memory consumption. Furthermore, conventional deep CNN approaches used layers in series-wise connection to create the deeper mode, because this later end layer cannot receive complete information and work as a dead layer. In this paper, we propose Inception-ResNet-based Network for MRI Image Super-Resolution known as IRMRIS. In our proposed approach, a bicubic interpolation is replaced with a deconvolution layer to learn the upsampling filters. Furthermore, a residual skip connection with the Inception block is used to reconstruct a high-resolution output image from a low-quality input image. Quantitative and qualitative evaluations of the proposed method are supported through extensive experiments in reconstructing sharper and clean texture details as compared to the state-of-the-art methods.

Lightweight Dual Mutual-Feedback Network for Artificial Intelligence in Medical Image Super-Resolution

As a result of hardware resource constraints, it is difficult to obtain medical images with a sufficient resolution to diagnose small lesions. Recently, super-resolution (SR) was introduced into the field of medicine to enhance and restore medical image details so as to help doctors make more accurate diagnoses of lesions. High-frequency information enhances the accuracy of the image reconstruction, which is demonstrated by deep SR networks. However, deep networks are not applicable to resource-constrained medical devices because they have too many parameters, which requires a lot of memory and higher processor computing power. For this reason, a lightweight SR network that demonstrates good performance is needed to improve the resolution of medical images. A feedback mechanism enables the previous layers to perceive high-frequency information of the latter layers, but no new parameters are introduced, which is rarely used in lightweight networks. Therefore, in this work, a lightweight dual mutual-feedback network (DMFN) is proposed for medical image super-resolution, which contains two back-projection units that operate in a dual mutual-feedback manner. The features generated by the up-projection unit are fed back into the down-projection unit and, simultaneously, the features generated by the down-projection unit are fed back into the up-projection unit. Moreover, a contrast-enhanced residual block (CRB) is proposed as each cell block used in projection units, which enhances the pixel contrast in the channel and spatial dimensions. Finally, we designed a unity feedback to down-sample the SR result as the inverse process of SR. Furthermore, we compared it with the input LR to narrow the solution space of the SR function. The final ablation studies and comparison results show that our DMFN performs well without utilizing a large amount of computing resources. Thus, it can be used in resource-constrained medical devices to obtain medical images with better resolutions.

Super-Resolution Swin Transformer and Attention Network for Medical CT Imaging.

Computerized tomography (CT) is widely used for clinical screening and treatment planning. In this study, we aimed to reduce X-ray radiation and achieve high-quality CT imaging by using low-intensity X-rays because CT radiation is damaging to the human body. An innovative vision transformer for medical image super-resolution (SR) is applied to establish a high-definition image target. To achieve this, we proposed a method called swin transformer and attention network (STAN) that uses the swin transformer network, which employs an attention method to overcome the long-range dependency difficulties encountered in CNNs and RNNs to enhance and restore the quality of medical CT images. We adopted the peak signal-to-noise ratio for performance comparison with other mainstream SR reconstruction models used in medical CT imaging. Experimental results revealed that the proposed STAN model yields superior medical CT imaging results than the existing SR techniques based on CNNs. The proposed STAN model employs a self-attention mechanism to more effectively extract critical features and long-range information, hence enhancing the quality of medical CT image reconstruction.

Multi-modal medical image super-resolution fusion based on detail enhancement and weighted local energy deviation

Multi-modal medical image fusion (MMIF) integrates medical images of different modalities into an image with rich information to boost the accuracy and efficiency of clinical diagnosis and treatment. There are two main problems in medical image fusion: 1) It is difficult to balance the computational efficiency and fusion quality; 2) in the clinic, it is necessary to observe medical images in high resolution. To overcome these problems, a multi-modal medical image super-resolution (SR) fusion (MMISRF) method is proposed based on detail enhancement and weighted local energy deviation (WLED). The method has two major novelties. For the first problem, to improve the efficiency, the proposed method decomposes the SR image into one base layer and one detail layer through a two-scale decomposition method. Then, for the detail layer, a fusion rule is proposed based on detail enhancement and information refinement to enhance the detail information and retain the salient feature information. For the base layer, a WLED-based rule is designed to better preserve the energy information from the source images to the fused image. The final fused image is obtained by combining the fused detail and base layers. For the second problem, this paper introduces the bicubic interpolation-based SR into the field of MMIF for the first time. The experimental results indicate that the proposed MMISRF outperforms the state-of-the-art approaches in terms of subjective visual effect and objective evaluation. Furthermore, the proposed method is more effective for MMIF task at different resolutions.

Generating High-Resolution CT Slices from Two Image Series Using Deep-Learning-Based Resolution Enhancement Methods.

Medical image super-resolution (SR) has mainly been developed for a single image in the literature. However, there is a growing demand for high-resolution, thin-slice medical images. We hypothesized that fusing the two planes of a computed tomography (CT) study and applying the SR model to the third plane could yield high-quality thin-slice SR images. From the same CT study, we collected axial planes of 1 mm and 5 mm in thickness and coronal planes of 5 mm in thickness. Four SR algorithms were then used for SR reconstruction. Quantitative measurements were performed for image quality testing. We also tested the effects of different regions of interest (ROIs). Based on quantitative comparisons, the image quality obtained when the SR models were applied to the sagittal plane was better than that when applying the models to the other planes. The results were statistically significant according to the Wilcoxon signed-rank test. The overall effect of the enhanced deep residual network (EDSR) model was superior to those of the other three resolution-enhancement methods. A maximal ROI containing minimal blank areas was the most appropriate for quantitative measurements. Fusing two series of thick-slice CT images and applying SR models to the third plane can yield high-resolution thin-slice CT images. EDSR provides superior SR performance across all ROI conditions.

Gated Multi-Attention Feedback Network for Medical Image Super-Resolution

Medical imaging technology plays a crucial role in the diagnosis and treatment of diseases. However, the captured medical images are often in a low resolution (LR) due to the limited imaging condition. Super-resolution (SR) technology is a feasible solution to enhance the resolution of a medical image without increasing the hardware cost. However, the existing SR methods often ignore high-frequency details, which results in blurred edges and an unsatisfying visual perception. In this paper, a gated multi-attention feedback network (GAMA) is proposed for medical image SR. Specifically, a gated multi-feedback network is employed as the backbone to extract hierarchical features. Meanwhile, a layer attention feature extraction (LAFE) module is introduced to refine the feature map. In addition, a channel-space attention reconstruction (CSAR) module is built to enhance the representational ability of the semantic feature map. Furthermore, a gradient variance loss is tailored as the regularization in guiding the model learning to regularize the model in generating a faithful high-resolution image with rich textures and sharp edges. The experiments verify the effectiveness of the proposed GAMA compared with the state-of-the-art approaches.

Medical image super-resolution reconstruction via multi-scale information distillation network under multi-scale geometric transform domain

High resolution (HR) magnetic resonance images (MRI) or computed tomography (CT) images can provide clearer anatomical details of human body, which facilitates early diagnosis of the diseases. However, due to the imaging system, imaging environment and human factors, it is difficult to obtain clear high-resolution images. In this paper, we proposed a novel medical image super resolution (SR) reconstruction method via multi-scale information distillation (MSID) network in the non-subsampled shearlet transform (NSST) domain, namely NSST-MSID network. We first proposed a MSID network that mainly consisted of a series of stacked MSID blocks to fully exploit features from images and effectively restore the low resolution (LR) images to HR images. In addition, most previous methods predict the HR images in the spatial domain, producing over-smoothed outputs while losing texture details. Thus, we viewed the medical image SR task as the prediction of NSST coefficients, which make further MSID network keep richer structure details than that in spatial domain. Finally, the experimental results on our constructed medical image datasets demonstrated that the proposed method was capable of obtaining better peak signal to noise ratio (PSNR), structural similarity (SSIM) and root mean square error (RMSE) values and keeping global topological structure and local texture detail better than other outstanding methods, which achieves good medical image reconstruction effect.