Image Super-resolution Method Research Articles

A Fast and Super-Resolution Method of Vortex-Based Imaging

With characteristics of particular wavefronts and orthogonal modes, the sparse imaging method based on vortex waves could obtain high-resolution reconstruction. However, building a large-scale observation matrix is time-consuming and heavily burdens the whole imaging procedure. Inspired by the spherical harmonic expansion of plane waves, this paper provides a fast super-resolution imaging method based on vortex waves. Featuring spherical harmonic functions and a transformation, an orthogonal matching pursuit algorithm for high-resolution imaging exploits a low-rank matrix decomposition of the observation matrix. Simulation results show that the proposed method exhibits significant efficiency advantages while retaining high resolution, achieving an efficiency of up to 51.4% when applied to a large-scale observation matrix.

Improved sparse domain super-resolution reconstruction algorithm based on CMUT.

A novel breast ultrasound tomography system based on a circular array of capacitive micromechanical ultrasound transducers (CMUT) has broad application prospects. However, the images produced by this system are not suitable as input for the training phase of the super-resolution (SR) reconstruction algorithm. To solve the problem, this paper proposes an improved medical image super-resolution (MeSR) method based on the sparse domain. First, we use the simultaneous algebraic reconstruction technique (SART) with high imaging accuracy to reconstruct the image into a training image in a sparse domain model. Secondly, we denoise and enhance the contrast of the SART images to obtain improved detail images before training the dictionary. Then, we use the original detail image as the guide image to further process the improved detail image. Therefore, a high-precision dictionary was obtained during the testing phase and applied to filtered back projection SR reconstruction. We compared the proposed algorithm with previously reported algorithms in the Shepp Logan model and the model based on the CMUT background. The results showed significant improvements in peak signal-to-noise ratio, entropy, and average gradient compared to previously reported algorithms. The experimental results demonstrated that the proposed MeSR method can use noisy reconstructed images as input for the training phase of the SR algorithm and produce excellent visual effects.

AIF-LFNet: All-in-Focus Light Field Super-Resolution Method Considering the Depth-Varying Defocus

As an aperture-divided computational imaging system, microlens array (MLA) -based light field (LF) imaging is playing an increasingly important role in computer vision. As the trade-off between the spatial and angular resolutions, deep learning (DL) -based image super-resolution (SR) methods have been applied to enhance the spatial resolution. However, in existing DL-based methods, the depth-varying defocus is not considered both in dataset development and algorithm design, which restricts many applications such as depth estimation and object recognition. To overcome this shortcoming, a super-resolution task that reconstructs all-in-focus high-resolution (HR) LF images from low-resolution (LR) LF images is proposed by designing a large dataset and proposing a convolutional neural network (CNN) -based SR method. The dataset is constructed by using Blender software, consisting of 150 light field images used as training data, and 15 light field images used as validation and testing data. The proposed network is designed by proposing the dilated deformable convolutional network (DCN) -based feature extraction block and the LF subaperture image (SAI) Deblur-SR block. The experimental results demonstrate that the proposed method achieves more appealing results both quantitatively and qualitatively.

DASR: Dual-Attention Transformer for infrared image super-resolution

The infrared image super-resolution (SR) method successfully overcomes the hardware limitations of infrared cameras that reconstruct higher-quality images with improved efficiency and cost-effectiveness. However, existing infrared SR methods do not take into account the specificity of infrared images and are primarily designed for small-scale factors. In this paper, we first investigate the domain differences between infrared and visible images and the impact of these differences on super-resolution tasks. It was found that compared with visible SR, the infrared SR will require more global edge structure information rather than local texture information reconstructed by previous CNN-based methods mainly. To address this disparity, we propose a novel infrared SR model, named DASR, which incorporates a Transformer with spatial and channel dual-attention mechanisms. In DASR, spatial attention captures both local and global information through window-based and cross-window contextual long-range interactions, while channel attention captures channel-wise global information through cross-channel interactions. With this new Transformer architecture, our method effectively extracts spatial and channel global information, which cannot be captured by the local receptive field of convolution, thus making it more suitable for infrared SR. Extensive experiments on benchmark datasets have indicated that our method outperforms the state-of-the-art infrared SR methods with less number of parameters and lower computational complexity while having the best visual results.

A super-resolution ultrasound imaging method based on active-modulated super-resolution optical fluctuation imaging.

Super resolution ultrasound imaging (SR-US) methods including super-resolution optical fluctuation imaging (SOFI) have been successfully demonstrated to improve imaging performance of ultrasound (US). However, the imaging quality of US improved by conventional SOFI depends on the probability of microbubbles (MB) appearing in imaging regions. Current SOFI-based ultrasound imaging methods usually fix the probability of MBs, ignoring the effect of probability characteristics, leading to artifacts in high-order SOFI images. Inspired by active-modulated SOFI (AR-SOFI), in this paper, we propose a new method, termed as AR-SOFI-US, for further improving the performance of SR-US, which is achieved by effectively controlling the probabilities of MBs on an appropriate range. Through a series of numerical simulations, we compare the imaging resolution at differing MB probabilities and demonstrate that by controlling the probabilities of MBs when they appear in the imaging regions, incorporating the proposed AR-SOFI-US method, we can improve the spatial resolution of SR-US to a higher degree, especially for the high-order SOFI imaging results.

Progressive spatial-angular feature enhancement network for light field image super-resolution

The intensity and direction of the light field (LF) can be recorded simultaneously by using LF cameras. However, since LF cameras sacrifice spatial resolution for higher angular resolution, the images acquired by LF cameras tend to have low spatial resolution. Therefore, LF image super-resolution (SR) has become an integral part of LF studies. Many existing LF image SR methods fail to fully utilize angular and spatial information due to only using partial sub-aperture images (SAIs). In this paper, we propose a progressive spatial-angular feature enhancement network (PSAFENet) to deal with the problem of missing information in LF image SR. Specifically, we first extract the spatial features of SAIs, the spatial and angular features contained in the macro-pixel images (MacPIs) by three different feature extraction modules. Then, these features are fed into a spatial-angular feature enhancement (SAFE) module to perform enhancement of spatial-angular information on the SAIs. To improve the reconstruction accuracy, we also use the information multi-distillation block (IMDB) to remove the redundant information before upsampling. Our network can well merge the angular and spatial information into each SAI, which facilitates the reconstruction of the LF images. Experimental results on five public datasets show that the proposed PSAFENet method outperforms existing methods in both qualitative and quantitative comparisons.

Image super-resolution method based on attention aggregation hierarchy feature

Image super-resolution method based on attention aggregation hierarchy feature

A Versatile Synaptotagmin-1 Nanobody Provides Perturbation-Free Live Synaptic Imaging And Low Linkage-Error in Super-Resolution Microscopy.

Imaging of living synapses has relied for over two decades on the overexpression of synaptic proteins fused to fluorescent reporters. This strategy alters the stoichiometry of synaptic components and ultimately affects synapse physiology. To overcome these limitations, here a nanobody ispresented that binds the calcium sensor synaptotagmin-1 (NbSyt1). This nanobody functions as an intrabody (iNbSyt1) in living neurons and is minimally invasive, leaving synaptic transmission almost unaffected, as suggested by the crystal structure of the NbSyt1 bound to Synaptotagmin-1 and by thephysiological data. Its single-domain nature enables the generation of protein-based fluorescent reporters, as showcased here by measuring spatially localized presynaptic Ca2+ with a NbSyt1- jGCaMP8 chimera. Moreover, the small size of NbSyt1 makes it ideal for various super-resolution imaging methods. Overall, NbSyt1 is a versatile binder that will enable imaging in cellular and molecular neuroscience with unprecedented precision across multiple spatiotemporal scales.

Medical Image Super-Resolution Based on Semantic Perception Transfer Learning.

Medical images are an important basis for doctors to diagnose diseases, but some medical images have low resolution due to hardware technology and cost constraints. Super-resolution technology can reconstruct low-resolution medical images into high-resolution images and enhance the quality of low-resolution images, thus assisting doctors in diagnosing diseases. However, traditional super-resolution methods mainly learn the mapping relationships among modal pixels from low resolution to high resolution, lacking the learning of high-level semantic features, resulting in a lack of understanding and utilization of semantic information, such as reconstructed objects, object attributes, and spatial relationships between two objects. In this paper, we propose a medical image super-resolution method based on semantic perception transfer learning. First, we propose a novel semantic perception super-resolution method that empowers super-resolution models to perceive high-level semantics by transferring features of the image description generation network in natural language processing. Second, we construct a semantic feature extraction network and an image description generation network and comprehensively utilized image and text modal data to learn transferable, high-level semantic features. Third, we train an end-to-end, semantic perception super-resolution model by fusing dynamic perceptual convolution, a semantic extraction network, and distillation polarization self-attention. Experiments show that semantic perception transfer learning can effectively improve the quality of super-resolution reconstruction.

MapSR: A Deep Neural Network for Super-Resolution of Raster Map

The purpose of multisource map super-resolution is to reconstruct high-resolution maps based on low-resolution maps, which is valuable for content-based map tasks such as map recognition and classification. However, there is no specific super-resolution method for maps, and the existing image super-resolution methods often suffer from missing details when reconstructing maps. We propose a map super-resolution (mapSR) model that fuses local and global features for super-resolution reconstruction of low-resolution maps. Specifically, the proposed model consists of three main modules: a shallow feature extraction module, a deep feature fusion module, and a map reconstruction module. First, the shallow feature extraction module initially extracts the image features and embeds the images with appropriate dimensions. The deep feature fusion module uses Transformer and Convolutional Neural Network (CNN) to focus on extracting global and local features, respectively, and fuses them by weighted summation. Finally, the map reconstruction module uses upsampling methods to reconstruct the map features into the high-resolution map. We constructed a high-resolution map dataset for training and validating the map super-resolution model. Compared with other models, the proposed method achieved the best results in map super-resolution.

A Hybrid Approach for retinal image super-resolution

Experts require large high-resolution retinal images to detect tiny abnormalities, such as microaneurysms or issues of vascular branches. However, these images often suffer from low quality (e.g., resolution) due to poor imaging device configuration and misoperations. Many works utilized Convolutional Neural Network-based (CNN) methods for image super-resolution. The authors focused on making these models more complex by adding layers and various blocks. It leads to additional computational expenses and obstructs the application in real-life scenarios. Thus, this paper proposes a novel, lightweight, deep-learning super-resolution method for retinal images. It comprises a Vision Transformer (ViT) encoder and a convolutional neural network decoder. To our best knowledge, this is the first attempt to use a transformer-based network to solve the issue of accurate retinal image super-resolution. A progressively growing super-resolution training technique is applied to increase the resolution of images by factors of 2, 4, and 8. The prominent architecture remains constant thanks to the adaptive patch embedding layer, which does not lead to additional computational expense due to increased up-scaling factors. This patch embedding layer includes 2-dimensional convolution with specific values of kernel size and strides that depend on the input shape. This strategy has removed the need to append additional super-resolution blocks to the model. The proposed method has been evaluated through quantitative and qualitative measures. The qualitative analysis also includes vessel segmentation of super-resolved and ground truth images. Experimental results indicate that the proposed method outperforms the current state-of-the-art methods.

Context-aware lightweight remote-sensing image super-resolution network.

In recent years, remote-sensing image super-resolution (RSISR) methods based on convolutional neural networks (CNNs) have achieved significant progress. However, the limited receptive field of the convolutional kernel in CNNs hinders the network's ability to effectively capture long-range features in images, thus limiting further improvements in model performance. Additionally, the deployment of existing RSISR models to terminal devices is challenging due to their high computational complexity and large number of parameters. To address these issues, we propose a Context-Aware Lightweight Super-Resolution Network (CALSRN) for remote-sensing images. The proposed network primarily consists of Context-Aware Transformer Blocks (CATBs), which incorporate a Local Context Extraction Branch (LCEB) and a Global Context Extraction Branch (GCEB) to explore both local and global image features. Furthermore, a Dynamic Weight Generation Branch (DWGB) is designed to generate aggregation weights for global and local features, enabling dynamic adjustment of the aggregation process. Specifically, the GCEB employs a Swin Transformer-based structure to obtain global information, while the LCEB utilizes a CNN-based cross-attention mechanism to extract local information. Ultimately, global and local features are aggregated using the weights acquired from the DWGB, capturing the global and local dependencies of the image and enhancing the quality of super-resolution reconstruction. The experimental results demonstrate that the proposed method is capable of reconstructing high-quality images with fewer parameters and less computational complexity compared with existing methods.

Cross-View Attention Interaction Fusion Algorithm for Stereo Super-Resolution

In the process of stereo super-resolution reconstruction, in addition to the richness of the extracted feature information directly affecting the texture details of the reconstructed image, the texture details of the corresponding pixels between stereo image pairs also have an important impact on the reconstruction accuracy in the process of network learning. Therefore, aiming at the information interaction and stereo consistency of stereo image pairs, a cross-view attention interaction fusion stereo super-resolution algorithm is proposed. Firstly, based on parallax attention mechanism and triple attention mechanism, an attention stereo fusion module is constructed. The attention stereo fusion module is inserted between different levels of two single image super-resolution network branches, and the attention weight is calculated through the cross dimensional interaction of the three branches. It makes full use of the ability of single image super-resolution network to extract single view information and further maintaining the stereo consistency between stereo image pairs. Then, an enhanced cross-view interaction strategy including three fusion methods is proposed. Specifically, the vertical sparse fusion method is used to integrate the interior view information of different levels in the two single image super-resolution sub branches, the horizontal dense fusion method is used to connect the adjacent attention stereo fusion modules and the constraint between stereo image consistency is further strengthened in combination with the feature fusion method. Finally, the experimental results on Flickr 1024, Middlebury and KITTI benchmark datasets show that the proposed algorithm is superior to the existing stereo image super-resolution methods in quantitative measurement and qualitative visual quality while maintaining the stereo consistency of image pairs.

CGC-Net: A Context-Guided Constrained Network for Remote-Sensing Image Super Resolution

In remote-sensing image processing tasks, images with higher resolution always result in better performance on downstream tasks, such as scene classification and object segmentation. However, objects in remote-sensing images often have low resolution and complex textures due to the imaging environment. Therefore, effectively reconstructing high-resolution remote-sensing images remains challenging. To address this concern, we investigate embedding context information and object priors from remote-sensing images into current deep learning super-resolution models. Hence, this paper proposes a novel remote-sensing image super-resolution method called Context-Guided Constrained Network (CGC-Net). In CGC-Net, we first design a simple but effective method to generate inverse distance maps from the remote-sensing image segmentation maps as prior information. Combined with prior information, we propose a Global Context-Constrained Layer (GCCL) to extract high-quality features with global context constraints. Furthermore, we introduce a Guided Local Feature Enhancement Block (GLFE) to enhance the local texture context via a learnable guided filter. Additionally, we design a High-Frequency Consistency Loss (HFC Loss) to ensure gradient consistency between the reconstructed image (HR) and the original high-quality image (HQ). Unlike existing remote-sensing image super-resolution methods, the proposed CGC-Net achieves superior visual results and reports new state-of-the-art (SOTA) performance on three popular remote-sensing image datasets, demonstrating its effectiveness in remote-sensing image super-resolution (RSI-SR) tasks.

Spectral-Spatial MLP Network for Hyperspectral Image Super-Resolution

Many hyperspectral image (HSI) super-resolution (SR) methods have been proposed and have achieved good results; however, they do not sufficiently preserve the spectral information. It is beneficial to sufficiently utilize the spectral correlation. In addition, most works super-resolve hyperspectral images using high computation complexity. To solve the above problems, a novel method based on a channel multilayer perceptron (CMLP) is presented in this article, which aims to obtain a better performance while reducing the computational cost. To sufficiently extract spectral features, a local-global spectral integration block is proposed, which consists of CMLP and some parameter-free operations. The block can extract local and global spectral features with low computational cost. In addition, a spatial feature group extraction block based on the CycleMLP framework is designed; it can extract local spatial features well and reduce the computation complexity and number of parameters. Extensive experiments demonstrate that our method achieves a good performance compared with other methods.

Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion.

When traditional super-resolution reconstruction methods are applied to infrared thermal images, they often ignore the problem of poor image quality caused by the imaging mechanism, which makes it difficult to obtain high-quality reconstruction results even with the training of simulated degraded inverse processes. To address these issues, we proposed a thermal infrared image super-resolution reconstruction method based on multimodal sensor fusion, aiming to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming the limitations of imaging mechanisms. First, we designed a novel super-resolution reconstruction network, which consisted of primary feature encoding, super-resolution reconstruction, and high-frequency detail fusion subnetwork, to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming limitations of imaging mechanisms. We designed hierarchical dilated distillation modules and a cross-attention transformation module to extract and transmit image features, enhancing the network's ability to express complex patterns. Then, we proposed a hybrid loss function to guide the network in extracting salient features from thermal infrared images and reference images while maintaining accurate thermal information. Finally, we proposed a learning strategy to ensure the high-quality super-resolution reconstruction performance of the network, even in the absence of reference images. Extensive experimental results show that the proposed method exhibits superior reconstruction image quality compared to other contrastive methods, demonstrating its effectiveness.

Learning the degradation distribution for medical image superresolution via sparse swin transformer

High-resolution (HR) medical images can provide rich details, which are important for discovering subtle lesions to make diagnoses. Convolutional neural networks (CNNs) are widely used in this field, but struggle to model long-range dependencies. Although transformer-based methods have improved in this respect, this method requires large quantities of data. Unfortunately, large quantities of low-resolution (LR) and HR medical image pairs may not always be available. In addition, most medical image superresolution (SR) methods are deterministic, while the degradation in real scenarios is stochastic. To address these problems, we introduce a probabilistic degradation model that combines natural and medical images for training. This design alleviates the problem of insufficient medical image pairs and learns the degradation process of the natural scene. In addition, we propose a new medical image SR model that consists of CNNs and the Swin Transformer structure to excavate both local and global semantic features. Moreover, to reduce computational stress, the spherical locality-sensitive hashing (SLSH) module is employed in the nonlocal attention (NLA) mechanism to form the ENLA module. This design enables the proposed Sparse Swin Transformer (SSFormer) model to generate HR medical images without extensive training images. Experiments on diverse datasets (natural images and medical images) demonstrate that the proposed method is robust and effective, qualitatively and quantitatively outperforming other medical image SR methods. Code is available at https://github.com/codehxj/SSFormer.

Method of depth simulation imaging and depth image super-resolution reconstruction for a 2D/3D compatible CMOS image sensor.

This paper presents a depth simulation imaging and depth image super-resolution (SR) method for two-dimensional/three-dimensional compatible CMOS image sensors. A depth perception model is established to analyze the effects of depth imaging parameters and evaluate the real imaging effects. We verify its validity by analyzing the depth error, imaging simulation, and auxiliary physical verification. By means of the depth simulation images, we then propose a depth SR reconstruction algorithm to recover the low-resolution depth maps to the high-resolution depth maps in two types of datasets. With the best situation in depth accuracy kept, the root mean square error (RMSE) of Middlebury dataset images are 0.0156, 0.0179, and 0.0183m. The RMSE of RGB-D dataset images are 0.0223 and 0.0229m. Compared with other listed conventional algorithms, our algorithm reduces the RMSE by more than 16.35%, 17.19%, and 23.90% in the Middlebury dataset images. Besides, our algorithm reduces the RMSE by more than 9.71% and 8.76% in the RGB-D dataset images. The recovery effects achieve optimized results.

Multi-Stage Edge-Guided Stereo Feature Interaction Network for Stereoscopic Image Super-Resolution

Stereo image super-resolution (SR) aims to simultaneously increase the resolution of stereo image pairs, which benefits many downstream three-dimensional (3D) multimedia broadcasting and stereo vision-related tasks, such as 3D television broadcasting and stereo matching. A key insight in convolutional neural networks-based stereo image SR is to enforce stereo feature interactions between the two stereo views to explore complementary cross-view features that can facilitate the SR in both views. To fully exploit the cross-view stereo features, in this paper we propose a new multi-stage network, cascaded by several stereo feature interactions, progressively improving the SR quality from coarse to fine. In particular, an edge-guided stereo attention mechanism is proposed to be embedded into each stereo feature interaction to better capture consistent structure details of the cross-views. Followed by stereo feature fusion and reconstruction modules, we finally put together a multi-stage edge-guided stereo feature interaction network (MESFINet) for stereo image SR. Comprehensive experiments on KITTI2012, KITTI2015, Middlebury, and Flickr1024 benchmark datasets show that the proposed MESFINet achieves superior performance against the state-of-the-art stereo image SR methods and can be used to improve the accuracy of stereo matching.

A far-field super-resolution imaging method based on phase constructive

Super-resolution imaging under passive illumination is always a crucial issue to be addressed. Because the wavefront of an incoherent light field is not easily detected, it is hard to achieve super-resolution imaging by synthetic aperture under passive illumination. In this work, we proposed a super-resolution method based on phase constructive. We use a quadrature phase-shifting algorithm to retrieve the wavefront from a series of correlation patterns and reconstruct the super-resolution image with a wavefront synthesis algorithm. A simulation result verifies the effectiveness of our proposed method with a USAF-1951 target as the object.