Super-resolution Image Restoration Research Articles

IoT-Enhanced local attention dual networks for pathological image restoration in healthcare

High-resolution pathological images play a pivotal role in accurate disease diagnosis and are important in precision medicine. However, obtaining real-time high-resolution images faces challenges due to hardware limitations and scanning time constraints. Conventional image super-resolution restoration algorithms struggle to provide satisfactory results for pathological images, resulting in blurred and unrealistic restorations. In response to this challenge, this research proposes a pioneering approach by integrating Internet of Things (IoT) technology with Local Attention Dual Network (IOT-LAT) for super-resolution restoration of pathological images. The enhanced IOT-LATincorporates IoT-based data acquisition and processing, IoT-Enhanced non-local attention mechanisms, Gaussian constraint, and parameter-sharing strategies in up-sampling and down-sampling branches. This integration enables real-time super-resolution restoration of pathological images with improved accuracy. The reconstructed images exhibit a structural similarity of 0.914 and a peak signal-to-noise ratio of 30.84 dB. These results validate the effectiveness of the suggested approach in precisely reconstructing high-frequency details and improving modelling efficiency via a lightweight non-local attention mechanism enhanced by the Internet of Things. This work discusses IoT and non-local attention dual networks to improve pathological image super-resolution restoration. This allows for faster and more accurate disease diagnosis and treatment in precision medicine.

Super-resolution image restoration for microlens array imaging system

Microlens array (MLA) has been widely applied in optical imaging fields. However, the existing MLA imaging system (MLAIS) often suffers the problems of low imaging resolutions and luminance contrast due to the limitation of the sensor size and microlens aperture. To address these problems, we propose a super-resolution (SR) image restoration method for the microlens array imaging system (MLAIS). The proposed method uses an Otsu-Kmeans gravity-based center extraction method to segment the image captured by MLAIS into many sub-images and an enhanced correlation coefficient-based approach to align the sub-images. A maximum a posteriori probability algorithm is developed to recover the SR image from the aligned sub-images. A two-step gray transformation algorithm is designed to enhance the SR image. Numerical simulations and practical experiments are implemented to verify the effectiveness of the proposed method, and its SR restoration performance is evaluated by comparing the results with classical and state-of-the-art methods. The results indicate that the proposed method can achieve high-quality SR image restoration for the MLAIS, and its SR performance is better than the comparison methods. The proposed method may find applications in multispectral SR imaging, remote sensing SR imaging, muti-sensor SR imaging, and so on.

Deep learning-based image super-resolution restoration for mobile infrared imaging system

Mobile infrared imaging system (MIIS) has important applications in the areas of the military affairs, medical diagnosis, and industrial detection. However, MIIS usually suffers the problems of the low resolution, low contrast, and noise interference. To solve these problems, we propose a deep learning-based image super-resolution restoration method for the MIIS. We design a multiscale feature distillation residual network to retain image features at various stages during the training process. The network uses dilated convolution to expand the receptive field and outputs the super-resolution (SR) infrared image with a sub-pixel method. Many infrared images that are captured by a MIIS are used for experiments. Experimental results indicate that the proposed method can realize the infrared image SR restoration and performs outstandingly against the four recently published SR methods both in visual quality and indicator performance. The proposed method is helpful for the practical applications of the MIIS.

A method for optimal linear super-resolution image restoration

In this paper, we propose a super-resolution (pixel grid refinement) method for digital images. It is based on the linear filtering of a zero-padded discrete signal. We introduce a continuous-discrete observation model to create a reconstruction system. The proposed observation model is typical of real-world imaging systems - an initially continuous signal first undergoes linear (dynamic) distortions and then is subjected to sampling and the effect of additive noise. The proposed method is optimal in the sense of mean square recovery error minimization. In the theoretical part of the article, a general scheme of the linear super-resolution of the signal is presented and expressions for the impulse and frequency responses of the optimal reconstruction system are derived. An expression for the error of such restoration is also derived. For the sake of brevity, the entire description is presented for one-dimensional signals, but the obtained results can easily be generalized for the case of two-dimensional images. The experimental section of the paper is devoted to the analysis of the super-resolution reconstruction error depending on the parameters of the observation model. The significant superiority of the proposed method in terms of the reconstruction error is demonstrated in comparison with linear interpolation, which is usually used to refine the grid of image pixels.

Ultra-High-Resolution 1 m/pixel CaSSIS DTM Using Super-Resolution Restoration and Shape-from-Shading: Demonstration over Oxia Planum on Mars

We introduce a novel ultra-high-resolution Digital Terrain Model (DTM) processing system using a combination of photogrammetric 3D reconstruction, image co-registration, image super-resolution restoration, shape-from-shading DTM refinement, and 3D co-alignment methods. Technical details of the method are described, and results are demonstrated using a 4 m/pixel Trace Gas Orbiter Colour and Stereo Surface Imaging System (CaSSIS) panchromatic image and an overlapping 6 m/pixel Mars Reconnaissance Orbiter Context Camera (CTX) stereo pair to produce a 1 m/pixel CaSSIS Super-Resolution Restoration (SRR) DTM for different areas over Oxia Planum on Mars—the future ESA ExoMars 2022 Rosalind Franklin rover’s landing site. Quantitative assessments are made using profile measurements and the counting of resolvable craters, in comparison with the publicly available 1 m/pixel High-Resolution Imaging Experiment (HiRISE) DTM. These assessments demonstrate that the final resultant 1 m/pixel CaSSIS DTM from the proposed processing system has achieved comparable and sometimes more detailed 3D reconstruction compared to the overlapping HiRISE DTM.

Single Image Super-Resolution Restoration of TGO CaSSIS Colour Images: Demonstration with Perseverance Rover Landing Site and Mars Science Targets

The ExoMars Trace Gas Orbiter (TGO)’s Colour and Stereo Surface Imaging System (CaSSIS) provides multi-spectral optical imagery at 4–5 m/pixel spatial resolution. Improving the spatial resolution of CaSSIS images would allow greater amounts of scientific information to be extracted. In this work, we propose a novel Multi-scale Adaptive weighted Residual Super-resolution Generative Adversarial Network (MARSGAN) for single-image super-resolution restoration of TGO CaSSIS images, and demonstrate how this provides an effective resolution enhancement factor of about 3 times. We demonstrate with qualitative and quantitative assessments of CaSSIS SRR results over the Mars2020 Perseverance rover’s landing site. We also show examples of similar SRR performance over 8 science test sites mainly selected for being covered by HiRISE at higher resolution for comparison, which include many features unique to the Martian surface. Application of MARSGAN will allow high resolution colour imagery from CaSSIS to be obtained over extensive areas of Mars beyond what has been possible to obtain to date from HiRISE.

Research on region selection super resolution restoration algorithm based on infrared micro-scanning optical imaging model

With spring up of infrared imaging related industry, infrared imaging technology has become mainstream development direction of intelligent photoelectrical detection due to its good concealment, wide detection range, high positioning accuracy, long distant penetration, light weight, little volume, low power dissipation and high solidity. However, the features of infrared dim-small target image such as less details and low SNR become bottleneck of infrared image application. How to enhance imaging effect of infrared dim-small target becomes research hotspot. Starting from the point of ‘restoration as foundation’, the theory and technology of infrared dim-small target super-resolution restoration by utilizing the theory and technology of super-resolution restoration are explored in this paper. This paper mainly focuses on the research of super-resolution restoration algorithm of infrared dim-small target based on infrared micro-scanning optical model. Aiming at solving super-resolution restoration problem of infrared dim-small target, the traditional super-resolution restoration algorithm is optimized and the improved algorithm is proposed. Meanwhile, infrared micro-scanning optical model is introduced to break theoretical limit of simple image processing algorithm. And the performance of infrared image super-resolution restoration is improved.

Challenges in Sparse Image Reconstruction

Handling huge amount of data from different sources more so in the images is the latest challenge. One of the solutions to this is sparse representation. The idea of sparsity has been receiving much attention recently from many researchers in the areas such as satellite image processing, signal processing, medical image processing, microscopy image processing, pattern recognition, neuroscience, seismic imaging, etc. Many algorithms have been developed for various areas of sparse representation. The main objective of this paper is to provide a comprehensive study and highlight the challenges in the area of sparse representation which will be helpful for researchers. Also, the current challenges and opportunities of applying sparsity to image reconstruction, namely, image super-resolution, image denoising and image restoration are discussed. This survey on sparse representation categorizes the existing methods into three groups: dictionary learning approach, greedy strategy approximation approach and deep learning approach.

Synchronous nanoscale topographic and chemical mapping by differential-confocal controlled Raman microscopy

Confocal Raman microscopy is currently used for label-free optical sensing and imaging within the biological, engineering, and physical sciences as well as in industry. However, currently these methods have limitations, including their low spatial resolution and poor focus stability, that restrict the breadth of new applications. This paper now introduces differential-confocal controlled Raman microscopy as a technique that fuses differential confocal microscopy and Raman spectroscopy, enabling the point-to-point collection of three-dimensional nanoscale topographic information with the simultaneous reconstruction of corresponding chemical information. The microscope collects the scattered Raman light together with the Rayleigh light, both as Rayleigh scattered and reflected light (these are normally filtered out in conventional confocal Raman systems). Inherent in the design of the instrument is a significant improvement in the axial focusing resolution of topographical features in the image (to ∼ 1 nm ), which, when coupled with super-resolution image restoration, gives a lateral resolution of 220 nm. By using differential confocal imaging for controlling the Raman imaging, the system presents a significant enhancement of the focusing and measurement accuracy, precision, and stability (with an antidrift capability), mitigating against both thermal and vibrational artefacts. We also demonstrate an improved scan speed, arising as a consequence of the nonaxial scanning mode.

Image Super-Resolution Restoration Instance-Based Learning and Iterative Kernel Regression

This paper proposes an image super-resolution restoration algorithm based on example learning and iterative directional kernel regression, which is used to solve the problem that the existing super-resolution restoration algorithm based on example learning cannot effect the restored image has a small degree of matching with the sample library and the problem of image restoration in the presence of noise. Among them, the example learning can achieve basic image restoration. In the directional kernel regression, the estimated smoothing matrix can obtain the minimum mean square estimation after multiple iterations, and further optimize the super-resolution restored image. Simulation results show that the improved algorithm improves the robustness and edge preservation characteristics of the super-resolution restoration image. Compared with the classical algorithm, the algorithm has better visual effects and the root mean square error can be improved over 15%, and reflect the effectiveness of the algorithm.

Nanoscale Spatial Resolution in Far-Field Raman Imaging Using Hyperspectral Unmixing in Combination with Positivity Constrained Super-Resolution.

This work introduces hyper-resolution (HyRes), a numerical approach for spatial resolution enhancement that combines hyperspectral unmixing and super-resolution image restoration (SRIR). HyRes yields a substantial increase in spatial resolution of Raman spectroscopy while simultaneously preserving the undistorted spectral information. The resolving power of this technique is demonstrated on Raman spectroscopic data from a polymer nanowire sample. Here, we demonstrate an achieved resolution of better than 14 nm, a more than eightfold improvement on single-channel image-based SRIR and better than regular far-field Raman spectroscopy, and comparable to near-field probing techniques.

Image Super-Resolution Using Capsule Neural Networks

Convolutional neural networks (CNNs) have been widely applied in super-resolution (SR) and other image restoration tasks. Recently, Hinton et al. proposed capsule neural networks to resolve the problem of viewpoint variations in image classification tasks. Each capsule is represented as either a vector or a matrix to encode more object information, such as position, size, direction, etc. Instead of detecting specific features, these capsule neural networks search for the most relevant features using an iterative process. Therefore, capsule neural networks require fewer parameters compared to traditional neural networks. Inspired by these advances, we make use of a capsule neural network to exploit more potential features for image SR. In this paper, we develope two frameworks: the Capsule Image Restoration Neural Network (CIRNN) and the Capsule Attention and Reconstruction Neural Network (CARNN), to incorporate capsules into image SR convolutional neural networks. The CIRNN takes advantage of the rich information encoded in the capsules to reconstruct accurate high-resolution images. The CARNN generates SR attention features by utilizing the robust segmentation capability of the capsules. Our experiments show that both frameworks can enhance SR for most testing datasets. The CIRNN performs better than the CARNN and can achieve better performance than other traditional CNN methods with a similar amount of parameters.

Wavelet-based segmentation on the sphere

Segmentation, a useful/powerful technique in pattern recognition, is the process of identifying object outlines within images. There are a number of efficient algorithms for segmentation in Euclidean space that depend on the variational approach and partial differential equation modelling. Wavelets have been used successfully in various problems in image processing, including segmentation, inpainting, noise removal, super-resolution image restoration, and many others. Wavelets on the sphere have been developed to solve such problems for data defined on the sphere, which arise in numerous fields such as cosmology and geophysics. In this work, we propose a wavelet-based method to segment images on the sphere, accounting for the underlying geometry of spherical data. Our method is a direct extension of the tight-frame based segmentation method used to automatically identify tube-like structures such as blood vessels in medical imaging. It is compatible with any arbitrary type of wavelet frame defined on the sphere, such as axisymmetric wavelets, directional wavelets, curvelets, and hybrid wavelet constructions. Such an approach allows the desirable properties of wavelets to be naturally inherited in the segmentation process. In particular, directional wavelets and curvelets, which were designed to efficiently capture directional signal content, provide additional advantages in segmenting images containing prominent directional and curvilinear features. We present several numerical experiments, applying our wavelet-based segmentation method, as well as the common K-means method, on real-world spherical images, including an Earth topographic map, a light probe image, solar data-sets, and spherical retina images. These experiments demonstrate the superiority of our method and show that it is capable of segmenting different kinds of spherical images, including those with prominent directional features. Moreover, our algorithm is efficient with convergence usually within a few iterations.

CT Super-Resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble (GAN-CIRCLE).

In this paper, we present a semi-supervised deep learning approach to accurately recover high-resolution (HR) CT images from low-resolution (LR) counterparts. Specifically, with the generative adversarial network (GAN) as the building block, we enforce the cycle-consistency in terms of the Wasserstein distance to establish a nonlinear end-to-end mapping from noisy LR input images to denoised and deblurred HR outputs. We also include the joint constraints in the loss function to facilitate structural preservation. In this process, we incorporate deep convolutional neural network (CNN), residual learning, and network in network techniques for feature extraction and restoration. In contrast to the current trend of increasing network depth and complexity to boost the imaging performance, we apply a parallel 1×1 CNN to compress the output of the hidden layer and optimize the number of layers and the number of filters for each convolutional layer. The quantitative and qualitative evaluative results demonstrate that our proposed model is accurate, efficient and robust for super-resolution (SR) image restoration from noisy LR input images. In particular, we validate our composite SR networks on three large-scale CT datasets, and obtain promising results as compared to the other state-of-the-art methods.

Super resolution reconstruction algorithm of video image based on deep self encoding learning

Super resolution reconstruction of video image is a research hotspot in the field of image processing, and it is widely used in video surveillance, image processing, criminal analysis and other fields. Super resolution image reconstruction can reconstruct a high resolution image from low resolution images, and this technology has become a research hotspot in the field of image processing. In recent years, deep learning has been developed rapidly in the field of multimedia processing, and image super resolution restoration technology based on deep learning has gradually become the mainstream technology. In view of the existing image super-resolution algorithm problems, such as more parameters, larger amount of calculation, longer training time, blurred image texture, we use the deep self-coding learning method to improve it. We analyze the advantages and disadvantages of the existing technology from the network type, network structure, training methods and so on, and sort out the development of the technology. The experimental results show that the improved network model achieves better super-resolution results, and the subjective visual effect and objective evaluation index are improved obviously, and the image sharpness and edge sharpness are improved obviously.

Image super-resolution via Gaussian scale patch group sparse representation

This passage puts forward a Gaussian scale patch group sparse representation method, to solve the shortage problem of traditional image super-resolution restoration schemes. Our image reconstruction method is focused on the optimisation of sparse representation method model, which brings the method and performance improvement to image sparse reconstruction. The overall framework of our approach is as follows. First of all, we utilised the nonlocal similar patches to extract the patch groups, and then using the simultaneous sparse coding to develop a nonlocal extension of Gaussian scale mixture model. In the end, we integrate the patch group model and Gaussian scale sparsity model into encoding framework. The experimental simulation results show that the proposed framework method can both maintain the clarity of the edge and also inhibit the bad artefacts. Our method often provides a higher subjective/objective quality of reconstructed images than other competing methods.

自外而内的单幅图像超分辨率复原算法

自外而内的单幅图像超分辨率复原算法

Improving spatial resolution of confocal Raman microscopy by super-resolution image restoration.

A new super-resolution image restoration confocal Raman microscopy method (SRIR-RAMAN) is proposed for improving the spatial resolution of confocal Raman microscopy. This method can recover the lost high spatial frequency of the confocal Raman microscopy by using Poisson-MAP super-resolution imaging restoration, thereby improving the spatial resolution of confocal Raman microscopy and realizing its super-resolution imaging. Simulation analyses and experimental results indicate that the spatial resolution of SRIR-RAMAN can be improved by 65% to achieve 200 nm with the same confocal Raman microscopy system. This method can provide a new tool for high spatial resolution micro-probe structure detection in physical chemistry, materials science, biomedical science and other areas.

Impact of detector-element active-area shape and fill factor on super-resolution

In many undersampled imaging systems, spatial integration from the individual detector elements is the dominant component of the system point spread function (PSF). Conventional focal plane arrays (FPAs) utilize square detector elements with a nearly 100% fill factor, where fill factor is defined as the fraction of the detector element area that is active in light detection. A large fill factor is generally considered to be desirable because more photons are collected for a given pitch, and this leads to a higher signal-to-noise-ratio (SNR). However, the large active area works against super-resolution (SR) image restoration by acting as an additional low pass filter in the overall PSF when modeled on the SR sampling grid. A high fill factor also tends to increase blurring from pixel cross-talk. In this paper, we study the impact of FPA detector-element shape and fill factor on SR. A detailed modulation transfer function analysis is provided along with a number of experimental results with both simulated data and real data acquired with a midwave infrared (MWIR) imaging system. We demonstrate the potential advantage of low fill factor detector elements when combined with SR image restoration. Our results suggest that low fill factor circular detector elements may be the best choice. New video results are presented using robust adaptive Wiener filter SR processing applied to data from a commercial MWIR imaging system with both high and low detector element fill factors.

Blind super-resolution restoration with frame-by-frame nonparametric blur estimation

In this paper, we investigate super-resolution image restoration from multiple images, which are possibly degraded with large nonparametric motion blur. The blur kernel for each input image is separately estimated. This is unlike many existing super-resolution algorithms, which assume identical blur kernel for all input images. We also do not make any restrictions on the motion field among images; that is, we estimate dense motion field without simplifications such as parametric motion. We present a two-step algorithm: In the first step, each input image is deblurred using its estimated blur kernel. In the second step, multi-frame super-resolution restoration is applied to the deblurred images. Because the estimated blur kernels may not be accurate, we propose a weighted cost function for the super-resolution restoration step, where a weight associated with an input image reflects the reliabilities of the corresponding kernel estimate and deblurred image. We provide experimental results with both simulated and real data, and show the effectiveness and robustness of the proposed method compared to some alternative approaches and state-of-the-art methods.