Proposed Super-resolution Method Research Articles

Exploiting incoherent synthetic apertures in integral imaging for optical super-resolution.

Integral imaging (InIm) working with a pixelated device (e.g., a display panel) and a microlens array (MLA) suffers from low spatial resolution because of a significant trade-off between the spatial and angular resolution. The system bandwidth is presumed to be limited by the Nyquist frequency set by the pixel pitch. This study demonstrates that InIm intrinsically works in an incoherent synthetic aperture (ISA) manner with unexploited resolution capabilities. The sampling shifts between lenslets can be controlled and utilized to construct "computational galvos" to introduce varying aliasing; as a result, the Nyquist frequency is broken for optical super-resolution (SR). In particular, an InIm system can be configured for an N-fold oversampling rate with N lenslets. Furthermore, in an InIm display, the fill factor of a pixel's emitting area is always lower than 100%, so the bandwidth limit set by the pixel shape, i.e., two times the Nyquist frequency, is loosened. An InIm display prototype was built with an oversampling rate of four and a pixel fill factor of 75%. In the experiment, the proposed SR method achieved a 2.12 times resolution without dynamic devices or time-multiplexing.

Super-resolution reconstruction for early cervical cancer magnetic resonance imaging based on deep learning

This study aims to develop a super-resolution (SR) algorithm tailored specifically for enhancing the image quality and resolution of early cervical cancer (CC) magnetic resonance imaging (MRI) images. The proposed method is subjected to both qualitative and quantitative analyses, thoroughly investigating its performance across various upscaling factors and assessing its impact on medical image segmentation tasks. The innovative SR algorithm employed for reconstructing early CC MRI images integrates complex architectures and deep convolutional kernels. Training is conducted on matched pairs of input images through a multi-input model. The research findings highlight the significant advantages of the proposed SR method on two distinct datasets at different upscaling factors. Specifically, at a 2× upscaling factor, the sagittal test set outperforms the state-of-the-art methods in the PSNR index evaluation, second only to the hybrid attention transformer, while the axial test set outperforms the state-of-the-art methods in both PSNR and SSIM index evaluation. At a 4× upscaling factor, both the sagittal test set and the axial test set achieve the best results in the evaluation of PNSR and SSIM indicators. This method not only effectively enhances image quality, but also exhibits superior performance in medical segmentation tasks, thereby providing a more reliable foundation for clinical diagnosis and image analysis.

Battery Energy Storage System Planning Based on Super-Resolution Source-Load Uncertainty Reconstruction

Low-time resolution electricity data have been used to drive battery energy storage system (BESS) planning due to data barriers. However, the coarse-resolution time series cannot reflect real power variation, and the planning results may be inappropriate due to the unrealistic representation of source-load uncertainties. To this end, this paper proposes a BESS planning method based on super-resolution (SR) source-load uncertainty reconstruction. The Gramian angular field (GAF) is employed to transform the 1D time series into 2D images to enrich the time dependency of signals. With advanced computer vision (CV), the entropy decrease generative adversarial network (EDGAN) based on information-growth attention is designed to recover the source-load profile from low resolution to high resolution. To depict source-load uncertainties in BESS sizing, a two-stage multi-distributionally robust optimization (2S-MDRO) is established under SR ambiguity sets. Case studies validated the accuracy of the proposed SR method and proved the effectiveness of the proposed sizing model. The recommended time resolution for BESS planning is presented in the comparative results.

A Lightweight Remote Sensing Image Super-Resolution Method and Its Application in Smart Cities

With the growth of urban population, a series of urban problems have emerged, and how to speed up smart city construction has received extensive attention. Remote sensing images have the advantages of wide spatial coverage and rich information, and it is suitable for use as research data for smart cities. However, due to limitations in the imaging sensor conditions and complex weather, remote sensing images face the problems of insufficient resolution and cloud occlusion, which cannot meet the resolution requirements of smart city tasks. The remote sensing image super-resolution (SR) technique can improve the details and texture information without upgrading the imaging sensor system, which becomes a feasible solution for the above problems. In this paper, we propose a novel remote sensing image super-resolution method which leverages the texture features from internal and external references to help with SR reconstruction. We introduce the transformer attention mechanism to select and extract parts of texture features with high reference values to ensure that the network is lightweight, effective, and easier to deploy on edge computing devices. In addition, our network can automatically learn and adjust the alignment angles and scales of texture features for better SR results. Extensive comparison experiments show that our proposed method achieves superior performance compared with several state-of-the-art SR methods. In addition, we also evaluate the application value of our proposed SR method in urban region function recognition in smart cities. The dataset used in this task is low-quality. The comparative experiment between the original dataset and the SR dataset generated by our proposed SR method indicates that our method can effectively improve the recognition accuracy.

A No-Reference CNN-Based Super-Resolution Method for KOMPSAT-3 Using Adaptive Image Quality Modification

In recent years, research on increasing the spatial resolution and enhancing the quality of satellite images using the deep learning-based super-resolution (SR) method has been actively conducted. In a remote sensing field, conventional SR methods required high-quality satellite images as the ground truth. However, in most cases, high-quality satellite images are difficult to acquire because many image distortions occur owing to various imaging conditions. To address this problem, we propose an adaptive image quality modification method to improve SR image quality for the KOrea Multi-Purpose Satellite-3 (KOMPSAT-3). The KOMPSAT-3 is a high performance optical satellite, which provides 0.7-m ground sampling distance (GSD) panchromatic and 2.8-m GSD multi-spectral images for various applications. We proposed an SR method with a scale factor of 2 for the panchromatic and pan-sharpened images of KOMPSAT-3. The proposed SR method presents a degradation model that generates a low-quality image for training, and a method for improving the quality of the raw satellite image. The proposed degradation model for low-resolution input image generation is based on Gaussian noise and blur kernel. In addition, top-hat and bottom-hat transformation is applied to the original satellite image to generate an enhanced satellite image with improved edge sharpness or image clarity. Using this enhanced satellite image as the ground truth, an SR network is then trained. The performance of the proposed method was evaluated by comparing it with other SR methods in multiple ways, such as edge extraction, visual inspection, qualitative analysis, and the performance of object detection. Experimental results show that the proposed SR method achieves improved reconstruction results and perceptual quality compared to conventional SR methods.

Wavelet-based super resolution of downsampling-based compressed video

Downsampling performed prior to encoding in video compression discards the high-frequency information of the original video frames. This necessitates super resolution (SR) to recover the lost information after decoding. In this paper, we propose a framework for the downsampling-based video compression problem along with a method for wavelet-based SR. We develop a framework by integrating the downsampling and transform steps of the conventional models. We then use the proposed SR method implemented directly on the wavelet sub-bands at the decoder side. Experimental results demonstrate the superiority of the proposed method in comparison with the conventional techniques employed for resolution enhancement of decoded videos.

Super-Resolution of Cardiac MR Cine Imaging using Conditional GANs and Unsupervised Transfer Learning.

High-resolution (HR), isotropic cardiac Magnetic Resonance (MR) cine imaging is challenging since it requires long acquisition and patient breath-hold times. Instead, 2D balanced steady-state free precession (SSFP) sequence is widely used in clinical routine. However, it produces highly-anisotropic image stacks, with large through-plane spacing that can hinder subsequent image analysis. To resolve this, we propose a novel, robust adversarial learning super-resolution (SR) algorithm based on conditional generative adversarial nets (GANs), that incorporates a state-of-the-art optical flow component to generate an auxiliary image to guide image synthesis. The approach is designed for real-world clinical scenarios and requires neither multiple low-resolution (LR) scans with multiple views, nor the corresponding HR scans, and is trained in an end-to-end unsupervised transfer learning fashion. The designed framework effectively incorporates visual properties and relevant structures of input images and can synthesise 3D isotropic, anatomically plausible cardiac MR images, consistent with the acquired slices. Experimental results show that the proposed SR method outperforms several state-of-the-art methods both qualitatively and quantitatively. We show that subsequent image analyses including ventricle segmentation, cardiac quantification, and non-rigid registration can benefit from the super-resolved, isotropic cardiac MR images, to produce more accurate quantitative results, without increasing the acquisition time. The average Dice similarity coefficient (DSC) for the left ventricular (LV) cavity and myocardium are 0.95 and 0.81, respectively, between real and synthesised slice segmentation. For non-rigid registration and motion tracking through the cardiac cycle, the proposed method improves the average DSC from 0.75 to 0.86, compared to the original resolution images.

Super-resolution Image Reconstruction from Projections Using Non-local and Local Regularizations

Abstract This article presents a super-resolution (SR) method dedicated to tomographic imaging, where an image is reconstructed from projections obtained with low-resolution detectors. In this work, upscaling the image resolution is performed by backprojecting the projection measurements into the high-resolution image space modeled on a finer grid. Since this upscaling process often creates irregular pixels, it is important to employ regularizers that can reduce the irregular pixels while preserving fine details. Here we consider two different types of regularizers, non-local and local regularizers, each of which has been independently used for image reconstruction and is known to have its own advantages and disadvantages depending on the edge structures in the underlying image. To achieve a good compromise between the two types of regularizers, we selectively combine them using a space-variant weighting factor, which is systematically determined by our own criterion to classify edges. The experimental results show that our proposed SR method improves the reconstruction accuracy in various image quality assessments and has the potential to be useful in a wide range of imaging applications.

Super Resolution Based an Adaptive Two Pass Normalized Convolution Using Multiple Low Resolution Images

Super resolution (SR) using signal processing is technique to identify the missing high frequency contents due to limitations of imaging hardware, accomplished by using spatio-temporal information available from LR images. The multi-image SR methods effort to reinstate High Resolution (HR) image commencing numerous wraparound artefacted and degraded Low Resolution (LR) images. In past three decades various researchers contributed in the field, but all are intuitive SR mechanisms. The proposed SR method is based on several LR images of the sight to be super resolved. Registration step of SR applies Keren’s algorithm which is iterative and more accurate. Registration algorithm findings include both translational and rotational motion parameters, where rotation angle is obtained by using an affine transformation produced using Gaussian pyramid. The reconstruction is accomplished by structure-adaptive Normalized Convolution with double pass (SANC-D), using first robust and then adaptive approach. Subjective and objective analysis of the results of reconstruction indicates that the algorithm gives better super resolution images than the others at the expense of execution time of algorithm and it is system dependent.

Cost-Efficient Super-Resolution Hardware Using Local Binary Pattern Classification and Linear Mapping for Real-Time 4K Conversion

We propose a new hardware-friendly super-resolution (SR) algorithm using computationally simple feature extraction and regression methods, i.e., local binary pattern (LBP) and linear mapping, respectively. The proposed method pre-trains dedicated linear mapping kernels for different texture types of low-resolution (LR) image patches where the texture type is classified based on LBP features. On inference operation, a high-resolution (HR) image patch is reconstructed by multiplying an LR image patch with a linear mapping kernel, which is inferred by the LBP feature class of the corresponding LR patch. Since, the LBP is a highly efficient feature extraction operator for local texture classification, our method is extremely fast and power-efficient while showing competitive reconstruction quality to the latest machine learning-based SR techniques. We also present a fully pipe-lined hardware architecture and its implementation for real-time operations of the proposed SR method. The proposed SR algorithm has been implemented on a field-programmable-gate-array (FPGA) platform including Xilinx KCU105 that can process 63 frames-per-second (fps) while converting full-high-definition (FHD) images to 4K ultra-high-definition (UHD) images. Extensive experimental results show that the proposed proposed algorithm and its hardware implementation can achieve high reconstruction performance compared to the latest machine-learning-based SR methods while utilizing minimum hardware resources, thereby having remarkably less computational complexity. Sometimes, the latest deep-learning-based SR approaches offer slightly higher reconstruction quality, but they require significantly larger amount of hardware resources than the proposed method.

Robust MR image super‐resolution reconstruction with cross‐modal edge‐preserving regularization

AbstractIn this article, we propose a novel image super‐resolution (SR) reconstruction method in the field of magnetic resonance imaging, which is based on a cross‐modal edge‐preserving regularization integrating the internal gradient prior from the target‐modal image itself and the external gradient prior from the reference‐modal image obtained by pre‐scan in many medical imaging scenes. The reference‐modal image is a high‐resolution guidance image that has much shareable information such as gradient orientation on edge regions, which can be used to improve the image resolution of the target modal. In addition, to be robust against the misalignment between the target‐modal image and reference‐modal image, a multimodal registration is incorporated in the SR reconstruction process. In this work, the proposed SR method can be formulated as an alternating optimization problem, that is, the target‐modal and reference‐modal images are alternately updated through iterations. Experimental results on simulated and realistic images show the superior performance of the proposed approach over several state‐of‐the‐art SR techniques.

Super-Resolution Simulation for Real-Time Prediction of Urban Micrometeorology

We propose a super-resolution (SR) simulation system that consists of a physics-based meteorological simulation and an SR method based on a deep convolutional neural network (CNN). The CNN is trained using pairs of high-resolution (HR) and low-resolution (LR) images created from meteorological simulation results for different resolutions so that it can map LR simulation images to HR ones. The proposed SR simulation system, which performs LR simulations, can provide HR prediction results in much shorter operating cycles than those required for corresponding HR simulation prediction system. We apply the SR simulation system to urban micrometeorology, which is strongly affected by buildings and human activity. Urban micrometeorology simulations that need to resolve urban buildings are computationally costly and thus cannot be used for operational real-time predictions even when run on supercomputers. We performed HR micrometeorology simulations on a supercomputer to obtain datasets for training the CNN in the SR method. It is shown that the proposed SR method can be used with a spatial scaling factor of 4 and that it outperforms conventional interpolation methods by a large margin. It is also shown that the proposed SR simulation system has the potential to be used for operational urban micrometeorology predictions.

Super-Resolution Reconstruction and Its Application Based on Multilevel Main Structure and Detail Boosting

Vivid main structure and rich texture detail are important factors with which to determine the quality of high-resolution images after super-resolution (SR) reconstruction. Owing to the loss of high-frequency information in the process of SR reconstruction and the limitation of the accurate estimation of the unknown information in the inversion process, a gap still exists between the high-resolution image and the real image. The main structure can better preserve the edge structure of the image, and detail boosting can compensate for the missing high-frequency information in the reconstruction process. Therefore, a novel single remote-sensing image SR reconstruction method based on multilevel main structure and detail boosting (MMSDB-SR) is put forward in this paper. First, the multilevel main structure was obtained based on the decomposition of the remote-sensing image through use of the relative total variation model. Subsequently, multilevel texture detail information was obtained by a difference process. Second, the multilevel main structure and texture detail were reconstructed separately. The detail-boosting function was used to compensate for the missing high-frequency details in the reconstruction process. Finally, the high-resolution remote-sensing image with clear edge and rich texture detail can be obtained by fusing the multilevel main structure and texture-detail information. The experimental results show that the reconstructed high-resolution image has high clarity, high fidelity, and multi-detail visual effects, and the objective evaluation index exhibits significant improvement. Actual results show an average gain in entropy of up to 0.34 dB for an up-scaling of 2. Real results show an average gain in enhancement measure evaluation of up to 2.42 for an up-scaling of 2. The robustness and universality of the proposed SR method are verified.

Learning recurrent residual regressors for single image super-resolution

Abstract Example regression-based single image super-resolution (SR) technique has been recognized as an effective way to produce a high-quality image with finer details from one low-resolution (LR) input. However, most current popular approaches usually establish the mappings from the LR feature space to the final HR one in one-pass scheme, which is insufficient to represent the complicated mapping relationship well. In this paper, we propose a novel single image SR framework by learning a group of linear residual regressors in a boosting manner so as to alleviate the gap between the underlying mappings and estimated mappings. In the training stage, we begin with the learning of a set of linear regressors by integrating the K-SVD dictionary learning algorithm and the ridge regression, and then further improve the HR estimate accuracy by learning multi-round residual regressors from the estimated errors in a cascade manner. Accordingly, in the testing stage more details can be gradually added into the input LR image by applying the learned multi-round residual regressors to SR reconstruction. The proposed SR method is fundamentally coarse-to-fine. Experimental results carried out on six publicly available datasets indicate that the proposed SR framework achieves promising performance in comparing with other state-of-the-art competitors in terms of both subjective and objective equality assessments.

CISRDCNN: Super-resolution of compressed images using deep convolutional neural networks

In recent years, much research has been conducted on image super-resolution (SR). To the best of our knowledge, however, few SR methods were concerned with compressed images. The SR of compressed images is a challenging task due to the complicated compression artifacts, while many images suffer from them in practice. The intuitive solution for this difficult task is to decouple it into two sequential but independent subproblems, i.e., compression artifacts reduction (CAR) and SR. Nevertheless, some useful details may be removed in CAR stage, which is contrary to the goal of SR and makes the SR stage more challenging. In this paper, an end-to-end trainable deep convolutional neural network is designed to perform SR on compressed images (CISRDCNN), which reduces compression artifacts and improves image resolution jointly. Experiments on compressed images produced by JPEG (we take the JPEG as an example in this paper) demonstrate that the proposed CISRDCNN yields state-of-the-art SR performance on commonly used test images and imagesets. The results of CISRDCNN on real low quality web images are also very impressive, with obvious quality enhancement. Further, we explore the application of the proposed SR method in low bit-rate image coding, leading to better rate-distortion performance than JPEG.

Learning local dictionaries and similarity structures for single image super-resolution

The central task of reconstruction-based single image super-resolution (SR) approaches is to design an effective prior to well pose the solution to unknown up-sampled image. In this paper, we present a novel single image SR method by learning a set of local dictionaries and non-local similar structures from the input low-resolution (LR) image itself. The local dictionaries are learned by segmenting structurally different regions into different clusters and then training an individual dictionary for each cluster. With the learned dictionaries and similar information, each HR pixel in the expected HR image is estimated as the weighted average of a non-local dictionary (NLD)-based regression which assembles the local structural regularity and the non-local similar redundancies. We further transform the proposed NLD-based regression model into a unified regularization term for a maximum a posteriori probability (MAP) based SR framework. Thorough experimental results carried out on five publicly available datasets indicate that the proposed SR method is promising in producing high-quality images with finer details and sharper edges in terms of both quantitative and perceptual quality assessments.

Optimized multiple linear mappings for single image super-resolution

Learning piecewise linear regression has been recognized as an effective way for example learning-based single image super-resolution (SR) in literature. In this paper, we employ an expectation–maximization (EM) algorithm to further improve the SR performance of our previous multiple linear mappings (MLM) based SR method. In the training stage, the proposed method starts with a set of linear regressors obtained by the MLM-based method, and then jointly optimizes the clustering results and the low- and high-resolution subdictionary pairs for regression functions by using the metric of the reconstruction errors. In the test stage, we select the optimal regressor for SR reconstruction by accumulating the reconstruction errors of m-nearest neighbors in the training set. Thorough experimental results carried on six publicly available datasets demonstrate that the proposed SR method can yield high-quality images with finer details and sharper edges in terms of both quantitative and perceptual image quality assessments.

A Mixed Non‐local Prior Model for Image Super‐resolution Reconstruction

Generating high-resolution image from a set of degraded low-resolution images is a challenge problem in image processing. Due to the ill-posed nature of Super-resolution (SR), it is necessary to find an effective image prior model to make it well-posed. For this purpose, we propose a mixed non-local prior model by adaptively combining the non-local total variation and non-local H1 models, and establish a multi-frame SR method based on this mixed non-local prior model. The unknown Highresolution (HR) image, motion parameters and hyperparameters related to the new prior model and noise statistics are determined automatically, resulting in an unsupervised super-resolution method. Extensive experiments demonstrate the effectiveness of the proposed SR method, which can not only preserve image details better but also suppress noise better.

SRLSP: A Face Image Super-Resolution Algorithm Using Smooth Regression With Local Structure Prior

The performance of traditional face recognition systems is sharply reduced when encountered with a low-resolution (LR) probe face image. To obtain much more detailed facial features, some face super-resolution (SR) methods have been proposed in the past decade. The basic idea of a face image SR is to generate a high-resolution (HR) face image from an LR one with the help of a set of training examples. It aims at transcending the limitations of optical imaging systems. In this paper, we regard face image SR as an image interpolation problem for domain-specific images. A missing intensity interpolation method based on smooth regression with a local structure prior (LSP), named SRLSP for short, is presented. In order to interpolate the missing intensities in a target HR image, we assume that face image patches at the same position share similar local structures, and use smooth regression to learn the relationship between LR pixels and missing HR pixels of one position patch. Performance comparison with the state-of-the-art SR algorithms on two public face databases and some real-world images shows the effectiveness of the proposed method for a face image SR in general. In addition, we conduct a face recognition experiment on the extended Yale-B face database based on the super-resolved HR faces. Experimental results clearly validate the advantages of our proposed SR method over the state-of-the-art SR methods in face recognition application.

Super-resolving barcode images with an edge-preserving variational Bayesian framework

Limited resolution, blurring, warping, and additive noise associated with image acquisition and storage often make the barcode image degraded, generating low-resolution (LR) barcode images. Barcodes in degraded LR images are difficult to recognize. The goal of this paper is to introduce the potential of super-resolution (SR) technique in conquering the aforementioned challenges, and a variational Bayesian SR method is proposed in this work. Different from the previous work, here, the high-resolution barcode image is estimated through its corresponding posterior probability distribution. The barcode image is made of some homogeneous regions separated by sharp edges, and sometimes it is anisotropic. A universal prior probability distribution was proposed for the barcode image by considering these characteristics. Mathematically, the efficiency of this prior distribution is demonstrated, which can preserve sharp edges and suppress artifacts in the reconstructed barcode images. Moreover, by using the variational Bayesian framework, the motion parameters and hyperparameters can be estimated accurately and efficiently, ensuring the success of the SR technique. In order to overcome the difficulty caused by nonlinearity, the Taylor expansion method is introduced to solve the proposed SR problem. Eventually, the simulated and real data experiments show the encouraging performance of the proposed SR method. It increases certainly the reconstruction quality, and could be considerably robust against blur and noise. It is believed that the variational SR technique in the barcode auto-identification technique should open a further perspective of coping with technology challenge.