Learning-based Super-resolution Technique Research Articles

DiffLense: a conditional diffusion model for super-resolution of gravitational lensing data

Abstract Gravitational lensing data is frequently collected at low resolution due to instrumental limitations and observing conditions. Machine learning-based super-resolution techniques offer a method to enhance the resolution of these images, enabling more precise measurements of lensing effects and a better understanding of the matter distribution in the lensing system. This enhancement can significantly improve our knowledge of the distribution of mass within the lensing galaxy and its environment, as well as the properties of the background source being lensed. Traditional super-resolution techniques typically learn a mapping function from lower-resolution to higher-resolution samples. However, these methods are often constrained by their dependence on optimizing a fixed distance function, which can result in the loss of intricate details crucial for astrophysical analysis. In this work, we introduce DiffLense, a novel super-resolution pipeline based on a conditional diffusion model specifically designed to enhance the resolution of gravitational lensing images obtained from the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP). Our approach adopts a generative model, leveraging the detailed structural information present in Hubble space telescope (HST) counterparts. The diffusion model, trained to generate HST data, is conditioned on HSC data pre-processed with denoising techniques and thresholding to significantly reduce noise and background interference. This process leads to a more distinct and less overlapping conditional distribution during the model’s training phase. We demonstrate that DiffLense outperforms existing state-of-the-art single-image super-resolution techniques, particularly in retaining the fine details necessary for astrophysical analyses.

Deep learning-based super-resolution gradient echo imaging of the pancreas: Improvement of image quality and reduction of acquisition time

PurposeThe purpose of this study was to evaluate the impact of a deep learning-based super-resolution technique on T1-weighted gradient-echo acquisitions (volumetric interpolated breath-hold examination; VIBE) on the assessment of pancreatic MRI at 1.5 T compared to standard VIBE imaging (VIBESTD). Materials and methodsThis retrospective single-center study was conducted between April 2021 and October 2021. Fifty patients with a total of 50 detectable pancreatic lesion entities were included in this study. There were 27 men and 23 women, with a mean age of 69 ± 13 (standard deviation [SD]) years (age range: 33–89 years). VIBESTD (precontrast, dynamic, postcontrast) was retrospectively processed with a deep learning-based super-resolution algorithm including a more aggressive partial Fourier setting leading to a simulated acquisition time reduction (VIBESR). Image analysis was performed by two radiologists regarding lesion detectability, noise levels, sharpness and contrast of pancreatic edges, as well as regarding diagnostic confidence using a 5-point Likert-scale with 5 being the best. ResultsVIBESR was rated better than VIBESTD by both readers regarding lesion detectability (5 [IQR: 5, 5] vs. 5 [IQR: 4, 5], for reader 1; 5 [IQR: 5, 5] vs. 4 [IQR: 4, 5]) for reader 2; both P <0.001), noise levels (5 [IQR: 5, 5] vs. 5 [IQR: 4, 5] for reader 1; 5 [IQR: 5, 5] vs. 4 [IQR: 4, 5] for reader 2; both P <0.001), sharpness and contrast of pancreatic edges (5 [IQR: 5, 5] vs. 5 [IQR: 4, 5] for reader 1; 5 [IQR: 5, 5] vs. 4 [IQR: 4, 5] for reader 2; both P <0.001), as well as regarding diagnostic confidence (5 [IQR: 5, 5] vs. 5 [IQR: 4, 5] for reader 1; 5 [IQR: 5, 5] vs. 4 [IQR: 4, 5] for reader 2; both P <0.001). There were no significant differences between lesion sizes as measured by the two readers on VIBESR and VIBESTD images (P > 0.05). The mean acquisition time for VIBESTD (15 ± 1 [SD] s; range: 11–16 s) was longer than that for VIBESR (13 ± 1 [SD] s; range: 11–14 s) (P < 0.001). ConclusionOur results indicate that the newly developed deep learning-based super-resolution algorithm adapted to partial Fourier acquisitions has a positive influence not only on shortening the examination time but also on improvement of image quality in pancreatic MRI.

Enhancing hyperspectral image compression using learning-based super-resolution technique

Enhancing hyperspectral image compression using learning-based super-resolution technique

Cost-Efficient Super-Resolution Hardware Using Local Binary Pattern Classification and Linear Mapping for Real-Time 4K Conversion

We propose a new hardware-friendly super-resolution (SR) algorithm using computationally simple feature extraction and regression methods, i.e., local binary pattern (LBP) and linear mapping, respectively. The proposed method pre-trains dedicated linear mapping kernels for different texture types of low-resolution (LR) image patches where the texture type is classified based on LBP features. On inference operation, a high-resolution (HR) image patch is reconstructed by multiplying an LR image patch with a linear mapping kernel, which is inferred by the LBP feature class of the corresponding LR patch. Since, the LBP is a highly efficient feature extraction operator for local texture classification, our method is extremely fast and power-efficient while showing competitive reconstruction quality to the latest machine learning-based SR techniques. We also present a fully pipe-lined hardware architecture and its implementation for real-time operations of the proposed SR method. The proposed SR algorithm has been implemented on a field-programmable-gate-array (FPGA) platform including Xilinx KCU105 that can process 63 frames-per-second (fps) while converting full-high-definition (FHD) images to 4K ultra-high-definition (UHD) images. Extensive experimental results show that the proposed proposed algorithm and its hardware implementation can achieve high reconstruction performance compared to the latest machine-learning-based SR methods while utilizing minimum hardware resources, thereby having remarkably less computational complexity. Sometimes, the latest deep-learning-based SR approaches offer slightly higher reconstruction quality, but they require significantly larger amount of hardware resources than the proposed method.

Resolution enhancement in scanning electron microscopy using deep learning

We report resolution enhancement in scanning electron microscopy (SEM) images using a generative adversarial network. We demonstrate the veracity of this deep learning-based super-resolution technique by inferring unresolved features in low-resolution SEM images and comparing them with the accurately co-registered high-resolution SEM images of the same samples. Through spatial frequency analysis, we also report that our method generates images with frequency spectra matching higher resolution SEM images of the same fields-of-view. By using this technique, higher resolution SEM images can be taken faster, while also reducing both electron charging and damage to the samples.

Zoom based image super-resolution using DCT with LBP as characteristic model

The prime intention of super-resolution (SR) technique is to restore the high-resolution images from one or more low-resolution (LR) images. These images are captured from the same scene with different acquisition systems with different resolution. Because these acquisition systems, images are suffered for an ill-posed problem with low visualization and picture information. Therefore, in this paper, the zoom-based super-resolution approach is proposed for super-resolution of low resolute images which are acquired from different camera zoom-lens. In this approach, three LR images of the same static scene which are acquired using three distinct zoom factors are used. Learning-based SR technique is used to enhance the spatial resolution of these LR images. The training dataset comprises three sets of captured images which are LR images, an enhanced version of LR images-HR1 and enhanced version of HR1 images-HR2. High-frequency details of the super-resolute image are learned in form of the discrete cosine transform (DCT) coefficients of HR training images. Finally, the super-resolved versions of LR observations, captured at different zoom-factors, are combined. The experimental results show that this proposed approach can be applied to various types of natural images in grayscale as well as color. The experimental results also show that this proposed approach performs better than existing approaches.

Alignment-free and high-frequency compensation in face hallucination.

Face hallucination is one of learning-based super resolution techniques, which is focused on resolution enhancement of facial images. Though face hallucination is a powerful and useful technique, some detailed high-frequency components cannot be recovered. It also needs accurate alignment between training samples. In this paper, we propose a high-frequency compensation framework based on residual images for face hallucination method in order to improve the reconstruction performance. The basic idea of proposed framework is to reconstruct or estimate a residual image, which can be used to compensate the high-frequency components of the reconstructed high-resolution image. Three approaches based on our proposed framework are proposed. We also propose a patch-based alignment-free face hallucination. In the patch-based face hallucination, we first segment facial images into overlapping patches and construct training patch pairs. For an input low-resolution (LR) image, the overlapping patches are also used to obtain the corresponding high-resolution (HR) patches by face hallucination. The whole HR image can then be reconstructed by combining all of the HR patches. Experimental results show that the high-resolution images obtained using our proposed approaches can improve the quality of those obtained by conventional face hallucination method even if the training data set is unaligned.

Learning-based super-resolution technique significantly improves detection of coronary artery stenoses on 1.5T whole-heart coronary MRA

Background Whole-heart coronary MRA permits noninvasive detection of coronary artery disease. In the brain MRI, superresolution (SR) technique has emerged as a method to enhance image resolution and improve diagnostic performance. In the current study, we developed a learning-based SR technique optimized for whole-heart coronary MRA (Figure 1). Summarizing briefly, this SR algorithm consists of (1) construction of the dictionary for lowand high-resolution pairs of image patches, and (2) generation of high-resolution coronary MRA images by embedding optimal patches selected form the dictionary by least square fitting. The purpose of this study was to determine if the SR technique can improve the diagnostic performance of 1.5T whole-heart coronary MRA in detecting significant stenosis on catheter coronary angiography.

Manifold learning-based sample selection method for facial image super-resolution

Learning efficiency, related to not only the size of training set but also to the usage of samples, is an important issue of learning-based super-resolution (SR). We propose a face hallucination method with adaptive sample selection ability by learning from the facial manifold. Employing locality preserving projections (LPP) to analyze the intrinsic features on the local facial manifold, our method searches out image patches dynamically in the LPP subspace, which makes the training set tailored to the input patch. Using the selected training set, we develop a patch-based eigen-transformation algorithm to efficiently restore the lost high-frequency components of the low-resolution face image. Experiments on synthetic and real-life images fully demonstrate that the proposed adaptive sample selection SR method can achieve better performance than some state-of-the-art learning-based SR techniques with less computational cost by utilizing a relative small sample, especially under the case of low quality input images.