Color Image Super-resolution Research Articles

CGIHE-VDSR: Color global image histogram equalization with very deep super resolution networks for color image super resolution

Medical and satellite image analysis require incredibly high resolution. Super-resolution combines several low-resolution images of the same scene to generate a high-resolution image. The Super resolution employing deep learning techniques still has an illumination issue. This paper proposes a novel CGIHE-VDSR algorithm that integrates the Very Deep Super Resolution (VDSR) Network with Color Global Image Histogram Equalization (CGIHE) to improve image resolution. In the proposed method, the low-resolution image is first histogram equalized using the CGIHE algorithm. Then, the VDSR network is applied to the histogram equalized image for super-resolution. The comparison of real-time data with the benchmark images is done using the proposed algorithm in the MATLAB platform. The PSNR and SSIM metrics demonstrate that the super resolution image obtained using the proposed method is significantly better than the existing methods.

Multiscale Attention Fusion for Depth Map Super-Resolution Generative Adversarial Networks.

Color images have long been used as an important supplementary information to guide the super-resolution of depth maps. However, how to quantitatively measure the guiding effect of color images on depth maps has always been a neglected issue. To solve this problem, inspired by the recent excellent results achieved in color image super-resolution by generative adversarial networks, we propose a depth map super-resolution framework with generative adversarial networks using multiscale attention fusion. Fusion of the color features and depth features at the same scale under the hierarchical fusion attention module effectively measure the guiding effect of the color image on the depth map. The fusion of joint color-depth features at different scales balances the impact of different scale features on the super-resolution of the depth map. The loss function of a generator composed of content loss, adversarial loss, and edge loss helps restore clearer edges of the depth map. Experimental results on different types of benchmark depth map datasets show that the proposed multiscale attention fusion based depth map super-resolution framework has significant subjective and objective improvements over the latest algorithms, verifying the validity and generalization ability of the model.

Large-scale individual building extraction from open-source satellite imagery via super-resolution-based instance segmentation approach

Building footprint is a primary dataset of an urban geographic information system (GIS) database. Therefore, it is essential to establish a robust and automated framework for large-scale building extraction. However, the characteristic of remote sensing images complicates the application of the instance segmentation method based on the Mask R-CNN model, which ought to be improved toward extracting and fusing multi-scale features. Moreover, open-source satellite image datasets with wider spatial coverage and temporal resolution than high-resolution images may exhibit different coloration and resolution. This study proposes a large-scale building extraction framework based on super-resolution (SR) and instance segmentation using a relatively lower-resolution (>0.6 m) open-sourced dataset. The framework comprises four steps: color normalization and image super-resolution, scene classification, building extraction, and scene mosaicking. We took Hyogo Prefecture, Japan (19,187 km2) as a test area and extracted 1,726,006 (29.12 km2) of the 3,301,488 buildings (32.46 km2), where the number of buildings and footprint area increased by 3.0 % and 5.0 % respectively. The result indicated that the color normalization and image super-resolution could improve the visual quality of open-source satellite images and contribute to building extraction accuracy. Moreover, the improved Mask R-CNN based on Multi-Path Vision Transformer (MPViT) backbone achieved F1 scores of 0.71, 0.70, 0.81, and 0.67 for non-built-up, rural, suburban, and urban areas, respectively, which is better than those of the baseline model and other mainstream instance segmentation approaches. This study demonstrates the potential of acquiring acceptable building footprint maps from open-source satellite images, which has significant practical implications.

Retraction Note: Rainfall soil erosion based on color image super resolution and rural revitalization design

Retraction Note: Rainfall soil erosion based on color image super resolution and rural revitalization design

RETRACTED ARTICLE: Rainfall soil erosion based on color image super resolution and rural revitalization design

Based on the super-resolution color image, this paper studies the design of rainfall soil erosion and rural revitalization. The steep slope system of the Loess Plateau is not only an important cause of soil erosion by rain, but also an important area for landslides, debris flows, and other natural disasters. In this study, high and steep slope system was used as the research object. Through indoor artificial rainfall experiments, under the conditions of intermittent rainfall, outflow and sediment yield characteristics, soil moisture content changes, and rainfall erosion dynamic parameters, the appearance of steep slope system under various conditions was studied from the beginning of rainfall to slope crack stage. At the same time, the development of agriculture and rural areas is not only an important foundation and support for the realization of national modernization, but also an important factor affecting economic and social development. At present, the development level of China’s agriculture and rural areas is still relatively backward, which is a significant defect in the development of China’s modernization. According to the relevant research results of domestic and foreign researchers, this study takes the main content of Rural Revitalization Strategy as the starting point, and, based on the explanation of development background and implication, deeply analyzes the development background of rural complex model and local economic revitalization strategy. At the same time, taking the rural complex as the main carrier to implement the rural revitalization strategy, this paper studies the theoretical source of the rural complex model in detail, and briefly summarizes the significance and characteristics of the rural complex model. This paper expounds the important role of the development of rural complex facilities in the implementation of China’s rural revitalization strategy.

Hypergraph-regularized sparse representation for single color image super resolution

Sparsity-based single image super resolution method generates the High-Resolution (HR) output via a corresponding dictionary from the Low-Resolution (LR) input. However, most of these existing methods ignore the complementary information from color channels, which causes the loss of a valid prior and the limitation of HR image quality improvement. In this paper, hypergraph regularization is first incorporated with Joint Color Dictionary Training (JCDT) model and HR image reconstruction (HRIR) model. A novel Hypergraph-regularized Sparse coding-based Super Resolution (HG-ScSR) is proposed. This regularization can not only focus on the illuminance information, but also exploit the self-channel and cross-channel information of three color RGB channels from high-resolution image patches. Especially, the complex relationship is explored among every color image patch pixel and the consistency of the similar pixels is enforced. Both simulated and real data experiments verify the higher performance of the proposed HG-ScSR.

Color Image Super-Resolution and Enhancement with Inter-Channel Details at Trivial Cost

Image super-resolution is essential for a variety of applications such as medical imaging, surveillance imaging, and satellite imaging, among others. Traditionally, the most popular color image super-resolution is performed in each color channel independently. In this paper, we show that the super-resolution quality can be further enhanced by exploiting the cross-channel correlation. Inspired by the High-Quality Linear Interpolation (HQLI) demosaicking algorithm by Malvar et al., we design an image super-resolution scheme that integrates intra-channel interpolation with cross-channel details by isotropic linear combinations. Despite its simplicity, our super-resolution method achieves the accuracy comparable with the existing fastest state-of-the-art super-resolution algorithm at 20 times faster speed. It is well applicable to applications that adopt traditional interpolations, for improved visual quality at trivial computation cost. Our comparative study verifies the effectiveness and efficiency of the proposed super-resolution algorithm.

Multi-Frame RGB/NIR Imaging for Low-Light Color Image Super-Resolution

We propose a method for super-resolution (SR) of low-resolution (LR) color images taken in low-light scenes. Our method is based on the multi-frame SR technique, which reconstructs a high-resolution color image by fusing multiple LR images that were taken at different camera positions so as to alter the alignment between scene details and pixels. Previous multi-frame SR methods have implicitly assumed that the LR images could be captured with less noise and blur. However, images taken in low-light scene will contain high level of noise and motion blur, making it difficult to achieve high quality SR. To overcome these problems, we utilize a single sensor that captures red, green, blue (RGB) and near-infrared (NIR) information. We capture multiple raw images of a low-light scene using our RGB/NIR single sensor with an NIR flash unit. Because a NIR flash can provide sufficient illumination for low-light scenes, the structural scene information that, contributes to an effective SR process, can be captured with high quality. With the help of the NIR information, we jointly perform deblurring, denoising and SR of LR RGB/NIR raw images. Our experiments demonstrate the effectiveness of the proposed method using synthetic and real RGB/NIR raw images.

Simultaneous color-depth super-resolution with conditional generative adversarial networks

In this paper, color-depth conditional generative adversarial networks (CDcGAN) are proposed to resolve the problems of simultaneous color image super-resolution and depth image super-resolution in 3D videos. Firstly, a generative network is presented to leverage the mutual information of the low-resolution color image and low-resolution depth image so that they can enhance each other considering their geometric structural similarity in the same scene. Secondly, three auxiliary losses of data loss, total variation loss, and 8-connected gradient difference loss are introduced to train this generative network to ensure that the generated images are close to the real ones in addition to the adversarial loss. Finally, we study the CDcGAN and its variants. Experimental results show that the proposed approach can produce the high-quality color image and depth image from a pair of low-quality images, and it is superior to several other leading methods. Additionally, it has also been used to resolve the problems of concurrent image smoothing and edge detection, as well as the problem of HR-color-image-guided depth super-resolution to show the effectiveness and universality of the proposed method.

Sparse Representation for Color Image Super-Resolution with Image Quality Difference Evaluation

Sparse Representation for Color Image Super-Resolution with Image Quality Difference Evaluation

Sparsity-Based Color Image Super Resolution via Exploiting Cross Channel Constraints.

Sparsity constrained single image super-resolution (SR) has been of much recent interest. A typical approach involves sparsely representing patches in a low-resolution (LR) input image via a dictionary of example LR patches, and then using the coefficients of this representation to generate the high-resolution (HR) output via an analogous HR dictionary. However, most existing sparse representation methods for SR focus on the luminance channel information and do not capture interactions between color channels. In this paper, we extend sparsity-based SR to multiple color channels by taking the color information into account. Edge similarities amongst RGB color bands are exploited as cross channel correlation constraints. These additional constraints lead to a new optimization problem, which is not easily solvable; however, a tractable solution is proposed to solve it efficiently. Moreover, to fully exploit the complementary information among color channels, a dictionary learning method is also proposed specifically to learn color dictionaries that encourage edge similarities. Merits of the proposed method over state of the art are demonstrated both visually and quantitatively using image quality metrics.

Vector Sparse Representation of Color Image Using Quaternion Matrix Analysis

Traditional sparse image models treat color image pixel as a scalar, which represents color channels separately or concatenate color channels as a monochrome image. In this paper, we propose a vector sparse representation model for color images using quaternion matrix analysis. As a new tool for color image representation, its potential applications in several image-processing tasks are presented, including color image reconstruction, denoising, inpainting, and super-resolution. The proposed model represents the color image as a quaternion matrix, where a quaternion-based dictionary learning algorithm is presented using the K-quaternion singular value decomposition (QSVD) (generalized K-means clustering for QSVD) method. It conducts the sparse basis selection in quaternion space, which uniformly transforms the channel images to an orthogonal color space. In this new color space, it is significant that the inherent color structures can be completely preserved during vector reconstruction. Moreover, the proposed sparse model is more efficient comparing with the current sparse models for image restoration tasks due to lower redundancy between the atoms of different color channels. The experimental results demonstrate that the proposed sparse image model avoids the hue bias issue successfully and shows its potential as a general and powerful tool in color image analysis and processing domain.

Nonlocal Mumford-Shah Regularizers for Color Image Restoration

We propose here a class of restoration algorithms for color images, based upon the Mumford-Shah (MS) model and nonlocal image information. The Ambrosio-Tortorelli and Shah elliptic approximations are defined to work in a small local neighborhood, which are sufficient to denoise smooth regions with sharp boundaries. However, texture is nonlocal in nature and requires semilocal/non-local information for efficient image denoising and restoration. Inspired from recent works (nonlocal means of Buades, Coll, Morel, and nonlocal total variation of Gilboa, Osher), we extend the local Ambrosio-Tortorelli and Shah approximations to MS functional (MS) to novel nonlocal formulations, for better restoration of fine structures and texture. We present several applications of the proposed nonlocal MS regularizers in image processing such as color image denoising, color image deblurring in the presence of Gaussian or impulse noise, color image inpainting, color image super-resolution, and color filter array demosaicing. In all the applications, the proposed nonlocal regularizers produce superior results over the local ones, especially in image inpainting with large missing regions. We also prove several characterizations of minimizers based upon dual norm formulations.

Color Image Super Resolution: A Two-Step Approach Based on Geometric Grouplets

Color Image Super Resolution: A Two-Step Approach Based on Geometric Grouplets

Robust Color Image Superresolution: An Adaptive M-Estimation Framework

This paper introduces a new color image superresolution algorithm in an adaptive, robust M-estimation framework. Using a robust error norm in the objective function, and adapting the estimation process to each of the low-resolution frames, the proposed method effectively suppresses the outliers due to violations of the assumed observation model, and results in color superresolution estimates with crisp details and no color artifacts, without the use of regularization. Experiments on both synthetic and real sequences demonstrate the superior performance over using the L2 and L1 error norms in the objective function.