Super-resolution Reconstruction Technique Research Articles

A lightweight convolutional neural network super-resolution method to improve the quality of target image in space measurement tasks

With the rapid advancement of space technology, space target observation images have become an essential tool for the precise measurement and shape analysis of spacecraft. However, due to the challenging conditions of the space environment, these images often suffer from blurring and distortion, which hampers the effectiveness of spacecraft observation and measurement missions. Although recent progress has been made in super-resolution reconstruction techniques, the limited processing capacity of on-board equipment prevents the direct deployment of these high-complexity methods. In this paper, we propose an efficient and lightweight super-resolution reconstruction algorithm called the Pyramid Frequency-Aware Network for space target observation images. Specifically, we use a divide-and-conquer strategy to separately process low-frequency and high-frequency features, ensuring high-quality feature extraction while reducing the number of parameters. To further improve the model’s ability to capture edge and detailed texture information, we introduce a pyramidal wavelet decomposition and a multi-scale large separable kernel attention model. For high-frequency information, we design an enhanced fusion convolution block that facilitates multi-scale feature extraction and channel mixing. Furthermore, we have established a dataset of space target observation images, which can serve as a valuable reference for future studies on the reconstruction of such images. Extensive experimental results demonstrate that our Pyramid Frequency-Aware Network achieves an excellent balance between peak signal-to-noise ratio, structural similarity index, number of parameters, flops, and running time, both on public standard datasets and our self-built space target observation image dataset. Additionally, the network is lightweight enough to be deployed on resource-constrained equipment, such as satellites.

Contrast-enhanced thin-slice abdominal CT with super-resolution deep learning reconstruction technique: evaluation of image quality and visibility of anatomical structures.

To compare image quality and visibility of anatomical structures on contrast-enhanced thin-slice abdominal CT images reconstructed using super-resolution deep learning reconstruction (SR-DLR), deep learning-based reconstruction (DLR), and hybrid iterative reconstruction (HIR) algorithms. This retrospective study included 54 consecutive patients who underwent contrast-enhanced abdominal CT. Thin-slice images (0.5mm thickness) were reconstructed using SR-DLR, DLR, and HIR. Objective image noise and contrast-to-noise ratio (CNR) for liver parenchyma relative to muscle were assessed. Two radiologists independently graded image quality using a 5-point rating scale for image noise, sharpness, artifact/blur, and overall image quality. They also graded the visibility of small vessels, main pancreatic duct, ureters, adrenal glands, and right adrenal vein on a 5-point scale. SR-DLR yielded significantly lower objective image noise and higher CNR than DLR and HIR (P < .001). The visual scores of SR-DLR for image noise, sharpness, and overall image quality were significantly higher than those of DLR and HIR for both readers (P < .001). Both readers scored significantly higher on SR-DLR than on HIR for visibility for all structures (P < .01), and at least one reader scored significantly higher on SR-DLR than on DLR for visibility for all structures (P < .05). SR-DLR reduced image noise and improved image quality of thin-slice abdominal CT images compared to HIR and DLR. This technique is expected to enable further detailed evaluation of small structures.

A Review of Satellite-Based CO2 Data Reconstruction Studies: Methodologies, Challenges, and Advances

Carbon dioxide is one of the most influential greenhouse gases affecting human life. CO2 data can be obtained through three methods: ground-based, airborne, and satellite-based observations. However, ground-based monitoring is typically composed of sparsely distributed stations, while airborne monitoring has limited coverage and spatial resolution; they cannot fully reflect the spatiotemporal distribution of CO2. Satellite remote sensing plays a crucial role in monitoring the global distribution of atmospheric CO2, offering high observation accuracy and wide coverage. However, satellite remote sensing still faces spatiotemporal constraints, such as interference from clouds (or aerosols) and limitations from satellite orbits, which can lead to significant data loss. Therefore, the reconstruction of satellite-based CO2 data becomes particularly important. This article summarizes methods for the reconstruction of satellite-based CO2 data, including interpolation, data fusion, and super-resolution reconstruction techniques, and their advantages and disadvantages, it also provides a comprehensive overview of the classification and applications of super-resolution reconstruction techniques. Finally, the article offers future perspectives, suggesting that ideas like image super-resolution reconstruction represent the future trend in the field of satellite-based CO2 data reconstruction.

An Effective Res-Progressive Growing Generative Adversarial Network-Based Cross-Platform Super-Resolution Reconstruction Method for Drone and Satellite Images

In recent years, accurate field monitoring has been a research hotspot in the domains of aerial remote sensing and satellite remote sensing. In view of this, this study proposes an innovative cross-platform super-resolution reconstruction method for remote sensing images for the first time, aiming to make medium-resolution satellites capable of field-level detection through a super-resolution reconstruction technique. The progressive growing generative adversarial network (PGGAN) model, which has excellent high-resolution generation and style transfer capabilities, is combined with a deep residual network, forming the Res-PGGAN model for cross-platform super-resolution reconstruction. The Res-PGGAN architecture is similar to that of the PGGAN, but includes a deep residual module. The proposed Res-PGGAN model has two main benefits. First, the residual module facilitates the training of deep networks, as well as the extraction of deep features. Second, the PGGAN structure performs well in cross-platform sensor style transfer, allowing for cross-platform high-magnification super-resolution tasks to be performed well. A large pre-training dataset and real data are used to train the Res-PGGAN to improve the resolution of Sentinel-2’s 10 m resolution satellite images to 0.625 m. Three evaluation metrics, including the structural similarity index metric (SSIM), the peak signal-to-noise ratio (PSNR), and the universal quality index (UQI), are used to evaluate the high-magnification images obtained by the proposed method. The images generated by the proposed method are also compared with those obtained by the traditional bicubic method and two deep learning super-resolution reconstruction methods: the enhanced super-resolution generative adversarial network (ESRGAN) and the PGGAN. The results indicate that the proposed method outperforms all the comparison methods and demonstrates an acceptable performance regarding all three metrics (SSIM/PSNR/UQI: 0.9726/44.7971/0.0417), proving the feasibility of cross-platform super-resolution image recovery.

A Super-Resolution and 3D Reconstruction Method Based on OmDF Endoscopic Images.

In the field of endoscopic imaging, challenges such as low resolution, complex textures, and blurred edges often degrade the quality of 3D reconstructed models. To address these issues, this study introduces an innovative endoscopic image super-resolution and 3D reconstruction technique named Omni-Directional Focus and Scale Resolution (OmDF-SR). This method integrates an Omnidirectional Self-Attention (OSA) mechanism, an Omnidirectional Scale Aggregation Group (OSAG), a Dual-stream Adaptive Focus Mechanism (DAFM), and a Dynamic Edge Adjustment Framework (DEAF) to enhance the accuracy and efficiency of super-resolution processing. Additionally, it employs Structure from Motion (SfM) and Multi-View Stereo (MVS) technologies to achieve high-precision medical 3D models. Experimental results indicate significant improvements in image processing with a PSNR of 38.2902 dB and an SSIM of 0.9746 at a magnification factor of ×2, and a PSNR of 32.1723 dB and an SSIM of 0.9489 at ×4. Furthermore, the method excels in reconstructing detailed 3D models, enhancing point cloud density, mesh quality, and texture mapping richness, thus providing substantial support for clinical diagnosis and surgical planning.

Co-Transplantation-Based Human-Mouse Chimeric Brain Models to Study Human Glial-Glial and Glial-Neuronal Interactions.

Human-mouse chimeric brain models, generated by transplanting human induced pluripotent stem cell (hiPSC)-derived neural cells, are valuable for studying the development and function of human neural cells in vivo. Understanding glial-glial and glial-neuronal interactions is essential for unraveling the complexities of brain function and developing treatments for neurological disorders. To explore these interactions between human neural cells within an intact brain environment, we employe a co-transplantation strategy involving the engraftment of hiPSC-derived neural progenitor cells along with primitive macrophage progenitors into the neonatal mouse brain. This approach creates human-mouse chimeric brains containing human microglia, macroglia (astroglia and oligodendroglia), and neurons. Using super-resolution imaging and 3D reconstruction techniques, we examine the dynamics between human neurons and glia, unveiling human microglia engulfing immature human neurons, microglia pruning synapses of human neurons, and significant interactions between human oligodendrocytes and neurons. Single-cell RNA sequencing analysis of the chimeric brain uncovers a close recapitulation of the human glial progenitor cell population, along with a dynamic stage in astroglial development that mirrors the processes found in the human brain. Furthermore, cell-cell communication analysis highlights significant neuronal-glial and glial-glial interactions, especially the interaction between adhesion molecules neurexins and neuroligins. This innovative co-transplantation model opens up new avenues for exploring the complex pathophysiological mechanisms underlying human neurological diseases. It holds particular promise for studying disorders where glial-neuronal interactions and non-cell-autonomous effects play crucial roles.

MGDUN: An interpretable network for multi-contrast MRI image super-resolution reconstruction

Magnetic resonance imaging (MRI) Super-Resolution (SR) aims to obtain high resolution (HR) images with more detailed information for precise diagnosis and quantitative image analysis. Deep unfolding networks outperform general MRI SR reconstruction methods by providing better performance and improved interpretability, which enhance the trustworthiness required in clinical practice. Additionally, current SR reconstruction techniques often rely on a single contrast or a simple multi-contrast fusion mechanism, ignoring the complex relationships between different contrasts. To address these issues, in this paper, we propose a Model-Guided multi-contrast interpretable Deep Unfolding Network (MGDUN) for medical image SR reconstruction, which explicitly incorporates the well-studied multi-contrast MRI observation model into an unfolding iterative network. Specifically, we manually design an objective function for MGDUN that can be iteratively computed by the half-quadratic splitting algorithm. The iterative MGDUN algorithm is unfolded into a special model-guided deep unfolding network that explicitly takes into account both the multi-contrast relationship matrix and the MRI observation matrix during the end-to-end optimization process. Extensive experimental results on the multi-contrast IXI dataset and the BraTs 2019 dataset demonstrate the superiority of our proposed model, with PSNR reaching 37.3366 and 35.9690 respectively. Our proposed MGDUN provides a promising solution for multi-contrast MR image super-resolution reconstruction. Code is available at https://github.com/yggame/MGDUN.

A Review of GAN-Based Super-Resolution Reconstruction for Optical Remote Sensing Images

High-resolution images have a wide range of applications in image compression, remote sensing, medical imaging, public safety, and other fields. The primary objective of super-resolution reconstruction of images is to reconstruct a given low-resolution image into a corresponding high-resolution image by a specific algorithm. With the emergence and swift advancement of generative adversarial networks (GANs), image super-resolution reconstruction is experiencing a new era of progress. Unfortunately, there has been a lack of comprehensive efforts to bring together the advancements made in the field of super-resolution reconstruction using generative adversarial networks. Hence, this paper presents a comprehensive overview of the super-resolution image reconstruction technique that utilizes generative adversarial networks. Initially, we examine the operational principles of generative adversarial networks, followed by an overview of the relevant research and background information on reconstructing remote sensing images through super-resolution techniques. Next, we discuss significant research on generative adversarial networks in high-resolution image reconstruction. We cover various aspects, such as datasets, evaluation criteria, and conventional models used for image reconstruction. Subsequently, the super-resolution reconstruction models based on generative adversarial networks are categorized based on whether the kernel blurring function is recognized and utilized during training. We provide a brief overview of the utilization of generative adversarial network models in analyzing remote sensing imagery. In conclusion, we present a prospective analysis of forthcoming research directions pertaining to super-resolution reconstruction methods that rely on generative adversarial networks.

Multi-Attention Multi-Image Super-Resolution Transformer (MAST) for Remote Sensing

Deep-learning-driven multi-image super-resolution (MISR) reconstruction techniques have significant application value in the field of aerospace remote sensing. In particular, Transformer-based models have shown outstanding performance in super-resolution tasks. However, current MISR models have some deficiencies in the application of multi-scale information and the modeling of the attention mechanism, leading to an insufficient utilization of complementary information in multiple images. In this context, we innovatively propose a Multi-Attention Multi-Image Super-Resolution Transformer (MAST), which involves improvements in two main aspects. Firstly, we present a Multi-Scale and Mixed Attention Block (MMAB). With its multi-scale structure, the network is able to extract image features from different scales to obtain more contextual information. Additionally, the introduction of mixed attention allows the network to fully explore high-frequency features of the images in both channel and spatial dimensions. Secondly, we propose a Collaborative Attention Fusion Block (CAFB). By incorporating channel attention into the self-attention layer of the Transformer, we aim to better establish global correlations between multiple images. To improve the network’s perception ability of local detailed features, we introduce a Residual Local Attention Block (RLAB). With the aforementioned improvements, our model can better extract and utilize non-redundant information, achieving a superior restoration effect that balances the global structure and local details of the image. The results from the comparative experiments reveal that our approach demonstrated a notable enhancement in cPSNR, with improvements of 0.91 dB and 0.81 dB observed in the NIR and RED bands of the PROBA-V dataset, respectively, in comparison to the existing state-of-the-art methods. Extensive experiments demonstrate that the method proposed in this paper can provide a valuable reference for solving multi-image super-resolution tasks for remote sensing.

A novel hybrid generative adversarial network for CT and MRI super-resolution reconstruction

Objective. Computed tomography (CT) and magnetic resonance imaging (MRI) are widely used in medical imaging modalities, and provide valuable information for clinical diagnosis and treatment. However, due to hardware limitations and radiation safety concerns, the acquired images are often limited in resolution. Super-resolution reconstruction (SR) techniques have been developed to enhance the resolution of CT and MRI slices, which can potentially improve diagnostic accuracy. To capture more useful feature information and reconstruct higher quality super-resolution images, we proposed a novel hybrid framework SR model based on generative adversarial networks. Approach. The proposed SR model combines frequency domain and perceptual loss functions, which can work in both frequency domain and image domain (spatial domain). The proposed SR model consists of 4 parts: (i) the discrete Fourier transform (DFT) operation transforms the image from the image domain to frequency domain; (ii) a complex residual U-net performs SR in the frequency domain; (iii) the inverse discrete Fourier transform (iDFT) operation based on data fusion transforms the image from the frequency domain to image domain; (iv) an enhanced residual U-net network is used for SR of image domain. Main results. Experimental results on bladder MRI slices, abdomen CT slices, and brain MRI slices show that the proposed SR model outperforms state-of-the-art SR methods in terms of visual quality and objective quality metric such as the structural similarity (SSIM) and the peak signal-to-noise ratio (PSNR), which proves that the proposed model has better generalization and robustness. (Bladder dataset: upscaling factor of 2: SSIM = 0.913, PSNR = 31.203; upscaling factor of 4: SSIM = 0.821, PSNR = 28.604. Abdomen dataset: upscaling factor of 2: SSIM = 0.929, PSNR = 32.594; upscaling factor of 4: SSIM = 0.834, PSNR = 27.050. Brain dataset: SSIM = 0.861, PSNR = 26.945). Significance. Our proposed SR model is capable of SR for CT and MRI slices. The SR results provide a reliable and effective foundation for clinical diagnosis and treatment.

Dual feature enhanced video super-resolution network based on low-light scenarios

Video super-resolution (VSR) reconstruction technique aims to improve the spatiotemporal resolution of consecutive low resolution (LR) frames. Most of previous methods use the correlation of inter-frame to restore pixels. It is still a challenge to exploit intra-frame and inter-frame correlations to recover high-resolution (HR) frames, especially reconstruction of video in low-light conditions for sharp imaging results has been a bottleneck in current industrial environments. Therefore, this paper proposes a novel VSR network. Specifically, we first generate the hidden information of the current frame by using the appropriate combination of the front and rear frames. Then, a re-fusion block (RFB) is designed, which utilizes the hidden information to re-fuse with corresponding LR frame. After that, we integrate an improved dual attention mechanism (DAM) into network to extract more accurate feature of intra-frame without increasing the number of parameters. The technology has important practical significance in low-light and dim scenes, so we collect some industrial video sequences and make datasets to complete VSR task. Experimental results show that our model significantly outperforms state-of-the-art methods in performance.

Terrain Self-Similarity-Based Transformer for Generating Super Resolution DEMs

High-resolution digital elevation models (DEMs) are important for relevant geoscience research and practical applications. Compared with traditional hardware-based methods, super-resolution (SR) reconstruction techniques are currently low-cost and feasible methods used for obtaining high-resolution DEMs. Single-image super-resolution (SISR) techniques have become popular in DEM SR in recent years. However, DEM super-resolution has not yet utilized reference-based image super-resolution (RefSR) techniques. In this paper, we propose a terrain self-similarity-based transformer (SSTrans) to generate super-resolution DEMs. It is a reference-based image super-resolution method that automatically acquires reference images using terrain self-similarity. To verify the proposed model, we conducted experiments on four distinct types of terrain and compared them to the results from the bicubic, SRGAN, and SRCNN approaches. The experimental results show that the SSTrans method performs well in all four terrains and has outstanding advantages in complex and uneven surface terrains.

Impact of a Deep Learning-based Super-resolution Image Reconstruction Technique on High-contrast Computed Tomography: A Phantom Study

Deep-learning-based super-resolution image reconstruction (DLSRR) is a novel image reconstruction technique that is expected to contribute to improvement in spatial resolution as well as noise reduction through learning from high-resolution computed tomography (CT). This study aims to evaluate image quality obtained with DLSRR and assess its clinical potential. CT images of a Mercury CT 4.0 phantom were obtained using a 320-row multi-detector scanner at tube currents of 100, 200, and 300 mA. Image data were reconstructed by filtered back projection (FBP), hybrid iterative reconstruction (HIR), model-based iterative reconstruction (MBIR), deep-learning-based image reconstruction (DLR), and DLSRR at image reconstruction strength levels of mild, standard, and strong. Noise power spectrum (NPS), task transfer function (TTF), and detectability index were calculated. The magnitude of the noise-reducing effect in comparison with FBP was in the order MBIR <HIR <DLR <DLSRR. The resolution property was in the order HIR <FBP <DLR <MBIR <DLSRR. The detectability index was highest for DLSRR. The maximum and mean of the NPS shifted towards lower frequencies for HIR and MBIR compared with FBP, and similar shifts were observed for DLR and DLSRR. For each image reconstruction technique, NPS decreased with increasing reconstruction strength level, but no change was observed in TTF. The present results suggest that DLSRR can achieve greater noise reduction and improved spatial resolution in the high-contrast region compared with conventional DLR and iterative reconstruction techniques.

Super-resolution fusion optimization for poultry detection: a multi-object chicken detection method.

Accurate poultry detection is crucial for studying poultry behavior using computer vision and video surveillance. However, in free-range farming environments, detecting chickens can often be challenging due to their small size and mutual occlusion. The current detection algorithms exhibit a low level of accuracy, with a high probability of false and missed detections. To address this, we proposed a multi-object chicken detection method named Super-resolution Chicken Detection, which utilizes super-resolution fusion optimization. The algorithm employs the residual-residual dense block to extract image features and used a generative adversarial network to compensate for the loss of details during deep convolution, producing high-resolution images for detection. The proposed algorithm was validated with the B1 data set and the MC1 multi-object data set, demonstrating that the reconstructed images possessed richer pixel features compared to original images, specifically it improved detection accuracy and reduced the number of missed detections. The structural similarity of the reconstructed images was 99.9%, and the peak signal-to-noise ratio was above 30. The algorithm improved the Average Precision50:95 of all You Only Look Once Version X (YOLOX) models, with the largest improvement for the B1 data set with YOLOX-Large (+6.3%) and for the MC1 data set with YOLOX-Small (+4.1%). This was the first time a super-resolution reconstruction technique was applied to multi-object poultry detection. Our method will provide a fresh approach for future poultry researchers to improve the accuracy of object detection using computer vision and video surveillance.

Research on Super-resolution Image Based on Deep Learning

Abstract Image super-resolution is a kind of important image processing technology in computer vision and image processing. It refers to the process of recovering high-resolution image from low-resolution image. It has a wide range of real-world applications, such as medical imaging, security and others. In addition to improving image perception quality, it also helps improve other computer vision tasks. Compared with traditional methods, deep learning methods show better reconstruction results in the field of image super-resolution reconstruction, and have gradually developed into the mainstream technology. This article will study the depth in the super resolution direction is important method of types of introduction, combed the main image super-resolution reconstruction method, expounds the depth study of several important super-resolution network model, the advantages and disadvantages of different algorithms and adaptive application scenarios are analyzed and compared, this paper expounds the different ways in the super resolution to liquidate, Finally, the potential problems of current image super-resolution reconstruction techniques are discussed, and the future development direction is prospected.

Whole-Brain Water Content Mapping Using Super-Resolution Reconstruction with MRI Acquisitionin3Orthogonal Orientations.

Brain water content provides rich tissue contrast comparable to that of longitudinal relaxation time T1 , but mapping is usually performed at modest resolution. In particular, the slice thickness in 2D mapping methods is limited. Here, we combine super-resolution reconstruction techniques with a fast water content mapping method to acquire high and isotropic resolution (0.75 mm) water content maps at 3Tesla. A high-resolution multi-echo gradient echo image is super-resolution-reconstructed from 3 low-resolution, orthogonal multi-echo gradient echo image acquisitions, followed by water content mapping. The mapping accuracy and SNR of the proposed method are assessed using numerical simulations, phantom studies, and in vivo data acquired from 6 healthy volunteers at 3Tesla. A high-resolution acquisition with an established mapping method is used as a reference. Whole-brain water content maps with 0.75 mm isotropic resolution are demonstrated. No bias in the water content values was seen following super-resolution reconstruction. In the in vivo experiments, a lower SD of the mean water content values was observed with the proposed method compared to the reference method. Super-resolution reconstruction of multi-echo gradient echo data is demonstrated, enabling whole-brain water content mapping with high and isotropic resolution. The accuracy of the proposed method is shown using phantoms and 6 healthy volunteers and was found to be unchanged compared to the conventional acquisition. The proposed method could increase the sensitivity of water content mapping sufficiently to enable the detection of very small lesions, such as cortical lesions in multiple sclerosis.

Application of Digital Image Processing and GIS Technology in the Evaluation of Rationality of Urban and Rural Planning Spatial Layout

This work investigates digital image processing technology and presents a super-resolution image reconstruction approach to increase the logic of urban and rural planning spatial layout. Furthermore, this work integrates GIS technology with super-resolution image reconstruction techniques to create a digital image processing and GIS-based urban and rural spatial layout design model. In addition, this paper mathematically expresses the objects and scenes that need to be simulated as a collection of three-dimensional graphic objects stored in the computer. The platform is constructed with a three-tier system of B/S architecture, and the logical structure of the entire system is composed of the client, application server, database server, and other functional servers. Furthermore, this paper constructs the functional structure of urban and rural spatial layout planning model based on digital image processing and GIS technology and verifies the model in this paper through experimental research. The experimental statistical results verify the reasonableness of the model in this paper.

Convolutional Neural Networks Based Video Reconstruction and Computation in Digital Twins

With the advancement of communication and computing technologies, multimedia technologies involving video and image applications have become an important part of the information society and have become inextricably linked to people's daily productivity and lives. Simultaneously, there is a growing interest in super-resolution (SR) video reconstruction techniques. At the moment, the design of digital twins in video computing and video reconstruction is based on a number of difficult issues. Although there are several SR reconstruction techniques available in the literature, most of the works have not considered the spatio-temporal relationship between the video frames. With this motivation in mind, this paper presents VDCNN-SS, a novel very deep convolutional neural networks (VDCNN) with spatiotemporal similarity (SS) model for video reconstruction in digital twins. The VDCNN-SS technique proposed here maps the relationship between interconnected low resolution (LR) and high resolution (HR) image blocks. It also considers the spatiotemporal non-local complementary and repetitive data among nearby low-resolution video frames. Furthermore, the VDCNN technique is used to learn the LR–HR correlation mapping learning process. A series of simulations were run to examine the improved performance of the VDCNN-SS model, and the experimental results demonstrated the superiority of the VDCNN-SS technique over recent techniques.

Hyperspectral Superresolution Reconstruction via Decomposition of Low-Rank and Sparse Tensor

Hyperspectral super-resolution reconstruction technique obtains a high-resolution hyperspectral image (HR-HSI) by fusing both a low-resolution hyperspectral image (LR-HSI) and a high-resolution multispectral image (HR-MSI). Existing methods of hyperspectral super-resolution reconstruction are mostly concentrated on the global low-rank property, while spatial and spectral information in individual regions is not considered. To address this issue, decomposition of low-rank and sparse tensor (DLST) is proposed in this study. First, HR-HSI was decomposed into low-rank and sparse components. The former was further separated into spatial and spectral domains according to the spectral low-rank property, and the later was used to compensate for information loss caused by low-rank property. Then, a nonlocal constraint of adaptive manifold extracting structural details by the manifold structure was designed to enforce nonlocal self-similarity of the spatial domain. In order to ensure the same spatial structure of different bands and reduce the false individual regions in the sparse component, a surface-aware regularization combined with group sparsity was utilized. Finally, HR-HSI was constructed by the alternating direction method of multipliers. Experiment results on three datasets show that the proposed method outperform five common existing methods by means of both visual and quantitative evaluations. It is concluded that the new method by taking into account of the low-rank and sparse properties can improve the result of the reconstruction.

Global field reconstruction from sparse sensors with Voronoi tessellation-assisted deep learning

Achieving accurate and robust global situational awareness of a complex time-evolving field from a limited number of sensors has been a longstanding challenge. This reconstruction problem is especially difficult when sensors are sparsely positioned in a seemingly random or unorganized manner, which is often encountered in a range of scientific and engineering problems. Moreover, these sensors can be in motion and can become online or offline over time. The key leverage in addressing this scientific issue is the wealth of data accumulated from the sensors. As a solution to this problem, we propose a data-driven spatial field recovery technique founded on a structured grid-based deep-learning approach for arbitrary positioned sensors of any numbers. It should be noted that the na\"ive use of machine learning becomes prohibitively expensive for global field reconstruction and is furthermore not adaptable to an arbitrary number of sensors. In the present work, we consider the use of Voronoi tessellation to obtain a structured-grid representation from sensor locations enabling the computationally tractable use of convolutional neural networks. One of the central features of the present method is its compatibility with deep-learning based super-resolution reconstruction techniques for structured sensor data that are established for image processing. The proposed reconstruction technique is demonstrated for unsteady wake flow, geophysical data, and three-dimensional turbulence. The current framework is able to handle an arbitrary number of moving sensors, and thereby overcomes a major limitation with existing reconstruction methods. The presented technique opens a new pathway towards the practical use of neural networks for real-time global field estimation.