Compressive Imaging Research Articles

Coded aperture compressive temporal imaging via unsupervised lightweight local-global networks with geometric characteristics

Coded aperture compressive temporal imaging (CACTI) utilizes compressive sensing (CS) theory to compress three dimensional (3D) signals into 2D measurements for sampling in a single snapshot measurement, which in turn acquires high-dimensional (HD) visual signals. To solve the problems of low quality and slow runtime often encountered in reconstruction, deep learning has become the mainstream for signal reconstruction and has shown superior performance. Currently, however, impressive networks are typically supervised networks with large-sized models and require vast training sets that can be difficult to obtain or expensive. This limits their application in real optical imaging systems. In this paper, we propose a lightweight reconstruction network that recovers HD signals only from compressed measurements with noise and design a block consisting of convolution to extract and fuse local and global features, stacking multiple features to form a lightweight architecture. In addition, we also obtain unsupervised loss functions based on the geometric characteristics of the signal to guarantee the powerful generalization capability of the network in order to approximate the reconstruction process of real optical systems. Experimental results show that our proposed network significantly reduces the model size and not only has high performance in recovering dynamic scenes, but the unsupervised video reconstruction network can approximate its supervised version in terms of reconstruction performance.

Simulation study of compressed ultrafast 3D imaging based on interferometry

Observing the 3D (three-dimensional) surface vibration in 100 MHz is challenging due to low speed of CCD/CMOS camera. In order to obtain the ultrafast dynamic 3D shape information, a compressed ultrafast 3D imaging framework based on interferometry is proposed in this paper. Based on this imaging framework, an reconstruction method combining plug and play-fast flexible denoising network is developed to recover high resolution interference fringe patterns and the corresponding 3D information. The simulation results show that the proposed method can observe the dynamic processes of the anti-phase vibration and the in-phase vibration of the crystal oscillator, therefore the dynamic 3D model is established. The proposed method outperforms other reconstruction methods in terms of average peak signal to noise ratio and structural similarity in recovery the ultrafast dynamic 3D surface.

Semantic regularization of electromagnetic inverse problems

Solving ill-posed inverse problems typically requires regularization based on prior knowledge. To date, only prior knowledge that is formulated mathematically (e.g., sparsity of the unknown) or implicitly learned from quantitative data can be used for regularization. Thereby, semantically formulated prior knowledge derived from human reasoning and recognition is excluded. Here, we introduce and demonstrate the concept of semantic regularization based on a pre-trained large language model to overcome this vexing limitation. We study the approach, first, numerically in a prototypical 2D inverse scattering problem, and, second, experimentally in 3D and 4D compressive microwave imaging problems based on programmable metasurfaces. We highlight that semantic regularization enables new forms of highly-sought privacy protection for applications like smart homes, touchless human-machine interaction and security screening: selected subjects in the scene can be concealed, or their actions and postures can be altered in the reconstruction by manipulating the semantic prior with suitable language-based control commands.

Multiple Complementary Priors for Multispectral Image Compressive Sensing Reconstruction.

Compressive sensing (CS) techniques using a few compressed measurements have drawn considerable interest in reconstructing multispectral imagery (MSI). Nonlocal-based tensor methods have been widely used for MSI-CS reconstruction, which employ the nonlocal self-similarity (NSS) property of MSI to obtain satisfactory results. However, such methods only consider the internal priors of MSI while ignoring important external image information, for example deep-driven priors learned from a corpus of natural image datasets. Meanwhile, they usually suffer from annoying ringing artifacts due to the aggregation of overlapping patches. In this article, we propose a novel approach for highly effective MSI-CS reconstruction using multiple complementary priors (MCPs). The proposed MCP jointly exploits nonlocal low-rank and deep image priors under a hybrid plug-and-play framework, which contains multiple pairs of complementary priors, namely, internal and external, shallow and deep, and NSS and local spatial priors. To make the optimization tractable, a well-known alternating direction method of multiplier (ADMM) algorithm based on the alternating minimization framework is developed to solve the proposed MCP-based MSI-CS reconstruction problem. Extensive experimental results demonstrate that the proposed MCP algorithm outperforms many state-of-the-art CS techniques in MSI reconstruction. The source code of the proposed MCP-based MSI-CS reconstruction algorithm is available at: https://github.com/zhazhiyuan/MCP_MSI_CS_Demo.git.

Single-Shot Intensity- and Phase-Sensitive Compressive Sensing-Based Coherent Modulation Ultrafast Imaging.

Ultrafast imaging can capture the dynamic scenes with a nanosecond and even femtosecond temporal resolution. Complementarily, phase imaging can provide the morphology, refractive index, or thickness information that intensity imaging cannot represent. Therefore, it is important to realize the simultaneous ultrafast intensity and phase imaging for achieving as much information as possible in the detection of ultrafast dynamic scenes. Here, we report a single-shot intensity- and phase-sensitive compressive sensing-based coherent modulation ultrafast imaging technique, shortened as CS-CMUI, which integrates coherent modulation imaging, compressive imaging, and streak imaging. We theoretically demonstrate through numerical simulations that CS-CMUI can obtain both the intensity and phase information of the dynamic scenes with ultrahigh fidelity. Furthermore, we experimentally build a CS-CMUI system and successfully measure the intensity and phase evolution of a multimode Q-switched laser pulse and the dynamical behavior of laser ablation on an indium tin oxide thin film. It is anticipated that CS-CMUI enables a profound comprehension of ultrafast phenomena and promotes the advancement of various practical applications, which will have substantial impact on fundamental and applied sciences.

Optical fiber bundle differential compressive imaging.

We present a differential compressive imaging method for an optical fiber bundle (OFB), which provides a solution for an ultrathin bend-resistant endoscope with high resolution. This method uses an OFB and a diffuser to generate speckle illumination patterns. Differential operation is additionally applied to the speckle patterns to produce sensing matrices, by which the correlation between the matrices is greatly reduced from 0.875 to 0.0275, which ensures the high quality of image reconstruction. Pixilation artifacts from the fiber core arrangement are also effectively eliminated with this configuration. We demonstrate high-resolution reconstruction of images of 132 × 132 pixels with a compression rate of 12% using 77 fiber cores, the total diameter of which is only about 91 µm. An experimental verification proves that this method is tolerant to a limited degree of fiber bending, which provides a potential approach for robust high-resolution fiber endoscopy.

Dual-domain mean-reverting diffusion model-enhanced temporal compressive coherent diffraction imaging

Temporal compressive coherent diffraction imaging is a lensless imaging technique with the capability to capture fast-moving small objects. However, the accuracy of imaging reconstruction is often hindered by the loss of frequency domain information, a critical factor limiting the quality of the reconstructed images. To improve the quality of these reconstructed images, a method dual-domain mean-reverting diffusion model-enhanced temporal compressive coherent diffraction imaging (DMDTC) has been introduced. DMDTC leverages the mean-reverting diffusion model to acquire prior information in both frequency and spatial domain through sample learning. The frequency domain mean-reverting diffusion model is employed to recover missing information, while hybrid input-output algorithm is carried out to reconstruct the spatial domain image. The spatial domain mean-reverting diffusion model is utilized for denoising and image restoration. DMDTC has demonstrated a significant enhancement in the quality of the reconstructed images. The results indicate that the structural similarity and peak signal-to-noise ratio of images reconstructed by DMDTC surpass those obtained through conventional methods. DMDTC enables high temporal frame rates and high spatial resolution in coherent diffraction imaging.

Lightweight super-resolution multimode fiber imaging with regularized linear regression

Super-resolution multimode fiber imaging provides the means to image samples quickly with compact and flexible setups finding many applications from biology and medicine to material science and nanolithography. Typically, fiber-based imaging systems suffer from low spatial resolution and long measurement times. State-of-the-art computational approaches can achieve fast super-resolution imaging through a multimode fiber probe but currently rely on either per-sample optimised priors or large data sets with subsequent long training and image reconstruction times. This unfortunately hinders any real-time imaging applications. Here we present an ultimately fast non-iterative algorithm for compressive image reconstruction through a multimode fiber. The proposed approach helps to avoid many constraints by determining the prior of the target distribution from a simulated set and solving the under-determined inverse matrix problem with a mathematical closed-form solution. We have demonstrated theoretical and experimental evidence for enhanced image quality and sub-diffraction spatial resolution of the multimode fiber optical system.

Direct object detection with snapshot multispectral compressed imaging in a short-wave infrared band.

Snapshot multispectral imaging (SMSI) has attracted much attention in recent years for its compact structure and superior performance. High-level image analysis based on SMSI, such as object classification and recognition, usually takes the image reconstruction as the first step, which hinders its application in many important real-time scenarios. Here we demonstrate the first, to our knowledge, reconstruction-free strategy for object detection with SMSI in the short-wave infrared (SWIR) band. The implementation of our SMSI is based on a modified 4f system which modulates the light with a random phase mask, and the distinctive point spread function in each narrowband endows the system with spectrum resolving ability. A deep learning network with a CenterNet structure is trained to detect a small object by constructing a dataset with the PSF of our SMSI system and the sky images as background. Our results indicate that a small object with a spectral feature can be detected directly with the compressed image output by our SMSI system. This work paves the way toward the use of SMSI to detect a multispectral object in practical applications.

Faster structured illumination microscopy using complementary encoding-based compressive imaging

Structured illumination microscopy (SIM) has been widely applied to investigate intricate biological dynamics due to its outstanding super-resolution imaging speed. Incorporating compressive sensing into SIM brings the possibility to further improve the super-resolution imaging speed. Nevertheless, the recovery of the super-resolution information from the compressed measurement remains challenging in experiments. Here, we report structured illumination microscopy with complementary encoding-based compressive imaging (CECI-SIM) to realize faster super-resolution imaging. Compared to the nine measurements to obtain a super-resolution image in a conventional SIM, CECI-SIM can achieve a super-resolution image by three measurements; therefore, a threefold improvement in the imaging speed can be achieved. This faster imaging ability in CECI-SIM is experimentally verified by observing tubulin and actin in mouse embryonic fibroblast cells. This work provides a feasible solution for high-speed super-resolution imaging, which would bring significant applications in biomedical research.

Dual-Window Multiscale Transformer for Hyperspectral Snapshot Compressive Imaging

Coded aperture snapshot spectral imaging (CASSI) system is an effective manner for hyperspectral snapshot compressive imaging. The core issue of CASSI is to solve the inverse problem for the reconstruction of hyperspectral image (HSI). In recent years, Transformer-based methods achieve promising performance in HSI reconstruction. However, capturing both long-range dependencies and local information while ensuring reasonable computational costs remains a challenging problem. In this paper, we propose a Transformer-based HSI reconstruction method called dual-window multiscale Transformer (DWMT), which is a coarse-to-fine process, reconstructing the global properties of HSI with the long-range dependencies. In our method, we propose a novel U-Net architecture using a dual-branch encoder to refine pixel information and full-scale skip connections to fuse different features, enhancing the extraction of fine-grained features. Meanwhile, we design a novel self-attention mechanism called dual-window multiscale multi-head self-attention (DWM-MSA), which utilizes two different-sized windows to compute self-attention, which can capture the long-range dependencies in a local region at different scales to improve the reconstruction performance. We also propose a novel position embedding method for Transformer, named con-abs position embedding (CAPE), which effectively enhances positional information of the HSIs. Extensive experiments on both the simulated and the real data are conducted to demonstrate the superior performance, stability, and generalization ability of our DWMT. Code of this project is at https://github.com/chenx2000/DWMT.

Enhanced deep unrolling networks for snapshot compressive hyperspectral imaging

Snapshot compressive hyperspectral imaging necessitates the reconstruction of a complete hyperspectral image from its compressive snapshot measurement, presenting a challenging inverse problem. This paper proposes an enhanced deep unrolling neural network, called EDUNet, to tackle this problem. The EDUNet is constructed via the deep unrolling of a proximal gradient descent algorithm and introduces two innovative modules for gradient-driven update and proximal mapping reflectivity. The gradient-driven update module leverages a memory-assistant descent approach inspired by momentum-based acceleration techniques, for enhancing the unrolled reconstruction process and improving convergence. The proximal mapping is modeled by a sub-network with a cross-stage spectral self-attention, which effectively exploits the inherent self-similarities present in hyperspectral images along the spectral axis. It also enhances feature flow throughout the network, contributing to reconstruction performance gain. Furthermore, we introduce a spectral geometry consistency loss, encouraging EDUNet to prioritize the geometric layouts of spectral curves, leading to a more precise capture of spectral information in hyperspectral images. Experiments are conducted using three benchmark datasets including KAIST, ICVL, and Harvard, along with some real data, comprising a total of 73 samples. The experimental results demonstrate that EDUNet outperforms 15 competing models across four metrics including PSNR, SSIM, SAM, and ERGAS.

Hyperchaotic bilateral random low-rank approximation random sequence generation method and its application on compressive ghost imaging

Hyperchaotic bilateral random low-rank approximation random sequence generation method and its application on compressive ghost imaging

Vector-guided Fourier single-pixel imaging.

The Fourier single-pixel imaging technique exhibits great potential for compressive imaging. However, the utilization of low sampling ratio can introduce unwanted ringing artifacts, thereby compromising the fidelity of reconstructed image detail. To address this issue, Vector guided Fourier single-pixel imaging (V-FSI) has been proposed. We analyze the statistical properties in the edge vector field derived from images with low sampling ratio. Based on this information, a tailored sampling map is designed to acquire the significant high-frequency components for image reconstruction. Experimental results demonstrate the remarkable effectiveness of the proposed V-FSI method in enhancing image quality. Notably, V-FSI exhibits exceptional capabilities in perceiving and preserving the details of the objects, particularly for objects characterized by pronounced periodicity and directionality.

Snapshot compressive imaging at 855 million frames per second for aluminium planar wire array Z-pinch.

This paper present a novel, integrated compressed ultrafast photography system for comprehensive measurement of the aluminium planar wire array Z-Pinch evolution process. The system incorporates a large array streak camera and embedded encoding to improve the signal-to-noise ratio. Based on the "QiangGuang-I" pulsed power facility, we recorded the complete continuous 2D implosion process of planar wire array Z-Pinch for the first time. Our results contribute valuable understanding of imploding plasma instabilities and offer direction for the optimization of Z-Pinch facilities.

Fast matrix inversion in compressive spectral imaging based on a tensorial representation

Fast matrix inversion in compressive spectral imaging based on a tensorial representation

Snapshot volumetric imaging using rainbow-sheet illumination

Traditional lens-based three-dimensional imaging methods struggle with speed, spatial resolution, field of view, and depth of field (DOF). Here, we propose a volumetric imaging method that combines rainbow-sheet illumination, chromatic-aberration-induced DOF extension, and compressive hyperspectral imaging to optically section transparent objects over 200 depth slices in a single snapshot. A proof-of-concept mesoscopic system with a lateral resolution of 12.7 line pairs per millimeter and a depth resolution of roughly 140 μm in a volume of 10 × 10 × 10 mm3 is constructed. The practicality of the suggested method is demonstrated by dynamic volumetric imaging of a transparent jellyfish at a rate of 15 volumes per second.

Advances in Mask-Modulated Lensless Imaging

Lensless imaging allows for designing imaging systems that are free from the constraints of traditional imaging architectures. As a broadly investigated technique, mask-modulated lensless imaging encodes light signals via a mask plate integrated with the image sensor, which is more compacted, with scalability and compressive imaging abilities. Here, we review the latest advancements in mask-modulated lensless imaging, lensless image reconstruction algorithms, related techniques, and future directions and applications.

Computational ghost image encryption method based on sparse speckles

Optical encryption based on ghost imaging has the advantages of high-speed parallel processing and multi-dimensional information. However, in practical application, a large number of speckle patterns are required to achieve the encryption, resulting in a significant increase in transmission costs and low encryption efficiency. In this paper, a compressive sensing-based ghost imaging encryption method using sparse speckles is proposed. The sparse speckle used in this method has the characteristics of easy compression and greatly reducing the amount of encryption keys. According to calculation, the memory overhead of keys using sparse speckles can reduce to 2.44% of that using random speckles. And the decryption effect and anti-noise performance of this method are also verified better than that using random speckles by simulation and experiment results.

Exploiting sparsity of hyperspectral image: A novel approach for compressive hyperspectral image reconstruction using deep learning

Compressive hyperspectral imaging is an emerging technology that captures compressed two-dimensional (2D) measurements and subsequently reconstructs three-dimensional (3D) hyperspectral images, capitalizing on the sparsity prior. Numerous end-to-end networks have been proposed, yielding notable improvements in reconstruction speed and quality compared to traditional algorithms. However, these networks fall short in exploiting the sparsity of hyperspectral images in their direct mapping from 2D measurements to 3D hyperspectral images. To address this limitation, this paper introduces a novel approach for compressive hyperspectral image reconstruction using deep learning. The deep learning network comprises two components: the sparse coefficient recovery net and the dictionary training net. The dictionary training net incorporates a pre-trained dictionary, whose parameters can also be updated during the backpropagation process. Concurrently, the sparse coefficient recovery net generates sparse coefficients under this dictionary. Our approach draws inspiration from conventional dictionary learning algorithms, enabling effective utilization of signal sparsity. In comparison to several state-of-the-art methods, our method achieves superior reconstruction quality and demonstrates enhanced robustness against noise. The proposed approach is envisioned to offer fresh insights into hyperspectral images reconstruction using deep learning models.