Compressive Spectral Imaging Systems Research Articles

Freeform-surface-based optical design of a broadband compressive spectral imager with co-aperture coding

Compressive spectral imaging (CSI) is an advanced computational imaging approach to reconstruct the three-dimensional (3D) spatio-spectral data cube of a target scene through a single or a few snapshots. However, limited by the response range of the image detector, the existing CSI systems mostly work within narrow spectral bands, such as the visible or shortwave-infrared (SWIR) spectral band. The work band of the CSI system constrains the detection capacity for the targets under complex environments (such as rain, snow, haze, etc.). In addition, most of the current CSI prototypes lack engineering design for practical applications. This paper develops a novel, to the best of our knowledge, optical design scheme of a broadband CSI system with co-aperture coding to simultaneously realize visible multi-spectral imaging (10 channels) and SWIR super-resolution imaging (16×). The freeform surfaces are used to design the front-end reflective objective lens, thus significantly improving the image quality and spatial modulation precision of the system. By means of performance evaluation and tolerance analysis, excellent image quality and manufacturability of the proposed system are demonstrated.

Dual-Channel Switchable Metasurface Filters for Compact Spectral Imaging with Deep Compressive Reconstruction.

Spectral imaging technology, which aims to capture images across multiple spectral channels and create a spectral data cube, has been widely utilized in various fields. However, conventional spectral imaging systems face challenges, such as slow acquisition speed and large size. The rapid development of optical metasurfaces, capable of manipulating light fields versatilely and miniaturizing optical components into ultrathin planar devices, offers a promising solution for compact hyperspectral imaging (HSI). This study proposes a compact snapshot compressive spectral imaging (SCSI) system by leveraging the spectral modulations of metasurfaces with dual-channel switchable metasurface filters and employing a deep-learning-based reconstruction algorithm. To achieve compactness, the proposed system integrates dual-channel switchable metasurface filters using twisted nematic liquid crystals (TNLCs) and anisotropic titanium dioxide (TiO2) nanostructures. These thin metasurface filters are closely attached to the image sensor, resulting in a compact system. The TNLCs possess a broadband linear polarization conversion ability, enabling the rapid switching of the incidence polarization state between x-polarization and y-polarization by applying different voltages. This polarization conversion facilitates the generation of two groups of transmittance spectra for wavelength-encoding, providing richer information for spectral data cube reconstruction compared to that of other snapshot compressive spectral imaging techniques. In addition, instead of employing classic iterative compressive sensing (CS) algorithms, an end-to-end residual neural network (ResNet) is utilized to reconstruct the spectral data cube. This neural network leverages the 2-frame snapshot measurements of orthogonal polarization channels. The proposed hyperspectral imaging technology demonstrates superior reconstruction quality and speed compared to those of the traditional compressive hyperspectral image recovery methods. As a result, it is expected that this technology will have substantial implications in various domains, including but not limited to object detection, face recognition, food safety, biomedical imaging, agriculture surveillance, and so on.

Hybrid compressive spectral imaging with coded aperture optimization based on coherence minimization

Compressive spectral imaging (CSI) obtains spectral images through reconstruction from under-sampled low-dimensional projections with spatial and spectral coding. However, the simultaneous realization of high spatial–spectral resolution is challenged due to limitations of a single CSI architecture. In this paper, a hybrid CSI system combining the coded aperture snapshot spectral imaging (CASSI) and the spectrometer-based single-pixel spectral imaging (SPSI), with the corresponding coded aperture optimization for both constituent systems are proposed. The hybrid CSI system sustains the complementary advantages of the spatial resolution in CASSI and the spectral fidelity in SPSI, ensuring high reconstruction quality in both spatial and spectral dimensions with increased sampling efficiency. The derivable matrix operation of the hybrid CSI system is formulated; meanwhile, based on coherence minimization, the individual optimization on the coded apertures in the CASSI or the SPSI system, and the joint optimization on the coded apertures in the hybrid system are demonstrated. The reconstruction is further improved with the optimization, indicated by the quantitative image quality metrics and the point spectra. Compared to the conventional unoptimized CASSI, the optimized hybrid CSI system improves the PSNR for more than 7 dB. This work may introduce opportunities for multipath compressive imaging systems toward high-efficiency, high-resolution future.

Superpixel-based linear reconstruction method for dual-camera compressed hyperspectral imaging system

Compressive spectral imaging systems have received significant research attention in recent years. Among these systems, the dual-camera compressive hyperspectral imaging (DCCHI) design, has the potential to improve imaging resolution while maintaining the snapshot advantage. In this paper, we propose superpixel-based linear reconstruction methods for DCCHI system. Specifically, we assume that the spectral intensity of the superpixels generated from the RGB image varies linearly in the spectral dimension. Instead of addressing an ill-posed problem, we reconstruct hyperspectral images via solving overdetermined equations using the least squares method, resulting in a significant reduction in both computation time and memory usage. To further enhance the reconstruction accuracy, we extend the spectra of superpixels from a 1D linear space into a 3D space with an RGB spectral response matrix. Despite the simplicity of our method, it offers improved reconstruction accuracy of hyperspectral images compared to other state-of-the-art methods. Furthermore, it is more efficient in terms of memory usage and computational time.

Review of physical implementation architecture in compressive spectral imaging system

The predecessor of China Optics was China Optics and Applied Optics Digest, which was founded in 1985. At that time, it was the only retrieval journal in the field of optics in China. At the end of 2008, China Optics and Applied Optics Digest was renamed

Compressive spectral imaging system for soil classification with three-dimensional convolutional neural network.

Compressive spectral imaging systems have promising applications in the field of object classification. However, for soil classification problem, conventional methods addressing this specific task often fail to produce satisfying results due to the tradeoff between the invariance and discrepancy of each soil. In this paper, we explore a liquid crystal tunable filters (LCTF)-based system and propose a three-dimensional convolutional neural network (3D-CNN) for soil classification. We first obtain a set of soil compressive measurements via a low spatial resolution detector, and soil hyperspectral images are reconstructed with improved resolution in spatial as well as spectral domains by a compressive sensing (CS) method. Furthermore, different from previous spectral-based object classification methods restricted to extract features from each type independently, on account of the potential of spectral property on individual solid, our method proposes to apply the principal component analysis(PCA) to achieve a dimensionality reduction in the spectral domain. Then, we explore a differential perception model for flexible feature extraction, and finally introduce a 3D-CNN framework to solve the multi-soil classification problem. Experimental results demonstrate that our algorithm not only is able to accelerate the ability of feature discriminability but also performs against conventional soil classification methods.

Adaptive grayscale compressive spectral imaging using optimal blue noise coding patterns

Imaging spectroscopy collects the spectral information of a scene by sensing all the spatial information across the electromagnetic wavelengths and are useful for applications in surveillance, agriculture, and medicine, etc. In contrast, compressive spectral imaging (CSI) systems capture compressed projections of the scene, which are then used to recover the whole spectral scene. A key component in such optical systems is the coded aperture which performs the scene codification and defines the sensing scheme of the system. The proper design of the coded aperture entries leads to good quality reconstructions with few compressed measurements. Commonly, the acquired measurements are prone to saturation due to the limited dynamic range of the sensor, however, the saturation is not usually taking into account in the coded aperture design. The saturation errors in compressed measurements are unbounded leading to poor reconstructions since CSI recovery algorithms only provide solutions for bounded or noisy-bounded errors. This paper proposes an adaptive grayscale coded aperture design which combines the advantages of blue noise and block-unblock coding patterns. Blue noise coding patterns are optimal and provide high-quality image reconstructions on regions of non-saturated compressed pixels. On the other hand, the block-unblock coding patterns provide redundancy in the sampling which helps to reduce the saturation in the detector. Further, the saturation is reduced between snapshots by using an adaptive filter which updates the entries of the grayscale coded aperture based on the previously acquired measurements. The proposed coded apertures are optimized such that the number of saturated measurements is minimized. Extensive simulations and an experimental setup were made using the coded aperture snapshot spectral imager (CASSI) sensing scheme, where the results show an improvement up to 2 dB of peak signal-to-noise ratio is achieved when the proposed adaptive grayscale blue noise and block-unblock coded aperture (AGBBCA) design is compared with adaptive grayscale block-unblock coded apertures (AGBCA).

Noniterative Hyperspectral Image Reconstruction From Compressive Fused Measurements

Compressive spectral imaging (CSI) enables the acquisition of spectral and spatial information of a scene using fewer projected measurements than traditional scanning approaches. Recently, research efforts have focused on obtaining high-resolution spectral images via expensive detectors and sophisticated CSI devices. Alternatively, high-resolution spectral images can be obtained using side information or fusion of compressed measurements, without significantly increasing acquisition costs. Indeed, these approaches retrieve improved resolution images applying iterative and computationally expensive algorithms. This paper proposes the fusion of compressed measurements obtained from two state-of-the-art CSI systems, the single-pixel camera (SPC) and the three-dimensional coded aperture snapshot imaging system (3D-CASSI), such that high-resolution images can be obtained by exploiting detailed spectra provided by the SPC and high spatial resolution of the 3D-CASSI. Specifically, a noniterative reconstruction algorithm is proposed, based on the fact that the spatial-spectral data lie in a low-dimensional subspace. In contrast to related works, the proposed approach relies on implementable CSI systems. Simulations and experimental results show the effectiveness of the proposed method compared to similar approaches, both in reconstruction quality and complexity. Specifically, the proposed method is up to 5.6 times faster than its counterparts and provides comparable quality of attained reconstructions.

Compressive spectral feature sensing

To reduce the size of spectral data, compressive sensing imaging systems are developed to sample fewer measurements than the Nyquist-rate ones, from which the original data can be recovered by the optimisation model and algorithm. However, this is not a cheap option for the case where the real-time acquisition of spectral information is required. To solve this problem, the authors propose a novel sensing approach for spectral features by combining the sampling, recovery and feature extraction. Inspired by the spectral feature representation, the sampling (sensing) matrix is designed from the training spectral samples to sense the spectral features of the imaging scene, which can be utilised for classification and recognition directly. Besides, the physical realisation of the sensing matrix for compressive spectral imaging systems is demonstrated by designing new modulation patterns of the digital micro-mirror device. The experimental results on real spectral data show the feasibility of the proposed scheme and the robustness to the quantisation error and the measurement noise. Moreover, the proposed sensing approach can reduce the cost of computation and time greatly by removing the sparse recovery and feature extraction..

Shifting colored coded aperture design for spectral imaging.

Compressive spectral imaging (CSI) systems sense 3D spatio-spectral data cubes with just a few two-dimensional (2D) projections by using a coded aperture, a dispersive element, and a focal plane array (FPA). The coded apertures in these systems, whose main function is the modulation of the data cube, are often implemented through photomasks attached to piezoelectric devices. A remarkable improvement on this configuration has been recently proposed, the replacement of the block-unblock coded apertures by patterned optical filter arrays, referred to as "colored" coded apertures, which allow spatial and spectral modulation. When using these colored coded apertures, its real implementation in terms of cost and complexity directly depends on the number of filters to be used, as well as the number of shots to be captured. A shifting colored coded aperture optimization featuring these observations is proposed, with the aim to improve the imaging quality reconstruction and to generate an achievable optical implementation with a limited number of filters requiring only one mask to acquire any number of shots. The mathematical model of the computational imaging strategy to overcome the practical limitations of actual CSI systems is presented along with a testbed implementation. Simulations, as well as experimental results, will prove the accuracy and performance of the proposed shifting colored coded aperture design over the current literature designs.

Compressive spectral imaging system based on liquid crystal tunable filter.

Liquid crystal tunable filters (LCTF) are extensively used in hyperspectral imaging systems to successively acquire different spectral components of scenes by adjusting the center wavelength of the filter. However, the spectral and spatial resolutions of the imager are limited by the bandwidth of LCTF, and the pitch dimension of the detector, respectively. This paper applies compressive sensing principles to improve both of the spatial and spectral resolutions of the LCTF-based hyperspectral imaging system. An accurate transmission model of the LCTF is used to represent its bandpass filtering effects on the spectra. In addition, a random coded aperture placed behind the LCTF is used to modulate the spectral images in the spatial domain. Then, the three-dimensional encoded spectral images are projected onto a two-dimensional detector. Benefiting from the spectral-dependent transmission property of the LCTF, information of the entire spectrum is collected by a few snapshots using different center wavelengths of the LCTF. Super-resolution hyperspectral images can be reconstructed from a small set of compressive measurements by solving a convex optimization problem. Simulations and experiments show that the proposed method can effectively improve the spectral and spatial resolutions of traditional LCTF-based spectral imager without changing the structures of the LCTF and detector. Finally, a multi-channel spectral coding method is proposed to further increase the compression capacity of the system.

Compressive spectral image super-resolution by using singular value decomposition

Compressive sensing (CS) has been recently applied to the acquisition and reconstruction of spectral images (SI). This field is known as compressive spectral imaging (CSI). The attainable resolution of SI depends on the sensor characteristics, whose cost increases in proportion to the resolution. Super-resolution (SR) approaches are usually applied to low-resolution (LR) CSI systems to improve the quality of the reconstructions by solving two consecutive optimization problems. In contrast, this work aims at reconstructing a high resolution (HR) SI from LR compressive measurements by solving a single convex optimization problem based on the fusion of CS and SR techniques. Furthermore, the truncated singular value decomposition is used to alleviate the computational complexity of the inverse reconstruction problem. The proposed method is tested by using the coded aperture snapshot spectral imager (CASSI), and the results are compared to HR-SI images directly reconstructed from LR-SI images by using an SR algorithm via sparse representation. In particular, a gain of up to 1.5 dB of PSNR is attained with the proposed method.

A quantitative and qualitative performance analysis of compressive spectral imagers

Contexto: Las imágenes espectrales (SI) contienen información espacio-espectral acerca de una escena disponible en un cubo de datos que usualmente comprende una cantidad significativa de éstos. Los sistemas tradicionales de (SI) adquieren datos redundantes ignorando la alta correlación entre las mediciones y las muestras redundantes. Los sistemas de compresión de imágenes espectrales comprimen los datos espectrales en la etapa de adquisición, lo que permite reducir la cantidad de datos y la redundancia. Actualmente, existen varios sistemas de imágenes espectrales disponibles que proporcionan nuevas funciones para los usuarios y abren un amplio campo de nuevas aplicaciones. Por ejemplo, los sistemas de CASSI, SCSI, SSCS, y HYCA son cuatro de los más destacados.Método: La revisión de algunos trabajos provee amplios estudios de tecnologías disponibles y muestra cómo se pueden utilizar las nuevas capacidades de los enfoques de formación de imágenes espectrales. Sin embargo, para la selección de una arquitectura específica se requiere una comparación cuantitativa y cualitativa de estos sistemas en los mismos escenarios.Resultados: En este trabajo se analiza el rendimiento cualitativo y cuantitativo de estos cuatro sistemas de compresión de imágenes espectrales para evaluarlos en los mismos escenarios. Para ello, cada arquitectura se modela como un sistema de ecuaciones lineales y el proceso de reconstrucción de las imágenes se logra con el mismo enfoque de optimización transmitancia, código de apertura y número de proyecciones.Conclusión: Se muestra que el sistema SSCSI alcanza el mejor rendimiento en la reconstrucción con el valor más alto PSNR.

Compressive spectral testbed imaging system based on thin-film color-patterned filter arrays.

Compressive spectral imaging systems can reliably capture multispectral data using far fewer measurements than traditional scanning techniques. In this paper, a thin-film patterned filter array-based compressive spectral imager is demonstrated, including its optical design and implementation. The use of a patterned filter array entails a single-step three-dimensional spatial-spectral coding on the input data cube, which provides higher flexibility on the selection of voxels being multiplexed on the sensor. The patterned filter array is designed and fabricated with micrometer pitch size thin films, referred to as pixelated filters, with three different wavelengths. The performance of the system is evaluated in terms of references measured by a commercially available spectrometer and the visual quality of the reconstructed images. Different distributions of the pixelated filters, including random and optimized structures, are explored.

A Compressive System Matrix Design in Spectral Imaging by a Homogenization Algorithm

Context: Compressive spectral imaging systems (CSI) use a focal plane array (FPA) to measure two-dimensional (2D) coded projections of a three-dimensional (3D) spatiospectral scene. A reconstruction algorithm based on compressive sensing theory exploits the projections to retrieve the underlying 3D scene. Compressive sensing relies on two principles: Sparsity and incoherence. Higher incoherence drives to better-reconstructed image quality. In CSI systems, a random design of the coded apertures elements guarantees a high incoherence between the sensing matrix and the representation basis. However, when a coded aperture is designed completely random, it is possible that some voxels not be sensed at all or they be sensed more than once. Method: This paper presents a random algorithm for a colored coded apertures design by homogenizing defined parameters of the colored coded aperture snapshot spectral imaging system (C-CASSI) representative matrix. Homogenization parameters guarantee that all voxels are sensed at least once. The homogenization is achieved by weighting the selected parameters of the matrix, in this case, the average of unblocking elements per column and the average of unblocking elements per row. Results/Conclusions: Simulations show a higher performance in the PSNR of the reconstructed images by using the proposed approach, as compared to traditional random coded apertures.

DMD-based implementation of patterned optical filter arrays for compressive spectral imaging.

Compressive spectral imaging (CSI) captures multispectral imagery using fewer measurements than those required by traditional Shannon-Nyquist theory-based sensing procedures. CSI systems acquire coded and dispersed random projections of the scene rather than direct measurements of the voxels. To date, the coding procedure in CSI has been realized through the use of block-unblock coded apertures (CAs), commonly implemented as chrome-on-quartz photomasks. These apertures block or permit us to pass the entire spectrum from the scene at given spatial locations, thus modulating the spatial characteristics of the scene. This paper extends the framework of CSI by replacing the traditional block-unblock photomasks by patterned optical filter arrays, referred to as colored coded apertures (CCAs). These, in turn, allow the source to be modulated not only spatially but spectrally as well, entailing more powerful coding strategies. The proposed CCAs are synthesized through linear combinations of low-pass, high-pass, and bandpass filters, paired with binary pattern ensembles realized by a digital micromirror device. The optical forward model of the proposed CSI architecture is presented along with a proof-of-concept implementation, which achieves noticeable improvements in the quality of the reconstruction.