Compressive Spectral Imaging Research Articles

Noniterative Hyperspectral Image Reconstruction From Compressive Fused Measurements

Compressive spectral imaging (CSI) enables the acquisition of spectral and spatial information of a scene using fewer projected measurements than traditional scanning approaches. Recently, research efforts have focused on obtaining high-resolution spectral images via expensive detectors and sophisticated CSI devices. Alternatively, high-resolution spectral images can be obtained using side information or fusion of compressed measurements, without significantly increasing acquisition costs. Indeed, these approaches retrieve improved resolution images applying iterative and computationally expensive algorithms. This paper proposes the fusion of compressed measurements obtained from two state-of-the-art CSI systems, the single-pixel camera (SPC) and the three-dimensional coded aperture snapshot imaging system (3D-CASSI), such that high-resolution images can be obtained by exploiting detailed spectra provided by the SPC and high spatial resolution of the 3D-CASSI. Specifically, a noniterative reconstruction algorithm is proposed, based on the fact that the spatial-spectral data lie in a low-dimensional subspace. In contrast to related works, the proposed approach relies on implementable CSI systems. Simulations and experimental results show the effectiveness of the proposed method compared to similar approaches, both in reconstruction quality and complexity. Specifically, the proposed method is up to 5.6 times faster than its counterparts and provides comparable quality of attained reconstructions.

Compressive spectral feature sensing

To reduce the size of spectral data, compressive sensing imaging systems are developed to sample fewer measurements than the Nyquist-rate ones, from which the original data can be recovered by the optimisation model and algorithm. However, this is not a cheap option for the case where the real-time acquisition of spectral information is required. To solve this problem, the authors propose a novel sensing approach for spectral features by combining the sampling, recovery and feature extraction. Inspired by the spectral feature representation, the sampling (sensing) matrix is designed from the training spectral samples to sense the spectral features of the imaging scene, which can be utilised for classification and recognition directly. Besides, the physical realisation of the sensing matrix for compressive spectral imaging systems is demonstrated by designing new modulation patterns of the digital micro-mirror device. The experimental results on real spectral data show the feasibility of the proposed scheme and the robustness to the quantisation error and the measurement noise. Moreover, the proposed sensing approach can reduce the cost of computation and time greatly by removing the sparse recovery and feature extraction..

Shifting colored coded aperture design for spectral imaging.

Compressive spectral imaging (CSI) systems sense 3D spatio-spectral data cubes with just a few two-dimensional (2D) projections by using a coded aperture, a dispersive element, and a focal plane array (FPA). The coded apertures in these systems, whose main function is the modulation of the data cube, are often implemented through photomasks attached to piezoelectric devices. A remarkable improvement on this configuration has been recently proposed, the replacement of the block-unblock coded apertures by patterned optical filter arrays, referred to as "colored" coded apertures, which allow spatial and spectral modulation. When using these colored coded apertures, its real implementation in terms of cost and complexity directly depends on the number of filters to be used, as well as the number of shots to be captured. A shifting colored coded aperture optimization featuring these observations is proposed, with the aim to improve the imaging quality reconstruction and to generate an achievable optical implementation with a limited number of filters requiring only one mask to acquire any number of shots. The mathematical model of the computational imaging strategy to overcome the practical limitations of actual CSI systems is presented along with a testbed implementation. Simulations, as well as experimental results, will prove the accuracy and performance of the proposed shifting colored coded aperture design over the current literature designs.

Multicolor fluorescent imaging by space-constrained computational spectral imaging.

Spectral imaging is a powerful technique used to simultaneously study multiple fluorophore labels with overlapping emissions. Here, we present a computational spectral imaging method, which uses sample spatial fluorescence information as a reconstruction constraint. Our method addresses both the under-sampling issue of compressive spectral imaging and the low throughput issue of scanning spectral imaging. With simulated and experimental data, we have demonstrated the reconstruction precision of our method in two and three-color imaging. We have experimentally validated this method for differentiating cellular structures labeled with two red-colored fluorescent proteins, tdTomato and mCherry, which have highly overlapping emission spectra. Our method has the advantage of totally free wavelength choice and can also be combined with conventional filter-based sequential multi-color imaging to further improve multiplexing capability.

Spectral Zooming and Resolution Limits of Spatial Spectral Compressive Spectral Imagers

The recently introduced Spatial Spectral Compressive Spectral Imager (SSCSI) has been proposed as an alternative to carry out spatial and spectral coding using a binary on-off coded aperture. In SSCSI, the pixel pitch size of the coded aperture, as well as its location with respect to the detector array, play a critical role in the quality of image reconstruction. In this paper, a rigorous discretization model for this architecture is developed, based on a light propagation analysis across the imager. The attainable spatial and spectral resolution, and the various parameters affecting them, is derived through this process. Much like the displacement of zoom lens components leads to higher spatial resolution of a scene, a shift of the coded aperture in the SSCSI in reference to the detector leads to higher spectral resolution. This allows the recovery of spectrally detailed datacubes by physically displacing the mask towards the spectral plane. To prove the underlying concepts, computer simulations and experimental data are presented in this paper.

Coded Aperture Design for Compressive Spectral Subspace Clustering

Compressive spectral imaging (CSI) acquires compressed observations of a spectral scene by applying different coding patterns at each spatial location and then performing a spectral-wise integration. Relying on compressive sensing, spectral image reconstruction is achieved by using nonlinear and relatively expensive optimization-based algorithms. In the CSI literature, several works have focused on improving reconstructions quality by properly designing the set of coding patterns. However, signal recovery is not actually necessary in many signal processing applications. For instance, assuming that compressed measurements with similar characteristics lie on the same subspace, unsupervised methods such as subspace clustering can be used to separate them into the same cluster. Since the structure of compressed measurements is defined by the applied codification, it is possible to improve clustering performance. This paper proposes to design a set of coding patterns such that inter-class and intra-class data structure is preserved after the CSI acquisition in order to improve clustering results directly on the compressed domain. To validate the coding pattern design, an algorithm based on sparse subspace clustering (SSC) is proposed to perform clustering on the compressed measurements. The proposed algorithm adds a three-dimensional (3-D) spatial regularizer to the SSC problem exploiting the spatial correlation of spectral images. In general, an overall accuracy up to 83.81% is obtained, when noisy measurements are assumed. In addition, a difference of at most 4% in terms of overall accuracy was observed when comparing the clustering results obtained by the full 3-D data with those achieved using CSI measurements acquired with the proposed coding pattern design.

Spectral Image Fusion from Compressive Measurements.

Compressive spectral imagers reduce the number of sampled pixels by coding and combining the spectral information. However, sampling compressed information with simultaneous high spatial and high spectral resolution demands expensive high-resolution sensors. This work introduces a model allowing data from high spatial/low spectral and low spatial/high spectral resolution compressive sensors to be fused. Based on this model, the compressive fusion process is formulated as an inverse problem that minimizes an objective function defined as the sum of a quadratic data fidelity term and smoothness and sparsity regularization penalties. The parameters of the different sensors are optimized and the choice of an appropriate regularization is studied in order to improve the quality of the high resolution reconstructed images. Simulation results conducted on synthetic and real data, with different CS imagers, allow the quality of the proposed fusion method to be appreciated.

SLIC super-pixels for multi-resolution compressive spectral imaging reconstruction

espanolLas imagenes espectrales (SI) se utilizan comunmente en diferentes aplicaciones que involucran identificacion de materiales debido a que contienen informacion espacial (𝑥, 𝑦) y espectral (𝜆). Sin embargo, los metodos de adquisicion tradicionales emplean gran cantidad de datos, lo que conlleva a altos costos de almacenamiento y procesamiento. Muestreo compresivo (CS) se ha aplicado en SI con el fin de recuperar el cubo de datos a partir de un numero reducido de medidas. Las reconstrucciones se obtienen mediante algoritmos basados en la norma 𝑙₂ − 𝑙₁ cuya complejidad computacional crece en proporcion al numero de incognitas. En este trabajo, se propone un modelo de reconstruccion de resolucion multiple basado en el algoritmo iterativo de agrupamiento lineal simple (SLIC) para disminuir el numero de incognitas a recuperar. Los resultados de simulacion muestran que el metodo propuesto es hasta un 86% mas rapido que las reconstrucciones de resolucion completa. Ademas, el enfoque de MR obtiene una reconstruccion mas precisa en hasta 12dB de PSNR EnglishSpectral imaging (SI) is widely used in different applications involving material identification since it contains both spatial (𝑥, 𝑦) and spectral information (𝜆). However, traditional SI scanning methods involve massive amounts of data, which increase the cost of storing and processing. Compressive sensing (CS) theory has been applied in SI, such that the underlying data cube can be recovered from a reduced number of measures. Reconstructions are obtained by 𝑙₂ − 𝑙₁ norm-based algorithms whose computational complexity grows in proportion to the number of unknowns. In this paper, a multiresolution reconstruction model based on the simple linear iterative clustering (SLIC) is proposed to reduce the number of unknown values to recover. Simulation results show that the proposed method is up to 86% faster than the full-resolution reconstructions. Additionally, MR approach obtains more accurate reconstructions with improvements of up to 12dB of PSNR.

Compressive spectral imaging system based on liquid crystal tunable filter.

Liquid crystal tunable filters (LCTF) are extensively used in hyperspectral imaging systems to successively acquire different spectral components of scenes by adjusting the center wavelength of the filter. However, the spectral and spatial resolutions of the imager are limited by the bandwidth of LCTF, and the pitch dimension of the detector, respectively. This paper applies compressive sensing principles to improve both of the spatial and spectral resolutions of the LCTF-based hyperspectral imaging system. An accurate transmission model of the LCTF is used to represent its bandpass filtering effects on the spectra. In addition, a random coded aperture placed behind the LCTF is used to modulate the spectral images in the spatial domain. Then, the three-dimensional encoded spectral images are projected onto a two-dimensional detector. Benefiting from the spectral-dependent transmission property of the LCTF, information of the entire spectrum is collected by a few snapshots using different center wavelengths of the LCTF. Super-resolution hyperspectral images can be reconstructed from a small set of compressive measurements by solving a convex optimization problem. Simulations and experiments show that the proposed method can effectively improve the spectral and spatial resolutions of traditional LCTF-based spectral imager without changing the structures of the LCTF and detector. Finally, a multi-channel spectral coding method is proposed to further increase the compression capacity of the system.

Multi-Resolution Compressive Spectral Imaging Reconstruction from Single Pixel Measurements.

Massive amounts of data in spectral imagery increase acquisition, storing and processing costs. Compressive spectral imaging (CSI) methods allow the reconstruction of spatial and spectral information from a small set of random projections. The single pixel camera is a low cost optical architecture which enables the compressive acquisition of spectral images. Traditional CSI reconstruction methods obtain a sparse approximation of the underlying spatial and spectral information, however the complexity of these algorithms increases in proportion to the dimensionality of the data. This work proposes a multiresolution (MR) CSI reconstruction approach from single pixel camera measurements that exploits spectral similarities between pixels to group them in super-pixels such that the total number of unknowns in the inverse problem is reduced. Specifically, two different types of super-pixels are considered: rectangular and irregular structures. Simulation and experimental results show that the proposed MR scheme improves reconstruction quality in up to 6dB of PSNR and reconstruction time in up to 90% with respect to the traditional full resolution reconstructions.

Adaptive filter design via a gradient thresholding algorithm for compressive spectral imaging.

Sensing a spectral image data cube has traditionally been a time-consuming task since it requires a scanning process. In contrast, compressive spectral imaging (CSI) has attracted widespread interest since it requires fewer samples than scanning systems to acquire the data cube, thus improving the sensing speed. CSI captures linear projections of the scene, and then a reconstruction algorithm estimates the underlying scene. One notable CSI architecture is the color coded aperture snapshot spectral imager (C-CASSI), which employs pixelated filter arrays as the coding patterns to spatially and spectrally encode the incoming light. Up to date works on C-CASSI have used non-adaptive color coded apertures. Non-adaptive sampling ignores prior information about the signal to design the coding patterns. Therefore, this work proposes a method to adaptively design the color coded aperture, such that the quality of image reconstruction is improved. In more detail, this work introduces a gradient thresholding algorithm, which computes the consecutive color coded aperture from a rapidly reconstructed low-resolution version of the data cube. The successive adaptive patterns enable recovering a data cube in the presence of Gaussian noise with higher image quality. Real reconstructions and simulations evidence an improvement of up to 3dB in the quality of image reconstruction of the proposed method in comparison with state-of-the-art non-adaptive techniques.

Single Aperture Spectral+ToF Compressive Camera: Toward Hyperspectral+Depth Imagery

Spectral imaging involves the sensing of a large amount of spatial information across a multitude of wavelengths. Conventional approaches rely on scanning techniques to construct a spectral data cube. Recently, compressive spectral imaging (CSI) has allowed to estimate spectral images with as few as a single coded snapshot. On a different front, 3-D ranging imaging often involves scanning along one of the spatial dimensions to estimate the depth of an scene using structured light, or the use of two cameras as required by stereo-imaging techniques. Recently, Time-of-Flight (ToF) snapshot imaging has gained considerable attention, due to its accuracy and speed. To date, however, these imaging modalities (CSI and ToF) have been realized and implemented by separate independent imaging sensors. This paper presents the development of a single aperture compressive spectral + depth imaging camera that employs a commodity 3-D range ToF sensor as the sensing device of a coded-aperture-based compressive spectral imager. The proposed system uses a single aperture/single sensor; thus, representing a significant improvement over existing RGB+D cameras that integrate two separate image sensors, one for RGB and another for depth. In addition, the observable wavelength range of the CSI device is expanded from the visible to the near-infrared, resolving up to as many as 16 independent channels. The proposed system allows the addition of side-information by placing a grayscale or RGB camera in the same path of the single-aperture system, such that the quality of the spectral estimation is improved, while maintaining high-frame rates. We demonstrate the proposed ideas through real experimentation conducted on an assembled CSI+ToF testbed camera system.

Compressive Spectral Imaging via Polar Coded Aperture

A compressive spectral imager based on a polar coded aperture and a continuous variable circular bandpass filter is proposed for spinning munitions. As the imager rotates with the munition, compressive projections are sequentially captured with embedded spatial and spectral modulation. The polar coded aperture design is introduced, aiming at optimizing the sensing process. Both discrete and continuous rotation models of the proposed imager are derived and used to characterize the compressive imager. Computer simulations validate the computational models and the reconstruction algorithm.

Joint sparse and low rank recovery algorithm for compressive hyperspectral imaging.

Compressive spectral imaging techniques encode and disperse a hyperspectral image (HSI) to sense its spatial and spectral information with few bidimensional (2D) multiplexed projections. Recovering the original HSI from the 2D projections is carried by traditional compressive sensing-based techniques that exploit the sparsity property of natural HSI as they are represented in a proper orthonormal basis. Nevertheless, HSIs also exhibit a low rank property inasmuch only a few numbers of spectral signatures are present in the images. Specifically, when an HSI is rearranged as a matrix whose columns represent vectorized 2D spatial images in a different wavelength, this matrix is said to be low rank. Therefore, this paper proposes an HSI recovering algorithm from compressed measurements involving a joint sparse and low rank optimization problem, which seeks to jointly minimize the ℓ2-, ℓ1-, and ℓ*-norm, leading the solution to fit the given projections, and be simultaneously sparse and low rank. Several simulations, along different data sets and optical sensing architectures, show that when the low rank property is included in the inverse problem formulation, the reconstruction quality increases up to four (dB) in terms of peak signal to noise ratio.

Coded aperture design in compressive spectral imaging based on side information.

Coded aperture compressive spectral imagers (CSI) sense a three-dimensional data cube by using two-dimensional projections of the coded and spectrally dispersed input image. Recently, it has been shown that by combining spectral images acquired from a CSI sensor and a complementary sensor leads to substantial improvement in the quality of the fused image. To maximally exploit the benefits of the complementary information, the spatial structure of the coded apertures must be optimized inasmuch as these structures determine the sensing matrix properties and, accordingly, the quality of the reconstructed images. This paper proposes a method to use side information from a red-green-blue sensor to design the coded aperture patterns of a CSI imager, such that more detailed spatial images and wavelength profiles can be reconstructed. The side information is used as the input of an edge detection algorithm to approximate a version of the edges of the spectral images. The coded apertures are designed to follow the spatial structure determined by the estimated spectral edges, such that the high frequencies are promoted, leading to more detailed reconstructed spectral images. Simulations and experimental results indicate that when compared with random coded aperture structures, the designed coded apertures based on side information obtain up to 3dB improvement in the quality of the reconstructed images.

Compressive spectral image super-resolution by using singular value decomposition

Compressive sensing (CS) has been recently applied to the acquisition and reconstruction of spectral images (SI). This field is known as compressive spectral imaging (CSI). The attainable resolution of SI depends on the sensor characteristics, whose cost increases in proportion to the resolution. Super-resolution (SR) approaches are usually applied to low-resolution (LR) CSI systems to improve the quality of the reconstructions by solving two consecutive optimization problems. In contrast, this work aims at reconstructing a high resolution (HR) SI from LR compressive measurements by solving a single convex optimization problem based on the fusion of CS and SR techniques. Furthermore, the truncated singular value decomposition is used to alleviate the computational complexity of the inverse reconstruction problem. The proposed method is tested by using the coded aperture snapshot spectral imager (CASSI), and the results are compared to HR-SI images directly reconstructed from LR-SI images by using an SR algorithm via sparse representation. In particular, a gain of up to 1.5 dB of PSNR is attained with the proposed method.

An sparsity-based approach for spectral image target detection from compressive measurements acquired by the CASSI architecture

<p>Hyperspectral imaging entails data typically spanning hundreds of contiguous wavebands in a certain spectral range. Each spatial point in hyperspectral images is therefore represented by a vector whose entries correspond to the intensity on each spectral band. These images enable object and feature detection, classification, or identification based on their spectral characteristics. Novel architectures have been developed for the acquisition of compressive spectral images with just a few coded aperture focal plane array measurements. This work focuses on the development of a target detection approach in hyperspectral images directly from compressive measurements without first reconstructing the full data cube that represents the real image. Specifically, a sparsity-based target detection model that uses compressive measurement for the detection task is designed and tested using an optimization algorithm. Simulations show that it is possible to perform certain transformations to the dictionaries used in traditional target detection, in order to achieve an accurate image representation in the compressed subspace</p>

Recovering spectral images from compressive measurements using designed coded apertures and Matrix Completion Theory

<p>Spectral imaging aims to capture and process a 3-dimensional spectral image with a large amount of spectral information for each spatial location. Compressive spectral imaging techniques (CSI) increases the sensing speed and reduces the amount of collected data compared to traditional spectral imaging methods. The coded aperture snapshot spectral imager (CASSI) is an optical architecture to sense a spectral image in a single 2D coded projection by applying CSI. Typically, the 3D scene is recovered by solving an L1-based optimization problem that assumes the scene is sparse in some known orthonormal basis. In contrast, the matrix completion technique (MC) allows to recover the scene without such prior knowledge. The MC reconstruction algorithms rely on a low-rank structure of the scene. Moreover, the CASSI system uses coded aperture patterns that determine the quality of the estimated scene. Therefore, this paper proposes the design of an optimal coded aperture set for the MC methodology. The designed set is attained by maximizing the distance between the translucent elements in the coded aperture. Visualization of the recovered spectral signals and simulations over different databases show average improvement when the designed coded set is used between 1-3 dBs compared to the complementary coded aperture set, and between 3-9 dBs compared to the conventional random coded aperture set.</p>

Colored Coded Aperture Design in Compressive Spectral Imaging via Minimum Coherence

Colored coded aperture optimization in compressive spectral imaging is discussed. Based on the analysis of the coherence of the underlying sensing matrix, a general family of codes is derived. These designs lead to reconstructions of multispectral scenes of better quality than the ones obtained using the traditional random black and white coded apertures. The approach used in this work exploits the structure of the sensing matrices and reduces the problem of the design of a large-scale matrix to a subset of substantially smaller problems for which it is possible to obtain a closed form solution, leading to fast design algorithms.

Further compression of focal plane array in compressive spectral imaging architectures

Contexto: Las imágenes híper-espectrales 3D de alta resolución pueden ser capturadas en una imagen 2D mediante técnicas basadas en “sensado compresivo” (compressive sensing, en inglés). Entre estas técnicas, hay una denominada Compressive Spectral Imaging (CSI), de la cual se han propuesto diversas arquitecturas en los últimos ocho años. Una cámara óptica especialmente diseñada captura la información espacio-espectral de la escena e imprime proyecciones en un plano focal 2D (Focal Plane Array, FPA). Estas muestras se pueden transmitir o almacenar; luego la imagen original puede ser reconstruida usando comúnmente un algoritmo de optimización de la norma . El tamaño en bytes del FPA es menor que la imagen original, y éste por lo tanto puede ser considerado una versión comprimida en 2D de la imagen original en 3D.Objetivo: Realizar una compresión adicional del FPA para cuatro arquitecturas CSI, para incrementar las velocidades de transmisión o disminuir tamaños de almacenamiento.Método: En este trabajo se presentan los resultados de esta compresión adicional usando la codificación aritmética y se propone una transformación inversa que se aplica al FPA con base en la estructura de los filtros ópticos y los códigos de apertura de las cámaras usadas en CSI, lo cual permite aumentar su factor de compresión.Resultados: Los resultados muestran que el factor de compresión aumenta entre 1 y 2 puntos en tres de las arquitecturas.Conclusiones: A pesar de que hay pérdidas de datos en el proceso de transformación-cuantificación-compresión-descompresión del FPA, para cada arquitectura CSI usada, la calidad del cubo de datos posteriormente reconstruido expresada en el PSNR entre la imagen original y la reconstruida, no difiere significativamente de la versión original.