Single-pixel Camera Research Articles

Cascaded compressed-sensing single-pixel camera for high-dimensional optical imaging

Single-pixel detectors are popular devices in optical sciences because of their fast temporal response, high sensitivity, and low cost. However, when being used for imaging, they face a fundamental challenge in acquiring high-dimensional information of an optical field because they are essentially zero-dimensional sensors and measure only the light intensity. To address this problem, we developed a cascaded compressed-sensing single-pixel camera, which decomposes the measurement into multiple stages, sequentially reducing the dimensionality of the data from a high-dimensional space to zero dimension. This measurement scheme allows us to exploit the compressibility of a natural scene in multiple domains, leading to highly efficient data acquisition. We demonstrated our method in several demanding applications, including enabling tunable single-pixel full-waveform hyperspectral light detection and ranging (LIDAR) for the first time.

High throughput compressive fluorescence lifetime imaging with a silicon photomultiplier detector.

Fluorescence lifetime imaging microscopy (FLIM) is a powerful technique for studying biological processes. There exists a growing interest in developing strategies to enhance throughput and reduce acquisition time of FLIM systems, which commonly employ laser scanning excitation and time-correlated single-photon counting (TCSPC) detection. In this work, we propose a wide-field FLIM microscope based on compressive sensing and high photon rate detection (beyond pile-up limit) based on a high-efficiency silicon photomultiplier detector as a single-pixel camera. We experimentally validate the capabilities of this design achieving 20 frames per second FLIM images on free-moving green algae sample.

Time-domain diffuse optical imaging technique for monitoring rheumatoid arthritis disease activity: experimental validation in tissue-mimicking finger phantoms

Objective. Effective treatment within 3–5 months of disease onset significantly improves rheumatoid arthritis (RA) prognosis. Nevertheless, 30% of RA patients fail their first treatment, and it takes 3–6 months to identify failure with current monitoring techniques. Time-domain diffuse optical imaging (TD-DOI) may be more sensitive to RA disease activity and could be used to detect treatment failure. In this report, we present the development of a TD-DOI hand imaging system and validate its ability to measure simulated changes in RA disease activity using tissue-mimicking finger phantoms. Approach. A TD-DOI system was built, based on a single-pixel camera architecture, and used to image solid phantoms which mimicked a proximal interphalangeal finger joint. For reference, in silico images of virtual models of the solid phantoms were also generated using Monte Carlo simulations. Spatiotemporal Fourier components were extracted from both simulated and experimental images, and their ability to distinguish between phantoms representing different RA disease activity was quantified. Main results. Many spatiotemporal Fourier components extracted from TD-DOI images could clearly distinguish between phantoms representing different states of RA disease activity. Significance. A TD-DOI system was built and validated using finger-mimicking solid phantoms. The findings suggest that the system could be used to monitor RA disease activity. This single-pixel TD-DOI system could be used to acquire longitudinal measures of RA disease activity to detect early treatment failure.

Enhancing in full-color single-pixel imaging: integrating variable density sampling with hyper-Laplacian priors

Full-color single-pixel imaging aims to restore chromatic images using a single detector element, such as a photodiode or a single-pixel camera. However, image quality is inevitably compromised at low sampling rates due to inefficient sampling methods or incomplete representation of spectrum information. To address these challenges, we meticulously consider the distribution of the image frequency spectrum and the correlation between multiple bands and make further improvements in sampling strategy and reconstruction methods. First, we propose a variable density random sampling strategy based on the exponential distribution to enhance image sampling efficiency. Second, we discover that in most cases, there exists a hyper-Laplacian distribution between spectral mixed images and monochromatic images. Building upon this observation, we designed a hyper-Laplacian prior and seamlessly integrated it into our reconstruction method to enhance the performance of full-color images. Experimental results demonstrate that our method significantly improves the quality of reconstructed full-color images compared to state-of-the-art methods.

Single-pixel Fresnel incoherent correlation holography for 3D imaging

Fresnel incoherent correlation holography (FINCH) is an incoherent, non-scanning, and high-precision 3D imaging technique, but its area imaging characteristics limit the application of FINCH to some extent. Accordingly, we developed a single-pixel imaging method based on FINCH to realize 3D imaging. We utilized spatial light modulators (SLM) and loaded a measurement matrix on a digital micromirror array (DMD), thereby achieving phase modulation and compressed encoding simultaneously; one-dimensional data acquisition was achieved using a single-pixel camera. Finally, we combined the single-pixel reconstruction and phase-shifting algorithm to achieve incoherent correlation holography imaging with subsampling. The experimental results demonstrated that the proposed method could achieve comparable quality results at a sampling rate of 20 % compared to a 100 % sampling rate, and the method had good noise immunity. This not only avoided the limitations of area array sensors but also enabled high-fidelity imaging with a small data volume, which broadened the scope of research on the application of FINCH in the field of 3D imaging.

Fast data fitting scheme for compressive multispectral fluorescence lifetime imaging.

A single-pixel camera combined with compressive sensing techniques is a promising fluorescence microscope scheme for acquiring a multidimensional dataset (space, spectrum, and lifetime) and for reducing the measurement time with respect to conventional microscope schemes. However, upon completing the acquisition, a computational step is necessary for image reconstruction and data analysis, which can be time-consuming, potentially canceling out the beneficial effect of compressive sensing. In this work, we propose and experimentally validate a fast-fit workflow based on global analysis and multiple linear fits, which significantly reduces the computation time from tens of minutes to less than 1 s. Moreover, as the method is interlaced with the measurement flow, it can be applied in parallel with the acquisitions.

Scan-less microscopy based on acousto-optic encoded illumination.

Several optical microscopy methods are now available for characterizing scientific and industrial processes at sub-micron resolution. However, they are often ill-suited for imaging rapid events. Limited by the trade-off between camera frame-rate and sensitivity, or the need for mechanical scanning, current microscopes are optimized for imaging at hundreds of frames-per-second (fps), well-below what is needed in processes such as neuronal signaling or moving parts in manufacturing lines. Here, we present a scan-less technology that allows sub-micrometric imaging at thousands of fps. It is based on combining a single-pixel camera with parallelized encoded illumination. We use two acousto-optic deflectors (AODs) placed in a Mach-Zehnder interferometer and drive them simultaneously with multiple and unique acoustic frequencies. As a result, orthogonal light stripes are obtained that interfere with the sample plane, forming a two-dimensional array of flickering spots - each with its modulation frequency. The light from the sample is collected with a single photodiode that, after spectrum analysis, allows for image reconstruction at speeds only limited by the AOD's bandwidth and laser power. We describe the working principle of our approach, characterize its imaging performance as a function of the number of pixels - up to 400 × 400 - and characterize dynamic events at 5000 fps.

Application of compressive sensing techniques for advanced image processing and digital image transmission

The field of compressive sensing (CS) has emerged as a transformative approach in the acquisition and processing of high-dimensional data. This paper presents a comprehensive study on the application of compressive sensing techniques to advanced image processing and digital image transmission. By leveraging the inherent sparsity in natural images, CS allows for significant reductions in the amount of data required for accurate reconstruction, thereby overcoming the limitations imposed by the traditional Shannon-Nyquist sampling theorem. We explore the theoretical foundations of CS, including the principles of sparsity and incoherence, and provide a detailed overview of the Orthogonal Matching Pursuit (OMP) algorithm, a prominent greedy algorithm used for sparse signal recovery. Experimental results demonstrate the efficacy of CS in improving image reconstruction quality, as evidenced by enhancements in peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). Additionally, we discuss the practical implementation of CS in single-pixel cameras and its potential impact on future imaging technologies. The findings suggest that CS offers a robust framework for efficient image acquisition and processing, making it a valuable tool for various applications in multimedia, medical imaging, and remote sensing.

Photon-noise: is a single-pixel camera better than point scanning? A signal-to-noise ratio analysis for Hadamard and Cosine positive modulation

In a single-pixel camera, an unknown object is sequentially illuminated by intensity patterns. The total reflected or transmitted intensity is summed in a single-pixel detector from which the object is computationally reconstructed. In the situation where the measurements are limited by photon-noise, it is questionable whether a single-pixel camera performs better or worse than simply scanning the object with a focused intensity spot—a modality known as point raster scanning and employed in many laser scanning systems. Here, we solve this general question and report that positive intensity modulation based on Hadamard or Cosine patterns does not necessarily improve the single-to-noise ratio (SNR) of single-pixel cameras, as compared to point raster scanning (RS). Instead, we show that the SNR is only improved on object pixels at least k times brighter than the object mean signal , where k is a constant that depends on the modulation scheme (modulation matrix, number of detectors, etc). The constant k is derived for several widespread cases and has important consequences on the choice of the optical deign. This fundamental property is demonstrated theoretically, numerically, and is experimentally confirmed in the spatial domain (widefield fluorescence imaging) and in the spectral domain (spontaneous Raman spectral measurements). Finally, we provide user-oriented guidelines that help decide when and how multiplexing under photon-noise should be used instead of point RS.

Spatially encoded hyperspectral compressive microscope for ultrabroadband VIS/NIR hyperspectral imaging.

Hyperspectral imaging (HSI) has become a valuable tool in sample characterization in various scientific fields. While many approaches have been tested, specific applications and technology usually lead to only a narrow part of the spectrum being studied. We demonstrate the use of a broadband HSI setup based on compressed sensing capable of capturing data in visible (VIS), near-infrared (NIR), and short-wave infrared (SWIR) spectral regions. Using a tested design, we developed a dual configuration and tested its performance on a set of samples demonstrating spatial resolution and spectral reconstruction. Samples showing a potential use of the setup in optical defect detection are also tested. The setup showcases a dual single-pixel camera configuration capable of combining various detectors with a shared spatial modulation, further improving data efficiency and providing an affordable instrument from broadband spectral studies.

Maximizing atmospheric-disturbed fiber coupling efficiency with speckle-based phase retrieval and a single-pixel camera.

An approach to adaptive optics utilizing a single-pixel camera (SPC) is proposed to maximize fiber coupling efficiency at the receiver side of an optical satellite-to-ground link perturbed by atmospheric turbulence. Using a single-pixel wavefront sensor enables operation at longer optical wavelengths, such as near and far infrared, which have advantageous propagation characteristics for free space optical communication. In this approach, a focal plane intensity image of the atmospheric-disturbed wavefront is taken via an SPC using a compressed sensing technique. An iterative speckle-based phase retrieval algorithm is then applied to infer the phase distortion corrected by a deformable mirror in a feedback loop. This computational approach to inferring the phase of the wavefront overcomes the limitations of traditional Shack-Hartman-based approaches, which are difficult to implement at high speed and at the long infrared wavelengths proposed for future optical satellite communication downlinks. It has been shown that fiber coupling efficiency is increased from less than 5% to 40%-50% in medium-to-strong turbulence scenarios with the phase retrieval algorithm proposed in this work.

Computational based time-resolved multispectral fluorescence microscopy

Multispectral imaging and time-resolved imaging are two common acquisition schemes in fluorescence microscopy, and their combination can be beneficial to increase specificity. The multidimensionality of the dataset (space, time, and spectrum) introduces some challenges, such as the acquisition of big datasets and long measurement times. In this work, we present a time-resolved multispectral fluorescence microscopy system with a short measurement time, achieved by exploiting Compressive Sensing (CS) based on the Single-Pixel Camera (SPC) scheme. Data Fusion (DF) with a high-resolution camera allows us to tackle the problem of low spatial resolution, typical of SPC. The combined use of SPC, CS, and DF, in which hardware and algorithms are integrated, represents a computational imaging framework to reduce the number of measurements while preserving the information content. This approach has been exploited to demonstrate a zoom feature without moving the optical system. We describe and characterize the system in terms of spatial, spectral, and temporal properties, along with validation on a cellular sample.

Mid-infrared single-pixel imaging at the single-photon level

Single-pixel cameras have recently emerged as promising alternatives to multi-pixel sensors due to reduced costs and superior durability, which are particularly attractive for mid-infrared (MIR) imaging pertinent to applications including industry inspection and biomedical diagnosis. To date, MIR single-pixel photon-sparse imaging has yet been realized, which urgently calls for high-sensitivity optical detectors and high-fidelity spatial modulators. Here, we demonstrate a MIR single-photon computational imaging with a single-element silicon detector. The underlying methodology relies on nonlinear structured detection, where encoded time-varying pump patterns are optically imprinted onto a MIR object image through sum-frequency generation. Simultaneously, the MIR radiation is spectrally translated into the visible region, thus permitting infrared single-photon upconversion detection. Then, the use of advanced algorithms of compressed sensing and deep learning allows us to reconstruct MIR images under sub-Nyquist sampling and photon-starving illumination. The presented paradigm of single-pixel upconversion imaging is featured with single-pixel simplicity, single-photon sensitivity, and room-temperature operation, which would establish a new path for sensitive imaging at longer infrared wavelengths or terahertz frequencies, where high-sensitivity photon counters and high-fidelity spatial modulators are typically hard to access.

Nanosecond compressive fluorescence lifetime microscopy imaging via the RATS method with a direct reconstruction of lifetime maps.

The RAndom Temporal Signals (RATS) method has proven to be a useful and versatile method for measuring photoluminescence (PL) dynamics and fluorescence lifetime imaging (FLIM). Here, we present two fundamental development steps in the method. First, we demonstrate that by using random digital laser modulation in RATS, it is possible to implement the measurement of PL dynamics with temporal resolution in units of nanoseconds. Secondly, we propose an alternative approach to evaluating FLIM measurements based on a single-pixel camera experiment. In contrast to the standard evaluation, which requires a lengthy iterative reconstruction of PL maps for each timepoint, here we use a limited set of predetermined PL lifetimes and calculate the amplitude maps corresponding to each lifetime. The alternative approach significantly saves post-processing time and, in addition, in a system with noise present, it shows better stability in terms of the accuracy of the FLIM spectrogram. Besides simulations that confirmed the functionality of the extension, we implemented the new advancements into a microscope optical setup for mapping PL dynamics on the micrometer scale. The presented principles were also verified experimentally by mapping a LuAG:Ce crystal surface.

Illumination Temporal Fluctuation Suppression for Single-Pixel Imaging.

Single-pixel cameras offer improved performance in non-visible imaging compared with modern digital cameras which capture images with an array of detector pixels. However, the quality of the images reconstructed by single-pixel imaging technology fails to match traditional cameras. Since it requires a sequence of measurements to retrieve a single image, the temporal fluctuation of illumination intensity during the measuring will cause inconsistence for consecutive measurements and thus noise in reconstructed images. In this paper, a normalization protocol utilizing the differential measurements in single-pixel imaging is proposed to reduce such inconsistence with no additional hardware required. Numerical and practical experiments are performed to investigate the influences of temporal fluctuation of different degrees on image quality and to demonstrate the feasibility of the proposed normalization protocol. Experimental results show that our normalization protocol can match the performance of the system with the reference arm. The proposed normalization protocol is straightforward with the potential to be easily applied in any temporal-sequence imaging strategy.

Adaptive High-Resolution Imaging Method Based on Compressive Sensing.

Compressive sensing (CS) is a signal sampling theory that originated about 16 years ago. It replaces expensive and complex receiving devices with well-designed signal recovery algorithms, thus simplifying the imaging system. Based on the application of CS theory, a single-pixel camera with an array-detection imaging system is established for high-pixel detection. Each detector of the detector array is coupled with a bundle of fibers formed by fusion of four bundles of fibers of different lengths, so that the target area corresponding to one detector is split into four groups of target information arriving at different times. By comparing the total amount of information received by the detector with the threshold set in advance, it can be determined whether the four groups of information are calculated separately. The simulation results show that this new system can not only reduce the number of measurements required to reconstruct high quality images but can also handle situations wherever the target may appear in the field of view without necessitating an increase in the number of detectors.

Single-pixel MWIR camera for acquiring thermal images of silicone rubber encapsulated LEDs

A single-pixel mid-wavelength infrared (MWIR) thermal camera is proposed and realized. By measuring 3.7–4.7 µm MWIR, the thermal image of silicone rubber encapsulated LED chips can be reconstructed using a compressive sensing algorithm without prior measurement of the thermoelectric coefficient of the LED chips.

Compressive phase object classification using single-pixel digital holography.

A single-pixel camera (SPC) is a computational imaging system that obtains compressed signals of a target scene using a single-pixel detector. The compressed signals can be directly used for image classification, thereby bypassing image reconstruction, which is computationally intensive and requires a high measurement rate. Here, we extend this direct inference to phase object classification using single-pixel digital holography (SPDH). Our method obtains compressed measurements of target complex amplitudes using SPDH and trains a classifier using those measurements for phase object classification. Furthermore, we present a joint optimization of the sampling patterns used in SPDH and a classifier to improve classification accuracy. The proposed method successfully classified phase object images of handwritten digits from the MNIST database, which is challenging for SPCs that can only capture intensity images.

Compressive Domain Deep CNN for Image Classification and Performance Improvement Using Genetic Algorithm-Based Sensing Mask Learning

The majority of digital images are stored in compressed form. Generally, image classification using convolution neural network (CNN) is done in uncompressed form rather than compressed one. Training the CNN in the compressed domain eliminates the requirement for decompression process and results in improved efficiency, minimal storage, and lesser cost. Compressive sensing (CS) is one of the effective and efficient method for signal acquisition and recovery and CNN training on CS measurements makes the entire process compact. The most popular sensing phenomenon used in CS is based on image acquisition using single pixel camera (SPC) which has complex design implementation and usually a matrix simulation is used to represent the SPC process in numerical demonstration. The CS measurements using this phenomenon are visually different from the image and to add this in the training set of the compressed learning framework, there is a need for an inverse SPC process that is to be applied all through the training and testing dataset image samples. In this paper we proposed a simple sensing phenomenon which can be implemented using the image output of a standard digital camera by retaining few pixels and forcing the rest of the pixels to zero and this reduced set of pixels is assumed as CS measurements. This process is modeled by a binary mask application on the image and the resultant image still subjectively legible for human vision and can be used directly in the training dataset. This sensing mask has very few active pixels at arbitrary locations and there is a lot of scope to heuristically learn the sensing mask suitable for the dataset. Only very few attempts had been made to learn the sensing matrix and the sole effect of this learning process on the improvement of CNN model accuracy is not reported. We proposed to have an ablation approach to study how this sensing matrix learning improves the accuracy of the basic CNN architecture. We applied CS for two class image dataset by applying a Primitive Walsh Hadamard (PWH) binary mask function and performed the classification experiment using a basic CNN. By retaining arbitrary amount of pixel in the training and testing dataset we applied CNN on the compressed measurements to perform image classification and studied and reported the model performance in terms of training and validation accuracies by varying the amount of pixels retained. A novel Genetic Algorithm-based compressive learning (GACL) method is proposed to learn the PWH mask to optimize the model training accuracy by using two different crossover techniques. In the experiment conducted for the case of compression ratio (CR) 90% by retaining only 10% of the pixels in every images both in training and testing dataset that represent two classes, the training accuracy is improved from 67% to 85% by using diagonal crossover in offspring creation of GACL. The robustness of the method is examined by applying GACL for user defined multiclass dataset and achieved better CNN model accuracies. This work will bring out the strength of sensing matrix learning which can be integrated with advanced training models to minimize the amount of information that is to be sent to central servers and will be suitable for a typical IoT frame work.

A super-resolution fusion video imaging spectrometer based on single-pixel camera

Multispectral cameras collect image data through more spectral channels, thereby providing higher level of detail spectral information. The high resolution of large-scale multispectral data in the spectrum, space and time dimensions has become a major challenge in the design of imaging spectrometers. The idea of single-pixel imaging (SPI) has the potential to balance many indicators of imaging spectrometers due to its inherent high sensitivity and low cost. For the limitation of the reconstruction mechanism, SPI is more difficult to image transformed scenes, and is more significant for spectral imaging of transformed scenes. This paper proposes a dual optical path spectral imaging system based on SPI, SPFS (Single pixel fusion spectrometer), which greatly reduces the sampling time by reducing the SPI imaging resolution. SPFS does not require spectral unmixing work since the observed vector does not directly contain spatial information, plus the data of different bands are directly completed by the linear array spectrometer. The high spatial resolution image acquired by the RGB camera is used to fuse with the low spatial resolution spectral image reconstructed by SPI. In addition, a lightweight deep learning algorithm is used to cooperate with this system to process image fusion. Experimental verification and theoretical analysis show that the working frame rate of this scheme is able to reach at least 24 Hz, and it has the potential for special optimization under different wavelength ranges and a higher frame rate improvement space.