Single-pixel Camera Research Articles

Efficient location, imaging and recognition of faces by single-pixel camera

Face recognition is a problem with many practical applications. Modern image analysis methods such as object tracking, feature extraction, classification or verification that explore advanced techniques of machine learning and compressive sensing have been used for this purpose. Many of these methods, which are usually applied in image post-processing, are adaptable to fast intelligent viewing with a single-pixel camera. We study such a camera in the context of face recognition with limited information. We seek the optimal basis of patterns for the camera as well as to resolve practical issues related to face localisation and alignment. We compare the use of the Hadamard and eigenface bases for imaging and verification of faces. For the latter task, we develop a simple algorithm based on compressive sensing.

Hyperspectral imaging system based on a single-pixel camera design for detecting differences in tissue properties.

Optical spectroscopy can be used to distinguish between healthy and diseased tissue. In this study, the design and testing of a single-pixel hyperspectral imaging (HSI) system that uses autofluorescence emission from collagen (400nm) and nicotinamide adenine dinucleotide phosphate (475nm) along with differences in the optical reflectance spectra to differentiate between healthy and thermally damaged tissue is discussed. The changes in protein autofluorescence and reflectance due to thermal damage are studied in ex vivo porcine tissue models. Thermal lesions were created in porcine skin (n=12) and liver (n=15) samples using an IR laser. The damaged regions were clearly visible in the hyperspectral images. Sizes of the thermally damaged regions as measured via HSI are compared to sizes of these regions as measured in white-light images and via physical measurement. Good agreement between the sizes measured in the hyperspectral images, white-light imaging, and physical measurements were found. The HSI system can differentiate between healthy and damaged tissue. Possible applications of this imaging system include determination of tumor margins during surgery/biopsy and cancer diagnosis and staging.

Quantitative traceable temperature measurement using novel thermal imaging camera.

Conventional thermal imaging cameras, based on focal-plane array (FPA) sensors, exhibit inherent problems: such as stray radiation, cross-talk and the calibration uncertainty of ensuring each pixel behaves as if it were an identical temperature sensor. Radiation thermometers can largely overcome these issues, comprising of only a single detector element that can be optimised and calibrated. Although the latter approach can provide excellent accuracy for single-point temperature measurement, it does not provide a temperature image of the target object. In this work, we present a micromechanical systems (MEMS) mirror and silicon (Si) avalanche photodiode (APD) based single-pixel camera, capable of producing quantitative thermal images at an operating wavelength of 1 µm. This work utilises a custom designed f-theta wide-angle lens and MEMS mirror, to scan +/- 30° in both x- and y-dimensions, without signal loss due to vignetting at any point in the field of view (FOV). Our single-pixel camera is shown to perform well, with 3 °C size-of-source effect (SSE) related temperature error and can measure below 700 °C whilst achieving ± 0.5 °C noise related measurement uncertainty. Our measurements were calibrated and traceable to the International Temperature Scale of 1990 (ITS-90). The combination of low SSE and absence of vignetting enables quantitative temperature measurements over a spatial field with measurement uncertainty at levels lower than would be possible with FPA based thermal imaging cameras.

Multi-Resolution Compressive Spectral Imaging Reconstruction from Single Pixel Measurements.

Massive amounts of data in spectral imagery increase acquisition, storing and processing costs. Compressive spectral imaging (CSI) methods allow the reconstruction of spatial and spectral information from a small set of random projections. The single pixel camera is a low cost optical architecture which enables the compressive acquisition of spectral images. Traditional CSI reconstruction methods obtain a sparse approximation of the underlying spatial and spectral information, however the complexity of these algorithms increases in proportion to the dimensionality of the data. This work proposes a multiresolution (MR) CSI reconstruction approach from single pixel camera measurements that exploits spectral similarities between pixels to group them in super-pixels such that the total number of unknowns in the inverse problem is reduced. Specifically, two different types of super-pixels are considered: rectangular and irregular structures. Simulation and experimental results show that the proposed MR scheme improves reconstruction quality in up to 6dB of PSNR and reconstruction time in up to 90% with respect to the traditional full resolution reconstructions.

Imaging using hyperuniform sampling with a single-pixel camera.

Digital cameras use detector arrays with regular geometry for optical sampling. Though regular arrangement was demonstrated to be optimal for two-dimensional sampling, it causes aliasing at high frequencies exceeding its Nyquist limit. Here, we proposed a randomization procedure to generate 2D hyperuniform patterns that can be used to suppress aliasing in image retrieval. Experiments are performed using a single-pixel camera, where the sampling patterns do not necessarily follow a fixed Cartesian geometry. Results demonstrate that the images reconstructed by hyperuniform patterns have a lower root mean squared error and exhibit less moiré fringes at high frequencies than the images reconstructed by regular square patterns do. Furthermore, the same conclusion can be applied to the production of conventional detector arrays, where manufacturing imperfection could be utilized to suppress frequency aliasing in image retrieval.

Complex-amplitude single-pixel imaging.

A single-pixel camera can be represented using complex-amplitude. The complex-amplitude representation of input signals and output signals enables us to perform complex-amplitude imaging of an object, particularly profilometry with reflectance measurements or quantitative phase imaging with transmittance measurements. The complex-amplitude representation of optical coding masks and the coherent addition that is performed by interference can directly represent Hadamard patterns having positive and negative values. Furthermore, the residual area of the mask can be used for a reference beam with phase shifting. Such a complex-amplitude imaging system with a single-beam line structure is highly stabile against external disturbances.

Compressed sensing hyperspectral imaging in the 0.9-2.5 μm shortwave infrared wavelength range using a digital micromirror device and InGaAs linear array detector.

A hyperspectral imaging system based on compressed sensing has been developed to image in the 0.9-2.5μm shortwave infrared wavelengths. With a programmable digital micromirror device utilized as spatial light modulator, we have successfully performed spectrally resolved image reconstruction with a 256-element InGaAs linear array detector without traditional raster scanning or a push-broom mechanism by a compressed sensing (CS) single-pixel camera approach. The chemical sensitivity of the imaging sensor to near-infrared (NIR) overtone signatures of hydrocarbons was demonstrated using hydrocarbon and ink patterns on glass, showing spectral selectivity for the chemical components. Compared to point-by-point raster scanning, we show that the CS scheme can effectively accelerate image acquisition with lower but reasonable quality.

A Semi Nonnegative Matrix Factorization Technique for Pattern Generalization in Single-Pixel Imaging

A single-pixel camera is a computational imaging device that only requires a single point detector to capture the image of a scene. This device measures the inner product of the scene and the spatial light modulator patterns. The image of the scene can be recovered through postprocessing the measurements obtained for a set of different patterns. Independent of the strategy used for image recovery, real acquisitions require the spatial light modulator patterns to be positive. In addition, the dark current measured in the absence of modulation must be rejected. To date, both experimental issues have been addressed empirically. In this paper, we solve these from a general perspective. Indeed, we propose to seek positive patterns that are linear combinations of the desired patterns (with negative values), and the linear transformation matrices are chosen to reject the dark current. We refer to the problem of finding the positive patterns and the linear combinations as “pattern generalization.” To the best of our knowledge, this is the first time that this problem has been introduced. In addition, we show that pattern generalization can be solved using a semi nonnegative matrix factorization algorithm. The data obtained from simulations demonstrate that our approach performs similarly to or better than conventional methods, while using fewer measurements.

Time-resolved multispectral imaging based on an adaptive single-pixel camera.

Time-resolved multispectral imaging has many applications in different fields, which range from characterization of biological tissues to environmental monitoring. In particular, optical techniques, such as lidar and fluorescence lifetime imaging, require imaging at the subnanosecond scales over an extended area. In this paper, we demonstrate experimentally a time-resolved multispectral acquisition scheme based on single-pixel imaging. Single-pixel imaging is an emerging paradigm that provides low-cost high-quality images. Here, we use an adaptive strategy that allows acquisition and image reconstruction times to be reduced drastically or full basis scans. Adaptive time-resolved multispectral imaging scheme can have significant applications in biological imaging, at scales from macroscopic to microscopic.

Mask Responses for Single-Pixel Terahertz Imaging

Terahertz (THz) radiation meaning electromagnetic radiation in the range from 0.1 (3) to 10 (30) has the unique advantage of easily penetrating many obstructions while being non-hazardous to organic tissue since it is non-ionizing. A shortcoming of this domain is the limited availability of high-sensitivity detector arrays respective THz cameras with >1k pixels. To overcome the imaging limitations of the THz domain, compressive imaging in combination with an optically controllable THz spatial light modulator is a promising approach especially when used in a single-pixel imaging modality. The imaging fidelity, performance and speed of this approach depend crucially on the imaging patterns also called masks and their properties used in the imaging process. Therefore, in this paper, it is investigated how the image quality after reconstruction is specifically influenced by the different mask types and their properties in a compressive imaging modality. The evaluation uses an liquid-crystal display based projector as spatial light modulator to derive specific guidelines for the use of binary and true greyscale masks in THz single-pixel imaging setups respective THz single-pixel cameras when used in far-field applications e.g. stand-off security imaging.

Deep learning for real-time single-pixel video

Single-pixel cameras capture images without the requirement for a multi-pixel sensor, enabling the use of state-of-the-art detector technologies and providing a potentially low-cost solution for sensing beyond the visible spectrum. One limitation of single-pixel cameras is the inherent trade-off between image resolution and frame rate, with current compressive (compressed) sensing techniques being unable to support real-time video. In this work we demonstrate the application of deep learning with convolutional auto-encoder networks to recover real-time 128 × 128 pixel video at 30 frames-per-second from a single-pixel camera sampling at a compression ratio of 2%. In addition, by training the network on a large database of images we are able to optimise the first layer of the convolutional network, equivalent to optimising the basis used for scanning the image intensities. This work develops and implements a novel approach to solving the inverse problem for single-pixel cameras efficiently and represents a significant step towards real-time operation of computational imagers. By learning from examples in a particular context, our approach opens up the possibility of high resolution for task-specific adaptation, with importance for applications in gas sensing, 3D imaging and metrology.

Signal-to-noise ratio of single-pixel cameras based on photodiodes

Single-pixel cameras have been successfully used in different imaging applications in the last years. One of the key elements affecting the quality of these cameras is the photodetector. Here, we develop a numerical model of a single-pixel camera, which takes into account not only the characteristics of the incident light but also the physical properties of the detector. In particular, our model considers the photocurrent, the dark current, the photocurrent shot noise, the dark-current shot noise, and the Johnson-Nyquist (thermal) noise of the photodiode used as a light detector. The model establishes a clear relationship between the electric signal and the quality of the final image. This allows us to perform a systematic study of the quality of the image obtained with single-pixel cameras in different contexts. In particular, we study the signal-to-noise ratio as a function of the optical power of the incident light, the wavelength, and the photodiode temperature. The results of the model are compared with those obtained experimentally with a single-pixel camera.

Single-Pixel Camera in the Visible Band With Fiber Signal Collection

Computational imaging based on projected patterns is an alternative technique to conventional imaging and removes the need for arrayed detectors. Over the last decade, compressive sensing (CS) has emerged as a potential way to perform computational imaging and has found applications in various fields. One approach with which to realize CS is the single-pixel camera (SPC), which makes it possible to acquire an object scene using a small number of coded measurements with a low-cost single-pixel sensor. A computational reconstruction algorithm is then used to recover the image from the linear measurements, which are far fewer than the number of pixels being reconstructed. Unfortunately, the noise suppression of an SPC imposes a challenging issue for reconstructing high-quality images. Nonetheless, realizing multispectral imaging with an SPC is attractive. In this paper, we present a fiber signal collection SPC that achieves high-signal-to-noise visible light imaging and multispectral imaging. We use a fiber to collect and transport the light through an ideal transmission path to replace the lens collection scheme. We develop a proof-of-concept prototype using fiber collection and analyze the system performance on object reflectivity, active area of the detector, and system noises. The fiber-based prototype system provides noticeable improvements in image quality (correlation coefficient) and the ability of multispectral (RGB) imaging compared with a conventional SPC. Our simplified approach to multispectral imaging can be readily extended to other wavebands.

Single Pixel Black Phosphorus Photodetector for Near-Infrared Imaging.

Infrared imaging systems have wide range of military or civil applications and 2D nanomaterials have recently emerged as potential sensing materials that may outperform conventional ones such as HgCdTe, InGaAs, and InSb. As an example, 2D black phosphorus (BP) thin film has a thickness-dependent direct bandgap with low shot noise and noncryogenic operation for visible to mid-infrared photodetection. In this paper, the use of a single-pixel photodetector made with few-layer BP thin film for near-infrared imaging applications is demonstrated. The imaging is achieved by combining the photodetector with a digital micromirror device to encode and subsequently reconstruct the image based on compressive sensing algorithm. Stationary images of a near-infrared laser spot (λ = 830 nm) with up to 64 × 64 pixels are captured using this single-pixel BP camera with 2000 times of measurements, which is only half of the total number of pixels. The imaging platform demonstrated in this work circumvents the grand challenges of scalable BP material growth for photodetector array fabrication and shows the efficacy of utilizing the outstanding performance of BP photodetector for future high-speed infrared camera applications.

Lensless Photoluminescence Hyperspectral Camera Employing Random Speckle Patterns

We propose and demonstrate a spectrally-resolved photoluminescence imaging setup based on the so-called single pixel camera – a technique of compressive sensing, which enables imaging by using a single-pixel photodetector. The method relies on encoding an image by a series of random patterns. In our approach, the image encoding was maintained via laser speckle patterns generated by an excitation laser beam scattered on a diffusor. By using a spectrometer as the single-pixel detector we attained a realization of a spectrally-resolved photoluminescence camera with unmatched simplicity. We present reconstructed hyperspectral images of several model scenes. We also discuss parameters affecting the imaging quality, such as the correlation degree of speckle patterns, pattern fineness, and number of datapoints. Finally, we compare the presented technique to hyperspectral imaging using sample scanning. The presented method enables photoluminescence imaging for a broad range of coherent excitation sources and detection spectral areas.

High throughput dual-wavelength temperature distribution imaging via compressive imaging

Thermal imaging is an essential tool in a wide variety of research areas. In this work we demonstrate high-throughput double-wavelength temperature distribution imaging using a modified single-pixel camera without the requirement of a beam splitter (BS). A digital micro-mirror device (DMD) is utilized to display binary masks and split the incident radiation, which eliminates the necessity of a BS. Because the spatial resolution is dictated by the DMD, this thermal imaging system has the advantage of perfect spatial registration between the two images, which limits the need for the pixel registration and fine adjustments. Two bucket detectors, which measures the total light intensity reflected from the DMD, are employed in this system and yield an improvement in the detection efficiency of the narrow-band radiation. A compressive imaging algorithm is utilized to achieve under-sampling recovery. A proof-of-principle experiment was presented to demonstrate the feasibility of this structure.

Super-Resolution Imaging Through Scattering Medium Based on Parallel Compressed Sensing

Recent studies show that compressed sensing (CS) can recover sparse signal with much fewer measurements than traditional Nyquist theorem. From another point of view, it provides a new idea for super-resolution imaging, like the emergence of single pixel camera. However, traditional methods implemented measurement matrix by digital mirror device (DMD) or spatial light modulator, which is a serial imaging process and makes the method inefficient. In this paper, we propose a super resolution imaging system based on parallel compressed sensing. The proposed method first measures the transmission matrix of the scattering sheet and then recover high resolution objects by “two-step phase shift” technology and CS reconstruction algorithm. Unlike traditional methods, the proposed method realizes parallel measurement matrix by a simple scattering sheet. Parallel means that charge-coupled device camera can obtain enough measurements at once instead of changing the patterns on the DMD repeatedly. Simulations and experimental results show the effectiveness of the proposed method.

Principles and applications of high-speed single-pixel imaging technology

Single-pixel imaging (SPI) technology has garnered great interest within the last decade because of its ability to record high-resolution images using a single-pixel detector. It has been applied to diverse fields, such as magnetic resonance imaging (MRI), aerospace remote sensing, terahertz photography, and hyperspectral imaging. Compared with conventional silicon-based cameras, single-pixel cameras (SPCs) can achieve image compression and operate over a much broader spectral range. However, the imaging speed of SPCs is governed by the response time of digital micromirror devices (DMDs) and the amount of compression of acquired images, leading to low (ms-level) temporal resolution. Consequently, it is particularly challenging for SPCs to investigate fast dynamic phenomena, which is required commonly in microscopy. Recently, a unique approach based on photonic time stretch (PTS) to achieve high-speed SPI has been reported. It achieves a frame rate far beyond that can be reached with conventional SPCs. In this paper, we first introduce the principles and applications of the PTS technique. Then the basic architecture of the high-speed SPI system is presented, and an imaging flow cytometer with high speed and high throughput is demonstrated experimentally. Finally, the limitations and potential applications of high-speed SPI are discussed.

Lensless Imaging With Compressive Ultrafast Sensing

Lensless imaging is an important and challenging problem. One notable solution to lensless imaging is a single-pixel camera that benefits from ideas central to compressive sampling. However, traditional single-pixel cameras require many illumination patterns that result in a long acquisition process. Here, we present a method for lensless imaging based on compressive ultrafast sensing. Each sensor acquisition is encoded with a different illumination pattern and produces a time series where time is a function of the photon's origin in the scene. Currently available hardware with picosecond time resolution enables time tagging photons as they arrive to an omnidirectional sensor. This allows lensless imaging with significantly fewer patterns compared to regular single-pixel imaging. To that end, we develop a framework for designing lensless imaging systems that use ultrafast detectors. We provide an algorithm for ideal sensor placement and an algorithm for optimized active illumination patterns. We show that efficient lensless imaging is possible with ultrafast measurement and compressive sensing. This paves the way for novel imaging architectures and remote sensing in extreme situations where imaging with a lens is not possible.

Imaging around corners with single-pixel detector by computational ghost imaging

We have designed a single-pixel camera with imaging around corners based on computational ghost imaging. It can obtain the image of an object when the camera cannot look at the object directly. Our imaging system explores the fact that a bucket detector in a ghost imaging setup has no spatial resolution capability. A series of experiments have been designed to confirm our predictions. This camera has potential applications for imaging around corner or other similar environments where the object cannot be observed directly.