High-speed Image Reconstruction Research Articles

Rapid single-photon color imaging of moving objects.

This paper outlines an experimental demonstration of a Bayesian image reconstruction approach to achieve rapid single-photon color imaging of moving objects. The capacity to extract the color of objects is important in a variety of target identification and computer vision applications. Nonetheless, it remains challenging to achieve high-speed color imaging of moving objects in low-photon flux environments. The low-photon regime presents particular challenges for efficient spectral separation and identification, while unsupervised image reconstruction algorithms are often slow and computationally expensive. In this paper, we address both of these difficulties using a combination of hardware and computational solutions. We demonstrate color imaging using a Single-Photon Avalanche Diode (SPAD) detector array for rapid, low-light-level data acquisition, with an integrated color filter array (CFA) for efficient spectral unmixing. High-speed image reconstruction is achieved using a bespoke Bayesian algorithm to produce high-fidelity color videos. The analysis is conducted first on simulated data allowing different pixel formats and photon flux scenarios to be investigated. Experiments are then performed using a plasmonic metasurface-based CFA, integrated with a 64 × 64 pixel format SPAD array. Passive imaging is conducted using white-light illumination of multi-colored, moving targets. Intensity information is recorded in a series of 2D photon-counting SPAD frames, from which accurate color information is extracted using the fast Bayesian method introduced herein. The per-frame reconstruction rate proves to be hundreds of times faster than the previous computational method. Furthermore, this approach yields additional information in the form of uncertainty measures, which can be used to assist with imaging system optimization and decision-making in real-world applications. The techniques demonstrated point the way towards rapid video-rate single-photon color imaging. The developed Bayesian algorithm, along with more advanced SPAD technology and utilization of time-correlated single-photon counting (TCSPC) will permit live 3D, color videography in extremely low-photon flux environments.

Capture the Moment: High-Speed Imaging With Spiking Cameras Through Short-Term Plasticity.

High-speed imaging can help us understand some phenomena that are too fast to be captured by our eyes. Although ultra-fast frame-based cameras (e.g., Phantom) can record millions of fps at reduced resolution, they are too expensive to be widely used. Recently, a retina-inspired vision sensor, spiking camera, has been developed to record external information at 40, 000 Hz. The spiking camera uses the asynchronous binary spike streams to represent visual information. Despite this, how to reconstruct dynamic scenes from asynchronous spikes remains challenging. In this paper, we introduce novel high-speed image reconstruction models based on the short-term plasticity (STP) mechanism of the brain, termed TFSTP and TFMDSTP. We first derive the relationship between states of STP and spike patterns. Then, in TFSTP, by setting up the STP model at each pixel, the scene radiance can be inferred by the states of the models. In TFMDSTP, we use the STP to distinguish the moving and stationary regions, and then use two sets of STP models to reconstruct them respectively. In addition, we present a strategy for correcting error spikes. Experimental results show that the STP-based reconstruction methods can effectively reduce noise with less computing time, and achieve the best performances on both real-world and simulated datasets.

High-speed image reconstruction for optically sectioned, super-resolution structured illumination microscopy

Super-resolution structured illumination microscopy (SR-SIM) is an outstanding method for visualizing the subcellular dynamics in living cells. To date, by using elaborately designed systems and algorithms, SR-SIM can achieve rapid, optically sectioned, SR observation with hundreds to thousands of time points. However, real-time observation is still out of reach for most SIM setups as conventional algorithms for image reconstruction involve a heavy computing burden. To address this limitation, an accelerated reconstruction algorithm was developed by implementing a simplified workflow for SR-SIM, termed joint space and frequency reconstruction. This algorithm results in an 80-fold improvement in reconstruction speed relative to the widely used Wiener-SIM. Critically, the increased processing speed does not come at the expense of spatial resolution or sectioning capability, as demonstrated by live imaging of microtubule dynamics and mitochondrial tubulation.

Electrical Capacitance Tomography Sensor With House Structure for Assisting Recognition of Objects

As a fast and non-invasive visualization technology, electrical capacitance tomography(ECT) avoids the problem that camera imaging technology is interfered by light and can’t identify material properties. However, the traditional ECT sensor research mainly focused on the image reconstruction of multiphase flow between electrodes, without considering the object recognition performance through the dielectric distribution of the objects. Therefore, we propose a new ECT sensor (called House Sensor) with house-like structure, which is suitable for multi features detection of household objects. We analyze the sensitivity distribution of House Sensor, and use linear back projection (LBP) and Tikhonov regularization for high-speed image reconstruction. 20 objects with different sizes and dielectric constants are visually clustered by t-distributed stochastic neighbor embedding (t-SNE) to verify the feasibility of classification, and the accuracy of Random Forest (RF) is up to 95.3% within 3.5ms. This implies that House Sensor provide a new feasible scheme to assist object recognition under non-vision conditions.

GPU accelerated Cartesian GRAPPA reconstruction using CUDA

Background and ObjectiveGRAPPA (Generalized Auto-calibrating Partially Parallel Acquisition) is an advanced parallel MRI reconstruction method (pMRI) that enables under-sampled data acquisition with multiple receiver coils to reduce the MRI scan time and reconstructs artifact free image from the acquired under-sampled data. However, the reduction in MRI scan time comes at the expense of long reconstruction time. It is because the GRAPPA reconstruction time shows exponential growth with increasing number of receiver coils. Consequently, the conventional CPU platforms may not adhere to the requirements of fast data processing for MR image reconstruction. MethodsGraphics Processing Units (GPUs) have recently emerged as a viable commodity hardware to reduce the reconstruction time of pMRI methods. This paper presents a novel GPU based implementation of GRAPPA using custom built CUDA kernels, to meet the rising demands of fast MRI processing. The proposed framework exploits intrinsic parallelism in the calibration and synthesis phases of GRAPPA reconstruction process, aiming to achieve high speed MR image reconstruction for various GRAPPA configuration settings using different number of receiver coils, auto-calibration signals (ACS), sizes of GRAPPA kernel and acceleration factors. In-vivo experiments (using 8, 12 and 30 receiver coils) are performed to compare the performance of the proposed GPU accelerated GRAPPA with the CPU based GRAPPA extensions and GPU counterpart. ResultsThe results indicate that the proposed method achieves up to ≈47.8× , ≈17× and ≈3.8× speed up gains over multicore CPU (single thread), multicore CPU (8 thread) and Gadgetron (GPU based GRAPPA) respectively, without compromising the reconstruction accuracy. ConclusionsThe proposed method reduces the GRAPPA reconstruction time by employing the calibration phase (GRAPPA weights estimation) and synthesis phase (interpolation) on GPU. Our study shows that the proposed GPU based parallel framework for GRAPPA reconstruction provides a solution for high-speed image reconstruction while maintaining the quality of the reconstructed images.

Design and Implementation of Special-Purpose Computer for Incoherent Digital Holography

We designed and implemented a special-purpose computer for incoherent color digital holography for application in three-dimensional (3D) microscopy. We focused on multiwavelength-multiplexed phase-shifting incoherent color digital holography as a method for acquiring 3D color-reconstructed images. This method records holograms faster than conventional temporal division methods. However, the reconstruction calculation of this method requires a significant amount of time, similar to the conventional method. Therefore, we designed a circuit for high-speed image reconstruction. We designed a Fresnel diffraction circuit that requires repetitive calculations to obtain a 3D color image but incurs a high calculation cost. We implemented the circuit on a field-programmable gate array (FPGA) and successfully calculated reconstructed images that were of the same quality as those calculated by the central processing unit (CPU) of a personal computer (PC). The calculation time was approximately 2.6 times faster than that of the CPU.

Three-dimensional ultrasonic imaging operation using FPGA

The feasibility of three-dimensional (3D) ultrasound imaging methods that involve computations depends on the performance of a computing system, which requires high-speed image reconstruction. Therefore, we examine the hardware (HW) implementation of the algorithm utilizing a field-programmable-gate-array (FPGA). Subsequently, we analyze the critical path delay of the HW and reduce the delay by modifying the architecture using FPGA resources to increase the maximum frequency.This paper presents a HW implementation approach for performing 3D ultrasound imaging.

On the possibility of direct Fourier reconstruction from divergent-beam projections

A great number of tomographic systems, especially those equipped with fast data acquisition techniques, scan the objects investigated by divergent (fan) X-ray beams. Fan-beam projection data require special reconstruction techniques to be implemented. Among reconstruction techniques from parallel projection data, the direct Fourier method (DFM) proved to be one of the most promising ones, especially for high-speed image reconstruction in the high-end 3-D and dynamic tomographic systems. The goal of this work is to answer the topical question: how would direct use of the DFM influence the quality of image reconstruction from the fan-beam data? The formula describing the error caused by such an approximation is derived. The conclusions deduced from the formula are confirmed by computer simulations. The boundary values of data acquisition geometry parameters have been estimated for the case of using the DFM without recalculating the fan-beam data.

An evaluation of Radon transform computations using DSP chips

The Radon transform and its inverse (a filtered backprojection) are receiving increasing attention for applications in image reconstruction. As data collection capabilities and image reconstruction algorithms have become more sophisticated, the computational intensity of these problems has drastically increased. Parallel processing techniques are being used to implement highspeed hardware designs that will speed up this computationally burdensome task. Parallel arrays of digital signal processing (DSP) chips may be used to compute the Radon transform and back-projection for high-speed image reconstruction. In this paper we describe computation of the Radon transform and back-projection using a parallel pipelined processor architecture of DSP chips and evaluate the accuracy of the computations and quality of reconstructed images. To justify the computational approach selected, alternative procedures for computation of the Radon transform and back-projection are described and their performance using the 32-bit fixed-point arithmetic of the selected DSP chips are compared. We present, evaluate, and compare the simulated performances of implementations of these procedures on two fixed-point DSP chips: the TI TMS32020 and the AT&T DSP16.

MR fluoroscopy: initial clinical studies.

Magnetic resonance (MR) fluoroscopy is a method for high-speed MR image acquisition with the goals of short acquisition time per image (500 msec or less), high image rate (10 images or more per second), and high-speed image reconstruction (150 msec or less from data acquisition to image display). The authors present their results with the first two goals in volunteers. MR fluoroscopic image data were acquired with a limited flip angle pulse sequence with reduced repetition times (TRs) and fewer phase encodings used per image. The sequence was applied continuously, and images were formed by updating one set of data with data from the most recently taken measurements. Sample head images were generated with TR/echo times as small as 11/5.5 msec and 48 phase encodings for a total acquisition time of about 500 msec. Images were acquired while the volunteer flexed his head. Artifacts from the motion became less evident on images as progressively shorter acquisition times were used.