Computational Reconstruction Algorithm Research Articles

High-Speed Clearing and High-Resolution Staining for Analysis of Various Markers for Neurons and Vessels.

Tissue clearing enables deep imaging in various tissues by increasing the transparency of tissues, but there were limitations of immunostaining of the large-volume tissues such as the whole brain. Here, we cleared and immune-stained whole mouse brain tissues using a novel clearing technique termed high-speed clearing and high-resolution staining (HCHS). We observed neural structures within the cleared brains using both a confocal microscope and a light-sheet fluorescence microscope (LSFM). The reconstructed 3D images were analyzed using a computational reconstruction algorithm. Various neural structures were well observed in three-dimensional (3D) images of the cleared brains from Gad-green fluorescent protein (GFP) mice and Thy 1-yellow fluorescent protein (YFP) mice. The intrinsic fluorescence signals of both transgenic mice were preserved after HCHS. In addition, large-scale 3D imaging of brains, immune-stained by the HCHS method using a mild detergent-based solution, allowed for the global topological analysis of several neuronal markers such as c-Fos, neuronal nuclear protein (NeuN), Microtubule-associated protein 2 (Map2), Tuj1, glial fibrillary acidic protein (GFAP), and tyrosine hydroxylase (TH) in various anatomical regions in the whole mouse brain tissues. Finally, through comparisons with various existing tissue clearing methodologies such as CUBIC, Visikol, and 3DISCO, it was confirmed that the HCHS methodology results in relatively less tissue deformation and higher fluorescence retention. In conclusion, the development of 3D imaging based on novel tissue-clearing techniques (HCHS) will enable detailed spatial analysis of neural and vascular networks present within the brain.

A fast and flexible algorithm for microstructure reconstruction combining simulated annealing and deep learning

Microstructural analyses of porous media have considerable research value when studying of macroscopic properties,and the accurate reconstruction of a digital microstructure model is an important component of this research. Computational reconstruction algorithms for microstructures have attracted much attention due to their low cost and excellent performance. However, achieving faster and more efficient reconstruction remains a challenge for computational reconstruction algorithms. The bottleneck lies in the computational reconstruction algorithms themselves, which are either too slow (traditional reconstruction algorithms) or not flexible to the training process (deep learning reconstruction algorithms). To address these limitations, we propose a fast and flexible deep learning algorithm using a neural network based on an improved simulated annealing framework (ISAF-NN). The proposed algorithm adopts structural information of the reference image to guide the network design, and uses the description function to extract feature distribution of the reference image as the objective function to complete the network optimization. Benefit from the network structure is simple and flexible, the proposed algorithm can complete training and reconstruction in a short time. By adjusting the input size, the algorithm can also achieve arbitrary sized reconstruction. The proposed algorithm is experimentally applied to several materials to verify its effectiveness and generalizability.

Ultrasonic subwavelength imaging with blind structured illumination

The resolution of ultrasonic imaging systems suffers from the diffraction limit. To solve this problem, we develop a blind structured illumination microscopy (blind-SIM) system that achieves far-field subwavelength resolution in underwater ultrasonic imaging systems. By illuminating the object with multiple structured illumination (SI) patterns generated by scattering media with subwavelength features, the spatial frequency mixing between the object and the SIs converts evanescent waves to propagate waves that can reach sensors in the far field. In this work, the acoustic SIs are generated by randomly distributed micro particles in water. The image of the object is then reconstructed from multiple measurements (each produced by a SI pattern) utilizing computational reconstruction algorithms where the precise knowledge of the SIs is not required. The developed ultrasonic blind-SIM system has the potential in improving the ultrasonic imaging resolution in medical diagnoses, underwater acoustic imaging, and communications. Our system largely reduces the imaging hardware complexity and improves the imaging speed compared with existing ultrasonic subwavelength technologies that mostly rely on the precise localization of time-varying contrast agents.

Ultrathin Optics-Free Spectrometer with Monolithically Integrated LED Excitation.

A semiconductor spectrometer chip with a monolithically integrated light-emitting diode was demonstrated. The spectrometer design was based on a computational reconstruction algorithm and a series of absorptive spectral filters directly built in to the photodetectors’ active regions. The result is the elimination of the need to employ external optics to control the incident angle of light. In the demonstration, an array of gallium nitride (GaN) based photodetectors with wavelength selectivity generated via the principle of local strain engineering were designed and fabricated. Additionally, a GaN based LED was monolithically integrated. An optical blocking structure was used to suppress the LED-photodetector interference and was shown to be essential for the spectroscopic functionality. A proof of concept using a reflection spectroscopy configuration was experimentally conducted to validate the feasibly of simultaneously operating the LED excitation light source and the photodetectors. Spectral reconstruction using a non-negative least squares (NNLS) algorithm enhanced with orthogonal matching pursuit was shown to reconstruct the signal from the reflection spectroscopy. Optics-free operation was also demonstrated.

Deep Probabilistic Imaging: Uncertainty Quantification and Multi-modal Solution Characterization for Computational Imaging

Computational image reconstruction algorithms generally produce a single image without any measure of uncertainty or confidence. Regularized Maximum Likelihood (RML) and feed-forward deep learning approaches for inverse problems typically focus on recovering a point estimate. This is a serious limitation when working with under-determined imaging systems, where it is conceivable that multiple image modes would be consistent with the measured data. Characterizing the space of probable images that explain the observational data is therefore crucial. In this paper, we propose a variational deep probabilistic imaging approach to quantify reconstruction uncertainty. Deep Probabilistic Imaging (DPI) employs an untrained deep generative model to estimate a posterior distribution of an unobserved image. This approach does not require any training data; instead, it optimizes the weights of a neural network to generate image samples that fit a particular measurement dataset. Once the network weights have been learned, the posterior distribution can be efficiently sampled. We demonstrate this approach in the context of interferometric radio imaging, which is used for black hole imaging with the Event Horizon Telescope, and compressed sensing Magnetic Resonance Imaging (MRI).

Compressive single-pixel hyperspectral imaging using RGB sensors.

Hyperspectral imaging that obtains the spatial-spectral information of a scene has been extensively applied in various fields but usually requires a complex and costly system. A single-pixel detector based hyperspectral system mitigates the complexity problem but simultaneously brings new difficulties on the spectral dispersion device. In this work, we propose a low-cost compressive single-pixel hyperspectral imaging system with RGB sensors. Based on the structured illumination single-pixel imaging configuration, the lens-free system directly captures data by the RGB sensors without dispersion in the spectral dimension. The reconstruction is performed with a pre-trained spatial-spectral dictionary, and the hyperspectral images are obtained through compressive sensing. In addition, the spatial patterns for the structured illumination and the dictionary for the sparse representation are optimized by coherence minimization, which further improve the reconstruction quality. In both spatial and spectral dimensions, the intrinsic sparse properties of the hyperspectral images are made full use of for high sampling efficiency and low reconstruction cost. This work may introduce opportunities for optimization of computational imaging systems and reconstruction algorithms towards high speed, high resolution, and low cost future.

Electron tomography for functional nanomaterials

Abstract

Fourier focusing in integral imaging with optimum visualization pixels

In this paper, we propose Fourier Volumetric Computational Reconstruction (FVCR), which is a new computational integral imaging reconstruction (CIIR) algorithm. It handles physical pixel overlapping by refocusing in the Fourier domain. Thus, the longitudinal resolution of the reconstructed 3D image is the same as optical reconstruction. It also considers the lateral resolution limit of integral imaging and minimizes the computational load in CIIR. To verify the improvements in both lateral and longitudinal resolution of the reconstructed 3D image, in this paper, we implemented optical experiments. To the best of our knowledge, this is the first report of the efficient CIIR algorithm that considers the longitudinal and lateral resolution limits of integral imaging.

Spectral-depth imaging with deep learning based reconstruction.

We develop a compact imaging system to enable simultaneous acquisition of the spectral and depth information in real time. Our system consists of a spectral camera with low spatial resolution and an RGB camera with high spatial resolution, which captures two measurements from two different views of the same scene at the same time. Relying on an elaborate computational reconstruction algorithm with deep learning, our system can eventually obtain a spectral cube with a spatial resolution of 1920 × 1080 and a total of 16 spectral bands in the visible light section, as well as the corresponding depth map with the same spatial resolution. Quantitative and qualitative results on benchmark datasets and real-world scenes show that our reconstruction results are accurate and reliable. To the best of our knowledge, this is the first attempt to capture 5D information (3D space + 1D spectrum + 1D time) with a miniaturized apparatus and without active illumination.

Abstract 46: 3D reconstruction and quantitative analysis of histology for prostate cancer

Abstract The integration of serial sectioning of tissue, digital whole slide imaging (WSI) and computational reconstruction algorithms enable the examination of histological samples in 3D at subcellular resolution. This allows visualizing and analyzing the imaged sample in its true three-dimensional context, offering a more comprehensive view on its spatial and morphological characteristics than that obtained via typical 2D examination. The advantages of reconstructing 3D models computationally using WSI over direct 3D imaging include the combination of high resolution, large sample sizes and compatibility with existing biochemical techniques such as in situ hybridization, immunohistochemistry and established histological staining and interpretation protocols. For this purpose, we developed a software pipeline to perform routine 3D reconstruction tasks for large image sizes and datasets in a fully automatic manner. The steps in our 3D reconstruction process are image acquisition, alignment of images to a shared coordinate space, and visualization of the reconstructed 3D data. We optimized algorithm selection and computationally intensive hyperparameter tuning using a quantitative benchmarking framework and Bayesian optimization. Further, we applied our existing pipeline for feature based analysis of tissue and extended it into 3D, allowing quantification and visualization of hundreds of features characterizing the tissue. We currently apply this system to characterize prostate cancer tumors. Prostate cancer is a heterogenous, often multifocal disease. In order to understand why and how certain tumor foci develop to a life-threatening disease over others, the tissue growth patterns and evolution of tumors with different genetic backgrounds need to be studied. We use mouse models of prostate cancer to study early tumor development connected to common genetic cancer alterations. By computing hundreds of features from the histology, and studying these in the spatial context, we gain important information of tumor characteristics as well as intra- and intertumor heterogeneity. In the future, we will further scale up the protocol to perform 3D reconstruction for serially sectioned human prostates in the near future. Citation Format: Pekka Ruusuvuori, Kimmo Kartasalo, Mira Valkonen, Masi Valkonen, Tapio Visakorpi, Matti Nykter, Leena Latonen. 3D reconstruction and quantitative analysis of histology for prostate cancer [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 46.

Visual quality enhancement of integral imaging by using pixel rearrangement technique with convolution operator (CPERTS)

In this paper, we propose a new visual quality enhancement of a three-dimensional (3D) computational reconstruction algorithm in integral imaging. Integral imaging can record 3D images easily using a lenslet array. However, the elemental images may have low resolution, because each image cannot use full resolution of an image sensor. To solve this problem, a computational reconstruction technique can be used to reconstruct visual-quality-enhanced 3D images from low-resolution elemental images. Our method is based on the pixel of elemental images rearrangement technique (PERT), which can provide enhanced visual quality of the reconstructed 3D image compared with that of conventional computational reconstruction algorithms. However, it has a problem in which the size of 3D scenes is different from the optical reconstruction results. Therefore, in this paper, we propose a solution considering empty spaces between back-projected pixels on the reconstruction planes and enhance the extensibility using the convolution operator. Our experimental results show the enhancement of the visual quality and resolution of the reconstructed 3D images using the point-spread function filter.

High-Speed Hyperspectral Video Acquisition By Combining Nyquist and Compressive Sampling.

We propose a novel hybrid imaging system to acquire 4D high-speed hyperspectral (HSHS) videos with high spatial and spectral resolution. The proposed system consists of two branches: one branch performs Nyquist sampling in the temporal dimension while integrating the whole spectrum, resulting in a high-frame-rate panchromatic video; the other branch performs compressive sampling in the spectral dimension with longer exposures, resulting in a low-frame-rate hyperspectral video. Owing to the high light throughput and complementary sampling, these two branches jointly provide reliable measurements for recovering the underlying HSHS video. Moreover, the panchromatic video can be used to learn an over-complete 3D dictionary to represent each band-wise video sparsely, thanks to the inherent structural similarity in the spectral dimension. Based on the joint measurements and the self-adaptive dictionary, we further propose a simultaneous spectral sparse (3S) model to reinforce the structural similarity across different bands and develop an efficient computational reconstruction algorithm to recover the HSHS video. Both simulation and hardware experiments validate the effectiveness of the proposed approach. To the best of our knowledge, this is the first time that hyperspectral videos can be acquired at a frame rate up to 100fps with commodity optical elements and under ordinary indoor illumination.

Single-Pixel Camera in the Visible Band With Fiber Signal Collection

Computational imaging based on projected patterns is an alternative technique to conventional imaging and removes the need for arrayed detectors. Over the last decade, compressive sensing (CS) has emerged as a potential way to perform computational imaging and has found applications in various fields. One approach with which to realize CS is the single-pixel camera (SPC), which makes it possible to acquire an object scene using a small number of coded measurements with a low-cost single-pixel sensor. A computational reconstruction algorithm is then used to recover the image from the linear measurements, which are far fewer than the number of pixels being reconstructed. Unfortunately, the noise suppression of an SPC imposes a challenging issue for reconstructing high-quality images. Nonetheless, realizing multispectral imaging with an SPC is attractive. In this paper, we present a fiber signal collection SPC that achieves high-signal-to-noise visible light imaging and multispectral imaging. We use a fiber to collect and transport the light through an ideal transmission path to replace the lens collection scheme. We develop a proof-of-concept prototype using fiber collection and analyze the system performance on object reflectivity, active area of the detector, and system noises. The fiber-based prototype system provides noticeable improvements in image quality (correlation coefficient) and the ability of multispectral (RGB) imaging compared with a conventional SPC. Our simplified approach to multispectral imaging can be readily extended to other wavebands.

A training-based speech regeneration approach with cascading mapping models

Computational speech reconstruction algorithms have the ultimate aim of returning natural sounding speech to aphonic and dysphonic patients as well as those who can only whisper. In particular, individuals who have lost glottis function due to disease or surgery, retain the power of vocal tract modulation to some degree but they are unable to speak anything more than hoarse whispers without prosthetic aid. While whispering can be seen as a natural and secondary aspect of speech communications for most people, it becomes the primary mechanism of communications for those who have impaired voice production mechanisms, such as laryngectomees.In this paper, by considering the current limitations of speech reconstruction methods, a novel algorithm for converting whispers to normal speech is proposed and the efficiency of the algorithm is explored. The algorithm relies upon cascading mapping models and makes use of artificially generated whispers (called whisperised speech) to regenerate natural phonated speech from whispers. Using a training-based approach, the mapping models exploit whisperised speech to overcome frame to frame time alignment problems that are inherent in the speech reconstruction process. This algorithm effectively regenerates missing information in the conventional frameworks of phonated speech reconstruction, and is able to outperform the current state-of-the-art regeneration methods using both subjective and objective criteria.

基于衍射追迹的集成成像重构算法

基于衍射追迹的集成成像重构算法

Depth Extraction of Partially Occluded 3D Objects Using Axially Distributed Stereo Image Sensing

There are several methods to record three dimensional (3D) information of objects such as lens array based integral imaging, synthetic aperture integral imaging (SAII), computer synthesized integral imaging (CSII), axially distributed image sensing (ADS), and axially distributed stereo image sensing (ADSS). ADSS method is capable of recording partially occluded 3D objects and reconstructing high-resolution slice plane images. In this paper, we present a computational method for depth extraction of partially occluded 3D objects using ADSS. In the proposed method, the high resolution elemental stereo image pairs are recorded by simply moving the stereo camera along the optical axis and the recorded elemental image pairs are used to reconstruct 3D slice images using the computational reconstruction algorithm. To extract depth information of partially occluded 3D object, we utilize the edge enhancement and simple block matching algorithm between two reconstructed slice image pair. To demonstrate the proposed method, we carry out the preliminary experiments and the results are presented.

3D Visualization of Partially Occluded Objects Using Axially Distributed Image Sensing With a Wide-Angle Lens

In this paper we propose an axially distributed image-sensing method with a wide-angle lens to capture the wide-area scene of 3D objects. A lot of parallax information can be collected by translating the wide-angle camera along the optical axis. The recorded wide-area elemental images are calibrated using compensation of radial distortion. With these images we generate volumetric slice images using a computational reconstruction algorithm based on ray back-projection. To show the feasibility of the proposed method, we performed optical experiments for visualization of a partially occluded 3D object.

Visualization of partially occluded 3D object using wedge prism-based axially distributed sensing

Recently, an axially distributed sensing system (ADS) was proposed for three-dimensional imaging and visualization. In ADS system, the 3D information cannot be collected when the coordinates of the object are close to the optical axis. In this paper, we present a wedge-prism based axially distributed sensing for improving the visual quality of 3D reconstructed images. In the proposed method, a wedge prism is placed in front of a camera and the parallax information is collected through the wedge prism by translating the wedge prism and the image sensor together along the optical axis. Accordingly, the 3D object is recorded as the improved multiple 2D perspective images within the full area of the image sensor. The volumetric images are generated from the recorded elemental images using a computational reconstruction algorithm based on ray back-projection. The proposed method is applied to partially occluded 3D object visualization. Preliminary experiments are performed to verify the approach.

Depth extraction of 3D objects using axially distributed image sensing

Axially distributed image sensing (ADS) technique is capable of capturing 3D objects and reconstructing high-resolution slice plane images for 3D objects. In this paper, we propose a computational method for depth extraction of 3D objects using ADS. In the proposed method, the high-resolution elemental images are recorded by simply moving the camera along the optical axis and the recorded elemental images are used to generate a set of 3D slice images using the computational reconstruction algorithm based on ray back-projection. To extract depth of 3D object, we propose the simple block comparison algorithm between the first elemental image and a set of 3D slice images. This provides a simple computation process and robustness for depth extraction. To demonstrate our method, we carry out the preliminary experiments of three scenarios for 3D objects and the results are presented. To our best knowledge, this is the first report to extract the depth information using an ADS method.

3D optical microscopy method based on synthetic aperture integral imaging

In this paper, we propose a 3D optical microscopy system based on synthetic aperture integral imaging (SAII) and apply it to a surface extraction application. In the proposed system, the micro-object is optically magnified in optical microscope system and the elemental images of magnified micro-object are recorded using the SAII sensing method. The recorded elemental images are used to reconstruct a set of 3D slice images using the computational reconstruction algorithm based on ray back-projection. We obtain the surface extraction of micro-object based on the estimation of depth by using the block matching algorithm between the elemental image and a set of the reconstructed 3D slice images. The longitudinal and lateral resolutions are analyzed for the proposed system. To demonstrate our system, we carry out the preliminary experiments of a micro-object and the results are presented.