Filter-based Imaging Research Articles

DeepQR: single-molecule QR codes for optical gene-expression analysis

Abstract Optical imaging and single-molecule imaging, in particular, utilize fluorescent tags in order to differentiate observed species by color. The degree of color multiplexing is dependent on the available spectral detection window and the ability to distinguish between fluorophores of different colors within this window. Consequently, most single-molecule imaging techniques rely on two to four colors for multiplexing. DeepQR combines compact spectral imaging with deep learning to enable 4 color acquisition with only 3 spectral detection windows. It allows rapid high-throughput acquisition and decoding of hundreds of unique single-molecule color combinations applied here to tag native RNA targets. We validate our method with clinical samples analyzed with the NanoString gene-expression inflammation panel side by side with the commercially available NanoString nCounter system. We demonstrate high concordance with “gold-standard” filter-based imaging and over a four-fold decrease in acquisition time by applying a single snapshot to record four-color barcodes. The new approach paves the path for extreme single-molecule multiplexing.

The Calibration Methods of Geometric Parameters of Crystal for Mid-Infrared Acousto-Optic Tunable Filter-Based Imaging Systems Design.

AOTF calibration is a complex topic that has various aspects. As far as geometric calibration is concerned, it includes not only processing errors and fixing errors in the optical system, but also the error of geometric parameters of crystal (GPC). GPC is the preset input in the optical design and optimization of Zemax, which determines the key parameters, including the spatial resolution, the field of view, and aberration. In particular, the compensation of aberration during the optical design requires accurate values of GPC. However, it is currently considered ideal. Therefore, two calibration methods based on the principle of parallel tangent are proposed: (1) the minimum-central wavelength method; (2) the minimum-frequency method. The deviation of the parallel tangent incident angle calibrated by the two methods is 0.03°. As a result, the tuning curve calculated in theory with the calibrated geometric parameters of AOTF is consistent with the tuning curve measured in practice.

Polarized Intensity Ratio Constraint Demosaicing for the Division of a Focal-Plane Polarimetric Image

Polarization is an independent dimension of light wave information that has broad application prospects in machine vision and remote sensing tasks. Polarization imaging using a division-of-focal-plane (DoFP) polarimetric sensor can meet lightweight and real-time application requirements. Similar to Bayer filter-based color imaging, demosaicing is a basic and important processing step in DoFP polarization imaging. Due to the differences in the physical properties of polarization and the color of light waves, the widely studied color demosaicing method cannot be directly applied to polarization demosaicing. We propose a polarized intensity ratio constraint demosaicing model to efficiently account for the characteristics of polarization detection in this work. First, we discuss the special constraint relationship between the polarization channels. It can be simply described as: for a beam of light, the sum of the intensities detected by any two vertical ideal analyzers should be equal to the total light intensity. Then, based on this constraint relationship and drawing on the concept of guided filtering, a new polarization demosaicing method is developed. A method to directly use raw images captured by the DoFP detector as the ground truth for comparison experiments is then constructed to aid in the convenient collection of experimental data and extensive image scenarios. Results of both qualitative and quantitative experiments illustrate that our method is an effective and practical method to faithfully recover the full polarization information of each pixel from a single mosaic input image.

Target-Oriented SAR Imaging for SCR Improvement via Deep MF-ADMM-Net

Synthetic aperture radar (SAR) is an important means for target surveillance through reconstructing the microwave image of the observation area. However, under the condition of low signal-to-clutter ratio (SCR), such as a strong sea clutter situation, it is difficult to surveil targets from SAR images acquired by the traditional matched filter-based imaging methods. To improve the target surveillance performance of SAR, this article proposes a target-oriented SAR imaging method, which can enhance the desired target and improve the SCR in the reconstructed SAR images. By separating the target area from the clutter area, we first establish a target-oriented SAR imaging model, where the generalized regularization is used to characterize the features of the target, contributing to the improvement of SCR in the reconstructed image. Then, the imaging model is solved through a deep network, MF-ADMM-Net, which is obtained by unfolding an alternating direction method of multipliers (ADMM)-based iterative solution. In addition, the training strategy is formulated with the consideration of complex values. Experiments are conducted to verify the performance of image reconstruction and SCR improvement of the proposed method, and comparisons show the superiority of MF-ADMM-Net in effect and efficiency.

Measurement of 3-Dimensional cAMP Distributions in Living Cells using 4-Dimensional (x, y, z, and λ) Hyperspectral FRET Imaging and Analysis.

Cyclic AMP is a second messenger that is involved in a wide range of cellular and physiological activities. Several studies suggest that cAMP signals are compartmentalized, and that compartmentalization contributes to signaling specificity within the cAMP signaling pathway. The development of Fӧrster resonance energy transfer (FRET) based biosensors has furthered the ability to measure and visualize cAMP signals in cells. However, these measurements are often confined to two spatial dimensions, which may result in misinterpretation of data. To date, there have been only very limited measurements of cAMP signals in three spatial dimensions (x, y, and z), due to the technical limitations in using FRET sensors that inherently exhibit low signal to noise ratio (SNR). In addition, traditional filter-based imaging approaches are often ineffective for accurate measurement of cAMP signals in localized subcellular regions due to a range of factors, including spectral crosstalk, limited signal strength, and autofluorescence. To overcome these limitations and allow FRET-based biosensors to be used with multiple fluorophores, we have developed hyperspectral FRET imaging and analysis approaches that provide spectral specificity for calculating FRET efficiencies and the ability to spectrally separate FRET signals from confounding autofluorescence and/or signals from additional fluorescent labels. Here, we present the methodology for implementing hyperspectral FRET imaging as well as the need to construct an appropriate spectral library that is neither undersampled nor oversampled to perform spectral unmixing. While we present this methodology for measurement of three-dimensional cAMP distributions in pulmonary microvascular endothelial cells (PMVECs), this methodology could be used to study spatial distributions of cAMP in a range of cell types.

Sparse radar imaging based on <italic>L</italic><sub>1/2</sub> regularization theory

Sparse radar imaging seeks to provide high-performance radar imaging technique for observed scenes by using L 1/2 regularization method. This article surveys our systematic explorations and innovative practices for tackling such problem. The key contributions include a new sparse radar imaging theory based on L 1/2 regularization theory, a new sparse radar imaging model based on echo simulation operator, a new sparse radar imaging system evaluate approach based on 3D phase diagram. The synthetic aperture radar (SAR) imaging can be formulated as a L 1/2 regularization problem, which optimizes a quadratic error term of radar imaging. The method can implement SAR imaging effectively at lower sampling rates than Nyquist rate and produce higher quality images with reduced sidelobes. Echo simulation operator is constructed to generate SAR raw data by means of inverse of matched filter-based imaging procedure, instead of the exact observation matrix. The echo simulation operator based model forms a new radar imaging method, which reduces the computational complexity and memory occupation to the same order as matched filter imaging algorithm, and hence, makes L 1/2 regularization reconstruction of large-scale considered scene become possible. 3D phase diagram is employed to analyze and evaluate the performance of sparse radar imaging system. In 3D phase diagram, success rates of signal recovery can be derived from the statistics of relative errors between the ground truth and the recovered signals. The success rates for each combination of SNR, sparsity and sampling ratio are provided to form a three-dimensional phase diagram. The trend of success rates is presented accurately by taking advantage of 3D phase transition boundary. We design the sparse radar prototype and conduct a series of airborne experiments, which demonstrate the applicability and potentials of the sparse radar imaging. 3D phase diagram is used to evaluate the relationship between reconstruction performance and signal-to- noise ratio. The experimental results verity the effectiveness of the L 1/2 regularization method and the feasibility of 3D phase diagram. The sparse radar imaging based on L 1/2 regularization has promising application prospects. Firstly, it could be generalized to deal with 3D-SAR imaging, which achieves better flexibility for system design and super-resolution capability of height dimension for reconstruction property. Then, we review some of the most popular techniques to exploit sparsity, for recovering high-dimensional matrices and higher order tensors from compressive measurements, and further emphasize the broad application prospects of higher order compressive sensing in 3D-SAR imaging.

Fluorescence hyperspectral imaging (fHSI) using a spectrally resolved detector array.

The ability to resolve multiple fluorescent emissions from different biological targets in video rate applications, such as endoscopy and intraoperative imaging, has traditionally been limited by the use of filter‐based imaging systems. Hyperspectral imaging (HSI) facilitates the detection of both spatial and spectral information in a single data acquisition, however, instrumentation for HSI is typically complex, bulky and expensive. We sought to overcome these limitations using a novel robust and low cost HSI camera based on a spectrally resolved detector array (SRDA). We integrated this HSI camera into a wide‐field reflectance‐based imaging system operating in the near‐infrared range to assess the suitability for in vivo imaging of exogenous fluorescent contrast agents. Using this fluorescence HSI (fHSI) system, we were able to accurately resolve the presence and concentration of at least 7 fluorescent dyes in solution. We also demonstrate high spectral unmixing precision, signal linearity with dye concentration and at depth in tissue mimicking phantoms, and delineate 4 fluorescent dyes in vivo. Our approach, including statistical background removal, could be directly generalised to broader spectral ranges, for example, to resolve tissue reflectance or autofluorescence and in future be tailored to video rate applications requiring snapshot HSI data acquisition.

Blind source separation of ex-vivo aorta tissue multispectral images.

Blind Source Separation methods (BSS) aim for the decomposition of a given signal in its main components or source signals. Those techniques have been widely used in the literature for the analysis of biomedical images, in order to extract the main components of an organ or tissue under study. The analysis of skin images for the extraction of melanin and hemoglobin is an example of the use of BSS. This paper presents a proof of concept of the use of source separation of ex-vivo aorta tissue multispectral Images. The images are acquired with an interference filter-based imaging system. The images are processed by means of two algorithms: Independent Components analysis and Non-negative Matrix Factorization. In both cases, it is possible to obtain maps that quantify the concentration of the main chromophores present in aortic tissue. Also, the algorithms allow for spectral absorbance of the main tissue components. Those spectral signatures were compared against the theoretical ones by using correlation coefficients. Those coefficients report values close to 0.9, which is a good estimator of the method's performance. Also, correlation coefficients lead to the identification of the concentration maps according to the evaluated chromophore. The results suggest that Multi/hyper-spectral systems together with image processing techniques is a potential tool for the analysis of cardiovascular tissue.

Development of a dual-path system for band-to-band registration of an acousto-optic tunable filter-based imaging spectrometer

Accurate band-to-band registration is crucial when an acousto-optic tunable filter (AOTF)-based imaging spectrometer is used to acquire spectral data on an unstable platform. However, it is difficult to registrate hyperspectral images accurately using traditional image registration algorithms, which will seriously affect the accurate acquisition of spectra and reduce the classification accuracy. Here we demonstrate an optical configuration that acquires the diffracted and undiffracted beam of the AOTF simultaneously. These two beams have identical geometrical optical characteristic and suffer from the same image motion caused by the platform jitter. Compared with the diffracted beam, however, the intensity of the undiffracted beam is changeless when tuning the AOTF. Therefore, the use of undiffracted beams allows accurate measurement of image motion in different bands. The experiment shows that this technique greatly improves the registration accuracy of spectral images with poor correlation and low SNR.

Tunable filter-based multispectral imaging for detection of blood stains on construction material substrates part 2: realization of rapid blood stain detection

Based on the blood stain detection method and criteria established in part 1 of this article, we combine and organize all necessary tasks to realize the multispectral imaging-based rapid blood stain detection system. To rapidly detect blood stains on the test surface, the developed system automatically captures the spectral images, extracts their spectral data, determines the positions of blood stains, and accurately highlights the positions of blood stains on the display. To achieve such a system, several tasks are newly introduced, including adjustment of camera exposure times to prevent image saturation or excessive darkness, the search for the sampled clean positions of the substrate to determine the substrate reflectance spectrum, and suitable detection procedures and proper arrangement of criteria to eliminate unnecessary calculations. Parallel processes between image capturing and blood stain identification help shorten the time for blood stain identifications despite a large amount of spectral data to be processed. The developed system can identify blood against several other reddish brown stains on several substrates. The measured average identification times on different test surfaces range from only 23.3 to 28.7 s, including the image capturing process.

Improved Wavelet-based Spatial Filter of Damage Imaging Method on Composite Structures

Piezoelectric sensor array-based spatial filter technology is a new promising method presented in research area of structural health monitoring (SHM) in the recent years. To apply this method to composite structures and give the actual position of damage, this paper proposes a spatial filter-based damage imaging method improved by complex Shannon wavelet transform. The basic principle of spatial filter is analyzed first. Then, this paper proposes a method of using complex Shannon wavelet transform to construct analytic signals of time domain signals of PZT sensors array. The analytic signals are synthesized depending on the principle of the spatial filter to give a damage imaging in the form of angle-time. A method of converting the damage imaging to the form of angle-distance is discussed. Finally, an aircraft composite oil tank is adopted to validate the damage imaging method. The validating results show that this method can recognize angle and distance of damage successfully.

Additional correction for energy transfer efficiency calculation in filter-based Förster resonance energy transfer microscopy for more accurate results

Forster resonance energy transfer (FRET) microscopy is commonly used to monitor protein interactions with filter-based imaging systems, which require spectral bleedthrough (or cross talk) correction to accurately measure energy transfer efficiency (E). The double-label (donor+acceptor) specimen is excited with the donor wavelength, the acceptor emission provided the uncorrected FRET signal and the donor emission (the donor channel) represents the quenched donor (qD), the basis for the E calculation. Our results indicate this is not the most accurate determination of the quenched donor signal as it fails to consider the donor spectral bleedthrough (DSBT) signals in the qD for the E calculation, which our new model addresses, leading to a more accurate E result. This refinement improves E comparisons made with lifetime and spectral FRET imaging microscopy as shown here using several genetic (FRET standard) constructs, where cerulean and venus fluorescent proteins are tethered by different amino acid linkers.

Mapping the Architecture of Secretin Receptors with Intramolecular Fluorescence Resonance Energy Transfer Using Acousto-Optic Tunable Filter-Based Spectral Imaging

The molecular structure and agonist-induced conformational changes of class II G protein-coupled receptors are poorly understood. In this work, we developed and characterized a series of dual cyan fluorescent protein (CFP)-tagged and yellow fluorescent protein (YFP)-tagged secretin receptor constructs for use in various functional and fluorescence analyses of receptor structural variants. CFP insertions within the first or second intracellular loop domains of this receptor were tolerated poorly or partially, respectively, in receptors tagged with a carboxyl-terminal yellow fluorescent protein that itself had no effect on secretin binding or cAMP production. A similar CFP insertion into the third intracellular loop resulted in a plasma membrane-localized receptor that bound secretin and signaled normally. This fully active third-loop variant exhibited a significant decrease in fluorescence resonance energy transfer signals that were recorded with an acousto-optic tunable filter microscope after exposure to secretin agonist but not to a receptor antagonist. These data demonstrate changes in the relative positions of intracellular structures that support a model for secretin receptor activation.