All-optical Imaging Research Articles

Neural single-shot GHz FMCW correlation imaging

Depth sensing is essential for 3D environmental perception across application domains, including autonomous driving, topographical mapping, and augmented and virtual reality (AR/VR). Traditional correlation time-of-flight (ToF) methods, while are able to produce dense high-resolution depth maps, are plagued by phase wrapping artifacts which limit their effective depth range. Though multi-frequency methods can help reduce this problem by simultaneously solving for phase wrap counts in multiple wavelengths, this requires multiple measurements per pixel, necessitating additional hardware and imaging time. We introduce a 3D imaging method that requires a single per-point measurement by combining frequency-modulated continuous wave (FMCW) operation, all-optical correlation ToF imaging, and a specialized frequency-decoding network. Our system performs all-optical correlation imaging at GHz rates. The method is validated through both simulations and real-world experiments, comparing favorably to existing methods in all experiments.

Brillouin gain microscopy

Optical imaging techniques with mechanical contrast, including passive microrheology, optical coherence elastography and Brillouin microscopy, are critical for material and biological discovery owing to their less perturbative nature compared with traditional mechanical imaging methods. An emerging optical microscopy approach for mechanical imaging is stimulated Brillouin scattering microscopy, which has been shown to be useful for biomechanical imaging with high sensitivity and specificity. However, the excitation energy used is high and the temporal resolution remains limited by the need to acquire full spectra. Here we develop Brillouin gain microscopy that detects the Brillouin gain at a specific mechanically contrasting frequency corresponding to a Brillouin acoustic-vibrational mode of interest in the sample. Brillouin gain microscopy affords a 200-fold improvement in temporal resolution compared with stimulated Brillouin scattering microscopy, down to 100 μs at excitation energy as low as 23 μJ. Using Brillouin gain microscopy, we demonstrate cross-sectional, all-optical mechanical imaging of materials as well as of the structure and dynamics in living systems with low excitation energy and at high temporal resolution.

All-optical complex field imaging using diffractive processors.

Complex field imaging, which captures both the amplitude and phase information of input optical fields or objects, can offer rich structural insights into samples, such as their absorption and refractive index distributions. However, conventional image sensors are intensity-based and inherently lack the capability to directly measure the phase distribution of a field. This limitation can be overcome using interferometric or holographic methods, often supplemented by iterative phase retrieval algorithms, leading to a considerable increase in hardware complexity and computational demand. Here, we present a complex field imager design that enables snapshot imaging of both the amplitude and quantitative phase information of input fields using an intensity-based sensor array without any digital processing. Our design utilizes successive deep learning-optimized diffractive surfaces that are structured to collectively modulate the input complex field, forming two independent imaging channels that perform amplitude-to-amplitude and phase-to-intensity transformations between the input and output planes within a compact optical design, axially spanning ~100 wavelengths. The intensity distributions of the output fields at these two channels on the sensor plane directly correspond to the amplitude and quantitative phase profiles of the input complex field, eliminating the need for any digital image reconstruction algorithms. We experimentally validated the efficacy of our complex field diffractive imager designs through 3D-printed prototypes operating at the terahertz spectrum, with the output amplitude and phase channel images closely aligning with our numerical simulations. We envision that this complex field imager will have various applications in security, biomedical imaging, sensing and material science, among others.

An S-CNN-based phase conjugation method in imaging through random media

Imaging through random media is a pervasive issue in many areas. Recently, deep learning (DL) has shown strong capability in speckle reconstruction, especially in extracting the correlation properties of speckle patterns. However, in practical imaging scenarios, most DL methods need numerous training data to adapt to various imaging condition changes. Here, we develop an S-CNN-based phase conjugation method in imaging through random media. Specifically, we propose the amplitude-phase distortion model to characterize the light scattering process. Then, we further develop a scattering convolutional neural network (S-CNN) based on our model to map speckle patterns to scattered light fields in the changed imaging conditions. Finally, we reconstruct the object through phase conjugation. We quantitatively validated the imaging performance of S-CNN-based phase conjugation imaging, meeting the possible changes of practical imaging conditions, encompassing unseen types of different sparse objects, different diffusers, different detection positions, and different apertures on the imaging path. We found that the Jaccard index (JI), Pearson correlation coefficient (PCC), and structural similarity measure (SSIM) in different physical scenarios were increased by more than 21%, 22%, and 8%, compared with digital CNN. The proposed method provides a new optical imaging system that can be adapted to accommodate various condition changes in practical imaging scenarios and paves the way for an all-optical imaging system in imaging through random media.

Short-wave Infrared Photoluminescence Lifetime Mapping of Rare-Earth Doped Nanoparticles Using All-Optical Streak Imaging.

The short-wave infrared (SWIR) photoluminescence lifetimes of rare-earth doped nanoparticles (RENPs) have found diverse applications in fundamental and applied research. Despite dazzling progress in the novel design and synthesis of RENPs with attractive optical properties, existing optical systems for SWIR photoluminescence lifetime imaging are still considerably restricted by inefficient photon detection, limited imaging speed, and low sensitivity. To overcome these challenges, SWIR photoluminescence lifetime imaging microscopy using an all-optical streak camera (PLIMASC) is developed. Synergizing scanning optics and a high-sensitivity InGaAs CMOS camera, SWIR-PLIMASC has a 1D imaging speed of up to 138.9kHz in the spectral range of 900-1700nm, which quantifies the photoluminescence lifetime of RENPs in a single shot. A 2D photoluminescence lifetime map can be acquired by 1D scanning of the sample. To showcase the power of SWIR-PLIMASC, a series of core-shell RENPs with distinct SWIR photoluminescence lifetimes is synthesized. In particular, using Er3+ -doped RENPs, SWIR-PLIMASC enables multiplexed anti-counterfeiting. Leveraging Ho3+ -doped RENPs as temperature indicators, this system is applied to SWIR photoluminescence lifetime-based thermometry. Opening up a new avenue for efficient SWIR photoluminescence lifetime mapping, this work is envisaged to contribute to advanced materials characterization, information science, and biomedicine.

Direct and integrating sampling in terahertz receivers from wafer-scalable InAs nanowires

Terahertz (THz) radiation will play a pivotal role in wireless communications, sensing, spectroscopy and imaging technologies in the decades to come. THz emitters and receivers should thus be simplified in their design and miniaturized to become a commodity. In this work we demonstrate scalable photoconductive THz receivers based on horizontally-grown InAs nanowires (NWs) embedded in a bow-tie antenna that work at room temperature. The NWs provide a short photoconductivity lifetime while conserving high electron mobility. The large surface-to-volume ratio also ensures low dark current and thus low thermal noise, compared to narrow-bandgap bulk devices. By engineering the NW morphology, the NWs exhibit greatly different photoconductivity lifetimes, enabling the receivers to detect THz photons via both direct and integrating sampling modes. The broadband NW receivers are compatible with gating lasers across the entire range of telecom wavelengths (1.2–1.6 μm) and thus are ideal for inexpensive all-optical fibre-based THz time-domain spectroscopy and imaging systems. The devices are deterministically positioned by lithography and thus scalable to the wafer scale, opening the path for a new generation of commercial THz receivers.

Advances in Atomic Time Scale Imaging with a Fine Intrinsic Spatial Resolution

Atomic time scale imaging, opening a new era for studying dynamics in microcosmos, is presently attracting immense research interest on the global level due to its powerful ability. On the atom level, physics, chemistry, and biology are identical for researching atom motion and atomic state change. The light possesses twoness, the information carrier and the research resource. The most fundamental principle of this imaging is that light records the event-modulated light field by itself, so-called all-optical imaging. This paper can answer what is the essential standard to develop and evaluate atomic time scale imaging, what is the optimal imaging system, and what are the typical techniques to implement this imaging, up to now. At present, the best record in the experiment, made by multistage optical parametric amplification (MOPA), is realizing 50-fs resolved optical imaging with a spatial resolution of ~83 lp/mm at an effective framing rate of 15 × 10 12 fps for recording an ultrafast optical lattice with its rotating speed up to 13.5 × 10 12 rad/s.

Voltage-Seq: all-optical postsynaptic connectome-guided single-cell transcriptomics

Understanding the routing of neuronal information requires the functional characterization of connections. Neuronal projections recruit large postsynaptic ensembles with distinct postsynaptic response types (PRTs). PRT is typically probed by low-throughput whole-cell electrophysiology and is not a selection criterion for single-cell RNA-sequencing (scRNA-seq). To overcome these limitations and target neurons based on specific PRTs for soma harvesting and subsequent scRNA-seq, we created Voltage-Seq. We established all-optical voltage imaging and recorded the PRT of 8,347 neurons in the mouse periaqueductal gray (PAG) evoked by the optogenetic activation of ventromedial hypothalamic (VMH) terminals. PRTs were classified and spatially resolved in the entire VMH-PAG connectome. We built an onsite analysis tool named VoltView to navigate soma harvesting towards target PRTs guided by a classifier that used the VMH-PAG connectome database as a reference. We demonstrated Voltage-seq by locating VMH-driven γ-aminobutyric acid-ergic neurons in the PAG, guided solely by the onsite classification in VoltView.

Miniaturised dual-modality all-optical ultrasound probe for laser interstitial thermal therapy (LITT) monitoring.

All-optical ultrasound (OpUS) has emerged as an imaging paradigm well-suited to minimally invasive imaging due to its ability to provide high resolution imaging from miniaturised fibre optic devices. Here, we report a fibre optic device capable of concurrent laser interstitial thermal therapy (LITT) and real-time in situ all-optical ultrasound imaging for lesion monitoring. The device comprised three optical fibres: one each for ultrasound transmission, reception and thermal therapy light delivery. This device had a total lateral dimension of <1 mm and was integrated into a medical needle. Simultaneous LITT and monitoring were performed on ex vivo lamb kidney with lesion depth tracked using M-mode OpUS imaging. Using one set of laser energy parameters for LITT (5 W, 60 s), the lesion depth varied from 3.3 mm to 8.3 mm. In all cases, the full lesion depth could be visualised and measured with the OpUS images and there was a good statistical agreement with stereomicroscope images acquired after ablation (t=1.36, p=0.18). This work demonstrates the feasibility and potential of OpUS to guide LITT in tumour resection.

Ultrasound pulse generation through continuous-wave laser excited thermo-cavitation for all-optical ultrasound imaging

The optical generation of pulsed ultrasound is attractive to nondestructive testing and biological imaging, especially for those involving narrow operation space or strong electro-magnetic interference. However, conventional techniques based on the photoacoustic effect inevitably required an expensive high-energy short pulsed laser and dedicated preparation of the optically absorptive composite film. Here, a fiber-optic ultrasound pulse transmitter based on continuous-wave (CW) laser triggered thermo-cavitation was demonstrated. The fiber-delivered CW laser light heated the highly-absorptive copper nitrate solution and generated explosive bubbles , which emitted strong ultrasound waves. Omnidirectional ultrasound pulses with an amplitude up to 0.3 MPa and a repetition rate of 5 kHz in the frequency range of 5–12 MHz were obtained by using a 50 mW optical heating power at a wavelength of 980 nm. The fiber-tip ultrasound transmitter was integrated with a polymer-cavity-based fiber ultrasound detector to construct an all-fiber ultrasound endoscopic imaging probe. Without the need for a wavelength-tunable laser, the ultrasound detector was interrogated by CW laser light with a fixed wavelength, coupled with feedback-controlled heating of the cavity to stabilize its spectral fringe. The CW laser-driven fiber ultrasound transmitter, in combination with the photothermally stabilized fiber ultrasound detector, opens new routes for a number of ultrasound-related industrial and biomedical applications.

Fast interrogation wavelength tuning for all-optical photoacoustic imaging.

Optical detection of ultrasound for photoacoustic imaging provides a large bandwidth and high sensitivity at high acoustic frequencies. Therefore, higher spatial resolutions can be achieved using Fabry-Pérot cavity sensors than conventional piezoelectric detection. However, fabrication constraints during the deposition of the sensing polymer layer require precise control of the interrogation beam wavelength to provide optimal sensitivity. This is commonly achieved by employing slowly tunable narrowband lasers as interrogation sources, hence limiting the acquisition speed. We propose instead to use a broadband source and a fast-tunable acousto-optic filter to adjust the interrogation wavelength at each pixel within a few microseconds. We demonstrate the validity of this approach by performing photoacoustic imaging with a highly inhomogeneous Fabry-Pérot sensor.

Investigating mechanisms of laser pulse-induced reflectivity modulations in photoacoustic remote sensing with a 10 million frames-per-second camera

Photoacoustic remote sensing has been recently developed as an all-optical imaging modality capable of imaging a variety of endogenous contrast agents label-free. Initially predicted laser pulse-induced refractive index perturbation-based interrogation beam reflectivity modulations have been found to be orders of magnitude smaller than those typically observed experimentally. In this report we utilize a 10 million frames-per-second camera to further investigate these predicted reflectivity modulations, while also exploring other potential mechanisms of laser pulse-induced reflectivity modulations. Laser-induced motion is demonstrated both laterally for gold wires suspended and submerged in air and water, respectively, and carbon fibers submerged in water, and axial motion is observed in gold wires submerged in a depth gradient of intralipid solution. This laser-induced sample motion is anticipated to cause reflectivity modulations local to the interrogation beam profile in microscopy set-ups. Non-motion-based maximum intensity modulations of 3% are also observed in gold wires submerged in water, indicating the presence of the originally predicted reflectivity modulations. Overall, these observations are important as they provide a widefield view of laser-pulse interactions unavailable in previous point scanning-based photoacoustic remote sensing microscopy configurations, where observed mechanisms occur on time-scales orders of magnitude faster than equivalent field of view point scanning capabilities.

Bicistronic Expression of a High-Performance Calcium Indicator and Opsin for All-Optical Stimulation and Imaging at Cellular Resolution.

State-of-the-art all-optical systems promise unprecedented access to neural activity in vivo, using multiphoton optogenetics to allow simultaneous imaging and control of activity in selected neurons at cellular resolution. However, to achieve wide use of all-optical stimulation and imaging, simple strategies are needed to robustly and stably express opsins and indicators in the same cells. Here, we describe a bicistronic adeno-associated virus (AAV) that expresses both the fast and bright calcium indicator jGCaMP8s, and a soma-targeted (st) and two-photon-activatable opsin, ChrimsonR. With this method, stChrimsonR stimulation with two-photon holography in the visual cortex of mice drives robust spiking in targeted cells, and neural responses to visual sensory stimuli and spontaneous activity are strong and stable. Cells expressing this bicistronic construct show responses to both photostimulation and visual stimulation that are similar to responses measured from cells expressing the same opsin and indicator via separate viruses. This approach is a simple and robust way to prepare neurons in vivo for two-photon holography and imaging.

All-optical edge-enhanced proton imaging driven by an intense vortex laser

An all-optical approach to edge-enhanced proton radiography is realized by using a relativistic vortex laser irradiating on nanometer-thick foil. In the proof-of-principle experiments, the hollow proton beam was successfully produced by the transparent target normal electric field sheath in the break-out after-burner acceleration mechanism, using a superintense Laguerre–Gauss laser with the highest intensity of the laser exceeded 1020 W/cm2. An insect was imaged with the proton beam; the leg structures on the edge were clearly captured. By contrast, the dot proton source produced by a Gaussian laser was almost completely blocked by the insect's body, losing most edge information. Hollow-structured proton beams driven by vortex lasers conquer the dot imaging limit for high-energy proton beams, which may benefit imaging of capsule implosions in inertial confined fusion, instability research on expanding plasma, and precise positioning in medical therapy.

Analysis of Diffractive Neural Networks for Seeing Through Random Diffusers

Imaging through diffusive media is a challenging problem, where the existing solutions heavily rely on digital computers to reconstruct distorted images. We provide a detailed analysis of a computer-free, all-optical imaging method for seeing through random, unknown phase diffusers using diffractive neural networks, covering different deep learning-based training strategies. By analyzing various diffractive networks designed to image through random diffusers with different correlation lengths, a trade-off between the image reconstruction fidelity and distortion reduction capability of the diffractive network was observed. During its training, random diffusers with a range of correlation lengths were used to improve the diffractive network's generalization performance. Increasing the number of random diffusers used in each epoch reduced the overfitting of the diffractive network's imaging performance to known diffusers. We also demonstrated that the use of additional diffractive layers improved the generalization capability to see through new, random diffusers. Finally, we introduced deliberate misalignments in training to “vaccinate” the network against random layer-to-layer shifts that might arise due to the imperfect assembly of the diffractive networks. These analyses provide a comprehensive guide in designing diffractive networks to see through random diffusers, which might profoundly impact many fields, such as biomedical imaging, atmospheric physics, and autonomous driving.

A Simple Ca2+-Imaging Approach of Network-Activity Analyses for Human Neurons.

Rapid advances in light microscopy and development of all-optical electrophysiological imaging tools have greatly leveraged the speed and the depth of neurobiology studies. Calcium imaging is a common method that is useful for measuring calcium signals in cells and has been used as a functional proxy for neuronal activity. Here I describe a simple, stimulation-free approach that measures neuronal network activity and single-neuron dynamics in human neurons. This protocol provides the experimental workflow that includes step-wise illustrations of sample preparations, data processing, and analyses that can be used for quick phenotypical assessment and serves as a quick functional readout for mutagenesis or screen effort for neurodegenerative studies.

Computing metasurface capable of broad-band switchable anisotropic edge-enhanced imaging

We reported a novel all-optical imaging device based on a designed 2D material (metasurface) for anisotropic 2D edge enhancements. This metasurface can work as a switchable anisotropic edge filter at broadband wavelengths.

Dual-modality fibre optic probe for simultaneous ablation and ultrasound imaging

All-optical ultrasound (OpUS) is an emerging high resolution imaging paradigm utilising optical fibres. This allows both therapeutic and imaging modalities to be integrated into devices with dimensions small enough for minimally invasive surgical applications. Here we report a dual-modality fibre optic probe that synchronously performs laser ablation and real-time all-optical ultrasound imaging for ablation monitoring. The device comprises three optical fibres: one each for transmission and reception of ultrasound, and one for the delivery of laser light for ablation. The total device diameter is < 1 mm. Ablation monitoring was carried out on porcine liver and heart tissue ex vivo with ablation depth tracked using all-optical M-mode ultrasound imaging and lesion boundary identification using a segmentation algorithm. Ablation depths up to 2.1 mm were visualised with a good correspondence between the ultrasound depth measurements and visual inspection of the lesions using stereomicroscopy. This work demonstrates the potential for OpUS probes to guide minimally invasive ablation procedures in real time.

Angular dependence of spatial frequency modulation in diffusion media

An optical field will undergo coherent diffusion when it is mapped into thermal-motioned atoms, e.g., in a slow or storage light process. As was demonstrated before, such diffusion effect is equivalent to a spatial low-pass filter attenuating the high spatial frequency (SF) components of the optical field. Here, employing electromagnetically induced transparency (EIT) based light storage in hot atomic vapor, we demonstrate that the angular deviation between the control and probe beams could be utilized as a degree of freedom to modulate the SF of the probe beam. The principle is to change the diffusion-induced low-pass filter into a band-pass filter, whose SF response can be tuned by varying the direction and magnitude of the angular deviation. Transverse multimode light fields, such as optical images and Laguerre-Gaussian modes are utilized to study such SF modulation. Our findings could be broadly applied to the fields of quantum information processing, all-optical image manipulation and imaging through diffusive media.

A deep-learning approach for online cell identification and trace extraction in functional two-photon calcium imaging

In vivo two-photon calcium imaging is a powerful approach in neuroscience. However, processing two-photon calcium imaging data is computationally intensive and time-consuming, making online frame-by-frame analysis challenging. This is especially true for large field-of-view (FOV) imaging. Here, we present CITE-On (Cell Identification and Trace Extraction Online), a convolutional neural network-based algorithm for fast automatic cell identification, segmentation, identity tracking, and trace extraction in two-photon calcium imaging data. CITE-On processes thousands of cells online, including during mesoscopic two-photon imaging, and extracts functional measurements from most neurons in the FOV. Applied to publicly available datasets, the offline version of CITE-On achieves performance similar to that of state-of-the-art methods for offline analysis. Moreover, CITE-On generalizes across calcium indicators, brain regions, and acquisition parameters in anesthetized and awake head-fixed mice. CITE-On represents a powerful tool to speed up image analysis and facilitate closed-loop approaches, for example in combined all-optical imaging and manipulation experiments.