Light Field Microscopy Research Articles (Page 1)

A hybrid Fourier light field microscopy (HFLFM) system with deep learning for 3D high-resolution reconstruction

Volumetric functional imaging of transient cellular signaling and motion dynamics is often limited by hardware bandwidth and the scarcity of photons under short exposures. To overcome these challenges, we introduce squeezed light field microscopy (SLIM), a computational imaging approach that rapidly captures high-resolution three-dimensional light signals using only a single, low-format camera sensor. SLIM records over 1,000 volumes per second across a 550-µm diameter field of view and 300-µm depth, achieving 3.6-µm lateral and 6-µm axial resolution. Here we demonstrate its utility in blood cell velocimetry within the embryonic zebrafish brain and in freely moving tails undergoing high-frequency swings. Millisecond-scale temporal resolution further enables precise voltage imaging of neural membrane potentials in the leech ganglion and hippocampus of behaving mice. Together, these results establish SLIM as a versatile and robust tool for high-speed volumetric microscopy across diverse biological systems.

3D localization for light-field microscopy via convergent accelerated inertial algorithm

Light-field microscopy (LFM) enables rapid volumetric imaging through single-frame acquisition and fast 3D reconstruction algorithms. The high speed and low phototoxicity of LFM make it highly suitable for real-time 3D fluorescence imaging, such as studies of neural activity monitoring and blood flow analysis. However, in in vivo fluorescence imaging scenarios, the light intensity needs to be reduced as much as possible to achieve longer-term observations. The resulting low signal-to-noise ratio (SNR) caused by reduced light intensity significantly degrades the quality of 3D reconstruction in LFM. Existing deep-learning-based methods struggle to incorporate the structured intensity distribution and noise characteristics inherent to LFM data, often leading to artifacts and uneven energy distributions. To address these challenges, we propose the denoise-weighted view-channel-depth (DNW-VCD) network, integrating a two-step noise model and energy weight matrix into an LFM reconstruction framework. Additionally, we developed an attenuator-induced imaging system for dual-SNR image acquisition to validate DNW-VCD’s performance. Experimental results show that our method achieves artifact-reduced, real-time 3D imaging with isotropic resolution and lower phototoxicity, as verified through imaging of fluorescent beads, algae, and zebrafish heart.

Long-term and high-spatiotemporal-resolution 3D imaging of living cells remains an unmet challenge for super-resolution microscopy, owing to the noticeable phototoxicity and limited scanning speed. While emerging light-field microscopy can mitigate this issue through three-dimensionally capturing biological dynamics with merely single snapshot, it suffers from suboptimal resolution insufficient for resolving subcellular structures. Here we propose an Adaptive Learning PHysics-Assisted Light-Field Microscopy (Alpha-LFM) with a physics-assisted deep learning framework and adaptive-tuning strategies capable of light-field reconstruction of diverse subcellular dynamics. Alpha-LFM delivers sub-diffraction-limit spatial resolution (up to ~120 nm) while maintaining high temporal resolution and low phototoxicity. It enables rapid and mild 3D super-resolution imaging of diverse intracellular dynamics at hundreds of volumes per second with exceptional details. Using Alpha-LFM approach, we finely resolve the lysosome-mitochondrial interactions, capture rapid motion of peroxisome and the endoplasmic reticulum at 100 volumes per second, and reveal the variations in mitochondrial fission activity throughout two complete cell cycles of 60 h.

Innovative Applications of Light-Sheet and Light-Field Microscopy for Developmental Cardiovascular Imaging

Volumetric imaging and computation to explore contractile function in zebrafish hearts.

This work presents the Fourier Lightfield Multi-view Stereoscope (FiLM-Scope), a novel imaging device that combines concepts from Fourier Light Field Microscopy and Multi-view Stereo imaging to capture high-resolution 3D videos over large fields-of-view. The FiLM-Scope optical hardware consists of a multi-camera array, with 48 individual micro-cameras, placed behind a high-throughput primary lens. This allows the FiLM-Scope to simultaneously capture 48 unique 12.8 megapixel images of a 28 × 37 mm field-of-view, from unique angular perspectives over a 21° × 29° range, with down to 22 μm lateral resolution. We additionally describe a self-supervised algorithm to reconstruct 3D height maps from these images. Our approach demonstrates height accuracy down to 11 μm. To showcase the utility of our system, we perform tool tracking over the surface of an ex-vivo rat skull and visualize the 3D deformation in stretching human skin, with videos captured at up to 100 frames-per-second. The FiLM-Scope has the potential to improve 3D visualization in a range of micro-surgical settings.

Deep-learning based colorectal cancer pathological analysis with hyperspectral light field microscopy

Three-dimensional (3D) observation of moving cells offers critical insights into fundamental biological processes. However, the limited temporal resolution of traditional scanning-based 3D imaging techniques constrains their use in live-cell imaging. Light-field microscopy (LFM) has emerged as a scanning-free alternative, enabling rapid 3D volumetric data acquisition. Despite this advantage, LFM's spatial resolution and reconstruction artifacts remain challenges that hinder its broader application in biological imaging. This study proposes a custom-designed staggered bifocal microlens array (MLA), fabricated through a cost-effective, high-throughput method, to overcome these limitations. This approach enhances spatial resolution while minimizing reconstruction artifacts compared to conventional uniform MLAs. We demonstrate the technique through 3D imaging of microbeads at varying depths and live cells suspended in droplets. This bifocal MLA presents a promising advancement for extending LFM's applications in biological research.

Five-dimensional single-shot fluorescence imaging using a polarized Fourier light-field microscope

Light-field microscopy (LFM) and its variants have significantly advanced intravital high-speed 3D imaging. However, their practical applications remain limited due to trade-offs among processing speed, fidelity, and generalization in existing reconstruction methods. Here we propose a physics-driven self-supervised reconstruction network (SeReNet) for unscanned LFM and scanning LFM (sLFM) to achieve near-diffraction-limited resolution at millisecond-level processing speed. SeReNet leverages 4D information priors to not only achieve better generalization than existing deep-learning methods, especially under challenging conditions such as strong noise, optical aberration, and sample motion, but also improve processing speed by 700 times over iterative tomography. Axial performance can be further enhanced via fine-tuning as an optional add-on with compromised generalization. We demonstrate these advantages by imaging living cells, zebrafish embryos and larvae, Caenorhabditiselegans, and mice. Equipped with SeReNet, sLFM now enables continuous day-long high-speed 3D subcellular imaging with over 300,000 volumes of large-scale intercellular dynamics, such as immune responses and neural activities, leading to widespread practical biological applications.

In fluorescence microscopy, a persistent challenge is the defocused background that obscures cellular details and introduces artifacts. Here, we introduce Dark sectioning, a method inspired by natural image dehazing for removing backgrounds that leverages dark channel prior and dual frequency separation to provide single-frame optical sectioning. Unlike denoising or deconvolution, Dark sectioning specifically targets and removes out-of-focus backgrounds, stably improving the signal-to-background ratio by nearly 10 dB and structural similarity index measure of images by approximately tenfold. Dark sectioning was validated using wide-field, confocal, two/three-dimensional structured illumination and one/two-photon microscopy with high-fidelity reconstruction. We further demonstrate its potential to improve the segmentation accuracy in deep tissues, resulting in better recognition of neurons in the mouse brain and accurate assessment of nuclei in prostate lesions or mouse brain sections. Dark sectioning is compatible with many other microscopy modalities, including light-sheet and light-field microscopy, as well as processing algorithms, including deconvolution and super-resolution optical fluctuation imaging.

Volumetric fluorescence imaging techniques, such as confocal, multiphoton, light sheet, and light field microscopy, have become indispensable tools across a wide range of cellular, developmental, and neurobiological applications. However, it is difficult to scale such techniques to the large 3D fields of view (FOV), volume rates, and synchronicity requirements for high-resolution 4D imaging of freely behaving organisms. Here, we present reflective Fourier light field computed tomography (ReFLeCT), a high-speed volumetric fluorescence computational imaging technique. ReFLeCT synchronously captures entire tomograms of multiple unrestrained, unanesthetized model organisms across multi-millimeter 3D FOVs at 120 volumes per second. In particular, we applied ReFLeCT to reconstruct 4D videos of fluorescently labeled zebrafish and Drosophila larvae, enabling us to study their heartbeat, fin and tail motion, gaze, jaw motion, and muscle contractions with nearly isotropic 3D resolution while they are freely moving. To our knowledge, as a novel approach for snapshot tomographic capture, ReFLeCT is a major advance toward bridging the gap between current volumetric fluorescence microscopy techniques and macroscopic behavioral imaging.

Abstract Non‐invasive, dynamic monitoring of cerebral blood flow in awake animals is essential to a comprehensive understanding of the hemodynamics associated with natural sensorimotor activities. This requires both high imaging speed and deep tissue penetration for microscopy. Current second near‐infrared window (NIR‐II, 1000–1700 nm) fluorescence microscopy techniques, including multi‐photon microscopy and light‐sheet microscopy, offer greater imaging depth due to low scattering properties, but are still limited by their imaging speed. In contrast, light‐field microscopy (LFM) enables 3D image reconstruction from a single‐shot, facilitating high‐speed volumetric imaging but with limited depth penetration. Here, a NIR‐II light field microscopy (NIR‐II LFM) system is developed for real‐time 3D monitoring of cerebral blood vessels in awake mice, combined with skull‐clearing treatment. With high speed and reduced light scattering, the system achieves volumetric vessel imaging, monitors blood flow dynamics, and performs 3D cerebral hemodynamic analysis. Additionally, in an acute pathology model, the system enables real‐time observation of cerebral thrombus formation in the cortex through the skull in awake mice. As a novel optical approach, NIR‐II LFM not only facilitates dynamic 3D imaging of cerebral vasculature in vivo but also demonstrates unique advantages in studying acute vascular pathologies and beyond.

Fourier light field microscopy (FLFM) has emerged as a valuable tool for single-shot three-dimensional imaging largely due to its ability to reduce reconstruction artifacts and facilitate efficient parallel processing. However, existing research primarily concentrates on fluorescence imaging, where detection signals are incoherent, and suffer from resolution limitations inherent to the parallel sampling nature of the microlens array. This paper introduces a partially coherent FLFM (pc-FLFM) for weakly scattering samples by integrating annular partially coherent illumination (PCI) with a spectrum filtering strategy. By implementing filtering at the Fourier plane of the objective, we effectively suppress the background noise associated with PCI, thereby enhancing the accuracy of 3D image reconstruction through the Richardson-Lucy algorithm. Numerical experiments demonstrate that pc-FLFM achieves a resolution that is approximately 20% superior to conventional incoherent image techniques, signifying a notable enhancement in image quality. Furthermore, the proposed approach exhibits a significant reduction in computational complexity (over two orders of magnitude). This facilitates efficient simulation of diverse imaging scenarios, enabling the development of an optimized experimental strategy before resource-intensive physical experiments. Thus, pc-FLFM emerges as a transformative tool for single-shot, high-resolution 3D imaging for weakly scattering samples, pushing the boundaries of current optical microscopy techniques.

Adaptive specular reflection removal in light field microscopy using multi-polarization hybrid illumination and deep learning

Achieving fast, large-scale volumetric imaging with micrometer resolution has been a persistent challenge in biological microscopy. To address this challenge, we report an augmented version of light field microscopy, incorporating a motorized tilting mirror upstream of the camera. Depending on the scanning pattern, the field of view and/or the lateral resolution can be greatly improved. Our microscope technique is simple, versatile, and configured for bright-field and epifluorescence modes. We demonstrate its performance with imaging of multi-cellular aggregates of various shapes and sizes.

Animals engaged in naturalistic behavior can exhibit a large degree of behavioral variability even under sensory invariant conditions. Such behavioral variability can include not only variations of the same behavior, but also variability across qualitatively different behaviors driven by divergent cognitive states, such as fight-or-flight decisions. However, the neural circuit mechanisms that generate such divergent behaviors across trials are not well understood. To investigate this question, here we studied the visual-evoked responses of larval zebrafish to moving objects of various sizes, which we found exhibited highly variable and divergent responses across repetitions of the same stimulus. Given that the neuronal circuits underlying such behaviors span sensory, motor, and other brain areas, we built a novel Fourier light field microscope which enables high-resolution, whole-brain imaging of larval zebrafish during behavior. This enabled us to screen for neural loci which exhibited activity patterns correlated with behavioral variability. We found that despite the highly variable activity of single neurons, visual stimuli were robustly encoded at the population level, and the visual-encoding dimensions of neural activity did not explain behavioral variability. This robustness despite apparent single neuron variability was due to the multi-dimensional geometry of the neuronal population dynamics: almost all neural dimensions that were variable across individual trials, i.e. the "noise" modes, were nearly orthogonal to those encoding for sensory information. Investigating this neuronal variability further, we identified two sparsely-distributed, brain-wide neuronal populations whose pre-motor activity predicted whether the larva would respond to a stimulus and, if so, which direction it would turn on a single-trial level. These populations predicted single-trial behavior seconds before stimulus onset, indicating they encoded time-varying internal modulating behavior, perhaps organizing behavior over longer timescales or enabling flexible behavior routines dependent on the animal's internal state. Our results provide the first whole-brain confirmation that sensory, motor, and internal variables are encoded in a highly mixed fashion throughout the brain and demonstrate that de-mixing each of these components at the neuronal population level is critical to understanding the mechanisms underlying the brain's remarkable flexibility and robustness.

Droplet microfluidics (DMF) generates, manipulates and processes discrete sub-microlitre droplets, which allows for precise control and high efficiency in conducting biological assays. On-chip 3D characterization of droplets and samples moving within them is challenging. Light field microscopy (LFM) based on a microlens array (MLA) has emerged as an instantaneous volumetric imaging method, with application to DMF where it can rapidly capture 3D information. However, the trade-off between spatial resolution and depth of field and challenges with reconstruction artifacts have so far limited LFM applications in microfluidics. In this work, a novel integrated system is introduced that combines a DMF device and a bifocal MLA-based LFM system. The system enables precise droplet manipulation alongside on-chip 3D imaging and tracking of particles and live cells in a volume exceeding 500 × 500 × 300 μm3 with a temporal resolution of 100 ms. The LFM has higher spatial resolution and less reconstruction artifacts compared to LFM systems based on conventional MLAs. Experiments applying microbeads and SW480 cells validate the system's capability for effective on-chip sample manipulation and 3D characterization with a best lateral resolution of approximately 1.83 μm and an axial resolution of about 6.8 μm. Additionally, the system successfully demonstrates manipulating rapid on-chip cell lysis and 3D monitoring with a temporal resolution of 300 ms over several minutes, highlighting the synergistic benefits of combining LFM with DMF.