Intensity-only Measurements Research Articles

Ptychographic lensless coherent endomicroscopy through a flexible fiber bundle

Conventional fiber-bundle-based endoscopes allow minimally invasive imaging through flexible multi-core fiber (MCF) bundles by placing a miniature lens at the distal tip and using each core as an imaging pixel. In recent years, lensless imaging through MCFs was made possible by correcting the core-to-core phase distortions pre-measured in a calibration procedure. However, temporally varying wavefront distortions, for instance, due to dynamic fiber bending, pose a challenge for such approaches. Here, we demonstrate a coherent lensless imaging technique based on intensity-only measurements insensitive to core-to-core phase distortions. We leverage a ptychographic reconstruction algorithm to retrieve the phase and amplitude profiles of reflective objects placed at a distance from the fiber tip, using as input a set of diffracted intensity patterns reflected from the object when the illumination is scanned over the MCF cores. Our approach thus utilizes an acquisition process equivalent to confocal microendoscopy, only replacing the single detector with a camera.

Experimentally realized physical-model-based frugal wave control in metasurface-programmable complex media

Metasurface-programmable radio environments are considered a key ingredient of next-generation wireless networks. Yet, identifying a metasurface configuration that yields a desired wireless functionality in an unknown complex environment was so far only achieved with closed-loop iterative feedback schemes. Here, we introduce open-loop wave control in metasurface-programmable complex media by estimating the parameters of a compact physics-based forward model. Our experiments demonstrate orders-of-magnitude advantages over deep-learning-based digital-twin benchmarks in terms of accuracy, compactness and required calibration examples. Strikingly, our parameter estimation also works without phase information and without providing measurements for all considered scattering coefficients. These unique generalization capabilities of our pure-physics model unlock unforeseen and previously inaccessible frugal wave control protocols that significantly alleviate the measurement complexity. For instance, we achieve coherent wave control (focusing or perfect absorption) and phase-shift-keying backscatter communications in metasurface-programmable complex media with intensity-only measurements. Our approach is also directly relevant to dynamic metasurface antennas, microwave-based signal processors and emerging in situ reconfigurable nanophotonic, optical and room-acoustical systems.

Alternating projection combined with fast gradient projection (FGP-AP) method for intensity-only measurement optical diffraction tomography in LED array microscopy.

Optical diffraction tomography (ODT) is a powerful label-free measurement tool that can quantitatively image the three-dimensional (3D) refractive index (RI) distribution of samples. However, the inherent "missing cone problem," limited illumination angles, and dependence on intensity-only measurements in a simplified imaging setup can all lead to insufficient information mapping in the Fourier domain, affecting 3D reconstruction results. In this paper, we propose the alternating projection combined with the fast gradient projection (FGP-AP) method to compensate for the above problem, which effectively reconstructs the 3D RI distribution of samples using intensity-only images captured from LED array microscopy. The FGP-AP method employs the alternating projection (AP) algorithm for gradient descent and the fast gradient projection (FGP) algorithm for regularization constraints. This approach is equivalent to incorporating prior knowledge of sample non-negativity and smoothness into the 3D reconstruction process. Simulations demonstrate that the FGP-AP method improves reconstruction quality compared to the original AP method, particularly in the presence of noise. Experimental results, obtained from mouse kidney cells and label-free blood cells, further affirm the superior 3D imaging efficacy of the FGP-AP method.

Ultrafast Bragg coherent diffraction imaging of epitaxial thin films using deep complex-valued neural networks

Domain wall structures form spontaneously due to epitaxial misfit during thin film growth. Imaging the dynamics of domains and domain walls at ultrafast timescales can provide fundamental clues to features that impact electrical transport in electronic devices. Recently, deep learning based methods showed promising phase retrieval (PR) performance, allowing intensity-only measurements to be transformed into snapshot real space images. While the Fourier imaging model involves complex-valued quantities, most existing deep learning based methods solve the PR problem with real-valued based models, where the connection between amplitude and phase is ignored. To this end, we involve complex numbers operation in the neural network to preserve the amplitude and phase connection. Therefore, we employ the complex-valued neural network for solving the PR problem and evaluate it on Bragg coherent diffraction data streams collected from an epitaxial La2-xSrxCuO4 (LSCO) thin film using an X-ray Free Electron Laser (XFEL). Our proposed complex-valued neural network based approach outperforms the traditional real-valued neural network methods in both supervised and unsupervised learning manner. Phase domains are also observed from the LSCO thin film at an ultrafast timescale using the complex-valued neural network.

Motion-resolved, reference-free holographic imaging via spatiotemporally regularized inversion

Holography is a powerful technique that records the amplitude and phase of an optical field simultaneously, enabling a variety of applications such as label-free biomedical analysis and coherent diffraction imaging. Holographic recording without a reference wave has been long pursued because it obviates the high experimental requirements of conventional interferometric methods. However, due to the ill-posed nature of the underlying phase retrieval problem, reference-free holographic imaging is faced with an inherent tradeoff between imaging fidelity and temporal resolution. Here, we propose a general computational framework, termed spatiotemporally regularized inversion (STRIVER), to achieve motion-resolved, reference-free holographic imaging with high fidelity. Specifically, STRIVER leverages signal priors in the spatiotemporal domain to jointly eliminate phase ambiguities and motion artifacts, and, when combined with diversity measurement schemes, produces a physically reliable, time-resolved holographic video from a series of intensity-only measurements. We experimentally demonstrate STRIVER in near-field ptychography, where dynamic holographic imaging of freely swimming paramecia is performed at a framerate-limited speed of 112 fps. The proposed method can be potentially extended to other measurement schemes, spectral regimes, and computational imaging modalities, pushing the temporal resolution toward higher limits.

Retrieval of Aerosol Optical Depth and FMF over East Asia from Directional Intensity and Polarization Measurements of PARASOL

The advantages of performing aerosol retrieval with multi-angle, multi-spectral photopolarimetric measurements over intensity-only measurements come from this technique’s sensitivity to aerosols’ microphysical properties, such as their particle size, shape, and complex refraction index. In this study, an extended LUT (Look Up Table) algorithm inherited from a previous work based on the assumption of surface reflectance spectral shape invariance is proposed and applied to PARASOL (Polarization and Anisotropy of Reflectances for Atmospheric Science coupled with Observations from a Lidar) measurements to retrieve aerosols’ optical properties including aerosol optical depth (AOD) and aerosol fine-mode fraction (FMF). Case studies conducted over East China for different aerosol scenes are investigated. A comparison between the retrieved AOD regional distribution and the corresponding MODIS (Moderate-resolution Imaging Spectroradiometer) C6 AOD products shows similar spatial distributions in the Jing-Jin-Ji (Beijing–Tianjin–Hebei, China’s mega city cluster) region. The PARASOL AOD retrievals were compared against the AOD measurements of seven AERONET (Aerosol Robotic Network) stations in China to evaluate the performance of the retrieval algorithm. In the fine-particle-dominated regions, lower RMSEs were found at Beijing and Hefei urban stations (0.16 and 0.18, respectively) compared to those at other fine-particle-dominated AERONET stations, which can be attributed to the assumption of surface reflectance spectral shape invariance that has significant advantages in separating the contribution of surface and aerosol scattering in urban areas. For the FMF validation, an RMSE of 0.23, a correlation of 0.57, and a bias of −0.01 were found. These results show that the algorithm performs reasonably in distinguishing the contribution of fine and coarse particles.

Fast reconstruction of programmable interferometers with intensity-only measurements

Programmable linear optical interferometers are promising for classical and quantum applications. Their integrated design makes it possible to create more scalable and stable devices. To use them in practice, one has to reconstruct the whole device model taking the manufacturing errors into account. The inability to address individual interferometer elements complicates the reconstruction problem. A naive approach is to train the model via some complex optimization procedure. A faster optimization-free algorithm has been recently proposed (Bantysh et al 2023 Opt. Express 31 16729–42). However, it requires the full transfer matrix tomography while a more practical setup measures only the fields intensities at the interferometer output. In this paper, we propose the modification of the fast algorithm, which uses additional set of interferometer configurations in order to reconstruct the model in the case of intensity-only measurements. We show that it performs slightly worse than the original fast algorithm but it is more practical and still does not require intensive numerical optimization.

Towards practical single-shot phase retrieval with physics-driven deep neural network.

Phase retrieval (PR), a long-established challenge for recovering a complex-valued signal from its Fourier intensity-only measurements, has attracted considerable attention due to its widespread applications in optical imaging. Recently, deep learning-based approaches were developed and allowed single-shot PR. However, due to the substantial disparity between the input and output domains of the PR problems, the performance of these approaches using vanilla deep neural networks (DNN) still has much room to improve. To increase the reconstruction accuracy, physics-informed approaches were suggested to incorporate the Fourier intensity measurements into an iterative estimation procedure. Since the approach is iterative, they require a lengthy computation process, and the accuracy is still not satisfactory for images with complex structures. Besides, many of these approaches work on simulation data that ignore some common problems such as saturation and quantization errors in practical optical PR systems. In this paper, a novel physics-driven multi-scale DNN structure dubbed PPRNet is proposed. Similar to other deep learning-based PR methods, PPRNet requires only a single Fourier intensity measurement. It is physics-driven that the network is guided to follow the Fourier intensity measurement at different scales to enhance the reconstruction accuracy. PPRNet has a feedforward structure and can be end-to-end trained. Thus, it is much faster and more accurate than the traditional physics-driven PR approaches. Extensive simulations and experiments on an optical platform were conducted. The results demonstrate the superiority and practicality of the proposed PPRNet over the traditional learning-based PR methods.

Three-dimensional refractive index estimation based on deep-inverse non-interferometric optical diffraction tomography (ODT-Deep).

Optical diffraction tomography (ODT) solves an inverse scattering problem to obtain label-free, 3D refractive index (RI) estimation of biological specimens. This work demonstrates 3D RI retrieval methods suitable for partially-coherent ODT systems supported by intensity-only measurements consisting of axial and angular illumination scanning. This framework allows for access to 3D quantitative RI contrast using a simplified non-interferometric technique. We consider a traditional iterative tomographic solver based on a multiple in-plane representation of the optical scattering process and gradient descent optimization adapted for focus-scanning systems, as well as an approach that relies solely on 3D convolutional neural networks (CNNs) to invert the scattering process. The approaches are validated using simulations of the 3D scattering potential for weak phase 3D biological samples.

Two-frame advanced iterative self-tuning algorithm for accurate phase retrieval

Two-shot random phase-shifting interferometry has been actively investigated and improved owing to more demanding requirements in investigating dynamic phenomena or sensing some transient events. Given the background intensity and phase step knowledge, a reconstruction algorithm can be used to recover a complex-valued signal from intensity-only measurements. Most algorithms assume the background intensity and modulation amplitude in a frame of interferogram are constant variables; however, in realistic systems, they vary laterally and axially across the field of view. The model can be used but often appears less precise. In this work, we propose an approach that leverages the iterative method to cope with spatially varying background intensity and modulation amplitude. Our approach, termed the two-frame advanced iterative self-tuning algorithm, uses the spatial mean algorithm paired with an iterative procedure to search the phase step from two captured fringe patterns. It represents a novel approach to reliable and practical two-step phase-shifting interferometry without pre-filtering due to the cancelation of simultaneously obtaining the phase shifts and measured phase through iterative operation in the spatial–temporal domain. Experimental results obtained using the proposed method indicate that it is a simple and robust solution for phase extraction from a two-frame unknown phase shift fringe pattern with spatially varying background intensity and modulation amplitude.

Polarization-sensitive intensity diffraction tomography

Optical anisotropy, which is an intrinsic property of many materials, originates from the structural arrangement of molecular structures, and to date, various polarization-sensitive imaging (PSI) methods have been developed to investigate the nature of anisotropic materials. In particular, the recently developed tomographic PSI technologies enable the investigation of anisotropic materials through volumetric mappings of the anisotropy distribution of these materials. However, these reported methods mostly operate on a single scattering model, and are thus not suitable for three-dimensional (3D) PSI imaging of multiple scattering samples. Here, we present a novel reference-free 3D polarization-sensitive computational imaging technique—polarization-sensitive intensity diffraction tomography (PS-IDT)—that enables the reconstruction of 3D anisotropy distribution of both weakly and multiple scattering specimens from multiple intensity-only measurements. A 3D anisotropic object is illuminated by circularly polarized plane waves at various illumination angles to encode the isotropic and anisotropic structural information into 2D intensity information. These information are then recorded separately through two orthogonal analyzer states, and a 3D Jones matrix is iteratively reconstructed based on the vectorial multi-slice beam propagation model and gradient descent method. We demonstrate the 3D anisotropy imaging capabilities of PS-IDT by presenting 3D anisotropy maps of various samples, including potato starch granules and tardigrade.

Parallel synthetic aperture transport-of-intensity diffraction tomography with annular illumination.

Transport-of-intensity diffraction tomography (TIDT) is a recently developed label-free computational microscopy technique that retrieves high-resolution three-dimensional (3D) refractive index (RI) distribution of biological specimens from 3D intensity-only measurements. However, the non-interferometric synthetic aperture in TIDT is generally achieved sequentially through the acquisition of a large number of through-focus intensity stacks captured at different illumination angles, resulting in a very cumbersome and redundant data acquisition process. To this end, we present a parallel implementation of a synthetic aperture in TIDT (PSA-TIDT) with annular illumination. We found that the matched annular illumination provides a mirror-symmetric 3D optical transfer function, indicating the analyticity in the upper half-plane of the complex phase function, which allows for recovery of the 3D RI from a single intensity stack. We experimentally validated PSA-TIDT by conducting high-resolution tomographic imaging of various unlabeled biological samples, including human breast cancer cell lines (MCF-7), human hepatocyte carcinoma cell lines (HepG2), Henrietta Lacks (HeLa) cells, and red blood cells (RBCs).

Phase Retrieval: From Computational Imaging to Machine Learning: A tutorial

Phase retrieval consists in the recovery of a complex-valued signal from intensity-only measurements. As it pervades a broad variety of applications, many researchers have striven to develop phase-retrieval algorithms. Classical approaches involve techniques as varied as generic gradient-descent routines or specialized spectral methods, to name a few. Yet, the phase-recovery problem remains a challenge to this day. Recently, however, advances in machine learning have revitalized the study of phase retrieval in two ways: significant theoretical advances have emerged from the analogy between phase retrieval and single-layer neural networks; practical breakthroughs have been obtained thanks to deep-learning regularization. In this tutorial, we review phase retrieval under a unifying framework that encompasses classical and machine-learning methods. We focus on three key elements: applications, overview of recent reconstruction algorithms, and the latest theoretical results.

Bond-selective intensity diffraction tomography

Recovering molecular information remains a grand challenge in the widely used holographic and computational imaging technologies. To address this challenge, we developed a computational mid-infrared photothermal microscope, termed Bond-selective Intensity Diffraction Tomography (BS-IDT). Based on a low-cost brightfield microscope with an add-on pulsed light source, BS-IDT recovers both infrared spectra and bond-selective 3D refractive index maps from intensity-only measurements. High-fidelity infrared fingerprint spectra extraction is validated. Volumetric chemical imaging of biological cells is demonstrated at a speed of ~20 s per volume, with a lateral and axial resolution of ~350 nm and ~1.1 µm, respectively. BS-IDT’s application potential is investigated by chemically quantifying lipids stored in cancer cells and volumetric chemical imaging on Caenorhabditis elegans with a large field of view (~100 µm x 100 µm).

NON-INVASIVE PREDICTION OF OOCYTE MATURATION TO METAPHASE II (MII) FROM GERMINAL VESICLE (GV) BY MITOCHONDRIAL FUNCTION MARKERS USING FLUORESCENCE LIFETIME IMAGING MICROSCOPY (FLIM)

Metabolic function changes through meiosis and is critical for the oocyte developmental competence. As NADH and FAD+ are both autofluorescent, monitoring their fluorescence yields valuable information for characterizing cellular metabolism. Fluorescence Lifetime Imaging Microscopy (FLIM) generates timelapse profiling of metabolic state, without the need for additional markers. Superior to intensity-only measurements, FLIM generates 3 additional parameters (fraction bound, long and short lifetime) that are sensitive to metabolic shifts.

Recovery of continuous 3D refractive index maps from discrete intensity-only measurements using neural fields

Intensity diffraction tomography (IDT) refers to a class of optical microscopy techniques for imaging the three-dimensional refractive index (RI) distribution of a sample from a set of two-dimensional intensity-only measurements. The reconstruction of artefact-free RI maps is a fundamental challenge in IDT due to the loss of phase information and the missing-cone problem. Neural fields has recently emerged as a new deep learning approach for learning continuous representations of physical fields. The technique uses a coordinate-based neural network to represent the field by mapping the spatial coordinates to the corresponding physical quantities, in our case the complex-valued refractive index values. We present Deep Continuous Artefact-free RI Field (DeCAF) as a neural-fields-based IDT method that can learn a high-quality continuous representation of a RI volume from its intensity-only and limited-angle measurements. The representation in DeCAF is learned directly from the measurements of the test sample by using the IDT forward model without any ground-truth RI maps. We qualitatively and quantitatively evaluate DeCAF on the simulated and experimental biological samples. Our results show that DeCAF can generate high-contrast and artefact-free RI maps and lead to an up to 2.1-fold reduction in the mean squared error over existing methods. Producing high-quality 3D refractive index maps from 2D intensity-only measurements is a long-standing objective in computational microscopy, with many applications involving the visualization of cellular and subcellular structures. A new method can reconstruct high-contrast and artefact-free images by employing the neural fields technique, which can learn a continuous 3D representation using a neural network that maps spatial coordinates to the refractive index values.

Ring Resonator Gap Determination Design Rule and Parameter Extraction Method for Sub-GHz Resolution Whole C-Band Si3N4 Integrated Spectrometer

A panoramic ultra-high resolution photonic integrated circuit spectrometer is under development by the authors. The architecture comprises a tunable ring resonator (RR) stage and an AWG stage. The resolution defines the bandwidth of the RR, determined by the cross-coupled power and hence the gap between the access and ring waveguides. The AWG channel frequency spacing determines the required free-spectral range (FSR) and hence the perimeter of the ring resonator. The specified <1 GHz resolution combined with an FSR of 50 GHz renders accurate simulation difficult, obstructing the design process. In this report, a simplified design rule to determine the minimum gap between straight access waveguides and a circular ring waveguide is proposed. Realistic assumptions such as the existence of local bisymmetry and adiabatic mode evolution throughout the coupling region permit a simple mode solver to determine the relationship between the cross-coupled power and the minimum gap size. A parameter extraction method is also formulated for add-drop rings equipped with two nominally identical couplers that disentangles the loss and coupling ring parameters from intensity-only transmission measurements. The proposed rule is applied to the design of ring resonators fabricated on a Si3N4 platform. The parameter extraction method is used to analyze the measured characterization data of the ring resonators. The results show good agreement within ~43 nm between the design rule and the gaps size determined by the parameters extracted from the measured data and provide experimental confirmation of the technological viability of the ring resonators required by the spectrometer.

Ptychographic reconstruction with object initialization.

X-ray ptychography is a cutting edge imaging technique providing ultra-high spatial resolutions. In ptychography, phase retrieval, i.e., the recovery of a complex valued signal from intensity-only measurements, is enabled by exploiting a redundancy of information contained in diffraction patterns measured with overlapping illuminations. For samples that are considerably larger than the probe we show that during the iteration the bulk information has to propagate from the sample edges to the center. This constitutes an inherent limitation of reconstruction speed for algorithms that use a flat initialization. Here, we experimentally demonstrate that a considerable improvement of computational speed can be achieved by utilizing a low resolution sample wavefront retrieved from measured diffraction patterns as object initialization. In addition, we show that this approach avoids phase artifacts associated with large phase gradients and may alleviate the requirements on phase structure within the probe. Object initialization is computationally fast, potentially beneficial for bulky sample and compatible with flat samples. Therefore, the presented approach is readily adaptable with established ptychographic reconstruction algorithms implying a wide spread use.

Transport of intensity diffraction tomography with non-interferometric synthetic aperture for three-dimensional label-free microscopy

We present a new label-free three-dimensional (3D) microscopy technique, termed transport of intensity diffraction tomography with non-interferometric synthetic aperture (TIDT-NSA). Without resorting to interferometric detection, TIDT-NSA retrieves the 3D refractive index (RI) distribution of biological specimens from 3D intensity-only measurements at various illumination angles, allowing incoherent-diffraction-limited quantitative 3D phase-contrast imaging. The unique combination of z-scanning the sample with illumination angle diversity in TIDT-NSA provides strong defocus phase contrast and better optical sectioning capabilities suitable for high-resolution tomography of thick biological samples. Based on an off-the-shelf bright-field microscope with a programmable light-emitting-diode (LED) illumination source, TIDT-NSA achieves an imaging resolution of 206 nm laterally and 520 nm axially with a high-NA oil immersion objective. We validate the 3D RI tomographic imaging performance on various unlabeled fixed and live samples, including human breast cancer cell lines MCF-7, human hepatocyte carcinoma cell lines HepG2, mouse macrophage cell lines RAW 264.7, Caenorhabditis elegans (C. elegans), and live Henrietta Lacks (HeLa) cells. These results establish TIDT-NSA as a new non-interferometric approach to optical diffraction tomography and 3D label-free microscopy, permitting quantitative characterization of cell morphology and time-dependent subcellular changes for widespread biological and medical applications.

Multi-slice ptychographic imaging with multistage coarse-to-fine reconstruction.

The ability to image 3D samples with optical sectioning is essential for the study of tomographic morphology in material and biological sciences. However, it is often hampered by limitations of acquisition speed and equipment complexity when performing 3D volumetric imaging. Here, we propose, to the best of our knowledge, a new method for 3D reconstruction from a minimum of four intensity-only measurements. The complementary structured patterns provided by the digital micromirror device (DMD) irradiate the outermost layer of the sample to generate the corresponding diffraction intensities for recording, which enables rapid scanning of loaded patterns for fast acquisition. Our multistage reconstruction algorithm first extracts the overall coarse-grained information, and then iteratively optimizes the information of different layers to obtain fine features, thereby achieving high-resolution 3D tomography. The high-fidelity reconstruction in experiments on two-slice resolution targets, unstained Polyrhachis vicina Roger and freely moving C. elegans proves the robustness of the method. Compared with traditional 3D reconstruction methods such as interferometry-based methods or Fourier ptychographic tomography (FPT), our method increases the reconstruction speed by at least 10 times and is suitable for label-free dynamic imaging in multiple-scattering samples. Such 3D reconstruction suggests potential applications in a wide range of fields.