In-line Holographic Microscopy Research Articles

High-resolution lensless holographic microscopy using a physics-aware deep network.

Lensless digital inline holographic microscopy (LDIHM) is an emerging quantitative phase imaging modality that uses advanced computational methods for phase retrieval from the interference pattern. The existing end-to-end deep networks require a large training dataset with sufficient diversity to achieve high-fidelity hologram reconstruction. To mitigate this data requirement problem, physics-aware deep networks integrate the physics of holography in the loss function to reconstruct complex objects without needing prior training. However, the data fidelity term measures the data consistency with a single low-resolution hologram without any external regularization, which results in a low performance on complex biological data. We aim to mitigate the challenges with trained and physics-aware untrained deep networks separately and combine the benefits of both methods for high-resolution phase recovery from a single low-resolution hologram in LDIHM. We propose a hybrid deep framework (HDPhysNet) using a plug-and-play method that blends the benefits of trained and untrained deep models for phase recovery in LDIHM. The high-resolution phase is generated by a pre-trained high-definition generative adversarial network (HDGAN) from a single low-resolution hologram. The generated phase is then plugged into the loss function of a physics-aware untrained deep network to regulate the complex object reconstruction process. Simulation results show that the SSIM of the proposed method is increased by 0.07 over the trained and 0.04 over the untrained deep networks. The average phase-SNR is elevated by 8.2dB over trained deep models and 9.8dB over untrained deep networks on the experimental biological cells (cervical cells and red blood cells). We showed improved performance of the HDPhysNet against the unknown perturbation in the imaging parameters such as the propagation distance, the wavelength of the illuminating source, and the imaging sample compared with the trained network (HDGAN). LDIHM, combined with HDPhysNet, is a portable and technology-driven microscopy best suited for point-of-care cytology applications.

Measuring colloidomer hydrodynamics with holographic video microscopy.

In-line holographic video microscopy records a wealth of information about the microscopic structure and dynamics of colloidal materials. Powerful analytical techniques are available to retrieve that information when the colloidal particles are well separated. Large assemblies of close-packed particles create holograms that are substantially more challenging to interpret. We demonstrate that Rayleigh-Sommerfeld back propagation is useful for analyzing holograms of colloidomer chains, close-packed linear assemblies of micrometer-scale emulsion droplets. Colloidomers are fully flexible chains and undergo three-dimensional configurational changes under the combined influence of random thermal forces and hydrodynamic forces. We demonstrate the ability of holographic reconstruction to track these changes as colloidomers sediment through water in a horizontal slit pore. Comparing holographically measured configurational trajectories with predictions of hydrodynamic models both validates the analytical technique for this valuable class of self-organizing materials and also provides insights into the influence of geometric confinement on colloidomer hydrodynamics.

Single-beam digital holographic reconstruction: a phase-support enhanced complex wavefront on phase-only function for twin-image elimination.

In in-line digital holographic microscopy (DHM), twin-image artifacts pose a significant challenge, and reduction or complete elimination is essential for object reconstruction. To facilitate object reconstruction using a single hologram, significantly reduce inaccuracies, and avoid iterative processing, a digital holographic reconstruction algorithm called phase-support constraint on phase-only function (PCOF) is presented. In-line DHM simulations and tabletop experiments employing the sliding-window approach are used to compute the arithmetic mean and variance of the phase values in the reconstructed image. A support constraint mask, through variance thresholding, effectively enabled twin-image artifacts. Quantitative evaluations using metrics such as mean squared error, peak signal-to-noise ratio, and mean structural similarity index show PCOF's superior capability in eliminating twin-image artifacts and achieving high-fidelity reconstructions compared with conventional methods such as angular spectrum and iterative phase retrieval methods. PCOF stands as a promising approach to in-line digital holographic reconstruction, offering a robust solution to mitigate twin-image artifacts and enhance the fidelity of reconstructed objects.

Glimpsing the 2020 spring bloom in the Strait of Georgia (Canada) with autonomous ferry-based sensors

Spatiotemporal observations are data rich and offer insights into links between ecological patterns and underlying processes. We present fine-scale autonomous observations from repeated ferry transects in the Strait of Georgia (British Columbia, Canada) during the 2020 spring bloom period using a FerryBox system (temperature, salinity, chlorophyll a fluorescence) and a digital inline holographic microscope. Despite instrument cleaning interruptions related to COVID-19 restrictions, 3 periods from late winter (February) to springtime (March and April) contained 14 days of high-quality holograms (>70000) capturing >10500 identifiable micro- to mesoplankton using automatic object detection. The ferry set-up provided automatic data storage through Ocean Networks Canada, which also automatized data flagging and guaranteed remote access. The highest-quality holograms repeatedly covered the central and eastern Strait and showed aspects of bloom succession. Fast-growing diatoms (Skeletonema sp.) emerged first, followed by a diverse assemblage including Chaetoceros spp., Ditylum spp., and Eucampia spp., and by April, larger centric cells prevailed. The combined approach captured local suppression of chlorophyll a fluorescence and diatom concentrations in Fraser River plume waters during the freshet, suggesting fine-scale spatial patterns in seasonal planktonic community composition. This work is among the first of its kind to autonomously generate in situ imaging and physicochemical data with spatiotemporal resolution.

Measuring Vesicle Loading with Holographic Microscopy and Bulk Light Scattering.

We report efforts to quantify the loading of cell-sized lipid vesicles using in-line digital holographic microscopy. This method does not require fluorescent reporters, fluorescent tracers, or radioactive tracers. A single-color LED light source takes the place of conventional illumination to generate holograms rather than bright field images. By modeling the vesicle's scattering in a microscope with a Lorenz-Mie light scattering model and comparing the results to data holograms, we are able to measure the vesicle's refractive index and thus loading. Performing the same comparison for bulk light scattering measurements enables the retrieval of vesicle loading for nanoscale vesicles.

Characterization of Optical, Thermal, and Viscoelastic Properties of Pollenkitt in Angiosperm Pollen Using In-Line Digital Holographic Microscopy.

Pollen grains are remarkable material composites, with various organelles in their fragile interior protected by a strong shell made of sporopollenin. The outermost layer of angiosperm pollen grains contains a lipid-rich substance called pollenkitt, which is a natural bioadhesive that helps preserve structural integrity when the pollen grain is exposed to external environmental stresses. In addition, its viscous nature enables it to adhere to various floral and insect surfaces, facilitating the pollination process. To examine the physicochemical properties of aqueous pollenkitt droplets, we used in-line digital holographic microscopy to capture light scattering from individual pollenkitt particles. Comparison of pollenkitt holograms to those modeled using the Lorenz-Mie theory enables investigations into the minute variations in the refractive index and size resulting from changes in local temperature and pollen aging.

Nanoplastics in Water: Artificial Intelligence-Assisted 4D Physicochemical Characterization and Rapid In Situ Detection.

For the first time, we present a much-needed technology for the in situ and real-time detection of nanoplastics in aquatic systems. We show an artificial intelligence-assisted nanodigital in-line holographic microscopy (AI-assisted nano-DIHM) that automatically classifies nano- and microplastics simultaneously from nonplastic particles within milliseconds in stationary and dynamic natural waters, without sample preparation. AI-assisted nano-DIHM identifies 2 and 1% of waterborne particles as nano/microplastics in Lake Ontario and the Saint Lawrence River, respectively. Nano-DIHM provides physicochemical properties of single particles or clusters of nano/microplastics, including size, shape, optical phase, perimeter, surface area, roughness, and edge gradient. It distinguishes nano/microplastics from mixtures of organics, inorganics, biological particles, and coated heterogeneous clusters. This technology allows 4D tracking and 3D structural and spatial study of waterborne nano/microplastics. Independent transmission electron microscopy, mass spectrometry, and nanoparticle tracking analysis validates nano-DIHM data. Complementary modeling demonstrates nano- and microplastics have significantly distinct distribution patterns in water, which affect their transport and fate, rendering nano-DIHM a powerful tool for accurate nano/microplastic life-cycle analysis and hotspot remediation.

Improving cellular analysis throughput of lens-free holographic microscopy with circular Hough transform and convolutional neural networks

In recent years, lens-free Digital In-Line Holographic Microscopy (DIHM) has become an essential tool for in-vitro experiments. Specifically, it offers many innovative approaches to the characterization of cancer cell assays, which are pioneering studies in the prevention and treatment of cancer, the most common disease of our time. With DIHM, images with very high resolution, high magnification coefficient, and wide FOV can be obtained. However, the success of the DIHM is directly dependent on the successful coordination of the imaging and analysis processes. This study proposes a combined approach of circular Hough transform (CHT) and Convolutional Neural Network (CNN) algorithms to improve the cellular analysis aspect of DIHM. The design, installation, and testing of the DIHM for cellular imaging are presented. Resolution analyses of the microscopy system are presented using a standard USAF 1951 resolution slide. MCF-7 cell lines were imaged with the designed microscope at various concentrations. CHT was used in the wide search space to detect the particles in the cell slides. The proposed method detected all overlapping cells and non-cell particles in the images. CNN was used to classify particles as living, dead, and noncell cells. A GPU-based GUI has been designed to perform imaging and analysis operations. The performance of the proposed method is statistically compared with a reference system using various performance metrics. The accuracy of the model is calculated as %99.92 and also the results obtained with the proposed method show more than R2=0.99 correlation with a reference system. The results of the proposed method have improved the analytical performance of DIHM in in-vitro cell cancer cell line experiments. In addition, the method contributes to the analysis of high-resolution images.

Real‐Time Jones Matrix Synthesis by Compact Polarization Inline Holographic Microscopy

AbstractThe Jones matrix is a powerful quantitative tool for polarimetric imaging and characterization, where the matrix elements represent the intrinsic optical properties of the anisotropic objects on interaction with the polarized light. Most of the state‐of‐the‐art single‐shot Jones‐matrix imaging techniques rely on interferometric methods, which makes the design complex with the use of large number of polarization optics and produces space‐bandwidth limitations. The simultaneous extraction of Jones matrix elements with a compact system design is still a technical challenge in view of its potential applications in biomedical and real‐time characterization scenarios. In this paper, the imaging features of the in‐line holography are exploited to develop a compact polarization in‐line holography (PIH) system capable of single‐shot extraction of Jones‐matrix elements. The technique achieves simultaneous orthogonal polarization state generation and corresponding polarization multiplexed in‐line hologram detection by utilizing a compact polarization geometry. The imaging compatibility and measurement accuracy of the method is experimentally validated by the real‐time synthesis of Jones‐matrix elements corresponding to custom designed polarization sensitive samples and standard birefringent resolution target. Furthermore, to demonstrate the application of the system in dynamic imaging, the transient polarization changes of a custom‐designed parallel nematic liquid crystal display (LCD) are investigated.

Improved resolution in fiber bundle inline holographic microscopy using multiple illumination sources.

Recent work has shown that high-quality inline holographic microscopy images can be captured through fiber imaging bundles. Speckle patterns arising from modal interference within the bundle cores can be minimized by use of a partially-coherent optical source such as an LED delivered via a multimode fiber. This allows numerical refocusing of holograms from samples at working distances of up to approximately 1 mm from the fiber bundle before the finite coherence begins to degrade the lateral resolution. However, at short working distances the lateral resolution is limited not by coherence, but by sampling effects due to core-to-core spacing in the bundle. In this article we demonstrate that multiple shifted holograms can be combined to improve the resolution by a factor of two. The shifted holograms can be rapidly acquired by sequentially firing LEDs, which are each coupled to their own, mutually offset, illumination fiber. Following a one-time calibration, resolution-enhanced images are created in real-time at an equivalent net frame rate of up to 7.5 Hz. The resolution improvement is demonstrated quantitatively using a resolution target and qualitatively using mounted biological slides. At longer working distances, beyond 0.6 mm, the improvement is reduced as resolution becomes limited by the source spatial and temporal coherence.

Application of GRIN-lens-based in-line digital holographic microscopy to automatic detection and localization of particles released from a MEMS device.

Recently, we developed a compact and easy-to-implement in-line digital holographic microscope (DHM) using a GRIN rod lens, which provides better resolution (1.3µm) compared with commonly used pinhole-based DHM setups. Here, we employ this microscope to acquire 3D holographically reconstructed images of silica microparticles, within the 10-300µm size range, launched/released from a microelectro-mechanical systems (MEMS) device. To the best of our knowledge, this is the first time that a MEMS device is implemented to store and launch microparticles. The custom-designed MEMS device consists of a 50µm thick flat circular silicon ultrasonic membrane mounted on an off-the-shelf piezoelectric transducer. Moreover, we propose and experimentally demonstrate a new automatic hybrid detection and localization method for particle field holography that benefits from a combination of well-known minimum intensity and variance of gray level distribution focus metrics. This robust method is fast and provides precise d e p t h/z position of particles. The proposed method is applied for particle testing of the MEMS device, reconstructing 3D visualization, and measuring the size and velocity of released particles. The obtained experimental results show that the velocity of released particles, previously dry-loaded onto the MEMS device, is of the order of a few tens of cm/s.

Physics-driven universal twin-image removal network for digital in-line holographic microscopy.

Digital in-line holographic microscopy (DIHM) enables efficient and cost-effective computational quantitative phase imaging with a large field of view, making it valuable for studying cell motility, migration, and bio-microfluidics. However, the quality of DIHM reconstructions is compromised by twin-image noise, posing a significant challenge. Conventional methods for mitigating this noise involve complex hardware setups or time-consuming algorithms with often limited effectiveness. In this work, we propose UTIRnet, a deep learning solution for fast, robust, and universally applicable twin-image suppression, trained exclusively on numerically generated datasets. The availability of open-source UTIRnet codes facilitates its implementation in various DIHM systems without the need for extensive experimental training data. Notably, our network ensures the consistency of reconstruction results with input holograms, imparting a physics-based foundation and enhancing reliability compared to conventional deep learning approaches. Experimental verification was conducted among others on live neural glial cell culture migration sensing, which is crucial for neurodegenerative disease research.

Holographic imaging platform for particle discrimination based on simultaneous mass density and refractive index measurements.

Real-time detection, classification and identification of aerosol particles is crucial in various industries and public health areas. In order to circumvent the limitations of existing particle analysis methods for efficient discrimination, we demonstrate a compact digital in-line holographic microscopy platform with an inertial spectrometer for simultaneous measurement of two independent fingerprint parameters at single species level. In particular, by interrogating the particle location and size captured with the platform, particle mass density can be estimated. Furthermore, by employing Monte Carlo fitting to the Lorenz-Mie theory, the refractive index of each particle can also be extracted from the interference patterns. It is demonstrated that the combination of mass density and optical density characterization unambiguously enhances the discriminatory power of the system, especially when dealing with particles that exhibit similar mass densities but distinctive refractive indices or vice versa. This innovative approach represents a significant advancement in particle characterization and composition identification, with potential applications in various industrial, scientific, and research domains. An iOS-based app interface is then customized for wireless controlling of the CMOS imager, image acquisition, reconstruction, and data analysis. The imaging platform proposed in this work has prominent advantages including compactness, accuracy, efficiency, high throughput, and remote sensing capability, which is especially relevant for applications where on-site/remote metrology and identification of particles is required.

Multispectral in-line hologram reconstruction with aberration compensation applied to Gram-stained bacteria microscopy

In multispectral digital in-line holographic microscopy (DIHM), aberrations of the optical system affect the repeatability of the reconstruction of transmittance, phase and morphology of the objects of interest. Here we address this issue first by model fitting calibration using transparent beads inserted in the sample. This step estimates the aberrations of the optical system as a function of the lateral position in the field of view and at each wavelength. Second, we use a regularized inverse problem approach (IPA) to reconstruct the transmittance and phase of objects of interest. Our method accounts for shift-variant chromatic and geometrical aberrations in the forward model. The multi-wavelength holograms are jointly reconstructed by favouring the colocalization of the object edges. The method is applied to the case of bacteria imaging in Gram-stained blood smears. It shows our methodology evaluates aberrations with good repeatability. This improves the repeatability of the reconstructions and delivers more contrasted spectral signatures in transmittance and phase, which could benefit applications of microscopy, such as the analysis and classification of stained bacteria.

Real-time 4D tracking of airborne virus-laden droplets and aerosols

There is currently no real-time airborne virus tracking method, hindering the understanding of rapid virus changes and associated health impacts. Nano-digital in-line holographic microscopy (Nano-DIHM) is a lensless technology that can directly obtain the interference patterns of objects by recording the scattered light information originating from the objects. Here, we provide evidence for real-time physicochemical tracking of virus-laden droplets and aerosols in the air using desktop label-free Nano-DIHM. The virus interference patterns, as single and ensemble particles, were imaged by the Nano-DIHM with 32.5 ms resolution. The next-generation Stingray and Octopus software was used to automate object detection, characterization and classification from the recorded holograms. The detection system was demonstrated to detect active MS2 bacteriophages, inactivated SARS-CoV-2 and RNA fragments, and an MS2 mixture with metallic and organic compounds. This work demonstrates the feasibility of using Nano-DIHM to provide rapid virus detection to improve transmission management in real time.

AI-based analysis of 3D position and orientation of red blood cells using a digital in-line holographic microscopy.

The morphological and mechanical characteristics of red blood cells (RBCs) largely vary depending on the occurrence of hematologic disorders. Variations in the rheological properties of RBCs affect the dynamic motions of RBCs, especially their rotational behavior. However, conventional techniques for measuring the orientation of biconcave-shaped RBCs still have some technical limitations, including complicated optical setups, complex post data processing, and low throughput. In this study, we propose a novel image-based technique for measuring 3D position and orientation of normal RBCs using digital in-line holographic microscopy (DIHM) and artificial intelligence (AI). Formaldehyde-fixed RBCs are immobilized in coagulated polydimethylsiloxane (PDMS). Holographic images of RBCs positioned at various out-of-plane angles are acquired by precisely manipulating the PDMS-trapped RBC sample attached to a 4-axis optical stage. With the aid of deep learning algorithms for data augmentation and regression analysis, the out-of-plane angle of RBCs is directly predicted from the captured holographic images. The 3D position and in-plane angle of RBCs are acquired by employing numerical reconstruction and ellipse detection methods. Combining these digital image processing techniques, the 3D positional and orientational information of each RBC recorded in a single holographic image is measured within 23.5 and 3.07s, respectively. The proposed AI-based DIHM technique that can extract the 3D position, orientation, and morphology of individual RBCs would be utilized to analyze the dynamic translational and rotational motions of abnormal RBCs with hematologic disorders in shear flows through further research.

In-situ and real-time nano/microplastic coatings and dynamics in water using nano-DIHM: A novel capability for the plastic life cycle research

A novel nano-digital inline holographic microscope (nano-DIHM) was used to advance in-situ and real-time nano/microplastic physicochemical research, such as particle coatings and dynamic processes in water. Nano-DIHM data provided evidence of distinct coating patterns on nano/microplastic particles by oleic acid, magnetite, and phytoplankton, representing organic, inorganic, and biological coatings widely present in the natural surroundings. A high-resolution scanning transmission electron microscopy confirmed nano-DIHM data, demonstrating its nano/microplastic research capabilities. The sedimentation of two plastic size categories was examined: (a) ∼10 to 700 µm, and (b) ∼ 1 to 5 mm. Particle size was the primary factor affecting the sedimentation for studied (a) microplastics and (b) pellets. Two types of silicone rubbers exhibited different sedimentation processes. We also demonstrated that inorganic ions in seawater and oleic acid organic coatings altered the sedimentation velocity of studied plastics by 9 – 13% and 5 – 9%, respectively. Semi-empirical probability functions were developed and incorporated into a numerical model (CaMPSim-3D) to simulate the transport of studied microplastics and pellets in the Saint John River estuary. Water dynamics was the driving force of plastic transport, yet the accumulation of plastics was selectively dependant on particle physicochemical properties such as size and density by ∼ 7%. The usage of nano-DIHM for targeted identification of nano/microplastic hotspots and aquatic plastic wastes remediation were discussed.

A sub-wavelength Si LED integrated in a CMOS platform

A nanoscale on-chip light source with high intensity is desired for various applications in integrated photonics systems. However, it is challenging to realize such an emitter using materials and fabrication processes compatible with the standard integrated circuit technology. In this letter, we report an electrically driven Si light-emitting diode with sub-wavelength emission area fabricated in an open-foundry microelectronics complementary metal-oxide-semiconductor platform. The light-emitting diode emission spectrum is centered around 1100 nm and the emission area is smaller than 0.14 μm2 (~varnothing 400 nm). This light-emitting diode has high spatial intensity of >50 mW/cm2 which is comparable with state-of-the-art Si-based emitters with much larger emission areas. Due to sub-wavelength confinement, the emission exhibits a high degree of spatial coherence, which is demonstrated by incorporating the light-emitting diode into a compact lensless in-line holographic microscope. This centimeter-scale, all-silicon microscope utilizes a single emitter to simultaneously illuminate ~9.5 million pixels of a complementary metal-oxide-semiconductor imager.

GRIN-lens-based in-line digital holographic microscopy.

In-line digital holographic microscopy (DHM) provides three-dimensional images with large fields of view and depths of field and micrometer-scale resolution, using a compact, cost-effective, and stable setup. Here, we develop the theoretical background and experimentally demonstrate an in-line DHM based on a gradient-index (GRIN) rod lens. In addition, we develop a conventional pinhole-based in-line DHM with different configurations to compare the resolution and image quality of both GRIN-based and pinhole-based systems. We show that in a high-magnification regime, where the sample is positioned near a source that produces spherical waves, our optimized GRIN-based setup provides better resolution (∼1.38µm). Furthermore, we employed this microscope to holographically image dilute polystyrene micro-particles with diameters of 3.0 and 2.0µm. We investigated the effect of light source-detector and sample-detector distances on the resolution, by both theory and experiment. Our theoretical and experimental results are in good agreement.

Accelerated settling velocity of airborne particulate matter on hairy plant leaves

Phytoremediation has emerged as an ecofriendly technique to reduce hazardous particulate matter (PM) in the air. Although previous studies have conducted statistical analyses to reveal PM removal capabilities of various plant species according to their leaf characteristics, the underlying physical mechanism of PM adsorption of plants remains unclear. Conventional methodologies for measuring PM accumulation usually require long-term field tests and provide limited understanding on PM removal effects of individual leaf traits of various plants. In this study, we propose a novel methodology which can compare the electrostatic interactions between PMs and plant leaves according to their trichome structures by using digital in-line holographic microscopy (DIHM). Surface characteristics of Perilla frutescens and Capsicum annuum leaves are measured to examine electrostatic effects according to the morphological features of trichomes. 3D settling motions of PMs near the microstructures of trichomes of the two plant species are compared in detail. To validate the PM removal effect of the hairy microstructures, a polydimethylsiloxane (PDMS) replica model of a P. frutescens leaf is fabricated to demonstrate accelerated settling velocities of PMs near trichome-like microstructures. The size and electric charge distributions of PMs with irregular shapes are analyzed using DIHM. Numerical simulation of the PM deposition near a trichome-like structure is conducted to verify the empirical results. As a result, the settling velocities of PMs on P. frutescens leaves and a PDMS replica sample are 12.11 ± 1.88% and 34.06 ± 4.19% faster than those on C. annuum leaves and a flat PDMS sample, respectively. These findings indicate that the curved microstructures of hairy trichomes of plant leaves increase the ability to capture PMs by enhancing the electric field intensity just near trichomes. Compared with conventional methods, the proposed methodology can quantitatively evaluate the settling velocity of PMs on various plant leaves according to the morphological structure and density of trichomes within a short period of time. The present research findings would be widely utilized in the selection of suitable air-purifying plants for sustainable removal of harmful air pollutants in urban and indoor environments.