Digital In-line Holographic Microscopy Research Articles

High-resolution lensless holographic microscopy using a physics-aware deep network.

Lensless digital inline holographic microscopy (LDIHM) is an emerging quantitative phase imaging modality that uses advanced computational methods for phase retrieval from the interference pattern. The existing end-to-end deep networks require a large training dataset with sufficient diversity to achieve high-fidelity hologram reconstruction. To mitigate this data requirement problem, physics-aware deep networks integrate the physics of holography in the loss function to reconstruct complex objects without needing prior training. However, the data fidelity term measures the data consistency with a single low-resolution hologram without any external regularization, which results in a low performance on complex biological data. We aim to mitigate the challenges with trained and physics-aware untrained deep networks separately and combine the benefits of both methods for high-resolution phase recovery from a single low-resolution hologram in LDIHM. We propose a hybrid deep framework (HDPhysNet) using a plug-and-play method that blends the benefits of trained and untrained deep models for phase recovery in LDIHM. The high-resolution phase is generated by a pre-trained high-definition generative adversarial network (HDGAN) from a single low-resolution hologram. The generated phase is then plugged into the loss function of a physics-aware untrained deep network to regulate the complex object reconstruction process. Simulation results show that the SSIM of the proposed method is increased by 0.07 over the trained and 0.04 over the untrained deep networks. The average phase-SNR is elevated by 8.2dB over trained deep models and 9.8dB over untrained deep networks on the experimental biological cells (cervical cells and red blood cells). We showed improved performance of the HDPhysNet against the unknown perturbation in the imaging parameters such as the propagation distance, the wavelength of the illuminating source, and the imaging sample compared with the trained network (HDGAN). LDIHM, combined with HDPhysNet, is a portable and technology-driven microscopy best suited for point-of-care cytology applications.

Single-beam digital holographic reconstruction: a phase-support enhanced complex wavefront on phase-only function for twin-image elimination.

In in-line digital holographic microscopy (DHM), twin-image artifacts pose a significant challenge, and reduction or complete elimination is essential for object reconstruction. To facilitate object reconstruction using a single hologram, significantly reduce inaccuracies, and avoid iterative processing, a digital holographic reconstruction algorithm called phase-support constraint on phase-only function (PCOF) is presented. In-line DHM simulations and tabletop experiments employing the sliding-window approach are used to compute the arithmetic mean and variance of the phase values in the reconstructed image. A support constraint mask, through variance thresholding, effectively enabled twin-image artifacts. Quantitative evaluations using metrics such as mean squared error, peak signal-to-noise ratio, and mean structural similarity index show PCOF's superior capability in eliminating twin-image artifacts and achieving high-fidelity reconstructions compared with conventional methods such as angular spectrum and iterative phase retrieval methods. PCOF stands as a promising approach to in-line digital holographic reconstruction, offering a robust solution to mitigate twin-image artifacts and enhance the fidelity of reconstructed objects.

Glimpsing the 2020 spring bloom in the Strait of Georgia (Canada) with autonomous ferry-based sensors

Spatiotemporal observations are data rich and offer insights into links between ecological patterns and underlying processes. We present fine-scale autonomous observations from repeated ferry transects in the Strait of Georgia (British Columbia, Canada) during the 2020 spring bloom period using a FerryBox system (temperature, salinity, chlorophyll a fluorescence) and a digital inline holographic microscope. Despite instrument cleaning interruptions related to COVID-19 restrictions, 3 periods from late winter (February) to springtime (March and April) contained 14 days of high-quality holograms (>70000) capturing >10500 identifiable micro- to mesoplankton using automatic object detection. The ferry set-up provided automatic data storage through Ocean Networks Canada, which also automatized data flagging and guaranteed remote access. The highest-quality holograms repeatedly covered the central and eastern Strait and showed aspects of bloom succession. Fast-growing diatoms (Skeletonema sp.) emerged first, followed by a diverse assemblage including Chaetoceros spp., Ditylum spp., and Eucampia spp., and by April, larger centric cells prevailed. The combined approach captured local suppression of chlorophyll a fluorescence and diatom concentrations in Fraser River plume waters during the freshet, suggesting fine-scale spatial patterns in seasonal planktonic community composition. This work is among the first of its kind to autonomously generate in situ imaging and physicochemical data with spatiotemporal resolution.

Measuring Vesicle Loading with Holographic Microscopy and Bulk Light Scattering.

We report efforts to quantify the loading of cell-sized lipid vesicles using in-line digital holographic microscopy. This method does not require fluorescent reporters, fluorescent tracers, or radioactive tracers. A single-color LED light source takes the place of conventional illumination to generate holograms rather than bright field images. By modeling the vesicle's scattering in a microscope with a Lorenz-Mie light scattering model and comparing the results to data holograms, we are able to measure the vesicle's refractive index and thus loading. Performing the same comparison for bulk light scattering measurements enables the retrieval of vesicle loading for nanoscale vesicles.

Characterization of Optical, Thermal, and Viscoelastic Properties of Pollenkitt in Angiosperm Pollen Using In-Line Digital Holographic Microscopy.

Pollen grains are remarkable material composites, with various organelles in their fragile interior protected by a strong shell made of sporopollenin. The outermost layer of angiosperm pollen grains contains a lipid-rich substance called pollenkitt, which is a natural bioadhesive that helps preserve structural integrity when the pollen grain is exposed to external environmental stresses. In addition, its viscous nature enables it to adhere to various floral and insect surfaces, facilitating the pollination process. To examine the physicochemical properties of aqueous pollenkitt droplets, we used in-line digital holographic microscopy to capture light scattering from individual pollenkitt particles. Comparison of pollenkitt holograms to those modeled using the Lorenz-Mie theory enables investigations into the minute variations in the refractive index and size resulting from changes in local temperature and pollen aging.

Improving cellular analysis throughput of lens-free holographic microscopy with circular Hough transform and convolutional neural networks

In recent years, lens-free Digital In-Line Holographic Microscopy (DIHM) has become an essential tool for in-vitro experiments. Specifically, it offers many innovative approaches to the characterization of cancer cell assays, which are pioneering studies in the prevention and treatment of cancer, the most common disease of our time. With DIHM, images with very high resolution, high magnification coefficient, and wide FOV can be obtained. However, the success of the DIHM is directly dependent on the successful coordination of the imaging and analysis processes. This study proposes a combined approach of circular Hough transform (CHT) and Convolutional Neural Network (CNN) algorithms to improve the cellular analysis aspect of DIHM. The design, installation, and testing of the DIHM for cellular imaging are presented. Resolution analyses of the microscopy system are presented using a standard USAF 1951 resolution slide. MCF-7 cell lines were imaged with the designed microscope at various concentrations. CHT was used in the wide search space to detect the particles in the cell slides. The proposed method detected all overlapping cells and non-cell particles in the images. CNN was used to classify particles as living, dead, and noncell cells. A GPU-based GUI has been designed to perform imaging and analysis operations. The performance of the proposed method is statistically compared with a reference system using various performance metrics. The accuracy of the model is calculated as %99.92 and also the results obtained with the proposed method show more than R2=0.99 correlation with a reference system. The results of the proposed method have improved the analytical performance of DIHM in in-vitro cell cancer cell line experiments. In addition, the method contributes to the analysis of high-resolution images.

Application of GRIN-lens-based in-line digital holographic microscopy to automatic detection and localization of particles released from a MEMS device.

Recently, we developed a compact and easy-to-implement in-line digital holographic microscope (DHM) using a GRIN rod lens, which provides better resolution (1.3µm) compared with commonly used pinhole-based DHM setups. Here, we employ this microscope to acquire 3D holographically reconstructed images of silica microparticles, within the 10-300µm size range, launched/released from a microelectro-mechanical systems (MEMS) device. To the best of our knowledge, this is the first time that a MEMS device is implemented to store and launch microparticles. The custom-designed MEMS device consists of a 50µm thick flat circular silicon ultrasonic membrane mounted on an off-the-shelf piezoelectric transducer. Moreover, we propose and experimentally demonstrate a new automatic hybrid detection and localization method for particle field holography that benefits from a combination of well-known minimum intensity and variance of gray level distribution focus metrics. This robust method is fast and provides precise d e p t h/z position of particles. The proposed method is applied for particle testing of the MEMS device, reconstructing 3D visualization, and measuring the size and velocity of released particles. The obtained experimental results show that the velocity of released particles, previously dry-loaded onto the MEMS device, is of the order of a few tens of cm/s.

Physics-driven universal twin-image removal network for digital in-line holographic microscopy.

Digital in-line holographic microscopy (DIHM) enables efficient and cost-effective computational quantitative phase imaging with a large field of view, making it valuable for studying cell motility, migration, and bio-microfluidics. However, the quality of DIHM reconstructions is compromised by twin-image noise, posing a significant challenge. Conventional methods for mitigating this noise involve complex hardware setups or time-consuming algorithms with often limited effectiveness. In this work, we propose UTIRnet, a deep learning solution for fast, robust, and universally applicable twin-image suppression, trained exclusively on numerically generated datasets. The availability of open-source UTIRnet codes facilitates its implementation in various DIHM systems without the need for extensive experimental training data. Notably, our network ensures the consistency of reconstruction results with input holograms, imparting a physics-based foundation and enhancing reliability compared to conventional deep learning approaches. Experimental verification was conducted among others on live neural glial cell culture migration sensing, which is crucial for neurodegenerative disease research.

Holographic imaging platform for particle discrimination based on simultaneous mass density and refractive index measurements.

Real-time detection, classification and identification of aerosol particles is crucial in various industries and public health areas. In order to circumvent the limitations of existing particle analysis methods for efficient discrimination, we demonstrate a compact digital in-line holographic microscopy platform with an inertial spectrometer for simultaneous measurement of two independent fingerprint parameters at single species level. In particular, by interrogating the particle location and size captured with the platform, particle mass density can be estimated. Furthermore, by employing Monte Carlo fitting to the Lorenz-Mie theory, the refractive index of each particle can also be extracted from the interference patterns. It is demonstrated that the combination of mass density and optical density characterization unambiguously enhances the discriminatory power of the system, especially when dealing with particles that exhibit similar mass densities but distinctive refractive indices or vice versa. This innovative approach represents a significant advancement in particle characterization and composition identification, with potential applications in various industrial, scientific, and research domains. An iOS-based app interface is then customized for wireless controlling of the CMOS imager, image acquisition, reconstruction, and data analysis. The imaging platform proposed in this work has prominent advantages including compactness, accuracy, efficiency, high throughput, and remote sensing capability, which is especially relevant for applications where on-site/remote metrology and identification of particles is required.

Multispectral in-line hologram reconstruction with aberration compensation applied to Gram-stained bacteria microscopy

In multispectral digital in-line holographic microscopy (DIHM), aberrations of the optical system affect the repeatability of the reconstruction of transmittance, phase and morphology of the objects of interest. Here we address this issue first by model fitting calibration using transparent beads inserted in the sample. This step estimates the aberrations of the optical system as a function of the lateral position in the field of view and at each wavelength. Second, we use a regularized inverse problem approach (IPA) to reconstruct the transmittance and phase of objects of interest. Our method accounts for shift-variant chromatic and geometrical aberrations in the forward model. The multi-wavelength holograms are jointly reconstructed by favouring the colocalization of the object edges. The method is applied to the case of bacteria imaging in Gram-stained blood smears. It shows our methodology evaluates aberrations with good repeatability. This improves the repeatability of the reconstructions and delivers more contrasted spectral signatures in transmittance and phase, which could benefit applications of microscopy, such as the analysis and classification of stained bacteria.

AI-based analysis of 3D position and orientation of red blood cells using a digital in-line holographic microscopy.

The morphological and mechanical characteristics of red blood cells (RBCs) largely vary depending on the occurrence of hematologic disorders. Variations in the rheological properties of RBCs affect the dynamic motions of RBCs, especially their rotational behavior. However, conventional techniques for measuring the orientation of biconcave-shaped RBCs still have some technical limitations, including complicated optical setups, complex post data processing, and low throughput. In this study, we propose a novel image-based technique for measuring 3D position and orientation of normal RBCs using digital in-line holographic microscopy (DIHM) and artificial intelligence (AI). Formaldehyde-fixed RBCs are immobilized in coagulated polydimethylsiloxane (PDMS). Holographic images of RBCs positioned at various out-of-plane angles are acquired by precisely manipulating the PDMS-trapped RBC sample attached to a 4-axis optical stage. With the aid of deep learning algorithms for data augmentation and regression analysis, the out-of-plane angle of RBCs is directly predicted from the captured holographic images. The 3D position and in-plane angle of RBCs are acquired by employing numerical reconstruction and ellipse detection methods. Combining these digital image processing techniques, the 3D positional and orientational information of each RBC recorded in a single holographic image is measured within 23.5 and 3.07s, respectively. The proposed AI-based DIHM technique that can extract the 3D position, orientation, and morphology of individual RBCs would be utilized to analyze the dynamic translational and rotational motions of abnormal RBCs with hematologic disorders in shear flows through further research.

GRIN-lens-based in-line digital holographic microscopy.

In-line digital holographic microscopy (DHM) provides three-dimensional images with large fields of view and depths of field and micrometer-scale resolution, using a compact, cost-effective, and stable setup. Here, we develop the theoretical background and experimentally demonstrate an in-line DHM based on a gradient-index (GRIN) rod lens. In addition, we develop a conventional pinhole-based in-line DHM with different configurations to compare the resolution and image quality of both GRIN-based and pinhole-based systems. We show that in a high-magnification regime, where the sample is positioned near a source that produces spherical waves, our optimized GRIN-based setup provides better resolution (∼1.38µm). Furthermore, we employed this microscope to holographically image dilute polystyrene micro-particles with diameters of 3.0 and 2.0µm. We investigated the effect of light source-detector and sample-detector distances on the resolution, by both theory and experiment. Our theoretical and experimental results are in good agreement.

Accelerated settling velocity of airborne particulate matter on hairy plant leaves

Phytoremediation has emerged as an ecofriendly technique to reduce hazardous particulate matter (PM) in the air. Although previous studies have conducted statistical analyses to reveal PM removal capabilities of various plant species according to their leaf characteristics, the underlying physical mechanism of PM adsorption of plants remains unclear. Conventional methodologies for measuring PM accumulation usually require long-term field tests and provide limited understanding on PM removal effects of individual leaf traits of various plants. In this study, we propose a novel methodology which can compare the electrostatic interactions between PMs and plant leaves according to their trichome structures by using digital in-line holographic microscopy (DIHM). Surface characteristics of Perilla frutescens and Capsicum annuum leaves are measured to examine electrostatic effects according to the morphological features of trichomes. 3D settling motions of PMs near the microstructures of trichomes of the two plant species are compared in detail. To validate the PM removal effect of the hairy microstructures, a polydimethylsiloxane (PDMS) replica model of a P. frutescens leaf is fabricated to demonstrate accelerated settling velocities of PMs near trichome-like microstructures. The size and electric charge distributions of PMs with irregular shapes are analyzed using DIHM. Numerical simulation of the PM deposition near a trichome-like structure is conducted to verify the empirical results. As a result, the settling velocities of PMs on P. frutescens leaves and a PDMS replica sample are 12.11 ± 1.88% and 34.06 ± 4.19% faster than those on C. annuum leaves and a flat PDMS sample, respectively. These findings indicate that the curved microstructures of hairy trichomes of plant leaves increase the ability to capture PMs by enhancing the electric field intensity just near trichomes. Compared with conventional methods, the proposed methodology can quantitatively evaluate the settling velocity of PMs on various plant leaves according to the morphological structure and density of trichomes within a short period of time. The present research findings would be widely utilized in the selection of suitable air-purifying plants for sustainable removal of harmful air pollutants in urban and indoor environments.

Quantitative phase imaging of biological cells using lensless inline holographic microscopy through sparsity-assisted iterative phase retrieval algorithm

The twin image-free phase reconstruction is still a challenge with single-shot inline holographic systems. Existing solutions mostly are based on the inverse problem approaches or alternating projections. However, there exists a trade-off between phase retrieval and twin image elimination. Recent studies have introduced a hybrid method involving both the approaches to mitigate this trade-off. Following these works, we propose a single-shot sparsity-assisted iterative phase retrieval approach that applies a sparsity constraint in the object domain and formulates phase retrieval as a minimization problem. We demonstrate lensless digital inline holographic microscopy for imaging transparent and weakly scattering biological samples over a large field-of-view of ∼29mm2. The proposed method achieves high fidelity phase reconstruction with faster convergence compared to the existing single-shot phase retrieval methods. We further demonstrate the phase quantification of label-free biological samples, such as cervical cells and RBCs, to highlight the potential of our technique in clinical applications.

Programmable Aperture Using a Digital Micromirror Device for In-Line Holographic Microscopy

We present the novel use of a digital micromirror device (DMD) as a digitally adjustable and programmable aperture for lens-free digital in-line holographic microscopy (LFDIHMY). The proposed micro-electro-mechanical system (MEMS) based DMD aperture can be used as a compact and inexpensive replacement for existing mechanical based aperture control methods required for many resolution enhancement methods. Simulation of the DMD aperture in an in-line holography setup is presented using a general DMD model from literature. A hardware prototype was successfully implemented and demonstrated. The resolution of both simulated and experimental holograms recorded with a conventional pinhole aperture and the proposed DMD aperture are compared using a 1951 USAF resolution test target. We show that a resolution of 2.46 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathrm { \mu {\textbf {m}}}$ </tex-math></inline-formula> , close to the sensor pixel limit, can be obtained with each aperture. This demonstrates that a DMD can be used as a programmable replacement for a pinhole aperture.

Accurate unsupervised estimation of aberrations in digital holographic microscopy for improved quantitative reconstruction.

In the context of digital in-line holographic microscopy, we describe an unsupervised methodology to estimate the aberrations of an optical microscopy system from a single hologram. The method is based on the Inverse Problems Approach reconstructions of holograms of spherical objects. The forward model is based on a Lorenz-Mie model distorted by optical aberrations described by Zernike polynomials. This methodology is thus able to characterize most varying aberrations in the field of view in order to take them into account to improve the reconstruction of any sample. We show that this approach increases the repeatability and quantitativity of the reconstructions in both simulations and experimental data. We use the Cramér-Rao lower bounds to study the accuracy of the reconstructions. Finally, we demonstrate the efficiency of this aberration calibration with image reconstructions using a phase retrieval algorithm as well as a regularized inverse problems algorithm.

Novel Dynamic Technique, Nano-DIHM, for Rapid Detection of Oil, Heavy Metals, and Biological Spills in Aquatic Systems.

Numerous anthropogenic and natural particle contaminants exist in diverse aquatic systems, with widely unknown environmental fates. We coupled a flow tube with a digital in-line holographic microscopy (nano-DIHM) technique for aquatic matrices, for in situ real-time analysis of particle size, shape, and phase. Nano-DIHM enables 4D tracking of particles in water and their transformations in three-dimensional space. We demonstrate that nano-DIHM can be automated to detect and track oil spills/oil droplets in dynamic systems. We provide evidence that nano-DIHM can detect the MS2 bacteriophage as a representative biological-viral material and mercury-containing particles alongside other heavy metals as common toxic contaminants. Nano-DIHM shows the capability of observation of combined materials in water, characterizing the interactions of various particles in mixtures, and particles with different coatings in a suspension. The observed sizes of the particles and droplets ranged from ∼1 to 200 μm. We herein demonstrate the ability of nano-DIHM to characterize and distinguish particle-based contaminants in water and their interactions in both stationary and dynamic modes with a 62.5 millisecond time resolution. The fully automated software for dynamic and real-time detection of contaminants will be of global significance. A comparison is also made between nano-DIHM and established techniques such as S/TEM for their different capabilities. Nano-DIHM can provide a range of physicochemical information in stationary and dynamic modes, allowing life cycle analysis of diverse particle contaminants in different aquatic systems, and serve as an effective tool for rapid response for spills and remediation of natural waters.

Combining multi-marker metabarcoding and digital holography to describe eukaryotic plankton across the Newfoundland Shelf

The planktonic diversity throughout the oceans is vital to ecosystem functioning and linked to environmental change. Plankton monitoring tools have advanced considerably with high-throughput in-situ digital cameras and genomic sequencing, opening new challenges for high-frequency observations of community composition, structure, and species discovery. Here, we combine multi-marker metabarcoding based on nuclear 18S (V4) and plastidial 16S (V4–V5) rRNA gene amplicons with a digital in-line holographic microscope to provide a synoptic diversity survey of eukaryotic plankton along the Newfoundland Shelf (Canada) during the winter transition phase of the North Atlantic bloom phenomenon. Metabarcoding revealed a rich eukaryotic diversity unidentifiable in the imaging samples, confirming the presence of ecologically important saprophytic protists which were unclassifiable in matching images, and detecting important groups unobserved or taxonomically unresolved during similar sequencing campaigns in the Northwest Atlantic Ocean. In turn, imaging analysis provided quantitative observations of widely prevalent plankton from every trophic level. Despite contrasting plankton compositions portrayed by each sampling method, both capture broad spatial differences between the northern and southern sectors of the Newfoundland Shelf and suggest complementary estimations of important features in eukaryotic assemblages. Future tasks will involve standardizing digital imaging and metabarcoding for wider use and consistent, comparable ocean observations.

Accurate automatic object 4D tracking in digital in-line holographic microscopy based on computationally rendered dark fields

Building on Gabor seminal principle, digital in-line holographic microscopy provides efficient means for space–time investigations of large volumes of interest. Thus, it has a pivotal impact on particle tracking that is crucial in advancing various branches of science and technology, e.g., microfluidics and biophysical processes examination (cell motility, migration, interplay etc.). Well-established algorithms often rely on heavily regularized inverse problem modelling and encounter limitations in terms of tracking accuracy, hologram signal-to-noise ratio, accessible object volume, particle concentration and computational burden. This work demonstrates the DarkTrack algorithm—a new approach to versatile, fast, precise, and robust 4D holographic tracking based on deterministic computationally rendered high-contrast dark fields. Its unique capabilities are quantitatively corroborated employing a novel numerical engine for simulating Gabor holographic recording of time-variant volumes filled with predefined dynamic particles. Our solution accounts for multiple scattering and thus it is poised to secure an important gap in holographic particle tracking technology and allow for ground-truth-driven benchmarking and quantitative assessment of tracking algorithms. Proof-of-concept experimental evaluation of DarkTrack is presented via analyzing live spermatozoa. Software supporting both novel numerical holographic engine and DarkTrack algorithm is made open access, which opens new possibilities and sets the stage for democratization of robust holographic 4D particle examination.

DWT‐SVD Based Watermarking for High‐Resolution Medical Holographic Images

Watermarking is one of the most common techniques used to protect data’s authenticity, integrity, and security. The obfuscation in the frequency domain used in the watermarking method makes the watermarking stronger than the obfuscation in the spatial domain. It occupies an important place in watermarking works in imperceptibility, capacity, and robustness. Finding the optimal location to hide the watermarking is one of the most challenging tasks in these methods and affects the method’s performance. In this article, sample identification information is processed with the method of watermaking on the hiding environment created by using a chaos‐based random number generator on biomedical data to provide solutions to problems such as visual attack, identity theft, and information confusion. In order to obtain biomedical data, a lensless digital in‐line holographic microscopy (DIHM) setup was designed, and holographic data of human blood and cancer cell lines, which are widely used in the laboratory environment, were obtained. The standard USAF 1951 target was used to evaluate the resolution of our imaging setup. Various QR codes were generated for medical sample identification, and the captured medical data were processed by watermarking it with chaos‐based random number generators. A new method using chaos‐based discrete wavelet transform (DWT) and singular value decomposition (SVD) has been developed and applied to high‐resolution data to eliminate the problem of encrypted data being directly targeted by third‐party attacks. The performance of the proposed new watermarking method has been demonstrated by various robustness and invisibility tests. Experimental results showed that the proposed scheme reached an average PSNR value of 564588 dB and SSIM value of 0.9972 against several geometric and destructive attacks, which means that the proposed method does not affect the image quality and also ensures the security of the watermarking information. The results of the proposed method have shown that it can be used efficiently in various fields.