Lens-free Holographic Microscopy Research Articles

Deep learning optimization for small object classification in lensfree holographic microscopy

Lensfree holographic microscopy is a compact and cost-effective modality for imaging large fields of view with high resolution. When combined with automated image processing, it can be used for biomolecular sensing where biochemically functionalized micro- and nano-beads are used to label biomolecules of interest. Neural networks for image feature classification provide faster and more robust sensing results than traditional image processing approaches. While neural networks have been widely applied to other types of image classification problems, and even image reconstruction in lensfree holographic microscopy, it is unclear what type of network architecture performs best for the types of small object image classification problems involved in holographic-based sensors. Here, we apply a shallow convolutional neural network to this task, and thoroughly investigate how different layers and hyperparameters affect network performance. Layers include dropout, convolutional, normalization, pooling, and activation. Hyperparameters include dropout fraction, filter number and size, stride, and padding. We ultimately achieve a network accuracy of ∼83%, and find that the choice of activation layer is most important for maximizing accuracy. We hope that these results can be helpful for researchers developing neural networks for similar classification tasks.

Clinical and Biomedical Applications of Lensless Holographic Microscopy

AbstractMany clinical procedures and biomedical research workflows rely on microscopy, including diagnosis of cancer, genetic disorders, autoimmune diseases, infections, and quantification of cell culture. Despite its widespread use, traditional image acquisition and review by trained microscopists is often lengthy and expensive, limited to large hospitals or laboratories, precluding use in point‐of‐care settings. In contrast, lensless or lensfree holographic microscopy (LHM) is inexpensive and widely deployable because it can achieve performance comparable to expensive and bulky objective‐based benchtop microscopes while relying on components that cost only a few hundred dollars or less. Lab‐on‐a‐chip integration is practical and enables LHM to be combined with single‐cell isolation, sample mixing, and in‐incubator imaging. Additionally, many manual tasks in conventional microscopy are instead computational in LHM, including image focusing, stitching, and classification. Furthermore, LHM offers a field of view hundreds of times greater than that of conventional microscopy without sacrificing resolution. Here, the basic LHM principles are summarized, as well as recent advances in artificial intelligence integration and enhanced resolution. How LHM is applied to the above clinical and biomedical applications is discussed in detail. Finally, emerging clinical applications, high‐impact areas for future research, and some current challenges facing widespread adoption are identified.

Improving cellular analysis throughput of lens-free holographic microscopy with circular Hough transform and convolutional neural networks

In recent years, lens-free Digital In-Line Holographic Microscopy (DIHM) has become an essential tool for in-vitro experiments. Specifically, it offers many innovative approaches to the characterization of cancer cell assays, which are pioneering studies in the prevention and treatment of cancer, the most common disease of our time. With DIHM, images with very high resolution, high magnification coefficient, and wide FOV can be obtained. However, the success of the DIHM is directly dependent on the successful coordination of the imaging and analysis processes. This study proposes a combined approach of circular Hough transform (CHT) and Convolutional Neural Network (CNN) algorithms to improve the cellular analysis aspect of DIHM. The design, installation, and testing of the DIHM for cellular imaging are presented. Resolution analyses of the microscopy system are presented using a standard USAF 1951 resolution slide. MCF-7 cell lines were imaged with the designed microscope at various concentrations. CHT was used in the wide search space to detect the particles in the cell slides. The proposed method detected all overlapping cells and non-cell particles in the images. CNN was used to classify particles as living, dead, and noncell cells. A GPU-based GUI has been designed to perform imaging and analysis operations. The performance of the proposed method is statistically compared with a reference system using various performance metrics. The accuracy of the model is calculated as %99.92 and also the results obtained with the proposed method show more than R2=0.99 correlation with a reference system. The results of the proposed method have improved the analytical performance of DIHM in in-vitro cell cancer cell line experiments. In addition, the method contributes to the analysis of high-resolution images.

Lens-free auto-focusing imaging algorithm for the ultra-broadband light source.

Auto-focusing is an essential task for lens-free holographic microscopy, which has developed many methods for high precision or fast refocusing. In this work, we derive the relationship among intensity derivation, the derivative of spectral distribution, as well as the distribution of the object, and propose a new auto-focusing criterion, the Robert critical function with axial difference (RCAD), to enhance the accuracy of distance estimation for lens-free imaging with the ultra-broadband light source. This method consists of three steps: image acquisition and preprocessing, axial-difference calculation, and distance estimation with sharpness analysis. The simulations and experiments demonstrate that the accuracy of this metric on auto-focusing with the ultra-broadband spectrum can effectively assist in determining the off-focus distance. The experiments are conducted in an ultra-broad-spectrum on-chip system, where the samples including the resolution target and the cross-section of the Tilia stem are employed to maximize the applicability of this method. We believe that the RCAD criterion is expected to be a useful auxiliary tool for lens-free on-chip microscopes with ultra-broadband spectrum illumination.

Wavelength-scanning pixel-super-resolved lens-free on-chip quantitative phase microscopy with a color image sensor

We report a wavelength-scanning-based lens-free on-chip microscope using a color CMOS sensor and a matching modified phase retrieval algorithm for pixel super-resolution. Compared to traditional monochrome industrial cameras, color sensors favored by the consumer electronics industry have smaller pixel sizes, higher performance, and lower costs. However, the color filtering array (CFA) introduces inherent modulation to the holograms acquired under quasi-monochromatic illumination, which complicates the data processing in lens-free on-chip microscopy. Without physically removing the CFA positioned on the sensor chip, we demonstrate quantitative phase imaging (QPI) with a lateral half-width resolution of 615 nm over a wide field-of-view of 51.88 mm2 by exploiting the green-channel data from Bayer-masked holograms. The resulting spatial bandwidth product is 137.2 megapixels, over 10 times that of a conventional optical microscope. The rationale for using only green-channel data is that the information from each sampling point is not lost during propagation but rather distributed to all pixels in the image. Therefore, the missing data in other channels can be recovered by exploiting the sufficient differences among the raw images captured at different wavelengths. Compared to the scheme with monochrome sensors, this method requires the acquisition of several more images to guarantee the convergence of the algorithm. Experimental results show that we can achieve high-quality QPI performance, thus demonstrating the applicability of cost-effective color sensors in the field of lens-free holographic microscopy.

Transformer oil quality assessment based on lens-free holographic microscopy

In order to avoid serious problems caused by excessive gas in transformer oil and to realize online monitoring of transformer oil, we propose a method based on lens-free holographic microscopy for transformer oil quality assessment. Using a lens-free device, we can obtain a hologram of the gas bubbles with a large field of view in transformer oil. After the reconstruction, the quantitative phase and shape information of the gas bubbles in the oil can be acquired, thus we can further calculate the parameter of the area, volume, amount, and gas to oil volume ratio of the gas bubbles in transformer oil, which can be used to evaluate the quality of transformer oil effectively. During the experiment, we measured the smallest diameter of the gas bubble in oil was 8 μm. The newly designed method provides a new solution for the assessment of transformer oil quality.

Label-free detection and quantification of Giardia duodenalis cysts using a lens-free microscope

Giardiasis leads to severe diarrhea, and its impacts are especially concerning in low-income sites that lack water and sanitation infrastructure. Monitoring water quality considering the infective form of Giardia is therefore an action that strongly aligns with public health. The standard protocol for cyst quantification in water involves immunofluorescence assays (IFA), which depend on fluorescent probes coupled to antibodies, making it an observer-dependent, and a relatively expensive method. Lens-free holographic microscopy (LHM) offers a possible alternative for imaging, as it relies on portable, lightweight, and low-cost equipment. It is based on phase and amplitude images of holograms recorded with partially coherent lighting. This study, therefore, aimed to test LHM for the detection and counting of Giardia cysts. Purified suspensions of G. duodenalis cysts, with known concentration, were quantified by IFA and LHM. Holograms were acquired by transmission, using three different wavelengths, and were then submitted to automated analysis, using an algorithm developed in Python. Slide preparation and cyst counting by IFA proved to be more laborious compared to LHM. Agglomerations of cysts impaired both methods. LHM quantification showed significant similarity with the reference number of cysts in suspension, which was not obtained by standard immunofluorescence. LHM proved to be an effective technique for detecting and counting Giardia cysts in purified samples and with high concentrations, showing potential for future application in water quality assessment targeting pathogens, but limitations include low sensitivity and a demand for future work considering matrix effect on real environmental samples.

Quantitative detection of cellulose particles in transformer oil based on a lens-free holographic microscope

Quantitative detection of cellulose particles in transformer oil based on a lens-free holographic microscope

Portable, Automated and Deep-Learning-Enabled Microscopy for Smartphone-Tethered Optical Platform Towards Remote Homecare Diagnostics: A Review.

Globally new pandemic diseases induce urgent demands for portable diagnostic systems to prevent and control infectious diseases. Smartphone-based portable diagnostic devices are significantly efficient tools to user-friendly connect personalized health conditions and collect valuable optical information for rapid diagnosis and biomedical research through at-home screening. Deep learning algorithms for portable microscopes also help to enhance diagnostic accuracy by reducing the imaging resolution gap between benchtop and portable microscopes. This review highlighted recent progress and continued efforts in a smartphone-tethered optical platform through portable, automated, and deep-learning-enabled microscopy for personalized diagnostics and remote monitoring. In detail, the optical platforms through smartphone-based microscopes and lens-free holographic microscopy are introduced, and deep learning-based portable microscopic imaging is explained to improve the image resolution and accuracy of diagnostics. The challenges and prospects of portable optical systems with microfluidic channels and a compact microscope to screen COVID-19 in the current pandemic are also discussed. It has been believed that this review offers a novel guide for rapid diagnosis, biomedical imaging, and digital healthcare with low cost and portability.

Lensfree auto-focusing imaging using nuclear norm of gradient

Auto-focusing is an essential task for lensfree holographic microscopy, where the optimal focal plane needs to be determined from refocused Z-stack data. In this work, we propose a new auto-focusing criterion, nuclear norm of gradient (NoG), to enhance the accuracy of distance estimation for lensfree imaging. The NoG metric is composed of three parts: shape acquisition, gradient calculation, and gradient fusion. The experiment is conducted in a multi-height on-chip system, where 20 sets of samples including resolution target, stained slide, and label-free cell are employed. The experimental results demonstrate that the accuracy of the NoG metric on auto-focusing is superior to state-of-the-art metrics. Also, the experiment of a multi-layer target proves that the NoG metric has a good optical-sectioning capability. We believe that the NoG metric is expected to be a useful tool for lensfree on-chip microscopes.

Characterization of exhaled e-cigarette aerosols in a vape shop using a field-portable holographic on-chip microscope

The past decade marked a drastic increase in the usage of electronic cigarettes. The adverse health impact of secondhand exposure due to exhaled e-cig particles has raised significant concerns, demanding further research on the characteristics of these particles. In this work, we report direct volatility measurements on exhaled e-cig aerosols using a field-portable device (termed c-Air) enabled by deep learning and lens-free holographic microscopy; for this analysis, we performed a series of field experiments in a vape shop where customers used/vaped their e-cig products. During four days of experiments, we periodically sampled the indoor air with intervals of ~ 16 min and collected the exhaled particles with c-Air. Time-lapse inline holograms of the collected particles were recorded by c-Air and reconstructed using a convolutional neural network yielding phase-recovered microscopic images of the particles. Volumetric decay of individual particles due to evaporation was used as an indicator of the volatility of each aerosol. Volatility dynamics quantified through c-Air experiments showed that indoor vaping increased the percentage of volatile and semi-volatile particles in air. The reported methodology and findings can guide further studies on volatility characterization of indoor e-cig emissions.

Point-of-care SARS-CoV-2 sensing using lens-free imaging and a deep learning-assisted quantitative agglutination assay.

The persistence of the global COVID-19 pandemic caused by the SARS-CoV-2 virus has continued to emphasize the need for point-of-care (POC) diagnostic tests for viral diagnosis. The most widely used tests, lateral flow assays used in rapid antigen tests, and reverse-transcriptase real-time polymerase chain reaction (RT-PCR), have been instrumental in mitigating the impact of new waves of the pandemic, but fail to provide both sensitive and rapid readout to patients. Here, we present a portable lens-free imaging system coupled with a particle agglutination assay as a novel biosensor for SARS-CoV-2. This sensor images and quantifies individual microbeads undergoing agglutination through a combination of computational imaging and deep learning as a way to detect levels of SARS-CoV-2 in a complex sample. SARS-CoV-2 pseudovirus in solution is incubated with acetyl cholinesterase 2 (ACE2)-functionalized microbeads then loaded into an inexpensive imaging chip. The sample is imaged in a portable in-line lens-free holographic microscope and an image is reconstructed from a pixel superresolved hologram. Images are analyzed by a deep-learning algorithm that distinguishes microbead agglutination from cell debris and viral particle aggregates, and agglutination is quantified based on the network output. We propose an assay procedure using two images which results in the accurate determination of viral concentrations greater than the limit of detection (LOD) of 1.27 × 103 copies per mL, with a tested dynamic range of 3 orders of magnitude, without yet reaching the upper limit. This biosensor can be used for fast SARS-CoV-2 diagnosis in low-resource POC settings and has the potential to mitigate the spread of future waves of the pandemic.

Fractal dimension-based viability analysis of cancer cell lines in lens-free holographic microscopy via machine learning

Fractal dimension-based viability analysis of cancer cell lines in lens-free holographic microscopy via machine learning

High-Speed Lens-Free Holographic Sensing of Protein Molecules Using Quantitative Agglutination Assays.

Accurate, cost-effective, easy-to-use, and point-of-care sensors for protein biomarker levels are important for disease diagnostics. A cost-effective and compact readout approach that has been used for several diagnostic applications is lens-free holographic microscopy, which provides an ultralarge field of view and submicron resolution when it is coupled with pixel super-resolution techniques. Despite its potential as a diagnostic technique, lens-free microscopy has not previously been applied to quantitative protein molecule sensing in solution, which can simplify sensing protocols and ultimately enable measurements of binding kinetics in physiological conditions. Here, we sense interferon-γ (an immune system biomarker) and NeutrAvidin molecules in solution by combining lens-free microscopy with a one-step bead-based agglutination assay, enabled by a custom high-speed light-emitting diode (LED) array and automated image processing routines. We call this a quantitative large-area binding (QLAB) sensor. The high-speed light source provides, for the first time, pixel super-resolved imaging of >104 2 μm beads in solution undergoing Brownian motion, without significant motion blur. The automated image processing routines enable the counting of individual beads and clusters, providing a quantitative sensor readout that depends on both bead and analyte concentrations. Fits to the chemical binding theory are provided. For NeutrAvidin, we find a limit of detection (LOD) of <27 ng/mL (450 pM) and a dynamic range of 2-4 orders of magnitude. For mouse interferon-γ, the LOD is <3 ng/mL (200 pM) and the dynamic range is at least 4 orders of magnitude. The QLAB sensor holds promise for point-of-care applications in low-resource communities and where protocol simplicity is important.

Nonmechanical parfocal and autofocus features based on wave propagation distribution in lensfree holographic microscopy

Performing long-term cell observations is a non-trivial task for conventional optical microscopy, since it is usually not compatible with environments of an incubator and its temperature and humidity requirements. Lensless holographic microscopy, being entirely based on semiconductor chips without lenses and without any moving parts, has proven to be a very interesting alternative to conventional microscopy. Here, we report on the integration of a computational parfocal feature, which operates based on wave propagation distribution analysis, to perform a fast autofocusing process. This unique non-mechanical focusing approach was implemented to keep the imaged object staying in-focus during continuous long-term and real-time recordings. A light-emitting diode (LED) combined with pinhole setup was used to realize a point light source, leading to a resolution down to 2.76 μm. Our approach delivers not only in-focus sharp images of dynamic cells, but also three-dimensional (3D) information on their (x, y, z)-positions. System reliability tests were conducted inside a sealed incubator to monitor cultures of three different biological living cells (i.e., MIN6, neuroblastoma (SH-SY5Y), and Prorocentrum minimum). Altogether, this autofocusing framework enables new opportunities for highly integrated microscopic imaging and dynamic tracking of moving objects in harsh environments with large sample areas.

Deep learning virtual colorful lens-free on-chip microscopy

Currently, it is generally known that lens-free holographic microscopy, which has no imaging lens, can realize a large field-of-view imaging with a low-cost setup. However, in order to obtain colorful images, traditional lens-free holographic microscopy should utilize at least three quasi-chromatic light sources of discrete wavelengths, such as red LED, green LED, and blue LED. Here, we present a virtual colorization by deep learning methods to transfer a gray lens-free microscopy image into a colorful image. Through pairs of images, i.e., grayscale lens-free microscopy images under green LED at 550 nm illumination and colorful bright-field microscopy images, a generative adversarial network (GAN) is trained, and its effectiveness of virtual colorization is proved by applying it to hematoxylin and eosin stained pathological tissue samples imaging. Our computational virtual colorization method might strengthen the monochromatic illumination lens-free microscopy in medical pathology applications and label staining biomedical research.

Pixel super-resolution for lens-free holographic microscopy using deep learning neural networks.

Lens-free holographic microscopy (LFHM) provides a cost-effective tool for large field-of-view imaging in various biomedical applications. However, due to the unit optical magnification, its spatial resolution is limited by the pixel size of the imager. Pixel super-resolution (PSR) technique tackles this problem by using a series of sub-pixel shifted low-resolution (LR) lens-free holograms to form the high-resolution (HR) hologram. Conventional iterative PSR methods require a large number of measurements and a time-consuming reconstruction process, limiting the throughput of LFHM in practice. Here we report a deep learning-based PSR approach to enhance the resolution of LFHM. Compared with the existing PSR methods, our neural network-based approach outputs the HR hologram in an end-to-end fashion and maintains consistency in resolution improvement with a reduced number of LR holograms. Moreover, by exploiting the resolution degradation model in the imaging process, the network can be trained with a data set synthesized from the LR hologram itself without resorting to the HR ground truth. We validated the effectiveness and the robustness of our method by imaging various types of samples using a single network trained on an entirely different data set. This deep learning-based PSR approach can significantly accelerate both the data acquisition and the HR hologram reconstruction processes, therefore providing a practical solution to fast, lens-free, super-resolution imaging.

Localizing and Characterizing Colloidal Particles Scattering Using Lens-free Holographic Microscopy**Supported by the National Key R&D Program of China under Grant No 2016YFA0202102, the Fundamental Research Funds for the Central Universities under Grant No 14380078, and the Scientific Research Foundation of Graduate School of Nanjing University under Grant No 2017CL02.

Lens-free holographic microscopy could achieve both improved resolution and field of view (FOV), which has huge potential applications in biomedicine, fluid mechanics and soft matter physics. Unfortunately, due to the limited sensor pixel size, target objects could not be located to a satisfactory level. Recent studies have shown that electromagnetic scattering can be fitted to digital holograms to obtain the 3D positions of isolated colloidal spheres with nanometer precision and millisecond temporal resolution. Here, we describe a lens-free holographic imaging technique that fits multi-sphere superposition scattering to digital holograms to obtain in situ particle position and model parameters: size and refractive index of colloidal spheres. We show that the proposed method can be utilized to analyze the location and character of colloidal particles under large FOV with high density.

A deep learning-enabled portable imaging flow cytometer for cost-effective, high-throughput, and label-free analysis of natural water samples

We report a deep learning-enabled field-portable and cost-effective imaging flow cytometer that automatically captures phase-contrast color images of the contents of a continuously flowing water sample at a throughput of 100 mL/h. The device is based on partially coherent lens-free holographic microscopy and acquires the diffraction patterns of flowing micro-objects inside a microfluidic channel. These holographic diffraction patterns are reconstructed in real time using a deep learning-based phase-recovery and image-reconstruction method to produce a color image of each micro-object without the use of external labeling. Motion blur is eliminated by simultaneously illuminating the sample with red, green, and blue light-emitting diodes that are pulsed. Operated by a laptop computer, this portable device measures 15.5 cm × 15 cm × 12.5 cm, weighs 1 kg, and compared to standard imaging flow cytometers, it provides extreme reductions of cost, size and weight while also providing a high volumetric throughput over a large object size range. We demonstrated the capabilities of this device by measuring ocean samples at the Los Angeles coastline and obtaining images of its micro- and nanoplankton composition. Furthermore, we measured the concentration of a potentially toxic alga (Pseudo-nitzschia) in six public beaches in Los Angeles and achieved good agreement with measurements conducted by the California Department of Public Health. The cost-effectiveness, compactness, and simplicity of this computational platform might lead to the creation of a network of imaging flow cytometers for large-scale and continuous monitoring of the ocean microbiome, including its plankton composition.

Optimized sensing of sparse and small targets using lens-free holographic microscopy.

Lens-free holographic microscopy offers sub-micron resolution over an ultra-large field-of-view >20 mm2, making it suitable for bio-sensing applications that require the detection of small targets at low concentrations. Various pixel super-resolution techniques have been shown to enhance resolution and boost signal-to-noise ratio (SNR) by combining multiple partially-redundant low-resolution frames. However, it has been unclear which technique performs best for small-target sensing. Here, we quantitatively compare SNR and resolution in experiments using no regularization, cardinal-neighbor regularization, and a novel implementation of sparsity-promoting regularization that uses analytically-calculated gradients from Bayer-pattern image sensors. We find that sparsity-promoting regularization enhances the SNR by ~8 dB compared to the other methods when imaging micron-scale beads with surface coverages up to ~4%.