On-chip Microscopy Research Articles

Single-shot lensfree on-chip quantitative phase microscopy with partially coherent LED illumination.

We propose a single-shot lens-free phase retrieval (SSLFPR) method in a lens-free on-chip microscopy (LFOCM) system based on a partially coherent light emitting diode (LED) illumination. The finite bandwidth (∼23.95nm) of LED illumination is divided into a series of quasi-monochromatic components according to the LED spectrum measured by a spectrometer. When the "virtual wavelength scanning" phase retrieval method is combined with the dynamic phase support constraint, the resolution loss caused by the spatiotemporal partial coherence of the light source can be effectively compensated. At the same time, the nonlinearity characteristics of the support constraint help to further improve the imaging resolution, accelerate the convergence of the iteration process, and greatly eliminate the artifacts. Based on the proposed SSLFPR method, we demonstrate that the phase information of samples (including phase resolution target and polystyrene microspheres) illuminated by a LED can be accurately retrieved based on one single diffraction pattern. The SSLFPR method has a half-width resolution of 977nm across a large field-of-view (FOV) of 19.53mm2, which is 1.41 × the resolution of the conventional approach. We also imaged living Henrietta Lacks (HeLa) cells cultured in vitro, further demonstrating the real-time single-shot quantitative phase imaging (QPI) capability of SSLFPR for dynamic samples. Given its simple hardware, high throughput, and single-frame high-resolution QPI capability, SSLFPR is expected to be adopted in a wide range of biological and medical applications.

Particle tracking and identification using on-chip holographic imaging with inertial separation

Rapid and accurate monitoring and identification of micro-sized particles are of utmost importance in various industrial fields. Here, we present an on-chip microscopy holography platform with an inertial impactor for particle metrology and classification. In particular, the particles are first sampled and classified via an inertial impactor according to their inertia. By interrogating the particle location and size using digital in-line holography imaging techniques with calibrated models, we, for the first time, classify particles with different inertia and estimate the particle mass densities. We then develop an IOS-based App interface for wirelessly controlling of the CMOS imager and sample image reconstruction process. The imaging platform proposed in work have prominent advantages including compact in size, robust and efficient in reconstruction algorithm, high throughput characterization capability. Additionally, it can operate remotely via smartphone and a customized App interface, which is especially relevant for industrial application where on-site/remote metrology and identification of particles is required.

Accurate and fast registration algorithm for multi-height lensless in-line on-chip holographic microscopy

Lensless in-line on-chip holographic microscope has cost-effective and compact designs that can offer sub-micron resolution over a large field-of-view. In multi-height lensless on-chip holographic microscopy, the in-line hologram series are recorded at different positions along the optical axis, thus, there inevitably exists several kinds of error sources including shifts, scales, and rotations. Therefore, digitally registering and aligning these in-line holograms is of great concern for iterative phase-retrieved recovery. Here, we proposed an accurate and fast registration method, which consists of three major steps. First, Fourier-transform-based phase correlation method is used to roughly correct the spatial translation through locating the correlation peak. Second, we proposed a morphology-based method to accurately search for the matching feature points to obtain the accurate scaling factors, where samples themselves are used as markers without any labeling. Third, the phase correlation method is used for precisely registration again. Experimental results verified the validity and effectiveness of the proposed registration algorithm. And by the comparative experiments with the existing popular registration algorithms, our proposed registration algorithm can register and align the multi-height in-line holograms with greater precision and speed.

Pixel Super-Resolution Phase Retrieval for Lensless On-Chip Microscopy via Accelerated Wirtinger Flow.

Empowered by pixel super-resolution (PSR) and phase retrieval techniques, lensless on-chip microscopy opens up new possibilities for high-throughput biomedical imaging. However, the current PSR phase retrieval approaches are time consuming in terms of both the measurement and reconstruction procedures. In this work, we present a novel computational framework for PSR phase retrieval to address these concerns. Specifically, a sparsity-promoting regularizer is introduced to enhance the well posedness of the nonconvex problem under limited measurements, and Nesterov’s momentum is used to accelerate the iterations. The resulting algorithm, termed accelerated Wirtinger flow (AWF), achieves at least an order of magnitude faster rate of convergence and allows a twofold reduction in the measurement number while maintaining competitive reconstruction quality. Furthermore, we provide general guidance for step size selection based on theoretical analyses, facilitating simple implementation without the need for complicated parameter tuning. The proposed AWF algorithm is compatible with most of the existing lensless on-chip microscopes and could help achieve label-free rapid whole slide imaging of dynamic biological activities at subpixel resolution.

Lensfree on-chip microscopy based on single-plane phase retrieval.

We propose a novel single-plane phase retrieval method to realize high-quality sample reconstruction for lensfree on-chip microscopy. In our method, complex wavefield reconstruction is modeled as a quadratic minimization problem, where total variation and joint denoising regularization are designed to keep a balance of artifact removal and resolution enhancement. In experiment, we built a 3D-printed field-portable platform to validate the imaging performance of our method, where resolution chart, dynamic target, transparent cell, polystyrene beads, and stained tissue sections are employed for the imaging test. Compared to state-of-the-art methods, our method eliminates image degradation and obtains a higher imaging resolution. Different from multi-wavelength or multi-height phase retrieval methods, our method only utilizes a single-frame intensity data record to accomplish high-fidelity reconstruction of different samples, which contributes a simple, robust, and data-efficient solution to design a resource-limited lensfree on-chip microscope. We believe that it will become a useful tool for telemedicine and point-of-care application.

Atomic-resolution lithography with an on-chip scanning tunneling microscope

In this work, atomic-resolution lithography with a Microelectromechanical-System (MEMS) based Scanning Tunneling Microscope (STM) is demonstrated for the first time. The microscope consists of a commercial UltraHigh-Vacuum (UHV) STM whose regular tip is replaced with a 1-Degree-of-Freedom (1-DOF) MEMS nanopositioner. This results in a hybrid STM system where XY-plane motions are provided by the piezotube of the original system and Z-axis motion by the MEMS with a higher bandwidth. Sharp tips made of Pt or W are added to the MEMS devices with postfabrication techniques. With this hybrid system, STM-based lithography is demonstrated on an H-passivated Si (100)-2×1 sample under UHV condition. Results prove the capability of the hybrid STM system for atomic-scale lithography. This capability, paired with the small footprint of the MEMS device, makes this approach a candidate for building a high-throughput parallel STM lithography platform by incorporating an array of 1-DOF MEMS devices that perform lithography in parallel.

Lensfree auto-focusing imaging using nuclear norm of gradient

Auto-focusing is an essential task for lensfree holographic microscopy, where the optimal focal plane needs to be determined from refocused Z-stack data. In this work, we propose a new auto-focusing criterion, nuclear norm of gradient (NoG), to enhance the accuracy of distance estimation for lensfree imaging. The NoG metric is composed of three parts: shape acquisition, gradient calculation, and gradient fusion. The experiment is conducted in a multi-height on-chip system, where 20 sets of samples including resolution target, stained slide, and label-free cell are employed. The experimental results demonstrate that the accuracy of the NoG metric on auto-focusing is superior to state-of-the-art metrics. Also, the experiment of a multi-layer target proves that the NoG metric has a good optical-sectioning capability. We believe that the NoG metric is expected to be a useful tool for lensfree on-chip microscopes.

A Multi-Wavelength Phase Retrieval With Multi-Strategy for Lensfree On-Chip Holography

Due to convenient operation and robust imaging, multi-wavelength phase retrieval has been widely applied in lensfree on-chip digital holographic microscope (LFOCDHM). Nevertheless, the insufficient diffraction variation and small number of measurements on the LFOCDHM make it difficult to eliminate the twin image by multi-wavelength phase retrieval. We propose a multi-wavelength phase retrieval for LFOCDHM based on energy constraint, global update strategy, and vector extrapolation acceleration. Simulations and experiments on the LFOCDHM show that our proposed method realizes efficient elimination effect and robust reconstruction with three wavelengths for illumination while maintaining fast convergence. More importantly, the proposed method is simple and non-parametric. It is believed that the proposed method could provide a promising solution for LFOCDHM.

On the robustness of machine learning algorithms toward microfluidic distortions for cell classification via on-chip fluorescence microscopy.

Single-cell imaging and sorting are critical technologies in biology and clinical applications. The power of these technologies is increased when combined with microfluidics, fluorescence markers, and machine learning. However, this quest faces several challenges. One of these is the effect of the sample flow velocity on the classification performances. Indeed, cell flow speed affects the quality of image acquisition by increasing motion blur and decreasing the number of acquired frames per sample. We investigate how these visual distortions impact the final classification task in a real-world use-case of cancer cell screening, using a microfluidic platform in combination with light sheet fluorescence microscopy. We demonstrate, by analyzing both simulated and experimental data, that it is possible to achieve high flow speed and high accuracy in single-cell classification. We prove that it is possible to overcome the 3D slice variability of the acquired 3D volumes, by relying on their 2D sum z-projection transformation, to reach an efficient real time classification with an accuracy of 99.4% using a convolutional neural network with transfer learning from simulated data. Beyond this specific use-case, we provide a web platform to generate a synthetic dataset and to investigate the effect of flow speed on cell classification for any biological samples and a large variety of fluorescence microscopes (https://www.creatis.insa-lyon.fr/site7/en/MicroVIP).

Ptychography-based high-throughput lensless on-chip microscopy via incremental proximal algorithms.

Ptychography-based lensless on-chip microscopy enables high-throughput imaging by retrieving the missing phase information from intensity measurements. Numerous reconstruction algorithms for ptychography have been proposed, yet only a few incremental algorithms can be extended to lensless on-chip microscopy because of large-scale datasets but limited computational efficiency. In this paper, we propose the use of accelerated proximal gradient methods for blind ptychographic phase retrieval in lensless on-chip microscopy. Incremental gradient approaches are adopted in the reconstruction routine. Our algorithms divide the phase retrieval problem into sub-problems involving the evaluation of proximal operator, stochastic gradient descent, and Wirtinger derivatives. We benchmark the performances of accelerated proximal gradient, extended ptychographic iterative engine, and alternating direction method of multipliers, and discuss their convergence and accuracy in both noisy and noiseless cases. We also validate our algorithms using experimental datasets, where full field of view measurements are captured to recover the high-resolution complex samples. Among these algorithms, accelerated proximal gradient presents the overall best performance regarding accuracy and convergence rate. The proposed methods may find applications in ptychographic reconstruction, especially for cases where a wide field of view and high resolution are desired at the same time.

Hyperuniform scalar random fields for lensless, multispectral imaging systems.

We propose a novel framework for the systematic design of lensless imaging systems based on the hyperuniform random field solutions of nonlinear reaction-diffusion equations from pattern formation theory. Specifically, we introduce a new class of imaging point-spread functions (PSFs) with enhanced isotropic behavior and controllable sparsity. We investigate PSFs and modulated transfer functions for a number of nonlinear models and demonstrate that two-phase isotropic random fields with hyperuniform disorder are ideally suited to construct imaging PSFs with improved performances compared to PSFs based on Perlin noise. Additionally, we introduce a phase retrieval algorithm based on non-paraxial Rayleigh-Sommerfeld diffraction theory and introduce diffractive phase plates with PSFs designed from hyperuniform random fields, called hyperuniform phase plates (HPPs). Finally, using high-fidelity object reconstruction, we demonstrate improved image quality using engineered HPPs across the visible range. The proposed framework is suitable for high-performance lensless imaging systems for on-chip microscopy and spectroscopy applications.

Super-resolution generative adversarial network (SRGAN) enabled on-chip contact microscopy

We demonstrate a deep learning based contact imaging on a CMOS chip to achieve ∼1 μm spatial resolution over a large field of view of ∼24 mm2. By using regular LED illumination, we acquire the single lower-resolution image of the objects placed approximate to the sensor with unit fringe magnification. For the raw contact-mode lens-free image, the pixel size of the sensor chip limits the spatial resolution. We apply a super-resolution generative adversarial networks, a type of deep learning based single-image super-resolution (SR) algorithm, to circumvent this limitation and effectively recover much higher resolution image of the objects, permitting sub-micron spatial resolution to be achieved across the entire sensor chip active area, which is also equivalent to the imaging field-of-view (24 mm2) due to unit magnification. This SR contact imaging approach eliminates the need of either lens or multi-frame acquisition, being very powerful and cost-effective. We demonstrate the success of this approach by imaging the proliferation dynamics of large-scale cells and the Instantaneous behaviors of freely moving Caenorhabditis elegans directly on the chip.

Atomic precision imaging with an on-chip scanning tunneling microscope integrated into a commercial ultrahigh vacuum STM system

In this article, we replace the Z axis of the piezotube of a conventional Ultrahigh-Vacuum (UHV) Scanning Tunneling Microscope (STM) with a one-degree-of-freedom Microelectromechanical-System (MEMS) nanopositioner. As a result, a hybrid system is realized in which motions in the XY plane are carried out by the piezotube, while the MEMS device performs the Z-axis positioning with a smaller footprint and higher sensitivity. With the proposed system and a feedback loop, STM imaging is conducted on an H-passivated Si (100)-2×1 sample in a UHV condition, demonstrating that this on-chip STM is conducive to atomic precision scanning tunneling microscopy.

Wavelength-scanning lensfree on-chip microscopy for wide-field pixel-super-resolved quantitative phase imaging.

We propose a lensfree on-chip microscopy approach for wide-field quantitative phase imaging (QPI) based on wavelength scanning. Unlike previous methods, we found that a relatively large-range wavelength diversity not only provides information to overcome spatial aliasing of the image sensor but also creates sufficient diffraction variations that can be used to achieve motion-free, pixel-super-resolved phase recovery. Based on an iterative phase retrieval and pixel-super-resolution technique, the proposed wavelength-scanning approach uses only eight undersampled holograms to achieve a half-pitch lateral resolution of 691 nm across a large field-of-view of 29.85mm2, surpassing 2.41 times the theoretical Nyquist-Shannon sampling resolution limit imposed by the pixel size of the sensor (1.67 µm). We confirmed the effectiveness of this technique in QPI and resolution enhancement by measuring the benchmark quantitative phase microscopy target. We also showed that this method can track HeLa cell growth within an incubator, revealing cellular morphologies and subcellular dynamics of a large cell population over an extended period of time.

Autofocusing Algorithm for Pixel-Super-Resolved Lensfree On-Chip Microscopy

In recent years, lensfree on-chip microscopy has developed into a promising and powerful computational optical microscopy technique that allows for wide-field, high-throughput microscopic imaging without using any lenses. However, due to the limited pixel size of the state-of-the-art image sensors, lens-free on-chip microscopy generally suffers from low imaging resolution, which is far from enough to meet the current demand for high-resolution microscopy. Many pixel super-resolution techniques have been developed to solve or at least partially solve this problem by acquiring a series of low-resolution holograms with multiple lateral sub-pixel shifting or axial distances. However, the prerequisite of these pixel super-resolution techniques is that the propagation distance of each low-resolution hologram can be obtained precisely, which faces two major challenges. On the one hand, the captured hologram is inherent pixelated and of low resolution, making it difficult to determine the focal plane by evaluating the image sharpness accurately. On the other hand, the twin-image is superimposed on the backpropagated raw hologram, further exacerbating the difficulties in accurate focal plane determination. In this study, we proposed a high-precision autofocusing algorithm for multi-height pixel-super-resolved lensfree on-chip microscopy. Our approach consists of two major steps: individual preliminary estimation and global precise estimation. First, an improved critical function that combines differential critical function and frequency domain critical function is proposed to obtain the preliminary focus distances of different holograms. Then, the precise focus distances can be determined by further evaluating the global offset of the averaged, low-noise reconstruction from all backpropagated holograms with preliminary focus distances. Simulations and experimental results verified the validity and effectiveness of the proposed algorithm.

High-Precision Lensless Microscope on a Chip Based on In-Line Holographic Imaging.

The lensless on-chip microscope is an emerging technology in the recent decade that can realize the imaging and analysis of biological samples with a wide field-of-view without huge optical devices and any lenses. Because of its small size, low cost, and being easy to hold and operate, it can be used as an alternative tool for large microscopes in resource-poor or remote areas, which is of great significance for the diagnosis, treatment, and prevention of diseases. To improve the low-resolution characteristics of the existing lensless shadow imaging systems and to meet the high-resolution needs of point-of-care testing, here, we propose a high-precision on-chip microscope based on in-line holographic technology. We demonstrated the ability of the iterative phase recovery algorithm to recover sample information and evaluated it with image quality evaluation algorithms with or without reference. The results showed that the resolution of the holographic image after iterative phase recovery is 1.41 times that of traditional shadow imaging. Moreover, we used machine learning tools to identify and count the mixed samples of mouse ascites tumor cells and micro-particles that were iterative phase recovered. The results showed that the on-chip cell counter had high-precision counting characteristics as compared with manual counting of the microscope reference image. Therefore, the proposed high-precision lensless microscope on a chip based on in-line holographic imaging provides one promising solution for future point-of-care testing (POCT).

A combined high-throughput and high-content platform for unified on-chip synthesis, characterization and biological screening

Acceleration and unification of drug discovery is important to reduce the effort and cost of new drug development. Diverse chemical and biological conditions, specialized infrastructure and incompatibility between existing analytical methods with high-throughput, nanoliter scale chemistry make the whole drug discovery process lengthy and expensive. Here, we demonstrate a chemBIOS platform combining on-chip chemical synthesis, characterization and biological screening. We developed a dendrimer-based surface patterning that enables the generation of high-density nanodroplet arrays for both organic and aqueous liquids. Each droplet (among > 50,000 droplets per plate) functions as an individual, spatially separated nanovessel, that can be used for solution-based synthesis or analytical assays. An additional indium-tin oxide coating enables ultra-fast on-chip detection down to the attomole per droplet by matrix-assisted laser desorption/ionization mass spectrometry. The excellent optical properties of the chemBIOS platform allow for on-chip characterization and in-situ reaction monitoring in the ultraviolet, visible (on-chip UV-Vis spectroscopy and optical microscopy) and infrared (on-chip IR spectroscopy) regions. The platform is compatible with various cell-biological screenings, which opens new avenues in the fields of high-throughput synthesis and drug discovery.

Nanometric depth phase imaging using low-cost on-chip lensless inline holographic microscopy

Lensless microscopes are simple, portable, and cost effective compared with the sophisticated microscopes of today that require high-end objectives, lenses, and filters. We demonstrate a lensless on-chip phase microscope based on the holographic principle to image 3D nanometric depth information from transparent and weakly scattering biological samples. We characterize the microscope using standard quantitative phase resolution target (PRT) charts with feature depths from 50 to 350 nm and report a signal-to-noise ratio value of 23 dB. Further, we apply a gradient descent-based constrained optimization approach for phase retrieval to eliminate the twin image and noise from the hologram. The device, operating as an inline holographic microscope, can perform quantitative phase imaging with nanometric depth sensitivity for a field of view of 29.4 mm2 using an LED-based light source butt-coupled to an optical fiber cable.

Directly addressable GaN-based nano-LED arrays: fabrication and electro-optical characterization

The rapid development of display technologies has raised interest in arrays of self-emitting, individually controlled light sources atthe microscale. Gallium nitride (GaN) micro-light-emitting diode (LED) technology meets this demand. However, the current technology is not suitable for the fabrication of arrays of submicron light sources that can be controlled individually. Our approach is based on nanoLED arrays that can directly address each array element and a self-pitch with dimensions below the wavelength of light. The design and fabrication processes are explained in detail and possess two geometries: a 6 × 6 array with 400 nm LEDs and a 2 × 32 line array with 200 nm LEDs. These nanoLEDs are developed as core elements of a novel on-chip super-resolution microscope. GaN technology, based on its physical properties, is an ideal platform for such nanoLEDs.

Super-resolved multispectral lensless microscopy via angle-tilted, wavelength-multiplexed ptychographic modulation.

We report an angle-tilted, wavelength-multiplexed ptychographic modulation approach for multispectral lensless on-chip microscopy. In this approach, we illuminate the specimen with lights at five wavelengths simultaneously. A prism is added at the illumination path for spectral dispersion. Thus, lightwaves at different wavelengths hit the specimen at slightly different incident angles, breaking the ambiguities in mixed-state ptychographic reconstruction. At the detection path, we place a thin diffuser between the specimen and the monochromatic image sensor for encoding the spectral information into 2D intensity measurements. By scanning the sample to different x-y positions, we acquire a sequence of monochromatic images for reconstructing the five complex object profiles at the five wavelengths. An up-sampling procedure is integrated into the recovery process to bypass the resolution limit imposed by the imager pixel size. We demonstrate a half-pitch resolution of 0.55 µm using an image sensor with 1.85 µm pixel size. We also demonstrate quantitative and high-quality multispectral reconstructions of stained tissue sections for digital pathology applications.