Pixel Super-resolution Algorithm Research Articles

Lensless super-resolved quantitative phase imaging with cm-level lateral translations of two glass plates.

In this Letter, we propose a new, to the best of our knowledge, lensless on-chip holographic microscopy platform, which can acquire sub-pixel-shifting holograms through centimeter (cm)-level lateral translations. An LED light source is used to illuminate the sample, and two orthogonally tilted step-structure glass plates are inserted into the optical path. By merely displacing the glass plates under cm-level precision, a series of holograms with sub-pixel displacements can be obtained. Combined with our improved pixel super-resolution (PSR) algorithm, high-quality PSR phase imaging can be achieved. Tests on the high-resolution USAF1951 target demonstrate that the system can achieve a half-width resolution of 870 nm by a camera with a pixel size of 1.67 µm. Additionally, imaging experiments were conducted on phase-type sinusoidal gratings, yeasts, red blood cells, and lilium ovary sections, respectively. The results show that the system can achieve large field-of-view, high-resolution phase imaging under low-cost hardware conditions and holds promise for its applications in biology and medicine.

Deep nonlocal low-rank regularization for complex-domain pixel super-resolution.

Pixel super-resolution (PSR) has emerged as a promising technique to break the sampling limit for phase imaging systems. However, due to the inherent nonconvexity of phase retrieval problem and super-resolution process, PSR algorithms are sensitive to noise, leading to reconstruction quality inevitably deteriorating. Following the plug-and-play framework, we introduce the nonlocal low-rank (NLR) regularization for accurate and robust PSR, achieving a state-of-the-art performance. Inspired by the NLR prior, we further develop the complex-domain nonlocal low-rank network (CNLNet) regularization to perform nonlocal similarity matching and low-rank approximation in the deep feature domain rather than the spatial domain of conventional NLR. Through visual and quantitative comparisons, CNLNet-based reconstruction shows an average 1.4 dB PSNR improvement over conventional NLR, outperforming existing algorithms under various scenarios.

Pixel Super-Resolved Lensless on-Chip Sensor with Scattering Multiplexing

Lensless on-chip microscopy has shown great potential for biomedical imaging due to its large-area and high-throughput imaging capabilities. By combining the pixel super-resolution (PSR) technique, it can improve the resolution beyond the limit of the imaging detector. However, existing PSR techniques are restricted by the feature size and crosstalk of modulation components (such as spatial light modulator), which cannot efficiently encode target information. Besides, the reconstruction algorithms suffer from the trade-off between reconstruction quality, imaging resolution, and computational efficiency. In this work, we constructed a novel integrated lensless on-chip sensor via scattering multiplexing and reported a robust PSR algorithm for target reconstruction. We employed a scattering layer to replace conventional modulators and permanently integrated it with the image detector. Benefiting from the high-degree-of-freedom calibration, the scattering layer realized fine wavefront modulation with a small feature size. Besides, the integration engineering avoided repetitious calibration and reduced the complexity of data acquisition. The reported PSR algorithm combined both model-driven and data-driven strategies, with the advantages of high fidelity, strong generalization, and low computational complexity. The remarkable performance allows it to efficiently exploit the high-frequency information from the fine modulation. A series of experiments validate that the reported sensor and PSR algorithm provide a low-cost solution for large-scale microscopic imaging, realizing ∼1.1 μm imaging resolution and ∼7 dB enhancement on the PSNR index compared to existing methods.

High-fidelity pixel-super-resolved complex field reconstruction via adaptive smoothing.

Pixel super-resolution (PSR) techniques have been developed to overcome the sampling limit in lensless digital holographic imaging. However, the inherent non-convexity of the PSR phase retrieval problem can potentially degrade reconstruction quality by causing the iterations to tend toward a false local minimum. Furthermore, the ill posedness of the up-sampling procedure renders PSR algorithms highly susceptible to noise. In this Letter, we propose a heuristic PSR algorithm with adaptive smoothing (AS-PSR) to achieve high-fidelity reconstruction. By automatically adjusting the intensity constraints on the estimated field, the algorithm can effectively locate the optimal solution and converge with high reconstruction quality, pushing the resolution toward the diffraction limit. The proposed method is verified experimentally within a coherent modulation phase retrieval framework, achieving a twofold improvement in resolution. The AS-PSR algorithm can be further applied to other phase retrieval methods based on alternating projection.

Pixel super-resolved lens-free on-chip microscopy based on dual laterally shifting modulation.

Achieving high spatial resolution over a wide field of view (FOV) is the goal of many imaging systems. In a traditional lens-based microscope, designing a complex objective with high numerical aperture (NA) to achieve this goal is a tough and challenging task. The lens-free wide-field imaging method based on phase retrieval provides a new way to bypass the trade-off between the spatial resolution and FOV of conventional microscopy. However, the typical lens-free microscopy usually requires mechanical devices with high precision and repeatability. In this paper, we report a robust and cost-effective pixel super-resolved lens-free imaging method based on dual laterally shifting modulation. A thin diffuser is inserted between the object and the image sensor to be used as the modulator. The diffuser and the object are transversely scanned at the same time to add diversities for phase retrieval and pixel super-resolution, respectively. In this way, the positional shifts of the diffuser and the object can be directly recovered with the registration algorithm, thus addressing the low stability and inaccuracy issues of translation stages. We also propose a pixel super-resolution phase-retrieval algorithm to recover the object and the unknown diffuser. We first use numerical simulations to evaluate the proposed scheme. Then, we validate this approach by imaging a resolution target and a pollen sample, thus achieving an FOV of ${\sim}{30}\;{\text{mm}^2}$∼30mm2 and a half-pitch resolution of 0.78 µm, which surpasses 2.14 times the theoretical Nyquist-Shannon sampling resolution limit. Finally, the 3D refocusing ability is also verified by imaging a thick mosquito sample.

Noise-robust pixel-super-resolved multi-image phase retrieval with coherent illumination

Multi-image phase retrieval (MPR) algorithms play a significant role in the success of coherent diffractive imaging and in-line lensfree digital holographic imaging. However, they still suffer from the pixelation problem. By using them in combination with pixel super-resolution algorithms, the problem could be partially alleviated. But the existing pixel-super-resolved MPR method requires delicate parameter tuning and additional de-noising techniques. In this paper, we propose a new noise-robust pixel-super-resolved MPR scheme. It obviates the need for the mechanical movement of traditional pixel super-resolution methods and needs no extra measurements so that it can be directly applied in the existing system or dataset. Compared with the existing reconstruction method, it requires no sophisticated parameter selection and exhibits higher noise robustness, especially under laser illumination. Also, a new and effective focus measure is put forward so as to locate the object plane with higher precision and robustness. As a whole, our scheme provides a new label-free way of complex field imaging with a large field of view and high spatial resolution.

Compact lensless subpixel resolution large field of view microscope.

We report on a method to increase the spatial resolution in a compact lensless microscope. A compact side illumination is fabricated to illuminate the sample with a collimated beam by diffraction from a volume phase grating. The wavelength of a semi-conductor laser source (vertical-cavity surface-emitting laser) is tuned with the injection current to alter the illumination direction by wavelength selective diffraction from the volume phase grating. The angle tuning is such that several subpixel shifted digital inline holograms are obtained. The stack of holograms is then processed in a pixel super-resolution reconstruction algorithm. The amplitude of the sample is reconstructed with subpixel resolution over a large field of view (FOV). The technique is demonstrated on a 1951 USAF test target. A resolution of ∼2.76 μm, over a FOV of ∼28 mm2, is demonstrated for a device of <2 cm height. The original pixel size was 5.2 μm demonstrating the subpixel resolution.

3D imaging of optically cleared tissue using a simplified CLARITY method and on-chip microscopy.

High-throughput sectioning and optical imaging of tissue samples using traditional immunohistochemical techniques can be costly and inaccessible in resource-limited areas. We demonstrate three-dimensional (3D) imaging and phenotyping in optically transparent tissue using lens-free holographic on-chip microscopy as a low-cost, simple, and high-throughput alternative to conventional approaches. The tissue sample is passively cleared using a simplified CLARITY method and stained using 3,3'-diaminobenzidine to target cells of interest, enabling bright-field optical imaging and 3D sectioning of thick samples. The lens-free computational microscope uses pixel super-resolution and multi-height phase recovery algorithms to digitally refocus throughout the cleared tissue and obtain a 3D stack of complex-valued images of the sample, containing both phase and amplitude information. We optimized the tissue-clearing and imaging system by finding the optimal illumination wavelength, tissue thickness, sample preparation parameters, and the number of heights of the lens-free image acquisition and implemented a sparsity-based denoising algorithm to maximize the imaging volume and minimize the amount of the acquired data while also preserving the contrast-to-noise ratio of the reconstructed images. As a proof of concept, we achieved 3D imaging of neurons in a 200-μm-thick cleared mouse brain tissue over a wide field of view of 20.5 mm2. The lens-free microscope also achieved more than an order-of-magnitude reduction in raw data compared to a conventional scanning optical microscope imaging the same sample volume. Being low cost, simple, high-throughput, and data-efficient, we believe that this CLARITY-enabled computational tissue imaging technique could find numerous applications in biomedical diagnosis and research in low-resource settings.

Pixel super-resolution in digital holography by regularized reconstruction

In-line digital holography (DH) and lensless microscopy are 3D imaging techniques used to reconstruct the volume of micro-objects in many fields. However, their performances are limited by the pixel size of the sensor. Recently, various pixel super-resolution algorithms for digital holography have been proposed. A hologram with improved resolution was produced from a stack of laterally shifted holograms, resulting in better resolved reconstruction than a single low-resolution hologram. Algorithms for super-resolved reconstructions based on inverse problems approaches have already been shown to improve the 3D reconstruction of opaque spheres. Maximum a posteriori (MAP) approaches have also been shown capable of reconstructing the object field more accurately and more efficiently and to extend the usual field-of-view. Here we propose an inverse problem formulation for DH pixel super-resolution and an algorithm that alternates registration and reconstruction steps. The method is described in detail and used to reconstruct synthetic and experimental holograms of sparse 2D objects. We show that our approach improves both the shift estimation and reconstruction quality. Moreover, the reconstructed field-of-view can be expanded by up to a factor 3, thus making it possible to multiply the analyzed area 9 fold.

Computational out-of-focus imaging increases the space–bandwidth product in lens-based coherent microscopy

The space–bandwidth product (SBP) of modern objective lenses is often significantly larger than the pixel count of opto-electronic image sensor chips, and, therefore, much of the information transmitted by the optical system cannot be adequately sampled or digitized. To resolve this mismatch, microscopes are in general designed to maintain the resolution of the optical system while significantly wasting the field of view (FOV) and SBP of the objective lens. We introduce a wide-field and high-resolution coherent imaging method that uses a stack of out-of-focus images to provide much better utilization of the SBP of an objective lens. We demonstrate our approach on a benchtop microscope by using a demagnification camera adapter to match the active area of the image sensor chip to the FOV of an objective lens. We show that the resulting spatial undersampling caused by capturing a large FOV can be mitigated through an iterative pixel super-resolution algorithm that uses e.g., ∼three to five slightly out-of-focus images, yielding an ∼8-fold increase in the SBP of the microscope. Furthermore, the same pixel super-resolution algorithm also achieves phase retrieval, revealing the optical phase information of the specimen. We compared our method against traditional off-axis and phase-shifting digital holographic microscopy modalities and demonstrated at least 3-fold reduction in the number of images required to achieve the same SBP. This technique could be used to maximize the throughput and SBP of lens-based coherent imaging and holography systems and inspire new microscopy designs that benefit from the inherent autofocusing steps of a scanning microscope to increase its SBP.

Propagation phasor approach for holographic image reconstruction.

To achieve high-resolution and wide field-of-view, digital holographic imaging techniques need to tackle two major challenges: phase recovery and spatial undersampling. Previously, these challenges were separately addressed using phase retrieval and pixel super-resolution algorithms, which utilize the diversity of different imaging parameters. Although existing holographic imaging methods can achieve large space-bandwidth-products by performing pixel super-resolution and phase retrieval sequentially, they require large amounts of data, which might be a limitation in high-speed or cost-effective imaging applications. Here we report a propagation phasor approach, which for the first time combines phase retrieval and pixel super-resolution into a unified mathematical framework and enables the synthesis of new holographic image reconstruction methods with significantly improved data efficiency. In this approach, twin image and spatial aliasing signals, along with other digital artifacts, are interpreted as noise terms that are modulated by phasors that analytically depend on the lateral displacement between hologram and sensor planes, sample-to-sensor distance, wavelength, and the illumination angle. Compared to previous holographic reconstruction techniques, this new framework results in five- to seven-fold reduced number of raw measurements, while still achieving a competitive resolution and space-bandwidth-product. We also demonstrated the success of this approach by imaging biological specimens including Papanicolaou and blood smears.

Automation Highlights from the Literature

Automation Highlights from the Literature

Holographic pixel super-resolution in portable lensless on-chip microscopy using a fiber-optic array

We report a portable lensless on-chip microscope that can achieve <1 µm resolution over a wide field-of-view of ∼ 24 mm(2) without the use of any mechanical scanning. This compact on-chip microscope weighs ∼ 95 g and is based on partially coherent digital in-line holography. Multiple fiber-optic waveguides are butt-coupled to light emitting diodes, which are controlled by a low-cost micro-controller to sequentially illuminate the sample. The resulting lensfree holograms are then captured by a digital sensor-array and are rapidly processed using a pixel super-resolution algorithm to generate much higher resolution holographic images (both phase and amplitude) of the objects. This wide-field and high-resolution on-chip microscope, being compact and light-weight, would be important for global health problems such as diagnosis of infectious diseases in remote locations. Toward this end, we validate the performance of this field-portable microscope by imaging human malaria parasites (Plasmodium falciparum) in thin blood smears. Our results constitute the first-time that a lensfree on-chip microscope has successfully imaged malaria parasites.