Fourier Ptychographic Microscopy Research Articles

Gaussian coherent transfer function of partially coherent Fourier ptychographic microscopy

Gaussian coherent transfer function of partially coherent Fourier ptychographic microscopy

Fourier Ptychographic Coherent Anti-Stokes Raman Scattering Microscopy with Point-Scanning for Super-Resolution Imaging.

Fourier ptychography (FP) is a high resolution wide-field imaging method based on the extended aperture in the Fourier space, which is synthesized from raw images with varying illumination angles. If FP is extended to coherent nonlinear optical imaging, the resolution could be further improved due to the increase of the cutoff frequency of the synthesized coherent optical transfer function (C-OTF) with respect to the order of nonlinear optical processes. However, there is a fundamental conflict between wide-field FP and nonlinear optical imaging, whereby the nonlinear optical imaging typically requires a focused excitation laser beam with high power density. To tackle the problem, in this work, a unique point-scanning FP (PS-FP) method is presented for super-resolution nonlinear optical imaging, in which the nonlinear optical signal is obtained by using focused laser beam, while the conventional FP algorithm can still be used to retrieve the super-resolution image. PS-FP coherent anti-Stokes Raman scattering (PS-FP-CARS) imaging on a variety of samples, where a 1.8-fold expansion of the OTF is achieved experimentally for enhancing vibrational imaging. Further theoretical calculation shows that the C-OTF of PS-FP higher-order CARS (PS-FP-HO-CARS) can be expanded up to ≈4.9-fold, thereby improving the spatial resolution by ≈3-fold in comparison with conventional point-scanning CARS with under tightly focused beams. The generality of PS-FP method developed in this work can be adapted to other coherent nonlinear optical imaging modalities for super-resolution imaging in tissue and cells.

Leukocyte detection based on convolutional neural network fusion with color Fourier ptychographic microscopy

Abstract In the field of optical microscopic imaging, color Fourier Ptychographic Microscopy (FPM) technology has attracted much attention due to its advantages of large field of view, high resolution, and quantitative phase imaging. In this paper, a color FPM fusion algorithm based on deep learning is proposed in combination with Convolutional Neural Networks (CNN) and applied to leukocyte detection. Firstly, this paper introduces a fusion model of a convolutional neural network based on the traditional color FPM imaging method and fuses low-resolution color images and high-resolution grayscale images through a multilayer convolutional network. This method improves the quality of reconstructed images while reducing the reconstruction time. Secondly, this paper constructs a leukocyte detection dataset by using an improved color FPM reconstruction algorithm and builds a leukocyte detection system based on the YOLOv7 architecture. This paper shows that combining convolutional neural networks with color FPM technology can provide higher-quality reconstructed images in medical imaging and cell detection, which provides strong technical support for digital pathology and medical diagnosis.

Diagonal illumination scheme for Fourier ptychographic microscopy: resolution doubling and aliasing minimization

Diagonal illumination scheme for Fourier ptychographic microscopy: resolution doubling and aliasing minimization

Fourier ptychographic microscopy with multi-height illumination based on energy threshold pre-search

Fourier ptychographic microscopy (FPM) technology combines the concepts of synthetic aperture imaging, ptychography, and phase retrieval to address the contradiction between the large field of view and high resolution in traditional microscopy and can achieve high-resolution amplitude and phase images with a large field of view. However, for most samples, the primary information is concentrated in the low-frequency region, and traditional single-height FPM may suffer from insufficient sampling, leading to low reconstruction accuracy. In addition, the reconstruction process typically requires a large number of low-resolution images, which also significantly reduces the reconstruction efficiency. To overcome these issues, this paper proposes a form of FPM with multi-height illumination based on an energy threshold pre-search. This method simply involves moving the LED array to three planes for multi-height sample illumination on the traditional FPM hardware, thus improving the sampling conditions and enhancing the reconstruction accuracy. The low-resolution images acquired in this way are then screened using an energy threshold method to select images with higher energy, and a phase retrieval method is employed to reconstruct high-resolution complex amplitude images. The results of simulations and experiments demonstrate that compared to traditional methods, our approach not only improves the reconstruction accuracy but also reduces the number of low-resolution images by at least approximately 60%, thereby significantly enhancing the reconstruction efficiency.

Optimization of the FPM iterative process based on bright-field spectral overlap rate analysis.

Fourier ptychographic microscopy (FPM) reconstructs high-resolution images through multiple iterations on a large number of sub-images at different angles, a process that is time-consuming. For a long time, various methods for optimizing the efficiency of FPM based on the acquisition process and algorithms have been proposed. However, there has been no specific analysis of the impact that the sub-images involved in the reconstruction have on the final result. In this Letter, we conduct the first, to our knowledge, analysis of the impact of a single sub-image on the reconstruction result of a high-resolution image in different numbers of iterations and obtain a curve depicting the change in image quality after the sub-images are involved in the reconstruction in different cycles. By analyzing this curve, the sub-images that exert a negative impact on the resulting image are exported along with their corresponding LED positions. On this basis, we propose the concept of bright-field spectral overlap ratio to distinguish whether the sub-images have a positive impact on the reconstruction results under different acquisition conditions and remove the sub-images that have a negative impact on the results during the iterative process. Both simulation and real experimental results demonstrate that our algorithm can reduce the reconstruction time while maintaining image quality. Moreover, it can be combined with other methods to improve reconstruction efficiency.

Development and Assessment of Multiple Illumination Color Fourier Ptychographic Microscopy for High Throughput Sample Digitization.

In this study, we proposed a multiplexed color illumination strategy to improve the data acquisition efficiency of Fourier ptychography microscopy (FPM). Instead of sequentially lighting up one single channel LED, our method turns on multiple white light LEDs for each image acquisition via a color camera. Thus, each raw image contains multiplexed spectral information. An FPM prototype was developed, which was equipped with a 4×/0.13 NA objective lens to achieve a spatial resolution equivalent to that of a 20×/0.4 NA objective lens. Both two- and four-LED illumination patterns were designed and applied during the experiments. A USAF 1951 resolution target was first imaged under these illumination conditions, based on which MTF curves were generated to assess the corresponding imaging performance. Next, H&E tissue samples and analyzable metaphase chromosome cells were used to evaluate the clinical utility of our strategy. The results show that the single and multiplexed (two- or four-LED) illumination results achieved comparable imaging performance on all the three channels of the MTF curves. Meanwhile, the reconstructed tissue or cell images successfully retain the definition of cell nuclei and cytoplasm and can better preserve the cell edges as compared to the results from the conventional microscopes. This study initially validates the feasibility of multiplexed color illumination for the future development of high-throughput FPM scanning systems.

Simultaneous Multifocal Plane Fourier Ptychographic Microscopy Utilizing a Standard RGB Camera.

Fourier ptychographic microscopy (FPM) is a computational imaging technology that can acquire high-resolution large-area images for applications ranging from biology to microelectronics. In this study, we utilize multifocal plane imaging to enhance the existing FPM technology. Using an RGB light emitting diode (LED) array to illuminate the sample, raw images are captured using a color camera. Then, exploiting the basic optical principle of wavelength-dependent focal length variation, three focal plane images are extracted from the raw image through simple R, G, and B channel separation. Herein, a single aspherical lens with a numerical aperture (NA) of 0.15 was used as the objective lens, and the illumination NA used for FPM image reconstruction was 0.08. Therefore, simultaneous multifocal plane FPM with a synthetic NA of 0.23 was achieved. The multifocal imaging performance of the enhanced FPM system was then evaluated by inspecting a transparent organic light-emitting diode (OLED) sample. The FPM system was able to simultaneously inspect the individual OLED pixels as well as the surface of the encapsulating glass substrate by separating R, G, and B channel images from the raw image, which was taken in one shot.

Can deep neural networks work with amplitude and phase input of defocused images?

Deep neural network (DNN) models, particularly convolutional neural networks (CNNs), have demonstrated remarkable performance in biomedical image classification due to their ability to automatically learn features from large datasets. One common challenge in the preparation of large, microscopic datasets for DNN tasks is sample defocusing, potentially impairing the model performance. To handle defocusing, computational imaging, or specifically quantitative phase imaging (QPI), performs digital refocusing by using both the phase and the amplitude of the complex optical field. This leads us to investigate whether feeding the complex field into DNN would potentially address the defocusing problem as in-focus information is implicitly encoded in the complex field. In this paper, we assess the feasibility of employing neural networks to directly process full amplitude and phase data from a defocus plane without digital refocusing. Our specific focus lies in understanding the tolerance for defocus in image classification neural networks when amplitude and phase are taken as inputs. To accomplish this, we used Fourier ptychography microscopy (FPM) to acquire in-focus phase and amplitude images of two distinct object types – normal red blood cells and echinocytes. We then digitally propagate the complex field to generate progressively defocused images of the samples to serve as training and testing datasets for image classification neural networks. While the digitally defocused images contain sufficient information to recover the original in-focus images, we observed that current standard implementations of deep learning models are unable to effectively utilize the defocused field to distinguish between the two cell types. We conclude that the physical-model-based digital refocusing capability of QPI remains indispensable for overcoming defocusing issues in current standard DNN models.

Fourier Ptychographic Microscopy Reconstruction Method Based on Residual Local Mixture Network.

Fourier Ptychographic Microscopy (FPM) is a microscopy imaging technique based on optical principles. It employs Fourier optics to separate and combine different optical information from a sample. However, noise introduced during the imaging process often results in poor resolution of the reconstructed image. This article has designed an approach based on a residual local mixture network to improve the quality of Fourier ptychographic reconstruction images. By incorporating channel attention and spatial attention into the FPM reconstruction process, the network enhances the efficiency of the network reconstruction and reduces the reconstruction time. Additionally, the introduction of the Gaussian diffusion model further reduces coherent artifacts and improves image reconstruction quality. Comparative experimental results indicate that this network achieves better reconstruction quality, and outperforming existing methods in both subjective observation and objective quantitative evaluation.

Fast color fourier Ptychographic microscopic imaging technology with fusion color correction

In biomedicine, rapid and accurate acquisition of color pathology slides is of paramount importance for diagnostic analyses. Conventional microscopic full-color imaging cannot provide the physician with enough effective information at one time due to the "law of proportionality of objective lenses." Fourier ptychographic microscopy (FPM) has emerged as a solution for high-throughput digital pathology. It can simultaneously achieve a large field of view and high-resolution (HR) imaging. However, achieving color FPM typically requires capturing images separately for each of the red, green, and blue channels for reconstruction, resulting in a significant increase in the time cost. Existing improvement strategies require long coloring times, and have difficulty ensuring standardization between the reconstructed and original image colors owing to the influence of optical system stability and algorithm errors. In this study, a fast-color FPM imaging technology with fusion color correction (CCFPM) is proposed. It achieves high-precision color FPM imaging by separately capturing single low-resolution images for the red and blue channels, which are used to color the reconstructed green-channel HR grayscale image. The reconstructed HR color image is then processed using a color correction algorithm. Simulation and experimental analyses demonstrate that this method compresses the coloring algorithm's computation time to less than 1 s, reducing both the acquisition and reconstruction times by 66.7 % compared with the traditional R + G + B channel approach, achieving a significant improvement in speed. Meanwhile, it also achieves a breakthrough in algorithm speed compared with other fast color FPM solutions, compressing the coloring time to a few hundredths or even a few thousandths of the existing algorithms, and ensures the accuracy of the reconstructed image, reducing the RMSE by 50 % compared with the existing algorithms. Regarding color consistency, CCFPM can mitigate the chromatic aberration caused by fluctuations in optical system stability, maintaining consistency between the reconstructed and original image colors, thus achieving standardization of image color appearance. Therefore, the proposed method is promising for further advancements in the application of FPM in digital pathology.

Fourier ptychographic microscopy with adaptive resolution strategy.

Fourier ptychographic microscopy (FPM) is a method capable of reconstructing a high-resolution, wide field-of-view (FOV) image, where dark-field images provide the high-frequency information required for the iterative process. Theoretically, using more dark-field images can lead to results with higher resolution. However, the resolution required to clearly detect samples with different microscales varies. For certain samples, the limit resolution of the imaging system may exceed the one required to resolve the details. This suggests that simply increasing the number of dark-field images will not improve the recognition capability for such samples and may instead significantly increase the computational cost.To address this issue, this Letter proposes an adaptive resolution strategy that automatically assigns the resolution required for the sample. Based on a Tenengrad approach, this strategy determines the number of images required for reconstruction by evaluating a series of differential images among the reconstructions for a certain subregion and then efficiently completes the full-FOV reconstruction according to the determined resolution. We conducted the full-FOV reconstruction utilizing feature-domain FPM for both the USAF resolution test chart and a human red blood cell sample. Employing the adaptive resolution strategy, the preservation of reconstruction resolution can be ensured while respectively economizing approximately 76% and 89% of the time.

Multimodal 2D and 3D microscopic mapping of growth cartilage by computational imaging techniques – a short review including new research

Being able to image the microstructure of growth cartilage is important for understanding the onset and progression of diseases such as osteochondrosis and osteoarthritis, as well as for developing new treatments and implants. Studies of cartilage using conventional optical brightfield microscopy rely heavily on histological staining, where the added chemicals provide tissue-specific colours. Other microscopy contrast mechanisms include polarization, phase- and scattering contrast, enabling non-stained or ‘label-free’ imaging that significantly simplifies the sample preparation, thereby also reducing the risk of artefacts. Traditional high-performance microscopes tend to be both bulky and expensive. Computational imaging denotes a range of techniques where computers with dedicated algorithms are used as an integral part of the image formation process. Computational imaging offers many advantages like 3D measurements, aberration correction and quantitative phase contrast, often combined with comparably cheap and compact hardware. X-ray microscopy is also progressing rapidly, in certain ways trailing the development of optical microscopy. In this study, we first briefly review the structures of growth cartilage and relevant microscopy characterization techniques, with an emphasis on Fourier ptychographic microscopy (FPM) and advanced x-ray microscopies. We next demonstrate with our own results computational imaging through FPM and compare the images with hematoxylin eosin and saffron (HES)-stained histology. Zernike phase contrast, and the nonlinear optical microscopy techniques of second harmonic generation (SHG) and two-photon excitation fluorescence (TPEF) are explored. Furthermore, X-ray attenuation-, phase- and diffraction-contrast computed tomography (CT) images of the very same sample are presented for comparisons. Future perspectives on the links to artificial intelligence, dynamic studies and in vivo possibilities conclude the article.

High-resolution, large field-of-view label-free imaging via aberration-corrected, closed-form complex field reconstruction

Computational imaging methods empower modern microscopes to produce high-resolution, large field-of-view, aberration-free images. Fourier ptychographic microscopy can increase the space-bandwidth product of conventional microscopy, but its iterative reconstruction methods are prone to parameter selection and tend to fail under excessive aberrations. Spatial Kramers–Kronig methods can analytically reconstruct complex fields, but is limited by aberration or providing extended resolution enhancement. Here, we present APIC, a closed-form method that weds the strengths of both methods while using only NA-matching and darkfield measurements. We establish an analytical phase retrieval framework which demonstrates the feasibility of analytically reconstructing the complex field associated with darkfield measurements. APIC can retrieve complex aberrations of an imaging system with no additional hardware and avoids iterative algorithms, requiring no human-designed convergence metrics while always obtaining a closed-form complex field solution. We experimentally demonstrate that APIC gives correct reconstruction results where Fourier ptychographic microscopy fails when constrained to the same number of measurements. APIC achieves 2.8 times faster computation using image tile size of 256 (length-wise), is robust against aberrations compared to Fourier ptychographic microscopy, and capable of addressing aberrations whose maximal phase difference exceeds 3.8π when using a NA 0.25 objective in experiment.

Depth-of-field extended Fourier ptychographic microscopy without defocus distance priori.

Fourier ptychographic microscopy (FPM) provides a solution of high-throughput phase imaging. Thanks to its coherent imaging model, FPM has the capacity of depth-of-field (DOF) extension by simultaneously recovering the sample's transmittance function and pupil aberration, which contains a defocus term. However, existing phase retrieval algorithms (PRs) often struggle in the presence of a significant defocus. In this Letter, different PRs with embedded pupil recovery are compared, and the one based on the alternating direction multiplier method (ADMM-FPM) demonstrates promising potential for reconstructing highly defocused FPM images. Besides, we present a plug-and-play framework that integrates ADMM-FPM and total variation or Hessian denoiser for pupil function enhancement. Both simulations and experiments demonstrate that this framework enables robust reconstruction of defocused FPM images without any prior knowledge of defocus distance or sample characteristics. In experiments involving USAF 1951 targets and pathologic slides, ADMM-FPM combined with the Hessian denoiser successfully corrected the defocus up to approximately 200 µm, i.e., extending the DOF to 400 µm.

Momentum Fourier ptychographic topography

Reflective Fourier Ptychographic Microscopy (FPM) has demonstrated the potential as a technology for semiconductor metrology and inspection applications, offering both a wide field-of-view (FOV) and high resolution with nanoscale height reconstruction accuracy. However, to enhance resolution, the wavelength of the illumination source has been shortened to the ultraviolet (UV) range, leading to a significant decrease in height calculation accuracy due to low signal-to-noise issues. Additionally, the time-consuming aspect persists as a challenge. To address these issues, we introduce Momentum Fourier Ptychographic Topography (MFPT). MFPT can overcome the problem of getting trapped in local minima due to noise, and it does not require an iterative process of total variant regularization, allowing for significant computational time savings. Using UV illumination with MFPT, we measure a United States Air Force (USAF) 1951 target and two step-height samples, providing evidence that MFPT is robust to noise and accurately reconstructs nanoscale height. As a result, utilizing UV illumination with MFPT allows us to achieve a resolution of 173 nm, and we can rapidly and accurately calculate the height of step-height samples with nominal heights of 30 nm and 70 nm.

Linear-space-variant model for Fourier ptychographic microscopy.

Fourier ptychographic microscopy (FPM) needs to realize well-accepted reconstruction by image segmentation and discarding problematic data due to artifacts caused by vignetting. However, the imaging results have long suffered from uneven color blocks and the consequent digital stitching artifacts, failing to bring satisfying experiences to researchers and users over the past decade since the invention of FPM. In fact, the fundamental reason for vignetting artifacts lies in that the acquired data does not match the adopted linear-space-invariant (LSI) forward model, i.e., the actual object function is modulated by a quadratic phase factor during data acquisition, which has been neglected in the advancement of FPM. In this Letter, we rederive a linear-space-variant (LSV) model for FPM and design the corresponding loss function for FPM, termed LSV-FPM. Utilizing LSV-FPM for optimization enables the efficient removal of wrinkle artifacts caused by vignetting in the reconstruction results, without the need of segmenting or discarding images. The effectiveness of LSV-FPM is validated through data acquired in both 4f and finite conjugate single-lens systems.

FPM-WSI: Fourier ptychographic whole slide imaging via feature-domain backdiffraction

Fourier ptychographic microscopy (FPM) theoretically provides a solution to the trade-off between spatial resolution and field of view (FOV), and has promising prospects in digital pathology. However, block reconstruction and then stitching has become an unavoidable procedure for reconstruction of large FOV due to vignetting artifacts. This introduces digital stitching artifacts, as the existing image-domain optimization algorithms are highly sensitive to systematic errors. Such obstacles significantly impede the advancement and practical implementation of FPM, explaining why, despite a decade of development, FPM has not gained widespread recognition in the field of biomedicine. We report a feature-domain FPM (FD-FPM) based on the structure-aware forward model to realize stitching-free, full-FOV reconstruction. The loss function is uniquely formulated in the feature domain of images, which bypasses the troublesome vignetting effect and algorithmic vulnerability via feature-domain backdiffraction. Through massive simulations and experiments, we show that FD-FPM effectively eliminates vignetting artifacts for full-FOV reconstruction, and still achieves impressive reconstructions despite the presence of various systematic errors. We also found it has great potential in recovering the data with a lower spectrum overlapping rate, and in realizing digital refocusing without a prior defocus distance. With FD-FPM, we achieved full-color and high-throughput imaging (4.7 mm diameter FOV, 336 nm resolution in the blue channel) free of blocking-and-stitching procedures on a self-developed Fourier ptychographic microscopy whole slide imaging platform. The reported FD-FPM shows the value of FPM for various experimental circumstances, and offers physical insights useful for the developments of models for other computational imaging techniques. The reported platform demonstrates high-quality, high-speed imaging and low cost, and could find applications in many fields of biomedical research, as well as in clinical applications.

Multi-scale fractal Fourier Ptychographic microscopy to assess the dose-dependent impact of copper pollution on living diatoms

Accumulation of bioavailable heavy metals in aquatic environment poses a serious threat to marine communities and human health due to possible trophic transfers through the food chain of toxic, non-degradable, exogenous pollutants. Copper (Cu) is one of the most spread heavy metals in water, and can severely affect primary producers at high doses. Here we show a novel imaging test to assay the dose-dependent effects of Cu on live microalgae identifying stress conditions when they are still capable of sustaining a positive growth. The method relies on Fourier Ptychographic Microscopy (FPM), capable to image large field of view in label-free phase-contrast mode attaining submicron lateral resolution. We uniquely combine FPM with a new multi-scale analysis method based on fractal geometry. The system is able to provide ensemble measurements of thousands of diatoms in the liquid sample simultaneously, while ensuring at same time single-cell imaging and analysis for each diatom. Through new image descriptors, we demonstrate that fractal analysis is suitable for handling the complexity and informative power of such multiscale FPM modality. We successfully tested this new approach by measuring how different concentrations of Cu impact on Skeletonema pseudocostatum diatom populations isolated from the Sarno River mouth.

LED-based temporal variant noise model for Fourier ptychographic microscopy

Fourier ptychographic microscopy (FPM) is a technique to reconstruct a high-resolution image from a set of low-resolution images captured with different illumination angles, which is susceptible to ambient noise, system noise, and weak currents when acquiring large-angle images, especially dark field images. To effectively address the noise problem, we propose an adaptive denoising algorithm based on a LED-based temporal variant noise model. Taking the results of blank slide samples as the reference value of noise, and analyzing the distribution of noise, we establish a statistical model for temporal variant noise, describing the relationship between temporal noise and LED spatial location. Based on this model, Gaussian denoising parameters are selected to adaptively denoise the images with different locations, with which high-resolution images can be reconstructed. Compared with other methods, the experimental results show that the proposed method effectively suppresses the noise, recovers more image details, increases the image contrast, and obtains better visual effects. Meanwhile, better objective evaluation also mirrors the advantages of the proposed algorithms.