Fourier Ptychographic Research Articles

Long-range fourier ptychographic imaging of the object in multidimensional motion

Fourier Ptychographic (FP) is a cutting-edge technique for achieving high resolution in long-range imaging, holding significant research value. However, most of the research on FP has been limited to high-resolution imaging of stationary objects, considerably narrowing the scope of its applications. In real-world scenarios, the object may move in three dimensions and rotate during the image acquisition process. To address such scenarios, this paper proposes a method for achieving FP of the object in multidimensional motion using a single camera. Starting from the principles of Fourier optics and diffraction, the paper calculates the effects of an object's movement in different dimensions on the light field. The Fourier-Mellin algorithm is to be applied to deduce changes in the light field from captured intensity images and align all collected data under a chosen reference light field. During image reconstruction, we propose an additional phase retrieval algorithm that integrates total variation regularization to address aperture offset issues. The paper validates the proposed method's effectiveness through simulations and experiments. FP is successfully applied to objects in multidimensional motion. The method also doubles the imaging system's resolution.

Leukocyte detection based on convolutional neural network fusion with color Fourier ptychographic microscopy

Abstract In the field of optical microscopic imaging, color Fourier Ptychographic Microscopy (FPM) technology has attracted much attention due to its advantages of large field of view, high resolution, and quantitative phase imaging. In this paper, a color FPM fusion algorithm based on deep learning is proposed in combination with Convolutional Neural Networks (CNN) and applied to leukocyte detection. Firstly, this paper introduces a fusion model of a convolutional neural network based on the traditional color FPM imaging method and fuses low-resolution color images and high-resolution grayscale images through a multilayer convolutional network. This method improves the quality of reconstructed images while reducing the reconstruction time. Secondly, this paper constructs a leukocyte detection dataset by using an improved color FPM reconstruction algorithm and builds a leukocyte detection system based on the YOLOv7 architecture. This paper shows that combining convolutional neural networks with color FPM technology can provide higher-quality reconstructed images in medical imaging and cell detection, which provides strong technical support for digital pathology and medical diagnosis.

Development and Assessment of Multiple Illumination Color Fourier Ptychographic Microscopy for High Throughput Sample Digitization.

In this study, we proposed a multiplexed color illumination strategy to improve the data acquisition efficiency of Fourier ptychography microscopy (FPM). Instead of sequentially lighting up one single channel LED, our method turns on multiple white light LEDs for each image acquisition via a color camera. Thus, each raw image contains multiplexed spectral information. An FPM prototype was developed, which was equipped with a 4×/0.13 NA objective lens to achieve a spatial resolution equivalent to that of a 20×/0.4 NA objective lens. Both two- and four-LED illumination patterns were designed and applied during the experiments. A USAF 1951 resolution target was first imaged under these illumination conditions, based on which MTF curves were generated to assess the corresponding imaging performance. Next, H&E tissue samples and analyzable metaphase chromosome cells were used to evaluate the clinical utility of our strategy. The results show that the single and multiplexed (two- or four-LED) illumination results achieved comparable imaging performance on all the three channels of the MTF curves. Meanwhile, the reconstructed tissue or cell images successfully retain the definition of cell nuclei and cytoplasm and can better preserve the cell edges as compared to the results from the conventional microscopes. This study initially validates the feasibility of multiplexed color illumination for the future development of high-throughput FPM scanning systems.

Fourier Ptychographic Microscopy Reconstruction Method Based on Residual Local Mixture Network.

Fourier Ptychographic Microscopy (FPM) is a microscopy imaging technique based on optical principles. It employs Fourier optics to separate and combine different optical information from a sample. However, noise introduced during the imaging process often results in poor resolution of the reconstructed image. This article has designed an approach based on a residual local mixture network to improve the quality of Fourier ptychographic reconstruction images. By incorporating channel attention and spatial attention into the FPM reconstruction process, the network enhances the efficiency of the network reconstruction and reduces the reconstruction time. Additionally, the introduction of the Gaussian diffusion model further reduces coherent artifacts and improves image reconstruction quality. Comparative experimental results indicate that this network achieves better reconstruction quality, and outperforming existing methods in both subjective observation and objective quantitative evaluation.

Investigating the Joint Amplitude and Phase Imaging of Stained Samples in Automatic Diagnosis.

The diagnosis of many diseases relies, at least on first intention, on an analysis of blood smears acquired with a microscope. However, image quality is often insufficient for the automation of such processing. A promising improvement concerns the acquisition of enriched information on samples. In particular, Quantitative Phase Imaging (QPI) techniques, which allow the digitization of the phase in complement to the intensity, are attracting growing interest. Such imaging allows the exploration of transparent objects not visible in the intensity image using the phase image only. Another direction proposes using stained images to reveal some characteristics of the cells in the intensity image; in this case, the phase information is not exploited. In this paper, we question the interest of using the bi-modal information brought by intensity and phase in a QPI acquisition when the samples are stained. We consider the problem of detecting parasitized red blood cells for diagnosing malaria from stained blood smears using a Deep Neural Network (DNN). Fourier Ptychographic Microscopy (FPM) is used as the computational microscopy framework to produce QPI images. We show that the bi-modal information enhances the detection performance by 4% compared to the intensity image only when the convolution in the DNN is implemented through a complex-based formalism. This proves that the DNN can benefit from the bi-modal enhanced information. We conjecture that these results should extend to other applications processed through QPI acquisition.

A Physics-Inspired Deep Learning Framework for an Efficient Fourier Ptychographic Microscopy Reconstruction under Low Overlap Conditions.

Two-dimensional observation of biological samples at hundreds of nanometers resolution or even below is of high interest for many sensitive medical applications. Recent advances have been obtained over the last ten years with computational imaging. Among them, Fourier Ptychographic Microscopy is of particular interest because of its important super-resolution factor. In complement to traditional intensity images, phase images are also produced. A large set of N raw images (with typically N = 225) is, however, required because of the reconstruction process that is involved. In this paper, we address the problem of FPM image reconstruction using a few raw images only (here, N = 37) as is highly desirable to increase microscope throughput. In contrast to previous approaches, we develop an algorithmic approach based on a physics-informed optimization deep neural network and statistical reconstruction learning. We demonstrate its efficiency with the help of simulations. The forward microscope image formation model is explicitly introduced in the deep neural network model to optimize its weights starting from an initialization that is based on statistical learning. The simulation results that are presented demonstrate the conceptual benefits of the approach. We show that high-quality images are effectively reconstructed without any appreciable resolution degradation. The learning step is also shown to be mandatory.

Unsupervised adaptive coded illumination Fourier ptychographic microscopy based on a physical neural network.

Fourier Ptychographic Microscopy (FPM) is a computational technique that achieves a large space-bandwidth product imaging. It addresses the challenge of balancing a large field of view and high resolution by fusing information from multiple images taken with varying illumination angles. Nevertheless, conventional FPM framework always suffers from long acquisition time and a heavy computational burden. In this paper, we propose a novel physical neural network that generates an adaptive illumination mode by incorporating temporally-encoded illumination modes as a distinct layer, aiming to improve the acquisition and calculation efficiency. Both simulations and experiments have been conducted to validate the feasibility and effectiveness of the proposed method. It is worth mentioning that, unlike previous works that obtain the intensity of a multiplexed illumination by post-combination of each sequentially illuminated and obtained low-resolution images, our experimental data is captured directly by turning on multiple LEDs with a coded illumination pattern. Our method has exhibited state-of-the-art performance in terms of both detail fidelity and imaging velocity when assessed through a multitude of evaluative aspects.

Fourier ptychographic and deep learning using breast cancer histopathological image classification.

Automated, as well as accurate classification with breast cancer histological images, was crucial for medical applications because of detecting malignant tumors via histopathological images. In this work create a Fourier ptychographic (FP) and deep learning using breast cancer histopathological image classification. Here the FP method used in the process begins with such a random guess that builds a high-resolution complex hologram, subsequently uses iterative retrieval using FP constraints to stitch around each other low-resolution multi-view means of production owned from either the hologram's high-resolution hologram's elemental images captured via integral imaging. Next, the feature extraction process includes entropy, geometrical features, and textural features. The entropy-based normalization is used to optimize the features. Finally, it attains the classification process of the proposed ENDNN classifies the breast cancer images into normal or abnormal. The experimental outcomes demonstrate that our presented technique overtakes the traditional techniques.

Efficient Synthetic Aperture for Phaseless Fourier Ptychographic Microscopy with Hybrid Coherent and Incoherent Illumination (Laser Photonics Rev. 17(3)/2023)

Efficient Synthetic Aperture for Phaseless Fourier Ptychographic Microscopy In article number 2200201, Qian Chen, Chao Zuo, and co-workers introduce an efficient synthetic aperture for phaseless Fourier ptychographic microscopy (ESA-FPM), which employs both coherent and incoherent illuminations to maximize the efficiency of data and achieve high spatial-bandwidth product reconstruction with few acquisitions. Furthermore, a customized miniaturized ESA-FPM system with a high-numerical-aperture, low-magnification objective lens, and a high-brightness LED array is built, demonstrating its potential for biomedical and pathological applications.

Fourier Ptychographic Microscopy Reconstruction Method Based on Residual Transfer Networks

Fourier ptychographic microscopy reconstruction mostly adopts the traditional alternating iterative phase recovery method and optimization method, which has high computational complexity, high redundancy of image acquisition data, low reconstruction quality and high time consumption. In this paper, the model of residual transfer networks based on Resnet152 is proposed for Fourier ptychographic microscopy reconstruction, the learning process of deep convolution neural network is introduced, and the image reconstruction method based on deep learning realizes the end-to-end reconstruction of low-resolution images to high-resolution images. Through comparative experiments and analysis, the residual network can overcome the gradient explosion, make the feature information more complete and efficient, and the incremental up-sampling reconstruction network has higher image quality, lower computational complexity and shorter running time.

A Smartphone-Based Fourier Ptychographic Microscope Using the Display Screen for Illumination

We propose a smartphone-mountable computational microscope that achieves wide-field and high-resolution imaging, based on the Fourier ptychographic (FP) microscopy technique. Our device uses the sm...

Two-photon structured illumination microscopy imaging using Fourier ptychography scheme

Super-resolution fluorescence microscopy plays an important role in the field of biological science with a tremendous potential, for the capability of observing living cells in real time. Numerous super-resolution methods have been developed to surpass the diffraction limit in recent years. Two-photon structured illumination microscopy (TPSIM) combines structured illumination microscopy (SIM) with two-photon excitation, providing wide field of view with deep penetration, and considerable resolution enhancement simultaneously. Here, we report a new algorithm for TPSIM termed as Fourier ptychographic (FP) technique. The result of simulation is presented to demonstrate that FP scheme is able to reduce the number of raw images with substantial resolution enhancement. The proposed method enables TPSIM to achieve live-cell imaging with fewer effective illumination patterns, shorter acquisition time, deeper imaging depth, and less phototoxicity. In addition, we show that, the number of raw images can further reduce to 4 for acceptable resolution improvement.

Robustness of Fourier Ptychographic Imaging to Variation of System Parameters

In most optic systems images are captured using a CCD/CMOS sensor, where the phases of the converted photons are inevitably lost. Fourier Ptychographic Microscopy (FPM) circumvents this issue by capturing normal microscopy images, and Fourier transforming them computationally (hence the name). Reconstructing the complex object not only yields amplitude but also phase information, enhanced up to super-resolution. Yet one disadvantage remains unsolved: FPM is a very ill-posed problem, the algorithm is not guaranteed to converge to the correct solution, if it converges at all. In practice this means that there is reasonable doubt if the recovered image actually represents the object under the microscope. This work inquires the quality of FPM reconstruction under variation of important system parameters in simulation and experiment. It shows that the alignment of the illumination source is quite critical: even 0.2 degrees off renders reconstruction useless. This paper thus furthers the costbenefit analysis of which amount of computation time should be spent on digital post-correction.

加权切趾传递函数约束的傅里叶叠层显微成像

加权切趾传递函数约束的傅里叶叠层显微成像

Structure-dependent amplification for denoising and background correction in Fourier ptychographic microscopy.

Fourier Ptychographic Microscopy (FPM) allows high resolution imaging using iterative phase retrieval to recover an estimate of the complex object from a series of images captured under oblique illumination. FPM is particularly sensitive to noise and uncorrected background signals as it relies on combining information from brightfield and noisy darkfield (DF) images. In this article we consider the impact of different noise sources in FPM and show that inadequate removal of the DF background signal and associated noise are the predominant cause of artefacts in reconstructed images. We propose a simple solution to FPM background correction and denoising that outperforms existing methods in terms of image quality, speed and simplicity, whilst maintaining high spatial resolution and sharpness of the reconstructed image. Our method takes advantage of the data redundancy in real space within the acquired dataset to boost the signal-to-background ratio in the captured DF images, before optimally suppressing background signal. By incorporating differentially denoised images within the classic FPM iterative phase retrieval algorithm, we show that it is possible to achieve efficient removal of background artefacts without suppression of high frequency information. The method is tested using simulated data and experimental images of thin blood films, bone marrow and liver tissue sections. Our approach is non-parametric, requires no prior knowledge of the noise distribution and can be directly applied to other hardware platforms and reconstruction algorithms making it widely applicable in FPM.

基于弧形阵列LED光源旋转照明装置的傅里叶叠层显微术

基于弧形阵列LED光源旋转照明装置的傅里叶叠层显微术

Fourier Ptychographic Microscopy for Rapid, High-Resolution Imaging of Circulating Tumor Cells Enriched by Microfiltration.

Examining the hematogenous compartment for evidence of metastasis has increased significantly within the oncology research community in recent years, due to the development of technologies aimed at the enrichment of circulating tumor cells (CTCs), the subpopulation of primary tumor cells that gain access to the circulatory system and are responsible for colonization at distant sites. In contrast to other technologies, filtration-based CTC enrichment, which exploits differences in size between larger tumor cells and surrounding smaller, non-tumor blood cells, has the potential to improve CTC characterization through isolation of tumor cell populations with greater molecular heterogeneity. However, microscopic analysis of uneven filtration surfaces containing CTCs is laborious, time-consuming, and inconsistent, preventing widespread use of filtration-based enrichment technologies. Here, integrated with a microfiltration-based CTC and rare cell enrichment device we have previously described, we present a protocol for Fourier Ptychographic Microscopy (FPM), a method that, unlike many automated imaging platforms, produces high-speed, high-resolution images that can be digitally refocused, allowing users to observe objects of interest present on multiple focal planes within the same image frame. The development of a cost-effective and high-throughput CTC analysis system for filtration-based enrichment technologies could have profound clinical implications for improved CTC detection and analysis.

Wide-field Fourier ptychographic microscopy using laser illumination source.

Fourier ptychographic (FP) microscopy is a coherent imaging method that can synthesize an image with a higher bandwidth using multiple low-bandwidth images captured at different spatial frequency regions. The method's demand for multiple images drives the need for a brighter illumination scheme and a high-frame-rate camera for a faster acquisition. We report the use of a guided laser beam as an illumination source for an FP microscope. It uses a mirror array and a 2-dimensional scanning Galvo mirror system to provide a sample with plane-wave illuminations at diverse incidence angles. The use of a laser presents speckles in the image capturing process due to reflections between glass surfaces in the system. They appear as slowly varying background fluctuations in the final reconstructed image. We are able to mitigate these artifacts by including a phase image obtained by differential phase contrast (DPC) deconvolution in the FP algorithm. We use a 1-Watt laser configured to provide a collimated beam with 150 mW of power and beam diameter of 1 cm to allow for the total capturing time of 0.96 seconds for 96 raw FPM input images in our system, with the camera sensor's frame rate being the bottleneck for speed. We demonstrate a factor of 4 resolution improvement using a 0.1 NA objective lens over the full camera field-of-view of 2.7 mm by 1.5 mm.

Fourier ptychographic reconstruction using Poisson maximum likelihood and truncated Wirtinger gradient.

Fourier ptychographic microscopy (FPM) is a novel computational coherent imaging technique for high space-bandwidth product imaging. Mathematically, Fourier ptychographic (FP) reconstruction can be implemented as a phase retrieval optimization process, in which we only obtain low resolution intensity images corresponding to the sub-bands of the sample’s high resolution (HR) spatial spectrum, and aim to retrieve the complex HR spectrum. In real setups, the measurements always suffer from various degenerations such as Gaussian noise, Poisson noise, speckle noise and pupil location error, which would largely degrade the reconstruction. To efficiently address these degenerations, we propose a novel FP reconstruction method under a gradient descent optimization framework in this paper. The technique utilizes Poisson maximum likelihood for better signal modeling, and truncated Wirtinger gradient for effective error removal. Results on both simulated data and real data captured using our laser-illuminated FPM setup show that the proposed method outperforms other state-of-the-art algorithms. Also, we have released our source code for non-commercial use.

Aperture-Scanning Fourier Ptychographic Encoding with Phase Modulation**Supported by the National Natural Science Foundation of China under Grant No 61205144, the Research Project of National University of Defense Technology under Grant No JC13-07-01, and the key Laboratory of High Power Laser and Physics of Chinese Academy of Sciences.

Phase modulation is first introduced into aperture-scanning Fourier ptychography. A series of images are acquired with an aperture scanning the Fourier plane of an optical system with a phase modulator. Then the reported algorithm synthesizes the captured images in the frequency domain to recover a high-resolution complex object wavefront. The leading results are considerably improved in robustness against the noises and shift errors, with a much faster convergence speed compared with the conventional scheme without phase modulation.