Coded Diffraction Patterns Research Articles

3D tomographic phase retrieval and unwrapping

This paper develops uniqueness theory for 3D phase retrieval with finite, discrete measurement data for strong phase objects and weak phase objects, including: (i) Unique determination of (phase) projections from diffraction patterns—General measurement schemes with coded and uncoded apertures are proposed and shown to ensure unique reduction of diffraction patterns to the phase projection for a strong phase object (respectively, the projection for a weak phase object) in each direction separately without the knowledge of relative orientations and locations. (ii) Uniqueness for 3D phase unwrapping—General conditions for unique determination of a 3D strong phase object from its phase projection data are established, including, but not limited to, random tilt schemes densely sampled from a spherical triangle of vertexes in three orthogonal directions and other deterministic tilt schemes. (iii) Uniqueness for projection tomography—Unique determination of an object of n 3 voxels from generic n projections or n + 1 coded diffraction patterns is proved. This approach of reducing 3D phase retrieval to the problem of (phase) projection tomography has the practical implication of enabling classification and alignment, when relative orientations are unknown, to be carried out in terms of (phase) projections, instead of diffraction patterns. The applications with the measurement schemes such as single-axis tilt, conical tilt, dual-axis tilt, random conical tilt and general random tilt are discussed.

Unfolding-Aided Bootstrapped Phase Retrieval in Optical Imaging: Explainable AI reveals new imaging frontiers

Phase retrieval in optical imaging refers to the recovery of a complex signal from phaseless data acquired in the form of its diffraction patterns. These patterns are acquired through a system with a coherent light source that employs a diffractive optical element (DOE) to modulate the scene, resulting in coded diffraction patterns (CDPs) at the sensor. Recently, the hybrid approach of a model-driven network or deep unfolding has emerged as an effective alternative to conventional model- and learning-based phase-retrieval techniques because it allows for bounding the complexity of algorithms while also retaining their efficacy. Additionally, such hybrid approaches have shown promise in improving the design of DOEs that follow theoretical uniqueness conditions. There are opportunities to exploit novel experimental setups and resolve even more complex DOE phase-retrieval applications. This article presents an overview of algorithms and applications of deep unfolding for bootstrapped—regardless whether near, middle, or far zones—phase retrieval.

Large-scale phase retrieval from coded diffraction patterns with electrically tunable lens

General optical detection devices rely on converting photons to electrons (current), and do not allow for direct recording of the phase due to the high oscillation frequency. As so, the missing phase can only be recovered from the intensity measurements. The emerging non-convex phase retrieval algorithm, represented by the Wirtinger Flow (WF) algorithm requires multiple-shot coded diffraction patterns (CDPs) for accurate recovery. To achieve the real-time acquisition for multiple CDPs, this paper proposes a 4f imaging system based on an electrically tunable lens (ETL), which can be used for real-time acquisition of multiple-shot CDPs, and can take the advantages of highspeed, high-resolution, extended depth-of-field, high-sensitivity and low-cost imaging. In this paper, the performance of 4f-ETL based imaging system for phase retrieval with multiple CDPs is compared under different iteration times, different object size, different numbers of masks and different noise levels. Numerical experiments demonstrate the effectiveness and superiority of our proposed ETL-based imaging system, and ETL allows variable-distance focusing of imaging and display systems without mechanical structures, which reduces the mechanical complexity and power consumption, improves acquisition speed.

Uniqueness theorems for tomographic phase retrieval with few coded diffraction patterns

3D tomographic phase retrieval under the Born approximation for discrete objects supported on a n × n × n grid is analyzed. It is proved that n projections are sufficient and necessary for unique determination by computed tomography with full projected field measurements and that n + 1 coded projected diffraction patterns are sufficient for unique determination, up to a global phase factor, in tomographic phase retrieval. Hence n + 1 is nearly, if not exactly, the minimum number of diffractions patterns needed for 3D tomographic phase retrieval under the Born approximation.

Learning spectral initialization for phase retrieval via deep neural networks.

Phase retrieval (PR) arises from the lack of phase information in the measures recorded by optical sensors. Phase masks that modulate the optical field and reduce ambiguities in the PR problem by producing redundancy in coded diffraction patterns (CDPs) have been included in these diffractive optical systems. Several algorithms have been developed to solve the PR problem from CDPs. Also, deep neural networks (DNNs) are used for solving inverse problems in computational imaging by considering physical constraints in propagation models. However, traditional algorithms based on non-convex formulation include an initialization stage that requires a high number of iterations to properly estimate the optical field. This work proposes an end-to-end (E2E) approach for addressing the PR problem, which jointly learns the spectral initialization and network parameters. Mainly, the proposed deep network approach contains an optical layer that simulates the propagation model in diffractive optical systems, an initialization layer that approximates the underlying optical field from CDPs, and a double branch DNN that improves the obtained initial guess by separately recovering phase and amplitude information. Simulation results show that the proposed E2E approach for PR requires fewer snapshots and iterations than the state of the art.

On the Convergence of Non-Convex Phase Retrieval With Denoising Priors

Denoising priors have achieved empirical success in solving non-convex phase retrieval but still lack convergence guarantees in theory. In this paper, we provide a novel insight on the convergence guarantees for a general class of non-convex phase retrieval with denoising priors. Specifically, we propose a Wirtinger flow based framework, named DWF, that allows monotone and contractive iterative optimization. We demonstrate in theory that, under a milder demi-Lipschitz condition on denoisers, the proposed DWF framework converges to the actual signal (up to a global sign) at a geometric rate. The derived convergence guarantees and rates are general to accommodate to complex-valued and real-valued signals in the presence of noise perturbation. Furthermore, we exemplify the proposed framework with two prevailing denoising priors, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , plug-and-play priors (PnP) and regularization by denoising (RED), including specific conditions on denoisers and convergence rates. To our best knowledge, this paper is the first attempt to provide theoretical guarantees of convergence for non-convex phase retrieval with denoising priors. Numerical evaluations demonstrate the theoretical results for analytic denoiser like arithmetic mean filter and deep learning based denoiser DnCNN. Furthermore, extensive experiments under the Gaussian model and coded diffraction pattern show that the proposed framework outperforms existing denoising prior-based methods and evidently reduces the necessary sampling rate for stable reconstruction with a guarantee of convergence in theory.

Robust phase retrieval with green noise binary masks

Phase retrieval with pre-defined optical masks can provide extra constraints and thus achieve improved performance. Recent progress in optimization theory demonstrates the superiority of random masks in enhancing the accuracy of phase retrieval algorithms. However, traditional approaches only focus on the randomness of the masks but ignore their non-bandlimited nature. When using these masks for phase retrieval, the intensity measurements contain many significant high-frequency components that the phase retrieval algorithm cannot take care of and thus leads to degraded performance. Based on the concept of digital halftoning, this paper proposes a green noise binary masking scheme that can significantly reduce the high-frequency contents of the masks while fulfilling the randomness requirement. The resulting intensity measurements will contain data concentrated in the mid-frequency band and around zero frequency areas which can be fully utilized in the phase retrieval optimization process. Our experimental results show that the proposed green noise binary masking scheme consistently outperforms the traditional ones when using in binary coded diffraction pattern phase retrieval systems.

Large-scale phase retrieval

High-throughput computational imaging requires efficient processing algorithms to retrieve multi-dimensional and multi-scale information. In computational phase imaging, phase retrieval (PR) is required to reconstruct both amplitude and phase in complex space from intensity-only measurements. The existing PR algorithms suffer from the tradeoff among low computational complexity, robustness to measurement noise and strong generalization on different modalities. In this work, we report an efficient large-scale phase retrieval technique termed as LPR. It extends the plug-and-play generalized-alternating-projection framework from real space to nonlinear complex space. The alternating projection solver and enhancing neural network are respectively derived to tackle the measurement formation and statistical prior regularization. This framework compensates the shortcomings of each operator, so as to realize high-fidelity phase retrieval with low computational complexity and strong generalization. We applied the technique for a series of computational phase imaging modalities including coherent diffraction imaging, coded diffraction pattern imaging, and Fourier ptychographic microscopy. Extensive simulations and experiments validate that the technique outperforms the existing PR algorithms with as much as 17dB enhancement on signal-to-noise ratio, and more than one order-of-magnitude increased running efficiency. Besides, we for the first time demonstrate ultra-large-scale phase retrieval at the 8K level (7680times 4320 pixels) in minute-level time.

Toward simple, generalizable neural networks with universal training for low-SWaP hybrid vision

Speed, generalizability, and robustness are fundamental issues for building lightweight computational cameras. Here we demonstrate generalizable image reconstruction with the simplest of hybrid machine vision systems: linear optical preprocessors combined with no-hidden-layer, “small-brain” neural networks. Surprisingly, such simple neural networks are capable of learning the image reconstruction from a range of coded diffraction patterns using two masks. We investigate the possibility of generalized or “universal training” with these small brains. Neural networks trained with sinusoidal or random patterns uniformly distribute errors around a reconstructed image, whereas models trained with a combination of sharp and curved shapes (the phase pattern of optical vortices) reconstruct edges more boldly. We illustrate variable convergence of these simple neural networks and relate learnability of an image to its singular value decomposition entropy of the image. We also provide heuristic experimental results. With thresholding, we achieve robust reconstruction of various disjoint datasets. Our work is favorable for future real-time low size, weight, and power hybrid vision: we reconstruct images on a 15 W laptop CPU with 15,000 frames per second: faster by a factor of 3 than previously reported results and 3 orders of magnitude faster than convolutional neural networks.

The effect of noise, a constant background, and bit depth on the phase retrieval of pure phase objects

In the current study, we investigate the effect of uniform white noise, Poisson noise and a constant background on the phase retrieval of pure phase objects. We also study the influence of the aforementioned factors on phase retrieval at different bit depths of intensity measurements. An algorithm called PhaseLift is used for phase retrieval as it requires a small number of modulating masks and can retrieve the phase of an object from sparse intensity measurements of low bit depth. A test object is modulated by eight random masks generated from a single mask and the phase of the object is retrieved from coded diffraction patterns. Different levels of uniform white noise, Poisson noise and constant background are superimposed on the diffraction patterns and the root-mean-square error (RMSE) of the retrieved object is calculated at each level. The results suggest that Poisson noise and a constant background at the same level cause similar RMSE compared to uniform white noise. Lowering the bit depth from 18-bits to 14-bits resulted in the decrease of the RMSE caused by Poisson noise and a constant background. We conclude that the effects of noise and constant background can be reduced by lowering the bit depth.

Inverting spectrogram measurements via aliased Wigner distribution deconvolution and angular synchronization

Abstract We propose a two-step approach for reconstructing a signal $\textbf x\in \mathbb{C}^d$ from subsampled discrete short-time Fourier transform magnitude (spectogram) measurements: first, we use an aliased Wigner distribution deconvolution approach to solve for a portion of the rank-one matrix $\widehat{\textbf{x}}\widehat{\textbf{x}}^{*}.$ Secondly, we use angular synchronization to solve for $\widehat{\textbf{x}}$ (and then for $\textbf{x}$ by Fourier inversion). Using this method, we produce two new efficient phase retrieval algorithms that perform well numerically in comparison to standard approaches and also prove two theorems; one which guarantees the recovery of discrete, bandlimited signals $\textbf{x}\in \mathbb{C}^{d}$ from fewer than $d$ short-time Fourier transform magnitude measurements and another which establishes a new class of deterministic coded diffraction pattern measurements which are guaranteed to allow efficient and noise robust recovery.

Compressed Sensing-Based Robust Phase Retrieval via Deep Generative Priors

Algorithmic phase retrieval offers an alternative means to recover the phase of optical images without requiring sophisticated measurement setups such as holography. This article proposes a framework to regularize the highly ill-posed and non-linear phase retrieval problem through deep generative priors by simply using gradient descent algorithm. We experimentally show effectiveness of the proposed approach for coded diffraction pattern measurements that are relevant in optical imaging setup. We demonstrate that the proposed approach achieves impressive results when compared with conventional hand-engineered priors in terms of number of measurements and robustness against noise. The performance of traditional generative priors based approaches depends upon how well the range of the generator spans the image class. To address this issue, we further modify the proposed algorithm to allow the generative model to explore solutions outside its range, leading to improved performance. Finally, we verify the effectiveness of the proposed approach on a real transmission matrix dataset in an actual application of multiple scattering media imaging.

Single-shot Coded Diffraction System for 3D Object Shape Estimation

The three-dimensional (3D) shape reconstruction problem of an object is a task of high interest in autonomous vehicles, detection of moving objects, and precision agriculture. A common methodology to recover the 3D shape of an object is using its optical phase. However, this approach involves solving a non-convex computationally demanding inverse problem known as phase retrieval (PR) in a setup that records coded diffraction patterns (CDP). Usually, the acquisition of several snapshots from the scene is required to solve the PR problem. This work proposes a single-shot 3D shape estimation technique using the optical phase of the object from CDP. The presented approach consists on accurately estimating the optical phase of the object by low-passfiltering the leading eigenvector of a carefully constructed matrix. Then, the estimated phase is used to infer the 3D object shape. It is important to mention that the estimation procedure does not involve a full time demanding reconstruction of the objects. Numerical results on synthetic data demonstrate that the proposed methodology closely estimates the 3D surface of an object with a normalized Mean-Square-Error of up to 0.27, under both noiseless and noisy scenarios. Additionally, the proposed method requires up to 60% less measurements to accurately estimate the 3D surface compared to a state-of-the-art methodology.

Phase Recovery Guarantees from Designed Coded Diffraction Patterns in Optical Imaging.

Phase retrieval is an inverse problem that consists in estimating a scene from diffraction intensities. This problem appears in optical imaging, which has three main diffraction zones where the measurements can be acquired, i.e., near, middle and far. Recent works have theoretically solved this inverse problem for the far zone, creating redundancy in the measurement process by including a coded aperture, which allows to modulate the scene and acquire coded diffraction patterns (CDP). However, in the state-of-the-art, the PR problem has not been theoretically studied for CDP at the near and middle zones. Moreover, the structure of the coded aperture is selected at random, leading to suboptimal estimations. Indeed, some of the coding elements employed in the literature are unfeasible because they increase the power of the scene in the modulation process. This paper provides theoretical guarantees for the recovery of a scene from CDP acquired at the three diffraction zones using admissible modulations. Based on the theoretical results, it is established that the image reconstruction quality directly depends on the coded aperture structure; therefore, a design strategy is proposed. In fact, when the scene can be sparsely represented in some basis, its support can be better estimated for a suitable choice of the coding elements in the modulation process. Experimental results show that the scene is successfully recovered by using designed coded apertures with up to 40% less measurements compared to non-designed ensembles.

Super-Resolution Phase Retrieval from Designed Coded Diffraction Patterns.

Super-resolution phase retrieval is an inverse problem that appears in diffractive optical imaging (DOI) and consists in estimating a high-resolution image from low-resolution phaseless measurements. DOI has three diffraction zones where the data can be acquired, known as near, middle, and far fields. Recent works have studied super-resolution phase retrieval under a setup that records coded diffraction patterns at the near and far fields. However, the attainable resolution of the image is mainly governed by the sensor characteristics, whose cost increases in proportion to the resolution. Also, these methodologies lack theoretical analysis. Hence, this work derives super-resolution models from low-resolution coded phaseless measurements at any diffraction zone that in contrast to prior contributions, the attainable resolution of the image is determined by the resolution of the coded aperture. For the proposed models, the existence of a unique solution (up to a global unimodular constant) is guaranteed with high probability, which can be increased by designing the coded aperture. Therefore, a strategy that designs the spatial distribution of the coded aperture is developed. Additionally, a super-resolution phase retrieval algorithm that minimizes a smoothed nonconvex least-squares objective function is proposed. The method first approximates the image by a spectral algorithm, which is then refined based upon a sequence of alternate steps. Simulation results show that the proposed algorithm overcomes state-of-the-art methods in reconstructing the high-resolution image. In addition, the reconstruction quality using designed coded apertures is higher than that of the non-designed ensembles.

Deep prior-based sparse representation model for diffraction imaging: A plug-and-play method

Diffraction imaging problem, i.e. recovery of a high-resolution or high-quality image from the intensity diffraction pattern, arises in many science and engineering fields. Recent efforts to solve this problem are exploiting sparse representation techniques. However, existing sparse representation models cannot explore inherent priors of the images to be reconstructed sufficiently. Imaging algorithms employed such traditional sparse representation models often suffer from low-quality reconstructions in the case of the noise or low-resolution observation. To address this issue, we propose a deep prior-based sparse representation (DPSR) regularization model that can impose the sparsity and the deep prior on the unknown image. The DPSR model is a plug-and-play model, namely that one can plug any effective deep denoiser into this model. We apply this model to coded diffraction imaging. To perform high-resolution imaging, a sub-pixel resolution coded diffraction pattern observation model is proposed. Based on this observation model, a diffraction imaging optimization problem is formulated. The formulated optimization problem is tackled by using the alternating optimization strategy and the epigraph concept. Compared to the benchmark diffraction imaging algorithms, the proposed algorithm has a notable peak signal-to-noise ratio (PSNR) gain of about 2 dB. Meanwhile, the proposed algorithm can perform high-resolution diffraction imaging with the pixel super-resolution factor of 4 under the single observation case. A demo code of the proposed algorithm is available at https://github.com/shibaoshun/DPSR.

Optical phase retrieval using four rotated versions of a single binary amplitude modulating mask

In recent years, phase retrieval methods recovering the phase of an object from coded diffraction patterns have gained popularity. A numerical phase retrieval method called PhaseLift that recovers the phase of an object from a very limited number of coded diffraction patterns was recently proposed. Performance of PhaseLift has been analyzed for different types and the number of masks modulating an object. We present a unique application of PhaseLift that uses four rotations of a single mask, modulating only the amplitude of an object. In simulations, a phase screen with the root-mean-square (RMS) value 0.294 μm was used as the test object. The RMS value of the retrieved phase screen after smoothing was 0.257 μm. In experiments, the RMS value of a wavefront measured with a Shack–Hartmann wavefront sensor was 0.094 while that of the retrieved wavefront after smoothing was 0.054 μm. While PhaseLift is able to recover a wavefront using this kind of modulation, a serious limitation to applicability of this method is its high computational cost and time.

Non-locally coded Fourier-transform ghost imaging.

A non-locally coded Fourier-transform ghost imaging (FGI) scheme and relevant coded phase retrieval method have been proposed to improve the image quality in ghost imaging. By inserting masks in the reference beam, the sample in the test beam is non-locally modulated, and coded Fourier-transform diffraction patterns of the sample are obtained via intensity correlation calculations between the two beams. Encoding and decoding procedures are incorporated in the phase retrieval process based on traditional hybrid input-output algorithm. Simulation and experiment results show that the spatial information of samples is successfully recovered from the coded diffraction patterns obtained using three masks, and the image quality is improved remarkably. If promoting this approach to x-ray FGI systems, it may realize high-resolution x-ray microscopy without increasing the sample's radiation damage.

A Generalization of Wirtinger Flow for Exact Interferometric Inversion

Interferometric inversion involves recovery of a signal from cross-correlations of its linear transformations. A close relative of interferometric inversion is the generalized phase retrieval problem, which consists of recovering a signal from the auto-correlations of its linear transformations. Recently, significant advancements have been made in phase retrieval methods despite the ill-posed, and non-convex nature of the problem. One such method is Wirtinger Flow (WF), a non-convex optimization framework that provides high probability guarantees of exact recovery under certain measurement models, such as coded diffraction patterns, and Gaussian sampling vectors. In this paper, we develop a generalization of WF for interferometric inversion, which we refer to as Generalized Wirtinger Flow (GWF). GWF theory extends the probabilistic exact recovery results of WF to arbitrary measurement models characterized in the equivalent lifted problem, hence covers a larger class of measurement models. Our framework unifies the theory of low rank matrix recovery (LRMR) and the non-convex optimization approach of WF, thereby establishes theoretical advantages of the non-convex approach over LRMR. We identify a new sufficient condition on the lifted forward model that directly implies exact recovery conditions of standard WF. This condition is less stringent than those of LRMR, which is the state of the art approach for exact interferometric inversion. We establish our sufficient condition for the cross-correlations of linear measurements collected by complex Gaussian sampling vectors, and show that the exact recovery conditions of standard WF imply our sufficient condition. As a result, the regularity condition of WF becomes redundant in solving the interferometric inversion problem. Finally, we demonstrate the effectiveness of GWF numerically in a deterministic multi-static radar imaging scenario.

Raster Grid Pathology and the Cure

Blind ptychography is a phase retrieval method using multiple coded diffraction patterns from different, overlapping parts of the unknown extended object illuminated with an unknown window function. The window function is also known as the probe in the optics literature. As such blind ptychography is an inverse problem of simultaneous recovery of the object and the window function given the intensities of the windowed Fourier transform and has a multiscale set-up in which the probe has an intermediate scale between the pixel scale and the macroscale of the extended object. The uniqueness problem for blind ptychography is analyzed rigorously for the raster scan (of a constant step size $\tau$) and its perturbations. The block phases are shown to form an arithmetic progression and the complete characterization of the raster scan ambiguities is given, including, first, the periodic raster grid pathology of degrees of freedom proportional to $\tau^{2}$, and, second, a nonperiodic, arithmetically progressing phase shift from block to block. Finally, irregularly perturbed raster scans are shown to remove all ambiguities other than the inherent ambiguities of the scaling factor and the affine phase factor under general requirements, including roughly the minimum overlap ratio 50%.