Wavefront Coding Research Articles

Convolutional Neural Networks to Decode Images in a Wavefront Coding Imaging System

Convolutional Neural Networks to Decode Images in a Wavefront Coding Imaging System

Chromatic Aberration in Wavefront Coding Imaging with Trefoil Phase Mask

The refractive index of the lenses used in optical designs varies with wavelength, causing light rays to fail when focusing on a single plane. This phenomenon is known as chromatic aberration (CA), chromatic distortion, or color fringing, among other terms. Images affected by CA display colored halos and experience a loss of resolution. Fully achromatic systems can be achieved through complex and costly lens designs and/or computationally when digital sensors capture the image. In this work, we propose using the wavefront coding (WFC) technique with a trefoil-shaped phase modulation plate in the optical system to effectively increase the resolution of images affected by longitudinal chromatic aberration (LCA), significantly simplifying the optical design and reducing costs. Experimental results with three LEDs simulating RGB images verify that WFC with trefoil phase plates effectively corrects longitudinal chromatic aberration. Transverse chromatic aberration (TCA) is corrected computationally. Furthermore, we demonstrate that the optical system maintains depth of focus (DoF) for color images.

Learned phase mask to protect camera under laser irradiation

The electro-optical imaging system works under focus conditions for clear imaging. However, under unexpected laser irradiation, the focused light with extremely high intensity can easily damage the imaging sensor, resulting in permanent degradation of its perceptual capabilities. With the escalating prevalence of compact high-performance lasers, safeguarding cameras from laser damage presents a formidable challenge. Here, we report an end-to-end method to construct the wavefront coding (WFC) imaging systems with simultaneous superior laser protection and imaging performance. In the optical coding part, we employ four types of phase mask parameterization methods: pixel-wise, concentric rings, linear combinations of Zernike bases, and odd-order polynomial bases, with parameters that are learnable. In the algorithm decoding part, a method combined of deconvolution module and residual-Unet is proposed to furthest restore the phase-mask-induced image blurring. The optical and algorithm parts are jointly optimized within the end-to-end framework to determine the performance boundary. The governing rule of the laser protection capability versus imaging quality is revealed by tuning the optimization loss function, and the system database is established for various working conditions. Numerical simulations and experimental validations both demonstrate that the proposed laser-protection WFC imaging system can reduce the peak single-pixel laser power by 99.4% while maintaining high-quality imaging with peak signal-to-noise ratio more than 22 dB. This work pioneers what we believe to be a new path for the design of laser protection imaging systems, with promising applications in security and autonomous driving.

Super-Resolution Image Reconstruction of Wavefront Coding Imaging System Based on Deep Learning Network

Wavefront Coding (WFC) is an innovative technique aimed at extending the depth of focus (DOF) of optics imaging systems. In digital imaging systems, super-resolution digital reconstruction close to the diffraction limit of optical systems has always been a hot research topic. With the design of a point spread function (PSF) generated by a suitably phase mask, WFC could also be used in super-resolution image reconstruction. In this paper, we use a deep learning network combined with WFC as a general framework for images reconstruction, and verify its possibility and effectiveness. Considering the blur and additive noise simultaneously, we proposed three super-resolution image reconstruction procedures utilizing convolutional neural networks (CNN) based on mean square error (MSE) loss, conditional Generative Adversarial Networks (CGAN), and Swin Transformer Networks (SwinIR) based on mean absolute error (MAE) loss. We verified their effectiveness by simulation experiments. A comparison of experimental results shows that the SwinIR deep residual network structure based on MAE loss optimization criteria can generate more realistic super-resolution images with more details. In addition, we used a WFC camera to obtain a resolution test target and real scene images for experiments. Using the resolution test target, we demonstrated that the spatial resolution could be improved from 55.6 lp/mm to 124 lp/mm by the proposed super-resolution reconstruction procedure. The reconstruction results show that the proposed deep learning network model is superior to the traditional method in reconstructing high-frequency details and effectively suppressing noise, with the resolution approaching the diffraction limit.

Computational imaging-based single-lens imaging systems and performance evaluation.

The minimalist optical system has a simple structure, small size, and lightweight, but the low optical complexity will produce optical aberration. Addressing the significant aberration degradation in minimalist systems, we propose a high-quality computational optical framework. This framework integrates a global point spread function (PSF) change imaging model with a transformer-based U-Net deep learning algorithm to achieve high-quality imaging in minimalist systems. Additionally, we introduce an imaging performance evaluation method based on the modulation transfer degree of resolution (MTR). We addressed severe chromatic and spherical aberrations in single-lens systems, a typical example of minimalist optical systems, by simulating the degradation process and reconstructing the imaging effects. This approach demonstrated significant improvements, thus validating the feasibility of our method. Specifically, our technique calculated the MTR values in real images captured with the GCL010109 single lens at 0.8085, and with the GCL010110 single lens at 0.8055. Our method enhanced the imaging performance of minimalist systems by 4 times, upgrading minimalist system capabilities from poor to good lens grade. This work can provide reference for wavefront coding, matelens, diffraction optical systems, and other computational imaging work. It can also promote the application of miniaturization of medical, aerospace, and head-mounted optical systems.

Decoupled illumination detection in light sheet microscopy for 4D observation of spermatozoa at high-resolutions

We present the use of wavefront coding (WFC) combined with machine learning in a light sheet fluorescence microscopy (LSFM) system. We visualize the 3D dynamics of sperm flagellar motion at an imaging speed up to 80 volumes per second, which is faster than twice volumetric video rate. By using the WFC technique we achieve to extend the depth of field of the collection objective with high numerical aperture (NA=1) from 2.6 μm to 50 μm, i. e., more than one order of magnitude. To improve the quality of the final images, we applied a machine learning-based algorithm to the acquired sperm raw images and to the point spread function (PSF) of the generated cubic phase masks previous to the deconvolution process.

Optimizing wavefront coding for extended depth of field: a synchronous algorithm for optical element and decoding optimization.

Wavefront coding (WFC) combines phase mask design and image restoration algorithm to extend the depth of field (DOF) for various applications. However, discrete design limits finding globally optimal solutions, increasing the complexity of system design, and affecting the accuracy and robustness of image restoration. An end-to-end imaging system design has emerged to break through these limitations by integrating optical design and image processing algorithms. In this study, we propose an algorithm that synchronously optimizes the optical elements and decoding algorithm in WFC using ray-tracing simulation. We also derive formulas for the optical layer's forward and backward propagation for joint optimization of the optical layer and decoding algorithm. Experimental verification demonstrates the algorithm's effectiveness in optimizing the WFC system and offers improved performance under a unified design framework.

Extended Depth-of-Field Imaging Using Multi-Scale Convolutional Neural Network Wavefront Coding

Wavefront encoding (WFC) is a depth-of-field (DOF) extension technology that combines optical encoding and digital decoding. The system extends DOF at the expense of intermediate image quality and then decodes it through an image restoration algorithm to obtain a clear image. Affected by point spread differences, traditional decoding methods are often accompanied by artifacts and noise amplification problems. In this paper, based on lens-combined modulated wavefront coding (LM-WFC), we simulate the imaging process under different object distances, generate a simulation data set of WFC, and train a multi-scale convolutional neural network. The simulation experiment proves that this method can effectively reduce artifacts and improve image clarity. In addition, we used the LM-WFC camera to obtain real scene images with different target distances for experiments. The decoding results showed that the network model can enhance the quality of image restoration and generate clear images that are more in line with human vision, which is conducive to the improvement and practical application of wavefront coding systems.

Wavefront coding image reconstruction via physical prior and frequency attention.

Wavefront coding (WFC) is an effective technique for extending the depth-of-field of imaging systems, including optical encoding and digital decoding. We applied physical prior information and frequency domain model to the wavefront decoding, proposing a reconstruction method by a generative model. Specifically, we rebuild the baseline inspired by the transformer and propose three modules, including the point spread function (PSF) attention layer, multi-feature fusion block, and frequency domain self-attention block. These models are used for end-to-end learning to extract PSF feature information, fuse it into the image features, and further re-normalize the image feature information, respectively. To verify the validity, in the encoding part, we use the genetic algorithm to design a phase mask in a large field-of-view fluorescence microscope system to generate the encoded images. And the experimental results after wavefront decoding show that our method effectively reduces noise, artifacts, and blur. Therefore, we provide a deep-learning wavefront decoding model, which improves reconstruction image quality while considering the large depth-of-field (DOF) of a large field-of-view system, with good potential in detecting digital polymerase chain reaction (dPCR) and biological images.

Low-cost and simple optical system based onwavefront coding and deep learning.

With the development of computational imaging, the integration of optical system design and digital algorithms has made more imaging tasks easier to perform. Wavefront coding (WFC) is a typical computational imaging technique that is used to address the constraints of optical aperture and depth of field. In this paper, we demonstrated a low-cost and simple optical system based on WFC and deep learning. We constructed an optimized encoding method for the phase plate under the framework of deep learning, which reduces the requirement for aberration correction in the full field of view. Optical coding was achieved with just a double-bonded lens and a simple cubic phase mask, and digital decoding used the deep residual UNet++ network framework. The final image obtained has good resolution, whereas the depth of field of the system expanded by a factor of 13, which is of great significance for the high-precision inspection and attaching of small parts of machine vision.

Phase Mask Design Based on an Improved Particle Swarm Optimization Algorithm for Depth of Field Extension

Phase mask optimization is one of the critical steps in designing a wavefront coding system to extend the depth of field (DoF). As a classical phase mask, a cubic phase mask was taken as an example. An improved particle swarm optimization (PSO) algorithm was applied to calculate the parameters of the cubic phase mask by introducing the modulation transfer function as the optimization criterion and a threshold as a constraint. The quality of the subsequent image restoration is guaranteed on the premise of the extended DoF. Finally, the improved PSO was proved to be faster, more efficient, and more accurate compared to the simulated annealing algorithm and the traditional PSO. The experimental results verify that the cubic phase mask optimized by the improved PSO can achieve DoF extension in the wavefront coding system. The improved PSO can also be applied to other phase masks of wavefront coding systems.

Correcting the effect of temperature on image quality of thermal imaging objectives by wavefront coding technique

The paper has proposed a method for correcting the effect of temperature on the imaging quality of thermal imaging systems by wavefront coding technology. A cubic phase mask was added to the aperture diaphragm of the thermal imaging objective to obtain a temperature-invariant point spread function (PSF). The received images will be of low quality but almost invariant with the change in temperature. An inverse filter was used to recover high-quality images over a variable temperature range. To demonstrate the effectiveness of the proposed method, a thermal imaging objective was used to experiment. The simulation results demonstrate that the proposed method can effectively eliminate the temperature influence on the image quality of the thermal imaging objective.

Extended depth-of-field infrared imaging with deeply learned wavefront coding

Wavefront coding (WFC) techniques, including optical coding and digital image processing stages, enable significant capabilities for extending the depth of field of imaging systems. In this study, we demonstrated a deeply learned far-infrared WFC camera with an extended depth of field. We designed and optimized a high-order polynomial phase mask by a genetic algorithm, exhibiting a higher defocus consistency of the modulated transfer functions than works published previously. Additionally, we trained a generative adversarial network based on a synthesized WFC dataset for the digital processing part, which is more effective and robust than conventional decoding methods. Furthermore, we captured real-world infrared images using the WFC camera with far, middle, and near object distances. Their results after wavefront coding/decoding showed that the model of deeply learned networks improves the image quality and signal-to-noise ratio significantly and quickly. Therefore, we construct a novel artificial intelligent method of deeply learned WFC optical imaging by applying infrared wavelengths, but not limited to, and provide good potential for its practical application in "smart" imaging and large range target detection.

Development of cubic freeform optical surface for wavefront coding application for extended depth of field Infrared camera

Freeform surfaces are being developed for number of applications like illumination systems, compact projection systems, head up display, ophthalmic and automotive industries. One of the important use of such surfaces in optical imaging application is wavefront coding for large depth of focus camera. The thermal induced defocus is the well-known problem in infrared (IR) imaging systems. The efficient technique to mitigate the effect of defocus is wavefront coding, The phase profile for wavefront coding application with IR camera is a cubic freeform profile, that enables the focus invariant across the region near the focal plane. This work presents the development of a high precision cubic freeform profile on Germanium (Ge) substrate and its application for extended depth of focus for IR imaging camera. The cubic freeform profile on the surface is generated using single point diamond turning (SPDT) in slow tool servo (STS) configuration. It is further polished by using bonnet polisher with polyurethane and uninap-13 pads in seven axis polishing machine. The precise work piece alignment during the generation and characterization of the surface is very important aspect which is taken care by introducing fiducials from the beginning of the process of realization of the component. Full field measurement of grinded surface is carried out by using long wavelength interferometer. Computer generated hologram (CGH) in Fizeau interferometer is used for corrective polishing metrology and final testing of the surface profile. Precise work piece alignment and full field measurement starting from the grinding stage helped to achieve surface with residual error of 0.2 µm PV and 1.4 nm RMS roughness (Rq). The developed cubic profile is used in infrared camera for wavefront coding in an experimental setup for the verification of the intended results.

New family of Jacobi-Fourier aberrations for wavefront coding

New family of Jacobi-Fourier aberrations for wavefront coding

Improved Exponential Phase Mask for Generating Defocus Invariance of Wavefront Coding Systems

Wavefront coding is an effective way to extend the depth of field of optical imaging systems. The invariant defocusing imaging feature can be obtained by adding a phase mask with a suitable form to the aperture of a typical optical system. Traditional exponential phase mask defocusing optical characteristics exhibit strong invariance in the frequency domain, but the point spread function (PSF) variation is significant in the image plane To reduce PSF position deviation, we presented an improved exponential phase mask. The phase function of the improved mask is obtained by analyzing the relationship between Taylor expansion and wave aberration. Numerical analysis and imaging simulation are used to evaluate the performance of the proposed phase mask to that of other standard phase masks. The simulation results show that the improved exponential phase mask has a stronger defocus invariance of the modulation transfer function (MTF), and the position deviation of the PSF has been effectively controlled.

An apodized cubic phase mask used in a wavefront coding system to extend the depth of field

The point spread function (PSF) caused by a wavefront coding system with a cubic phase mask has big side-lobes which leads to bad image restoration. This paper proposes a novel apodized cubic phase mask to suppress the side-lobes of the PSF. Simulated annealing algorithm is used to optimize the cubic and the truncation parameter of the phase mask. The system with the novel phase mask has better performance in the modulation transfer function (MTF) especially in low-and-medium spatial frequency region. The simulation results show that the restored images with the novel phase mask are superior to the one with the classic cubic phase mask in contrast and ringing effect. The experimental results show that the side-lobes of the PSF are suppressed by using the apodized cubic phase mask.

The correction of dynamic aero-optical aberration of optical dome at different altitudes using wavefront coding

In our previous published paper [Opt. Commun. 490, 126876 (2021)], we proposed a method to correct the optical dome’s aero-optical aberration using wavefront coding (WFC). The simulation results were carried out under a fixed flight altitude. However, because the missile tracking the target is a dynamic process, the image quality deterioration of the aero-dynamically heated dome is diverse under different altitude. Therefore, it is of great significance to study the correction effect of WFC on dynamic aero-optical aberration of optical dome at different altitudes, which is performed in our new manuscript. The simulation results demonstrate that the WFC could effectively correct the dynamic aero-optical aberration and improve the accuracy of imaging guidance. These results confirm the possibility of the application of WFC imaging technology in missile-borne optical detection systems.

Research on corner detection algorithm based on the long depth of field binocular camera with wavefront coding

The long depth of field camera with wavefront coding is introduced in Design of Long Depth of Field Array Optical Components Defect Detection System [1]. It can obtain the whole object information at one time within 3–13 m of the depth of field. However, when binocular calibration is used in corner detection, the scalability and asymmetry of the point spread function will lead to images missing and redundancy. This article describes an improvement in the Hessian corner detector to solve this problem. Firstly, the local negative maximum with the largest change in curvature is extracted around the X corner, and the initial corner set is obtained according to their central symmetry. Then, by making use of the distance characteristics between corners of the checkerboard and constraining the angle characteristics of these corners, the false corners are filtered and the correct corner set is obtained. Finally, the corners set is sorted and the starting direction of the sorting is constrained. The corners extracted through this algorithm can be better applied to camera calibration and positioning. Experiments show that the proposed detector is better than OpenCV detector in the accuracy of positioning, calibration and measurement, and has certain robustness.

Wavefront Coding See-Through Display with Extension of Depth of Field

We propose a see-through display with a wavefront coding (WFC) technique using a cubic phase plate to extend the depth of field (DOF) of the projected virtual image. The image projected by the WFC see-through display allows the eyes to see an object in the range of accommodation and a clear projected image simultaneously without re-focusing. The DOF of the clear projected image by the system can be extended from 250 mm to an optical infinity of 10 m.