4f System Research Articles

Polarization-multiplexed zoom Moiré metalens for edge-enhanced imaging

Optical image processing with high operational efficiency has been applied as a pre-processing imaging system for image recognition. Edge-enhanced imaging as a high-efficiency optical image processing method is of great significance for feature extraction and target recognition. However, the edge-enhanced imaging system based on the 4F system and the spatial filter transforms mainly work under coherent light illumination conditions, without continuously zooming to track the spatial position of the target. Here, we demonstrate a polarization-multiplexed zoom Moiré metalens for edge-enhanced imaging under incoherent light illumination. Metalens is designed to generate polarization-dependent optical transfer functions that produce edge-enhanced images with a resolution of 1.2 µm by digital subtraction. Furthermore, continuous zoom at the range of 1-2× is realized by constructing a Moiré metalens composed of cascaded metasurfaces. The cascaded metasurfaces consist of two center-aligned dielectric metasurfaces, each with a Moiré phase sensitive to the rotation angle. By rotating the metasurface, the phase profile of the cascaded metasurfaces changes, and the effect of continuous zoom is realized. The focal length can be actively changed from 38 µm to 77 µm with the focusing efficiency of 50.3%. This metalens can be applied to machine vision, microscopic imaging, and promotes the development of multi-functional integrated optical systems.

Multifunctional multifocal lens for simultaneous edge and polarisation imaging

ABSTRACT The method of inserting ultrathin devices in the Fourier plane of the 4F imaging system can achieve multifunctional imaging. However, such a method generally requires precise alignment in the system, which also makes the system more complex. Here, we present an imaging system that incorporates a multifocal Pancharatnam–Berry (PB) phase liquid crystal (LC) lens capable of achieving simultaneous edge and polarisation imaging. Without placing optical elements in the Fourier plane of the 4F system, the proposed imaging system replace a refracted lens in 4F system by multifocal LC lens, which integrates edge and polarisation imaging modes. The phase distribution of multifocal lens is obtained through manipulating the positions and polarisation states of four foci with holographic principles. Furthermore, the imaging system based on the proposed multifocal lens has been shown to be effective in simultaneously achieving polarisation and edge imaging with broadband characteristics. The proposed optical system effectively reduces the complexity of the imaging system, which is expected to be applied in the fields of biomedical imaging, optical interconnection and microscopic imaging.

Identification of a Weak Interaction between the Spin-Only 4f-Electronic System of Trivalent Gd Ion and a Photoexcited Cyclic π Electronic System.

An electronic interaction between the spin-only 4f-electronic system and the photoexcited π cyclic system has been identified in the anionic bis(phthalocyaninato)gadolinium(III) complex [Pc2Gd]- by using variable-temperature variable-magnetic-field magnetic circular dichroism (VTVH MCD). Two positive MCD A-term patterns, corresponding to the QH and QL absorption bands, were observed to increase in intensity as the temperature decreased, indicating a ferromagnetic-type interaction between spin angular momentum S of the 4f system and orbital angular momentum L of the photoexcited π system. A theoretical model incorporating the S-L interaction constant CSL was fitted to the VTVH MCD spectra, confirming the presence of a weak interaction, whose intensity was determined as CSL = 0.27 cm-1 for QL and CSL = 0.02 cm-1 for QH. Ab initio calculations using the RASSCF and RASSI methods revealed that the inclusion of the excited state calculations with the spin multiplicity (2S + 1) of 6 was crucial for assigning ΔSL and CSL values. This underscores the essential role of the 6PJ term in accurately modeling the S-L interaction in [Pc2Gd]-.

Michelson Interferometric Methods for Full Optical Complex Convolution.

Optical real-time data processing is advancing fields like tensor algebra acceleration, cryptography, and digital holography. This technology offers advantages such as reduced complexity through optical fast Fourier transform and passive dot-product multiplication. In this study, the proposed Reconfigurable Complex Convolution Module (RCCM) is capable of independently modulating both phase and amplitude over two million pixels. This research is relevant for applications in optical computing, hardware acceleration, encryption, and machine learning, where precise signal modulation is crucial. We demonstrate simultaneous amplitude and phase modulation of an optical two-dimensional signal in a thin lens's Fourier plane. Utilizing two spatial light modulators (SLMs) in a Michelson interferometer placed in the focal plane of two Fourier lenses, our system enables full modulation in a 4F system's Fourier domain. This setup addresses challenges like SLMs' non-linear inter-pixel crosstalk and variable modulation efficiency. The integration of these technologies in the RCCM contributes to the advancement of optical computing and related fields.

Solution to the issue of high-order diffraction images for cylindrical computer-generated holograms

The cylindrical computer-generated hologram (CCGH), featuring a 360° viewing zone, has garnered widespread attention. However, the issue of high-order diffraction images due to pixelated structure in CCGH has not been previously reported and solved. For a cylindrical model offering a 360° viewing zone in the horizontal direction, the high-order diffraction images always overlap with the reconstruction image, leading to quality degradation. Furthermore, the 4f system is commonly used to eliminate high-order diffraction images in planar CGH, but its implementation is predictably complex for a cylindrical model. In this paper, we propose a solution to the issue of high-order diffraction images for CCGH. We derive the cylindrical diffraction formula from the outer hologram surface to the inner object surface in the spectral domain, and based on this, we subsequently analyze the effects brought by the pixel structure and propose the high-order diffraction model. Based on the proposed high-order diffraction model, we use the gradient descent method to optimize CCGH accounting for all diffraction orders simultaneously. Furthermore, we discuss the issue of circular convolution due to the periodicity of the Fast Fourier Transform (FFT) in cylindrical diffraction. The correctness of the proposed high-order diffraction model and the effectiveness of the proposed optimization method are demonstrated by numerical simulation. To our knowledge, this is the first time that the issue of high-order diffraction images in CCGH has been proposed, and we believe our solution can offer valuable guidance to practitioners in the field.

Direct object detection with snapshot multispectral compressed imaging in a short-wave infrared band.

Snapshot multispectral imaging (SMSI) has attracted much attention in recent years for its compact structure and superior performance. High-level image analysis based on SMSI, such as object classification and recognition, usually takes the image reconstruction as the first step, which hinders its application in many important real-time scenarios. Here we demonstrate the first, to our knowledge, reconstruction-free strategy for object detection with SMSI in the short-wave infrared (SWIR) band. The implementation of our SMSI is based on a modified 4f system which modulates the light with a random phase mask, and the distinctive point spread function in each narrowband endows the system with spectrum resolving ability. A deep learning network with a CenterNet structure is trained to detect a small object by constructing a dataset with the PSF of our SMSI system and the sky images as background. Our results indicate that a small object with a spectral feature can be detected directly with the compressed image output by our SMSI system. This work paves the way toward the use of SMSI to detect a multispectral object in practical applications.

Towards constructing a DOE-based practical optical neural system for ship recognition in remote sensing images

The optical neural network system based on the 4f system (4f-ONN) is a feasible solution for on-orbit real-time target detection and recognition on remote sensing images, as it can directly modulate and encode the two-dimensional image information. However, traditional 4f systems based on spatial light modulators (SLMs) often encounter misalignment errors during assembly due to the reflective optical path used in SLMs. Additionally, implementing electronic SLM in space-based applications introduces problems such as particle number reversal, significantly reducing the reliability of the system. To address these issues, this paper proposes the adoption of diffractive optical elements (DOEs) to construct the 4f-ONN system. The DOE-based design offers a more compact structure, enhanced reliability, and reduced energy consumption, making it highly suitable for on-orbit image processing applications. Due to the expensive and time-consuming nature of the DOE manufacturing process, a design approach for a 4f system based on DOE was pursued through software simulation in this paper. The simulation phase involved the utilization of electronic neural networks to acquire the physical parameters of the DOE mask, while incorporating the array theorem and Fraunhofer diffraction theorem to accurately calculate the physical dimensions of the DOE. The effectiveness of the adopted DOE-based 4f system was initially validated through experiments on a simple pattern dataset. Subsequently, simulation experiments were conducted on three public datasets, namely Mnist, Fashion-mnist, and QuickDraw16, to confirm the efficacy of the DOE-based 4f system in classification tasks. Lastly, target recognition experiments were performed on the GF-2 dataset, and a corresponding hardware system was developed to demonstrate the potential of the DOE-based 4f system in on-orbit image processing.

Isolation of phase edges using off-axis q-plate filters.

Edge-enhanced microscopes with a q-plate have attracted more attention to enhance the edges of phase-amplitude objects in biological samples due to their capacity for all-directional edge enhancement, while differential interference-contrast microscopy enhances edges in only one-direction. However, the edge-enhanced microscopes cannot distinguish the edges of phase and amplitude objects, as both edges are equally enhanced. This study introduces a novel method for isolating the edge of a phase object from an amplitude object using an off-axis q-plate filter in a 4f system. Herein, we combined off-axis q-plates with four different displacements to isolate the phase object edge from the amplitude object. To demonstrate the proposed method, we conducted experiments using two distinct samples. The first sample comprised a phase test target surrounded by an aperture, and the second sample involved an overlap between the phase test target and a white hair with non-zero transmittance. In the samples, the isolated phase object edge is in good agreement with the theoretical expectations, and the amplitude object edge was reduced by approximately 93%. The proposed method is a novel and effective approach for isolating the edge of a phase object from an amplitude object and can be useful in various biological imaging applications.

High-speed full-color computer-generated holography using a digital micromirror device and fiber-coupled RGB laser diode.

Computer-generated holography (CGH) can be used to display three-dimensional (3D) images and has a special feature that no other technology possesses: it can reconstruct arbitrary object wavefronts. In this study, we investigated a high-speed full-color reconstruction method for improving the realism of 3D images produced using CGH. The proposed method uses a digital micromirror device (DMD) with a high-speed switching capability as the hologram display device. It produces 3D video by time-division multiplexing using an optical system incorporating fiber-coupled laser diodes (LDs) operating in red, green, and blue wavelengths. The wavelength dispersion of the DMD is compensated for by multiplying plane waves on the hologram. Fourier transform optics are used to separate the object, conjugate, and zeroth-order light, thus eliminating the need for an extensive 4f system. The resources used in this research, such as the programs used for the hologram generation and the schematics of the LD driver, are available on GitHub.

Laplace Differentiator Based on Metasurface with Toroidal Dipole Resonance

AbstractAs an important method of image processing, image differentiation enables edge detection of targets to achieve recognition of object features and information compression and the computation speed can be boosted by optical information techniques. Traditional optical image differentiation methods mainly rely on spatial spectrum filtering by using classical 4f system, while some work focus on 1D or single‐direction differentiation. It is only in recent years that the rapid development of metasurfaces has facilitated image differentiation methods. In this work, a Laplace operation device is demonstrated based on a silicon hollow brick dielectric resonant metasurface for incident light field. The optical transfer function (OTF) required by the optical Laplace operation can be obtained by exciting an angularly selective toroidal dipole (TD) resonance supported by the metasurface. This hollow silicon brick differentiator not only realizes 2D second‐order edge detection but also has a numerical aperture close to 0.4 and a broad working wavelength range >100 nm and can be directly integrated with imaging systems. Such metadevice may be potentially applied in the fields of optical sensing, microscopy, machine vision, biomedical imaging, etc.

Edge-enhanced microscopy of complex objects using scalar and vectorial vortex filtering.

Recently, a 4f system containing a q-plate has been used to perform edge detection and enhancement of amplitude or phase objects. However, only a few studies have concentrated on edge enhancement of complex phase-amplitude objects. Here we experimentally verified the functional difference between scalar and vectorial vortex filtering with the q-plate using an onion cell as a complex object and the vectorial vortex filtering successfully enhanced the edges of phase and amplitude objects in the phase-amplitude object. One problem, however, is indistinguishability of the equally-enhanced edges of the phase and amplitude objects. To address this issue, we propose a method to isolate the edge of the phase object from the edge of the amplitude object using off-axis beam illumination. We theoretically calculated the isolation of the edge of the phase object from the amplitude object, and verified via numerical simulations.

Multiple sub-holograms optimization for holographic near-eye display based on holographic viewing-window

A multiple sub-holograms optimization method for holographic near-eye display (NED) based on holographic viewing-window (HVW) is proposed. The HVW is a hologram generated by the point source method (PSM) with the spatial frequency constraint, and the complete reconstructed images can be observed through it. In this paper, the HVW is a phase-only hologram (POH) of a three dimensional (3D) multiplane image and it is split into multiple sub-holograms that are optimized simultaneously for pupil movement by the convolutional neural network (CNN), where each optimized constant phase (OCP) is added to the wavefront of corresponding point source. Both the simulation and the experimental results show that the quality of the reconstructed image corresponding to each sub-hologram has significantly improved, especially compared with the random constant phase (RCP) method. Moreover, we establish a holographic NED system with a 4f system to pan the optimized POH to the pupil plane to generate the HVW, and then a camera with an additional aperture is used to capture the reconstructed images corresponding to each sub-hologram. In this paper, the high-quality 3D multiplane images with natural motion parallax can be observed through an 8.64 mm×8.64 mm HVW with nine sub-holograms.

Scanning single molecule localization microscopy (scanSMLM) for super-resolution volume imaging

Over the last decade, single-molecule localization microscopy (SMLM) has developed into a set of powerful techniques that have improved spatial resolution over diffraction-limited microscopy and demonstrated the ability to resolve biological features down to a few tens of nanometers. We introduce a single molecule-based scanning SMLM (scanSMLM) system that enables rapid volume imaging. Along with epi-illumination, the system employs a scanning-based 4f detection for volume imaging. The 4f system comprises a combination of an electrically-tunable lens and high NA detection objective lens. By rapidly changing the aperture (or equivalently the focus) of an electrically-tunable lens (ETL) in a 4f detection system, the selectivity of the axial object plane is achieved, for which the image forms in the image/detector plane. So, in principle, one can scan the object volume by just altering the aperture of ETL. Two schemes were adopted to carry out volume imaging: cyclic scan and conventional scan. The cyclic scheme scans the volume in each scan cycle, whereas plane-wise scanning is performed in the conventional scheme. Hence, the cyclic scan ensures uniform dwell time on each frame during data collection, thereby evenly distributing photobleaching throughout the cell volume. With a minimal change in the system hardware (requiring the addition of an ETL lens and related electronics for step-voltage generation) in the existing SMLM system, volume scanning (along the z-axis) can be achieved. To calibrate and derive critical system parameters, we imaged fluorescent beads embedded in a gel-matrix 3D block as a test sample. Subsequently, scanSMLM is employed to visualize the architecture of actin-filaments and the distribution of Meos-Tom20 molecules on the mitochondrial membrane. The technique is further exploited to understand the clustering of Hemagglutinin (HA) protein single molecules in a transfected cell for studying Influenza-A disease progression. The system, for the first time, enabled 3D visualization of HA distribution that revealed HA cluster formation spanning the entire cell volume, post 24 hrs of transfection. Critical biophysical parameters related to HA clusters (density, the number of HA molecules per cluster, axial span, fraction of clustered molecules, and others) are also determined, giving an unprecedented insight into Influenza-A disease progression at the single-molecule level.

Speckle-based optical encryption with complex-amplitude coding and deep learning.

We propose a speckle-based optical encryption scheme by using complex-amplitude coding and deep learning, which enables the encryption and decryption of complex-amplitude plaintext containing both amplitude and phase images. During encryption, the amplitude and phase images are modulated using a superpixel-based coding technique and feded into a digital micromirror device. After passing through a 4f system, the information undergoes disturbance modulation by a scattering medium, resulting in a diffracted speckle pattern serving as the ciphertext. A Y-shaped convolutional network (Y-Net) model is constructed to establish the mapping relationship between the complex-amplitude plaintext and ciphertext through training. During decryption, the Y-Net model is utilized to quickly extract high-quality amplitude and phase images from the ciphertext. Experimental results verify the feasibility and effectiveness of our proposed method, demonstrating that the potential of integrating speckle encryption and deep learning for optical complex-amplitude encryption.

Optimization of Longitudinal Alignment of an 4f System in a Compact Vectorial Optical-Field Generator Based on a High-Resolution Liquid Crystal Spatial Light Modulator

Vectorial optical fields have garnered significant attention due to their potential applications in areas such as optical nano-fabrication, optical micromachining, quantum information processing, optical imaging, and so on. Traditional compact vectorial optical generators with amplitude modulation perform poorly in terms of diffraction effect reduction. To tackle this problem, the refractive 4f system in amplitude modulation is longitudinally aligned using an optimization approach presented in this research. The phase images used for longitudinal alignment are loaded into the liquid crystal spatial light modulator (SLM), and the distance between the lens and the mirror in the reflective 4f system is adjusted for longitudinal alignment by compensating for the neglected phase in the integrated module for the compact vectorial optical-field generator. The spot images collected by the CCD are processed using the improved eight-direction Sobel operator and Roberts function, and the longitudinal alignment in the reflective 4f system is determined by the sharpness of the image. The sharpness of the edges of the lines and the overall image are both enhanced after optimization compared to before optimization. The results demonstrate that the proposed method can effectively reduce the longitudinal alignment error of the reflective 4f system in the amplitude modulation of the compact vectorial optical-field generator, lessen the diffraction effect, and improve the performance of the system.

Second-Harmonic Generation of the Vortex Beams with Integer and Fractional Topological Charges

The single-pass second-harmonic generation (SHG) of a vortex beam under low fundamental wave depletion is systematically studied. Vortex modes at 1064 nm with integer topological charges from ±1 to ±9 and fractional ones at ±0.75 are generated by modulating the fundamental Gaussian beam with different spiral phase plates. The frequency doubling of these fundamental vortex modes is realized via single-pass SHG through the KTP. A detailed theoretical model is set up in the single-pass SHG of the vortex beams. Theoretical analysis indicates that the higher the order of the vortex beams, the lower the SHG efficiency, when the beam waists and fundamental power are given. The experimentally measured SHG output characteristics verify those obtained via theoretical analysis. Conservation of the orbital angular momentum during the SHG process is also verified, regardless of the fractional or integer vortex beams. SH LG0,2l vortex beams with high mode purity are obtained. The beam waists of fundamental/SH in KTP measured using a 4f system demonstrate that the Rayleigh ranges of the fundamental wave and SH wave are the same. The paper comprehensively presents some basic laws in the single-pass SHG of a vortex beam. In addition, it also indicates that SHG is an effective method to improve the mode purity of vortex beam.

Optical edge detection with adjustable resolution based on cascaded Pancharatnam-Berry lenses.

We designed a versatile optical edge detection setup with two cascaded Pancharatnam-Berry lenses (PBLs) placed at the Fourier plane of a 4f system. When the two PBLs are parallel and close to each other, owing to the moiré-like effect, one-dimensional edge detection with adjustable resolution is achieved by introducing a transverse displacement of one PBL. Furthermore, two-dimensional edge detection with adjustable resolution can also be realized by tuning the longitudinal distance between the PBLs, and the transverse displacement is exploited to adjust the edge resolution in specified directions. The proposed scheme is verified by a proof-of-principle experiment in which the resolution-adjustable edges of different targets and cells were clearly observed, showing its flexibility and potential application in image processing and high-contrast microscopy.

Optical Diffractive Convolutional Neural Networks Implemented in an All-Optical Way

Optical neural networks can effectively address hardware constraints and parallel computing efficiency issues inherent in electronic neural networks. However, the inability to implement convolutional neural networks at the all-optical level remains a hurdle. In this work, we propose an optical diffractive convolutional neural network (ODCNN) that is capable of performing image processing tasks in computer vision at the speed of light. We explore the application of the 4f system and the diffractive deep neural network (D2NN) in neural networks. ODCNN is then simulated by combining the 4f system as an optical convolutional layer and the diffractive networks. We also examine the potential impact of nonlinear optical materials on this network. Numerical simulation results show that the addition of convolutional layers and nonlinear functions improves the classification accuracy of the network. We believe that the proposed ODCNN model can be the basic architecture for building optical convolutional networks.

FatNet: High-Resolution Kernels for Classification Using Fully Convolutional Optical Neural Networks

This paper describes the transformation of a traditional in silico classification network into an optical fully convolutional neural network with high-resolution feature maps and kernels. When using the free-space 4f system to accelerate the inference speed of neural networks, higher resolutions of feature maps and kernels can be used without the loss in frame rate. We present FatNet for the classification of images, which is more compatible with free-space acceleration than standard convolutional classifiers. It neglects the standard combination of convolutional feature extraction and classifier dense layers by performing both in one fully convolutional network. This approach takes full advantage of the parallelism in the 4f free-space system and performs fewer conversions between electronics and optics by reducing the number of channels and increasing the resolution, making this network faster in optics than off-the-shelf networks. To demonstrate the capabilities of FatNet, it was trained with the CIFAR100 dataset on GPU and the simulator of the 4f system. A comparison of the results against ResNet-18 shows 8.2 times fewer convolution operations at the cost of only 6% lower accuracy. This demonstrates that the optical implementation of FatNet results in significantly faster inference than the optical implementation of the original ResNet-18. These are promising results for the approach of training deep learning with high-resolution kernels in the direction toward the upcoming optics era.

Single-pixel phase microscopy without [formula omitted] system

We present single-pixel phase microscopy (SPPM) without 4f system, as a novel common-path quantitative phase imaging method, to acquire the amplitude and phase of any transmissive object. To get rid of the 4f system and make the experimental setup more compact and practical, SPPM adopts the phase-modulated detection mode. A single microscopy objective is employed for image magnification, and a collecting lens is used to circumvent the quadratic phase factor introduced by the objective. A phase-only liquid crystal on silicon is utilized to modulate the phase of object light to achieve Hadamard basis scan phase imaging and Fourier basis scan phase imaging with a single photomultiplier tube. We apply SPPM to observe several different types of objects, from simple artificial objects to complex biological specimens. The experimental results show that SPPM provides clear imaging, accurate phase, and spatial resolution up to 2.19 µm (a spatial frequency of 456 lp/mm). Without the 4f system, SPPM can be implemented as an add-on module to existing optical microscopy, which facilitates its adoption in biomedical science and optical metrology.