Diffractive Neural Networks Research Articles

Multimodality Image Segmentation Embedded with Metasurface Encoder

AbstractFeature dimensionality plays an important role for modern medical diagnosis and image processing. In this work, the study introduces an optoelectronic neural network for multimodal image segmentation, which dramatically improves computing speed and decreases imaging acquisition cost in brain tumor diagnostics. Multi‐layer metasurfaces are utilized as an image preprocessor that reduces image dimensionality at the physical layer. The low‐dimensional image is then processed to a U‐Net semantic segmentation network, to handle the complex and heterogeneous nature of brain image data. By using diffractive neural network, the metasurface encoder is optimized and physically constructed with high‐efficiency transmission metasurfaces. The entire optoelectronic network attains a structural similarity index measure (SSIM) of 96%, demonstrating its potential to revolutionize on‐site medical image processing with its high precision in segmenting brain imaging data.

Metasurface-based wide-angle reflective metalens with consistently constant focusing efficiency

Terahertz (THz) communication technology has drawn a lot of attention due to advancements in global research and aspirations for 6G communication as well as the development of THz technology. It is desirable to have a reflective metalens that can gather light from many directions and focus it at a fixed place. We employed the diffractive neural network (DNN) to design a wide-angle metasurface-based reflective metalens in order to meet this technical need. This reflective metalens is capable of consistently and efficiently reflecting incident light from a wide angle of incidence and focusing it into one spot. The DNN is used to quantitatively optimize the phase distribution of the reflective metalens and the resonance phase modulation approach is adopted to develop the unit structure. Both simulation and experimentation in a 220 GHz environment were used to confirm the reflective metalens's functionality. Through increased wide-angle reflection and directional focusing homogeneity and efficiency, the suggested technology offers a practical means of addressing the problems associated with 6G communication.

Compact eternal diffractive neural network chip for extreme environments

Artificial intelligence applications in extreme environments place high demands on hardware robustness, power consumption, and speed. Recently, diffractive neural networks have demonstrated superb advantages in high-throughput light-speed reasoning. However, the robustness and lifetime of existing diffractive neural networks cannot be guaranteed, severely limiting their compactness and long-term inference accuracy. Here, we have developed a millimeter-scale and robust bilayer-integrated diffractive neural network chip with virtually unlimited lifetime for optical inference. The two diffractive layers with binary phase modulation were engraved on both sides of a quartz wafer. Optical inference of handwritten digital recognition was demonstrated. The results showed that the chip achieved 82% recognition accuracy for ten types of digits. Moreover, the chip demonstrated high-performance stability at high temperatures. The room-temperature lifetime was estimated to be 1.84×1023 trillion years. Our chip satisfies the requirements for diffractive neural network hardware with high robustness, making it suitable for use in extreme environments.

Dynamically reconfigurable all-optical neural network based on a hybrid graphene metasurface array

In recent years, optical neural networks (ONNs) have received considerable attention for their intrinsic parallelism and low energy consumption, making them a vital area of research. However, the current passive diffractive ONNs lack dynamic tunability after fabrication for specific tasks. Here, we propose a dynamically reconfigurable diffractive deep neural network based on a hybrid graphene metasurface array, wherein the transmission and refractive index of each pixel can be finely adjusted via gate voltage. This capability enables the tailored modulation of the incident light’s amplitude and phase at each pixel, aligning with specific task requirements. The simulation results show the attainability of a dynamic modulation range of 7.97dB (ranging from −8.56dB to −0.591dB). Additionally, this proposed diffractive neural network platform incorporates an ultrathin structure comprising a one-atom-thick graphene layer and nanoscale metallic metastructures, rendering it compatible with complementary metal-oxide-semiconductor technology. Notably, a classification accuracy of 92.14% for a single-layer neural network operating in the terahertz spectrum is achieved based on the calculation result. This proposed platform presents compelling prospects for constructing various artificial neural network architectures with applications ranging from drug screening to automotive driving and vision sensing.

Optical mode manipulation using deep spatial diffractive neural networks

In this paper, we investigate the theoretical models and potential applications of spatial diffractive neural network (SDNN) structures, with a particular focus on mode manipulation. Our research introduces a novel diffractive transmission simulation method that employs matrix multiplication, alongside a parameter optimization algorithm based on neural network gradient descent. This approach facilitates a comprehensive understanding of the light field manipulation capabilities inherent to SDNNs. We extend our investigation to parameter optimization for SDNNs of various scales. We achieve the demultiplexing of 5, 11 and 100 orthogonal orbital angular momentum (OAM) modes using neural networks with 4, 10 and 50 layers, respectively. Notably, the optimized 100 OAM mode demultiplexer shows an average loss of 0.52 dB, a maximum loss of 0.62 dB, and a maximum crosstalk of -28.24 dB. Further exploring the potential of SDNNs, we optimize a 10-layer structure for mode conversion applications. This optimization enables conversions from Hermite-Gaussian (HG) to Laguerre-Gaussian (LG) modes, as well as from HG to OAM modes, showing the versatility of SDNNs in mode manipulation. We propose an innovative assembly of SDNNs on a glass substrate integrated with photonic devices. A 10-layer diffractive neural network, with a size of 49 mm × 7 mm × 7 mm, effectively demultiplexes 11 orthogonal OAM modes with minimal loss and crosstalk. Similarly, a 20-layer diffractive neural network, with a size of 67 mm × 7 mm × 7 mm, serves as a highly efficient 25-channel OAM to HG mode converter, showing the potential of SDNNs in advanced optical communications.

Metasurface‐Based Optical Logic Operators Driven by Diffractive Neural Networks (Adv. Mater. 9/2024)

Metasurface‐Based Optical Logic Operators Driven by Diffractive Neural Networks (Adv. Mater. 9/2024)

Broadband Diffractive Neural Networks Enabling Classification of Visible Wavelengths

Diffractive neural networks (DNNs) are emerging as a new machine learning hardware based on optical diffraction with parallel and high‐throughput information processing. The optical inputs to DNNs are spatially modulated by propagating through passive diffractive layers that work in succession to achieve an inference. Herein, visible wavelength classification using single‐ and two‐layer DNNs fabricated using direct laser writing is demonstrated. The proposed DNN approach accepts the point spread function of two different wavelengths modeled after a microscope objective as the input and modulates the input field toward the target detector for classification. Of the three models trained to classify different wavelength pairs, the highest performance observed is for the classification of 561 and 785 nm, achieving over 90% accuracy. This work demonstrates the potential of all‐optical artificial neural networks for applications requiring visible wavelengths, from visible light beam shaping to spectral analysis and optical imaging.

Polarization-based all-optical logic gates using diffractive neural networks

Optical logic operations are an essential part of optical computing. The inherent stability and low susceptibility of polarization to the external environment make it a suitable choice for acting as the logical state in computational tasks. Traditional polarization-based optical logic devices often rely on complex cascading structures to implement multiple logic gates. In this work, by leveraging the framework of deep diffractive neural networks (D2NN), we proposed a uniform approach to designing polarization-encoded all-optical logic devices with simpler and more flexible structures. We have implemented AND, OR, NOT, NAND, and NOR gates, as well as High-order Selector and Low-order Selector. These polarization-based all-optical logic devices using D2NN offer passive nature, stability, and high extinction ratio features, paving the way for a broader exploration of optical logic computing in the future.

High fidelity laser beam shaping using liquid crystal on silicon spatial light modulators as diffractive neural networks

Spatial light modulators (SLMs) based on liquid crystal on silicon (LCoS) are powerful tools for laser beam shaping as they can be used to dynamically create almost arbitrary intensity distributions. However, laser beam shaping with LCoS-SLMs often suffers from beam shaping artifacts in part caused by unconsidered properties of the LCoS devices: astigmatism that stems from the non-normal incidence of the laser beam on the SLM and the effect commonly referred to as the ’0-th diffraction order’ that is caused by both the crosstalk between neighboring pixels and the direct reflection at the cover glass of the SLM. We here present a method to consider and compensate for these inherent properties of LCoS devices by treating the SLM as a diffractive neural network.

All dielectric metasurface based diffractive neural networks for 1-bit adder

Abstract Diffractive deep neural networks (D 2 NNs) have brought significant changes in many fields, motivating the development of diverse optical computing components. However, a crucial downside in the optical computing components is employing diffractive optical elements (DOEs) which were fabricated using commercial 3D printers. DOEs simultaneously suffer from the challenges posed by high-order diffraction and low spatial utilization since the size of individual neuron is comparable to the wavelength scale. Here, we present a design of D 2 NNs based on all-dielectric metasurfaces which substantially reduces the individual neuron size of net to scale significantly smaller than the wavelength. Metasurface-based optical computational elements can offer higher spatial neuron density while completely eliminate high-order diffraction. We numerically simulated an optical half-adder and experimentally verified it in the terahertz frequency. The optical half-adder employed a compact network with only two diffraction layers. Each layer has a size of 2 × 2 cm2 but integrated staggering 40,000 neurons. The metasurface-based D 2 NNs can further facilitate miniaturization and integration of all optical computing devices and will find applications in numerous fields such as terahertz 6G communication, photonics integrated circuits, and intelligent sensors.

可变光学环境下衍射神经网络的仿真模型研究

可变光学环境下衍射神经网络的仿真模型研究

Two-photon nanolithography of micrometer scale diffractive neural network with cubical diffraction neurons at the visible wavelength

Two-photon nanolithography of micrometer scale diffractive neural network with cubical diffraction neurons at the visible wavelength

Piston sensing for sparse aperture systems via all-optical diffractive neural network

Piston sensing for sparse aperture systems via all-optical diffractive neural network

Pluggable multitask diffractive neural networks based on cascaded metasurfaces

Pluggable multitask diffractive neural networks based on cascaded metasurfaces

Simulating an Integrated Photonic Image Classifier for Diffractive Neural Networks.

The slowdown of Moore's law and the existence of the "von Neumann bottleneck" has led to electronic-based computing systems under von Neumann's architecture being unable to meet the fast-growing demand for artificial intelligence computing. However, all-optical diffractive neural networks provide a possible solution to this challenge. They can outperform conventional silicon-based electronic neural networks due to the significantly higher speed of the propagation of optical signals (≈108 m.s-1) compared to electrical signals (≈105 m.s-1), their parallelism in nature, and their low power consumption. The integrated diffractive deep neural network (ID2NN) uses an on-chip fully passive photonic approach to achieve the functionality of neural networks (matrix-vector operations) and can be fabricated via the CMOS process, which is technologically more amenable to implementing an artificial intelligence processor. In this paper, we present a detailed design framework for the integrated diffractive deep neural network and corresponding silicon-on-insulator integration implementation through Python-based simulations. The performance of our proposed ID2NN was evaluated by solving image classification problems using the MNIST dataset.

Metasurface-Based Optical Logic Operators Driven by Diffractive Neural Networks.

In this paper, a novel optical logic operator based on the multifunctional metasurface driven by all-optical diffractive neural network is reported, which can perform four principal quantum logic operations (Pauli-X, Pauli-Y, Pauli-Z, and Hadamard gates). The two ground states and are characterized by two orthogonal linear polarization states. The proposed spatial- and polarization-multiplexed all-optical diffractive neural network only contains a hidden layer physically mapped as a metasurface with simple and compact unit cells, which dramatically reduces the volume and computing resources required for the system. The designed optical quantum operator is proven to achieve high fidelities for all four quantum logical gates, up to 99.96% numerically and 99.88% experimentally. The solution will facilitate the construction of large-scale optical quantum computing systems and scalable optical quantum devices.

Training large-scale optoelectronic neural networks with dual-neuron optical-artificial learning

Optoelectronic neural networks (ONN) are a promising avenue in AI computing due to their potential for parallelization, power efficiency, and speed. Diffractive neural networks, which process information by propagating encoded light through trained optical elements, have garnered interest. However, training large-scale diffractive networks faces challenges due to the computational and memory costs of optical diffraction modeling. Here, we present DANTE, a dual-neuron optical-artificial learning architecture. Optical neurons model the optical diffraction, while artificial neurons approximate the intensive optical-diffraction computations with lightweight functions. DANTE also improves convergence by employing iterative global artificial-learning steps and local optical-learning steps. In simulation experiments, DANTE successfully trains large-scale ONNs with 150 million neurons on ImageNet, previously unattainable, and accelerates training speeds significantly on the CIFAR-10 benchmark compared to single-neuron learning. In physical experiments, we develop a two-layer ONN system based on DANTE, which can effectively extract features to improve the classification of natural images.

Gradient method for designing cascaded DOEs and its application in the problem of classifying handwritten digits

We consider a gradient method for calculating cascaded diffractive optical elements (DOEs) consisting of several sequentially placed phase DOEs. Using the unitarity property of the operator describing the light propagation through the cascaded DOE, we obtained explicit expressions for the derivatives of the error functional with the respect to the phase functions of the cascaded DOE. We consider the application of the gradient method in the problem of focusing several different incident beams to several domains with different intensity distributions, and in the problem of image classification. The presented description of the gradient method treats the problems of designing cascaded DOEs for both focusing the laser radiation and performing image classification in the framework of a single general approach. It is shown that the difference of the problem of optical classification from the problem of generating required intensity distributions consists only in the form of error functionals, the calculation of the derivatives of which is reduced to the same general formula. Using the proposed gradient method, we designed single and cascaded DOEs for optical classification of handwritten digits. The obtained results may find application in the development of diffractive neural networks and optical systems for laser beam focusing.

Event-based diffractive neural network chip for dynamic action recognition

Dynamic action recognition has promising applications in human–computer interaction, information encryption, and high-speed image processing. However, it is challenging for a high-dynamic scene due to the huge amount of data and the computation costs. Here, we have proposed an event-based diffractive neural network (EBDNN) chip to recognize human dynamic actions. An event camera with high dynamic range and high temporal resolution is used for dynamic action capture. Based on the approach of compressing dimensions, a series of static images can be obtained by superimposing the event frames of the intervals, which serve as the train dataset of a diffractive neural network with photon wave properties. Compared to a frame camera with the same temporal resolution, the required data volume is only 0.0048% of that, which greatly shortens the training time of the diffractive neural network on the computer and saves computing resources. Notably, the diffractive neural network was trained to perform image classification tasks at the speed of light with almost no energy consumption, and the diffractive devices, which includes 400 × 400 neurons, is printed based on the two-photon polymerization technology. In addition, the experimental accuracy was as high as 99.17%. Our work opens up a new direction for photoelectrically integrated intelligent device, paving the way for the development of real-time monitoring systems and other machine vision tasks, such as autonomous driving and video surveillance.

High-frame-rate reconfigurable diffractive neural network based on superpixels.

The existing implementations of reconfigurable diffractive neural networks rely on both a liquid-crystal spatial light modulator and a digital micromirror device, which results in complexity in the alignment of the optical system and a constrained computational speed. Here, we propose a superpixel diffractive neural network that leverages solely a digital micromirror device to control the neuron bias and connection. This approach considerably simplifies the optical system and achieves a computational speed of 326 Hz per neural layer. We validate our method through experiments in digit classification, achieving an accuracy of 82.6%, and action recognition, attaining a perfect accuracy of 100%. Our findings demonstrate the effectiveness of the superpixel diffractive neural network in simplifying the optical system and enhancing computational speed, opening up new possibilities for real-time optical information processing applications.