Photonic Neural Networks Research Articles

Quantization-aware training for low precision photonic neural networks

Recent advances in Deep Learning (DL) fueled the interest in developing neuromorphic hardware accelerators that can improve the computational speed and energy efficiency of existing accelerators. Among the most promising research directions towards this is photonic neuromorphic architectures, which can achieve femtojoule per MAC efficiencies. Despite the benefits that arise from the use of neuromorphic architectures, a significant bottleneck is the use of expensive high-speed and precision analog-to-digital (ADCs) and digital-to-analog conversion modules (DACs) required to transfer the electrical signals, originating from the various Artificial Neural Networks (ANNs) operations (inputs, weights, etc.) in the photonic optical engines. The main contribution of this paper is to study quantization phenomena in photonic models, induced by DACs/ADCs, as an additional noise/uncertainty source and to provide a photonics-compliant framework for training photonic DL models with limited precision, allowing for reducing the need for expensive high precision DACs/ADCs. The effectiveness of the proposed method is demonstrated using different architectures, ranging from fully connected and convolutional networks to recurrent architectures, following recent advances in photonic DL.

Massively scalable wavelength diverse integrated photonic linear neuron

As computing resource demands continue to escalate in the face of big data, cloud-connectivity and the internet of things, it has become imperative to develop new low-power, scalable architectures. Neuromorphic photonics, or photonic neural networks, have become a feasible solution for the physical implementation of efficient algorithms directly on-chip. This application is primarily due to the linear nature of light and the scalability of silicon photonics, specifically leveraging the wide-scale complementary metal-oxide-semiconductor manufacturing infrastructure used to fabricate microelectronics chips. Current neuromorphic photonic implementations stem from two paradigms: wavelength coherent and incoherent. Here, we introduce a novel architecture that supports coherent and incoherent operation to increase the capability and capacity of photonic neural networks with a dramatic reduction in footprint compared to previous demonstrations. As a proof-of-principle, we experimentally demonstrate simple addition and subtraction operations on a foundry-fabricated silicon photonic chip. Additionally, we experimentally validate an on-chip network to predict the logical 2 bit gates AND, OR, and XOR to accuracies of 96.8%, 99%, and 98.5%, respectively. This architecture is compatible with highly wavelength parallel sources, enabling massively scalable photonic neural networks.

Reconfigurable InP waveguide components using the Sb2S3 phase change material

Reconfigurable waveguide components are promising building blocks for photonic neural networks and as an optical analogue to field-programmable gate arrays. By changing the effective index of the waveguide, reconfigurable waveguide components can achieve on-chip light routing and modulation. In this paper, we design and demonstrate an Sb2S3-reconfigurable InP membrane Mach–Zehnder interferometer (MZI) on a silicon substrate. Sb2S3, which has tunable refractive index and low absorption in the near-infrared spectrum, was patterned on the InP waveguide MZIs to make an optical switch in the telecoms conventional-band. By laser induced crystallisation of the Sb2S3, it was possible to control interference in the MZI and achieve 18 dB on/off switching at 1540 nm. Laser reamorphisation and reversible switching of the Sb2S3 layer resulted in damage to the waveguide structure. However, simulations show that transition metal di-chalcogenide two-dimensional crystal layers can act as efficient thermal barriers that prevent thermal damage to the waveguide during laser amorphisation. Therefore, combining Sb2S3 with InP waveguides seems to be a feasible approach to achieve low-loss reprogrammable waveguide components for on-chip photonics routing and neural networks.

A Photonics-Inspired Compact Network: Toward Real-Time AI Processing in Communication Systems

Machine learning methods are ubiquitous in communication systems and have proven powerful for applications including radio-frequency (RF) fingerprinting, automatic modulation classification, and signal recovery in communication systems. However, the high throughput requirement of a communication link makes AI models difficult to implement in real-time on edge devices. In this work, we address this issue by improving both the algorithm and hardware to target real-time AI processing in communication systems. For algorithm development, we propose the first compact deep network consisting of a silicon photonic recurrent neural network model in combination with a simplified convolutional neural network classifier to identify RF emitters by their random transmissions. Our model achieves 96.32% classification accuracy over a set of 30 identical ZigBee devices when using 50 times fewer training parameters than an existing state-of-the-art CNN classifier (Merchant et al., 2018). Thanks to the large reduction in network size, we emulate the system using a small-scale FPGA board, the PYNQ-Z1, and demonstrate real-time RF fingerprinting with 0.219 ms latency. In addition, for hardware implementation, we further demonstrate a fully-integrated silicon photonic neural network for fiber nonlinearity compensation (Huang et al., 2021), which improves the received signal by 0.60 dB.

Computational metrics and parameters of an injection-locked large area semiconductor laser for neural network computing [Invited

Artificial neural networks have become a staple computing technique in many fields. Yet, they present fundamental differences with classical computing hardware in the way they process information. Photonic implementations of neural network architectures potentially offer fundamental advantages over their electronic counterparts in terms of speed, processing parallelism, scalability and energy efficiency. Scalable and high performance photonic neural networks (PNNs) have been demonstrated, yet they remain scarce. In this work, we study the performance of such a scalable, fully parallel and autonomous PNN based on large area vertical-cavity surface-emitting lasers (LA-VCSEL). We show how the performance varies with different physical parameters, namely, injection wavelength, injection power, and bias current. Furthermore, we link these physical parameters to the general computational measures of consistency and dimensionality. We present a general method of gauging dimensionality in high dimensional nonlinear systems subject to noise, which could be applied to many systems in the context of neuromorphic computing. Our work will inform future implementations of spatially multiplexed VCSEL PNNs.

All-optical graph representation learning using integrated diffractive photonic computing units.

Photonic neural networks perform brain-inspired computations using photons instead of electrons to achieve substantially improved computing performance. However, existing architectures can only handle data with regular structures but fail to generalize to graph-structured data beyond Euclidean space. Here, we propose the diffractive graph neural network (DGNN), an all-optical graph representation learning architecture based on the diffractive photonic computing units (DPUs) and on-chip optical devices to address this limitation. Specifically, the graph node attributes are encoded into strip optical waveguides, transformed by DPUs, and aggregated by optical couplers to extract their feature representations. DGNN captures complex dependencies among node neighborhoods during the light-speed optical message passing over graph structures. We demonstrate the applications of DGNN for node and graph-level classification tasks with benchmark databases and achieve superior performance. Our work opens up a new direction for designing application-specific integrated photonic circuits for high-efficiency processing large-scale graph data structures using deep learning.

Multicore Photonic Complex-Valued Neural Network with Transformation Layer

Photonic neural network chips have been widely studied because of their low power consumption, high speed and large bandwidth. Using amplitude and phase to encode, photonic chips can accelerate complex-valued neural network computations. In this article, a photonic complex-valued neural network (PCNN) chip is designed. The scale of the single-core PCNN chip is limited because of optical losses, and the multicore architecture of the chip is used to improve computing capability. Further, for improving the performance of the PCNN, we propose the transformation layer, which can be implemented by the designed photonic chip to transform real-valued encoding to complex-valued encoding, which has richer information. Compared with real-valued input, the transformation layer can effectively improve the classification accuracy from 93.14% to 97.51% of a 64-dimensional input on the MNIST test set. Finally, we analyze the multicore computation of the PCNN. Compared with the single-core architecture, the multicore architecture can improve the classification accuracy by implementing larger neural networks and has better phase noise robustness. The proposed architecture and algorithms are beneficial to promote the accelerated computing of photonic chips for complex-valued neural networks and are promising for use in many applications, such as image recognition and signal processing.

Silicon microring synapses enable photonic deep learning beyond 9-bit precision

Deep neural networks (DNNs) consist of layers of neurons interconnected by synaptic weights. A high bit-precision in weights is generally required to guarantee high accuracy in many applications. Minimizing error accumulation between layers is also essential when building large-scale networks. Recent demonstrations of photonic neural networks are limited in bit-precision due to cross talk and the high sensitivity of optical components (e.g., resonators). Here, we experimentally demonstrate a record-high precision of 9 bits with a dithering control scheme for photonic synapses. We then numerically simulated the impact with increased synaptic precision on a wireless signal classification application. This work could help realize the potential of photonic neural networks for many practical, real-world tasks.

Quantifying Power in Silicon Photonic Neural Networks

Due to challenging efficiency limits facing conventional and unconventional electronic architectures, information processors based on photonics have attracted renewed interest. Research communities have yet to settle on definitive techniques to describe the performance of this class of information processors. Photonic systems are different from electronic ones, so the existing concepts of computer performance measurement cannot necessarily apply. In this manuscript, we attempt to quantify the power use of photonic neural networks with state-of-the-art and future hardware. We derive scaling laws, physical limits, and new platform performance metrics. We find that overall performance is regime-like, which means that energy efficiency characteristics of a photonic processor can be completely described by no less than seven performance numbers. The introduction of these analytical strategies provides a much needed foundation for quantitative roadmapping and commercial value assignment for silicon photonic neural networks.

Matrix eigenvalue solver based on reconfigurable photonic neural network

Abstract The solution of matrix eigenvalues has always been a research hotspot in the field of modern numerical analysis, which has important value in practical application of engineering technology and scientific research. Despite the fact that currently existing algorithms for solving the eigenvalues of matrices are well-developed to try to satisfy both in terms of computational accuracy and efficiency, few of them have been able to be realized on photonic platform. The photonic neural network not only has strong judgment in solving inference tasks due to the superior learning ability, but also makes full use of the advantages of photonic computing with ultrahigh speed and ultralow energy consumption. Here, we propose a strategy of an eigenvalue solver for real-value symmetric matrices based on reconfigurable photonic neural networks. The strategy shows the feasibility of solving the eigenvalues of real-value symmetric matrices of n × n matrices with locally connected networks. Experimentally, we demonstrate the task of solving the eigenvalues of 2 × 2, 3 × 3, and 4 × 4 real-value symmetric matrices based on graphene/Si thermo-optical modulated reconfigurable photonic neural networks with saturated absorption nonlinear activation layer. The theoretically predicted test set accuracy of the 2 × 2 matrices is 93.6% with the measured accuracy of 78.8% in the experiment by the standard defined for simplicity of comparison. This work not only provides a feasible solution for the on-chip integrated photonic realization of eigenvalue solving of real-value symmetric matrices, but also lays the foundation for a new generation of intelligent on-chip integrated all-optical computing.

Programmable photonic neural networks combining WDM with coherent linear optics

Neuromorphic photonics has relied so far either solely on coherent or Wavelength-Division-Multiplexing (WDM) designs for enabling dot-product or vector-by-matrix multiplication, which has led to an impressive variety of architectures. Here, we go a step further and employ WDM for enriching the layout with parallelization capabilities across fan-in and/or weighting stages instead of serving the computational purpose and present, for the first time, a neuron architecture that combines coherent optics with WDM towards a multifunctional programmable neural network platform. Our reconfigurable platform accommodates four different operational modes over the same photonic hardware, supporting multi-layer, convolutional, fully-connected and power-saving layers. We validate mathematically the successful performance along all four operational modes, taking into account crosstalk, channel spacing and spectral dependence of the critical optical elements, concluding to a reliable operation with MAC relative error < 2%.

Programmable low-threshold optical nonlinear activation functions for photonic neural networks.

We experimentally demonstrate two types of programmable, low-threshold, optically controlled nonlinear activation functions, which are challenging to realize in photonic neural networks (PNNs). These devices rely on on-chip integrated Ge-Si photoelectric detectors and silicon electro-optical switches, and they generate rectified linear unit (ReLU) or sigmoid functions with arbitrary slopes without additional electrical processing. Both devices function at an extremely low threshold of 0.2 mW. The embedding of these nonlinear activation functions into convolutional neural networks facilitates the attainment of high inference accuracies of up to 95% when applied to Modified National Institute of Standards and Technology (MNIST) handwritten digit-classification tasks. The devices are suitable for low-power PNNs with an arbitrary number of propagation layers in photonic-computing chips.

Multilayer Photonic Spiking Neural Networks: Generalized Supervised Learning Algorithm and Network Optimization

We propose a generalized supervised learning algorithm for multilayer photonic spiking neural networks (SNNs) by combining the spike-timing dependent plasticity (STDP) rule and the gradient descent mechanism. A vertical-cavity surface-emitting laser with an embedded saturable absorber (VCSEL-SA) is employed as a photonic leaky-integrate-and-fire (LIF) neuron. The temporal coding strategy is employed to transform information into the precise firing time. With the modified supervised learning algorithm, the trained multilayer photonic SNN successfully solves the XOR problem and performs well on the Iris and Wisconsin breast cancer datasets. This indicates that a generalized supervised learning algorithm is realized for multilayer photonic SNN. In addition, network optimization is performed by considering different network sizes.

Channel response-aware photonic neural network accelerators for high-speed inference through bandwidth-limited optics.

Photonic neural network accelerators (PNNAs) have been lately brought into the spotlight as a new class of custom hardware that can leverage the maturity of photonic integration towards addressing the low-energy and computational power requirements of deep learning (DL) workloads. Transferring, however, the high-speed credentials of photonic circuitry into analogue neuromorphic computing necessitates a new set of DL training methods aligned along certain analogue photonic hardware characteristics. Herein, we present a novel channel response-aware (CRA) DL architecture that can address the implementation challenges of high-speed compute rates on bandwidth-limited photonic devices by incorporating their frequency response into the training procedure. The proposed architecture was validated both through software and experimentally by implementing the output layer of a neural network (NN) that classifies images of the MNIST dataset on an integrated SiPho coherent linear neuron (COLN) with a 3dB channel bandwidth of 7 GHz. A comparative analysis between the baseline and CRA model at 20, 25 and 32GMAC/sec/axon revealed respective experimental accuracies of 98.5%, 97.3% and 92.1% for the CRA model, outperforming the baseline model by 7.9%, 12.3% and 15.6%, respectively.

Quantized photonic neural network modeling method based on microring modulators

Photonic chips have great potential for neural network computing due to their fast speed, low power consumption, and parallelism. We propose a quantized neural network modeling method based on microring resonators (MRR). We analyze the optical properties of the MRRs and utilize lasers with different wavelengths as inputs of the neural network. The quantization aware method is adopted to train the neural network, and the stochastic search method is utilized to determine hyperparameters of the network. We transform the network parameters and hyperparameters into MRR parameters to simulate neural network matrix multiplication operations. Finally, we used the Mixed National Institute of Standards and Technology database for testing the proposed model. For 4-, 5-, and 6-bit quantization of weight parameters, we obtain classification accuracies of 94.23%, 94.73%, and 96.11%, respectively. Thus our study demonstrates the feasibility of building a neural network inference system using a microring structure and provides a theoretical support for applying MRRs in neural networks.

A photonic complex perceptron for ultrafast data processing

In photonic neural network a key building block is the perceptron. Here, we describe and demonstrate a complex-valued photonic perceptron that combines time and space multiplexing in a fully passive silicon photonics integrated circuit to process data in the optical domain. A time dependent input bit sequence is broadcasted into a few delay lines and detected by a photodiode. After detection, the phases are trained by a particle swarm algorithm to solve the given task. Since only the phases of the propagating optical modes are trained, signal attenuation in the perceptron due to amplitude modulation is avoided. The perceptron performs binary pattern recognition and few bit delayed XOR operations up to 16 Gbps (limited by the used electronics) with Bit Error Rates as low as 10^{-6}.

Nonlinear photonic dynamical systems for unconventional computing

Driven by the remarkable breakthroughs during the past decade, photonics neural networks have experienced a revival. Here, we provide a general overview of progress over the past decade, and sketch a roadmap of important future developments. We focus on photonic implementations of the reservoir computing machine learning paradigm, which offers a conceptually simple approach that is amenable to hardware implementations. In particular, we provide an overview of photonic reservoir computing implemented via either spatio temporal or delay dynamical systems. Going beyond reservoir computing, we discuss recent advances and future challenges of photonic implementations of deep neural networks, of the quest for learning methods that are hardware-friendly as well as realizing autonomous photonic neural networks, i.e. with minimal digital electronic auxiliary hardware.

Scalable and compact photonic neural chip with low learning-capability-loss

Abstract Photonic computation has garnered huge attention due to its great potential to accelerate artificial neural network tasks at much higher clock rate to digital electronic alternatives. Especially, reconfigurable photonic processor consisting of Mach–Zehnder interferometer (MZI) mesh is promising for photonic matrix multiplier. It is desired to implement high-radix MZI mesh to boost the computation capability. Conventionally, three cascaded MZI meshes (two universal N × N unitary MZI mesh and one diagonal MZI mesh) are needed to express N × N weight matrix with O(N 2) MZIs requirements, which limits scalability seriously. Here, we propose a photonic matrix architecture using the real-part of one nonuniversal N × N unitary MZI mesh to represent the real-value matrix. In the applications like photonic neural network, it probable reduces the required MZIs to O(Nlog2 N) level while pay low cost on learning capability loss. Experimentally, we implement a 4 × 4 photonic neural chip and benchmark its performance in convolutional neural network for handwriting recognition task. Low learning-capability-loss is observed in our 4 × 4 chip compared to its counterpart based on conventional architecture using O(N 2) MZIs. While regarding the optical loss, chip size, power consumption, encoding error, our architecture exhibits all-round superiority.

Ultralow Light‐Power Consuming Photonic Synapses Based on Ultrasensitive Perovskite/Indium‐Gallium‐Zinc‐Oxide Heterojunction Phototransistors

AbstractThe brain‐inspired neuromorphic parallel computing has become one of the most promising technologies for efficient information processing by overcoming the von Neumann bottleneck of sequential operations. Synapses transmit information among neurons and act as the basic component in neuromorphic computing platforms. Despite the rapid advances in developing artificial synapses, most synaptic devices function electronically and thus numerous merits for photonics such as visual/imaging information processing, less corsstalk, and fast response, have to be compromised. Herein, a light‐stimulated synaptic device (photonic synapses) based on perovskite/In‐Ga‐Zn‐O heterojunction phototransistor is reported. The combination of high‐efficiency light absorber, high‐mobility channel, and heterojunction device architecture leads to efficient photon‐to‐electron conversion and intrinsic high‐gain mechanism for such light‐stimulated synapses. As a result, high performance photonic synapses are obtained with the basic functions of excitatory postsynaptic current (EPSC), paired‐pulse facilitation (PPF), and short‐term memory to long‐term memory conversion (STM‐LTM). Importantly, owing to the ultrasensitive photodetection characteristics, the light power consumption of such photonic artificial synapse can be as low as 2.6 picojoule. This study proposes a simple, efficient, and industry‐compatible device concept providing the photosensitive synapses for photonic neural networks by combining the merits of appropriate materials and device architecture.

An ITO–graphene heterojunction integrated absorption modulator on Si-photonics for neuromorphic nonlinear activation

The high demand for machine intelligence of doubling every three months is driving novel hardware solutions beyond charging of electrical wires, given a resurrection to application specific integrated circuit (ASIC)-based accelerators. These innovations include photonic-based ASICs (P-ASICs) due to prospects of performing optical linear (and also nonlinear) operations, such as multiply–accumulate for vector matrix multiplications or convolutions, without iterative architectures. Such photonic linear algebra enables picosecond delay when photonic integrated circuits are utilized via “on-the-fly” mathematics. However, the neuron’s full function includes providing a nonlinear activation function, known as thresholding, to enable decision making on inferred data. Many P-ASIC solutions perform this nonlinearity in the electronic domain, which brings challenges in terms of data throughput and delay, thus breaking the optical link and introducing increased system complexity via domain crossings. This work follows the notion of utilizing enhanced light–matter interactions to provide efficient, compact, and engineerable electro-optic neuron nonlinearity. Here, we introduce and demonstrate a novel electro-optic device to engineer the shape of this optical nonlinearity to resemble a leaky rectifying linear unit—the most commonly used nonlinear activation function in neural networks. We combine the counter-directional transfer functions from heterostructures made out of two electro-optic materials to design a diode-like nonlinear response of the device. Integrating this nonlinearity into a photonic neural network, we show how the electrostatics of this thresholder’s gating junction improves machine learning inference accuracy and the energy efficiency of the neural network.