Neural Network Chip Research Articles

All-optical reconfigurable optical neural network chip based on wavelength division multiplexing

Optical computing has become an important way to achieve low power consumption and high computation speed. Optical neural network (ONN) is one of the key branches of optical computing due to its wide range of applications. However, the integrated ONN schemes proposed in previous works have some disadvantages, such as fixed network structure, complex matrix-vector multiplication (MVM) unit, and few all-optical nonlinear activation function (NAF) methods. Moreover, for the most compact MVM schemes based on wavelength division multiplexing (WDM), it is infeasible to employ intrinsic nonlinear effects to implement NAF, which brings frequent O-E-O conversion in ONN chips. Besides, it is also hard to realize a reconfigurable ONN with coherent MVMs, while it is much easier to implement in WDM schemes. We propose for the first time an all-optical silicon-based ONN chip based on WDM by adopting a new adjustment mechanism: optical gradient force (OGF). The proposed scheme is reconfigurable with tunable layers, variable neurons per layer, and adjustable NAF curves. In the task of classification of the MNIST dataset, our chip can realize an accuracy of 85.13% with 4 full-connected layers and only 50 neurons in total. In addition, we analyze the influence of the OGF-based NAF under fabrication errors and propose a calibration method. Compared to the previous works, our scheme has the two-fold advantages of compactness and reconfiguration, and it paves the way for the all-optical ONN based on WDM and opens the path to unblocking the bottleneck of integrated large-dimension ONNs.

Mixed precision quantization of silicon optical neural network chip

In recent years, the field of neural network research has witnessed remarkable advancements in various domains. One of the emerging approaches is the integration of photonic computing, which leverages the unique properties of light for ultra-fast information processing. In this article, we establish a mixed precision quantization model to silicon-based optical neural networks and evaluates their performance on the MNIST and Fashion-MNIST datasets. Through a genetic algorithm-based optimization process, we achieve significant parameter compression while maintaining competitive accuracy. Our findings demonstrate that with an average quantization bitwidth of 4.5 bits on the MNIST dataset, we achieve an impressive 85.94% reduction in parameter size compared to traditional 32-bit networks, with only a marginal accuracy drop of 0.65%. Similarly, on the Fashion-MNIST dataset, we achieve an average quantization bitwidth of 5.67 bits, resulting in an 82.28% reduction in parameter size with a slight accuracy drop of 0.8%.

Multimodal deep learning using on-chip diffractive optics with in situ training capability

Multimodal deep learning plays a pivotal role in supporting the processing and learning of diverse data types within the realm of artificial intelligence generated content (AIGC). However, most photonic neuromorphic processors for deep learning can only handle a single data modality (either vision or audio) due to the lack of abundant parameter training in optical domain. Here, we propose and demonstrate a trainable diffractive optical neural network (TDONN) chip based on on-chip diffractive optics with massive tunable elements to address these constraints. The TDONN chip includes one input layer, five hidden layers, and one output layer, and only one forward propagation is required to obtain the inference results without frequent optical-electrical conversion. The customized stochastic gradient descent algorithm and the drop-out mechanism are developed for photonic neurons to realize in situ training and fast convergence in the optical domain. The TDONN chip achieves a potential throughput of 217.6 tera-operations per second (TOPS) with high computing density (447.7 TOPS/mm2), high system-level energy efficiency (7.28 TOPS/W), and low optical latency (30.2 ps). The TDONN chip has successfully implemented four-class classification in different modalities (vision, audio, and touch) and achieve 85.7% accuracy on multimodal test sets. Our work opens up a new avenue for multimodal deep learning with integrated photonic processors, providing a potential solution for low-power AI large models using photonic technology.

Rapid species discrimination of similar insects using hyperspectral imaging and lightweight edge artificial intelligence.

Species discrimination of insects is an important aspect of ecology and biodiversity research. The traditional methods based on human visual experience and biochemical analysis cannot strike a balance between accuracy and timeliness. Morphological identification using computer vision and machine learning is expected to solve this problem, but image features have poor accuracy for very similar species and usually require complicated networks that are unfriendly to portable edge devices. In this work, we propose a fast and accurate species discrimination method of similar insects using hyperspectral features and lightweight machine learning algorithm. Feature regions selection, feature spectra selection and model quantification are used for the optimization of discriminating network. The experimental results of six similar butterfly species in the genus of Graphium show that, compared with morphological recognition with machine vision, our work achieves a higher accuracy of 92.36 ± 3.04% and a shorter inference time of 0.6 ms, with the tiny-size convolutional neural network deployed on a neural network chip. This study provides a rapid and high-accuracy species discrimination method for insects with high appearance similarity and paves the way for field discriminations using intelligent micro-spectrometer based on on-chip microstructure and artificial intelligence chip.

Quantifying the Impact of Single-Event Upsets on Neural Networks with Uncertainty Analysis

The Convolutional neural networks have gradually penetrated into various fields of social life, in addition to aerospace devices, neural network chips are also indispensable for target recognition and detection, etc. Due to the complex environment in space, neural network chips are extremely easy to be damaged by cosmic rays, so it is urgent to evaluate and enhance the robustness of neural network chips. However, most of the existing researches are monolithic error injection of neural networks and do not build relevant models for quantitative analysis. The goal of this paper is to analyse the effects of single-event upsets (SEUs) induced by the penetration of SRAM storage areas in a quantized neural network chip by energetic particles from space. Firstly, a SEU probability model as well as an uncertainty assessment model are established, and the effect of SEU on the reliability of CNNs is evaluated by fault injection with the upset rate of four chip types. The results of the error injection experiments show that the Virtex-5 type of rollover rate decreases the average accuracy of VGG-16 by 20%, while the other three types of error injection have an impact of about 11% on the accuracy of the neural network. Quantitative experimental results confirm the hypothesis that SEU increases the uncertainty of neural networks. The quantification of these effects contributes to the study of reinforcement of neural network chips.

Compact eternal diffractive neural network chip for extreme environments

Artificial intelligence applications in extreme environments place high demands on hardware robustness, power consumption, and speed. Recently, diffractive neural networks have demonstrated superb advantages in high-throughput light-speed reasoning. However, the robustness and lifetime of existing diffractive neural networks cannot be guaranteed, severely limiting their compactness and long-term inference accuracy. Here, we have developed a millimeter-scale and robust bilayer-integrated diffractive neural network chip with virtually unlimited lifetime for optical inference. The two diffractive layers with binary phase modulation were engraved on both sides of a quartz wafer. Optical inference of handwritten digital recognition was demonstrated. The results showed that the chip achieved 82% recognition accuracy for ten types of digits. Moreover, the chip demonstrated high-performance stability at high temperatures. The room-temperature lifetime was estimated to be 1.84×1023 trillion years. Our chip satisfies the requirements for diffractive neural network hardware with high robustness, making it suitable for use in extreme environments.

Cascadable excitability and inhibition in DFB laser-based photonic spiking neurons

Effective communication between neurons is crucial in both the human brain and neuromorphic computing. Excitatory and inhibitory spiking information are transmitted through axons and dendrites to adjacent neurons. In this paper, we propose a cascaded system for excitability and inhibition in photonic spiking neurons based on distributed feedback (DFB) laser subject to side-mode optical pulse injection (SMOPI). The feasibility of the cascaded system is proved by simulation and experiment. This system offers the advantage of eliminating the need for additional amplifiers. By appropriately adjusting the injection optical power and wavelength detuning between two DFB lasers, neuronal excitability and inhibition can be cascaded successfully. Both temporal encoded spikes and rate encoded spikes are considered. The operating conditions of the two lasers employed in the cascaded system are also analyzed. This work provides a promising technical pathway for the development of photonic spiking neural network (PSNN) chips, leveraging the benefits of low power consumption and high integration.

Event-based diffractive neural network chip for dynamic action recognition

Dynamic action recognition has promising applications in human–computer interaction, information encryption, and high-speed image processing. However, it is challenging for a high-dynamic scene due to the huge amount of data and the computation costs. Here, we have proposed an event-based diffractive neural network (EBDNN) chip to recognize human dynamic actions. An event camera with high dynamic range and high temporal resolution is used for dynamic action capture. Based on the approach of compressing dimensions, a series of static images can be obtained by superimposing the event frames of the intervals, which serve as the train dataset of a diffractive neural network with photon wave properties. Compared to a frame camera with the same temporal resolution, the required data volume is only 0.0048% of that, which greatly shortens the training time of the diffractive neural network on the computer and saves computing resources. Notably, the diffractive neural network was trained to perform image classification tasks at the speed of light with almost no energy consumption, and the diffractive devices, which includes 400 × 400 neurons, is printed based on the two-photon polymerization technology. In addition, the experimental accuracy was as high as 99.17%. Our work opens up a new direction for photoelectrically integrated intelligent device, paving the way for the development of real-time monitoring systems and other machine vision tasks, such as autonomous driving and video surveillance.

Photonic integrated spiking neuron chip based on a self-pulsating DFB laser with a saturable absorber

We proposed and experimentally demonstrated a simple and novel photonic spiking neuron based on a distributed feedback (DFB) laser chip with an intracavity saturable absorber (SA). The DFB laser with an intracavity SA (DFB-SA) contains a gain region and an SA region. The gain region is designed and fabricated by the asymmetric equivalent π-phase shift based on the reconstruction-equivalent-chirp technique. Under properly injected current in the gain region and reversely biased voltage in the SA region, periodic self-pulsation was experimentally observed due to the Q-switching effect. The self-pulsation frequency increases with the increase of the bias current and is within the range of several gigahertz. When the bias current is below the self-pulsation threshold, neuronlike spiking responses appear when external optical stimulus pulses are injected. Experimental results show that the spike threshold, temporal integration, and refractory period can all be observed in the fabricated DFB-SA chip. To numerically verify the experimental findings, a time-dependent coupled-wave equation model was developed, which described the physics processes inside the gain and SA regions. The numerical results agree well with the experimental measurements. We further experimentally demonstrated that the weighted sum output can readily be encoded into the self-pulsation frequency of the DFB-SA neuron. We also benchmarked the handwritten digit classification task with a simple single-layer fully connected neural network. By using the experimentally measured dependence of the self-pulsation frequency on the bias current in the gain region as an activation function, we can achieve a recognition accuracy of 92.2%, which bridges the gap between the continuous valued artificial neural networks and spike-based neuromorphic networks. To the best of our knowledge, this is the first experimental demonstration of a photonic integrated spiking neuron based on a DFB-SA, which shows great potential to realizing large-scale multiwavelength photonic spiking neural network chips.

Learning photons go backward.

Efficient learning algorithms are implemented in a silicon photonic neural network chip.

Convolutional networks with short-term memory effects

The scale expansion and depth increase of convolutional neural network lead to the increase in the number of devices and chip size, which makes the hardware implementation of neural network more and more difficult. To solve this problem, we design a convolutional neural network with short-term memory to save the number of devices and reduce the complexity of the hardware implementation. Memristor, as a natural nanoscale synapse, is one of the important devices to realize convolutional neural network chip. However, most current memristor-based neural networks use non-volatile memristors. Non-volatile memristors can only store long-term weights because of the stability of resistance, but volatile memristor can store short-term and long-term weights due to its forgetting property. This paper designs a LeNet-5 convolutional neural network by using forgetting memristor bridges composed of volatile memristors. Due to the short-term memory of volatile memristors, 49% of the number of memristors can be effectively saved compared to the neural networks using non-volatile memristors. By writing different weights for the forgetting memristor bridges, a recognition accuracy of about 97% is achieved on the MNIST dataset, and a good recognition accuracy is still achieved at 40,000 images if a recovery signal is given. What is more, in the FASHION-MNIST and ORL datasets, we also achieve 85% and 91% recognition accuracy respectively with the same network.

Multicore Photonic Complex-Valued Neural Network with Transformation Layer

Photonic neural network chips have been widely studied because of their low power consumption, high speed and large bandwidth. Using amplitude and phase to encode, photonic chips can accelerate complex-valued neural network computations. In this article, a photonic complex-valued neural network (PCNN) chip is designed. The scale of the single-core PCNN chip is limited because of optical losses, and the multicore architecture of the chip is used to improve computing capability. Further, for improving the performance of the PCNN, we propose the transformation layer, which can be implemented by the designed photonic chip to transform real-valued encoding to complex-valued encoding, which has richer information. Compared with real-valued input, the transformation layer can effectively improve the classification accuracy from 93.14% to 97.51% of a 64-dimensional input on the MNIST test set. Finally, we analyze the multicore computation of the PCNN. Compared with the single-core architecture, the multicore architecture can improve the classification accuracy by implementing larger neural networks and has better phase noise robustness. The proposed architecture and algorithms are beneficial to promote the accelerated computing of photonic chips for complex-valued neural networks and are promising for use in many applications, such as image recognition and signal processing.

Total-Ionization-Dose Radiation Effects and Hardening Techniques of a Mixed-Signal Spike Neural Network in 180 nm SOI-Pavlov Process

A mixed-signal spiking neural network (SNN) chip is presented, and its radiation effect-Total Ionizing Dose (TID) was studied. The chip was fabricated in a 180 nm silicon-on-insulator (SOI) integration process with an area of 3.75 mm2; the total doses were set at 300 krad (Si), 500 krad (Si), and 1 Mrad (Si). The TID radiation experimental results showed that the average spike frequency and spike amplitude of the output signal of the SNN circuit decreased after the irradiation because of the leakage current caused by the charge trapped in the buried oxide. Sensitive nodes were identified through the analysis of the critical path of the circuit, and guidance toward a radiation-hardening neuron circuit was proposed. The proposed circuit maintains good robustness with firing frequency variation.

A Survey of Intelligent Chip Design Research Based on Spiking Neural Networks

The traditional neural network Intelligent chip has the problem of high power consumption due to classical computing architecture, limiting the development of neural network Intelligent chips. Stochastic computing (SC) encodes binary numbers into stochastic pulse sequences in operation, taking advantage of low power consumption and high performance. The application of SC in spiking neural networks (SNNs) Intelligent chips is beneficial to solving the high power consumption of traditional neural network chips. This article first summarizes the basic elements of SNNs and the basic principles of SC. Then, we review the development trends of the stochastic computation-based neural network chips and existing SNN chips under research at home and abroad, respectively, and analyze the current problems. Finally, a review of SNN chips based on SC is highlighted. This paper aims to provide new research directions and to learn ideas for the field of SNN chips through systematic summaries.

An Efficient Interconnection System for Neural NOC Using Fault Tolerant Routing Method

Large scale Neural Network (NN) accelerators typically have multiple processing nodes that can be implemented as a multi-core chip, and can be organized on a network of chips (noise) corresponding to neurons with heavy traffic. Portions of several NoC-based NN chip-to-chip interconnect networks are linked to further enhance overall nerve amplification capacity. Large volumes of multicast on-chip or cross-chip can further complicate the construction of a cross-link network and create a NN barrier of device capacity and resources. In this paper, this refer to inter-chip and inter-chip communication strategies known as neuron connection for NN accelerators. Interconnect for powerful fault-tolerant routing system neural NoC is implemented in this paper. This recommends crossbar arbitration placement, virtual interrupts, and path-based parallelization strategies in terms of intra-chip communications for the virtual channel routing resulting in higher NoC output at lower hardware costs. A lightweight NoC compatible chip-to-chip interconnection scheme is proposed regarding to inter-chip communication for multicast-based data traffic to enable efficient interconnection for NoC-based NN chips. Moreover, the proposed methods will be tested with four Field Programmable Gate Arrays (FPGAs) on four hard-wired deep neural network (DNN) chips. From the experimental results it can be illustrate that a high throguput can obtained effectively by the proposed interconnection network in handling thedata traffic and low DNN through advanced links.

An artificial neural network chip based on two-dimensional semiconductor

Recently, research on two-dimensional (2D) semiconductors has begun to translate from the fundamental investigation into rudimentary functional circuits. In this work, we unveil the first functional MoS2 artificial neural network (ANN) chip, including multiply-and-accumulate (MAC), memory and activation function circuits. Such MoS2 ANN chip is realized through fabricating 818 field-effect transistors (FETs) on a wafer-scale and high-homogeneity MoS2 film, with a gate-last process to realize top gate structured FETs. A 62-level simulation program with integrated circuit emphasis (SPICE) model is utilized to design and optimize our analog ANN circuits. To demonstrate a practical application, a tactile digit sensing recognition was demonstrated based on our ANN circuits. After training, the digit recognition rate exceeds 97%. Our work not only demonstrates the protentional of 2D semiconductors in wafer-scale integrated circuits, but also paves the way for its future application in AI computation.

A Low-Power Spiking Neural Network Chip Based on a Compact LIF Neuron and Binary Exponential Charge Injector Synapse Circuits.

To realize a large-scale Spiking Neural Network (SNN) on hardware for mobile applications, area and power optimized electronic circuit design is critical. In this work, an area and power optimized hardware implementation of a large-scale SNN for real time IoT applications is presented. The analog Complementary Metal Oxide Semiconductor (CMOS) implementation incorporates neuron and synaptic circuits optimized for area and power consumption. The asynchronous neuronal circuits implemented benefit from higher energy efficiency and higher sensitivity. The proposed synapse circuit based on Binary Exponential Charge Injector (BECI) saves area and power consumption, and provides design scalability for higher resolutions. The SNN model implemented is optimized for 9 × 9 pixel input image and minimum bit-width weights that can satisfy target accuracy, occupies less area and power consumption. Moreover, the spiking neural network is replicated in full digital implementation for area and power comparisons. The SNN chip integrated from neuron and synapse circuits is capable of pattern recognition. The proposed SNN chip is fabricated using 180 nm CMOS process, which occupies a 3.6 mm2 chip core area, and achieves a classification accuracy of 94.66% for the MNIST dataset. The proposed SNN chip consumes an average power of 1.06 mW—20 times lower than the digital implementation.

Intelligent coding metasurface holograms by physics-assisted unsupervised generative adversarial network

Intelligent coding metasurface is a kind of information-carrying metasurface that can manipulate electromagnetic waves and associate digital information simultaneously in a smart way. One of its widely explored applications is to develop advanced schemes of dynamic holographic imaging. By now, the controlling coding sequences of the metasurface are usually designed by performing iterative approaches, including the Gerchberg–Saxton (GS) algorithm and stochastic optimization algorithm, which set a large barrier on the deployment of the intelligent coding metasurface in many practical scenarios with strong demands on high efficiency and capability. Here, we propose an efficient non-iterative algorithm for designing intelligent coding metasurface holograms in the context of unsupervised conditional generative adversarial networks (cGANs), which is referred to as physics-driven variational auto-encoder (VAE) cGAN (VAE-cGAN). Sharply different from the conventional cGAN with a harsh requirement on a large amount of manual-marked training data, the proposed VAE-cGAN behaves in a physics-driving way and thus can fundamentally remove the difficulties in the conventional cGAN. Specifically, the physical operation mechanism between the electric-field distribution and metasurface is introduced to model the VAE decoding module of the developed VAE-cGAN. Selected simulation and experimental results have been provided to demonstrate the state-of-the-art reliability and high efficiency of our VAE-cGAN. It could be faithfully expected that smart holograms could be developed by deploying our VAE-cGAN on neural network chips, finding more valuable applications in communication, microscopy, and so on.

A 617-TOPS/W All-Digital Binary Neural Network Accelerator in 10-nm FinFET CMOS

A binary neural network (BNN) chip explores the limits of energy efficiency and computational density for an all-digital deep neural network (DNN) inference accelerator. The chip intersperses data storage and computation using computation near memory (CNM) to reduce interconnect and data movement costs. It performs wide inner product operations to leverage parallelism inherent in DNN computations. The BNN chip leverages lightweight pipelining at a near-threshold voltage (NTV) to reduce the overhead of sequential elements. It employs optimized data access patterns to reduce memory accesses for convolutional operation with pooling layers. The combination of these techniques enables the BNN chip to achieve a peak energy efficiency of 617 TOPS/W. The digital BNN chip approaches the energy efficiency of analog in-memory techniques while also ensuring deterministic, scalable, and bit-accuracy operation. Moreover, the all-digital design leverages process scaling and does not require additional memory transistors or passive devices to attain a peak compute density of 418 TOPS/mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> and a memory density of 414 KB/mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> . The binary design is extended to enable bit-serial integer precision operation with a reconfigurable 1-b multiplication circuit and element-wise partial sum shift and accumulate. This technique allows for fine-grain mixed precision and retains energy efficiency by exploiting parallelism inherent in DNNs. The bit-serial binary operation allows for bit-accurate operation and high DNN accuracy that multibit analog compute-in-memory designs struggle to attain. It provides favorable energy tradeoffs compared with small-integer digital DNN accelerators.

NeuronLink: An Efficient Chip-to-Chip Interconnect for Large-Scale Neural Network Accelerators

Large-scale neural network (NN) accelerators typically consist of several processing nodes, which could be implemented as a multi- or many-core chip and organized via a network-on-chip (NoC) to handle the heavy neuron-to-neuron traffic. Multiple NoC-based NN chips are connected through chip-to-chip interconnection networks to further boost the overall neural acceleration capability. Huge amounts of multicast-based traffic travel on-chip or cross chips, making the interconnection network design more challenging and become the bottleneck of the NN system performance and energy. In this article, we propose coupling intrachip and interchip communication techniques, called NeuronLink, for NN accelerators. Regarding the intrachip communication, we propose scoring crossbar arbitration, arbitration interception, and route computation parallelization techniques for virtual-channel routing, leading to a high-throughput NoC with a lower hardware cost for multicast-based traffic. Regarding the interchip communication, we propose a lightweight and NoC-aware chip-to-chip interconnection scheme, enabling efficient interconnection for NoC-based NN chips. In addition, we evaluate the proposed techniques on a four connected NoC-based deep neural network (DNN) chips with four field-programmable gate arrays (FPGAs). The experimental results show that the proposed interconnection network can efficiently manage the data traffic inside DNNs with high-throughput and low-overhead against state-of-the-art interconnects.