Digital Neural Network Research Articles

Photonic neuromorphic accelerators for event-based imaging flow cytometry

In this work, we present experimental results of a high-speed label-free imaging cytometry system that seamlessly merges the high-capturing rate and data sparsity of an event-based CMOS camera with lightweight photonic neuromorphic processing. This combination offers high classification accuracy and a massive reduction in the number of trainable parameters of the digital machine-learning back-end. The event-based camera is capable of capturing 1 Gevents/sec, where events correspond to pixel contrast changes, similar to the retina’s ganglion cell function. The photonic neuromorphic accelerator is based on a hardware-friendly passive optical spectrum slicing technique that is able to extract meaningful features from the generated spike-trains using a purely analogue version of the convolutional operation. The experimental scenario comprises the discrimination of artificial polymethyl methacrylate calibrated beads, having different diameters, flowing at a mean speed of 0.1 m/sec. Classification accuracy, using only lightweight digital machine-learning schemes has topped at 98.2%. On the other hand, by experimentally pre-processing the raw spike data through the proposed photonic neuromorphic spectrum slicer at a rate of 3 × 106 images per second, we achieved an accuracy of 98.6%. This performance was accompanied by a reduction in the number of trainable parameters at the classification back-end by a factor ranging from 8 to 22, depending on the configuration of the digital neural network. These results confirm that neuromorphic sensing and neuromorphic computing can be efficiently merged to a unified bio-inspired system, offering a holistic enhancement in emerging bio-imaging applications.

Concurrent Image Differentiation and Integration Processings Enabled By Polarization‐Multiplexed Metasurface

AbstractOptical computing and image processing performed by sensor front‐end metasurfaces is receiving increasing interest because of advantages such as significant reduction of latency time, energy consumption, and system complexity. Despite the rapid progress, concurrent processing, the most important feature of electronic computing, has not yet been well implemented in optical computing. Here, a metasurface‐based optical image processor that can perform optical differentiation and integration tasks simultaneously is proposed. This optical front‐end processor integrates two coherent transfer functions corresponding to differential and integral convolution kernels into a built‐in metasurface by polarization encoding, allowing concurrent processing of multiple all‐optical computational tasks. The simultaneous differentiation and integration operations on images for edge enhancement and denoising are demonstrated at multiple visible wavelengths. This concurrent processing architecture paves a promising pathway toward multifunctional and higher‐speed image processing for machine vision and biomedical imaging and shows the potential to expedite and potentially supplant certain digital neural network algorithms.

Integration of Programmable Diffraction with Digital Neural Networks

Integration of Programmable Diffraction with Digital Neural Networks

Nonlinear encoding in diffractive information processing using linear optical materials

Nonlinear encoding of optical information can be achieved using various forms of data representation. Here, we analyze the performances of different nonlinear information encoding strategies that can be employed in diffractive optical processors based on linear materials and shed light on their utility and performance gaps compared to the state-of-the-art digital deep neural networks. For a comprehensive evaluation, we used different datasets to compare the statistical inference performance of simpler-to-implement nonlinear encoding strategies that involve, e.g., phase encoding, against data repetition-based nonlinear encoding strategies. We show that data repetition within a diffractive volume (e.g., through an optical cavity or cascaded introduction of the input data) causes the loss of the universal linear transformation capability of a diffractive optical processor. Therefore, data repetition-based diffractive blocks cannot provide optical analogs to fully connected or convolutional layers commonly employed in digital neural networks. However, they can still be effectively trained for specific inference tasks and achieve enhanced accuracy, benefiting from the nonlinear encoding of the input information. Our results also reveal that phase encoding of input information without data repetition provides a simpler nonlinear encoding strategy with comparable statistical inference accuracy to data repetition-based diffractive processors. Our analyses and conclusions would be of broad interest to explore the push-pull relationship between linear material-based diffractive optical systems and nonlinear encoding strategies in visual information processors.

Computational Investigation of the Potential and Limitations of Machine Learning with Neural Network Circuits Based on Synaptic Transistors.

Synaptic transistors have been proposed to implement neuron activation functions of neural networks (NNs). While promising to enable compact, fast, inexpensive, and energy-efficient dedicated NN circuits, they also have limitations compared to digital NNs (realized as codes for digital processors), including shape choices of the activation function using particular types of transistor implementation, and instabilities due to noise and other factors present in analog circuits. We present a computational study of the effects of these factors on NN performance and find that, while accuracy competitive with traditional NNs can be realized for many applications, there is high sensitivity to the instability in the shape of the activation function, suggesting that, when highly accurate NNs are required, high-precision circuitry should be developed beyond what has been reported for synaptic transistors to date.

Programming the scalable optical learning operator with spatial-spectral optimization

Electronic computers have evolved drastically over the past years with an ever-growing demand for improved performance. However, the transfer of information from memory and high energy consumption have emerged as issues that require solutions. Optical techniques are considered promising solutions to these problems with higher speed than their electronic counterparts and with reduced energy consumption. Here, we use the optical reservoir computing framework we have previously described (Scalable Optical Learning Operator or SOLO [1]) to program the spatial-spectral output of the light after nonlinear propagation in a multimode fiber. The novelty in the current paper is that the system is programmed through an output sampling scheme, similar to that used in hyperspectral imaging in astronomy. Linear and nonlinear computations are performed by light in the multimode fiber and the high dimensional spatial-spectral information at the fiber output is optically programmed before it reaches the camera. We then used a digital computer to classify the programmed output of the multi-mode fiber using a simple, single layer network. When combining front-end programming and the proposed spatial-spectral programming, we were able to achieve 89.9 % classification accuracy on the dataset consisting of chest X-ray images from COVID-19 patients. At the same time, we obtained a decrease of 99 % in the number of tunable parameters compared to an equivalently performing digital neural network. These results show that the performance of programmed SOLO is comparable with cutting-edge electronic computing platforms, albeit with a much-reduced number of electronic operations.

Multichannel meta-imagers for accelerating machine vision.

Rapid developments in machine vision technology have impacted a variety of applications, such as medical devices and autonomous driving systems. These achievements, however, typically necessitate digital neural networks with the downside of heavy computational requirements and consequent high energy consumption. As a result, real-time decision-making is hindered when computational resources are not readily accessible. Here we report a meta-imager designed to work together with a digital back end to offload computationally expensive convolution operations into high-speed, low-power optics. In this architecture, metasurfaces enable both angle and polarization multiplexing to create multiple information channels that perform positively and negatively valued convolution operations in a single shot. We use our meta-imager for object classification, achieving 98.6% accuracy in handwritten digits and 88.8% accuracy in fashion images. Owing to its compactness, high speed and low power consumption, our approach could find a wide range of applications in artificial intelligence and machine vision applications.

Conceptual foundations of a corporate intelligent risk-based system for analysis, prediction and prevention of occupational and work-related health disorders of workers

The relevance of the study is determined by the strategic interests of the Russian Federation dictating the need to achieve a national priority, which is the development of human potential. Given the ongoing reduction in labor force against a background of depopulation processes, finding a solution to the problem seems especially relevant. The purpose of this study is to develop a scientific conceptual framework for an intelligent risk-based innovative system for analysis, prediction and prevention of occupational and work-related health disorders of workers. This will help substantiate keynote sanitary-hygienic and medical-preventive measures aimed at reducing losses in healthy life expectancy and increasing occupational longevity. The main theoretical idea is based on a unique information-intellectual innovative system for analyzing and predicting cause-and-effect relationships between the effects of harmful and hazardous occupational factors and the resulting health problems of workers, occupational diseases included. The developed predictive digital neural network models, trained on retrospective or actual data on working conditions, health status, socioeconomic conditions and lifestyle factors, are the information and analytical basis for carrying out calculations and assessing the evolution of personal and group (occupation, age, and work records) health risks caused by occupational or work-related diseases in workers. This provides solid grounds for making a forecast of a prevented period of reduction in occupational working capacity associated with working conditions, specifically for each examined occupation, age, and work records. Such forecasts are an eligible information basis for developing and making well-grounded managerial decisions including those concerning sanitary, hygienic and medical preventive measures aimed at preserving occupational longevity. This will significantly increase effectiveness of corporate health-preserving policies.

Концептуальные основы корпоративной интеллектуальной риск-ориентированной системы анализа, прогноза и профилактики профессиональных и производственно-обусловленных нарушений здоровья работников

The relevance of the study is determined by the strategic interests of the Russian Federation dictating the need to achieve a national priority, which is the development of human potential. Given the ongoing reduction in labor force against a background of depopulation processes, finding a solution to the problem seems especially relevant. The purpose of this study is to develop a scientific conceptual framework for an intelligent risk-based innovative system for analysis, prediction and prevention of occupational and work-related health disorders of workers. This will help substantiate keynote sanitary-hygienic and medical-preventive measures aimed at reducing losses in healthy life expectancy and increasing occupational longevity. The main theoretical idea is based on a unique information-intellectual innovative system for analyzing and predicting cause-and-effect relationships between the effects of harmful and hazardous occupational factors and the resulting health problems of workers, occupational diseases included. The developed predictive digital neural network models, trained on retrospective or actual data on working conditions, health status, socioeconomic conditions and lifestyle factors, are the information and analytical basis for carrying out calculations and assessing the evolution of personal and group (occupation, age, and work records) health risks caused by occupational or work-related diseases in workers. This provides solid grounds for making a forecast of a prevented period of reduction in occupational working capacity associated with working conditions, specifically for each examined occupation, age, and work records. Such forecasts are an eligible information basis for developing and making well-grounded managerial decisions including those concerning sanitary, hygienic and medical preventive measures aimed at preserving occupational longevity. This will significantly increase effectiveness of corporate health-preserving policies.

Neuromorphic Computing via Fission-based Broadband Frequency Generation.

The performance limitations of traditional computer architectures have led to the rise of brain-inspired hardware, with optical solutions gaining popularity due to the energy efficiency, high speed, and scalability of linear operations. However, the use of optics to emulate the synaptic activity of neurons has remained a challenge since the integration of nonlinear nodes is power-hungry and, thus, hard to scale. Neuromorphic wave computing offers a new paradigm for energy-efficient information processing, building upon transient and passively nonlinear interactions between optical modes in a waveguide. Here, an implementation of this concept is presented using broadband frequency conversion by coherent higher-order soliton fission in a single-mode fiber. It is shown that phase encoding on femtosecond pulses at the input, alongside frequency selection and weighting at the system output, makes transient spectro-temporal system states interpretable and allows for the energy-efficient emulation of various digital neural networks. The experiments in a compact, fully fiber-integrated setup substantiate an anticipated enhancement in computational performance with increasing system nonlinearity. The findings suggest that broadband frequency generation, accessible on-chip and in-fiber with off-the-shelf components, may challenge the traditional approach to node-based brain-inspired hardware design, ultimately leading to energy-efficient, scalable, and dependable computing with minimal optical hardware requirements.

Nonlinear optical feature generator for machine learning

Modern machine learning models use an ever-increasing number of parameters to train (175 × 109 parameters for GPT-3) with large datasets to achieve better performance. Optical computing has been rediscovered as a potential solution for large-scale data processing, taking advantage of linear optical accelerators that perform operations at lower power consumption. However, to achieve efficient computing with light, it remains a challenge to create and control nonlinearity optically rather than electronically. In this study, a reservoir computing approach (RC) is investigated using a 14-mm waveguide in LiNbO3 on an insulator as an optical processor to validate the benefit of optical nonlinearity. Data are encoded on the spectrum of a femtosecond pulse, which is launched into the waveguide. The output of the waveguide is a nonlinear transform of the input, enabled by optical nonlinearities. We show experimentally that a simple digital linear classifier using the output spectrum of the waveguide increases the classification accuracy of several databases by ∼10% compared to untransformed data. In comparison, a digital neural network (NN) with tens of thousands of parameters was required to achieve similar accuracy. With the ability to reduce the number of parameters by a factor of at least 20, an integrated optical RC approach can attain a performance on a par with a digital NN.

Recognition of handwritten digital neural network construction and improvement

More and more neural network algorithms are being invented and improved, but the most basic algorithms are still useful in learning or practice areas. This essay is about building a digital recognition neural network from scratch, whose implementation has a completely original core code and details and precisely elaborates the whole process of basic neural network construction through detailed mathematical derivation, combined with the idea of programming. Furthermore, this project deeply researched and analyzed recognition accuracy according to the identified data of the neural network built for the project, then improved the algorithm used by increasing the accuracy from 86.93% to 99.1% and finally successfully explained the role of raw data preprocessing.

Multiplexed gradient descent: Fast online training of modern datasets on hardware neural networks without backpropagation

We present multiplexed gradient descent (MGD), a gradient descent framework designed to easily train analog or digital neural networks in hardware. MGD utilizes zero-order optimization techniques for online training of hardware neural networks. We demonstrate its ability to train neural networks on modern machine learning datasets, including CIFAR-10 and Fashion-MNIST, and compare its performance to backpropagation. Assuming realistic timescales and hardware parameters, our results indicate that these optimization techniques can train a network on emerging hardware platforms orders of magnitude faster than the wall-clock time of training via backpropagation on a standard GPU, even in the presence of imperfect weight updates or device-to-device variations in the hardware. We additionally describe how it can be applied to existing hardware as part of chip-in-the-loop training or integrated directly at the hardware level. Crucially, because the MGD framework is model-free it can be applied to nearly any hardware platform with tunable parameters, and its gradient descent process can be optimized to compensate for specific hardware limitations, such as slow parameter-update speeds or limited input bandwidth.

AnalogVNN: A fully modular framework for modeling and optimizing photonic neural networks

In this paper, we present AnalogVNN, a simulation framework built on PyTorch that can simulate the effects of optoelectronic noise, limited precision, and signal normalization present in photonic neural network accelerators. We use this framework to train and optimize linear and convolutional neural networks with up to nine layers and ∼1.7 × 106 parameters, while gaining insights into how normalization, activation function, reduced precision, and noise influence accuracy in analog photonic neural networks. By following the same layer structure design present in PyTorch, the AnalogVNN framework allows users to convert most digital neural network models to their analog counterparts with just a few lines of code, taking full advantage of the open-source optimization, deep learning, and GPU acceleration libraries available through PyTorch.

Optical neural network via loose neuron array and functional learning

This research proposes a deep-learning paradigm, termed functional learning (FL), to physically train a loose neuron array, a group of non-handcrafted, non-differentiable, and loosely connected physical neurons whose connections and gradients are beyond explicit expression. The paradigm targets training non-differentiable hardware, and therefore solves many interdisciplinary challenges at once: the precise modeling and control of high-dimensional systems, the on-site calibration of multimodal hardware imperfectness, and the end-to-end training of non-differentiable and modeless physical neurons through implicit gradient propagation. It offers a methodology to build hardware without handcrafted design, strict fabrication, and precise assembling, thus forging paths for hardware design, chip manufacturing, physical neuron training, and system control. In addition, the functional learning paradigm is numerically and physically verified with an original light field neural network (LFNN). It realizes a programmable incoherent optical neural network, a well-known challenge that delivers light-speed, high-bandwidth, and power-efficient neural network inference via processing parallel visible light signals in the free space. As a promising supplement to existing power- and bandwidth-constrained digital neural networks, light field neural network has various potential applications: brain-inspired optical computation, high-bandwidth power-efficient neural network inference, and light-speed programmable lens/displays/detectors that operate in visible light.

Unconventional Integrated Photonic Accelerators for High-Throughput Convolutional Neural Networks

We provide an overview of the rapidly evolving landscape of integrated photonic neuromorphic architectures, specifically targeting the implementation of convolutional neural networks. The exploding research momentum stems from the well-known advantages of photonic circuits compared to digital electronics, and at the same time, it is driven by the massive need for cognitive image/video processing. In this context, we provide a detailed literature review on photonic cores operating as convolutional neural networks, covering either the functionality of a conventional neural network or its spiking counterpart. Moreover, we propose 2 alternative photonic approaches that refrain from simply transferring neural network concepts directly into the optical domain; instead, they focus on fusing photonic, digital electronic, and event-based bioinspired processing to optimally exploit the virtues of each scheme. These approaches can offer beyond state-of-the-art performance while relying on realistic, scalable technology. The first approach is based on a photonic integrated platform and a bioinspired spectrum-slicing technique. The photonic chip allows feature extraction through optical filtering with low power consumption and an equivalent computational efficiency of 72 femtojoules per multiply-and-accumulate operation for 5-bit precision. When combined with typical digital neural networks, an almost 5-fold reduction in the number of parameters was achieved with a minor loss of accuracy compared to established convolutional neural networks. The second approach follows a bioisomorphic route in which miniaturized spiking laser neurons and unsupervised bioinspired training are unified in a deep architecture, revealing a noise-resilient and power-efficient proposition.

Digital computing through randomness and order in neural networks

We propose that coding and decoding in the brain are achieved through digital computation using three principles: relative ordinal coding of inputs, random connections between neurons, and belief voting. Due to randomization and despite the coarseness of the relative codes, we show that these principles are sufficient for coding and decoding sequences with error-free reconstruction. In particular, the number of neurons needed grows linearly with the size of the input repertoire growing exponentially. We illustrate our model by reconstructing sequences with repertoires on the order of a billion items. From this, we derive the Shannon equations for the capacity limit to learn and transfer information in the neural population, which is then generalized to any type of neural network. Following the maximum entropy principle of efficient coding, we show that random connections serve to decorrelate redundant information in incoming signals, creating more compact codes for neurons and therefore, conveying a larger amount of information. Henceforth, despite the unreliability of the relative codes, few neurons become necessary to discriminate the original signal without error. Finally, we discuss the significance of this digital computation model regarding neurobiological findings in the brain and more generally with artificial intelligence algorithms, with a view toward a neural information theory and the design of digital neural networks.

Meta-optic accelerators for object classifiers.

Rapid advances in deep learning have led to paradigm shifts in a number of fields, from medical image analysis to autonomous systems. These advances, however, have resulted in digital neural networks with large computational requirements, resulting in high energy consumption and limitations in real-time decision-making when computation resources are limited. Here, we demonstrate a meta-optic–based neural network accelerator that can off-load computationally expensive convolution operations into high-speed and low-power optics. In this architecture, metasurfaces enable both spatial multiplexing and additional information channels, such as polarization, in object classification. End-to-end design is used to co-optimize the optical and digital systems, resulting in a robust classifier that achieves 93.1% accurate classification of handwriting digits and 93.8% accuracy in classifying both the digit and its polarization state. This approach could enable compact, high-speed, and low-power image and information processing systems for a wide range of applications in machine vision and artificial intelligence.

Neural Schrödinger Equation: Physical Law as Deep Neural Network.

We show a new family of neural networks based on the Schrödinger equation (SE-NET). In this analogy, the trainable weights of the neural networks correspond to the physical quantities of the Schrödinger equation. These physical quantities can be trained using the complex-valued adjoint method. Since the propagation of the SE-NET can be described by the evolution of physical systems, its outputs can be computed by using a physical solver. The trained network is transferable to actual optical systems. As a demonstration, we implemented the SE-NET with the Crank-Nicolson finite difference method on Pytorch. From the results of numerical simulations, we found that the performance of the SE-NET becomes better when the SE-NET becomes wider and deeper. However, the training of the SE-NET was unstable due to gradient explosions when SE-NET becomes deeper. Therefore, we also introduced phase-only training, which only updates the phase of the potential field (refractive index) in the Schrödinger equation. This enables stable training even for the deep SE-NET model because the unitarity of the system is kept under the training. In addition, the SE-NET enables a joint optimization of physical structures and digital neural networks. As a demonstration, we performed a numerical demonstration of end-to-end machine learning (ML) with an optical frontend toward a compact spectrometer. Our results extend the application field of ML to hybrid physical-digital optimizations.

DNN-assisted phase distance tuned PSK modulation for PAM4-to-QPSK format conversion gateway node.

This paper proposes an optical gateway that converts pulse amplitude modulation (PAM) format to phase shift keying (PSK) modulation format, enabling flexible intensity-to-phase mapping without relying on optical-electrical-optical data conversion at heterogenous network connections. A proof-of-principle experiment shows that optically converted PSK signals from regular PAM signals will induce non-uniform irregular phase noise distortion. The proposed optical gateway is designed to provide an optimized phase distance for PSK signals such that an achievable information rate is maximized by a deep learning-based decision on the receiver side. The phase distance-tuned PSK signals offer performance improvement of about 4 dB gain at a target generalized mutual information by making use of a digital neural network recovery.