Analog Hardware Research Articles

An ultra-low power adjustable current-mode analog integrated general purpose artificial neural network classifier

This study introduces a methodology tailored to analog hardware architecture for implementing an artificial neural network. The fundamental components of the architecture include current-mode circuits, representing the class, and a voltage-mode comparator. Specifically, the current mode circuits comprise the Mahalanobis distance circuit, Sigmoid function circuit, analog multiplier, and current mirrors. Regarding the voltage comparator, which receives the final decision, a folded-cascode operational amplifier is employed. The operational principles of the architecture are extensively explained and applied in a power-efficient configuration (operating under 976nW) with low power supply rails (0.6 V). The proposed implementation is tested on real-world biomedical classification tasks, achieving classification accuracy exceeding 91.6%. The designs are implemented using a 90nm CMOS process and developed using the Cadence IC Suite for both schematic and layout design. Monte-Carlo analysis, encompassing both process and mismatch, as well as corner analysis, are provided to confirm the robust characteristics of the proposed classifier. Through comparative analysis of post-layout simulation results with an equivalent software-based classifier and related literature, the proper operation of the proposed architecture is confirmed.

A blueprint for precise and fault-tolerant analog neural networks

Analog computing has reemerged as a promising avenue for accelerating deep neural networks (DNNs) to overcome the scalability challenges posed by traditional digital architectures. However, achieving high precision using analog technologies is challenging, as high-precision data converters are costly and impractical. In this work, we address this challenge by using the residue number system (RNS) and composing high-precision operations from multiple low-precision operations, thereby eliminating the need for high-precision data converters and information loss. Our study demonstrates that the RNS-based approach can achieve ≥99% FP32 accuracy with 6-bit integer arithmetic for DNN inference and 7-bit for DNN training. The reduced precision requirements imply that using RNS can achieve several orders of magnitude higher energy efficiency while maintaining the same throughput compared to conventional analog hardware with the same precision. We also present a fault-tolerant dataflow using redundant RNS to protect the computation against noise and errors inherent within analog hardware.

DenRAM: neuromorphic dendritic architecture with RRAM for efficient temporal processing with delays

Neuroscience findings emphasize the role of dendritic branching in neocortical pyramidal neurons for non-linear computations and signal processing. Dendritic branches facilitate temporal feature detection via synaptic delays that enable coincidence detection (CD) mechanisms. Spiking neural networks highlight the significance of delays for spatio-temporal pattern recognition in feed-forward networks, eliminating the need for recurrent structures. Here, we introduce DenRAM, a novel analog electronic feed-forward spiking neural network with dendritic compartments. Utilizing 130 nm technology integrated with resistive RAM (RRAM), DenRAM incorporates both delays and synaptic weights. By configuring RRAMs to emulate bio-realistic delays and exploiting their heterogeneity, DenRAM mimics synaptic delays and efficiently performs CD for pattern recognition. Hardware-aware simulations on temporal benchmarks show DenRAM’s robustness against hardware noise, and its higher accuracy over recurrent networks. DenRAM advances temporal processing in neuromorphic computing, optimizes memory usage, and marks progress in low-power, real-time signal processing

A fractional-order hyperchaotic system that is period in integer-order case and its application in a novel high-quality color image encryption algorithm

A new fractional-order 5D hyperchaotic system based on memristor is constructed in this paper, with the speciality that the system exists chaotic and hyperchaotic states in the fractional-order case, while in periodic state in the integer-order. In addition, it has a variety of special phenomena at fractional-order such as infinite initial value range, parameter-dependent offset-boosting and amplitude control, attractor coexistence, and fractional order complexity greater than integer order. The correctness and feasibility of the system is verified by analog circuit simulation and hardware circuit implementation. Combining this system with image encryption algorithms, two new scrambling algorithms and a diffusion algorithm are proposed. And a high-quality encryption scheme that can be applied to a wide range of color images is proposed. The scheme is found to have excellent security after verification by various security analyses and comparison with other literatures. This paper provides a basis for the superiority of fractional-order chaotic systems and provides new methods in the field of image encryption.

Gradient-descent hardware-aware training and deployment for mixed-signal neuromorphic processors

Mixed-signal neuromorphic processors provide extremely low-power operation for edge inference workloads, taking advantage of sparse asynchronous computation within spiking neural networks (SNNs). However, deploying robust applications to these devices is complicated by limited controllability over analog hardware parameters, as well as unintended parameter and dynamical variations of analog circuits due to fabrication non-idealities. Here we demonstrate a novel methodology for offline training and deployment of SNNs to the mixed-signal neuromorphic processor DYNAP-SE2. Our methodology applies gradient-based training to a differentiable simulation of the mixed-signal device, coupled with an unsupervised weight quantization method to optimize the network’s parameters. Parameter noise injection during training provides robustness to the effects of quantization and device mismatch, making the method a promising candidate for real-world applications under hardware constraints and non-idealities. This work extends Rockpool, an open-source deep-learning library for SNNs, with support for accurate simulation of mixed-signal SNN dynamics. Our approach simplifies the development and deployment process for the neuromorphic community, making mixed-signal neuromorphic processors more accessible to researchers and developers.

Training self-learning circuits for power-efficient solutions

As the size and ubiquity of artificial intelligence and computational machine learning models grow, the energy required to train and use them is rapidly becoming economically and environmentally unsustainable. Recent laboratory prototypes of self-learning electronic circuits, such as “physical learning machines,” open the door to analog hardware that directly employs physics to learn desired functions from examples at a low energy cost. In this work, we show that this hardware platform allows for an even further reduction in energy consumption by using good initial conditions and a new learning algorithm. Using analytical calculations, simulations, and experiments, we show that a trade-off emerges when learning dynamics attempt to minimize both the error and the power consumption of the solution—greater power reductions can be achieved at the cost of decreasing solution accuracy. Finally, we demonstrate a practical procedure to weigh the relative importance of error and power minimization, improving the power efficiency given a specific tolerance to error.

Comparison of Bagging and Sparcity Methods for Connectivity Reduction in Spiking Neural Networks with Memristive Plasticity

Developing a spiking neural network architecture that could prospectively be trained on energy-efficient neuromorphic hardware to solve various data analysis tasks requires satisfying the limitations of prospective analog or digital hardware, i.e., local learning and limited numbers of connections, respectively. In this work, we compare two methods of connectivity reduction that are applicable to spiking networks with local plasticity; instead of a large fully-connected network (which is used as the baseline for comparison), we employ either an ensemble of independent small networks or a network with probabilistic sparse connectivity. We evaluate both of these methods with a three-layer spiking neural network, which are applied to handwritten and spoken digit classification tasks using two memristive plasticity models and the classical spike time-dependent plasticity (STDP) rule. Both methods achieve an F1-score of 0.93–0.95 on the handwritten digits recognition task and 0.85–0.93 on the spoken digits recognition task. Applying a combination of both methods made it possible to obtain highly accurate models while reducing the number of connections by more than three times compared to the basic model.

Periodic and chaotic spiking behaviors in a simplified memristive Hodgkin-Huxley circuit

The famous Hodgkin-Huxley circuit contains two time-varying resistors to describe the electrophysiological characteristics of sodium and potassium ion channels. But the time-varying resistors are expressed by mixed exponential equations, which are unavailable to directly implement in analog circuit and hinders hardware application of the Hodgkin-Huxley circuit. To hit this issue, a simplified memristive Hodgkin-Huxley (m-HH) circuit is proposed, which only involves two locally active memristors (LAMs) to depict the sodium and potassium ion channels, a capacitor to describe the neuron membrane, a DC current to represent external stimulus, and two DC voltages to express the reversal potentials. MATLAB-based numerical simulations and analog circuit-based hardware measurements display that the simplified m-HH circuit can generate memristor parameter- and DC current-related periodic spiking behaviors with different periodicities and chaotic spiking behavior. This delights that electrophysiological characteristics of ion channels and external stimulus can be employed for regulating these periodic and chaotic spiking behaviors. It is interesting that the simplified m-HH circuit can generate frequency self-adaptation and coexisting firing patterns. The inter-spike interval bifurcation diagram shows that the frequency of the periodic spiking behavior increases with the increase of externally applied DC current. Besides, the hybrid parameter bifurcation diagram displays that the simplified m-HH circuit can generate memristor initial state-related coexisting firing patterns of period-6 with chaos, period-3 with chaos, and period-3 with period-8. These offer us the unique insight into explaining some biological firing patterns. The most significance is that the analog hardware circuit is feasible in reproducing these periodic and chaotic spiking behaviors and benefits for developing spiking-based neuromorphic hardware.

A Radar-Based System for Detection of Human Fall Utilizing Analog Hardware Architectures of Decision Tree Model

A Radar-Based System for Detection of Human Fall Utilizing Analog Hardware Architectures of Decision Tree Model

Using the IBM analog in-memory hardware acceleration kit for neural network training and inference

Analog In-Memory Computing (AIMC) is a promising approach to reduce the latency and energy consumption of Deep Neural Network (DNN) inference and training. However, the noisy and non-linear device characteristics and the non-ideal peripheral circuitry in AIMC chips require adapting DNNs to be deployed on such hardware to achieve equivalent accuracy to digital computing. In this Tutorial, we provide a deep dive into how such adaptations can be achieved and evaluated using the recently released IBM Analog Hardware Acceleration Kit (AIHWKit), freely available at https://github.com/IBM/aihwkit. AIHWKit is a Python library that simulates inference and training of DNNs using AIMC. We present an in-depth description of the AIHWKit design, functionality, and best practices to properly perform inference and training. We also present an overview of the Analog AI Cloud Composer, a platform that provides the benefits of using the AIHWKit simulation in a fully managed cloud setting along with physical AIMC hardware access, freely available at https://aihw-composer.draco.res.ibm.com. Finally, we show examples of how users can expand and customize AIHWKit for their own needs. This Tutorial is accompanied by comprehensive Jupyter Notebook code examples that can be run using AIHWKit, which can be downloaded from https://github.com/IBM/aihwkit/tree/master/notebooks/tutorial.

A Low-Power Analog Cell for Implementing Spiking Neural Networks in 65 nm CMOS

A Spiking Neural Network (SNN) is realized within a 65 nm CMOS process to demonstrate the feasibility of its constituent cells. Analog hardware neural networks have shown improved energy efficiency in edge computing for real-time-inference applications, such as speech recognition. The proposed network uses a leaky integrate and fire neuron scheme for computation, interleaved with a Spike Timing Dependent Plasticity (STDP) circuit for implementing synaptic-like weights. The low-power, asynchronous analog neurons and synapses are tailored for the VLSI environment needed to effectively make use of hardware SSN systems. To demonstrate functionality, a feed-forward Spiking Neural Network composed of two layers, the first with ten neurons and the second with six, is implemented. The neuron design operates with 2.1 pJ of power per spike and 20 pJ per synaptic operation.

Forward-forward training of an optical neural network.

Neural networks (NNs) have demonstrated remarkable capabilities in various tasks, but their computation-intensive nature demands faster and more energy-efficient hardware implementations. Optics-based platforms, using technologies such as silicon photonics and spatial light modulators, offer promising avenues for achieving this goal. However, training multiple programmable layers together with these physical systems poses challenges, as they are difficult to fully characterize and describe with differentiable functions, hindering the use of error backpropagation algorithm. The recently introduced forward-forward algorithm (FFA) eliminates the need for perfect characterization of the physical learning system and shows promise for efficient training with large numbers of programmable parameters. The FFA does not require backpropagating an error signal to update the weights, rather the weights are updated by only sending information in one direction. The local loss function for each set of trainable weights enables low-power analog hardware implementations without resorting to metaheuristic algorithms or reinforcement learning. In this paper, we present an experiment utilizing multimode nonlinear wave propagation in an optical fiber demonstrating the feasibility of the FFA approach using an optical system. The results show that incorporating optical transforms in multilayer NN architectures trained with the FFA can lead to performance improvements, even with a relatively small number of trainable weights. The proposed method offers a new path to the challenge of training optical NNs and provides insights into leveraging physical transformations for enhancing the NN performance.

Material to system-level benchmarking of CMOS-integrated RRAM with ultra-fast switching for low power on-chip learning

Analog hardware-based training provides a promising solution to developing state-of-the-art power-hungry artificial intelligence models. Non-volatile memory hardware such as resistive random access memory (RRAM) has the potential to provide a low power alternative. The training accuracy of analog hardware depends on RRAM switching properties including the number of discrete conductance states and conductance variability. Furthermore, the overall power consumption of the system inversely correlates with the RRAM devices conductance. To study material dependence of these properties, TaOx and HfOx RRAM devices in one-transistor one-RRAM configuration (1T1R) were fabricated using a custom 65 nm CMOS fabrication process. Analog switching performance was studied with a range of initial forming compliance current (200–500 µA) and analog switching tests with ultra-short pulse width (300 ps) was carried out. We report that by utilizing low current during electroforming and high compliance current during analog switching, a large number of RRAM conductance states can be achieved while maintaining low conductance state. While both TaOx and HfOx could be switched to more than 20 distinct states, TaOx devices exhibited 10× lower conductance, which reduces total power consumption for array-level operations. Furthermore, we adopted an analog, fully in-memory training algorithm for system-level training accuracy benchmarking and showed that implementing TaOx 1T1R cells could yield an accuracy of up to 96.4% compared to 97% for the floating-point arithmetic baseline, while implementing HfOx devices would yield a maximum accuracy of 90.5%. Our experimental work and benchmarking approach paves the path for future materials engineering in analog-AI hardware for a low-power environment training.

Programmable Energy-Efficient Analog Multilayer Perceptron Architecture Suitable for Future Expansion to Hardware Accelerators

A programmable, energy-efficient analog hardware implementation of a multilayer perceptron (MLP) is presented featuring a highly programmable system that offers the user the capability to create an MLP neural network hardware design within the available framework. In addition to programmability, this implementation provides energy-efficient operation via analog/mixed-signal design. The configurable system is made up of 12 neurons and is fabricated in a standard 130 nm CMOS process occupying approximately 1 mm2 of on-chip area. The system architecture is analyzed in several different configurations with each achieving a power efficiency of greater than 1 tera-operations per watt. This work offers an energy-efficient and scalable alternative to digital configurable neural networks that can be built upon to create larger networks capable of standard machine learning applications, such as image and text classification. This research details a programmable hardware implementation of an MLP that achieves a peak power efficiency of 5.23 tera-operations per watt while consuming considerably less power than comparable digital and analog designs. This paper describes circuit elements that can readily be scaled up at the system level to create a larger neural network architecture capable of improved energy efficiency.

Transferable learning on analog hardware.

While analog neural network (NN) accelerators promise massive energy and time savings, an important challenge is to make them robust to static fabrication error. Present-day training methods for programmable photonic interferometer circuits, a leading analog NN platform, do not produce networks that perform well in the presence of static hardware errors. Moreover, existing hardware error correction techniques either require individual retraining of every analog NN (which is impractical in an edge setting with millions of devices), place stringent demands on component quality, or introduce hardware overhead. We solve all three problems by introducing one-time error-aware training techniques that produce robust NNs that match the performance of ideal hardware and can be exactly transferred to arbitrary highly faulty photonic NNs with hardware errors up to five times larger than present-day fabrication tolerances.

Spiking Neural Network Nonlinear Demapping on Neuromorphic Hardware for IM/DD Optical Communication

Neuromorphic computing implementing spiking neural networks (SNN) is a promising technology for reducing the footprint of optical transceivers, as required by the fast-paced growth of data center traffic. In this work, an SNN nonlinear demapper is designed and evaluated on a simulated intensity-modulation direct-detection link with chromatic dispersion. The SNN demapper is implemented in software and on the analog neuromorphic hardware system BrainScaleS-2 (BSS-2). For comparison, linear equalization (LE), Volterra nonlinear equalization (VNLE), and nonlinear demapping by an artificial neural network (ANN) implemented in software are considered. At a pre-forward error correction bit error rate of exponent-product2e-3, the software SNN outperforms LE by 1.5 dB, VNLE by 0.3 dB and the ANN by 0.5 dB. The hardware penalty of the SNN on BSS-2 is only 0.2 dB, i.e., also on hardware, the SNN performs better than all software implementations of the reference approaches. Hence, this work demonstrates that SNN demappers implemented on electrical analog hardware can realize powerful and accurate signal processing fulfilling the strict requirements of optical communications.

A 4D chaotic system with four-wing attractors and hidden extreme multistability

In this paper, a novel four-wing chaotic system is constructed based on the Sprott-A system. The novel system contains three nonlinearly quadratic terms, which have rich dynamics such as four-wing hidden attractors, hidden extreme multistability and transient transitions. It is found that this system has high complexity by spectral entropy analysis. In addition, a corresponding hardware analog circuit is designed based on this system by operational amplifiers and multipliers. The experimental results agree with those of the theoretical analysis and confirm that this system is practically feasible. This system will have a good application value in secure communication and cryptography.

Accelerated quantum Monte Carlo with probabilistic computers

Quantum Monte Carlo (QMC) techniques are widely used in a variety of scientific problems and much work has been dedicated to developing optimized algorithms that can accelerate QMC on standard processors (CPU). With the advent of various special purpose devices and domain specific hardware, it has become increasingly important to establish clear benchmarks of what improvements these technologies offer compared to existing technologies. In this paper, we demonstrate 2 to 3 orders of magnitude acceleration of a standard QMC algorithm using a specially designed digital processor, and a further 2 to 3 orders of magnitude by mapping it to a clockless analog processor. Our demonstration provides a roadmap for 5 to 6 orders of magnitude acceleration for a transverse field Ising model (TFIM) and could possibly be extended to other QMC models as well. The clockless analog hardware can be viewed as the classical counterpart of the quantum annealer and provides performance within a factor of < 10 of the latter. The convergence time for the clockless analog hardware scales with the number of qubits as ∼ N, improving the ∼ N2 scaling for CPU implementations, but appears worse than that reported for quantum annealers by D-Wave.

Enhancing the Dynamic Range of OFDM Radars Using Non-Linear Operation and Symbol-Based Equalization

Due to the increased flexibility in terms of waveform generation, digitally-modulated radar systems are gaining increasing popularity among research and academia. The ability to dynamically allocate frequency and time slots, as well as to employ different coding, enables novel multiplexing techniques for multiple-input-multiple-output radars and sensor networks, compared to state-of-the-art chirp-modulated systems. Another significant benefit is the possibility for joint radar and communication with the same hardware. A popular modulation for digital radar is orthogonal frequency-division multiplexing (OFDM), due to its well-known properties from communication applications. It offers all the above-mentioned possibilities, however OFDM radar has two main drawbacks: high peak-to-average power ratio (PAPR) due to the varying signal envelope which significantly increases the linearity requirements on the analog hardware and reduces the power efficiency, as well as a decoupling of target range and frequency-shift, which eliminates the possibility of using a high-pass filter to suppress close targets and thus decreases the sensor's dynamic range in comparison to conventional frequency-modulated continuous-wave (FMCW). The latter weakness is sometimes also viewed as a positive, due to the fact that range-frequency coupling can lead to undesired effects in some cases. This work addresses both of these problems together, which exist in every real-world OFDM radar sensor, and presents detailed analysis, simulation, characterization and measurement validation results that show how these effects negatively influence the signal quality and the obtained range-Doppler map of an automotive radar. Furthermore, a non-linear operation mode with receiver-sided healing that does not require any additional processing steps or computation power during radar operation is proposed. Its high potential is verified by measurements with a 78 GHz prototype setup in an anechoic chamber with static reflectors, as well as an automotive radar echo generator for dynamic targets and extended ranges.

Time-shift selection for reservoir computing using a rank-revealing QR algorithm.

Reservoir computing, a recurrent neural network paradigm in which only the output layer is trained, has demonstrated remarkable performance on tasks such as prediction and control of nonlinear systems. Recently, it was demonstrated that adding time-shifts to the signals generated by a reservoir can provide large improvements in performance accuracy. In this work, we present a technique to choose the time-shifts by maximizing the rank of the reservoir matrix using a rank-revealing QR algorithm. This technique, which is not task dependent, does not require a model of the system and, therefore, is directly applicable to analog hardware reservoir computers. We demonstrate our time-shift selection technique on two types of reservoir computer: an optoelectronic reservoir computer and the traditional recurrent network with a t a n h activation function. We find that our technique provides improved accuracy over random time-shift selection in essentially all cases.