Machine Learning with Stochastic Magnetic Domain Wall Based Neurons and Synapses

Abstract

The bottleneck between data storage and processing on conventional computing hardware is seen as a significant limitation to many forms of machine learning algorithms [1]. One route to solve this issue is to employ physical devices that can naturally mimic brain-like behaviour, known as neuromorphic computing. Magnetic materials are already widely used for long-term data storage, but ongoing work seeks to make use of their potential as both working memory and computing architectures. Particularly, devices based on magnetic domain walls (DWs) can perform logic operations [2] and can readily store information [3]. However, the stochasticity of DW pinning limits the feasibility of creating technologically viable devices. Here, we demonstrate how stochasticity can be changed from a technologically inhibitive behaviour into an integral property of machine learning algorithms that could be used in specialised neuromorphic devices.We first present experimental measurements that demonstrate the feasibility of tuning stochastic processes by applying external stimuli. Focused magneto-optic Kerr effect measurements were used to probe the pinning of DWs at notch-shaped defect sites in 400 nm wide Permalloy nanowires (Fig. 1(a)). DWs were injected into the nanowires and propagated to the defect sites using a field parallel to the nanowire (Hx). The probability of the DWs being pinned at the defect sites was found to depend sigmoidally on the magnitude of a field transverse to the nanowire length (Hy), (Fig. 1(b)).We apply this stochastic behaviour in two possible contexts, as a stochastic neuron for a binary network or as a stochastic synapse for feed forward network, where we model the pinning behaviour based on the experimental results. In the first case, we have modelled these DW wires as neurons in a Boltzmann machine, which exploits their randomness in order to sample a defined, or trained, probability distribution. We apply this to the task of integer factorisation [4], where the connectivity of the neurons is defined so that the binary representation of the integer factors are the most probable states. Such a device inherently exploits the tunable stochasticity of the system to solve complex problems.In the second case, we have explored the properties of feedforward artificial neural networks where the nanowires acted as binary stochastic synapses (BSS). These synapses are individually random but the output can be improved by repeatedly sampling the network to include redundancy. We have developed both mean-field and stochastic based learning methods, where for low repeats of the synapse the stochastic learning is seen to outperform the mean-field based method. It is found that for a single cycle of the network it can be trained to recognise handwritten digits with error rate of 17%. Repeated sampling of the network decreased the error rate to 8% for 128 cycles - close to mean field performance (Fig. 2). Following this we have expanded our network to become a multi-layer perceptron with both inhibitory and excitatory synapses. The additional layer increases the pathways through the network thus reducing the impact of the stochasticity on the classification. For the same classification test the error rate is seen to reduce to 5.5% with a hidden layer of 500 neurons.In conclusion, the pinning of DWs in nanowires can be tuned by applying a magnetic field transverse to the wire and the resultant pinning probability has been used as both stochastic neurons and synapses. As a neuron, we have demonstrated its ability to perform integer factorisation as part of a Boltzmann machine. While as a stochastic synapse, we have been able to train single and multi-layer networks to classify handwritten digits with close to digital network performance. Our work illustrates how the intrinsic stochasticity of DW devices can be harnessed to provide tunable bespoke hardware for machine learning tasks. **