Adaptive Weight Quantization Method for Nonlinear Synaptic Devices

Abstract

In this paper, we propose an adaptive quantization method that can easily transfer the weights, which are trained in software network with floating point operation, to the real synaptic devices in hardware-based neural networks and maintain high performance. An n-type gated Schottky diode is investigated as a synaptic device, and the conductance behavior of this device is modeled successfully. Max value normalization and $3\sigma $ normalization are applied to the weights trained with an accuracy of 98.29% on fully connected neural network ( $784\times 256\times10$ ) using software network. Then, the weights are quantized using the adaptive quantization method and can be transferred by adjusting the number of identical pulses applied to the synaptic devices. After applying the adaptive quantization method, accuracy rates of 98.09% and 97.20% in MNIST classification are obtained for both max value normalization and $3\sigma $ normalization, respectively. The proposed quantization method works well even when there is nonideality of synaptic devices such as nonlinearity of conductance behavior, limited conductance levels, and variation of conductance.