Abstract
Computing-In-Memory (CIM), based on non-von Neumann architecture, has lately received significant attention due to its lower overhead in delay and higher energy efficiency in convolutional and fully-connected neural network computing. Growing works have given the priority to researching the array of memory and peripheral circuits to achieve multiply-and-accumulate (MAC) operation, but not enough attention has been paid to the high-precision hardware implementation of non-linear layers up to now, which still causes time overhead and power consumption. Sigmoid is a widely used non-linear activation function and most of its studies provided an approximation of the function expression rather than totally matched, inevitably leading to considerable error. To address this issue, we propose a high-precision circuit implementation of the sigmoid, matching the expression exactly for the first time. The simulation results with the SMIC 40 nm process suggest that the proposed circuit implemented high-precision sigmoid perfectly achieves the properties of the ideal sigmoid, showing the maximum error and average error between the proposed simulated sigmoid and ideal sigmoid is 2.74% and 0.21%, respectively. In addition, a multi-layer convolutional neural network based on CIM architecture employing the simulated high-precision sigmoid activation function verifies the similar recognition accuracy on the test database of handwritten digits compared to utilize the ideal sigmoid in software, with online training achieving 97.06% and with offline training achieving 97.74%.
Highlights
Convolutional Neural Network (CNN) [1] has shown a satisfying performance on recognition/classification tasks owing to its characteristics of weight sharing, multi-core convolution, and local perception [2]
We demonstrated features of the sigmoid function with adjustable slope by simulating the high-precision sigmoid circuit with the SMIC 40 nm process, achieving the output value close to the ideal sigmoid, which means the simulated sigmoid function is high-precision, so it will theoretically not reduce the classification accuracy of the neural network
Compared to prior CIM works of the sigmoid activation function implemented on software, our approach can dramatically reduce data transfer back and forth and latency by putting the circuit of sigmoid into CIM architecture
Summary
Convolutional Neural Network (CNN) [1] has shown a satisfying performance on recognition/classification tasks owing to its characteristics of weight sharing, multi-core convolution, and local perception [2]. LeNet-5, which consists of two convolutional layers (C1, C2), two max-pooling layers (S1, S2), and three fully connected layers (F1, F2, F3). CNN plays a crucial role in the artificial intelligence (AI) world, but a massive amount of data transfers back and forth between CPU and memory causes high power consumption for conventional all-digital implemented CNN computation, which is called “memory bottleneck” [3,4,5] resulting from von-Neumann computing architecture. The Computing-In-Memory (CIM) [4,5,6,7]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
