Evaluation of the effects of K-Means Clustered-Based Weight Quantization on a Keras Library Based Convolutional Neural Network for Hand Written Digit Image Recognition

Abstract

A lot of Convolutional Neural Networks (CNNs) have been implemented using FPGAs for the past years. Subsequently, memory saving features were added to the CNN through weight quantization using K-means clustering. A future goal on an ASIC design, involving CNN and weight quantization working together in one chip, can give way to an automated procedure of memory-saving CNN design. In this paper an evaluation was done on the effect of quantizing the weights of a Keras library-based CNN using K means clustering. Various values of K in K-means clustering were tested to see its effects on the CNN accuracy performance. This paper presents first the design approach of a Keras library based Convolutional Neural Network (CNN) for hand-written digit images. It then presents a hardware model design of K-Means clustering algorithm using VHDL. The performance of CNN for image recognition was then tested for various levels of weight quantization using K-means clustering algorithm. Simulation results showed a compression of weights as high as 60% resulted to less than 1% reduction in CNN’s accuracy. The findings in this paper will serve as guide in determining the relevant values of K i.e. the compression ratio, for future ASIC design on this topic.