Improving Classification Accuracy by Optimizing Activation Function for Convolutional Neural Network on Homomorphic Encryption

Abstract

AbstractA secure machine learning technology that performs prediction while encrypting data using homomorphic encryption is being developed. However, Convolutional Neural Networks (CNN) on homomorphic encryption cannot use general non-linear activation functions, Thus, the classification accuracy is low. We proposed a novel method to create an activation function that improves the classification accuracy by performing a pre-training optimization on the coefficients of the polynomial approximation of the Mish function. We confirmed the improvement of classification accuracy for MNIST, Fashion-MNIST, and CIFAR-10 by optimizing the Mish function through pre-training. The classification accuracy can be improved by 4.27% for CIFAR-10. Furthermore, we showed that classification accuracy improves for Fashion-MNIST and CIFAR-10 even when different networks and datasets are optimized by pre-training the activation function. These results show that the activation function of CNNs on a homomorphic encryption can be optimized to improve classification accuracy.