Better approximation of sigmoid function for privacy-preserving neural networks

Abstract

Abstract With the development of artificial intelligence technology, privacy protection has become a prominent research direction. Homomorphic encryption is an important and promising technology in the construction of privacy-preserving neural networks. However, computing nonlinear activation functions remains a challenging task in the context of homomorphic encryption. Considering Sigmoid is one of the most commonly used activation functions, this paper proposes a polynomial approximation method for the Sigmoid function and applies it to the homomorphic encryption scheme of CKKS. We design polynomial approximation functions using Improved Taylor Expansion Segmentation (ITES), Least Squares method (LS), and Random Noise Addition method, respectively. These methods are implemented through the CKKS scheme in the homomorphic encryption library of SEAL. The experimental results show that the ITES achieves high fitting accuracy, reducing the average absolute error by 11.09% and the mean squared error by 38.06% compared to previous fitting methods. Therefore, experiments have identified methods with better fitting results in the CKKS scheme, which can meet the requirements of using homomorphic encryption in deep learning technologies. Finally, our analysis points out that different technologies have unique advantages and adapt to different scenarios, e.g., ITES has higher accuracy, LS has higher performance in scenarios with limited computing resources.