Study on the influence of different number of neuron layers and activation functions on neural networks

Abstract

Nowadays, the gradual development of neural network has reached a certain level, and the constantly updated and improved architecture has been applied in many interdisciplinary fields. It is well known that the performance of a neural network often depends on the depth of the network and the activation function used. In order to further explore the influence of neural network depth and activation function on network performance, the author constructs a small neural network to test the performance of a neural network with different depths and different activation functions. This neural network is used to solve the problem of multi-classification of iris. It adopts the dataset provided by TensorFlow to judge the categories of iris according to four features. The author observed the accuracy of iris classification by changing the depth and activation function of the network, and finally analyzed the possible influence of the network depth and activation function on the network through the accuracy. After ten tests respectively (the network adopts sigmoid as activation function), the average accuracy of single-layer neural network is 0.965, the average accuracy of two-layer neural network is 0.961, and the average accuracy of three-layer neural network is 0.958. The average accuracy of two-layer neural network with the hyperbolic tangent as its activation function is 0.960. Through analyzing these results and some reasoning, the author is able to draw a conclusion on the corresponding influence exerted on the network, which would simplify the esoteric process of the construction of a neural network.