Design of nonlinear segmentation activation functions for object detection

Abstract

The existing activation functions ReLU, Tanh, and Mish have problems such as "neuronal death", offset, and poor robustness. Aiming at these problems, the XExp activation function is proposed by combining the advantages of ReLU, Swish, and Mish functions, and the problem of negative half-axis neuronal death is optimized by using the nonlinearity of non-RELU family functions and the non-zero characteristics of negative half-axis functions, and the soft saturation of negative semi-axis is retained. By designing the position of the origin of the function, the problem of positive half-axis offset in the Swish and Mish functions are solved. In terms of convergence speed, the MNIST dataset achieved 93.87% training accuracy during the first batch training on the newly proposed activation function XExp function, which was more than 85% higher in convergence speed compared with the Relu function; In terms of model convergence stability, compared with the accuracy of the Relu function, the XExp function can still achieve 98.05% accuracy when the number of convolutional layers is increased to 25 layers. The two data sets of CIFAR-10 and CIFAR-100 verify their versatility and practicality in the field of object detection.