Wide Residual Lightweight Network Using Simplified Adaptive Parameter Rectifying Units

Abstract

Aiming at the problems of complex network structure, long training time and insufficient feature learning ability for deep learning, a lightweight network structure is designed. A kind of new activation function (namely rectifying linear unit) whose adaptive parameter is achieved by simplified training is proposed. The activation function is inserted into convolutional neural network to improve the feature learning ability by making each input signal has its own set of nonlinear transformation. Compared with traditional convolutional neural network, the number of network parameters is reduced by 51.61%, while the structure remains the ability of feature extraction before simplification. The proposed network structure can greatly reduce the network training time and improve the target recognition speed. The experiments on CIFAR-10 and CIFAR-100 datasets respectively show that the accuracies reach 95.26% and 76.54%, which are 1.67% and 3.76% higher than those of the traditional convolutional neural network.