A max-piecewise-linear neural network for function approximation

Abstract

This paper proposes a Max-Piecewise-Linear (MPWL) Neural Network for function approximation. The MPWL network consists of a single hidden layer and employs the Piecewise-Linear (PWL) Basis Functions as the activation functions of hidden neurons. Since a PWL Basis Function possesses a simple functional form and universal representation capability, the MPWL network achieves a good balance between the computational simplicity and approximation accuracy. In addition, a PWL version of Back-Propagation (PBP) algorithm is developed, whose computational complexity is lower than the training algorithms for the Canonical PWL network, and the Back-Propagation algorithm for the sigmoid network with same number of training cycles. Another advantage of the MPWL network is its amenability to hardware implementation. This facilitates many applications such as nonlinear circuit synthesis, dynamic identification and control.