An adaptive training method for optimal interpolative neural nets.

Abstract

In contrast to conventional multilayered feedforward networks which are typically trained by iterative gradient search methods, an optimal interpolative (OI) net can be trained by a noniterative least squares algorithm called RLS-OI. The basic idea of RLS-OI is to use a subset of the training set, whose inputs are called subprototypes, to constrain the OI net solution. A subset of these subprototypes, called prototypes, is then chosen as the parameter vectors of the activation functions of the OI net to satisfy the subprototype constraints in the least squares (LS) sense. By dynamically increasing the numbers of subprototypes and prototypes, RLS-OI evolves the OI net from scratch to the extent sufficient to solve a given classification problem. To improve the performance of RLS-OI, this paper addresses two important problems in OI net training: the selection of the subprototypes and the selection of the prototypes. By choosing subprototypes from poorly classified regions, this paper proposes a new subprototype selection method which is adaptive to the changing classification performance of the growing OI net. This paper also proposes a new prototype selection criterion to reduce the complexity of the OI net. For the same training accuracy, simulation results demonstrate that the proposed approach produces smaller OI net than the RLS-OI algorithm. Experimental results also show that the proposed approach is less sensitive to the variation of the training set than RLS-OI.