Representing bounded fault classes using neural networks with ellipsoidal activation functions

Abstract

Feedforward neural networks are characterized by the choice of the squashing function, the activation function and the error criterion function. The standard feedforward network considers a sigmoid squashing function, linear activation function and RMS error minimization as the error criterion. These networks exhibit generalization problems because the linear discriminant function creates half spaces of high activation. For fault diagnosis, we are specifically interested in classifiers that generate bounded regions. To address this and related issues, we propose two ideas in this paper: (i) the use of an ellipsoidal activation function; and (ii) an algorithm for generating hidden nodes and hence the network structure adaptively. Ellipsoidal activation leads to the generation of bounded ellipsoidal regions in the space of the network inputs and is thus suitable for fault classification. The standard backpropagation algorithm does not guarantee the proper determination of the ellipsoidal activation functions as it may run into local minima problems, attributable mainly to poor choices of initial network weights. The notion of a bounding ellisoid is introduced to determine proper initial weights and also avoid unnecessarily large ellipsoids. A new algorithm for determining the number of hidden nodes adaptively is also proposed. New hidden nodes are added in regions of input space where the representation is not adequate and nodes are stripped away when they do not contribute to proper classification. This removes the arbitrary nature of assigning hidden nodes to the network as practiced in the standard approaches. The success of the proposed framework is demonstrated on a variety of examples including adjacent distributions with nonlinear boundaries. The diagnosis of a reactor—distillation column system is presented to show the suitability of the ellipsoidal representation by analyzing the structure of the fault space. The advantages of ellipsoidal units over radially symmetric units is discussed in this context.