Networks For Pattern Classification Problems Research Articles

Pattern Classification Based on RBF Networks with Self-Constructing Clustering and Hybrid Learning

Radial basis function (RBF) networks are widely adopted to solve problems in the field of pattern classification. However, in the construction phase of such networks, there are several issues encountered, such as the determination of the number of nodes in the hidden layer, the form and initialization of the basis functions, and the learning of the parameters involved in the networks. In this paper, we present a novel approach for constructing RBF networks for pattern classification problems. An iterative self-constructing clustering algorithm is used to produce a desired number of clusters from the training data. Accordingly, the number of nodes in the hidden layer is determined. Basis functions are then formed, and their centers and deviations are initialized to be the centers and deviations of the corresponding clusters. Then, the parameters of the network are refined with a hybrid learning strategy, involving hyperbolic tangent sigmoid functions, steepest descent backpropagation, and least squares method. As a result, optimized RBF networks are obtained. With this approach, the number of nodes in the hidden layer is determined and basis functions are derived automatically, and higher classification rates can be achieved. Furthermore, the approach is applicable to construct RBF networks for solving both single-label and multi-label pattern classification problems.

A comparative study of general fuzzy min-max neural networks for pattern classification problems

General fuzzy min-max (GFMM) neural network is a generalization of fuzzy neural networks formed by hyperbox fuzzy sets for classification and clustering problems. Two principle algorithms are deployed to train this type of neural network, i.e., incremental learning and agglomerative learning. This paper presents a comprehensive empirical study of performance influencing factors, advantages, and drawbacks of the general fuzzy min-max neural network on pattern classification problems. The subjects of this study include (1) the impact of maximum hyperbox size, (2) the influence of the similarity threshold and measures on the agglomerative learning algorithm, (3) the effect of data presentation order, (4) comparative performance evaluation of the GFMM with other types of fuzzy min-max neural networks and prevalent machine learning algorithms. The experimental results on benchmark datasets widely used in machine learning showed overall strong and weak points of the GFMM classifier. These outcomes also informed potential research directions for this class of machine learning algorithms in the future.

Studies on Initialization for Multilayer Networks

This paper proposes an initialization of back propagation (BP) networks for pattern classification problems; the weights of hidden units are initialized so that hyperplanes should pass through the center of input pattern set, and those of the output layer are initialized to zero. Several simulation results confirm that the proposed initialization gives better convergence than the ordinary initialization that all the weights are initialized by uniform random values with zero mean.

A Study on the Relationship between Internal Representation and Generalization Ability of Multi-Layered Neural Networks for Pattern Classification Problems

A Study on the Relationship between Internal Representation and Generalization Ability of Multi-Layered Neural Networks for Pattern Classification Problems

Constructive neural networks as estimators of bayesian discriminant functions

Of crucial importance to the successful use of artificial neural networks for pattern classification problems is how the appropriate network size can be automatically determined. This issue is addressed by formulating the process as an automatic search in the space of functions that corresponds to a subclass of multilayer feedforward networks. Learning is thus a dynamic network construction process which involves adjusting both the network weights and the topology. Adding new hidden units corresponds to extracting higher-level features from the original input features for reducing the residual classification errors. It can be shown that the resultant network approximates a Bayesian classifier that implements the Bayesian decision rule for classification. The empirical results of several pattern classification experiments are also reported.