Differential Competitive Learning Research Articles

Noise-enhanced clustering and competitive learning algorithms

Noise can provably speed up convergence in many centroid-based clustering algorithms. This includes the popular k-means clustering algorithm. The clustering noise benefit follows from the general noise benefit for the expectation–maximization algorithm because many clustering algorithms are special cases of the expectation–maximization algorithm. Simulations show that noise also speeds up convergence in stochastic unsupervised competitive learning, supervised competitive learning, and differential competitive learning.

Centroid neural network adaptive resonance theory for vector quantization

In this paper, a novel unsupervised competitive learning algorithm, called the centroid neural network adaptive resonance theory (CNN-ART) algorithm, is proposed to relieve the dependence on the initial codewords of the codebook in contrast to the conventional algorithms with vector quantization in lossy image compression. The design of the CNN-ART algorithm is mainly based on the adaptive resonance theory structure, and then a gradient-descent-based learning rule is derived so that the CNN-ART algorithm does not require a predetermined schedule for learning rate. Furthermore, the appropriate initial weights obtained by the CNN-ART algorithm can be applied as an initial codebook for the Linde–Buzo–Gray (LBG) algorithm such that the compression performance can be greatly improved. In this paper, the extensive simulations demonstrate that the CNN-ART algorithm does outperform other algorithms like LBG, self-organizing feature map and differential competitive learning.

Centroid neural network for unsupervised competitive learning

An unsupervised competitive learning algorithm based on the classical -means clustering algorithm is proposed. The proposed learning algorithm called the centroid neural network (CNN) estimates centroids of the related cluster groups in training date. This paper also explains algorithmic relationships among the CNN and some of the conventional unsupervised competitive learning algorithms including Kohonen's self-organizing map (SOM) and Kosko's differential competitive learning (DCL) algorithm. The CNN algorithm requires neither a predetermined schedule for learning coefficient nor a total number of iterations for clustering. The simulation results on clustering problems and image compression problems show that CNN converges much faster than conventional algorithms with compatible clustering quality while other algorithms may give unstable results depending on the initial values of the learning coefficient and the total number of iterations.

On Adaptive Fuzzy Control Using Neural Networks

This paper presents investigations toward adaptive fuzzy control using neural networks for the control of nonlinear and time variable physical processes. A neural network in the con-trol system can work as an adaptive rule estimator for online acquisition and modification of rules for a fuzzy logic controller. Such a neural network estimator generates rules adaptively in the fuzzy controller as a fuzzy associative memory (FAM). Neural networks with unsupervised learning are the most appropriate for cases, such as: competitive learning, differential competitive learning, learning vector quantization, self-organizing maps etc. The neural systems perform well but require extensive computation for training. A new technique of learning in two stages was applied to improve this: first, in off-line system for initial learning of neural network with desired properties of control system, second in on-line system with adaptation to the changes in the system. The best results were received with differential competitive learning in two stages of learning.

Adaptive fuzzy systems for backing up a truck-and-trailer

Fuzzy control systems and neural-network control systems for backing up a simulated truck, and truck-and-trailer, to a loading dock in a parking lot are presented. The supervised backpropagation learning algorithm trained the neural network systems. The robustness of the neural systems was tested by removing random subsets of training data in learning sequences. The neural systems performed well but required extensive computation for training. The fuzzy systems performed well until over 50% of their fuzzy-associative-memory (FAM) rules were removed. They also performed well when the key FAM equilibration rule was replaced with destructive, or ;sabotage', rules. Unsupervised differential competitive learning (DCL) and product-space clustering adaptively generated FAM rules from training data. The original fuzzy control systems and neural control systems generated trajectory data. The DCL system rapidly recovered the underlying FAM rules. Product-space clustering converted the neural truck systems into structured sets of FAM rules that approximated the neural system's behavior.

Differential competitive learning for centroid estimation and phoneme recognition

A comparison is made of a differential-competitive-learning (DCL) system with two supervised competitive-learning (SCL) systems for centroid estimation and for phoneme recognition. DCL provides a form of unsupervised adaptive vector quantization. Standard stochastic competitive-learning systems learn only if neurons win a competition for activation induced by randomly sampled patterns. DCL systems learn only if the competing neurons change their competitive signal. Signal-velocity information provides unsupervised local reinforcement during learning. The sign of the neuronal signal derivative rewards winners and punishes losers. Standard competitive learning ignores instantaneous win-rate information. Synaptic fan-in vectors adaptively quantize the randomly sampled pattern space into nearest-neighbor decision classes. More generally, the synaptic-vector distribution estimates the unknown sampled probability density function p( x). Simulations showed that unsupervised DCL-trained synaptic vectors converged to class centroids at least as fast as, and wandered less about these centroids than, SCL-trained synaptic vectors did. Simulations on a small set of English phonemes favored DCL over SCL for classification accuracy.

Stochastic competitive learning

Competitive learning systems are examined as stochastic dynamical systems. This includes continuous and discrete formulations of unsupervised, supervised, and differential competitive learning systems. These systems estimate an unknown probability density function from random pattern samples and behave as adaptive vector quantizers. Synaptic vectors, in feedforward competitive neural networks, quantize the pattern space and converge to pattern class centroids or local probability maxima. A stochastic Lyapunov argument shows that competitive synaptic vectors converge to centroids exponentially quickly and reduces competitive learning to stochastic gradient descent. Convergence does not depend on a specific dynamical model of how neuronal activations change. These results extend to competitive estimation of local covariances and higher order statistics.