Frequency Sensitive Competitive Learning Research Articles

Designing non-linear minimax and related discriminants by disjoint tangent configurations applied to RBF networks

Non-linear classification machines seldom are trained under criteria that are usual and useful for linear discriminants, such as minimax, Fisher’s, and other similar criteria. The reason is the learning difficulties that transformation-trainable machines suffer when applying such criteria. However, the possibility of using non-linear machines whose transformations are pre-designed merits attention.In this contribution, we propose and study an efficient and potentially effective option: Applying Disjoint Tangent Configurations (DTC), a formulation that includes discriminants such as Fisher’s, Bayes for normal distributions, Minimax Probabilistic Decision Hyperplane (MPDH), and others, to the output of a Radial Basis Function (RBF) network which has been previously designed with a moderate number of nodes to reduce the computational load, but with a high quality centroid selection algorithm, Frequency Sensitive Competitive Learning (FSCL), which allows to obtain networks with high representation capabilities. Experiments demonstrate that this approach leads to good performance results with acceptable computational efforts.

Multimodel Representation of Complex Nonlinear Systems: A Multifaceted Approach for Real-Time Application

Presenting an important potential in the representation of nonlinear systems, the multimodel approach remains an attractive axis for research. One of the important problems in the multimodel structure concerns the validity calculation which is a fundamental point especially when the process is corrupted with noise and/or its parameters are of high variations. A new approach based on the use of both two type of validity is proposed. A developed specification of the need of each one is explained by an optimization procedure. The conduct of this approach requires, first, the classification of the numerical data into a set of clusters. The frequency-sensitive competitive learning (FSCL) algorithm is used to select the number of models and the fuzzy k-means algorithm identify the operating clusters. From the satisfactory results in terms of precision and robustness obtained on theoretical examples, we are incited to confirm our contribution to real process reactor. The results obtained are compared to the classical approaches showing its ability to represent adequately the nonlinear process with a superior precision and accuracy and from this the classic strategy of multimodel representation is oriented towards a multifaceted approach.

Riemannian competitive learning for symmetric positive definite matrices clustering

Abstract Symmetric positive definite (SPD) matrices have achieved considerable success in numerous computer vision applications including activity recognition, texture classification, and diffusion tensor imaging. Traditional pattern recognition methods developed in Euclidean space are not suitable for direct processing of SPD matrices because they lie in a Riemannian manifold of negative curvature. In this paper, we draw inspiration from neural network based simple competitive learning, which updates the weight of a neural network in Euclidean space, and propose Riemannian competitive learning which updates the weight in the Riemannian weight space, for SPD manifold clustering. This framework is based on the Fisher-Rao metric which is theoretically very good. We further introduce a conscious competition mechanism to enhance the performance of the RCL algorithm and have developed a robust algorithm termed Riemannian Frequency Sensitive Competitive Learning (rFSCL). Compared with existing methods, the advantages are threefold: (1) rFSCL inherits the online nature of competitive learning, which makes it capable of handling very large data sets; (2) rFSCL inherits the advantage of conscious competitive learning, which indicates that it is less sensitive to the initial values of the cluster centers and that all clusters are fully utilized without the “dead unit” problem associated with many clustering algorithms; and (3) As an intrinsic Riemannian clustering method, rFSCL operates along the geodesic on the manifold in an closed-form analytic manner and is completely independent of the choice of local coordinate systems. Extensive experiments on both synthetic data sets and real data sets show its superior performance to that of other state-of-the-art SPD clustering methods.

Competitive learning for self organizing maps used in classification of partial discharges

In this paper different competitive learning algorithms for self-organizing maps (SOM) are experimentally examined. The characterization of the results obtained is presented in terms of quality of SOM. The competitive learning algorithms evaluated through SOM are winner-takes-all, frequency sensitive competitive learning, and rival penalized competitive learning. Case study: their performance in the classification of partial discharges on power cables.

Color Quantization Based on Hierarchical Frequency Sensitive Competitive Learning

In this paper, we propose a Hierarchical Frequency Sensitive Competitive Learning (HFSCL) method to achieve Color Quantization (CQ). In HFSCL, the appropriate number of quantized colors and the palette can be obtained by an adaptive procedure following a binary tree structure with nodes and layers. Starting from the root node that contains all colors in an image until all nodes are examined by split conditions, a binary tree will be generated. In each node of the tree, a Frequency Sensitive Competitive Learning (FSCL) network is used to achieve two-way division. To avoid over-split, merging condition is defined to merge the clusters that are close enough to each other at each layer. Experimental results show that the proposed HFSCL has desired ability for CQ.

Text clustering using frequent itemsets

Frequent itemset originates from association rule mining. Recently, it has been applied in text mining such as document categorization, clustering, etc. In this paper, we conduct a study on text clustering using frequent itemsets. The main contribution of this paper is three manifolds. First, we present a review on existing methods of document clustering using frequent patterns. Second, a new method called Maximum Capturing is proposed for document clustering. Maximum Capturing includes two procedures: constructing document clusters and assigning cluster topics. We develop three versions of Maximum Capturing based on three similarity measures. We propose a normalization process based on frequency sensitive competitive learning for Maximum Capturing to merge cluster candidates into predefined number of clusters. Third, experiments are carried out to evaluate the proposed method in comparison with CFWS, CMS, FTC and FIHC methods. Experiment results show that in clustering, Maximum Capturing has better performances than other methods mentioned above. Particularly, Maximum Capturing with representation using individual words and similarity measure using asymmetrical binary similarity achieves the best performance. Moreover, topics produced by Maximum Capturing distinguished clusters from each other and can be used as labels of document clusters.

Learning to display high dynamic range images

In this paper, we present a learning-based image processing technique. We have developed a novel method to map high dynamic range scenes to low dynamic range images for display in standard (low dynamic range) reproduction media. We formulate the problem as a quantization process and employ an adaptive conscience learning strategy to ensure that the mapped low dynamic range displays not only faithfully reproduce the visual features of the original scenes, but also make full use of the available display levels. This is achieved by the use of a competitive learning neural network that employs a frequency sensitive competitive learning mechanism to adaptively design the quantizer. By optimizing an L 2 distortion function, we ensure that the mapped low dynamic images preserve the visual characteristics of the original scenes. By incorporating a frequency sensitive competitive mechanism, we facilitate the full utilization of the limited displayable levels. We have developed a deterministic and practicable learning procedure which uses a single variable to control the display result. We give a detailed description of the implementation procedure of the new learning-based high dynamic range compression method and present experimental results to demonstrate the effectiveness of the method in displaying a variety of high dynamic range scenes.

Condition monitoring of 3G cellular networks through competitive neural models

We develop an unsupervised approach to condition monitoring of cellular networks using competitive neural algorithms. Training is carried out with state vectors representing the normal functioning of a simulated CDMA2000 network. Once training is completed, global and local normality profiles (NPs) are built from the distribution of quantization errors of the training state vectors and their components, respectively. The global NP is used to evaluate the overall condition of the cellular system. If abnormal behavior is detected, local NPs are used in a component-wise fashion to find abnormal state variables. Anomaly detection tests are performed via percentile-based confidence intervals computed over the global and local NPs. We compared the performance of four competitive algorithms [winner-take-all (WTA), frequency-sensitive competitive learning (FSCL), self-organizing map (SOM), and neural-gas algorithm (NGA)] and the results suggest that the joint use of global and local NPs is more efficient and more robust than current single-threshold methods.

Frequency-sensitive competitive learning for scalable balanced clustering on high-dimensional hyperspheres.

Competitive learning mechanisms for clustering, in general, suffer from poor performance for very high-dimensional (>1000) data because of "curse of dimensionality" effects. In applications such as document clustering, it is customary to normalize the high-dimensional input vectors to unit length, and it is sometimes also desirable to obtain balanced clusters, i.e., clusters of comparable sizes. The spherical kmeans (spkmeans) algorithm, which normalizes the cluster centers as well as the inputs, has been successfully used to cluster normalized text documents in 2000+ dimensional space. Unfortunately, like regular kmeans and its soft expectation-maximization-based version, spkmeans tends to generate extremely imbalanced clusters in high-dimensional spaces when the desired number of clusters is large (tens or more). This paper first shows that the spkmeans algorithm can be derived from a certain maximum likelihood formulation using a mixture of von Mises-Fisher distributions as the generative model, and in fact, it can be considered as a batch-mode version of (normalized) competitive learning. The proposed generative model is then adapted in a principled way to yield three frequency-sensitive competitive learning variants that are applicable to static data and produced high-quality and well-balanced clusters for high-dimensional data. Like kmeans, each iteration is linear in the number of data points and in the number of clusters for all the three algorithms. A frequency-sensitive algorithm to cluster streaming data is also proposed. Experimental results on clustering of high-dimensional text data sets are provided to show the effectiveness and applicability of the proposed techniques. Index Terms-Balanced clustering, expectation maximization (EM), frequency-sensitive competitive learning (FSCL), high-dimensional clustering, kmeans, normalized data, scalable clustering, streaming data, text clustering.

A neural-fuzzy classifier for recognition of power quality disturbances

This paper presents a neural-fuzzy technology-based classifier for the recognition of power quality disturbances. The classifier adopts neural networks in the architecture of frequency sensitive competitive learning and learning vector quantization (LVQ). With given size of codewords, the neural networks are trained to determine the optimal decision boundaries separating different categories of disturbances. To cope with the uncertainties in the involved pattern recognition, the neural network outputs, instead of being taken as the final classification, are used to activate the fuzzy-associative-memory (FAM) recalling for identifying the most possible type that the input waveform may belong to. Furthermore, the input waveforms are preprocessed by the wavelet transform for feature extraction so as to improve the classifier with respect to recognition accuracy and scheme simplicity. Each subband of the transform coefficients is then utilized to recognize the associated disturbances.

A Neural-Fuzzy Classifier for Recognition of Power Quality Disturbances

This article presents a neural-fuzzy technology-based classifier for the recognition of power quality disturbances. The classifier adopts neural networks in the architecture of frequency-sensitive competitive leaning and learning vector quantization. With given size of code words, the neural networks are trained to determine the optimal decision boundaries separating different categories of disturbances. To cope with the uncertainties in the involved patten recognition, the neural network outputs, instead of being taken as the final classification, are used to activate the fuzzy-associative-memory recalling for identifying the most possible type that the input waveform may belong to. Furthermore, the input waveforms are preprocessed by the wavelet transform for feature extraction so as to improve the classifier with respect to recognition accuracy and scheme simplicity. Each sub-band of the transform coefficients is then utilized to recognize the associated disturbances.

An investigation of competitive learning for autonomous cluster identification in embedded systems.

Robust signal processing for embedded systems requires the effective identification and representation of features within raw sensory data. This task is inherently difficult due to unavoidable long-term changes in the sensory systems and/or the sensed environment. In this paper we explore four variations of competitive learning and examine their suitability as an unsupervised technique for the automated identification of data clusters within a given input space. The relative performance of the four techniques is evaluated through their ability to effectively represent the structure underlying artificial and real-world data distributions. As a result of this study it was found that frequency sensitive competitive learning provides both reliable and efficient solutions to complex data distributions. As well, frequency sensitive and soft competitive learning are shown to exhibit properties which may permit the evolution of an appropriate network structure through the use of growing or pruning procedures.

Efficient Vector Quantization Using the WTA-Rule with Activity Equalization

We propose a new algorithm for vector quantization, the Activity Equalization Vector quantization (AEV). It is based on the winner takes all rule with an additional supervision of the average node activities over a training interval and a subsequent re-positioning of those nodes with low average activities. The re-positioning is aimed to both an exploration of the data space and a better approximation of already discovered data clusters by an equalization of the node activities. We introduce a learning scheme for AEV which requires as previous knowledge about the data only their bounding box. Using an example of Martinetz et al. l1r, AEV is compared with the Neural Gas, Frequency Sensitive Competitive Learning (FSCL) and other standard algorithms. It turns out to converge much faster and requires less computational effort.

Competitive Learning and its Application in Adaptive Vision for Autonomous Mobile Robots

The task of providing robust vision for autonomous mobile robots is a complex signal processing problem which cannot be solved using traditional deterministic computing techniques. In this article we investigate four unsupervised neural learning algorithms, known collectively as competitive learning, in order to assess both their theoretical operation and their ability to learn to represent a basic robotic vision task. This task involves the ability of a modest robotic system to identify the components of basic motion and to generalize upon that learned knowledge to classify correctly novel visual experiences. This investigation shows that standard competitive learning and the DeSieno version of frequency-sensitive competitive learning (FSCL) are unsuitable for solving this problem. Soft competitive learning, while capable of producing an appropriate solution, is too computationally expensive in its present form to be used under the constraints of this application. However, the Krishnamurthy version of FSCL is found to be both computationally efficient and capable of reliably learning a suitable solution to the motion identification problem both in simulated tests and in actual hardware-based experiments.

Basic Competitive Neural Networks as Adaptive Mechanisms for Non-Stationary Colour Quantisation

In this paper we consider the application of two basic Competitive Neural Networks (CNN) to the adaptive computation of colour representatives on image sequences that show non-stationary distributions of pixel colours. The tested algorithms are the Simple Competitive Learning (SCL) algorithm and the Frequency-Sensitive Competitive Learning (FSCL) algorithm. Both, SCL and FCSL are the simplest adaptive methods based, respectively, on minimising the distortion and on the search for a uniform quantisation. The aim of this paper is to study several computational properties of these methods when applied to non-stationary clustering as adaptive vector quantisation algorithms. Non-stationary colour quantisation is, therefore, representative of the more general class of non-station-ary clustering problems. We expect our results to be meaningful for other algorithms that involve either the minimisation of the distortion or the search for uniform quantisers. We study experimentally the effect of the size of the image sample employed in the one-pass adaptation, their robustness to initial conditions, and the effect of local versus global scheduling of the learning rate.

High-dimensional clustering using frequency sensitive competitive learning

In this paper a clustering algorithm for sparsely sampled high-dimensional feature spaces is proposed. The algorithm performs clustering by employing a distance measure that compensates for differently sized clusters. A sequential version of the algorithm is constructed in the form of a frequency-sensitive competitive learning scheme. Experiments are conducted on an artificial Gaussian data set and on wavelet-based texture feature sets, where classification performance is used as a clustering significance measure. It is shown that the proposed technique improves classification performance dramatically for high-dimensional problems.

Configuring RBF neural networks

A novel method (based on the characteristics of scatter matrices and frequency-sensitive competitive learning) for training the hidden layer of a radial basis function neural network is proposed. The method is demonstrated to be robust and to outperform the state-of-the-art algorithm.

Analog VLSI Circuits for Competitive Learning Networks

An investigation is made concerning implementations of competitive learning algorithms in analog VLSI circuits and systems. Analog and low power digital circuits for competitive learning are currently important for their applications in computationally-efficient speech and image compression by vector quantization, as required for example in portable multi-media terminals. A summary of competitive learning models is presented to indicate the type of VLSI computations required, and the effects of weight quantization are discussed. Analog circuit representations of computational primitives for learning and evaluation of distortion metrics are discussed. The present state of VLSI implementations of hard and soft competitive learning algorithms are described, as well as those for topological feature maps. Tolerance of learning algorithms to observed analog circuit properties is reported. New results are also presented from simulations of frequency-sensitive and soft competitive learning concerning sensitivity of these algorithms to precision in VLSI learning computations. Applications of these learning algorithms to unsupervised feature extraction and to vector quantization of speech and images are also described.

Competitive learning and vector quantization in digital VLSI systems

This paper reviews implementations of competitive learning algorithms in digital VLSI circuits and systems, including recent results from our laboratory on neural circuits for efficient learning computations and for vector quantization. Digital circuits for competitive learning, especially those operating with low-power requirements, are currently important for their applications in computationally efficient speech and image compression by vector quantization, as required, for example, in portable multimedia terminals. A summary of competitive learning models is presented to indicate the type of VLSI computations required, and the effects of weight quantization are discussed. Circuit representations of computational primitives for learning and evaluation of distortion metrics by digital circuits are also reviewed. The present state of VLSI implementations of hard and soft competitive learning algorithms are discussed, as well as those for topological feature maps. Recent results are also presented from simulations of frequency-sensitive competitive learning concerning sensitivity of these algorithms to limited precision in VLSI learning computations. Applications of these learning algorithms to unsupervised feature extraction and to vector quantization are also described.

Diffusion approximation of frequency sensitive competitive learning

The focus of this paper is a convergence study of the frequency sensitive competitive learning (FSCL) algorithm. We approximate the final phase of FSCL learning by a diffusion process described by the Fokker-Plank equation. Sufficient and necessary conditions are presented for the convergence of the diffusion process to a local equilibrium. The analysis parallels that by Ritter-Schulten (1988) for Kohonen's self-organizing map. We show that the convergence conditions involve only the learning rate and that they are the same as the conditions for weak convergence described previously. Our analysis thus broadens the class of algorithms that have been shown to have these types of convergence characteristics.