Generalized Relevance Learning Vector Quantization Research Articles

Generalized Learning Riemannian Space Quantization: A Case Study on Riemannian Manifold of SPD Matrices.

Learning vector quantization (LVQ) is a simple and efficient classification method, enjoying great popularity. However, in many classification scenarios, such as electroencephalogram (EEG) classification, the input features are represented by symmetric positive-definite (SPD) matrices that live in a curved manifold rather than vectors that live in the flat Euclidean space. In this article, we propose a new classification method for data points that live in the curved Riemannian manifolds in the framework of LVQ. The proposed method alters generalized LVQ (GLVQ) with the Euclidean distance to the one operating under the appropriate Riemannian metric. We instantiate the proposed method for the Riemannian manifold of SPD matrices equipped with the Riemannian natural metric. Empirical investigations on synthetic data and real-world motor imagery EEG data demonstrate that the performance of the proposed generalized learning Riemannian space quantization can significantly outperform the Euclidean GLVQ, generalized relevance LVQ (GRLVQ), and generalized matrix LVQ (GMLVQ). The proposed method also shows competitive performance to the state-of-the-art methods on the EEG classification of motor imagery tasks.

Cyber-Physical Security with RF Fingerprint Classification through Distance Measure Extensions of Generalized Relevance Learning Vector Quantization

Radio frequency (RF) fingerprinting extracts fingerprint features from RF signals to protect against masquerade attacks by enabling reliable authentication of communication devices at the “serial number” level. Facilitating the reliable authentication of communication devices are machine learning (ML) algorithms which find meaningful statistical differences between measured data. The Generalized Relevance Learning Vector Quantization-Improved (GRLVQI) classifier is one ML algorithm which has shown efficacy for RF fingerprinting device discrimination. GRLVQI extends the Learning Vector Quantization (LVQ) family of “winner take all” classifiers that develop prototype vectors (PVs) which represent data. In LVQ algorithms, distances are computed between exemplars and PVs, and PVs are iteratively moved to accurately represent the data. GRLVQI extends LVQ with a sigmoidal cost function, relevance learning, and PV update logic improvements. However, both LVQ and GRLVQI are limited due to a reliance on squared Euclidean distance measures and a seemingly complex algorithm structure if changes are made to the underlying distance measure. Herein, the authors (1) develop GRLVQI-D (distance), an extension of GRLVQI to consider alternative distance measures and (2) present the Cosine GRLVQI classifier using this framework. To evaluate this framework, the authors consider experimentally collected Z-wave RF signals and develop RF fingerprints to identify devices. Z-wave devices are low-cost, low-power communication technologies seen increasingly in critical infrastructure. Both classification and verification, claimed identity, and performance comparisons are made with the new Cosine GRLVQI algorithm. The results show more robust performance when using the Cosine GRLVQI algorithm when compared with four algorithms in the literature. Additionally, the methodology used to create Cosine GRLVQI is generalizable to alternative measures.

Plant Leaf Recognition Using Competitive Based Learning Algorithm

Plant recognition based on digital leaf image has received as particular attention in computer vision and intelligence system, due its important implication in automatic plant identification. Plant species have the unique leaf characteristics such as the shape, texture, margin, and colour, which different each other. This study presents a novel method for automation plant recognition using Generalized Relevance Learning Vector Quantization (GRLVQ). GRLVQ is a competitive based learning algorithm which is integrating features extraction and classification phases. The experimental result shows that GRLVQ has better performance than the predecessor algorithm.

An online and incremental GRLVQ algorithm for prototype generation based on granular computing

In supervised classification, learning vector quantization (LVQ) methods are commonly used due to their intuitive structure based on prototypical instances that reduce considerably the computations in the classification process. Several improvements of LVQ have been proposed based on heuristics including LVQ3, and GLVQ. All these methods use the Euclidean distance to evaluate the similarity between prototypes and objects, which may be inappropriate if features are not equally scaled. Metric adaption techniques try to alleviate this problem by learning discriminative distance measures from the training data. Generalized relevance learning vector quantization is one of such improvements. However, in big data problems LVQ algorithms require incremental learning mechanisms. This paper introduces an LVQ-algorithm based on granular computing for prototye-based classification equipped with incremental learning mechanisms. The proposed algorithm is able to group entities with similar features, and at the same time proposes new prototypes to better cover the class distribution with prototyping elements. Two steps for the automatic control of prototypes are proposed: the first one controls the number of prototypes by a usage-frequency indicator; whereas the second one, is designed to learn the relevance of data dimensions, producing an automatic pruning of useless dimensions, avoiding a high computational load and increasing the interpretability of the resulting model. The proposed method is evaluated in benchmark data and obtains competitive performance with state-of-the-art solutions. In the case of big data sets, we obtained the best accuracy rate of about 72 % with a good compression rate of around 94 %.

Identification of precancerous lesions by multispectral gastroendoscopy

Gastric cancer is one of the fifth most deadly cancers worldwide. Nowadays the diagnosis is performed through gastroendoscopy under white light and histological analysis. However, the precancerous lesions are multifocalized and present low differences with respect to healthy tissue. Several systems have been proposed based on light tissue interaction to improve the visualization of malignancies. However, these systems are limited to few wavelengths. In this paper, we propose a minimally invasive technique based on multispectral imaging and a methodology to identify malignancies in the stomach. We developed a multispectral gastroendoscopic system compatible with current gastroendoscopes, where only the illumination is changed. The spectra are extracted from the acquired multispectral images in order to compute statistical features that are used to classify the data in two classes: healthy and malignant. The features are ranked by pooled variance t test to train three classifiers. Neural networks using generalized relevance learning vector quantization, support vector machine (SVM) with a Gaussian kernel and k-nn are evaluated using leave one patient out cross-validation. Taking into consideration the data collected in this work, the quantitative results from the classification using SVM show high accuracy and sensitivity using a low number of features. These results show the ability to discriminate malignancies of the gastric tissue. Therefore, multispectral imaging could help in the identification of malignancies during gastroendoscopy.

Learning vector quantization for (dis-)similarities

Prototype-based methods often display very intuitive classification and learning rules. However, popular prototype based classifiers such as learning vector quantization (LVQ) are restricted to vectorial data only. In this contribution, we discuss techniques how to extend LVQ algorithms to more general data characterized by pairwise similarities or dissimilarities only. We propose a general framework how the methods can be combined based on the background of a pseudo-Euclidean embedding of the data. This covers the existing approaches kernel generalized relevance LVQ and relational generalized relevance LVQ, and it opens the way towards two novel approach, kernel robust soft LVQ and relational robust soft LVQ. Interestingly, also unsupervised prototype based techniques which are based on a cost function can be put into this framework including kernel and relational neural gas and kernel and relational self-organizing maps (based on Heskes' cost function). We demonstrate the performance of the LVQ techniques for similarity or dissimilarity data in several benchmarks, reaching state of the art results.

Relevance-Based Feature Extraction for Hyperspectral Images

Hyperspectral imagery affords researchers all discriminating details needed for fine delineation of many material classes. This delineation is essential for scientific research ranging from geologic to environmental impact studies. In a data mining scenario, one cannot blindly discard information because it can destroy discovery potential. In a supervised classification scenario, however, the preselection of classes presents one with an opportunity to extract a reduced set of meaningful features without degrading classification performance. Given the complex correlations found in hyperspectral data and the potentially large number of classes, meaningful feature extraction is a difficult task. We turn to the recent neural paradigm of generalized relevance learning vector quantization (GRLVQ) [B. Hammer and T. Villmann, Neural Networks vol. 15, pp. 1059-1068, 2002], which is based on, and substantially extends, learning vector quantization (LVQ) [T. Kohonen, Self-Organizing Maps, Berlin, Germany: Springer-Verlag, 2001] by learning relevant input dimensions while incorporating classification accuracy in the cost function. By addressing deficiencies in GRLVQ, we produce an improved version, GRLVQI, which is an effective analysis tool for high-dimensional data such as remotely sensed hyperspectral data. With an independent classifier, we show that the spectral features deemed relevant by our improved GRLVQI result in a better classification for a predefined set of surface materials than using all available spectral channels.

Incremental GRLVQ: Learning relevant features for 3D object recognition

We present a new variant of generalized relevance learning vector quantization (GRLVQ) in a computer vision scenario. A version with incrementally added prototypes is used for the non-trivial case of high-dimensional object recognition. Training is based upon a generic set of standard visual features, the learned input weights are used for iterative feature pruning. Thus, prototypes and input space are altered simultaneously, leading to very sparse and task-specific representations. The effectiveness of the approach and the combination of the incremental variant together with pruning was tested on the COIL100 database. It exhibits excellent performance with regard to codebook size, feature selection and recognition accuracy.

Generalized relevance LVQ ( GRLVQ) with correlation measures for gene expression analysis

A correlation-based similarity measure is derived for generalized relevance learning vector quantization ( GRLVQ). The resulting GRLVQ-C classifier makes Pearson correlation available in a classification cost framework where data prototypes and global attribute weighting terms are adapted into directions of minimum cost function values. In contrast to the Euclidean metric, the Pearson correlation measure makes input vector processing invariant to shifting and scaling transforms, which is a valuable feature for dealing with functional data and with intensity observations like gene expression patterns. Two types of data measures are derived from Pearson correlation in order to make its benefits for data processing available in compact prototype classification models. Fast convergence and high accuracies are demonstrated for cDNA-array gene expression data. Furthermore, the automatic attribute weighting of GRLVQ-C is successfully used to rate the functional relevance of analyzed genes.

On the Generalization Ability of GRLVQ Networks

We derive a generalization bound for prototype-based classifiers with adaptive metric. The bound depends on the margin of the classifier and is independent of the dimensionality of the data. It holds for classifiers based on the Euclidean metric extended by adaptive relevance terms. In particular, the result holds for relevance learning vector quantization (RLVQ) [4] and generalized relevance learning vector quantization (GRLVQ) [19].

Neural maps in remote sensing image analysis

We study the application of self-organizing maps (SOMs) for the analyses of remote sensing spectral images. Advanced airborne and satellite-based imaging spectrometers produce very high-dimensional spectral signatures that provide key information to many scientific investigations about the surface and atmosphere of Earth and other planets. These new, sophisticated data demand new and advanced approaches to cluster detection, visualization, and supervised classification. In this article we concentrate on the issue of faithful topological mapping in order to avoid false interpretations of cluster maps created by an SOM. We describe several new extensions of the standard SOM, developed in the past few years: the growing SOM, magnification control, and generalized relevance learning vector quantization, and demonstrate their effect on both low-dimensional traditional multi-spectral imagery and ∼200-dimensional hyperspectral imagery.

Generalized relevance learning vector quantization

We propose a new scheme for enlarging generalized learning vector quantization (GLVQ) with weighting factors for the input dimensions. The factors allow an appropriate scaling of the input dimensions according to their relevance. They are adapted automatically during training according to the specific classification task whereby training can be interpreted as stochastic gradient descent on an appropriate error function. This method leads to a more powerful classifier and to an adaptive metric with little extra cost compared to standard GLVQ. Moreover, the size of the weighting factors indicates the relevance of the input dimensions. This proposes a scheme for automatically pruning irrelevant input dimensions. The algorithm is verified on artificial data sets and the iris data from the UCI repository. Afterwards, the method is compared to several well known algorithms which determine the intrinsic data dimension on real world satellite image data.