Generalized Learning Vector Quantization Research Articles

Undersampling based on generalized learning vector quantization and natural nearest neighbors for imbalanced data

Undersampling based on generalized learning vector quantization and natural nearest neighbors for imbalanced data

Generalized Learning Vector Quantization With Log-Euclidean Metric Learning on Symmetric Positive-Definite Manifold.

In many classification scenarios, the data to be analyzed can be naturally represented as points living on the curved Riemannian manifold of symmetric positive-definite (SPD) matrices. Due to its non-Euclidean geometry, usual Euclidean learning algorithms may deliver poor performance on such data. We propose a principled reformulation of the successful Euclidean generalized learning vector quantization (GLVQ) methodology to deal with such data, accounting for the nonlinear Riemannian geometry of the manifold through log-Euclidean metric (LEM). We first generalize GLVQ to the manifold of SPD matrices by exploiting the LEM-induced geodesic distance (GLVQ-LEM). We then extend GLVQ-LEM with metric learning. In particular, we study both 1) a more straightforward implementation of the metric learning idea by adapting metric in the space of vectorized log-transformed SPD matrices and 2) the full formulation of metric learning without matrix vectorization, thus preserving the second-order tensor structure. To obtain the distance metric in the full LEM learning (LEML) approaches, two algorithms are proposed. One method is to restrict the distance metric to be full rank, treating the distance metric tensor as an SPD matrix, and readily use the LEM framework (GLVQ-LEML-LEM). The other method is to cast no such restriction, treating the distance metric tensor as a fixed rank positive semidefinite matrix living on a quotient manifold with total space equipped with flat geometry (GLVQ-LEML-FM). Experiments on multiple datasets of different natures demonstrate the good performance of the proposed methods.

Riemannian dynamic generalized space quantization learning

Many existing works represent signals by covariance matrices and then develop learning methods on the Riemannian symmetric positive-definite (SPD) manifold to deal with such data. However, they summarize each instance with a single covariance matrix, omitting some potential important information, such as the time evolution of the correlation in signals. In this paper, we represent each instance by a sequence of covariance matrices and develop a novel dynamic generalized learning Riemannian space quantization (DGLRSQ) method to deal with such data representations. The proposed DGLRSQ method incorporates short-term memory mechanism in generalized learning Riemannian space quantization (GLRSQ), which is an extension of Euclidean generalized learning vector quantization to deal with SPD matrix-valued data. The proposed method can capture the temporal evolution of the correlation in signals and thus provides better performance to its the counterpart – GLRSQ, which treats each instance as a signal covariance matrix. Empirical investigations on synthetic data and motor imagery EEG data show the superior performance of the proposed method.

Variants of recurrent learning vector quantization

Learning Vector Quantization (LVQ) and its cost-function-based variant called Generalized Learning Vector Quantization (GLVQ) are powerful, yet simple and interpretable Machine Learning (ML) algorithms for multi-class classification. Even though GLVQ is an effective tool for classifying vectorial data in its native form, it is not particularly suited to handle raw sequence data of potentially different lengths. Usually, this problem is addressed by manually engineering fixed-length features from the raw data or by employing recurrent networks. Therefore, a natural idea is to try to incorporate recurrent units for data processing into the GLVQ network structure. The processed data can then be compared in a latent space for classification decisions. Yet, not much work has been done in direction as far as we are aware. Existing methods to handle sequential data in the framework of GLVQ are rather unsophisticated and severely outdated. In this paper, we provide a general framework for incorporating recurrent structures in an LVQ network and derive two classification models as variants of Recurrent Learning Vector Quantization, namely RecLVQ and LVQRNN. We also demonstrate the abilities of such approaches on illustrative classification problems.

Grouped Subspace Linear Semantic Alignment for Hyperspectral Image Transfer Learning

Transfer learning (TL) offers an effective way to reduce the demand for labeled samples in remote sensing image classification. However, existing TL methods have some limitations. Simple linear TL methods cannot align the source and target domains well when the data shift is complicated, while nonlinear methods consist of many learnable parameters and often need many labeled samples. To overcome these issues, we design a novel grouped subspace linear semantic alignment (G-SLSA) algorithm, which consists of four main ingredients. First, inspired by the linear supervised transfer learning (LSTL) approach, we propose subspace linear semantic alignment (SLSA) aiming to reduce the demand for labeled samples. Second, considering the heterogeneity of class-level data shift, we extend SLSA to G-SLSA through a grouped alignment strategy, which can reduce the data shift by decomposing a multiclass transfer learning task into a set of binary subtasks. Third, considering the demand of subtasks fusion on posterior probabilities, we propose a robust posterior probability estimation method for the binary generalized learning vector quantization (GLVQ) that is used in G-SLSA. Finally, a pairwise coupling (PWC) method is applied to fuse the results of each subtask. Experimental results conducted on three popular hyperspectral datasets demonstrate that G-SLSA outperforms other traditional and state-of-the-art deep learning (DL) methods, with an OA of 80.78±4.28% for PC-UP transfer learning scenario when 5 samples per class are available in the target domain.

The Resolved Mutual Information Function as a Structural Fingerprint of Biomolecular Sequences for Interpretable Machine Learning Classifiers.

In the present article we propose the application of variants of the mutual information function as characteristic fingerprints of biomolecular sequences for classification analysis. In particular, we consider the resolved mutual information functions based on Shannon-, Rényi-, and Tsallis-entropy. In combination with interpretable machine learning classifier models based on generalized learning vector quantization, a powerful methodology for sequence classification is achieved which allows substantial knowledge extraction in addition to the high classification ability due to the model-inherent robustness. Any potential (slightly) inferior performance of the used classifier is compensated by the additional knowledge provided by interpretable models. This knowledge may assist the user in the analysis and understanding of the used data and considered task. After theoretical justification of the concepts, we demonstrate the approach for various example data sets covering different areas in biomolecular sequence analysis.

You May Enter: automated ID checks face scrutiny [Olympic Games Facial Recognition

This article discusses the use of a contactless AI-driven facial-recognition system being used for ID checks and queue management at the Tokyo Olympics 2020. This will be used for staff, volunteers, athletes and media who have already given their consent, not the general public. The system is based on NEC NeoFace Watch technology, and uses generalised matching face detection method, along with an artificial neural network model called the generalised learning vector quantisation algorithm, to search and select facial features for facial matching. The perturbation space method then coverts the two-dimensional images into three dimensions, allowing rotation of the candidates head. Further processing with illumination filters and adaptive regional blend matching method, minimises the impact of local changes such as hats etc. providing a better photograph match against a central database image. An IC chip embedded security lanyard pass, containing personal ID and authentication details will also be used. Data privacy concerns are discussed, especially due to the current use of NEC technology in criminal surveillance applications, with the Tokyo 2020 organisation responding with a promise to comply with local privacy law. Possibilities of using the technology to enhance Covid-19 security and aid in contact-tracing for the event is also discussed.

Development of Adaptive Fuzzy-Neuro Generalized Learning-Vector Quantization Using PI Membership Function (AFNGLVQ-PI)

In a real-world environment, there are several difficult obstacles to overcome in classification. Those obstacles are data overlapping and skewness of data distribution. Overlapping data occur when many data from different classes overlap with each other; this condition often occurs when there are many classes in a data set. On other hand, skewness of data distribution occurs when the data distribution is not a Gaussian (normal) distribution. To overcome these two problems, a new method called Adaptive Fuzzy-Neuro Generalized Learning Vector Quantization using PI membership function (AFNGLVQ-PI) is proposed in this study. AFNGLVQ-PI is derived from Fuzzy-Neuro Generalized Learning Vector Quantization using the PI membership function (FNGLVQ-PI). In FNGLVQ-PI, the updated values for minimum and maximum variables in the fuzzy membership function are set based on the mean of the updated values. Whereas, in the newly proposed AFNGLVQ-PI, updated values for minimum, maximum, and mean variables are derived based on the differential equations to approximate the data distribution better. In this study, the newly proposed AFNGLVQ-PI algorithm was tested and verified on twelve different data sets. Two of the data sets are synthetic data sets where we could compare the performance of the data sets in different overlapping conditions and levels of skewness. The rest of the data sets were chosen and used as a benchmark to compare the performance of the proposed algorithm. In the experiment, AFNGLVQ-PI took first place in 18 out of 29 experiments. Furthermore, AFNGLVQ-PI also achieved positive improvements for all data sets used in the experiments, which could not be achieved by the Learning Vector Quantization (LVQ), Generalized Learning Vector Quantization (GLVQ), and other commonly used algorithms, such as SVM, kNN, and MLP.

Generalized Learning Riemannian Space Quantization: A Case Study on Riemannian Manifold of SPD Matrices.

Learning vector quantization (LVQ) is a simple and efficient classification method, enjoying great popularity. However, in many classification scenarios, such as electroencephalogram (EEG) classification, the input features are represented by symmetric positive-definite (SPD) matrices that live in a curved manifold rather than vectors that live in the flat Euclidean space. In this article, we propose a new classification method for data points that live in the curved Riemannian manifolds in the framework of LVQ. The proposed method alters generalized LVQ (GLVQ) with the Euclidean distance to the one operating under the appropriate Riemannian metric. We instantiate the proposed method for the Riemannian manifold of SPD matrices equipped with the Riemannian natural metric. Empirical investigations on synthetic data and real-world motor imagery EEG data demonstrate that the performance of the proposed generalized learning Riemannian space quantization can significantly outperform the Euclidean GLVQ, generalized relevance LVQ (GRLVQ), and generalized matrix LVQ (GMLVQ). The proposed method also shows competitive performance to the state-of-the-art methods on the EEG classification of motor imagery tasks.

Quantum-inspired learning vector quantizers for prototype-based classification

Prototype-based models like the Generalized Learning Vector Quantization (GLVQ) belong to the class of interpretable classifiers. Moreover, quantum-inspired methods get more and more into focus in machine learning due to its potential efficient computing. Further, its interesting mathematical perspectives offer new ideas for alternative learning scenarios. This paper proposes a quantum computing-inspired variant of the prototype-based GLVQ for classification learning. We start considering kernelized GLVQ with real- and complex-valued kernels and their respective feature mapping. Thereafter, we explain how quantum space ideas could be integrated into a GLVQ using quantum bit vector space in the quantum state space {mathcal {H}}^{n} and show the relations to kernelized GLVQ. In particular, we explain the related feature mapping of data into the quantum state space {mathcal {H}}^{n}. A key feature for this approach is that {mathcal {H}}^{n} is an Hilbert space with particular inner product properties, which finally restrict the prototype adaptations to be unitary transformations. The resulting approach is denoted as Qu-GLVQ. We provide the mathematical framework and give exemplary numerical results.

Passive concept drift handling via variations of learning vector quantization

Concept drift is a change of the underlying data distribution which occurs especially with streaming data. Besides other challenges in the field of streaming data classification, concept drift has to be addressed to obtain reliable predictions. Robust Soft Learning Vector Quantization as well as Generalized Learning Vector Quantization has already shown good performance in traditional settings and is modified in this work to handle streaming data. Further, momentum-based stochastic gradient descent techniques are applied to tackle concept drift passively due to increased learning capabilities. The proposed work is tested against common benchmark algorithms and streaming data in the field and achieved promising results.

Classification of Emotional State Based on EEG Signal using AMGLVQ

Identifying humans emotional state using electroencephalogram (EEG) signal more precisely than using non-verbal and verbal signals, because emotions are psychological and physiological processes that are connected with personality, motivation, mood, and temperament. EEG is a physiological signal that recorded from brain activity in the form of brain waves through the scalp. In this study, emotional states will be identified based on EEG signals using the Adaptive Multilayer Generalized Learning Vector Quantization (AMGLVQ) algorithm. The dataset used is DEAP: A Database for Emotion Analysis using Physiological and Audiovisual Signals. Emotional conditions that are classified are valence, that is low and high valence. DEAP dataset has imbalanced data characteristics, and one of the advantages of AMGLVQ algorithm is handling classification in imbalanced data conditions. The test results show that AMGLVQ has better performance compared to Random Forest (RF) and Support Vector Machine (SVM).

Asymmetric learning vector quantization for efficient nearest neighbor classification in dynamic time warping spaces

The nearest neighbor method together with the dynamic time warping (DTW) distance is one of the most popular approaches in time series classification. This method suffers from high storage and computation requirements for large training sets. As a solution to both drawbacks, this article extends learning vector quantization (LVQ) from Euclidean spaces to DTW spaces. The proposed generic LVQ scheme uses asymmetric weighted averaging as update rule. We theoretically justify the asymmetric LVQ scheme via subgradient techniques and by the margin-growth principle. In addition, we show that the decision boundary of two prototypes from different classes is piecewise quadratic. Empirical results exhibited superior performance of asymmetric generalized LVQ (GLVQ) over other state-of-the-art prototype generation methods for nearest neighbor classification.

ONLINE KERNEL AMGLVQ FOR ARRHYTHMIA HEARBEATS CLASSIFICATION

This study proposes Online Kernel Adaptive Multilayer Generalized Learning Vector Quantization (KAMGLVQ) for handling imbalanced data sets. KAMGLVQ is extended version of AMGLVQ that used kernel function to handling non-linear classification problems. Basically AMGLVQ is vector quantization based learning. The vector quantization based learning is very simple algorithm that can be applied to the multiclass problem and the complexity of LVQ can be controlled during training process. KAMGLVQ works at online kernel learning system that integrating feature extraction and classification. The architecture network of KAMGLVQ consists of three layers, input layer, hidden layer, and an output layer. The hidden layer of KAMGLVQ is adaptive; this algorithm will generate a number of hidden layer nodes. The algorithm implement on real ECG signals from the MIT-BIH arrhythmias database and synthetic data. The experiments showed that KAMGLVQ able improve the accuracy of classification better than SVM or back-propagation NN; also able to reduce the time computational cost.

EOG feature relevance determination for microsleep detection

Abstract Automatic relevance determination (ARD) was applied to two-channel EOG recordings for microsleep event (MSE) recognition. 10 s immediately before MSE and also before counterexamples of fatigued, but attentive driving were analysed. Two type of signal features were extracted: the maximum cross correlation (MaxCC) and logarithmic power spectral densities (PSD) averaged in spectral bands of 0.5 Hz width ranging between 0 and 8 Hz. Generalised learn-ing vector quantisation (GRLVQ) was used as ARD method to show the potential of feature reduction. This is compared to support-vector machines (SVM), in which the feature reduction plays a much smaller role. Cross validation yielded mean normalised relevancies of PSD features in the range of 1.6 - 4.9 % and 1.9 - 10.4 % for horizontal and vertical EOG, respectively. MaxCC relevancies were 0.002 - 0.006 % and 0.002 - 0.06 %, respectively. This shows that PSD features of vertical EOG are indispensable, whereas MaxCC can be neglected. Mean classification accuracies were estimated at 86.6±b 1.3 % and 92.3±b 0.2 % for GRLVQ and SVM, respec-tively. GRLVQ permits objective feature reduction by inclu-sion of all processing stages, but is not as accurate as SVM.

EOG feature relevance determination for microsleep detection

Abstract Automatic relevance determination (ARD) was applied to two-channel EOG recordings for microsleep event (MSE) recognition. 10 s immediately before MSE and also before counterexamples of fatigued, but attentive driving were analysed. Two type of signal features were extracted: the maximum cross correlation (MaxCC) and logarithmic power spectral densities (PSD) averaged in spectral bands of 0.5 Hz width ranging between 0 and 8 Hz. Generalised learn-ing vector quantisation (GRLVQ) was used as ARD method to show the potential of feature reduction. This is compared to support-vector machines (SVM), in which the feature reduction plays a much smaller role. Cross validation yielded mean normalised relevancies of PSD features in the range of 1.6 – 4.9 % and 1.9 – 10.4 % for horizontal and vertical EOG, respectively. MaxCC relevancies were 0.002 – 0.006 % and 0.002 – 0.06 %, respectively. This shows that PSD features of vertical EOG are indispensable, whereas MaxCC can be neglected. Mean classification accuracies were estimated at 86.6±b 1.3 % and 92.3±b 0.2 % for GRLVQ and SVM, respectively. GRLVQ permits objective feature reduction by inclusion of all processing stages, but is not as accurate as SVM.

A New Classification Algorithm: Optimally Generalized Learning Vector Quantization (OGLVQ)

We present a new Generalized Learning Vector Quantization classifier called Optimally Generalized Learning Vector Quantization based on a novel weight-update rule for learning labelled samples. The algorithm attains stable prototype/weight vector dynamics in terms of estimated current and previous weights and their updates. Resulting weight update term is then related to the proximity measure used by Generalized Learning Vector Quantization classifiers. New algorithm and some major counterparts are tested and compared for synthetic and publicly available datasets. Results reveal that new classifier is faster in training and is more successful and robust in classifying test samples of datasets studied than the counterparts it is compared.

一般化学習ベクトル量子化による画像認識

一般化学習ベクトル量子化による画像認識

Building Objects Extracting and Recognizing Methodology from Satellite Image based on the Feature Extraction Algorithms and Sliding Window Technique

Based on the gray features and shape features of objects, some satisfied objects are detected by using sliding window algorithm from satellite image. To further recognize their identification and classification, more texture features of them are needed to obtain to compare between them. GLCM (Gray-Level Co-occurrence Matrix) statistics are used to representative each partition of them. These PGLCM (Partition-GLCM) statistics can combine into a feature vector and those detected objects can be accurately recognized and classified by using GLVQ (Generalized Learning Vector Quantization) Neural Network algorithm. Experiments show when we choose those adapted parameters, such as the length and width of the window, and the threshold of difference of adjacent pixels, the extraction rate of building objects is up to 76.1%. Using the classification algorithm based on the feature vector generating by the statistics of PGLCM, the recognition rate of building is more than 88.9%.

Automatic Arrhythmia Beat Detection: Algorithm, System, and Implementation

Cardiac disease is one of the major causes of death in the world. Early diagnose of the symptoms depends on abnormality on heart beat pattern, known as Arrhythmia. A novel fuzzy neuro generalized learning vector quantization for automatic Arrhythmia heart beat classification is proposed. The algorithm is an extension from the GLVQ algorithm that employs a fuzzy logic concept as the discriminant function in order to develop a robust algorithm and improve the classification performance. The algorithm is tested against MIT-BIH arrhythmia database to measure the performance. Based on the experiment result, FN-GLVQ is able to increase the accuracy of GLVQ by a soft margin. As we intend to build a device with automated Arrhythmia detection, FN-GLVQ is then implemented into Field Gate Programmable Array to prototype the system into a real device.