Probabilistic Self-organizing Map Research Articles

A Dynamic Stress-Scape Framework to Evaluate Potential Effects of Multiple Environmental Stressors on Gulf of Alaska Juvenile Pacific Cod

Quantifying the spatial and temporal footprint of multiple environmental stressors on marine fisheries is imperative to understanding the effects of changing ocean conditions on living marine resources. Pacific Cod(Gadus macrocephalus), an important marine species in the Gulf of Alaska ecosystem, has declined dramatically in recent years, likely in response to extreme environmental variability in the Gulf of Alaska related to anomalous marine heatwave conditions in 2014–2016 and 2019. Here, we evaluate the effects of two potential environmental stressors, temperature variability and ocean acidification, on the growth of juvenile Pacific Cod in the Gulf of Alaska using a novel machine-learning framework called “stress-scapes,” which applies the fundamentals of dynamic seascape classification to both environmental and biological data. Stress-scapes apply a probabilistic self-organizing map (prSOM) machine learning algorithm and Hierarchical Agglomerative Clustering (HAC) analysis to produce distinct, dynamic patches of the ocean that share similar environmental variability and Pacific Cod growth characteristics, preserve the topology of the underlying data, and are robust to non-linear biological patterns. We then compare stress-scape output classes to Pacific Cod growth rates in the field using otolith increment analysis. Our work successfully resolved five dynamic stress-scapes in the coastal Gulf of Alaska ecosystem from 2010 to 2016. We utilized stress-scapes to compare conditions during the 2014–2016 marine heatwave to cooler years immediately prior and found that the stress-scapes captured distinct heatwave and non-heatwave classes, which highlighted high juvenile Pacific Cod growth and anomalous environmental conditions during heatwave conditions. Dominant stress-scapes underestimated juvenile Pacific Cod growth across all study years when compared to otolith-derived field growth rates, highlighting the potential for selective mortality or biological parameters currently missing in the stress-scape model as well as differences in potential growth predicted by the stress-scape and realized growth observed in the field. A sensitivity analysis of the stress-scape classification result shows that including growth rate data in stress-scape classification adjusts the training of the prSOM, enabling it to distinguish between regions where elevated sea surface temperature is negatively impacting growth rates. Classifications that rely solely on environmental data fail to distinguish these regions. With their incorporation of environmental and non-linear physiological variables across a wide spatio-temporal scale, stress-scapes show promise as an emerging methodology for evaluating the response of marine fisheries to changing ocean conditions in any dynamic marine system where sufficient data are available.

A novel health indicator based on the Lyapunov exponent, a probabilistic self-organizing map, and the Gini-Simpson index for calculating the RUL of bearings

Abstract The estimation of the remaining useful life (RUL) of rolling element bearings has been an area of excessive research over the past few decades. Time-series forecasting approaches are the most popular methods for calculating the RUL of bearings. However, there exist two key challenges in predicting the RUL using time-series forecasting. The first is the development of an accurate health indicator (HI) that can indicate bearing degradation i.e. the developed HI must be capable of tracking the early and critical degradation stages in bearings. The second is the determination of a suitable failure threshold for the HI time-series. The HIs for different bearings fluctuate at different levels at the time of failure and consequently, it becomes difficult to set a definite failure threshold for them. To overcome these problems, a novel HI is proposed in this paper. First, the vibration signals acquired from the bearings are subjected to the feature extraction process. For this purpose, chaotic features determined using the Lyapunov exponent are utilized. Then, feature samples extracted under healthy bearing conditions are used to train probabilistic self-organizing map (p-SOM) algorithm. Finally, the trained p-SOM model is tested against the monitored feature samples to construct the HI. This HI has a major advantage in that it lies in a specific range from [0–1] and therefore allows for the selection of precise limits for defining bearing failure. Once the HI has been determined, a state-space model is employed to forecast the HI up to pre-set failure thresholds and subsequently compute the RUL of the bearings. The proposed technique is validated on publicly-available benchmark datasets. The experimental results confirm that the suggested HI is effective in predicting damage growth as well as forecasting the RUL of bearings. Further, it outperforms the traditional health indicator i.e. SOM-based-minimum quantization error (MQE).

Using continuous Hopfield neural network for solving a new optimization architecture model of probabilistic self organizing map

Probabilistic models have proven their strength to model many natural phenomena as close as possible to reality, in particular, the probabilistic self organizing map (PRSOM) that belongs to the unsupervised learning models. It allows to provide an estimation of the probability density function through the likelihood maximization. This latter function depends on several parameters given by the model architecture. In this context, the aim of this paper is to deal with the architecture choice problem that consists in determining the optimal number of components needed for a better performance. In this paper, we propose a new optimization model that describes the problem above. We present the architecture of PRSOM in a mathematical system of a non linear objective function with mixed variables under linear and quadratic constraints. Due to the complexity of the resolution, we suggest the continuous Hopfield neural network (CHN) that we support by a deep stability analysis. Performance of the proposed model is demonstrated through the dataset clustering.

Probabilistic Self-Organizing Maps for Text-Independent Speaker Identification

The present paper introduces a novel speaker modeling technique for text-independent speaker identification using probabilistic self-organizing maps (PbSOMs). The basic motivation behind the introduced technique was to combine the self-organizing quality of the self-organizing maps and generative power of Gaussian mixture models. Experimental results show that the introduced modeling technique using probabilistic self-organizing maps significantly outperforms the traditional technique using the classical GMMs and the EM algorithm or its deterministic variant. More precisely, a relative accuracy improvement of roughly 39% has been gained, as well as, a much less sensitivity to the model-parameters initialization has been exhibited by using the introduced speaker modeling technique using probabilistic self-organizing maps.

Architecture Optimization Model for the Probabilistic Self-Organizing Maps and Speech Compression

The probabilistic self-organizing map (PRSOM) is an improved version of the Kohonen classical model (SOM) that appeared in the late 1990’s. In the last years, the interest of probabilistic methods, especially in the fields of clustering and classification has increased, and the PRSOM has been successfully employed in many technological uses, such as: pattern recognition, speech recognition, data compression, medical diagnosis, etc. Mathematically, the PRSOM gives an estimation of the density probability function of a set of samples. And this estimation depends on the parameters given by the architecture of the model. Therefore, the main problem of this model, that we try to approach in this paper, is the architecture choice (the number of neurons and the initialization parameters). In summary, in the present paper, we describe a recent approach of PRSOM trying to find a solution to the problem below. For that, we propose an architecture optimization model that is a mixed integer nonlinear optimization model under linear constraints, resolved by the genetic algorithm. Then to prove the efficiency of the proposed model, we chose to apply it on a speech compression technique based on the determination of the optimal codebook, and finally, we give an implementation and an evaluation of the proposed method that we compare with the results of the classical model.

Data visualization via latent variables and mixture models: a brief survey

In the literature, data visualization is extensively studied via diverse parametric probabilistic distributions for the exploration of continuous, binary, and counting data. An overview of the existing methods for non-symmetric data matrices is presented in an unified framework via the Bernoulli law and binary variables. An extension to continuous or counting variables is available by using instead any another univariate distribution such as the Poisson or Gaussian one. Several approaches are possible when the model is with a distribution on the rows, the columns, the row clusters, the column clusters, the cells, the blocks, or a transformed matrix of the distances from the pairs of rows or columns. The objective functions are presented with their full expressions in separated sections, one for each method: Kohonen's map and related methods of self-organizing maps, generative topographic mapping as a probabilistic self-organizing map, linear principal component analysis and related matricial methods (non-negative factorization, factorization), probabilistic parametric embedding, probabilistic latent semantic visualization, latent cluster position model, t-distributed stochastic neighbor embedding. The conclusion is a discussion of the contribution with perspectives.

Probabilistic Self-Organizing Map for Clustering and Visualizing non-i.i.d Data

We present a generative approach to train a new probabilistic self-organizing map (PrSOMS) for dependent and nonidentically distributed data sets. Our model defines a low-dimensional manifold allowing friendly visualizations. To yield the topology preserving maps, our model has the SOM like learning behavior with the advantages of probabilistic models. This new paradigm uses hidden Markov models (HMM) formalism and introduces relationships between the states. This allows us to take advantage of all the known classical views associated to topographic map. The objective function optimization has a clear interpretation, which allows us to propose expectation-maximization (EM) algorithm, based on the forward–backward algorithm, to train the model. We demonstrate our approach on two data sets: The real-world data issued from the "French National Audiovisual Institute" and handwriting data captured using a WACOM tablet.

PCA filtering and probabilistic SOM for network intrusion detection

The growth of the Internet and, consequently, the number of interconnected computers, has exposed significant amounts of information to intruders and attackers. Firewalls aim to detect violations according to a predefined rule-set and usually block potentially dangerous incoming traffic. However, with the evolution of attack techniques, it is more difficult to distinguish anomalies from normal traffic. Different detection approaches have been proposed, including the use of machine learning techniques based on neural models such as Self-Organizing Maps (SOMs). In this paper, we present a classification approach that hybridizes statistical techniques and SOM for network anomaly detection. Thus, while Principal Component Analysis (PCA) and Fisher Discriminant Ratio (FDR) have been considered for feature selection and noise removal, Probabilistic Self-Organizing Maps (PSOM) aim to model the feature space and enable distinguishing between normal and anomalous connections. The detection capabilities of the proposed system can be modified without retraining the map, but only by modifying the units activation probabilities. This deals with fast implementations of Intrusion Detection Systems (IDS) necessary to cope with current link bandwidths.

Adaptive kernel regression and probabilistic self-organizing maps for JPEG image deblocking

Perhaps the most widely used format for lossy compression of still images is JPEG (Joint Photographic Experts Group). In this image format the quality loss and an important part of the file size reduction come from the application of the Discrete Cosine Transform (DCT), and the subsequent quantization of its coefficients. These processes lead to noticeable compression artifacts. In this paper we present an intelligent system which is capable of analyzing a compressed JPEG image by combining the knowledge extracted from the image domain and the transformed domain, so that the results are restored versions that are more similar to both the original from the qualitative and quantitative points of view. It builds an approximation of the original image by kernel regression, which is controlled by self-organizing neural maps, so that the resulting image does not suffer from oversmoothing.

Probabilistic Self-Organizing Maps for Continuous Data

The original self-organizing feature map did not define any probability distribution on the input space. However, the advantages of introducing probabilistic methodologies into self-organizing map models were soon evident. This has led to a wide range of proposals which reflect the current emergence of probabilistic approaches to computational intelligence. The underlying estimation theories behind them derive from two main lines of thought: the expectation maximization methodology and stochastic approximation methods. Here, we present a comprehensive view of the state of the art, with a unifying perspective of the involved theoretical frameworks. In particular, we examine the most commonly used continuous probability distributions, self-organization mechanisms, and learning schemes. Special emphasis is given to the connections among them and their relative advantages depending on the characteristics of the problem at hand. Furthermore, we evaluate their performance in two typical applications of self-organizing maps: classification and visualization.

Probabilistic self-organizing maps for qualitative data

We present a self-organizing map model to study qualitative data (also called categorical data). It is based on a probabilistic framework which does not assume any prespecified distribution of the input data. Stochastic approximation theory is used to develop a learning rule that builds an approximation of a discrete distribution on each unit. This way, the internal structure of the input dataset and the correlations between components are revealed without the need of a distance measure among the input values. Experimental results show the capabilities of the model in visualization and unsupervised learning tasks.

Visualization and clustering of categorical data with probabilistic self-organizing map

This paper introduces a self-organizing map dedicated to clustering, analysis and visualization of categorical data. Usually, when dealing with categorical data, topological maps use an encoding stage: categorical data are changed into numerical vectors and traditional numerical algorithms (SOM) are run. In the present paper, we propose a novel probabilistic formalism of Kohonen map dedicated to categorical data where neurons are represented by probability tables. We do not need to use any coding to encode variables. We evaluate the effectiveness of our model in four examples using real data. Our experiments show that our model provides a good quality of results when dealing with categorical data.

Model-Based Clustering by Probabilistic Self-Organizing Maps

In this paper, we consider the learning process of a probabilistic self-organizing map (PbSOM) as a model-based data clustering procedure that preserves the topological relationships between data clusters in a neural network. Based on this concept, we develop a coupling-likelihood mixture model for the PbSOM that extends the reference vectors in Kohonen's self-organizing map (SOM) to multivariate Gaussian distributions. We also derive three expectation-maximization (EM)-type algorithms, called the SOCEM, SOEM, and SODAEM algorithms, for learning the model (PbSOM) based on the maximum-likelihood criterion. SOCEM is derived by using the classification EM (CEM) algorithm to maximize the classification likelihood; SOEM is derived by using the EM algorithm to maximize the mixture likelihood; and SODAEM is a deterministic annealing (DA) variant of SOCEM and SOEM. Moreover, by shrinking the neighborhood size, SOCEM and SOEM can be interpreted, respectively, as DA variants of the CEM and EM algorithms for Gaussian model-based clustering. The experimental results show that the proposed PbSOM learning algorithms achieve comparable data clustering performance to that of the deterministic annealing EM (DAEM) approach, while maintaining the topology-preserving property.

A PROBABILISTIC SELF-ORGANIZING MAP FOR BINARY DATA TOPOGRAPHIC CLUSTERING

This paper introduces a probabilistic self-organizing map for topographic clustering, analysis and visualization of multivariate binary data or categorical data using binary coding. We propose a probabilistic formalism dedicated to binary data in which cells are represented by a Bernoulli distribution. Each cell is characterized by a prototype with the same binary coding as used in the data space and the probability of being different from this prototype. The learning algorithm, Bernoulli on self-organizing map, that we propose is an application of the EM standard algorithm. We illustrate the power of this method with six data sets taken from a public data set repository. The results show a good quality of the topological ordering and homogenous clustering.

Improvement of borrowing channel assignment for patterned traffic load by online cellular probabilistic self-organizing map

This paper describes an improvement of borrowing channel assignment (BCA) for patterned traffic load by using the short-term traffic prediction ability of cellular probabilistic self-organizing map (CPSOM). The fast growing cellular mobile systems demand more efficient and faster channel allocation techniques today. In case of patterned traffic load, the traditional BCA methods are not efficient to further enhance the performance because heavy-traffic cells cannot borrow channels from their neighboring cells with light or medium traffic that may have unused nominal channels. The performance can be increased if the short-term traffic load can be predicted. The predicted results can then be used for channel re-assignment. Therefore, the unused nominal channels of the light-or-medium-traffic cells can be transferred to the heavy-traffic cells that need more nominal channels. In this paper, CPSOM is used online for traffic prediction. In this sense, the proposed CPSOM-based BCA method is able to enhance the performance for patterned traffic load compared with the traditional BCA methods. Simulation results corroborate that the proposed method enables the system to work with better performance for patterned traffic load than the traditional BCA methods.

Using Cellular Probabilistic Self-Organizing Map in Borrowing Channel Assignment for Patterned Traffic Load

The fast growing cellular mobile systems demand more efficient and faster channel allocation techniques. Borrowing channel assignment (BCA) is a compromising technique between fixed channel allocation (FCA) and dynamic channel allocation (DCA). However, in the case of patterned traffic load, BCA is not efficient to further enhance the performance because some heavy-traffic cells are unable to borrow channels from neighboring cells that do not have unused nominal channels. The performance of the whole system can be raised if the short-term traffic load can be predicted and the nominal channels can be re-assigned for all cells. This paper describes an improved BCA scheme using traffic load prediction. The prediction is obtained by using the short-term forecasting ability of cellular probabilistic self-organizing map (CPSOM). This paper shows that the proposed CPSOM-based BCA method is able to enhance the performance of patterned traffic load compared with the traditional BCA methods. Simulation results corroborate that the proposed method delivers significantly better performance than BCA for patterned traffic load situations, and is virtually as good as BCA in the other situations analyzed.

An Online Cellular Probabilistic Self-Organizing Map for Static and Dynamic Data Sets

In this paper, a new online cellular probabilistic self-organizing map (CPSOM) is presented. The proposed online CPSOM is derived from the batch mode soft topological vector quantization (STVQ). It requires less storage than the STVQ such that it is able to deal with much larger data sets. It converges faster than the STVQ with the same effect when the map size is relatively small, and forms more ordered topology than the STVQ when the map size is relatively large. Most of all, by tuning a parameter in the CPSOM as a forgetting factor, the CPSOM can be used not only in static data sets, but also in dynamic data sets, where the input data come in endlessly and dynamically. The online CPSOM provides more information about the assignment probability for each neuron, which proved to be very useful for unsupervised clustering of the CPSOM.

Hierarchical clustering of self-organizing maps for cloud classification

This paper presents a new method for segmenting multispectral satellite images. The proposed method is unsupervised and consists of two steps. During the first step the pixels of a learning set are summarized by a set of codebook vectors using a Probabilistic Self-Organizing Map (PSOM, Statistique et méthodes neuronales, Dunod, Paris, 1997). In a second step the codebook vectors of the map are clustered using Agglomerative Hierarchical Clustering (AHC, Pattern Recognition and Neural Networks, Cambridge University Press, Cambridge, 1996). Each pixel takes the label of its nearest codebook vector. A practical application to Meteosat images illustrates the relevance of our approach.

Probabilistic self-organizing map and radial basis function networks

We propose in this paper a new learning algorithm probabilistic self-organizing map (PRSOM) using a probabilistic formalism for topological maps. This algorithm approximates the density distribution of the input set with a mixture of normal distributions. The unsupervised learning is based on the dynamic clusters principle and optimizes the likelihood function. A supervised version of this algorithm based on radial basis functions (RBF) is proposed. In order to validate the theoretical approach, we achieve regression tasks on simulated and real data using the PRSOM algorithm. Moreover, our results are compared with normalized Gaussian basis functions (NGBF) algorithm.