Unsupervised Competitive Learning Research Articles

Exploring energetic, exergetic, economic and environmental (4E) performance of waste heat power generation in nuclear power plant systems: A perspective of pattern recognition

Combined cogeneration systems are employed to recover waste heat from the sCO2 cycle in nuclear power generation systems. However, dealing with their multidimensional and nonlinear design and optimization spaces is a complex challenge for designers. In this study, the combined cogeneration system is parameterized and modeled: 4E models including energetic, exergetic, economic and environmental are established for sCO2 combined Organic Rankine Cycle (sCO2/ORC) and trans-critical CO2 Cycle (sCO2/tCO2) respectively. Several decision variables and five performance indicators are selected. To gain a deeper understanding of the mechanisms and optimize the combined cogeneration system, pattern recognition methods including the Self-Organizing Map (SOM), an unsupervised competitive learning approach, and Statistical Sobol Global Sensitivity Analysis methods are proposed. These techniques help identify key parameters and system patterns within the design space. Then, the Non-Dominated Sorting Whale Optimization Algorithm (NSWOA) is adopted to build a Pareto-optimal decision space, and the SOM and PCPs are proposed to visualize and mining based on pattern recognition and graph visualization interaction. The design parameters of the top cycle, P2, PRc, and x, have been confirmed as having a substantial impact on the system's performance. The multiple attributes, criterions and objectives problems in the combined cogeneration system are ultimately simplified to “Economy-Environment”, double objectives problem, and three trade-offs’ solutions are discussed: prioritizing economics, prioritizing the environment, and finding a balanced economic-environmental compromise. Pattern recognition method provides decision-makers with a fresh perspective for gaining a deeper understanding of the multi-criteria objectives space and potential trade-offs. The integration of pattern recognition into the thermodynamic, economic, and environmental models outlined in this paper represents a substantial and valuable advancement in the analysis and optimization of combined cogeneration cycles.

Atomically thin optomemristive feedback neurons.

Cognitive functions such as learning in mammalian brains have been attributed to the presence of neuronal circuits with feed-forward and feedback topologies. Such networks have interactions within and between neurons that provide excitory and inhibitory modulation effects. In neuromorphic computing, neurons that combine and broadcast both excitory and inhibitory signals using one nanoscale device are still an elusive goal. Here we introduce a type-II, two-dimensional heterojunction-based optomemristive neuron, using a stack of MoS2, WS2 and graphene that demonstrates both of these effects via optoelectronic charge-trapping mechanisms. We show that such neurons provide a nonlinear and rectified integration of information, that can be optically broadcast. Such a neuron has applications in machine learning, particularly in winner-take-all networks. We then apply such networks to simulations to establish unsupervised competitive learning for data partitioning, as well as cooperative learning in solving combinatorial optimization problems.

Centroid competitive learning approach for clustering and mapping the social vulnerability in Morocco

Over the last three decades, growing inequalities in countries have compounded the issues faced by society's most vulnerable populations. Students are facing the brunt of the effects in particular. A student's social vulnerability emerges as a result of the interaction of a variety of individual and environmental factors that accumulate over time. Poverty, material deprivation, and a lack of parental education can all have an impact on student assessment in school. Previous research has focused on the impact of psychological, cognitive, and physical functioning on children's education, ignoring students’ social vulnerability and its impact on educational achievements in developing countries. This paper aims to identify vulnerable regions that need attention and intervention by clustering Moroccan students based on their social vulnerability using an unsupervised competitive learning approach “Centroid neural network,” subsequently displaying the results in a choropleth map to visualize the results. For this purpose, we used the PISA dataset which contains the full set of responses from individual students focusing on specific information such as their parent’s backgrounds, socioeconomic position, and school conditions. Based on our current findings, we concluded that social vulnerability has a detrimental impact on students’ cognitive development. Moreover, the choropleth map shows that 'Beni Mellal-Khenifra' has the highest level of social vulnerability among all regions, besides "Marrakech-Safi" "Eddakhla-Oued Eddahab" and "Guelmim-Oued Noun" all of which have a high level of social vulnerability as well, urging academicians to incorporate resilience building into the design of policies in these regions in order to improve student’s educational outcomes.

Biologically motivated learning method for deep neural networks using hierarchical competitive learning

This study proposes a novel biologically motivated learning method for deep convolutional neural networks (CNNs). The combination of CNNs and backpropagation learning is the most powerful method in recent machine learning regimes. However, it requires a large amount of labeled data for training, and this requirement can occasionally become a barrier for real world applications. To address this problem and use unlabeled data, we introduce unsupervised competitive learning, which only requires forward propagating signals for CNNs. The method was evaluated on image discrimination tasks using the MNIST, CIFAR-10, and ImageNet datasets, and it achieved state-of-the-art performance with respect to other biologically motivated methods in the ImageNet benchmark. The results suggest that the method enables higher-level learning representations solely based on the forward propagating signals without the need for a backward error signal for training convolutional layers. The proposed method could be useful for a variety of poorly labeled data, for example, time series or medical data.

Search for H-Bonded Motifs in Liquid Ethylene Glycol Using a Machine Learning Strategy.

Trajectories of atomic positions derived from ab initio molecular dynamics (AIMD) simulations of H-bonded liquids contain a wealth of information on dominant structural motifs and recurrent patterns of association. Extracting this information from a detailed search of the trajectories over multiple time frames is, however, a daunting exercise. Here, we use a machine learning strategy based on the neural inspired approach of the self-organizing maps (SOM), a type of artificial neural network that uses unsupervised competitive learning, to analyze the AIMD trajectories of liquid ethylene glycol (EG). The objective was to find whether there are H-bonded fragments, of two or more H-bonded EG molecules, that are recurrent in the liquid and to identify them. The SOM represents a set of high-dimensional data mapped onto a two-dimensional, grid of neurons or nodes, while preserving the topological properties of the input space. We show here that clustering of the fragments by SOM in terms of the molecular conformation of the individual EG molecules of the fragment and their H-bond connectivity pattern facilitates the search for H-bonded motifs. Using this approach, we are able to identify a H-bonded cyclic dimer and a bifurcated H-bonded structure as recurring motifs that appear in the longer H-bonded fragments present in liquid EG.

Urban Crowd Detection Using SOM, DBSCAN and LBSN Data Entropy: A Twitter Experiment in New York and Madrid

The surfer and the physical location are two important concepts associated with each other in the social network-based localization service. This work consists of studying urban behavior based on location-based social networks (LBSN) data; we focus especially on the detection of abnormal events. The proposed crowd detection system uses the geolocated social network provided by the Twitter application programming interface (API) to automatically detect the abnormal events. The methodology we propose consists of using an unsupervised competitive learning algorithm (self-organizing map (SOM)) and a density-based clustering method (density-based spatial clustering of applications with noise (DBCSAN)) to identify and detect crowds. The second stage is to build the entropy model to determine whether the detected crowds fit into the daily pattern with reference to a spatio-temporal entropy model, or whether they should be considered as evidence that something unusual occurs in the city because of their number, size, location and time of day. To detect an abnormal event in the city, it is sufficient to determine the real entropy model and to compare it with the reference model. For the normal day, the reference model is constructed offline for each time interval. The obtained results confirm the effectiveness of our method used in the first stage (SOM and DBSCAN stage) to detect and identify clusters dynamically, and imitating human activity. These findings also clearly confirm the detection of special days in New York City (NYC), which proves the performance of our proposed model.

Simulation Algorithm Benefits by Connecting Geostatistics With Unsupervised Learning

This article, written by Special Publications Editor Adam Wilson, contains highlights of paper SPE 190087, “Unsupervised Statistical Learning With Integrated Pattern-Based Geostatistical Simulation,” by Q. Li and R. Aguilera, SPE, University of Calgary, prepared for the 2018 SPE Western Regional Meeting, Garden Grove, California, USA, 22–27 April. The paper has not been peer reviewed. This paper presents a new geostatistics modeling methodology that connects geostatistics and machine-learning methodologies, uses nonlinear topological mapping to reduce the original high-dimensional data space, and uses unsupervised-learning algorithms to bypass problems with supervised-learning algorithms. The algorithm presented is a neural topology-preserving pattern-based geostatistical simulation algorithm that integrates the self-organizing map (SOM) concept and its updated version—growing self-organizing map (GSOM)—with an unsupervised competitive learning structure. Introduction In oil and gas reservoir modeling, any model construction faces challenges of limited data to some extent. The heuristic behind all geostatistical techniques is the implicit existence of statistical relationships among available data. “Data” here is a broad term; it could be discrete points, such as porosity or permeability at certain locations, but it also could be training images (TIs), which are used in this work. Using TIs as input data originated with multiple-point geostatistics. The aim was to overcome the limitations of using traditional two-point statistical variograms to describe geological continuity, especially in the case of curvilinear structures, which are quite common in nature, such as in fracture networks and geological fluvial structures. The authors write that geostatistics could benefit from the machine-learning or statistical-learning areas. Machine-learning tasks can be divided into two protocols, supervised learning and unsupervised learning. The difference depends on whether the input data have correct labels or not. For the investigation considered in this paper, after retrieving image patches from TIs, machine-learning algorithms were used to cluster those image patches into different classifications. If the correct clusters to which the image patches belong is known beforehand, the data can be said to have correct labels. Further, a large amount of these labeled data could be used for a model to learn. Supervised learning occurs when the model is finely tuned by guidance of these correct labels using an error-correction process. Unsupervised learning, on the other hand, occurs when neither how many clusters exist nor the correct clusters to which the image patches belong is known. Thus, without correct labels for guidance, this problem is identified as an unsupervised-learning problem. This is the first major drawback of applying machine-learning algorithms to geostatistical simulations. Two other important issues are encountered when performing pattern-based geostatistical simulations. The first is related to the large number of image patches. This large image-patch database contains high pattern redundancy, which could make pattern similarity comparison inefficient. The second issue is the high dimensionality of each image patch, which could typically be described by low-dimensional nonlinear-structure manifolds. To model and visualize these high-dimensional data, a nonlinearity dimensionality reduction should be sought.

Fault Diagnosis and Application Based on SOM Neural Network

In the actual demand of traction, the field of fault diagnosis technology applications is more and more widely. SOM (Self-organizing Feature Map) neural network is an unsupervised competitive learning; it uses a self-learning mode without teachers, without the need for training or learning process in advance to specify the training input data belongs to the category. Based on the characteristics of SOM neural network, a typical fault data is used to simulate the fault diagnosis and the simulation results show that the proposed neural network can accomplish the fault diagnosis task well.

The EIS‐based Kohonen neural network for high strength steel coating degradation assessment

Electrochemical impedance spectroscopy (EIS) method is used for a long‐term and in‐depth study on the failure analysis of polymer coatings. With the assistance of neural networks, a deeper insight into the changing states of corrosion during certain exposure circumstances has been investigated by applying specific Kohonen intelligent learning networks. The Kohonen artificial network has been trained by using 4 sets of samples from sample 1# to sample 4# with unsupervised competitive learning methods. Each sample includes up to 14 cycles of EIS data. The trained network has been tested using sample 0# impedance data at 0.1 Hz. All the sample data were collected during exposure to accelerated corrosion environments, and it took the changing rate of impedance of each cycle as an input training sample. Compared with traditional classification, Kohonen artificial network method classifies corrosion process into 5 subprocesses, which is refinement of 3 typical corrosion processes. The 2 newly defined subprocesses of corrosion, namely, premiddle stage and postmiddle stage were introduced. The EIS data and macro‐morphology for both subprocesses were analyzed through accelerated experiments that considered general atmospheric environmental factors such as UV radiation, thermal shock, and salt fog. The classification results of Kohonen artificial network are highly consistent with the predictions based on impedance magnitude at low frequency, which illustrates that the Kohonen network classification is an effective method to predict the failure cycles of polymer coatings.

Compact Model of HfO<italic>X</italic>-Based Electronic Synaptic Devices for Neuromorphic Computing

HfOx-based resistive switching device has been explored as one of the promising candidates for the electronic synapses of neuromorphic computing systems due to its high performance, low cost, and compatibility with CMOS technology. To meet the codesign requirement of HfOx-based electronic synapses with CMOS neurons in the neuromorphic computing systems, a compact model that can capture the synaptic futures of HfOx-based resistive switching device is developed. The developed model can accurately describe the multilevel conductance transition behaviors during RESET process for depression learning as well as the binary stochastic transition behavior during SET process for potentiation learning. After the verification with experimental data, the model is used to simulate a winner-take-all neural network to classify patterns with unsupervised competitive learning algorithm. Simulation results imply that the average recognition accuracy would decrease with the increase of the resistance variation of low resistance state (LRS) due to the “trap” effect. Guided by the simulation, a synapse cell consisted of a HfO X -based device and a fixed resistor series connection is proposed to achieve almost 100% recognition accuracy even if the resistance variation of LRS is 50%.

The Development of Hand-Centered Visual Representations in the Primate Brain: A Computer Modeling Study Using Natural Visual Scenes.

Neurons that respond to visual targets in a hand-centered frame of reference have been found within various areas of the primate brain. We investigate how hand-centered visual representations may develop in a neural network model of the primate visual system called VisNet, when the model is trained on images of the hand seen against natural visual scenes. The simulations show how such neurons may develop through a biologically plausible process of unsupervised competitive learning and self-organization. In an advance on our previous work, the visual scenes consisted of multiple targets presented simultaneously with respect to the hand. Three experiments are presented. First, VisNet was trained with computerized images consisting of a realistic image of a hand and a variety of natural objects, presented in different textured backgrounds during training. The network was then tested with just one textured object near the hand in order to verify if the output cells were capable of building hand-centered representations with a single localized receptive field. We explain the underlying principles of the statistical decoupling that allows the output cells of the network to develop single localized receptive fields even when the network is trained with multiple objects. In a second simulation we examined how some of the cells with hand-centered receptive fields decreased their shape selectivity and started responding to a localized region of hand-centered space as the number of objects presented in overlapping locations during training increases. Lastly, we explored the same learning principles training the network with natural visual scenes collected by volunteers. These results provide an important step in showing how single, localized, hand-centered receptive fields could emerge under more ecologically realistic visual training conditions.

3D model reconstruction using neural gas accelerated on GPU

In this work, we propose the use of the neural gas (NG), a neural network that uses an unsupervised Competitive Hebbian Learning (CHL) rule, to develop a reverse engineering process. This is a simple and accurate method to reconstruct objects from point clouds obtained from multiple overlapping views using low-cost sensors. In contrast to other methods that may need several stages that include downsampling, noise filtering and many other tasks, the NG automatically obtains the 3D model of the scanned objects. To demonstrate the validity of our proposal we tested our method with several models and performed a study of the neural network parameterization computing the quality of representation and also comparing results with other neural methods like growing neural gas and Kohonen maps or classical methods like Voxel Grid. We also reconstructed models acquired by low cost sensors that can be used in virtual and augmented reality environments for redesign or manipulation purposes. Since the NG algorithm has a strong computational cost we propose its acceleration. We have redesigned and implemented the NG learning algorithm to fit it onto Graphics Processing Units using CUDA. A speed-up of 180× faster is obtained compared to the sequential CPU version.

Event signature extraction and traffic modeling in WSNs

Motivated by earlier work on adaptive event forecasting, this paper proposes an iterative event signature extraction method for wireless sensor networks, and a probabilistic approach to model non i.i.d. (independent and identically distributed) aperiodic traffic. The model is validated on well described (collection included) real world measurements of various types, and it is then used to demonstrate the effectiveness of the proposed signature extraction method to support reliable event forecasting. Our scheme can continuously keep the event signature database low on artifacts, dynamically estimate the number of sequences and extract the signatures from noisy, overlapped events. The proposed solution is inspired by unsupervised competitive Hebbian learning used in self-organizing Kohonen maps. We evaluate the proposed solution analytically, but also empirically, by means of simulations.

Noise-enhanced clustering and competitive learning algorithms

Noise can provably speed up convergence in many centroid-based clustering algorithms. This includes the popular k-means clustering algorithm. The clustering noise benefit follows from the general noise benefit for the expectation–maximization algorithm because many clustering algorithms are special cases of the expectation–maximization algorithm. Simulations show that noise also speeds up convergence in stochastic unsupervised competitive learning, supervised competitive learning, and differential competitive learning.

PRINCIPAL COMPONENT ANALYSIS AND SELF-ORGANIZING MAP FOR VISUAL CLUSTERING OF MACHINE-PART CELL FORMATION IN CELLULAR MANUFACTURING SYSTEM

The present paper attempts to generate visual clustering and data extraction of cell formation problem using both principal component analysis (PCA) and self organizing map (SOM) from input of sequence based machine-part incidence matrix. First, the focus is to utilize PCA for extracting high dimensionality of input variables and project the dataset onto a 2-D space. Second, the unsupervised competitive learning of SOM algorithm is used for data visualization and subsequently, to solve cell formation problem based on ordinal sequence data via the node cluster on the SOM map. Although the numerically illustrated results from dataset revealed that PCA has explained most of the cumulative variance of data but in reality when the very large dimensional cell formation problem based on sequence is available then to obtain the clustering structure from PCA projection is become very difficult. Most importantly, in the visual clustering of ordinal data, the use of U-matrix alone can not be efficient to get the cluster structure but with color extraction, hit map, labeling via the SOM node map it becomes a powerful clustering visualization methodology and thus the present research contribute significantly in the research of cellular manufacturing.

Using unsupervised incremental learning to cope with gradual concept drift

Current computational approach to incremental learning requires a constant stream of labelled data to cope with gradual environmental changes known as concept drift. This paper examines a case where labelled data are unavailable. Inspired by the performance of the human visual system, capable of adjusting its concepts using unlabelled stimuli, we introduce a variant to an unsupervised competitive learning algorithm known as the Leader Follower (LF). This variant can adjust pre-learned concepts to environmental changes using unlabelled data samples. We motivate the needed change in the existing LF algorithm and compare between two variants to enable the accumulation of environmental changes when facing unbalanced sample ratio.

Extraction of customer attitude using neural network and canonical correlation analysis

This paper describes a method that extracts customer attitude toward their choices of web pages and commodities from their web search histories using neural network (NN) and canonical correlation analysis (CCA). Customers visit websites and leave behind valuable information about their behaviour. Customer behaviour analysis aims to ultimately improve business performance through an understanding of past and present search histories of customers so as to determine and identify attitude of customers. Online traders often want the interesting websites which are attractive to their customers. In order to do that, firstly, we generate the equal length patterns from the heterogeneous search patterns in order to facilitate the further analysis. Secondly, we apply an unsupervised competitive NN learning to cluster the customers. Thirdly, we apply a statistics CCA in order to extract the attitude and/or interest of the customers towards the web pages and commodities.

Autonomous diagnostics and prognostics in machining processes through competitive learning-driven HMM-based clustering

A prerequisite to widespread deployment of condition-based maintenance (CBM) systems in industry is autonomous yet effective diagnostics and prognostics algorithms. The concept of ‘autonomy’ in the context of diagnostics and prognostics is usually based on unsupervised clustering techniques. This paper employs an unsupervised competitive learning algorithm to perform hidden Markov model (HMM) based clustering of multivariate temporal observation sequences derived from sensor signal(s). This method improves autonomy of the diagnostic engine over the more traditional classifier based diagnostics models. Classifier models, such as the model presented by Baruah and Chinnam [Baruah, P. and Chinnam, R.B., 2005. HMM for diagnostics and prognostics in machining processes. International Journal of Production Research, 43 (6), 1275–1293] employ ‘labelled’ feature vectors for supervised model building and subsequent health-state classification during condition monitoring. Improving the autonomy of the diagnostics model also improves the autonomy of the prognostics module that often builds upon information extracted through the diagnostics module. We have validated the proposed methods on a physical test-bed that involved monitoring drill-bits on a CNC machine outfitted with thrust-force and torque sensors. Experimental results demonstrate the ability of this method to estimate on-line the remaining-useful-life of a drill-bit with significant accuracy.

Modeling multisensory enhancement with self-organizing maps.

Self-organization, a process by which the internal organization of a system changes without supervision, has been proposed as a possible basis for multisensory enhancement (MSE) in the superior colliculus (Anastasio and Patton, 2003). We simplify and extend these results by presenting a simulation using traditional self-organizing maps, intended to understand and simulate MSE as it may generally occur throughout the central nervous system. This simulation of MSE: (1) uses a standard unsupervised competitive learning algorithm, (2) learns from artificially generated activation levels corresponding to driven and spontaneous stimuli from separate and combined input channels, (3) uses a sigmoidal transfer function to generate quantifiable responses to separate inputs, (4) enhances the responses when those same inputs are combined, (5) obeys the inverse effectiveness principle of multisensory integration, and (6) can topographically congregate MSE in a manner similar to that seen in cortex. Thus, the model provides a useful method for evaluating and simulating the development of enhanced interactions between responses to different sensory modalities.

A pattern reordering approach based on ambiguity detection for online category learning

Pattern reordering is proposed as an alternative to sequential and batch processing for online category learning. Upon detecting that the categorization of a new input pattern is ambiguous, the input is postponed for a predefined time, after which it is reexamined and categorized for good. This approach is shown to improve the categorization performance over purely sequential processing, while yielding a shorter input response time, or latency, than batch processing. In order to examine the response time of processing schemes, the latency of a typical implementation is derived and compared to lower bounds. Gaussian and softmax models are derived from reject option theory and are considered for detecting ambiguity and triggering pattern postponement. The average latency and Rand Adjusted clustering score of reordered, sequential, and batch processing are compared through computer simulation using two unsupervised competitive learning neural networks and a radar pulse data set.