Adaptive Resonance Theory Modules Research Articles

Applying deep learning in brain computer interface to classify motor imagery

Deep-learning (DL) is a new paradigm in the artificial intelligence field associated with learning structures able to connect directly numeric data with high-level patterns or categories. DL seems to be a suitable technique to deal with computationally challenging Brain Computer Interface (BCI) problems. Following DL strategy, a new modular and self-organized architecture to solve BCI problems is proposed. A pattern recognition system to translate the measured signals in order to establish categories representing thoughts, without previous pre-processing, is developed. To achieve an easy interpretability of the system internal functioning, a neuro-fuzzy module and a learning methodology are carried out. The whole learning process is based on machine learning. The architecture and the learning method are tested on a representative BCI application to detect and classify motor imagery thoughts. Data is gathered with a low-cost device. Results prove the efficiency and adaptability of the proposed DL architecture where the used classification module (S-dFasArt) exhibits a better behaviour compared with the usual classifiers. Additionally, it employs neuro-fuzzy modules which allow to offer results in a rules format. This improves the interpretability with respect to the black-box description. A DL architecture, going from the raw data to the labels, is proposed. The proposed architecture, based on Adaptive Resonance Theory (ART) and Fuzzy ART modules, performs data processing in a self-organized way. It follows the DL paradigm, but at the same time, it allows an interpretation of the operation stages. Therefore this approach could be called Transparent Deep Learning.

Distributed dual vigilance fuzzy adaptive resonance theory learns online, retrieves arbitrarily-shaped clusters, and mitigates order dependence

This paper presents a novel adaptive resonance theory (ART)-based modular architecture for unsupervised learning, namely the distributed dual vigilance fuzzy ART (DDVFA). DDVFA consists of a global ART system whose nodes are local fuzzy ART modules. It is equipped with distributed higher-order activation and match functions and a dual vigilance mechanism. Together, these allow DDVFA to perform unsupervised modularization, create multi-prototype cluster representations, retrieve arbitrarily-shaped clusters, and reduce category proliferation. Another important contribution is the reduction of order-dependence, an issue that affects any agglomerative clustering method. This paper demonstrates two approaches for mitigating order-dependence: pre-processing using visual assessment of cluster tendency (VAT) or post-processing using a novel Merge ART module. The former is suitable for batch processing, whereas the latter also works for online learning. Experimental results in online mode carried out on 30 benchmark data sets show that DDVFA cascaded with Merge ART statistically outperformed the best other ART-based systems when samples were randomly presented. Conversely, they were found to be statistically equivalent in offline mode when samples were pre-processed using VAT. Remarkably, performance comparisons to non-ART-based clustering algorithms show that DDVFA (which learns incrementally) was also statistically equivalent to the non-incremental (offline) methods of density-based spatial clustering of applications with noise (DBSCAN), single linkage hierarchical agglomerative clustering (SL-HAC), and k-means, while retaining the appealing properties of ART. Links to the source code and data are provided. Considering the algorithm’s simplicity, online learning capability, and performance, it is an ideal choice for many agglomerative clustering applications.

Behavioral similarity of residential customers using a neural network based on adaptive resonance theory

This article presents a methodology aiming at the comprehension and analysis of the residential electricity consumption habitual behavior by means of a similarity analysis, based on the use of an ART (Adaptive Resonance Theory) neural network, which is a neural network composed of two fuzzy ART modules whose training is performed in an unsupervised mode. ART neural networks are stable and plastic and these properties, combined with the processing of essentially binary data, give the neural system a wide capacity for producing objectives that may be easily modified to satisfy requirements predetermined by consumer. The expected result is to obtain information regarding the similarity of consumers. Thus, some benefits may be derived by consumers, such as improved habits of electricity consumption and better strategies for negotiating more favorable rates, especially in the case of smart grid systems. In this new electricity sector paradigm, there is a strong consumer trend for free choice among electricity suppliers. This methodology also benefits load forecasting studies at grid points, where there is greater uncertainty, e.g., the busbars that are closest to consumers, i.e., the uncertainties in the context of the total load forecasting system are increased from the global load to the final consumer.

Adaptive Resonance Theory-based Modular Networks for Incremental Learning of Hierarchical Clusterings

This paper introduces HART-S, a new modular neural network (NN) that can incrementally learn stable hierarchical clusterings of arbitrary sequences of input patterns by self-organization. The network is a cascade of adaptive resonance theory (ART) modules, in which each module learns to cluster the differences between the input pattern and the selected category prototype at the previous module. Input patterns are first classified into a few broad categories, and successive ART modules find increasingly specific categories until a threshold is reached, the level of which can be controlled by a global parameter called 'resolution'. The network thus essentially implements a divisive (or splitting) hierarchical clustering algorithm: hence the name HART-S (for 'hierarchical ART with splitting'). HART-S is also compared and contrasted with HART-J (for 'hierarchical ART with joining'), another variant that was proposed earlier by the first author. The network dynamics are specified and some useful properties of both networks are given and then proven. Experiments were carried out on benchmark data sets to demonstrate the representational and learning capabilities of both networks and to compare the developed clusterings with those of two classical methods and a conceptual clustering algorithm. A brief survey of related NN models is also provided.

Compound key word generation from document databases using a hierarchical clustering ART model

The growing availability of databases on the information highways motivates the development of new processing tools able to deal with a heterogeneous and changing information environment. A highly desirable feature of data processing systems handling this type of information is the ability to automatically extract its own key words. In this paper we address the specific problem of creating semantic term associations from a text database. The proposed method uses a hierarchical model made up of Fuzzy Adaptive Resonance Theory (ART) neural networks. First, the system uses several Fuzzy ART modules to cluster isolated words into semantic classes, starting from the database raw text. Next, this knowledge is used together with coocurrence information to extract semantically meaningful term associations. These associations are asymmetric and one-to-many due to the polisemy phenomenon. The strength of the associations between words can be measured numerically. Besides this, they implicitly define a hierarchy between descriptors. The underlying algorithm is appropriate for employment on large databases. The operation of the system is illustrated on several real databases.

Compound Key Word Generation from Document Databases Using A Hierarchical Clustering ART Model

The growing availability of databases on the information highways motivates the development of new processing tools able to deal with a heterogeneous and changing information environment. A highly desirable feature of data processing systems handling this type of information is the ability to automatically extract its own key words. In this paper we address the specific problem of creating semantic term associations from a text database. The proposed method uses a hierarchical model made up of Fuzzy Adaptive Resonance Theory (ART) neural networks. First, the system uses several Fuzzy ART modules to cluster isolated words into semantic classes, starting from the database raw text. Next, this knowledge is used together with coocurrence information to extract semantically meaningful term associations. These associations are asymmetric and one-to-many due to the polisemy phenomenon. The strength of the associations between words can be measured numerically. Besides this, they implicitly define a hierarchy between descriptors. The underlying algorithm is appropriate for employment on large databases. The operation of the system is illustrated on several real databases.

An ART-based modular architecture for learning hierarchical clusterings

This paper introduces a neural architecture (HART for “Hierarchical ART”) that is capable of learning hierarchical clusterings of arbitrary input sequences. The network is built up of layers of Adaptive Resonance Theory (ART) network modules where each layer learns to cluster the prototypes developed at the layer directly below it. The notion of effective vigilance is introduced to refer to the vigilance level of multiple ART modules in a HART network. An upper bound is derived for the number of HART layers needed in the case when all ART modules have the same vigilance. Experiments were carried out on a machine learning benchmark database to demonstrate the developed internal representation as well as some learning properties of two- and three-layer binary HART networks.