Connectionist Systems Research Articles

Коннекционистское расширение минимальной модели вычислений (Часть 1)

We analyzed the modern variants of connectionist systems. We singled out the five main variants: cellular automata, neural networks, cellular nonlinear network, computing clusters and Grid. The connectionism is considered as a synthesis of computing parallelism and spatially distributed system. Its description is based on Rummelhart principles. Computability similarities are indicated at least at the application level, i. e. at the level of computational methods in connectionist systems. Nontraditional computing, in particular, within the connectionist systems, requires an expansion of the classical von Neumann computing model, first of all the recognition of its nonabsoluteness. The necessity of transition to higher level of abstraction and elaboration of the term “computation model” are postulated. For the first time the issue of irreducible computation model is raised and its set-theoretic formalization is given. This one assumes five basic sets and five basic relations.

Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: Design methodology and selected applications

The paper describes a new type of evolving connectionist systems (ECOS) called evolving spatio-temporal data machines based on neuromorphic, brain-like information processing principles (eSTDM). These are multi-modular computer systems designed to deal with large and fast spatio/spectro temporal data using spiking neural networks (SNN) as major processing modules. ECOS and eSTDM in particular can learn incrementally from data streams, can include ‘on the fly’ new input variables, new output class labels or regression outputs, can continuously adapt their structure and functionality, can be visualised and interpreted for new knowledge discovery and for a better understanding of the data and the processes that generated it. eSTDM can be used for early event prediction due to the ability of the SNN to spike early, before whole input vectors (they were trained on) are presented. A framework for building eSTDM called NeuCube along with a design methodology for building eSTDM using this is presented. The implementation of this framework in MATLAB, Java, and PyNN (Python) is presented. The latter facilitates the use of neuromorphic hardware platforms to run the eSTDM. Selected examples are given of eSTDM for pattern recognition and early event prediction on EEG data, fMRI data, multisensory seismic data, ecological data, climate data, audio-visual data. Future directions are discussed, including extension of the NeuCube framework for building neurogenetic eSTDM and also new applications of eSTDM.

Spam E-mail Filtering using ECOS Algorithms

Spam known as Unsolicited Bulk E-mail (UBE) including undesirable electronic correspondence that is sent in bulk to massive mailing lists, sometimes with some business nature sent get into bulk. One of the main problems of spam E-mail detection is an attack from an unknown source known as ‘zero-day’ so named due to continued changes in timing. Zero day attacks are problematic for having the ability to escape spamming detection as the hosts do not show up in blacklists. This adds to the problem of False positives and OCR computational cost especially in dealing with a huge corpus of texts with images that run through server-side filters. Spammers are busy orchestrating various representation techniques thus making their ‘zero-day’ spam E-mail that infiltrate the defenses of detection. Our proposed is a novel system called Spamming Dynamic Evolving Neural Fuzzy System (SDENFS), which adapts the Evolving Connectionist System (ECoS) based on a hybrid (supervised/unsupervised) learning approach. SDENFS adaptive online is enhanced by offline learning to detect dynamically the spamming E-mail included unknown zero-day spamming E-mails before it get to user account. SDENFS is suggested to work for high-speed “life-long” learning with low memory footprint with few number of rules creation for E-mail classification. Two datasets composed of 6612 samples of spam and legitimate E-mails were used to assess the proposed system. The proposed system showed a high level of performance in detecting spam E-mail attacks. The average of the accuracy and F-measure of the classification process was 99%.

Evolving connectionist systems for adaptive learning and knowledge discovery: Trends and directions

This paper follows the 25years of development of methods and systems for knowledge-based neural network systems and more specifically the recent evolving connectionist systems (ECOS). ECOS combine the adaptive/evolving learning ability of neural networks and the approximate reasoning and linguistically meaningful explanation features of symbolic representation, such as fuzzy rules. This review paper presents the classical now hybrid expert systems and evolving neuro-fuzzy systems, along with new developments in spiking neural networks, neurogenetic systems, and quantum inspired systems, all discussed from the point of few of their adaptability, model interpretability and knowledge discovery. The paper discusses new directions for the integration of principles from neural networks, fuzzy systems, bio- and neuroinformatics, and nature in general.

Targets Classification on Multispectral Images using Connectionists Methods

Identification of targets on remote sensing images depends mainly on the databases representing different classes. In this context, this paper proposes a connectionist system using a two-dimensional Kohonen self-organizing map to build a database of some identified targets on a multi-band satellite image. After an enhancing process, essentially based on a non-linear filtering, the system performs a non-supervised classification of a reference image in order to extract the sequence of samples of the reflectance coefficients related to the desired targets. This classification will be used later for automatic extraction of these targets on other stages.

Intelligent Modeling and Analysis of Moisture Sorption Isotherms in Milk and Pearl Millet–Based Weaning Food “Fortified Nutrimix”

Soft computing–based intelligent models have been proposed to predict moisture sorption isotherms in milk and pearl millet–based weaning food, “fortified Nutrimix,” at four temperatures, 15, 25, 35, and 45°C over the water activity range 0.11–0.97. Connectionist and adaptive neuro-fuzzy inference system (ANFIS) models were investigated. A back-propagation algorithm with Bayesian regularization/Levenberg-Marquardt optimization mechanisms was employed to develop connectionist models. The ANFIS model was based on the Sugeno-type fuzzy inference system. In addition, several empirical models were explored for fitting the sorption data. The soft computing models, in particular, ANFIS, outperformed the conventional sorption models for predicting isotherms in Nutrimix.

Infusing Intent and Its Management into Turing Machine: A Path to Cognitive Distributed Computing

The holy grail of Artificial Intelligence (AI) has been the reproduction of the cognitive processes in Silicon based computing machines to mimic the human or animal cognitive behavior. Computationalismattempts to explain cognition in terms of different internal representations and symbol-manipulating processes operating over these representations. On the other hand, Connectionism explains cognition in terms of a finite automaton which computes associative function specified by a set of input -output pairs that in turn, introduce interaction into the computing model. However, both theories have critics who believe that they fall short in explaining the cognitive processes observed either in humans or ani mals. Computationalism tied to the Turing computing model which is restricted to single, sequential processes does not support concurrency, mobility and synchronization observed in cognitive processes. The connectionist systems are also dedicated to singletasks and do not support complex environments which require behaviors that are coordinated and integrated. Above all, cognition is associated with intent of a system and its accomplishment efficiently through various processes that monitor and control its elf and its environment. Any computing model incorporating cognition must accommodate dynamic coupling between various elements of the system,

Adaptive connectionist systems for engineering applications

Adaptive Connectionist Systems refer to systems that change with time through interaction with the environment, learning through the utilization of artificial neural networks (ANN). Many engineering applications require the use of real-time decision-making and adaptation to new input. In this context a number of researchers have explored the use of adaptive connectionist systems and their applicability to solve engineering problems. This special issue includes selected contributions from the 13th Engineering Applications of Neural Network conference held in London, UK in 2012 (EANN 2012). The EANN conferences promote neural networks and associated techniques and the significant benefits that can be derived from their use. The conferences are not only for reporting advances, but also for showing how neural networks provide practical solutions in a wide range of applications. The first paper ‘‘Neurocomputing techniques to dynamically forecast spatiotemporal air pollution data’’ by Antonios Papaleonidas and Lazaros Iliadis addresses the application of ANNs to the problem of forecasting dangerous levels of Ozone (O3) concentrations in the city center of Athens and more specifically in the ‘‘Athinas’’ air quality monitoring station. While most existing approaches develop static ANN models optimal only under specific conditions, this work suggests a dynamic flexible and adjustable neural spatiotemporal model. The proposed approach takes in account not only the values of the primitive air pollutants and meteorological measurements, but it is also evaluates the influence of the existing O3 values in monitoring stations close to the one under examination. The results suggest that the optimal ANN model is using data from a small number of measurement’s stations and it is capable of estimating O3 concentrations in real time mod. It also has the capacity of forecasting the same values for future time intervals. The second paper ‘‘Information dynamics based selfadaptive reservoir for delay temporal memory tasks’’ by Sakyasingha Dasgupta, Florentin Worgotter and Poramate Manoonpong, investigates the application of recurrent neural networks of the reservoir computing (RC) type for time-series processing tasks and more specifically for delayed response tasks involving the transient memorization of information (temporal memory). The authors combine a generalized intrinsic plasticity rule with a local information dynamics based schema of reservoir neuron leak adaptation. Utilizing this combination they propose a self-adaptive reservoir (RNN) framework that maintains homeostasis through the generic intrinsic plasticity mechanism. The suggested framework has been evaluated on two standard benchmark tasks and the results demonstrate that the adaptation mechanism clearly outperforms static reservoirs. The authors also demonstrate the application of the self-adaptive reservoir (RNN) to the control of autonomous robotic agents through maze navigation experiments. The results show that the proposed network is effective for reconstructing the original trajectories and has the ability to cope with the variable temporal delay memory problem. The third paper ‘‘Exploratory analysis of mobile phone traffic patterns using 1-dimensional Self Organizing Map (SOM), clustering and anomaly detection’’ by Pekka Kumpulainen, considers the problem of recognizing traffic patterns in continuously evolving mobile networks. This is C. Jayne (&) Faculty of Engineering and Computing, Coventry University, Room EC306, Coventry CV1 5FB, UK e-mail: ab1527@coventry.ac.uk; Chrisina.Jayne2@coventry.ac.uk

Biochemical connectionism

In this paper, we discuss computational architectures that are motivated by connectionist patterns that occur in biochemical networks, and speculate about how this biochemical approach to connectionism might complement conventional neural approaches. In particular, we focus on three features of biochemical networks that make them distinct from neural networks: their diverse, complex nodal processes, their emergent organisation, and the dynamical behaviours that result from higher-order, self-modifying processes. We also consider the growing use of evolutionary algorithms in the design of connectionist systems, noting how this enables us to explore a wider range of connectionist architectures, and how the close relationship between biochemical networks and biological evolution can guide us in this endeavour.

The formation of categories and the representation of feature saliency: Analysis with a computational model trained with an Hebbian paradigm

An important issue in semantic memory models is the formation of categories and taxonomies, and the different role played by shared vs. distinctive and salient vs. marginal features. Aim of this work is to extend our previous model to critically discuss the mechanisms leading to the formation of categories, and to investigate how feature saliency can be learned from past experience. The model assumes that an object is represented as a collection of features, which belong to different cortical areas and are topologically organized. Excitatory synapses among features are created on the basis of past experience of object presentation, with a Hebbian paradigm, including the use of potentiation and depression of synapses, and thresholding for the presynaptic and postsynaptic. The model was trained using simple schematic objects as input (i.e., vector of features) having some shared features (so as to realize a simple category) and some distinctive features with different frequency. Three different taxonomies of objects were separately trained and tested, which differ as to the number of correlated features and the structure of categories. Results show that categories can be formed from past experience, using Hebbian rules with a different threshold for postsynaptic and presynaptic activity. Furthermore, features have a different saliency, as a consequence of their different frequency during training. The trained network is able to solve simple object recognition tasks, by maintaining a distinction between categories and individual members in the category, and providing a different role for salient features vs. not-salient features. In particular, not-salient features are not evoked in memory when thinking about the object, but they facilitate the reconstruction of objects when provided as input to the model. The results can provide indications on which neural mechanisms can be exploited to form robust categories among objects and on which mechanisms could be implemented in artificial connectionist systems to extract concepts and categories from a continuous stream of input objects (each represented as a vector of features).

Phishing Dynamic Evolving Neural Fuzzy Framework for Online Detection “Zero-day” Phishing Email

Phishing is a kind of attack in which criminals use spoofed emails and fraudulent web sites to trick financial organization and customers. Criminals try to lure online users by convincing them to reveal the username, passwords, credit card number and updating account information or fill billing information. One of the main problems of phishing email detection is the unknown “zero-day” phishing attack, (we define zero-day attacks as attacks that phisher mount using hosts that do not appear in blacklists and not trained on the old data sample and it is a noise data), which increases the level of difficulty to detect phishing email. Nowadays, phishers are creating different representation techniques to create unknown “zero-day” phishing email to breach the defenses of those detectors. Our proposed is a novel framework called phishing dynamic evolving neural fuzzy framework (PDENF), which adapts the evolving connectionist system (ECoS) based on a hybrid (supervised/unsupervised) learning approach. PDENF adaptive online is enhanced by offline learning to detect dynamically the phishing email included unknown zero-day phishing e-mails before it get to user account. PDENF is suggested to work for high-speed “life-long” learning with low memory footprint and minimizes the complexity of the rule base and configuration with few number of rules creation for email classification. We expect to achieves high performance, including high level of true positive, true negative, sensitivity, precision, F-measure and overall accuracy compared with other approaches.

Accounting for Agency in a Connectionist World: A Response to Kati Breckenridge

In this meditation on the relationship between the constraints of connectionist systems on one side, and the possibility of human freedom on the other, the author suggests that a sense of agency emerges in the development of the reflective mind and imagination, a process that psychoanalysis facilitates and encourages.

Using connectionist systems for electric energy consumption forecasting in shopping centers

A solution is presented for the short-term electrical energy forecasting in shopping centers located in the Netherlands and Belgium. A forecasting method is proposed on the basis of connectionist systems. General description of the forecasting method is provided, as well as its specific features with respect to the forecasting problem are studied. Several connectionist models are generalized, stated and applied, notably, moving average model, linear regression model, and neural network model. In addition, changes in forecasting quality are demonstrated depending on different input variables. The results of using these connectionist models are discussed, and conclusions regarding specific features of every model are outlined.

ARTdECOS, adaptive evolving connectionist model and application to heart rate variability

ARTdECOS is a new artificial neural network model, which incorporates fuzzy adaptive resonance theory (ART) at its core and implements evolving connectionist system (ECOS) methodology to enable knowledge extraction from real time, ongoing processes. The model creates category clusters based on resonance threshold conditions with the fuzzy ART core. After a number of input presentations the model amalgamates categories based upon the fuzzy similarity of weight vectors. Four new parameters are introduced to control and evaluate category amalgamation: weight activation threshold, weight resonance threshold, sum of weight differences and minimum number of categories. A graphical visualization tool is presented that depicts category rules and their evolution. Further, a method is presented to normalize input data in real time without setting feature minimum and maximum levels a priori. ARTdECOS is shown to satisfactorily classify instances from the IRIS dataset into three categories in an unsupervised mode. ARTdECOS is implemented for examples of heart rate interval time series. Heart rate variability features are derived based upon fractal and time domains. Segmentation of the heart rate time series data is shown by category selection.

Bituminous mix characterisation: evolving intelligent systems

This paper describes a novel implementation of evolving connectionist systems (Ecos) based predictive modelling of the dynamic modulus (|E*|) of hot-mix asphalt, a construction material used in preparing the surface layer of roads. Ecos are open-architecture artificial neural networks with a dynamically evolving structure that is automatically adjusted by the training algorithm thereby overcoming the limitations associated with conventional and constructive artificial neural networks. Two Ecos methods are investigated – the dynamic evolving neuro-fuzzy inference system (Denfis) and the evolving fuzzy neural network (EFuNN). Preliminary investigations show that Ecos-based hot-mix asphalt |E*| predictive modelling is a promising alternative to regression-based methods, although not as powerful as traditional multi-layered perceptron back-propagation artificial neural networks.

Secuencia y delimitación en el procesamiento textual alemán: modelo conexionista.

In order to simulate human speech, connectionism has developed several techniques for representation of frames and scripts. These models are able to solve some of the shortcomings of classical cognitive systems. Specifically, classical systems are unable to control when a chunk of knowledge has to be activated and what part of the chunk should be more activated. Instead, connectionist systems can use degrees in mobilizing to try this case. However, connectionism has also some limitations if considered as models of human mind, but they also are a point of interest to the application in the prosecution textual. Besides, we examinate the plausibility of any of these approaches used in computers and their implications in our own connectionist model.

Computational Models of Neuron-Astrocyte Interactions Lead to Improved Efficacy in the Performance of Neural Networks

The importance of astrocytes, one part of the glial system, for information processing in the brain has recently been demonstrated. Regarding information processing in multilayer connectionist systems, it has been shown that systems which include artificial neurons and astrocytes (Artificial Neuron-Glia Networks) have well-known advantages over identical systems including only artificial neurons. Since the actual impact of astrocytes in neural network function is unknown, we have investigated, using computational models, different astrocyte-neuron interactions for information processing; different neuron-glia algorithms have been implemented for training and validation of multilayer Artificial Neuron-Glia Networks oriented toward classification problem resolution. The results of the tests performed suggest that all the algorithms modelling astrocyte-induced synaptic potentiation improved artificial neural network performance, but their efficacy depended on the complexity of the problem.

Mobility Timing for Agent Communities, a Cue for Advanced Connectionist Systems

We introduce a wait-and-chase scheme that models the contact times between moving agents within a connectionist construct. The idea that elementary processors move within a network to get a proper position is borne out both by biological neurons in the brain morphogenesis and by agents within social networks. From the former, we take inspiration to devise a medium-term project for new artificial neural network training procedures where mobile neurons exchange data only when they are close to one another in a proper space (are in contact). From the latter, we accumulate mobility tracks experience. We focus on the preliminary step of characterizing the elapsed time between neuron contacts, which results from a spatial process fitting in the family of random processes with memory, where chasing neurons are stochastically driven by the goal of hitting target neurons. Thus, we add an unprecedented mobility model to the literature in the field, introducing a distribution law of the intercontact times that merges features of both negative exponential and Pareto distribution laws. We give a constructive description and implementation of our model, as well as a short analytical form whose parameters are suitably estimated in terms of confidence intervals from experimental data. Numerical experiments show the model and related inference tools to be sufficiently robust to cope with two main requisites for its exploitation in a neural network: the nonindependence of the observed intercontact times and the feasibility of the model inversion problem to infer suitable mobility parameters.

Artificial Astrocytes Improve Neural Network Performance

Compelling evidence indicates the existence of bidirectional communication between astrocytes and neurons. Astrocytes, a type of glial cells classically considered to be passive supportive cells, have been recently demonstrated to be actively involved in the processing and regulation of synaptic information, suggesting that brain function arises from the activity of neuron-glia networks. However, the actual impact of astrocytes in neural network function is largely unknown and its application in artificial intelligence remains untested. We have investigated the consequences of including artificial astrocytes, which present the biologically defined properties involved in astrocyte-neuron communication, on artificial neural network performance. Using connectionist systems and evolutionary algorithms, we have compared the performance of artificial neural networks (NN) and artificial neuron-glia networks (NGN) to solve classification problems. We show that the degree of success of NGN is superior to NN. Analysis of performances of NN with different number of neurons or different architectures indicate that the effects of NGN cannot be accounted for an increased number of network elements, but rather they are specifically due to astrocytes. Furthermore, the relative efficacy of NGN vs. NN increases as the complexity of the network increases. These results indicate that artificial astrocytes improve neural network performance, and established the concept of Artificial Neuron-Glia Networks, which represents a novel concept in Artificial Intelligence with implications in computational science as well as in the understanding of brain function.

Knowledge-based evolving connectionist systems for condition evaluation of sustainable roadways: a feasibility study

In place, sustainable rehabilitation of existing deteriorated concrete highways through rubblization is considered to be a green and economical alternative to other options involving total reconstruction, etc. There is currently no standardised method of backcalculating the rubblized pavement layer moduli from pavement non-destructive test data through inverse analysis. This paper explores the feasibility of applying two evolving connectionist systems (ECOS), namely dynamic evolving neuro-fuzzy inference system (DENFIS) and evolving fuzzy neural network (EFuNN), to rubblized pavement moduli backcalculation. It is advantageous to employ ECOS networks for analysing complex engineered systems such as rubblized pavements since ECOS networks are resistant to catastrophic forgetting, have the ability to adapt to and learn new data as soon as they become available, do not have a limit to the amount of knowledge they can store and learn the examples very quickly compared to traditional multi-layered perceptron backpropagation neural networks (MLP-BP NN).