Autoassociative Neural Network Research Articles

Dimensionality Reduction with Input Training Neural Network and Its Application in Chemical Process Modelling

Many applications of principal component analysis (PCA) can be found in dimensionality reduction. But linear PCA method is not well suitable for nonlinear chemical processes. A new PCA method based on improved input training neural network (IT-NN) is proposed for the nonlinear system modelling in this paper. Momentum factor and adaptive learning rate are introduced into learning algorithm to improve the training speed of IT-NN. Contrasting to the auto-associative neural network (ANN), IT-NN has less hidden layers and higher training speed. The effectiveness is illustrated through a comparison of IT-NN with linear PCA and ANN with experiments. Moreover, the IT-NN is combined with RBF neural network (RBF-NN) to model the yields of ethylene and propylene in the naphtha pyrolysis system. From the illustrative example and practical application, IT-NN combined with RBF-NN is an effective method of nonlinear chemical process modelling.

Support for an auto-associative model of spoken cued recall: Evidence from fMRI

Cued recall and item recognition are considered the standard episodic memory retrieval tasks. However, only the neural correlates of the latter have been studied in detail with fMRI. Using an event-related fMRI experimental design that permits spoken responses, we tested hypotheses from an auto-associative model of cued recall and item recognition [Chappell, M., & Humphreys, M. S. (1994). An auto-associative neural network for sparse representations: Analysis and application to models of recognition and cued recall. Psychological Review, 101, 103–128]. In brief, the model assumes that cues elicit a network of phonological short term memory (STM) and semantic long term memory (LTM) representations distributed throughout the neocortex as patterns of sparse activations. This information is transferred to the hippocampus which converges upon the item closest to a stored pattern and outputs a response. Word pairs were learned from a study list, with one member of the pair serving as the cue at test. Unstudied words were also intermingled at test in order to provide an analogue of yes/no recognition tasks. Compared to incorrectly rejected studied items (misses) and correctly rejected (CR) unstudied items, correctly recalled items (hits) elicited increased responses in the left hippocampus and neocortical regions including the left inferior prefrontal cortex (LIPC), left mid lateral temporal cortex and inferior parietal cortex, consistent with predictions from the model. This network was very similar to that observed in yes/no recognition studies, supporting proposals that cued recall and item recognition involve common rather than separate mechanisms.

Extraction of speaker-specific excitation information from linear prediction residual of speech

In this paper, through different experimental studies we demonstrate that the excitation component of speech can be exploited for speaker recognition studies. Linear prediction (LP) residual is used as a representation of excitation information in speech. The speaker-specific information in the excitation of voiced speech is captured using the AutoAssociative Neural Network (AANN) models. The decrease in the error during training and recognizing correct speakers during testing demonstrates that the excitation component of speech contains speaker-specific information and is indeed being captured by the AANN models. The study on the effect of different LP orders demonstrates that for a speech signal sampled at 8 kHz, the LP residual extracted using LP order in the range 8–20 best represents the speaker-specific excitation information. It is also demonstrated that the proposed speaker recognition system using excitation information and AANN models requires significantly less amount of data both during training as well as testing, compared to the speaker recognition system using vocal tract information. Finally the speaker recognition studies on NIST 2002 database demonstrates that even though, the recognition performance from the excitation information alone is poor, when combined with evidence from vocal tract information, there is significant improvement in the performance. This result demonstrates the complementary nature of the excitation component of speech.

Clustering on the Net: Applying an autoassociative neural network to computer-mediated discussions

ProjectH, a research group of a hundred researchers, produced a huge amount of data from computer mediated discussions. The data classified several thousand postings from over 30 newsgroups into 46 categories. One approach to extract typical examples from this database is presented in this paper. An autoassociative neural network is trained on all 3000 coded messages and then used to construct typical messages under certain specified conditions. With this method the neural network can be used to create “typical” messages for several scenarios. This paper illustrates the architecture of the neural network that was used and explains the necessary modifications to the coding scheme. In addition several “typicality sets” produced by the neural net are shown and their generation is explained. In conclusion, the autoassociative neural network is used to explore threads and the types of messages that typically initiate or contribute longer lasting threads.

Nonlinear adaptive subspace SOM implemented by a modular network SOM

An extension of the adaptive subspace self-organizing map (ASSOM) for dealing with nonlinear subspaces is presented. A nonlinear ASSOM is implemented by a modular network SOM (mnSOM) with autoassociative neural network (ANN) modules. In this paper, the architecture and the algorithm of the nonlinear ASSOM based on an mnSOM are introduced, and the results of a simulation of texture image classification is presented.

Simulating the ‘Other-Race’ Effect with Autoassociative Neural Networks: Further Evidence in Favor of the Face-Space Model

Other-race (OR) faces are less accurately recognized than same-race (SR) faces, but faster classified by race. This phenomenon has often been reported as the 'other-race' effect (ORE). Valentine (1991 Quarterly Journal of Experimental Psychology A: Human Experimental Psychology 43 161-204) proposed a theoretical multidimensional face-space model that explained both of these results, in terms of variations in exemplar density between races. According to this model, SR faces are more widely distributed across the dimensions of the space than OR faces. However, this model does not quantify nor state the dimensions coded within this face space. The aim of the present study was to test the face-space explanation of the ORE with neural network simulations by quantifying its dimensions. We found the predicted density properties of Valentine's framework in the face-projection spaces of the autoassociative memories. This was supported by an interaction for exemplar density between the race of the learned face set and the race of the faces. In addition, the elaborated face representations showed optimal responses for SR but not for OR faces within SR face spaces when explored at the individual level, as gender errors occurred significantly more often in OR than in SR face-space representations. Altogether, our results add further evidence in favor of a statistical exemplar density explanation of the ORE as suggested by Valentine, and question the plausibility of such coding for faces in the framework of recent neuroimaging studies.

LARGE‐SCALE DATA VISUALIZATION WITH MISSING VALUES

Visualization of large‐scale data inherently requires dimensionality reduction to 1D, 2D, or 3D space. Autoassociative neural networks with a bottleneck layer are commonly used as a nonlinear dimensionality reduction technique. However, many real‐world problems suffer from incomplete data sets, i.e. some values can be missing. Common methods dealing with missing data include the deletion of all cases with missing values from the data set or replacement with mean or “normal” values for specific variables. Such methods are appropriate when just a few values are missing. But in the case when a substantial portion of data is missing, these methods can significantly bias the results of modeling. To overcome this difficulty, we propose a modified learning procedure for the autoassociative neural network that directly takes the missing values into account. The outputs of the trained network may be used for substitution of the missing values in the original data set.

The autoassociative neural network in signal analysis: III. Enhancing the reliability of a NN with application to a BWR

This is the third and last of a series of papers trying to unveil the opaqueness of neural networks structure through a geometrical approach [Marseguerra M., Zoia, A., 2005a. The autoassociative neural network in signal analysis: I. The data dimensionality reduction and its geometric interpretation. Ann. Nucl. Energy 32, 1191–1206, Marseguerra, M., Zoia, A., 2005b. The autoassociative neural network in signal analysis: II. Application to on-line monitoring of a simulated BWR component. Ann. Nucl. Energy 32, 1207–1223]. Artificial neural networks (NN) provide a powerful tool in the operation of complex systems, such as nuclear power plants, in that they are suitable to determine the relationship between measured variables and control parameters on the basis of input-output examples. However, their major drawback is the fact that they always provide an output to the user, regardless of the appropriateness of the input. In this paper, we propose to adopt an autoassociative neural network (AANN) to work in cooperation with the NN to first assess the well-posedness of the desired neural model and to successively establish the appropriateness of the input data. The neural algorithm has been applied to a nuclear problem: the estimation of the reactivity forcing function parameters from the values of the measured neutron flux in a BWR reactor (provided by a reduced-order literature model). In this example, the AANN was able to suggest through geometrical considerations how to decompose the dataset in order to obtain a successful training for the NN and thereafter to validate the input data, thus enhancing the reliability of the NN model output.

06/00936 The autoassociative neural network in signal analysis: I. The data dimensionality reduction and its geometric interpretation: Marseguerra, M. and Zoia, A. Annals of Nuclear Energy, 2005, 32, (11), 1191–1206.

06/00936 The autoassociative neural network in signal analysis: I. The data dimensionality reduction and its geometric interpretation: Marseguerra, M. and Zoia, A. Annals of Nuclear Energy, 2005, 32, (11), 1191–1206.

06/00937 The autoassociative neural network in signal analysis: II. Application to on-line monitoring of a simulated BWR component: Marseguerra, M. and Zoia, A. Annals of Nuclear Energy, 2005, 32, (11), 1207–1223.

06/00937 The autoassociative neural network in signal analysis: II. Application to on-line monitoring of a simulated BWR component: Marseguerra, M. and Zoia, A. Annals of Nuclear Energy, 2005, 32, (11), 1207–1223.

06/00934 sp 3/sp 2 characterization of carbon materials from first-principles calculations: X-ray photoelectron versus high energy electron energy-loss spectroscopy techniques : Titantah, J. T. and Lamoen, D. Carbon, 2005, 43, (6), 1311–1316.

In this study, an artificial autoassociative neural network (AANN) was used online to detect deviations from normal antibiotic production fermentation using conventional process variables. To improve the efficiency of extracting hidden information contained in multidimensional process variables, and to finally render the AANN adequate for fault detection, we explored the following methods: selection of process variables; preprocessing of data that involved normalizing the training data of the AANN; and evaluation of data that involved assessing the output of the AANN. A method for fault detection in virginiamycin M and S production by Streptomyces virginiae was successfully developed based on these techniques.

06/00938 The dynamic thermal characteristics of lightweight building elements with a forced ventilated cavity and radiation barriers: Cerne, B. and Medved, S. Energy and Buildings, 2005, 37, (9), 972–981.

A new procedure for classifying bioprocess performance using artificial neural networks is presented. Using historical records of processes with known good and bad outcomes as reference sets, patterns in the data are represented as data clusters generated from the bottleneck values of an autoassociative neural network (AANN). The strategy is similar to analyzing the cluster of points generated when plotting the first and second principal components. This method departs, however, from principal component analysis (PCA) in that it can handle data clouds generated from more than 2 variables and the classification is assessed quantitatively as opposed to visually. This is achieved by feeding the cloud patterns into a feed-forward network which compares the sample’s data clouds to reference sets. Using a classification scheme based on Soft Independent Modeling of Class Analogy (SIMCA), the net is able to classify a process as good, bad, neither, or both as early as possible.

Face Generation Using Emotional Regions for Sensibility Robot

We think that psychological interaction is necessary for smooth communication between robots and people. One way to psychologically interact with others is through facial expressions. Facial expressions are very important for communication because they show true emotions and feelings. The ``Ifbot'' robot communicates with people by considering its own ``emotions''. Ifbot has many facial expressions to communicate enjoyment. We developed a method for generating facial expressions based on human subjective judgements mapping Ifbot's facial expressions to its emotions. We first created Ifbot's emotional space to map its facial expressions. We applied a five-layer auto-associative neural network to the space. We then subjectively evaluated the emotional space and created emotional regions based on the results. We generated emotive facial expressions using the emotional regions.

Spectrum analysis by recursively pruned extended auto‐associative neural network

AbstractA method for automatic analysis of high‐dimensional spectra is presented. The method is based on a so‐termed extended auto‐associative neural network, which is an auto‐associative (bottleneck) neural network with an additional output that corresponds to the response variable of interest. The input of the neural network consists of the absorbance at selected wavelengths. Once the model has been calibrated, the contribution of each input is estimated, the least contributing input and the corresponding output are removed (pruned), and the training procedure is repeated. This procedure leads to a compact non‐linear mapping between the absorbance at a few wavelengths and the response variable of interest. The auto‐associative architecture of the model prevents the overparametrization and overfitting that occur in straightforward mapping between spectra and variable of interest. Overparametrization and overfitting are further reduced by retaining only the inputs required for predicting the variable of interest. The method is illustrated on two case studies involving mid‐infrared absorbance spectra. The results are compared to those obtained with partial least squares (PLS) and principal component analysis (PCA) followed by a neural network. In both case studies, the proposed approach leads to models with fewer parameters and smaller prediction errors. Copyright © 2006 John Wiley & Sons, Ltd.

The AutoAssociative Neural Network in signal analysis: II. Application to on-line monitoring of a simulated BWR component

In this paper, Robust AutoAssociative Neural Networks (RAANN) are applied to a series of signals produced by the Halden simulator of the 1200 MWe BWR Forsmark-3 plant in Sweden. The applications concern: • correction of drifts and gross errors in sensors, for diagnostic and control purposes, • cluster analysis, to individuate a failed component and the intensity of the failure, • forecasting system signals, for safety or economic purposes, • reconstruction of unmeasured signals (virtual sensors). In the attainment of the above results, the geometric interpretation of the mapping performed by the network, propounded in Part I of this work, has provided a reasoned choice of the most critical free parameter, i.e., the number f of nodes of the bottleneck layer, thus allowing a deep understanding of the network functioning and also avoiding the traditional and troubling procedure of selection by trial-and-error. The theoretical basis of this analysis, discussed in details in the companion paper, is founded on the idea of dimension and in particular of fractal dimension, which has been used as a numerical estimator of f.

The autoassociative neural network in signal analysis: I. The data dimensionality reduction and its geometric interpretation

In complex and risky plants, such as the nuclear reactors, the analysis of the signals released by the many sensors which monitor the plant represents a difficult task due to the high-dimensionality of the data. This paper is the first of two in which we tackle the problem of the dimensionality reduction by the nonlinear principal components analysis as performed by an autoassociative neural network (AANN). This network filters the many input data and releases at the bottleneck output a relatively small number of signals which capture the significant properties of the original data, thus realizing the data reduction. In the present paper, we show that the network ability in correctly reproducing as output the given input after a passage through the bottleneck layer (which by definition should have fewer nodes than either input or output layers) could be conceived as a topological mapping between abstract spaces. Apart from the less critical choice of the number of nodes in the mapping and demapping layers, the topological mapping will be successful – and the AANN will be able to perform the required data reconstruction – provided that the number of nodes of the bottleneck layer is related to the dimensionality d of the abstract projection space. We show how to obtain a numerical estimate d* for the real dimension d. This numerical estimate will firmly base the choice of the number of nodes f of the bottleneck layer, thus avoiding the usual troubling trial-and-error procedure. The power of the proposed approach is demonstrated firstly on a few geometrical cases and then on the analysis of nuclear transients simulated by the classic Chernick’s model.

Sequential Memory: A Putative Neural and Synaptic Dynamical Mechanism

A key issue in the neurophysiology of cognition is the problem of sequential learning. Sequential learning refers to the ability to encode and represent the temporal order of discrete elements occurring in a sequence. We show that the short-term memory for a sequence of items can be implemented in an autoassociation neural network. Each item is one of the attractor states of the network. The autoassociation network is implemented at the level of integrate-and-fire neurons so that the contributions of different biophysical mechanisms to sequence learning can be investigated. It is shown that if it is a property of the synapses or neurons that support each attractor state that they adapt, then every time the network is made quiescent (e.g., by inhibition), then the attractor state that emerges next is the next item in the sequence. We show with numerical simulations implementations of the mechanisms using (1) a sodium inactivation-based spike-frequency-adaptation mechanism, (2) a Ca(2+)-activated K+ current, and (3) short-term synaptic depression, with sequences of up to three items. The network does not need repeated training on a particular sequence and will repeat the items in the order that they were last presented. The time between the items in a sequence is not fixed, allowing the items to be read out as required over a period of up to many seconds. The network thus uses adaptation rather than associative synaptic modification to recall the order of the items in a recently presented sequence.

USE OF AN AUTOASSOCIATIVE NEURAL NETWORK FOR DYNAMIC DATA RECONCILIATION

The technique of dynamic data reconciliation has been previously studied in the literature and shown to be an effective tool to better estimate the true values of process variables by using information from both measured values and process models. Real-time implementation of dynamic data reconciliation involves solving complex optimization problem, leading to large computation time. This paper presents a study on the use of a dynamic Autoassociative Neural Network (AANN) for dynamic data reconciliation. Once trained, the AANN can be directly used for online signal validation. Closed-loop performance of the AANN for both linear and nonlinear processes was evaluated using simulations of two storage tank processes. The AANN provided accurate estimates of measured values for the two processes studied in this investigation.

Development of a new diagnostic protocol using a neuro-dynamical tool

Abstract A general framework is presented where time series properly embedded in a suitable embedding space with the time-delayed method are used by an artificial neural network to diagnose diseases and/or possible dynamical changes in the status of a subject, e.g. the progression of the disease itself. The first step includes the choice of an optimal delay time, that can be carried out with the aid of the auto-correlation or the mutual information functions. This is followed by the choice of the embedding dimension, carried out using the false nearest neighbours method. An auto-associative artificial neural network is then trained to recognize the dynamical trajectory in the pseudo state space. Finally, the error function of the ANN is used to evaluate outliers in the dynamics, that can be recognized as diseases or state transitions.

Feature Extraction Techniques for the Analysis of Spectral Polarization Profiles

This paper introduces a novel feature extraction technique for the analysis of spectral line Stokes profiles. The procedure is based on the use of an autoassociative artificial neural network containing nonlinear hidden layers. The neural network extracts a small subset of parameters from the profiles (features), from which it is then able to reconstruct the original profile. This new approach is compared to two other procedures that have been proposed in previous works, namely principal component analysis and Hermitian function expansions. Depending on the target application, each of these three techniques has some advantages and disadvantages, which are discussed here.