Autoassociative Neural Network Research Articles

Robust Nonlinear PLS Based on Neural Networks and Application to Composition Estimator for High-Purity Distillation Columns

The accurate and reliable on-line estimation of product quality is an essential task for successful process operation and control. This paper proposes a new estimation method that extends the conventional linear PLS (Partial Least Squares) regression method to a nonlinear framework in a more robust manner. To handle the nonlinearities, nonlinear PLS based on linear PLS and neural network has been employed. To improve the robustness of the nonlinear PLS, the autoassociative neural network has been integrated with nonlinear PLS. The integration allows us to handle the nonlinear correlation as well as nonlinear functional relationship with fewer components in a more robust manner. The application results have shown that the proposed Robust Nonlinear PLS (RNPLS) performs better than previous linear and nonlinear regression methods such as PLS, NNPLS, even for the nonlinearities due to operating condition changes, limited observations, and measurement noise.

Backpropagation of an Image Similarity Metric for Autoassociative Neural Networks

Autoassociative Neural Networks (AANNs) are most commonly used for image data compression. The goal of an AANN for image data is to have the network output be `similar' to the input. Most of the research in this area use backpropagation training with Mean-Squared Error (MSE) as the optimisation criteria. This paper presents an alternative error function called the Visual Difference Predictor (VDP) based on concepts from the human-visual system. Using the VDP as the error function provides a criteria to train an AANN more efficiently, and results in faster convergence of the weights, while producing an output image perceived to be very similar by a human observer.

CLUSTER ANALYSIS OF MINERAL PROCESS DATA WITH AUTOASSOCIATIVE NEURAL NETWORKS

Experimental results indicate that autoassociative neural networks provide a robust method for the identification of clusters in process data. Cluster identification is accomplished by extracting a single feature from each multivariate data vector. The ranked features can be used to construct a feature curve, which is subsequently used as a basis for partitioning of the data space. In three case studies, involving two sets of ore samples, and a set of flotation froth features, with 11, 13 and 5 variables respectively, the clusters identified with the neural network appeared to be better than those obtained by conventional means.

Hysteresis studies in a noisy autoassociative neural network

We define magnetization for a noisy autoassociative neural network driven by an external periodic field. Numerical simulations are carried out to investigate the effect of drive amplitude and frequency and noise strength on the area of the hysteresis loop. We observe that in the presence of weak periodic signal, the network exhibits a maximum in the hysteresis loop area at a nonzero noise intensity indicating maximum synchronization between the periodic signal and the response. It also goes through a maximum as a function of signal frequency.

On impulsive autoassociative neural networks

Many systems existing in physics, chemistry, biology, engineering, and information science can be characterized by impulsive dynamics caused by abrupt jumps at certain instants during the process. These complex dynamical behaviors can be modeled by impulsive differential systems or impulsive neural networks. This paper formulates and studies a new model of impulsive autoassociative neural networks. Several fundamental issues, such as global exponential stability and existence and uniqueness of equilibria of such neural networks, are established.

Reliable neuro self-tuning control using autoassociative neural networks for the water treatment

This paper extends the results of the research presented recently in Vienna and incorporates an algorithm to obtain very precise control of the addition of the coagulation chemical by developing a neuro self-tuning policy for automatically adjusting the parameters of a conventional PI controller.

Neuronal state feedback learning of Cohen-Grossberg networks

This paper proposes a direct synaptic weight training technique for a class of additive dynamic auto-associative neural networks based on the Cohen-Grossberg neuronal activation model. The proposed technique is based on the Jurdjevic-Quinn stabilization method for control affine systems. Asymptotic stability of the training law is guaranteed and regions of attraction around each point attractor are predefined. The proposed technique requires the solution of significantly fewer non-linear differential equations and is considerably simpler and faster than existing training techniques.

A comparative study of neural network based feature extraction paradigms

The projection maps and derived classification accuracies of a neural network (NN) implementation of Sammon's mapping, an auto-associative NN (AANN) and a multilayer perceptron (MLP) feature extractor are compared with those of the conventional principal component analysis (PCA). Tested on five real-world databases, the MLP provides the highest classification accuracy at the cost of deforming the data structure, whereas the linear models preserve the structure but usually with inferior accuracy.

The diabolo classifier

We present a new classification architecture based on autoassociative neural networks that are used to learn discriminant models of each class. The proposed architecture has several interesting properties with respect to other model-based classifiers like nearest-neighbors or radial basis functions: it has a low computational complexity and uses a compact distributed representation of the models. The classifier is also well suited for the incorporation of a priori knowledge by means of a problem-specific distance measure. In particular, we will show that tangent distance (Simard, Le Cun, & Denker, 1993) can be used to achieve transformation invariance during learning and recognition. We demonstrate the application of this classifier to optical character recognition, where it has achieved state-of-the-art results on several reference databases. Relations to other models, in particular those based on principal component analysis, are also discussed.

Predicting ultra-hard binary compounds via cascaded auto- and hetero-associative neural networks

When an auto-associative neural network is trained within any conceptual space, the many rules and schema embodied within that knowledge domain are encoded through its many connection weights and biases. For instance, if such a network's input/output exemplars consist of numerous formulas representing known chemical compounds, subsequent network training will produce connection traces that embody the many implicit rules governing the constraints between constituent elements and their allowed proportionalities. That is to say, the net has gained a statistical `insight' into the patterns of bonding, valence, and charge balance that must be observed in theorizing new chemical compounds. If that network is now made chaotic by random perturbation of its processing elements and connection weights, the resulting network activations will represent the formulas of a wide variety of plausible compounds, many of which may be considered novel from the standpoint of network training. We therefore attain an all-neural search engine for generating a stream of plausible chemical possibilities. Adding subsequent `policing' networks to associate these emerging chemical formulas with various physical and chemical properties, we are able to either filter for sought characteristics or alternatively, assemble expanding materials tabulations of potentially new compounds and their estimated properties. Here, we describe the theory, construction, function, and results of just such an autonomous materials discovery machine, tailored specifically to the search for new ultrahard binary compounds.

Visualization of transformed multivariate data sets with autoassociative neural networks

Artificial neural networks have recently gained prominence as powerful tools for the projection of high-dimensional data, where fast interactive mapping of multi-dimensional data onto 2D or 3D maps with as little distortion as possible is required. These methods typically generate static maps of the data, based on some optimization criterion. A new strategy based on the transformation of the data prior to use of autoassociative neural networks is therefore proposed and it is shown that this strategy allows more flexible visualization of the data than is possible with either Kohonen or hidden target backpropagation (Sammon) neural networks, in that various perspectives of the multi-dimensional space can be explored by dynamically mapping the data with respect to user-defined vantage points in the multi-dimensional space.

A direct learning law for a class of auto-associative dynamic neural networks

This paper proposes a new learning technique fur a class of additive dynamic auto-associative neural networks. In the proposed technique, which is based on the Jurdjevic-Quinn stabilization method for control affine systems, the network synaptic weights are directly related to the network states. Asymptotic stability of the training law is assured and a region of attraction around each point attractor can be predefined. The proposed learning law is simpler than existing techniques and requires the solution of significantly fewer nonlinear differential equations. The proposed technique is compared with existing techniques by way of an example.

Quantification of Data Clusters for Bioprocess Performance Classification via Artificial Neural Networks

A new procedure for classifying bioprocess performance using artificial neural networks is presented. Using historical records of processes with known good and bad outcomes as reference sets, patterns in the data are represented as data clusters generated from the bottleneck values of an autoassociative neural network (AANN). The strategy is similar to analyzing the cluster of points generated when plotting the first and second principal components. This method departs, however, from principal component analysis (PCA) in that it can handle data clouds generated from more than 2 variables and the classification is assessed quantitatively as opposed to visually. This is achieved by feeding the cloud patterns into a feed-forward network which compares the sample’s data clouds to reference sets. Using a classification scheme based on Soft Independent Modeling of Class Analogy (SIMCA), the net is able to classify a process as good, bad, neither, or both as early as possible.

On pattern classification with Sammon's nonlinear mapping an experimental study

Sammon's mapping is conventionally used for exploratory data projection, and as such is usually inapplicable for classification. In this paper we apply a neural network (NN) implementation of Sammon's mapping to classification by extracting an arbitrary number of projections. The projection map and classification accuracy of the mapping are compared with those of the auto-associative NN (AANN), multilayer perceptron (MLP) and principal component (PC) feature extractor for chromosome data. We demonstrate that chromosome classification based on Sammon's (unsupervised) mapping is superior to the classification based on the AANN and PC feature extractor and highly comparable with that based on the (supervised) MLP. c 1998 Pattern Recognition Society.

Bioprocess fault detection by nonlinear multivariate analysis: application of an artificial autoassociative neural network and wavelet filter bank.

A nonlinear multivariate analysis, artificial autoassociative neural network (AANN), was applied to bioprocess fault detection. In an optimal production process of a recombinant yeast with a temperature controllable expression system, faults in test cases with faulty temperature sensors and plasmid instability of recombinant cells could be detected by the AANN. Since the raw data of measured variables included high-frequency noise, a wavelet filter bank (WFB) was applied to noise elimination before training of the AANN. The filtering performance of the WFB was compared with those of some classical first-order digital filters. The filtered signals at several resolution scales by the WFB were employed as the training data of the AANN. The computing time and summation of square of errors in training were compared, and the appropriate degree of the noise filtering and the density of the training data of the AANN were discussed. The performance of the feature capturing by the AANN was compared with that by a linear multivariate analysis, principal component analysis. A J index defined in this paper, using inputs and outputs of the AANN, was used for fault detection successfully. The output of the first unit of the trained AANN functioned effectively for the discrimination of the data in the abnormal cases from the data in the normal cases.

Use of Autoassociative Neural Networks for Signal Validation

Recently, the use of Autoassociative Neural Networks (AANNs) to perform on-line calibration monitoring of process sensors has been shown to not only be feasible, but practical as well. This paper summarizes the results of applying AANNs to instrument surveillance and calibration monitoring at Florida Power Corporation’s Crystal River #3 Nuclear Power Plant and at the Oak Ridge National Laboratory High Flux Isotope Reactor. In both cases sensor drifts are detectable at a nominal level of 0.5% instrument’s full scale range. This paper will discuss the selection of a five layer neural network architecture, a robust training paradigm, the input selection criteria, and a retuning algorithm.

Data reconciliation for simulated flotation process

This paper introduces a novel neural network-based technique called system balance-related autoassociative neural networks (SBANN) for steady state data reconciliation. This neural network has the same architecture as traditional feedforward neural networks but the main difference lies in the minimization of an objective function that includes process material and/or energy imbalance terms in addition to the traditional least-squares prediction term. Accordingly, this neural network with the system balance-related objective criterion is able to perform the two basic functions necessary for proper steady state data reconciliation: data smoothing to reduce the data variance and data correction to satisfy material and/or energy balance constraints. This novel technique is illustrated for data reconciliation of a simulated flotation circuit that is widely used in mineral processing.

Use of a novel autoassociative neural network for nonlinear steady‐state data reconciliation

AbstractA novel autoassociative neural‐network‐based estimator for nonlinear steady‐state data reconciliation was developed, which is a modified autoassociative feedforward neural network. The main difference between them lies in the minimization of an objective function that includes material imbalance terms of flow rates and compositions as well as the traditional least‐square prediction term. Accordingly, this neural network, with the material balance‐related equations included in the objective criterion, can perform simultaneously the following basic functions necessary for proper steady‐state data rectification: (1) eliminate the nonrandom errors, such as the biases and gross erros, from measurements; (2) filter out the random errors from measured data; and (3) estimate the values of unmeasured process variables, provided data redundancy prevails. This novel neural‐network‐based reconciliation method is demonstrated on a flotation system.

An exact learning algorithm for autoassociative neural networks with binary couplings

Exact solutions for the learning problem of autoassociative networks with binary couplings are determined by a new method. The use of a branch-and-bound algorithm leads to a substantial saving of computational time compared with complete enumeration. As a result, fully connected networks with up to 40 neurons could be investigated. The network capacity is found to be close to 0.83.

On-line fault diagnosis for optimal rice α-amylase production process of a temperature-sensitive mutant of Saccharomyces cerevisiae by an autoassociative neural network

A nonlinear statistical approach to data analysis, an autoassociative neural network (AANN), was applied to fault diagnosis in the optimal production process of a recombinant yeast with a temperature controllable expression system. High frequency noise in the data could be eliminated by a wavelet transform before the fault diagnosis was performed. The diagnosis system could accurately and immediately detect the faults on-line in the test cases of a faulty temperature sensor and plasmid instability of the recombinant cells. The same faults were not detected by linear principal component analysis (PCA). By implementing corrective action after fault detection, the final production amount was increased to twice the amount it would have been without diagnosis.