Hetero-associative Neural Network Research Articles

Gene selection and disease prediction from gene expression data using a two-stage hetero-associative memory

In general, gene expression microarrays consist of a vast number of genes and very few samples, which represents a critical challenge for disease prediction and diagnosis. This paper develops a two-stage algorithm that integrates feature selection and prediction by extending a type of hetero-associative neural networks. In the first level, the algorithm generates the associative memory, whereas the second level picks the most relevant genes. With the purpose of illustrating the applicability and efficiency of the method proposed here, we use four different gene expression microarray databases and compare their classification performance against that of other renowned classifiers built on the whole (original) feature (gene) space. The experimental results show that the two-stage hetero-associative memory is quite competitive with standard classification models regarding the overall accuracy, sensitivity and specificity. In addition, it also produces a significant decrease in computational efforts and an increase in the biological interpretability of microarrays because worthless (irrelevant and/or redundant) genes are discarded.

Enhancing the Delta Training Rule for a Single Layer Feedforward Heteroassociative Memory Neural Network

this paper, an algorithm is suggested to train a single layer feedforward neural network to function as a heteroassociative memory. This algorithm enhances the ability of the memory to recall the stored patterns when partially described noisy inputs patterns are presented. The algorithm relies on adapting the standard delta rule by introducing new terms, first order term and second order term to it. Results show that the heteroassociative neural network trained with this algorithm perfectly recalls the desired stored pattern when 1.6% and 3.2% special partially described noisy inputs patterns are presented.

Обучаемая модель заучивания последовательности движений на основе гетероассоциативной нейронной сети

Обучаемая модель заучивания последовательности движений на основе гетероассоциативной нейронной сети

Problems of coding stereo images in human memory

This paper discusses the memorization and recall by man of a sequence of planar or stereoscopic images, including six frames that contain a planar strip (8×8 positions of the stimulus) or a volume strip (8×4×2 positions). At the recall stage, the subject chose between the stimulus and three distractors in each frame. It is shown that the times for recognition and recall are less for volume stimuli, while the percent of correct responses is greater for planar stimuli. For volume stimuli, the distribution of errors depends on the disparity between the target and the selected distractor. A model based on a heteroassociative neural network reproduces the error distribution for planar but not for volume stimuli. The resulting data are evidence that the internal representations are substantially different for planar and three-dimensional objects: disparity has a substantial effect on the memorization and recognition of three-dimensional objects.

Autoassociative–heteroassociative neural networks

The Autoassociative–Heteroassociative Neural Network (A-HNN) is a unique integration of autoassociative and heteroassociative neural network mappings to provide a functional approximation of two variables from one. This new architecture provides three features: the autoassociative mapping enables a stability metric for assessing the robustness or accuracy of the heteroassociative mapping; the A-HNN generates the inverse of the encoding portion of an associated autoassociative neural network (AANN); and, empirically, the use of input data as target vectors (the autoassociative mapping) improves training performance of the network.

Reducing the Dimensions of Texture Features for Image Retrieval Using Multi-layer Neural Networks

This paper presents neural network-based dimension reduction of texture features in content-based image retrieval. In particular, we highlight the usefulness of hetero-associative neural networks to this task, and also propose a scheme to combine the hetero-associative and auto-associative functions. A multichannel Gabor-filtering approach is used to derive 30-dimensional texture features from a set of homogeneous texture images. Multi-layer feedforward neural networks are then trained to reduce the number of feature dimensions. Our results show that the methods lead to a reduction of up to 30% while keeping or even improving the performance of similarity ranking. This has the benefit of alleviating the ill-effects of the high dimensionality of features in current image indexing methods and resulting in significant speeding up retrieval rates. Results using principal component analysis are also provided for comparison.

Predicting ultra-hard binary compounds via cascaded auto- and hetero-associative neural networks

When an auto-associative neural network is trained within any conceptual space, the many rules and schema embodied within that knowledge domain are encoded through its many connection weights and biases. For instance, if such a network's input/output exemplars consist of numerous formulas representing known chemical compounds, subsequent network training will produce connection traces that embody the many implicit rules governing the constraints between constituent elements and their allowed proportionalities. That is to say, the net has gained a statistical `insight' into the patterns of bonding, valence, and charge balance that must be observed in theorizing new chemical compounds. If that network is now made chaotic by random perturbation of its processing elements and connection weights, the resulting network activations will represent the formulas of a wide variety of plausible compounds, many of which may be considered novel from the standpoint of network training. We therefore attain an all-neural search engine for generating a stream of plausible chemical possibilities. Adding subsequent `policing' networks to associate these emerging chemical formulas with various physical and chemical properties, we are able to either filter for sought characteristics or alternatively, assemble expanding materials tabulations of potentially new compounds and their estimated properties. Here, we describe the theory, construction, function, and results of just such an autonomous materials discovery machine, tailored specifically to the search for new ultrahard binary compounds.

On hetero-associative neural networks and adaptive interference cancellation

We discuss two novel adaptive algorithms for generalized eigendecomposition that are derived from a two-layer linear feedforward hetero-associative neural network. In addition, we provide a rigorous convergence analysis of the adaptive algorithms by using stochastic approximation theory. Finally, we use these algorithms for on-line multiuser access interference cancellation in code-division-multiple-access-based cellular communications. Numerical simulations are reported to demonstrate their rapid convergence.

Learning and retrieving spatio-temporal sequences with any static associative neural network

The purpose of the present paper is to report on the design of a general system that is capable of learning and retrieving spatio-temporal sequences using any static associative neural networks (ANNs), including both autoassociative and heteroassociative neural networks. This artificial neural system has three major components: a voting network, a parallel array of ANN's, and delayed feedback lines from the output of the system to the ANN layer. The system has separate primary and pairing input channels. During learning, pairs of sequences of spatial patterns are presented to the primary and the pairing input channels simultaneously. During retrieving, a cue sequence, which is presented to the primary input channel only, and which may have spatial imperfections and temporal gaps, causes the system to output the stored spatio-temporal sequence. As a demonstration of the applicability of the present general system, we present an implementation using a specific neural network, that is, our dynamically generated variant of the counterpropagation network. This system shows computational advantages such as fast and accurate learning and retrieving, and the ability to store a large number of sequences consisting of nonorthogonal spatial patterns.

A novel neural hetero-associative memory model for pattern recognition

A novel hetero-associative neural network model is proposed where the associative recall of pattern is achieved in a single pass through the system. Instead of forming the memory matrix by an outer product formulation, inner product cross-correlation of input data with each set of the library vector was performed. The limitation regarding the constraint imposed on the choice or selection of patterns that can be stored is avoided by such a formulation. The reliability of the proposed model is much improved in comparison to the heteroassociative memory models which uses outer product correlation formulation to construct the memory matrix.

Fast recognition of real objects by an optimized hetero-associative neural network

We have developed and realized a concept which is very well suited for a quick recognition of highly correlated patterns. For a hetero-associative memory we used a minimal optimized output code (index memory). We constructed a tree structure in which the assignment of indices has been optimized by simulated annealing. Thus the algorithm for optimal stability of the learned patterns works most effectively. Special care was taken of recognizing « real » objects, e.g. scanned letters. Here the characteristic noise is very anisotropic. We have slightly modified the minimal overlap strategy of Krauth and Mezard [1] by training with this specific noise, and could improve the performance of our network. In order to get insight into the network and its behaviour we used a measure called constructivity which shows clearly the anisotropic effects. We trained a network to recognize a scanned text and to produce the associated text file. Due to the architecture of the network many processes can be treated in parallel. Therefore we used transputers for the implementation.