Auto-associative Multilayer Perceptron Research Articles

Intrinsic Lexical Intentionality and the Mathematics of Homomorphism

Moisl [1, 2] proposed a model of how the brain implements intrinsic intentionality with respect to lexical and sentence meaning, where 'intrinsic' is understood as 'independent of interpretation by observers external to the cognitive agent'. The discussion in both was mainly philosophical and qualitative; the present paper gives a mathematical account of the distance structure preservation that underlies the proposed mechanism of intrinsic intentionality. The three-layer autoassociative multilayer perceptron (aMLP) architecture with nonlinear hidden and linear output layers is the component in the model which generates representations homomorphic with the environment. The discussion first cites existing work which identifies the aMLP as an implementation architecture for principal component analysis (PCA), and then goes on to argue that the homomorphism characteristic of linear functions like PCA extends to aMLPs with nonlinear activation functions in the hidden layer. The discussion is in two main parts: the first part outlines the model, and the second presents the mathematical account.

Neural Networks Principal Component Analysis for estimating the generative multifactor model of returns under a statistical approach to the Arbitrage Pricing Theory. Evidence from the Mexican Stock Exchange

A nonlinear principal component analysis (NLPCA) represents an extension of the standard principal component analysis (PCA) that overcomes the limitation of the PCA’s assumption about the linearity of the model. The NLPCA belongs to the family of nonlinear versions of dimension reduction or the extraction techniques of underlying features, including nonlinear factor analysis and nonlinear independent component analysis, where the principal components are generalized from straight lines to curves. The NLPCA can be achieved via an artificial neural network specification where the PCA classic model is generalized to a nonlinear mode, namely, Neural Networks Principal Component Analysis (NNPCA). In order to extract a set of nonlinear underlying systematic risk factors, we estimate the generative multifactor model of returns in a statistical version of the Arbitrage Pricing Theory (APT), in the context of the Mexican Stock Exchange. We used an auto-associative multilayer perceptron neural network or autoencoder, where the ‘bottleneck’ layer represented the nonlinear principal components, or in our context, the scores of the underlying factors of systematic risk. This neural network represents a powerful technique capable of performing a nonlinear transformation of the observed variables into the nonlinear principal components, and to execute a nonlinear mapping that reproduces the original variables. We propose a network architecture capable of generating a loading matrix that enables us to make a first approach to the interpretation of the extracted latent risk factors. In addition, we used a two stage methodology for the econometric contrast of the APT involving first, a simultaneous estimation of the system of equations via Seemingly Unrelated Regression (SUR), and secondly, a cross-section estimation via Ordinary Least Squared corrected by heteroskedasticity and autocorrelation by means of the Newey-West heteroskedasticity and autocorrelation consistent covariances estimates (HEC). The evidence found shows that the reproductions of the observed returns using the estimated components via NNPCA are suitable in almost all cases; nevertheless, the results in an econometric contrast lead us to a partial acceptance of the APT in the samples and periods studied.

Image compression approach with ridgelet transformation using modified neuro modeling for biomedical images

Image compression is applied to many fields such as television dissemination, remote sensing, image storage. Digitized images are compressed by a method which exploits the redundancy of the images so that the number of bits required to represent the image can be reduced with acceptable degradation of the decoded image. The humiliation of the image quality is limited with respect to the application used. There are various biomedical applications where accuracy is of major concern. To attain the objective of performance improvement with respect to decoded picture quality and compression ratios, in contrast to existing image compression techniques, an effective image coding technique which involves transforming the image into another domain with ridgelet function and then quantizing the coefficients with hybrid neural networks combining two different learning networks called auto-associative multilayer perceptron and self-organizing feature map is proposed. Ridge functions are effective in representing functions that have discontinuities along straight lines. Normal wavelet transforms not succeed to represent such functions effectively. The results obtained from the combination of finite ridgelet transform with hybrid neural networks found much better than that obtained from the JPEG2000 image compression system.

Laser spot detection-based computer interface system using autoassociative multilayer perceptron with input-to-output mapping-sensitive error back propagation learning algorithm

This paper presents a new computer interface system based on laser spot detection and moving pattern analysis of the detected laser spots in real-time processing. We propose a systematic method that uses either the frame difference of successive input images or an autoassociative multilayer perceptron (AAMLP) to detect laser spots. The AAMLP is applied only to areas of the input images where the frame difference of the successive images is not effective for detecting laser spots. In order to enhance the detection performance, the AAMLP is trained by a new training algorithm that increases the sensitivity of the input-to-output mapping of the AAMLP allowing a small variation in the input feature of the laser spot image to be successfully indicated. The proposed interface system is also able to keep track of the laser spot and recognize gesture commands. The moving pattern of the laser spot is recognized by using a multilayer perception. It is experimentally shown that the proposed computer interface system is fast enough for real-time operation with reliable accuracy.

Novel input and output mapping-sensitive error back propagation learning algorithm for detecting small input feature variations

This paper proposes a new error back propagation learning algorithm that properly enhances the sensitivity of input and output mapping by applying a high-pass filter characteristic to the conventional error back propagation learning algorithm, allowing small input feature variations to be successfully indicated. For the sensitive discrimination of novel class data with slightly different characteristics from the normal class data, the cost function in the proposed neural network algorithm is modified by further increasing the input and output sensitivity, where weight update rules are used to minimize the cost function using a gradient descent method. The proposed algorithm is applied to an auto-associative multilayer perceptron neural network and its performance evaluated with two real-world applications: a laser spot detection-based computer interface system for detecting a laser spot in complex backgrounds and an automatic inspection system for the reliable detection of Mura defects that occur during the manufacture of flat panel liquid crystal displays. When compared with the conventional error back propagation learning algorithm, the proposed algorithm shows a better performance as regards detecting small input feature variations by increasing the input–output mapping sensitivity.

D08: Neural network architecture for image compression

By using data compression techniques, it is possible to remove some of the redundant and irrelevant information contained in images, requiring less storage space and less time to transmit. The compression problem is well-suited for neural network based solutions. Neural networks are adaptive in nature, due to their use of a training phase for learning the relationships in the input data, and their generalization capabilities provide a means of coping with noisy or incomplete data. In this text there is effected application for still image compression using two significant neural network models, auto-associative multilayer perceptron (AMLP) and self-organizing feature map (SOFM). It was verified important characteristics of AMLP and SOFM to hardware implementation of neural networks by computer simulation. It was developed a software system NeuroCompressor for computer simulation of this application.

Autoassociative MLP in sleep spindle detection.

Spindles are one of the most important short-lasting waveforms in sleep EEG. They are the hallmarks of the so-called Stage 2 sleep. Visual spindle scoring is a tedious workload, since there are often a thousand spindles in one all-night recording of some 8 hr. Automated methods for spindle detection typically use some form of fixed spindle amplitude threshold, which is poor with respect to inter-subject variability. In this work a spindle detection system allowing spindle detection without an amplitude threshold was developed. This system can be used for automatic decision making of whether or not a sleep spindle is present in the EEG at a certain point of time. An Autoassociative Multilayer Perceptron (A-MLP) network was employed for the decision making. A novel training procedure was developed to remove inconsistencies from the training data, which was found to improve the system performance significantly.

Image compression using hybrid neural networks combining the auto-associative multi-layer perceptron and the self-organizing feature map

A new image compression technique is presented using hybrid neural networks that combine two different learning networks, the auto-associative multi-layer perceptron (AMLP) and the self-organizing feature map (SOFM). The neural networks simultaneously perform dimensionality reduction with the AMLP and categorization with the SOFM to compress image data. Two hybrid neural networks forming parallel and serial architectures are examined through theoretical analysis and computer simulation. The parallel structure network reduces the dimensionality of input pattern vectors by mapping them to different hidden layers of the AMLP selected by winner-take-all units of the SOFM. The serial structure network categorizes the input pattern vectors into several classes representing prototype vectors. Both the serial and parallel structures are combinations of the AMLP and SOFM networks. These hybrid neural networks achieve clear performance improvement with respect to decoded picture quality and compression ratios, compared to existing image compression techniques. >