Autoassociative Neural Network Models Research Articles

Asymmetric Weights and Retrieval Practice in an Autoassociative Neural Network Model of Paired-Associate Learning.

Rizzuto and Kahana (2001) applied an autoassociative Hopfield network to a paired-associate word learning experiment in which (1) participants studied word pairs (e.g., ABSENCE-HOLLOW), (2) were tested in one direction (ABSENCE-?) on a first test, and (3) were tested in the same direction again or in the reverse direction (?-HOLLOW) on a second test. The model contained a correlation parameter to capture the dependence between forward versus backward learning between the two words of a word pair, revealing correlation values close to 1.0 for all participants, consistent with neural network models that use the same weight for communication in both directions between nodes. We addressed several limitations of the model simulations and proposed two new models incorporating retrieval practice learning (e.g., the effect of the first test on the second) that fit the accuracy data more effectively, revealing substantially lower correlation values (average of .45 across participants, with zero correlation for some participants). In addition, we analyzed recall latencies, finding that second test recall was faster in the same direction after a correct first test. Only a model with stochastic retrieval practice learning predicted this effect. In conclusion, recall accuracy and recall latency suggest asymmetric learning, particularly in light of retrieval practice effects.

A visual familiarity account of evidence for orthographic processing in pigeons (Columbia livia): a reply to Scarf, Corballis, Güntürkün, and Colombo (2017).

Scarf et al. (Proc Natl Acad Sci 113(40):11272-11276, 2016) demonstrated that pigeons, as with baboons (Grainger et al. in Science 336(6078):245-248, 2012; Ziegler in Psychol Sci. https://doi.org/10.1177/0956797612474322 , 2013), can be trained to display several behavioural hallmarks of human orthographic processing. But, Vokey and Jamieson (Psychol Sci 25(4):991-996, 2014) demonstrated that a standard, autoassociative neural network model of memory applied to pixel maps of the words and nonwords reproduces all of those results. In a subsequent report, Scarf et al. (Anim Cognit 20(5):999-1002, 2017) demonstrated that pigeons can reproduce one more marker of human orthographic processing: the ability to discriminate visually presented four-letter words from their mirror-reversed counterparts (e.g. "LEFT" vs. " "). The current report shows that the model of Vokey and Jamieson (2014) reproduces the results of Scarf et al. (2017) and reinforces the original argument: the recent results thought to support a conclusion of orthographic processing in pigeons and baboons are consistent with but do not force that conclusion.

A Comprehensive Study on the Applications of Artificial Intelligence for the Medical Diagnosis and Prognosis of Asthma

An estimated 300 million people worldwide suffer from asthma, and this number is expected to increase to 400 million by 2025. Approximately 250,000 people die prematurely each year from asthma out of which, almost all deaths are avoidable. Most of these deaths occur because the patients are unaware of their asthmatic morbidity. If detected early, asthmatic mortality rate can be reduced by 78%, provided that the patients carry appropriate medication for the same and/or are in lose vicinity to medical equipment like nebulizers. This study focuses on the development and valuation of algorithms to diagnose asthma through symptom intensive questionary, clinical data and medical reports. Machine Learning Algorithms like Back-propagation model, Context Sensitive Auto-Associative Memory Neural Network Model, C4.5 Algorithm, Bayesian Network and Particle Swarm Optimization have been employed for the diagnosis of asthma and later a comparison is made between their respective prospects. All algorithms received an accuracy of over 80%. However, the use of Auto Associative Memory Model (on a layered Artificial Neural Network) displayed much better results. It reached to an accuracy of over 90% and an inconclusive diagnosis rate of less than 1% when trained with adequate data. In the end, naive mobile based applications were developed on Android and iOS that made use of the self-training auto associative memory model to achieve an accuracy of nearly 94.2%.

A new reconstruction-based auto-associative neural network for fault diagnosis in nonlinear systems

Auto-associative neural network (AANN) is a typical nonlinear principal component analysis method, which is widely used in industry for fault diagnosis purposes, especially in nonlinear systems. However, the basic AANN often suffers from “smearing effects” problems that may lead to misdiagnosis, particularly with regards to the complex faults involving multiple variables. In this work, a new reconstruction-based AANN (RBAANN) method is proposed to enhance the capacity of fault diagnosis. In RBAANN, a generic derivative equation is developed to investigate the effects of AANN model inputs on the prediction error between model inputs and outputs. Based on the derivative equation, the reconstruction-based index for single or multiple variables, which is defined as the minimum prediction error, is obtained by tuning the corresponding model inputs iteratively. However, without the prior knowledge of the real faulty variables, all the possible variable sets need to be evaluated by the reconstruction-based index, and this may result in an exhaustive search and cause a huge computational burden. Thus, a branch and bound algorithm is introduced into RBAANN to solve the variable selection problem. Finally, an efficient fault diagnosis strategy by integrating RBAANN and branch and bound algorithm (BAB-RBAANN) is implemented to further pinpoint the source of the detected faults. This BAB-RBAANN method can handle both single and multiple variable(s) faults for nonlinear systems without prior knowledge efficiently. The effectiveness of the proposed methods is evaluated on a validation example and an industrial example. Comparisons with other methods, including principal component analysis techniques, are also presented.

A Denoising Based Autoassociative Model for Robust Sensor Monitoring in Nuclear Power Plants

Sensors health monitoring is essentially important for reliable functioning of safety-critical chemical and nuclear power plants. Autoassociative neural network (AANN) based empirical sensor models have widely been reported for sensor calibration monitoring. However, such ill-posed data driven models may result in poor generalization and robustness. To address above-mentioned issues, several regularization heuristics such as training with jitter, weight decay, and cross-validation are suggested in literature. Apart from these regularization heuristics, traditional error gradient based supervised learning algorithms for multilayered AANN models are highly susceptible of being trapped in local optimum. In order to address poor regularization and robust learning issues, here, we propose a denoised autoassociative sensor model (DAASM) based on deep learning framework. Proposed DAASM model comprises multiple hidden layers which are pretrained greedily in an unsupervised fashion under denoising autoencoder architecture. In order to improve robustness, dropout heuristic and domain specific data corruption processes are exercised during unsupervised pretraining phase. The proposed sensor model is trained and tested on sensor data from a PWR type nuclear power plant. Accuracy, autosensitivity, spillover, and sequential probability ratio test (SPRT) based fault detectability metrics are used for performance assessment and comparison with extensively reported five-layer AANN model by Kramer.

Spectral and prosodic features-based speech pattern classification

Speech pattern produced by individuals are unique. This uniqueness is due to the accent influenced by individual's native dialect. Prior knowledge of spoken dialect provides valuable information for speaker profiling and incorporating them in the decision parameter can improve the system performance. In this paper, an auto-associative neural network model has been proposed to model intrinsic characteristics of speech features for dialect classification. This paper highlights the sufficiency of few spectral and prosodic features for identification of Hindi dialects. Experimental results show that system performance is the best when both spectral and prosodic features are combined to use as input. In the presence of noise, performance of a conventional ASR starts to degrade. The NOISEX-92 database is used to add white noise to the recorded utterances in the range of 0 dB to 20 dB. This paper evaluates the dialect classification system's performance for SNRs in this range.

Machine Learning Applications for a Wind Turbine Blade under Continuous Fatigue Loading

Structural health monitoring (SHM) systems will be one of the leading factors in the successful establishment of wind turbines in the energy arena. Detection of damage at an early stage is a vital issue as blade failure would be a catastrophic result for the entire wind turbine. In this study the SHM analysis will be based on experimental measurements of vibration analysis, extracted of a 9m CX-100 blade under fatigue loading. For analysis, machine learning techniques utilised for failure detection of wind turbine blades will be applied, like non-linear Neural Networks, including Auto-Associative Neural Network (AANN) and Radial Basis Function (RBF) networks models.

Film segmentation and indexing using autoassociative neural networks

In this paper, Autoassociative Neural Network (AANN) models are explored for segmentation and indexing the films (movies) using audio features. A two-stage method is proposed for segmenting the film into sequence of scenes, and then indexing them appropriately. In the first stage, music and speech plus music segments of the film are separated, and music segments are labelled as title and fighting scenes based on their position. At the second stage, speech plus music segments are classified into normal, emotional, comedy and song scenes. In this work, Mel frequency cepstral coefficients (MFCCs), zero crossing rate and intensity are used as audio features for segmentation and indexing the films. The proposed segmentation and indexing method is evaluated on manual segmented Hindi films. From the evaluation results, it is observed that title, fighting and song scenes are segmented and indexed without any errors, and most of the errors are observed in discriminating the comedy and normal scenes. Performance of the proposed AANN models used for segmentation and indexing of the films, is also compared with hidden Markov models, Gaussian mixture models and support vector machines.

Recognition of emotions from video using acoustic and facial features

In this paper, acoustic and facial features extracted from video are explored for recognizing emotions. The temporal variation of gray values of the pixels within eye and mouth regions is used as a feature to capture the emotion-specific knowledge from the facial expressions. Acoustic features representing spectral and prosodic information are explored for recognizing emotions from the speech signal. Autoassociative neural network models are used to capture the emotion-specific information from acoustic and facial features. The basic objective of this work is to examine the capability of the proposed acoustic and facial features in view of capturing the emotion-specific information. Further, the correlations among the feature sets are analyzed by combining the evidences at different levels. The performance of the emotion recognition system developed using acoustic and facial features is observed to be 85.71 and 88.14 %, respectively. It has been observed that combining the evidences of models developed using acoustic and facial features improved the recognition performance to 93.62 %. The performance of the emotion recognition systems developed using neural network models is compared with hidden Markov models, Gaussian mixture models and support vector machine models. The proposed features and models are evaluated on real-life emotional database, Interactive Emotional Dyadic Motion Capture database, which was recently collected at University of Southern California.

Improved speaker identification in wireless environment

The increasing use of wireless mobile systems is creating great deal of interest in the development of robust speech systems in wireless environment. The major degradations involved in wireless environment are: acoustic environment, speech coding and transmission errors. In this paper, we address the problem of Speaker Identification (SI) from coded and cellular speech. Since speaker-specific characteristics are preserved in steady vowel segments of speech even after coding, the features extracted from these steady vowel regions are used to build the SI system. We have proposed a method to determine the steady vowel region from the speech signal by using vowel onset points and epochs. SI studies are carried out using cellular, coded and microphone speech databases. Autoassociative Neural Network (AANN) models are explored for developing the SI models. Speech coders considered in this work are GSM CELP, and MELP. Significant improvement in the performance of SI system is observed by using proposed approach.

Audio-video based Segmentation and Classification using SVM and AANN

In this paper, we propose a method for combining audio and video for segmentation and classification. The objective of segmentation is to detect the category change point such news to advertisement. The classification system classify the audio-video data into one of the predefined categories such as news, advertisement, sports, serial and movies. Mel frequency cepstral coefficients(MFCC) are used as acoustic features and color histogram is used as visual features for segmentation and classification. Support vector machine(SVM) and autoassociative neural network(AANN) models are used for segmentation and classification. The evidence from audio and video are combined using weighted sum rule for both segmentation and classifications.

A New Method of On-Line Fault Monitoring Based on Nonlinear MPCA

Multiway principal components analysis (MPCA) is a linear model in nature, thus, limited when it is applied to batch process. In this paper, the linear model MPCA was complemented with an autoassociative neural network model in order to generate nonlinear principal components. The networks bottleneck layer outputs (nonlinear principal components) were made orthogonal. A method to estimate confidence limits based on a kernel probability density function was proposed since the nonlinear scores are no normally distributed. A statistic-like parameter () was proposed to evaluate on-line scores for new runs using the density estimated confidence bounds and replacing the statistic. The proposed method was applied to monitoring fed-batch streptomycete production, and the simulation results show that the nonlinear scores obtained with the autoassociative neural networks capture more process data variance than if obtained with a linear method and the density estimation method proved to be more reliable.

An Auto-Associative Residual Processing and K-means Clustering Approach for Anemometer Health Assessment

This paper presents a health assessment methodology, as well as specific residual processing and figure of merit algorithms for anemometers in two different configurations. The methodology and algorithms are applied to data sets provided by the Prognostics and Health Management Society 2011 Data Challenge. The two configurations consist of the “paired” data set in which two anemometers are positioned at the same height, and the “shear” data set which includes an array of anemometers at different heights. Various wind speed statistics, wind direction, and ambienttemperature information are provided, in which the objective is to classify the anemometer health status during a set of samples from a 5 day period. The proposed health assessment methodology consists of a set of data processing steps that include: data filtering and pre-processing, a residual or difference calculation, and a k-means clustering based figure of merit calculation. The residual processing for the paired data set was performed using a straightforward difference calculation, while the shear data set utilized an additional set of algorithm processing steps to calculate a weighted residual value for each anemometer. The residual processing algorithm for the shear data set used a set of auto-associative neural network models to learn the underlying correlation relationship between the anemometer sensors and to calculate a weighted residual value for each of the anemometer wind speed measurements. A figure of merit value based on the mean value of the smaller of the two clusters for the wind speed residual is used to determine the health status of each anemometer. Overall, the proposed methodology and algorithms show promise, in that the results from this approach resulted in the top score for the PHM 2011 Data Challenge Competition. Using different clustering algorithms or density estimation methods for the figure of merit calculation is being considered for future work.

Recognition of emotions from video using neural network models

In this paper, facial features from the video sequence are explored for characterizing the emotions. The emotions considered for this study are Anger, Fear, Happy, Sad and Neutral. For carrying out the proposed emotion recognition study, the required video data is collected from the studio, Center for Education Technology (CET), at Indian Institute of Technology (IIT) Kharagpur. The dynamic nature of the grey values of the pixels within the eye and mouth regions are used as the features to capture the emotion specific knowledge from the facial expressions. Multiscale morphological erosion and dilation operations are used to extract features from eye and mouth regions, respectively. The features extracted from left eye, right eye and mouth regions are used to develop the separate models for each emotion category. Autoassociative neural network (AANN) models are used to capture the distribution of the extracted features. The developed models are validated using subject dependent and independent emotion recognition studies. The overall performance of the proposed emotion recognition system is observed to be about 87%.

Structural damage alarming using auto-associative neural network technique: Exploration of environment-tolerant capacity and setup of alarming threshold

With the intention of avoiding false-positive and false-negative alarms in structural damage alarming using the auto-associative neural network (AANN) technique, two issues pertaining to this technique are addressed in this study. The first issue explored is the environment-tolerant capacity of the AANN. Efforts have been made to seek a generalization technique to enhance the environment-tolerant capacity. First, a baseline AANN model is formulated using the conventional training algorithm. Generalization techniques including AIC and FPE, early stopping, and Bayesian regularization are then investigated, resulting in three new AANN models. Their environment-tolerant capacity is evaluated as per their capability to avoid false-positive and false-negative alarms. The other issue addressed is the setup of alarming threshold, with intent to reduce the uncertainty in AANN-based structural damage alarming. A procedure based on the probability analysis of the novelty index is proposed for this purpose. First, the novelty index characterizing the intact structure is analyzed by the Kolmogorov–Smirnov goodness-of-fit test to obtain its best-fit continuous probability distribution. A confidence interval is then defined in consideration of the compromise between type I and type II errors. The alarming threshold of the novelty index is consequently set at the upper limit of the confidence interval. The above explorations are examined by using the long-term monitoring data on modal properties of the cable-stayed Ting Kau Bridge. The capability to eliminate false-positive alarm is verified by using unseen testing data which were not used in formulating the AANN models, while the capability to alleviate false-negative alarm is examined by using simulated data from the ‘damaged’ bridge with the help of a precise finite element model. The study indicates that the early stopping technique performs best in improving the environment-tolerant capacity of the AANN, and the alarming threshold set by the proposed procedure helps to reduce the uncertainty in AANN-based structural damage alarming.

Pattern classification models for classifying and indexing audio signals

In the age of digital information, audio data has become an important part in many modern computer applications. Audio classification and indexing has been becoming a focus in the research of audio processing and pattern recognition. In this paper, we propose effective algorithms to automatically classify audio clips into one of six classes: music, news, sports, advertisement, cartoon and movie. For these categories a number of acoustic features that include linear predictive coefficients, linear predictive cepstral coefficients and mel-frequency cepstral coefficients are extracted to characterize the audio content. The autoassociative neural network model (AANN) is used to capture the distribution of the acoustic feature vectors. Then the proposed method uses a Gaussian mixture model (GMM)-based classifier where the feature vectors from each class were used to train the GMM models for those classes. During testing, the likelihood of a test sample belonging to each model is computed and the sample is assigned to the class whose model produces the highest likelihood. Audio clip extraction, feature extraction, creation of index, and retrieval of the query clip are the major issues in automatic audio indexing and retrieval. A method for indexing the classified audio using LPCC features and k-means clustering algorithm is proposed.

PRINCIPAL COMPONENTS BASED SUPPORT VECTOR REGRESSION MODEL FOR ON-LINE INSTRUMENT CALIBRATION MONITORING IN NPPS

In nuclear power plants (NPPs), periodic sensor calibrations are required to assure that sensors are operating correctly. By checking the sensor's operating status at every fuel outage, faulty sensors may remain undetected for periods of up to 24 months. Moreover, typically, only a few faulty sensors are found to be calibrated. For the safe operation of NPP and the reduction of unnecessary calibration, on-line instrument calibration monitoring is needed. In this study, principal component-based auto-associative support vector regression (PCSVR) using response surface methodology (RSM) is proposed for the sensor signal validation of NPPs. This paper describes the design of a PCSVR-based sensor validation system for a power generation system. RSM is employed to determine the optimal values of SVR hyperparameters and is compared to the genetic algorithm (GA). The proposed PCSVR model is confirmed with the actual plant data of Kori Nuclear Power Plant Unit 3 and is compared with the Auto-Associative support vector regression (AASVR) and the auto-associative neural network (AANN) model. The auto-sensitivity of AASVR is improved by around six times by using a PCA, resulting in good detection of sensor drift. Compared to AANN, accuracy and cross-sensitivity are better while the auto-sensitivity is almost the same. Meanwhile, the proposed RSM for the optimization of the PCSVR algorithm performs even better in terms of accuracy, auto-sensitivity, and averaged maximum error, except in averaged RMS error, and this method is much more time efficient compared to the conventional GA method.

Classification of audio signals using AANN and GMM

Today, digital audio applications are part of our everyday lives. Audio classification can provide powerful tools for content management. If an audio clip automatically can be classified it can be stored in an organised database, which can improve the management of audio dramatically. In this paper, we propose effective algorithms to automatically classify audio clips into one of six classes: music, news, sports, advertisement, cartoon and movie. For these categories a number of acoustic features that include linear predictive coefficients, linear predictive cepstral coefficients and mel-frequency cepstral coefficients are extracted to characterize the audio content. The autoassociative neural network model (AANN) is used to capture the distribution of the acoustic feature vectors. The AANN model captures the distribution of the acoustic features of a class, and the backpropagation learning algorithm is used to adjust the weights of the network to minimize the mean square error for each feature vector. The proposed method also compares the performance of AANN with a Gaussian mixture model (GMM) wherein the feature vectors from each class were used to train the GMM models for those classes. During testing, the likelihood of a test sample belonging to each model is computed and the sample is assigned to the class whose model produces the highest likelihood.

Classification of sport videos using edge-based features and autoassociative neural network models

In this paper, we propose a method for classification of sport videos using edge-based features, namely edge direction histogram and edge intensity histogram. We demonstrate that these features provide discriminative information useful for classification of sport videos, by considering five sports categories, namely, cricket, football, tennis, basketball and volleyball. The ability of autoassociative neural network (AANN) models to capture the distribution of feature vectors is exploited, to develop class-specific models using edge-based features. We show that combining evidence from complementary edge features results in improved classification performance. Also, combination of evidence from different classifiers like AANN, hidden Markov model (HMM) and support vector machine (SVM) helps improve the classification performance. Finally, the performance of the classification system is examined for test videos which do not belong to any of the above five categories. A low rate of misclassification error for these test videos validates the effectiveness of edge-based features and AANN models for video classification.

Semi-physical Neural Network Model in Detecting Engine Transient Faults using the Local Approach

This paper investigates detection of an air leak fault in the intake manifold subsystem of an automotive engine during transient operation. Previously, it was shown that integrating the local approach with an auto-associative neural network model of the engine, significantly increased the sensitivity of fault detection. However, the drawback then is that the computational load is naturally dependent on the network complexity. This paper proposes the use of the available physical models to pre-process the original signals prior to model building for fault detection. This not only extracts existing relationships among the variables, but also helps in reducing the number of variables to be modelled and the related model complexity. The benefits of this improvement are demonstrated by practical application to a modern spark ignition 1.8 litre Nissan petrol engine.