Autoassociative Neural Network Research Articles

Sensor Fault Detection in Smart Extraterrestrial Habitats Using Unsupervised Learning

Various types of sensors are needed to monitor the health state of smart deep-space habitats. However, measured data can be affected by sensor faults, which influence the health management system and consequently the decision-making. In this paper, an unsupervised learning approach based on convolutional autoencoders (CAEs) is developed to detect anomalies in temperature and pressure sensors. The proposed method is systematically investigated using a habitat simulator (HabSim). Several illustrative examples are demonstrated in the nominal and hazardous states of the habitat, including micrometeorite impact and fire scenarios. The performance of the proposed method using CAEs is compared with that of existing methods using auto-associative neural networks (AANNs) and variational autoencoders. This comparison is based on typical evaluation metrics, including precision, recall, F1 score, training time, and testing time. The effect of temperature–pressure coupling on the detection performance of CAEs and AANNs is explored by training different data-driven models, including one with temperature sensors, one with pressure sensors, and one with both temperature and pressure sensors. The effect of the number of faulty sensors on the performance of CAEs is studied, as with an increase in the number of faulty sensors, redundant information among the sensors is reduced. The capability of CAEs to change the number of sensors without redesigning the network architecture and retraining the neural network is investigated and demonstrated. The capabilities and limitations of the proposed solution are discussed.

Autoassociative neural network for missing data imputation: A case study via the styrene production process

Autoassociative neural network for missing data imputation: A case study via the styrene production process

The Method of Restoring Lost Information from Sensors Based on Auto-Associative Neural Networks

The research aims to develop a neural network-based lost information restoration method when the complex nonlinear technical object (using the example of helicopter turboshaft engines) sensors fail during operation. The basis of the research is an auto-associative neural network (autoencoder), which makes it possible to restore lost information due to the sensor failure with an accuracy of more than 99%. An auto-associative neural network (autoencoder)-modified training method is proposed. It uses regularization coefficients that consist of the loss function to create a more stable and common model. It works well on the training sample of data and can produce good results on new data. Also, it reduces its overtraining risk when it adapts too much to the training data sample and loses its ability to generalize new data. This is especially important for small amounts of data or complex models. It has been determined based on the computational experiment results (the example of the TV3-117 turboshaft engine) that lost information restoration based on an auto-associative neural network provides a data restoring error of no more than 0.45% in the case of single failures and no more than 0.6% in case of double failures of the engine parameter registration sensor event.

Performance Evaluation of Laguerre Transform and Neural Network-based Cryptographic Techniques for Network Security

As the world evolves day by day with new technologies, there is a need to design a secure network in such a way that intruders and unauthorized persons should not have access to the network as well as information regarding the personnel in any firm. In this study, a new cryptographic technique for securing data transmission based on the LaplaceLaguerre polynomial (LLP) is developed and compared to an existing auto-associative neural network technique (AANNT).The performance of the LLPT and AANNT was tested with some selected files in a MATLAB environment and the results obtained provided comparative information (in respect of AANNT versus LLPT) as follows: encryption time (1.67 ms versus 3.9931s), decryption time (1.833 ms versus 2.1172s), throughput (26.2975 Kb/s versus 0.01098 Kb/s), memory consumption (3.349 KB versus 15.958 KB). From the compared results, it shows that AANNT offers a faster processing time, higher throughput, and takes up less memory space than the LLPT. However, cryptanalysis of the AANNT is possible if the network's weight and design are known; hence, the technique is unreliable for ensuring the data integrity and confidentiality of encrypted data. The proposed LLP cryptographic algorithm is designed to provide a higher security level by making the LLP algorithm computationally tedious to invert using the standard Laplace transform inversion method. When compared to the AANN-based cryptographic technique, cracking the algorithm to uncover the encryption key takes time. This shows the strength and robustness of the proposed LLP cryptographic algorithm against attacks, as well as its suitability for solving the problem of data privacy and security when compared to the AANN-based cryptographic algorithm.

Unsupervised damage assessment under varying ambient temperature based on an adjusted artificial neural network and new multivariate covariance-based distances

Temperature variability is one of the critical environmental conditions that causes confusing changes in structural properties and dynamic responses of bridges similar to damage. In this case, false alarms and mis-detection are among the major errors in health monitoring of such civil structures. High damage detectability is another significant challenge in bridge health monitoring. To deal with these issues, this article proposes an unsupervised damage assessment technique comprising two steps of data normalization and novelty detection. For the first step, an adjusted artificial neural network is considered to remove the effects of temperature variability from dynamic features (modal frequencies). This process is carried out by an auto-associative neural network by adjusting its hidden layer neurons through a new hyperparameter selection algorithm. Using normalized features obtained from the first step, this article proposes three multivariate covariance-based distances called linear dissimilarity analysis, multivariate Kullback–Leibler divergence, and multivariate Bregman distance to compute damage indices or novelty scores for damage assessment. The fundamental principles of these distances lie in three aspects: dividing the normalized features into segments, estimating the covariances of segmented feature sets, and incorporating the estimated covariances into the proposed distance measures. The major contributions of this article include proposing three non-parametric distance measures and developing an unsupervised data normalization framework via a new hyperparameter tuning algorithm for adjusting an artificial neural network. A concrete box-girder bridge is considered to verify the proposed approach, along with several comparative studies. Results show that the method presented here can mitigate severe temperature variability and increase damage detectability with superiority over some traditional and state-of-the-art damage assessment techniques.

Unsupervised data-driven method for damage localization using guided waves

Researchers have recently gained interest in ultrasonic guided waves (UGWs) for structural health monitoring (SHM). This interest has mainly grown thanks to the high sensitivity of UGWs to changes in the mechanical and material properties of the medium they travel in. Among the several types of UGWs, Lamb waves (LWs) have been successfully employed to detect, localize and quantify damage in thin-walled structures. To date, localization has been mainly performed through tomographic algorithms. Although those algorithms represent consolidated methods, they come with unsolved issues, such as artifacts generation in damage probability maps and strong dependency on the network of sensors installed to excite and sense diagnostic signals. Hence, more recently, supervised data-driven approaches have been propounded in the literature for LW-based damage diagnosis. Despite the good performance of such methods, a strong limitation is represented by the need for large high-fidelity datasets for training, which are generally not available for real-life structures. A possible solution is represented by unsupervised machine learning methods. Such methods have already been employed for damage detection in the SHM field, but no fully unsupervised approach to damage localization has been proposed yet. Hence, in this work, a framework combining an unsupervised data-driven method and an in-house tomographic approach is presented to process LWs to localize damage. Specifically, convolutional auto-associative neural networks (CAANNs) are employed to process diagnostic signals without requiring any prior feature extraction process. Furthermore, CAANNs predictions are processed to generate damage probability maps. The performance of the proposed method was tested against a numerical case study involving an Aluminum plate and against two experimental datasets of LWs acquired on a full-scale composite wing and a composite plate. Results showed that the proposed method outperformed classic damage diagnosis algorithms in terms of damage localization accuracy.

Deep learning denoising by dimension reduction: Application to the ORION-B line cubes

Context. The availability of large bandwidth receivers for millimeter radio telescopes allows for the acquisition of position-position-frequency data cubes over a wide field of view and a broad frequency coverage. These cubes contain a lot of information on the physical, chemical, and kinematical properties of the emitting gas. However, their large size coupled with an inhomogenous signal-to-noise ratio (S/N) are major challenges for consistent analysis and interpretation. Aims. We searched for a denoising method of the low S/N regions of the studied data cubes that would allow the low S/N emission to be recovered without distorting the signals with a high S/N. Methods. We performed an in-depth data analysis of the 13CO and C17O (1–0) data cubes obtained as part of the ORION-B large program performed at the IRAM 30 m telescope. We analyzed the statistical properties of the noise and the evolution of the correlation of the signal in a given frequency channel with that of the adjacent channels. This has allowed us to propose significant improvements of typical autoassociative neural networks, often used to denoise hyperspectral Earth remote sensing data. Applying this method to the 13CO (1–0) cube, we were able to compare the denoised data with those derived with the multiple Gaussian fitting algorithm ROHSA, considered as the state-of-the-art procedure for data line cubes. Results. The nature of astronomical spectral data cubes is distinct from that of the hyperspectral data usually studied in the Earth remote sensing literature because the observed intensities become statistically independent beyond a short channel separation. This lack of redundancy in data has led us to adapt the method, notably by taking into account the sparsity of the signal along the spectral axis. The application of the proposed algorithm leads to an increase in the S/N in voxels with a weak signal, while preserving the spectral shape of the data in high S/N voxels. Conclusions. The proposed algorithm that combines a detailed analysis of the noise statistics with an innovative autoencoder architecture is a promising path to denoise radio-astronomy line data cubes. In the future, exploring whether a better use of the spatial correlations of the noise may further improve the denoising performances seems to be a promising avenue. In addition, dealing with the multiplicative noise associated with the calibration uncertainty at high S/N would also be beneficial for such large data cubes.

Estimation of the desert dust balance and its relationship with environmental factors in the southern Baja California Peninsula

The emission, transport, and deposition of desert dust are influenced by environmental factors evaluated mainly as a function of precipitation, air temperature, and wind. The present study estimates the desert dust balance and its relationship with environmental factors for the period 1981–2020 in the Baja California Sur region, Mexico. Monthly data on wind, air temperature, precipitation, dust emission and deposition from in situ measurements, reanalysis, satellite estimates and a numerical parameterization scheme are used. First, the in-situ time series were reconstructed with a non-linear principal component analysis based on an autoassociative neural network. We then evaluated the performance of the satellite estimates and the reanalysis data using the matching technique, after which we quantified the dust emission in three source zones. To estimate the state of the climate, we obtained the annual cycle of monthly climatologies, and yearly averages. The effect of climate on the dust budget is estimated by means of non-parametric associations between the variables of interest. Finally, we estimate climate trends using Pettitt, Modified Man-Kendall and Theil-Sen tests. The results show that the data sets used represent the mean and variability of in situ measurements. There are three seasons with transitional periods for precipitation and air temperature. The wind shows a longitudinal west–east gradient. Its maximum intensity is from April to September. Northern zone was dominated by highest emissions. Dry deposition mechanism was the most representative. Emissions and dry deposition are controlled by wind intensity and precipitation, while precipitation controls wet deposition. The Pettitt test showed that abrupt changes in the time series coincide with periods where positive sunspot anomalies converge with negative PDO and SOI anomalies. Finally, trend analyses identified a temporal broadening of the dry season, warmer winters, and summers with increases of ~ 0.02 °C, predominantly increasing trends in dust emissions and wind intensity, and a progressive decrease in dust deposition mechanisms.

Nonlinear process monitoring based on generic reconstruction-based auto-associative neural network

A significant concern with statistical fault diagnosis is the large number of false alarms caused by the smearing effect. Although the reconstruction-based approach effectively solves this problem, most of them only focus on linear rather than nonlinear systems. In the present work, a generic reconstruction-based auto-associative neural network (GRBAANN) is proposed that uses the reconstruction-based approach to isolate simple and complex faults for nonlinear systems. Nevertheless, in GRBAANN, it is challenging to acquire a trivial solution for the reconstruction-based index, which is equivalent to a complex vector fixed-point problem. In this regard, the Steffensen method is employed to deal with this problem with an accelerated iterative process, which is appropriate for both single and multiple variable faults. The variable selection procedure is time-consuming but imperative for reconstruction-based approaches, with no exception to the proposed method. In order to ensure the real-time diagnosis for large-scale systems, the Sequential floating forward selection method with memory is proposed to minimize the computation time of the variable selection procedure. The effectiveness of the proposed GRBAANN scheme is illustrated through a validation example and an industrial example. Comparisons with the state-of-art methods are also presented.

Detecting anomalous sea-level states in North Sea tide gauge data using an autoassociative neural network

Abstract. The sea level in the North Sea is densely monitored by tide gauges. The data they provide can be used to solve different scientific and practical problems, including the validation of numerical models and the detection of extreme events. This study focuses on the detection of sea-level states with anomalous spatial correlations using autoassociative neural networks (AANNs), trained with different sets of observation- and model-based data. Such sea-level configurations are related to nonlinear ocean dynamics; therefore, neural networks appear to be the right candidate for their identification. The proposed network can be used to accurately detect such anomalies and localize them. We demonstrate that the atmospheric conditions under which anomalous sea-level states occur are characterized by high wind tendencies and pressure anomalies. The results show the potential of AANNs for accurately detecting the occurrence of such events. We show that the method works with AANNs trained on tide gauge records as well as with AANN trained with model-based sea surface height outputs. The latter can be used to enhance the representation of anomalous sea-level events in ocean models. Quantitative analysis of such states may help assess and improve numerical model quality in the future as well as provide new insights into the nonlinear processes involved. This method has the advantage of being easily applicable to any tide gauge array without preprocessing the data or acquiring any additional information.

Robust Hearing-Impaired Speaker Recognition from Speech using Deep Learning Networks in Native Language

Several research works in speaker recognition have grown recently due to its tremendous applications in security, criminal investigations and in other major fields. Identification of a speaker is represented by the way they speak, and not on the spoken words. Hence the identification of hearing-impaired speakers from their speech is a challenging task since their speech is highly distorted. In this paper, a new task has been introduced in recognizing Hearing Impaired (HI) speakers using speech as a biometric in native language Tamil. Though their speech is very hard to get recognized even by their parents and teachers, our proposed system accurately identifies them by adapting enhancement of their speeches. Due to the huge variety in their utterances, instead of applying the spectrogram of raw speech, Mel Frequency Cepstral Coefficient features are derived from speech and it is applied as spectrogram to Convolutional Neural Network (CNN), which is not necessary for ordinary speakers. In the proposed system of recognizing HI speakers, is used as a modelling technique to assess the performance of the system and this deep learning network provides 80% accuracy and the system is less complex. Auto Associative Neural Network (AANN) is used as a modelling technique and performance of AANN is only 9% accurate and it is found that CNN performs better than AANN for recognizing HI speakers. Hence this system is very much useful for the biometric system and other security related applications for hearing impaired speakers.

Effective Diagnosis of Lung Cancer via Various Data-Mining Techniques

One of the leading cancers for both genders worldwide is lung cancer. The occurrence of lung cancer has fully augmented since the early 19th century. In this manuscript, we have discussed various data mining techniques that have been employed for cancer diagnosis. Exposure to air pollution has been related to various adverse health effects. This work is subject to analysis of various air pollutants and associated health hazards and intends to evaluate the impact of air pollution caused by lung cancer. We have introduced data mining in lung cancer to air pollution, and our approach includes preprocessing, data mining, testing and evaluation, and knowledge discovery. Initially, we will eradicate the noise and irrelevant data, and following that, we will join the multiple informed sources into a common source. From that source, we will designate the information relevant to our investigation to be regained from that assortment. Following that, we will convert the designated data into a suitable mining process. The patterns are abstracted by utilizing a relational suggestion rule mining process. These patterns have revealed information, and this information is categorized with the help of an Auto Associative Neural Network classification method (AANN). The proposed method is compared with the existing method in various factors. In conclusion, the projected Auto associative neural network and relational suggestion rule mining methods accomplish a high accuracy status.

Selective connectivity enhances storage capacity in attractor models of memory function.

Autoassociative neural networks provide a simple model of how memories can be stored through Hebbian synaptic plasticity as retrievable patterns of neural activity. Although progress has been made along the last decades in understanding the biological implementation of autoassociative networks, their modest theoretical storage capacity has remained a major constraint. While most previous approaches utilize randomly connected networks, here we explore the possibility of optimizing network performance by selective connectivity between neurons, that could be implemented in the brain through creation and pruning of synaptic connections. We show through numerical simulations that a reconfiguration of the connectivity matrix can improve the storage capacity of autoassociative networks up to one order of magnitude compared to randomly connected networks, either by reducing the noise or by making it reinforce the signal. Our results indicate that the signal-reinforcement scenario is not only the best performing but also the most adequate for brain-like highly diluted connectivity. In this scenario, the optimized network tends to select synapses characterized by a high consensus across stored patterns. We also introduced an online algorithm in which the network modifies its connectivity while learning new patterns. We observed that, similarly to what happens in the human brain, creation of connections dominated in an initial stage, followed by a stage characterized by pruning, leading to an equilibrium state that was independent of the initial connectivity of the network. Our results suggest that selective connectivity could be a key component to make attractor networks in the brain viable in terms of storage capacity.

Development of Input Training Neural Networks for Multiple Sensor Fault Isolation

This paper considers the problem of inhibiting smearing effects for multiple sensor fault isolation. Although the reconstruction-based approach has received considerable attention and achieved excellent results, the investigation into nonlinear systems has been relatively limited. A new reconstruction-based input training neural network aided by the bound and branch method (BAB-RBITNN) is proposed for the nonlinear systems so as to achieve the desired performance of multiple sensor fault isolation. In BAB-RBITNN, a hypothetical faulty direction is naturally integrated into the objective testing function as residual weights for reducing the negative impacts of inaccurate sensor data, taking advantage of the non-static ITNN testing process. The weighted objective function is then considered as the fitness function to obtain the minimum predicted residual by the covariance matrix adaptation evolutionary strategy algorithm. The corresponding optimized fitness is regarded as the reconstruction-based index, which is the critical indicator of fault isolation in the proposed method. Besides, the BAB method is implemented to handle the combinatorial optimization problem instead of a brute search for intelligently pinpointing the real faulty sensors. The efficiency of the proposed method is shown through a validation example and two industrial examples. Comparisons with other methods, including reconstruction-based principal component analysis and reconstruction-based auto-associative neural network techniques, are also presented.

Satellite On-Board Change Detection via Auto-Associative Neural Networks

The increase in remote sensing satellite imagery with high spatial and temporal resolutions has enabled the development of a wide variety of applications for Earth observation and monitoring. At the same time, it requires new techniques that are able to manage the amount of data stored and transmitted to the ground. Advanced techniques for on-board data processing answer this problem, offering the possibility to select only the data of interest for a specific application or to extract specific information from data. However, the computational resources that exist on-board are limited compared to the ground segment availability. Alternatively, in applications such as change detection, only images containing changes are useful and worth being stored and sent to the ground. In this paper, we propose a change detection scheme that could be run on-board. It relies on a feature-based representation of the acquired images which is obtained by means of an auto-associative neural network (AANN). Once the AANN is trained, the dissimilarity between two images is evaluated in terms of the extracted features. This information can be subsequently turned into a change detection result. This study, which presents one of the first techniques for on-board change detection, yielded encouraging results on a set of Sentinel-2 images, even in light of comparison with a benchmark technique.

DEVELOPMENT OF A USER CLASSIFICATION METHOD ACCORDING TO THE LEVEL OF STRESS RESISTANCE USING A MODIFIED AUTO-ASSOCIATIVE NEURAL NETWORK

The work is aimed at solving a relevant for Ukraine and the world scientific and applied problem – the development of methods and means of constructing a psychophysical portrait of man to determine its stress resistance. For a number of professions, special methods of selecting personnel for special abilities are needed. Such abilities (qualities) include: health; motivation and will to act decisively, with the maximum effort of their physical, intellectual and psychological capabilities; physical training; level of mastery of technical means (or other means necessary in a critical situation); temperament; nature; emotional condition; accumulation of fatigue during the task and others. The article develops a method for classifying users according to the level of stress resistance on the basis of a modified auto-associative neural network using the Fletcher – Reeves algorithm. There are several ways to classify stress responses, but a more obvious study is to divide them into behavioral, intellectual, emotional and physiological manifestations of stress. The main signs of behavioral stress are psychomotor disorders (excessive muscle tension, winter breathing rhythm), lifestyle changes (changes in daily routine, sleep disturbances), occupational disorders (decreased productivity, increased fatigue), impaired social role functions, increased conflict, increasing aggression, etc.). In conditions of constant influence of stress factors on human consciousness, the decision can be ill-considered and made under the influence of emotions. To avoid such a situation in the workplace, the head of the company must monitor the condition of the operator. This human operator stress assessment system has been developed to predict human operator stress. The main task of the model is to predict based on the obtained parameters of the human condition (stress or not). Additional testing was added to increase the model’s performance based on input data on physical parameters, gender, age, height, and bad habits. The purpose of testing is to increase the accuracy of model prediction. Data from 92 respondents with 1180 records were selected for testing. Data set division: testing – 20 %, training / verification – 80%. The best results were achieved with the help of our own developed neural network based on Keras. Also in the paper the implementation of decision-making methods, the method of k-nearest neighbors and the modified auto-associative network was presented.

Asymmetric Weights and Retrieval Practice in an Autoassociative Neural Network Model of Paired-Associate Learning.

Rizzuto and Kahana (2001) applied an autoassociative Hopfield network to a paired-associate word learning experiment in which (1) participants studied word pairs (e.g., ABSENCE-HOLLOW), (2) were tested in one direction (ABSENCE-?) on a first test, and (3) were tested in the same direction again or in the reverse direction (?-HOLLOW) on a second test. The model contained a correlation parameter to capture the dependence between forward versus backward learning between the two words of a word pair, revealing correlation values close to 1.0 for all participants, consistent with neural network models that use the same weight for communication in both directions between nodes. We addressed several limitations of the model simulations and proposed two new models incorporating retrieval practice learning (e.g., the effect of the first test on the second) that fit the accuracy data more effectively, revealing substantially lower correlation values (average of .45 across participants, with zero correlation for some participants). In addition, we analyzed recall latencies, finding that second test recall was faster in the same direction after a correct first test. Only a model with stochastic retrieval practice learning predicted this effect. In conclusion, recall accuracy and recall latency suggest asymmetric learning, particularly in light of retrieval practice effects.

Fault Diagnosis and Reconstruction of Wind Turbine Anemometer Based on RWSSA-AANN

When the state of the wind turbine sensors, especially the anemometer, appears abnormal it will cause unnecessary wind loss and affect the correctness of other parameters of the whole system. It is very important to build a simple and accurate fault diagnosis model. In this paper, the model has been established based on the Random Walk Improved Sparrow Search Algorithm to optimize auto-associative neural network (RWSSA-AANN), and is used for fault diagnosis of wind turbine group anemometers. Using the cluster analysis, six wind turbines are determined to be used as a wind turbine group. The 20,000 sets of normal historical data have been used for training and simulating of the model, and the single and multiple fault states of the anemometer are simulated. Using this model to analyze the wind speed supervisory control and data acquisition system (SCADA) data of six wind turbines in a wind farm from 2013 to 2017, can effectively diagnose the fault state and reconstruct the fault data. A comparison of the results obtained using the model developed in this work has also been made with the corresponding results generated using AANN without optimization and AANN optimized by genetic algorithm. The comparison results indicate that the model has a higher accuracy and detection rate than AANN, genetic algorithm auto-associative neural network (GA-AANN), and principal component analysis (PCA).

자가연상신경망을 이용한 밀링 공정에서의 공구 마모 모니터링

자가연상신경망을 이용한 밀링 공정에서의 공구 마모 모니터링

Attribute‐Associated Neuron Modeling and Missing Value Imputation for Incomplete Data

The imputation of missing values is an important research content in incomplete data analysis. Based on the auto associative neural network (AANN), this paper conducts regression modeling for incomplete data and imputes missing values. Since the AANN can estimate missing values in multiple missingness patterns efficiently, we introduce incomplete records into the modeling process and propose an attribute cross fitting model (ACFM) based on AANN. ACFM reconstructs the path of data transmission between output and input neurons and optimizes the model parameters by training errors of existing data, thereby improving its own ability to fit relations between attributes of incomplete data. Besides, for the problem of incomplete model input, this paper proposes a model training scheme, which sets missing values as variables and makes missing value variables update with model parameters iteratively. The method of local learning and global approximation increases the precision of model fitting and the imputation accuracy of missing values. Finally, experiments based on several datasets verify the effectiveness of the proposed method.