Undercomplete Autoencoders Research Articles

Detection of Anomalies in Natural Complicated Data Structures Based on a Hybrid Approach

A hybrid approach is proposed to detect anomalies in natural complicated data structures with high noise levels. The approach includes the application of an autoencoder neural network and singular spectrum analysis (SSA) with an adaptive anomaly detection algorithm (AADA) developed by the authors. The autoencoder is the quintessence of the representation learning algorithm, and it projects (selects) data features. Here, under-complete autoencoders are used. They are a product of the development of the principal component method and allow one to approximate complex nonlinear dependencies. Singular spectrum analysis decomposes data through the singular decomposition of matrix trajectories and makes it possible to detect the data structure in the noise. The AADA is based on the combination of wavelet transforms with threshold functions. Combinations of different constructions of wavelet transformation with threshold functions are widely applied to tasks relating to complex data processing. However, when the noise level is high and there is no complete knowledge of a useful signal, anomaly detection is not a trivial problem and requires a complex approach. This paper considers the use of adaptive threshold functions, the parameters of which are estimated on a probabilistic basis. Adaptive thresholds and a moving time window are introduced. The efficiency of the proposed method in detecting anomalies in neutron monitor data is illustrated. Neutron monitor data record cosmic ray intensities. We used neutron monitor data from ground stations. Anomalies in cosmic rays can create serious radiation hazards for people as well as for space and ground facilities. Thus, the diagnostics of anomalies in cosmic ray parameters is quite topical, and research is being carried out by teams from different countries. A comparison of the results for the autoencoder + AADA and SSA + AADA methods showed the higher efficiency of the autoencoder + AADA method. A more flexible NN apparatus provides better detection of short-period anomalies that have complicated structures. However, the combination of SSA and the AADA is efficient in the detection of long-term anomalies in cosmic rays that occur during strong magnetic storms. Thus, cosmic ray data analysis requires a more complex approach, including the use of the autoencoder and SSA with the AADA.

A Fault Diagnosis Technique for Wind Turbine Gearbox: An Approach using Optimized BLSTM Neural Network with Undercomplete Autoencoder

The gearbox is one of the critical components of a wind turbine. Proactive maintenance of wind turbine gearboxes is crucial to decrease maintenance and operational costs and the long downtime of the complete system. As the gearbox is a significant part of the wind turbine, a fault in the gearbox leads to the breakdown of the wind turbine system. Hence, it is important to study and analyze the faults in wind turbine gearbox systems. In this article, a neural network-based model, a Bidirectional Long Short-Term Memory (BLSTM) fused with an autoencoder is intended to categorize the condition of the gearbox into good or bad (broken tooth) condition. Feature learning and reduction are achieved extensively through the autoencoder. This improves the performance of the BLSTM model regarding time complexity and classification accuracy. This model has been applied with time series vibration data of the gearbox in a wind turbine system. The suggested model's performance is analyzed using an openly available wind turbine gearbox vibration dataset. The result showed that BLSTM accuracy with an under-complete autoencoder is highly robust and appropriate for the health monitoring of wind turbine gearbox systems using time series data. Also, in order to illustrate the advantage of the projected model for fault analysis and diagnosis in wind turbine gearbox, the throughput or time complexity of training and testing of the split dataset is compared with the conventional BLSTM model.

Data-driven reduced-order modeling of spatiotemporal chaos with neural ordinary differential equations.

Dissipative partial differential equations that exhibit chaotic dynamics tend to evolve to attractors that exist on finite-dimensional manifolds. We present a data-driven reduced-order modeling method that capitalizes on this fact by finding a coordinate representation for this manifold and then a system of ordinary differential equations (ODEs) describing the dynamics in this coordinate system. The manifold coordinates are discovered using an undercomplete autoencoder-a neural network (NN) that reduces and then expands dimension. Then, the ODE, in these coordinates, is determined by a NN using the neural ODE framework. Both of these steps only require snapshots of data to learn a model, and the data can be widely and/or unevenly spaced. Time-derivative information is not needed. We apply this framework to the Kuramoto-Sivashinsky equation for domain sizes that exhibit chaotic dynamics with again estimated manifold dimensions ranging from 8 to 28. With this system, we find that dimension reduction improves performance relative to predictions in the ambient space, where artifacts arise. Then, with the low-dimensional model, we vary the training data spacing and find excellent short- and long-time statistical recreation of the true dynamics for widely spaced data (spacing of ∼ 0.7 Lyapunov times). We end by comparing performance with various degrees of dimension reduction and find a "sweet spot" in terms of performance vs dimension.

Denoising Letter Images from Scanned Invoices Using Stacked Autoencoders

Invoice document digitization is crucial for efficient management in industries. The scanned invoice image is often noisy due to various reasons. This affects the OCR (optical character recognition) detection accuracy. In this paper, letter data obtained from images of invoices are denoised using a modified autoencoder based deep learning method. A stacked denoising autoencoder (SDAE) is implemented with two hidden layers each in encoder network and decoder network. In order to capture the most salient features of training samples, a undercomplete autoencoder is designed with non-linear encoder and decoder function. This autoencoder is regularized for denoising application using a combined loss function which considers both mean square error and binary cross entropy. A dataset consisting of 59,119 letter images, which contains both English alphabets (upper and lower case) and numbers (0 to 9) is prepared from many scanned invoices images and windows true type (.ttf) files, are used for training the neural network. Performance is analyzed in terms of Signal to Noise Ratio (SNR), Peak Signal to Noise Ratio (PSNR), Structural Similarity Index (SSIM) and Universal Image Quality Index (UQI) and compared with other filtering techniques like Nonlocal Means filter, Anisotropic diffusion filter, Gaussian filters and Mean filters. Denoising performance of proposed SDAE is compared with existing SDAE with single loss function in terms of SNR and PSNR values. Results show the superior performance of proposed SDAE method.

Composition analysis of ceramic raw materials using laser-induced breakdown spectroscopy and autoencoder neural network.

In the ceramic production process, the content of Si, Al, Mg, Fe, Ti and other elements in the ceramic raw materials has an important impact on the quality of the ceramic products. Exploring a method that can quickly and accurately analyze the content of key elements in ceramic raw materials is of great significance to improve the quality of ceramic products. In this work, laser-induced breakdown spectroscopy (LIBS) is used for rapid analysis of ceramic raw materials. The chemical element composition and content of ceramic raw materials are quite different, which leads to serious matrix effects. Building an artificial neural network model is an effective way to solve the complex matrix effects, but model training can easily lead to overfitting due to the high number of spectral features and the limited number of samples. In order to solve this problem, we propose a feature extraction method that combines the linear regression (LR) and the sparse and under-complete autoencoder (SUAC) neural network. This LR + SUAC method performs nonlinear feature extraction and dimension reduction on high-dimensional spectral data. The spectral data dimension is reduced from 8188 to 100 through the LR layer, and further reduced to 32 through the SUAC encoding layer. Further, a quantitative analysis model for the elemental composition of ceramic raw materials is established by the combination of LR + SUAC and Back Propagation Neural Network (BPNN). Since the input data dimension and redundant information are greatly reduced by LR + SUAC, the overfitting problem of BPNN is greatly reduced. Experiment results showed that the LR + SUAC + BPNN method obtained the best quantitative analysis performance compared with several other methods in the cross-validation process.

A neuro evolutionary algorithm for patient calibrated prediction of survival in Glioblastoma patients

Background and objectivesGlioblastoma multiforme (GBM) is the most common and malignant type of primary brain tumors. Radiation therapy (RT) plus concomitant and adjuvant Temozolomide (TMZ) constitute standard treatment of GBM. Existing models for GBM growth do not consider the effect of different schedules on tumor growth and patient survival. However, clinical trials show that treatment schedule and drug dosage significantly affect patient survival. The goal is to provide a patient calibrated model for predicting survival according to the treatment schedule. MethodsWe propose a top-down method based on artificial neural networks (ANN) and genetic algorithm (GA) to predict survival of GBM patients. A feed forward undercomplete Autoencoder network is integrated with the neuro-evolutionary (NE) algorithm in order to extract a compressed representation of input clinical data. The proposed NE algorithm uses GA to obtain optimal architecture of a multi-layer perceptron (MLP). Taguchi L16 orthogonal design of experiments is used to tune parameters of the proposed NE algorithm. Finally, the optimal MLP is used to predict survival of GBM patients. ResultsData from 8 related clinical trials have been collected and integrated to train the model. From 847 evaluable cases, 719 were used for train and validation and the remaining 128 cases were used to test the model. Mean absolute error of the predictions on the test data is 0.087 months which shows excellent performance of the proposed model in predicting survival of the patients. Also, the results show that the proposed NE algorithm is superior to other existing models in both the mean and variability of the prediction error.

Triple-Regularized Latent Subspace Discriminative Regression for Hyperspectral Image Classification

There are various types and distributions of noise in hyperspectral images. However, the existing classification models are vulnerable to noise in the data. To improve the robustness of the classification models, a novel hyperspectral image classification model is proposed, named triple-regularized latent subspace discriminative regression (TRLSDR). The core idea of TRLSDR is to add a latent subspace to the standard discriminative least squares regression model to extract high-order features from the visual space by undercomplete autoencoder, and then use clean data for classification. Three regularizers are introduced in this process: Tikhonov regularizer is used to avoid overfitting; Laplacian regularizer is used to capture the neighborhood relationship; and low-rank regularizer is used to alleviate the error of Laplacian matrix construction caused by noise in original samples. We designed experiments on five hyperspectral image datasets and the results show that the proposed model is superior to the existing regression models.

Evaluation of Anomaly Detection of an Autoencoder Based on Maintenace Information and Scada-Data

The usage of machine learning techniques is widely spread and has also been implemented in the wind industry in the last years. Many of these techniques have shown great success but need to constantly prove the expectation of functionality. This paper describes a new method to monitor the health of a wind turbine using an undercomplete autoencoder. To evaluate the health monitoring quality of the autoencoder, the number of anomalies before an event has happened are to be considered. The results show that around 35% of all historical events that have resulted into a failure show many anomalies. Furthermore, the wind turbine subsystems which are subject to good detectability are the rotor system and the control system. If only one third of the service duties can be planned in advance, and thereby the scheduling time can be reduced, huge cost saving potentials can be seen.

Compact and Low-Power Neural Spike Compression Using Undercomplete Autoencoders.

Implantable microsystems that collect and transmit neural data are becoming very useful entities in the field of neuroscience. Limited by high data rates, on-chip compression is often required to transmit the recorded data without causing power dissipation at levels that would damage sensitive brain tissue. This paper presents a data compression system designed for brain-computer interfaces (BCIs) based on undercomplete autoencoders. To the best of our knowledge, the proposed system is the first to achieve an average spike reconstruction quality of 14-dB signal-to-noise-and-distortion ratio (SNDR) at a 32× compression ratio (CR), 18-dB SNDR at a 16× CR, 22-dB SNDR at an 8× CR, and 35-dB SNDR at a 4× CR of neural spikes. The spike detection and autoencoder-based compression modules are designed and implemented in a standard 45-nm CMOS process. The post-synthesis simulation results report that the compression module consumes between 1.4 and 222.5 [Formula: see text] of power per channel and takes between 0.018 and 0.082mm2 of silicon area, depending on the desired CR and number of channels.