Nonlinear Feature Extraction Research Articles

Temporal-Spatial Neighborhood Enhanced Sparse Autoencoder for Nonlinear Dynamic Process Monitoring

Data-based process monitoring methods have received tremendous attention in recent years, and modern industrial process data often exhibit dynamic and nonlinear characteristics. Traditional autoencoders, such as stacked denoising autoencoders (SDAEs), have excellent nonlinear feature extraction capabilities, but they ignore the dynamic correlation between sample data. Feature extraction based on manifold learning using spatial or temporal neighbors has been widely used in dynamic process monitoring in recent years, but most of them use linear features and do not take into account the complex nonlinearities of industrial processes. Therefore, a fault detection scheme based on temporal-spatial neighborhood enhanced sparse autoencoder is proposed in this paper. Firstly, it selects the temporal neighborhood and spatial neighborhood of the sample at the current time within the time window with a certain length, the spatial similarity and time serial correlation are used for weighted reconstruction, and the reconstruction combines the current sample as the input of the sparse stack autoencoder (SSAE) to extract the correlation features between the current sample and the neighborhood information. Two statistics are constructed for fault detection. Considering that both types of neighborhood information contain spatial-temporal structural features, Bayesian fusion strategy is used to integrate the two parts of the detection results. Finally, the superiority of the method in this paper is illustrated by a numerical example and the Tennessee Eastman process.

A new stable nonlinear textural feature extraction method based EEG signal classification method using substitution Box of the Hamsi hash function: Hamsi pattern

BackgroundThe EEG signal classification is crucial for epileptic seizure prediction. Therefore, many machine learning model has been presented to classify EEG signals accurately. Material and MethodThis work presents a novel automated EEG classification method by using a novel nonlinear feature extractor, and it is called as Hamsi-Pat. It uses the substitution box (S-Box) of the Hamsi hash function. As stated in the literature, S-Boxes have generally used for diffusion in symmetric encryption (especially block ciphers) methods and cryptologic hash functions. Since it is a nonlinear structure, this work aims to illustrate the merit of an S-Box for feature generation. Therefore, a new generation feature generator, which is Hamsi-Pat, is presented by using S-Box of the Hamsi hash function, and a novel EEG classification method is proposed by using Hamsi-Pat. The presented biomedical signal classification method has three elementary phases, and these phases are Hamsi-Pat based multileveled feature generation, iterative neighborhood component analysis (INCA) selector based feature dimension reduction, and classification using k nearest neighborhood (kNN) classifier. The presented Hamsi-Pat and INCA based methods were tested on Bonn electroencephalography (EEG) datasets. ResultThis model yielded 99.20% classification accuracy on the used EEG dataset for five classes case and it yielded 100.0% accuracies for other cases. ConclusionThese results obviously denoted that the S-Boxes can be considered as a feature generator, and a novel S-Box based feature generation research area can be defined as textural feature generation and statistical feature generation.

Nonlinear Feature Extraction Through Manifold Learning in an Electronic Tongue Classification Task.

A nonlinear feature extraction-based approach using manifold learning algorithms is developed in order to improve the classification accuracy in an electronic tongue sensor array. The developed signal processing methodology is composed of four stages: data unfolding, scaling, feature extraction, and classification. This study aims to compare seven manifold learning algorithms: Isomap, Laplacian Eigenmaps, Locally Linear Embedding (LLE), modified LLE, Hessian LLE, Local Tangent Space Alignment (LTSA), and t-Distributed Stochastic Neighbor Embedding (t-SNE) to find the best classification accuracy in a multifrequency large-amplitude pulse voltammetry electronic tongue. A sensitivity study of the parameters of each manifold learning algorithm is also included. A data set of seven different aqueous matrices is used to validate the proposed data processing methodology. A leave-one-out cross validation was employed in 63 samples. The best accuracy () was obtained when the methodology uses Mean-Centered Group Scaling (MCGS) for data normalization, the t-SNE algorithm for feature extraction, and k-nearest neighbors (kNN) as classifier.

Gated Stacked Target-Related Autoencoder: A Novel Deep Feature Extraction and Layerwise Ensemble Method for Industrial Soft Sensor Application.

These days, data-driven soft sensors have been widely applied to estimate the difficult-to-measure quality variables in the industrial process. How to extract effective feature representations from complex process data is still the difficult and hot spot in the soft sensing application field. Deep learning (DL), which has made great progresses in many fields recently, has been used for process monitoring and quality prediction purposes for its outstanding nonlinear modeling and feature extraction abilities. In this work, deep stacked autoencoder (SAE) is introduced to construct a soft sensor model. Nevertheless, conventional SAE-based methods do not take information related to target values in the pretraining stage and just use the feature representations in the last hidden layer for final prediction. To this end, a novel gated stacked target-related autoencoder (GSTAE) is proposed for improving modeling performance in view of the above two issues. By adding prediction errors of target values into the loss function when executing a layerwise pretraining procedure, the target-related information is used to guide the feature learning process. Besides, gated neurons are utilized to control the information flow from different layers to the final output neuron that take full advantage of different levels of abstraction representations and quantify their contributions. Finally, the effectiveness and feasibility of the proposed approach are verified in two real industrial cases.

A Primer on Machine Learning.

A Primer on Machine Learning.

Spatial Domain-Based Nonlinear Residual Feature Extraction for Identification of Image Operations

In this paper, a novel approach that uses a deep learning technique is proposed to detect and identify a variety of image operations. First, we propose the spatial domain-based nonlinear residual (SDNR) feature extraction method by constructing residual values from locally supported filters in the spatial domain. By applying minimum and maximum operators, diversity and nonlinearity are introduced; moreover, this construction brings nonsymmetry to the distribution of SDNR samples. Then, we propose applying a deep learning technique to the extracted SDNR features to detect and classify a variety of image operations. Many experiments have been conducted to verify the performance of the proposed approach, and the results indicate that the proposed method performs well in detecting and identifying the various common image postprocessing operations. Furthermore, comparisons between the proposed approach and the existing methods show the superiority of the proposed approach.

Parallel multi-scale entropy and it's application in rolling bearing fault diagnosis

As a prevalent tool for complexity measure of time series, multi-scale entropy has been utilized effectively to extract nonlinear dynamic features from rolling bearing vibration signals. However, the coarse-graining procedure of multi-scale entropy ignores the frequency-domain characteristics of the data, and finally leads to information aliasing and limited state identification capability. In this paper, the frequency characteristic changes of the data in the process of coarse-graining procedure are analyzed, and then a novel nonlinear dynamic method named parallel multi-scale entropy is proposed. In the proposed method, the aliasing effect caused by the reduction of data points is eliminated, and the final estimated sample entropy is replaced by the entropy average of multiple parallel intermediate series. Experimental results show that the proposed parallel multi-scale entropy holds better discrimination and stability compared with the original version, and can provide more excellent feature vectors for intelligent fault diagnosis.

Rolling Bearings Fault Diagnosis Based on Improved Complete Ensemble Empirical Mode Decomposition with Adaptive Noise, Nonlinear Entropy, and Ensemble SVM

Rolling bearings are fundamental elements that play a crucial role in the functioning of rotating machines; thus, fault diagnosis of rolling bearings is of great significance to reduce catastrophic failures and heavy economic loss. However, the vibration signals of rolling bearings are often nonlinear and nonstationary, resulting in difficulty for feature extraction and fault recognition. In this paper, a hybrid method for multiple fault diagnosis of rolling bearings is presented. The bearing vibration signals are decomposed with the improved complete ensemble empirical mode decomposition with adaptive noise (ICEEMDAN) to denoise and extract nonlinear entropy features. The nonlinear entropy features are further processed to select the more discriminative fault features and to reduce feature dimension. Then a multi-class intelligent recognition model based on ensemble support vector machine (ESVM) is constructed to diagnose different bearing fault modes as well as fault severities. The effectiveness of the proposed method is assessed via experimental case studies of rolling bearings under multiple operational conditions (i.e., speeds and loads). The results show that our method gives better diagnosis results as compared to some existing approaches.

Isomap-Based Three-Dimensional Operational Modal Analysis

In order to identify the modal parameters of time invariant three-dimensional engineering structures with damping and small nonlinearity, a novel isometric feature mapping (Isomap)-based three-dimensional operational modal analysis (OMA) method is proposed to extract nonlinear features in this paper. Using this Isomap-based OMA method, a low-dimensional embedding matrix is multiplied by a transformation matrix to obtain the original matrix. We find correspondence relationships between the low-dimensional embedding matrix and the modal coordinate response and between the transformation matrix and the modal shapes. From the low-dimensional embedding matrix, the natural frequencies can be determined using a Fourier transform and the damping ratios can be identified by the random decrement technique or natural excitation technique. The modal shapes can be estimated from the Moore–Penrose matrix inverse of the low-dimensional embedding matrix. We also discuss the effects of different parameters (i.e., number of neighbors and matrix assembly) on the results of modal parameter identification. The modal identification results from numerical simulations of the vibration response signals of a cylindrical shell under white noise excitation demonstrate that the proposed method can identify the modal shapes, natural frequencies, and ratios of three-dimensional structures in operational conditions only from the vibration response signals.

A hyperspectral anomaly detection framework based on segmentation and convolutional neural network algorithms

ABSTRACT Hyperspectral imagery (HSI) creates a lot of applications in target or anomaly detection due to their rich spectral content. Generally, one scene of an HSI contains more than one class. Therefore, the Gaussian distribution assumption of the background fails. Furthermore, the high dimensionality of data makes background modelling more difficult by increasing redundancy and disturbances. In this paper, a segmented-distance based anomaly detection method is proposed for HSI. The proposed method is based on segmentation and takes advantage of the statistical properties of the segmented areas to suppress the false-alarms. In addition to that, nonlinear feature extraction based on convolutional stacked auto-encoder (SAE) neural networks are implemented to extract deep and nonlinear relations from the input data. Both 1-D and 2-D convolutional layers are investigated. The proposed method is tested on the three different datasets. The experimental results show that the integration of segmentation and deep feature extraction generally performs better than other state-of-the-art methods.

Spiking network optimized for word recognition in noise predicts auditory system hierarchy.

The auditory neural code is resilient to acoustic variability and capable of recognizing sounds amongst competing sound sources, yet, the transformations enabling noise robust abilities are largely unknown. We report that a hierarchical spiking neural network (HSNN) optimized to maximize word recognition accuracy in noise and multiple talkers predicts organizational hierarchy of the ascending auditory pathway. Comparisons with data from auditory nerve, midbrain, thalamus and cortex reveals that the optimal HSNN predicts several transformations of the ascending auditory pathway including a sequential loss of temporal resolution and synchronization ability, increasing sparseness, and selectivity. The optimal organizational scheme enhances performance by selectively filtering out noise and fast temporal cues such as voicing periodicity, that are not directly relevant to the word recognition task. An identical network arranged to enable high information transfer fails to predict auditory pathway organization and has substantially poorer performance. Furthermore, conventional single-layer linear and nonlinear receptive field networks that capture the overall feature extraction of the HSNN fail to achieve similar performance. The findings suggest that the auditory pathway hierarchy and its sequential nonlinear feature extraction computations enhance relevant cues while removing non-informative sources of noise, thus enhancing the representation of sounds in noise impoverished conditions.

A new approach for small sample face recognition with pose variation by fusing Gabor encoding features and deep features

Small sample and pose variation are two critical technical problems in automatic face recognition (AFR). The combination of these two difficulties will seriously affect the performance of face recognition and restrict the wide application of AFR. To address this problem, we propose a new multi-feature fusion framework. First, we propose a pixel-level data augmentation algorithm based on manifold subspace partition, which constructs virtual samples in the original face image space to achieve training sample expansion and diversity enhancement. Then, we propose a feature-level data augmentation based on Gabor transformation, which can capture multi-level face features through multi-scale and multi-direction Gabor filters to realize the face expansion in feature space. To eliminate the data redundancy and interference information generated by Gabor feature augmentation, a Gabor feature encoding algorithm is proposed to construct the compressed Gabor feature vector. In addition, we propose a small scale adaptive deep CNN model, which is suitable for small sample datasets and can effectively extract nonlinear deep features of pose-varied faces. Finally, Gabor encoding features and nonlinear deep features are combined for small sample face recognition with pose variation. Experiment results based on two face datasets prove the effectiveness of the proposed multi-feature fusion framework.

An efficient technique for extraction of nonlinear dynamic features in a model-based feature extraction scheme for machine learning–based structural health monitoring

Super-harmonic components in response to the harmonic excitation are sensitive indicators of damages such as breathing cracks in beams or kissing bonds in adhesive joints. In a model-based damage identification process using pattern recognition, these damage indicators can be extracted from the finite element model for all probable damage cases using stepped-sine simulation that necessitates nonlinear transient dynamic analysis with high computational costs. In this study, a procedure based on nonlinear autoregressive with exogenous input model is introduced as an alternative shortcut method for extraction of the damage indicators. As a case study, the finite element model of a beam connected to a rigid support via a flexible adhesive layer was used to investigate the efficiency of the proposed method. Kissing bond was introduced to the model as the source of nonlinearity via contact elements. The results prove that the super-harmonic components of orders up to 3, extracted from the nonlinear autoregressive with exogenous input model, agreed well with those extracted directly from the finite element model, whereas the computational time is reduced by a factor of 1/5. Consequently, the proposed method is very advantageous in the stage of damage pattern database creation in a real-world model-based damage identification process based on pattern recognition.

Nonlinear ICA of fMRI reveals primitive temporal structures linked to rest, task, and behavioral traits

Accumulating evidence from whole brain functional magnetic resonance imaging (fMRI) suggests that the human brain at rest is functionally organized in a spatially and temporally constrained manner. However, because of their complexity, the fundamental mechanisms underlying time-varying functional networks are still not well understood. Here, we develop a novel nonlinear feature extraction framework called local space-contrastive learning (LSCL), which extracts distinctive nonlinear temporal structure hidden in time series, by training a deep temporal convolutional neural network in an unsupervised, data-driven manner. We demonstrate that LSCL identifies certain distinctive local temporal structures, referred to as temporal primitives, which repeatedly appear at different time points and spatial locations, reflecting dynamic resting-state networks. We also show that these temporal primitives are also present in task-evoked spatiotemporal responses. We further show that the temporal primitives capture unique aspects of behavioral traits such as fluid intelligence and working memory. These results highlight the importance of capturing transient spatiotemporal dynamics within fMRI data and suggest that such temporal primitives may capture fundamental information underlying both spontaneous and task-induced fMRI dynamics.

Class-information–incorporated kernel entropy component analysis with application to bearing fault diagnosis

Rolling bearings are present ubiquitously in industrial fields; timely fault diagnosis is of crucial significance in avoiding serious catastrophe. The extraction of ideal fault feature is a challenging task in vibration-based bearing fault detection. In this article, a novel method called class-information–incorporated kernel entropy component analysis is proposed for bearing fault diagnosis. The method is developed based on the Hebbian learning theory of neural network and the kernel entropy component analysis which attempts to compress the most Renyi quadratic entropy of input dataset after dimension reduction and presents a good performance for nonlinear feature extraction. Class-information–incorporated kernel entropy component analysis can take advantage of the label information of training samples to guide dimensional reduction and still follow the same simple mathematical formulation as kernel entropy component analysis. The high-dimensional feature dataset including time-domain, frequency-domain, and time–frequency domain characteristic parameters is first derived from the vibration signals. Then, the intrinsic geometric features are extracted by class-information–incorporated kernel entropy component analysis, and a classification strategy based on fusion information is applied to recognize different operating conditions of bearings. The experimental results demonstrated the feasibility and effectiveness of the proposed method.

A Deep Learning Approach for Identifying User Communities Based on Geographical Preferences and Its Applications to Urban and Environmental Planning

Understanding human mobility plays a vital role in urban and environmental planning as cities continue to grow. Ubiquitous geo-location, localization technology, and availability of big-data-ready computing infrastructure have enabled the development of more sophisticated models to characterize human mobility in urban areas. In this work, our main goal is to extract spatio-temporal features that characterize user mobility and, based on the similarity of these features, identify user communities . To this end, we propose a novel approach that leverages image processing techniques to represent user geographical preferences as images and then apply deep convolutional autoencoders to extract latent spatio-temporal mobility features from these images. These features are then fed to a clustering algorithm that identifies the underlying community structures. We use a diverse urban mobility dataset to validate the proposed framework. Our results show that the proposed framework is able to significantly increase the similarity between intra-community nodes (by up to 107%) as well as dissimilarity between inter-community nodes (up to 54%) when compared against no pre-processing of the datasets, i.e without pre-processing the datasets through any feature fusion method. Moreover, it was also able to reach up to 100% improvement when compared against community identification using Principal Component Analysis (PCA). Our results also show that the proposed approach yields significant increase in contact time amongst users belonging to the same community, by up to 80% when compared to the average contact time when not considering community structures, and by up to 150% when compared to the baseline. To the best of our knowledge, our proposal is the first to consider deep convolutional autoencoding to perform automatic extraction of non-linear spatio-temporal mobility features characterizing individual users from raw mobility datasets with the goal of identifying user communities.

Nonlinear Dynamic Feature Extraction Based on Phase Space Reconstruction for the Classification of Speech and Emotion

Due to the shortcomings of linear feature parameters in speech signals, and the limitations of existing time- and frequency-domain attribute features in characterizing the integrity of the speech information, in this paper, we propose a nonlinear method for feature extraction based on the phase space reconstruction (PSR) theory. First, the speech signal was analyzed using a nonlinear dynamic model. Then, the model was used to reconstruct a one-dimensional time speech signal. Finally, nonlinear dynamic (NLD) features based on the reconstruction of the phase space were extracted as the new characteristic parameters. Then, the performance of NLD features was verified by comparing their recognition rates with those of other features (NLD features, prosodic features, and MFCC features). Finally, the Korean isolated words database, the Berlin emotional speech database, and the CASIA emotional speech database were chosen for validation. The effectiveness of the NLD features was tested using the Support Vector Machine classifier. The results show that NLD features not only have high recognition rate and excellent antinoise performance for speech recognition tasks but also can fully characterize the different emotions contained in speech signals.

Graph-Based Decoding Model for Functional Alignment of Unaligned fMRI Data

Aggregating multi-subject functional magnetic resonance imaging (fMRI) data is indispensable for generating valid and general inferences from patterns distributed across human brains. The disparities in anatomical structures and functional topographies of human brains warrant aligning fMRI data across subjects. However, the existing functional alignment methods cannot handle well various kinds of fMRI datasets today, especially when they are not temporally-aligned, i.e., some of the subjects probably lack the responses to some stimuli, or different subjects might follow different sequences of stimuli. In this paper, a cross-subject graph that depicts the (dis)similarities between samples across subjects is used as a priori for developing a more flexible framework that suits an assortment of fMRI datasets. However, the high dimension of fMRI data and the use of multiple subjects makes the crude framework time-consuming or unpractical. To address this issue, we further regularize the framework, so that a novel feasible kernel-based optimization, which permits non-linear feature extraction, could be theoretically developed. Specifically, a low-dimension assumption is imposed on each new feature space to avoid overfitting caused by the high-spatial-low-temporal resolution of fMRI data. Experimental results on five datasets suggest that the proposed method is not only superior to several state-of-the-art methods on temporally-aligned fMRI data, but also suitable for dealing with temporally-unaligned fMRI data.

Batch process monitoring using multiway Laplacian autoencoders

AbstractIn this paper, a novel nonlinear method named multiway Laplacian autoencoder (MLAE) is proposed for batch process monitoring. Autoencoder (AE) is an effective unsupervised learning neural network for nonlinear feature extraction. Compared with traditional AEs, the proposed method has two main advantages. Firstly, traditional AEs usually ignore the local structure of the original dataset. The proposed MLAE method integrates graph Laplacian regularization to the loss function, and, thus, the local structure of the normal process data is fully considered. Secondly, the Laplacian matrix of the regularization term is constructed by an average local affinity matrix of all batch runs, which contains the information of the stochastic deviations among batches. Furthermore, two statistics, ie, H2 and SPE statistics, are developed based on the extracted hidden representation and the retained reconstruction error. The effectiveness and advantages of the MLAE‐based monitoring strategy are illustrated by a benchmark penicillin fermentation process and a real E. coli fermentation process.

Sparse representation preserving embedding based on extreme learning machine for process monitoring

Extreme learning machine (ELM) is a fast learning mechanism used in many domains. Unsupervised ELM has improved to extract nonlinear features. A nonlinear dynamic process monitoring method named sparse representation preserving embedding based on ELM (SRPE-ELM) is proposed in this paper. First, the noise is removed by sparse representation and the sparse coefficient is applied to construct the adjacency graph. The adjacency graph with a data-adaptive neighborhood can extract dynamic manifold structure better than a specified neighborhood parameter. Secondly, a new objection function considered the sparse reconstruction and output weights is established to extract nonlinear dynamic manifold structure. Thirdly, the statistic SPE and T2 based on SRPE-ELM are built to monitor the whole process. Finally, SRPE-ELM is applied in the IRIS data classification example, a numerical case and Tennessee Eastman benchmark process to verify the effectiveness of process monitoring.