Slow Feature Analysis Research Articles

A Systematic Approach to Dynamic Monitoring of Industrial Processes Based on Second-Order Slow Feature Analysis

Slow feature analysis has proven to be an effective process monitoring and fault diagnosis approach. By isolating temporal behaviors from steady-state variations in process data, slow feature analysis enables a concurrent monitoring of operating condition and process dynamics, based on which false alarms triggered by nominal operating condition deviations can be effectively removed. However, the present formulation of slow feature analysis only makes use of the first-order time difference of time series data, thereby falling short of addressing high-order dynamics in process operations. In this work, we propose a second-order formulation of slow feature analysis, and further develop a systematic framework for process monitoring and fault diagnosis, which can provide more meaningful information about process dynamics to assist decision-making of operators. Case studies on the Tennessee Eastman benchmark process are conducted to demonstrate the efficacy of the proposed method.

Fault Detection in Managed Pressure Drilling Using Slow Feature Analysis

Correct detection of drilling abnormal incidents while minimizing false alarms is a crucial measure to decrease the non-productive time and, thus, decrease the total drilling cost. With the recent development of drilling technology and innovation of down-hole signal transmitting method, abundant drilling data are collected and stored in the electronic driller’s database. The availability of such data provides new opportunities for rapid and accurate fault detection; however, data-driven fault detection has seen limited practical application in well drilling processes. One particular concern is how to distinguish “controllable” process changes, e.g., due to set-point changes, from truly abnormal events that should be considered as faults. This is highly relevant for the managed pressure drilling technology, where the operating pressure window is often narrow resulting in necessary set-point changes at different depths. However, the classical data-driven fault detection methods, such as principal component analysis and independent component analysis, are unable to distinguish normal set-point changes from abnormal faults. To address this challenge, a slow feature analysis (SFA)-based fault detection method is applied. The SFA-based method furnishes four monitoring charts containing more information that could be synthetically utilized to correctly differentiate set-point changes from faults. Furthermore, the evaluation about controller performance is provided for drilling operator. Simulation studies with a commercial high-fidelity simulator, Drillbench, demonstrate the effectiveness of the introduced approach.

Investigation of Incremental Learning as Temporal Feature Extraction

In this paper we discuss an effect of feature extraction using Incremental Learning Restricted Boltzmann Machine (IL-RBM). We trained the model on Moving MNIST and analyzed the obtained representation by visualizing hidden activities and reported some meaningful features obtained in incremental learning, similar to that of obtained in Slow Feature Analysis (SFA).

Control Performance Monitoring with Temporal Features and Dissimilarity Analysis for Nonstationary Dynamic Processes

Recently, the combination of cointegration analysis (CA) and slow feature analysis (SFA), has been adopted for concurrent monitoring of operation condition and process dynamics for nonstationary dynamic processes subject to time variant conditions. By isolating long-term temporal equilibrium features and specific temporal slow features from steady-state information, the CA-SFA based monitoring scheme can well distinguish between the changes of operation conditions and real faults. Considering that the temporal variation can provide an indication of control performance changes, the CA-SFA algorithm is further exploited based on dissimilarity analysis of temporal distribution to explore its unique efficacy in control performance monitoring (CPM). Two attractive features of the proposed approach are noticed. First, it is compatible with various operation conditions simultaneously including multifarious steady states and dynamic switchings between different working points. Second, a new performance monitoring index is used to monitor the control performance by quantifying the distribution structure of temporal features against the benchmark from both fast and slow dynamics aspects. Case study on a chemical industrial scale multiphase flow experimental rig shows the feasibility of the new CPM method.

Identification of robust probabilistic slow feature regression model for process data contaminated with outliers

Modeling of high dimensional dynamic process is considered as a challenging task. In this regard, probabilistic Slow Feature Analysis (PSFA), a dynamic latent variable model, is proven to be a useful tool which extracts temporally correlated dynamic features from the high-dimensional raw measurements. The extracted latent Slow Features (SFs) can capture process variations which are useful in developing dynamic models. Often times industrial data is affected by outliers, and modeling such data could result in inferior prediction performance. To deal with such scenarios, we propose a robust PSFA (RPSFA) based regression model that models outliers in the observation data using the Student's t-distribution. To estimate the parameters in RPSFA and to extract reduced dimension of SFs, we employ Expectation-Maximization (EM) algorithm under the Maximum Likelihood Estimation (MLE) framework considering SFs as hidden variables. To estimate the hidden SFs we propose a weighted gain Kalman filter based approach as the Normal distribution assumption of the observations is no longer valid. The validity and merits of the proposed approach are demonstrated though a simulated example, an industrial application and an experimental study.

A full‐condition monitoring method for nonstationary dynamic chemical processes with cointegration and slow feature analysis

Chemical processes are in general subject to time variant conditions because of load changes, product grade transitions, or other causes, resulting in typical nonstationary dynamic characteristic. It is of a considerable challenge for process monitoring to consider all possible operation conditions simultaneously including multifarious steady states and dynamic switchings. In this work, a novel full‐condition monitoring strategy is proposed based on both cointegration analysis (CA) and slow feature analysis (SFA) with the following considerations: (1) Despite that the operation conditions may vary over time, they may follow certain equilibrium relations that extend beyond the current time, and (2) there may exist certain dynamic relations that stay invariant under normal process operation despite process may operate at different operating conditions. To monitor both equilibrium and dynamic relations, in the proposed method, nonstationary variables are separated from stationary variables first. Then by CA and SFA, the long‐term equilibrium relation is distinguished from the specific relation held by the current conditions from both static and dynamic aspects. Various monitoring statistics are designed with clear physical interpretation. It can distinguish between the changes of operation conditions and real faults by checking deviations from equilibrium relation and deviations from the specific relation. Case study on a chemical industrial scale multiphase flow experimental rig shows the validity of the proposed full‐condition monitoring method. © 2017 American Institute of Chemical Engineers AIChE J, 64: 1662–1681, 2018

A dual fast and slow feature interaction in biologically inspired visual recognition of human action

Computational neuroscience studies have examined the human visual system through functional magnetic resonance imaging (fMRI) and identified a model where the mammalian brain pursues two independent pathways for recognizing biological movement tasks. On the one hand, the dorsal stream analyzes the motion information by applying optical flow, which considers the fast features. On the other hand, the ventral stream analyzes the form information with slow features. The proposed approach suggests that the motion perception of the human visual system comprises fast and slow feature interactions to identify biological movements. The form features in the visual system follow the application of the active basis model (ABM) with incremental slow feature analysis (IncSFA). Episodic observation is required to extract the slowest features, whereas the fast features update the processing of motion information in every frame. Applying IncSFA provides an opportunity to abstract human actions and use action prototypes. However, the fast features are obtained from the optical flow division, which gives an opportunity to interact with the system as the final recognition is performed through a combination of the optical flow and ABM-IncSFA information and through the application of kernel extreme learning machine. Applying IncSFA into the ventral stream and involving slow and fast features in the recognition mechanism are the major contributions of this research. The two human action datasets for benchmarking (KTH and Weizmann) and the results highlight the promising performance of this approach in model modification.

Multiple person tracking based on slow feature analysis

Object tracking is one of the most important components in numerous applications of computer vision. However, it still has many challenges to be solved, such as occlusion, matching, data association, etc. In this paper, we proposed to utilize slow feature analysis (SFA) method to handle the multiple person tracking problem. First, the part-based model is utilized to detect pedestrian in each frame. Then, a set of reliable tracklets is generated by utilizing spatial-temporal information of detection results. Third, SFA method is leveraged to extract slow-feature for these reliable tracklets. Finally, the traditional graph matching method is utilized to handle data association problem and consequently generate the final trajectory for individual tracking object. Some popular datasets are used in this study. The extensive comparison experiments demonstrate the superiority of the proposed method.

Belief Propagation for Probabilistic Slow Feature Analysis

Slow feature analysis (SFA) is a time-series analysis method for extracting slowly-varying latent features from multi-dimensional data. A recent study proposed a probabilistic framework of SFA using the Bayesian statistical framework. However, the conventional probabilistic framework of SFA can not accurately extract the slow feature in noisy environments since its marginal likelihood function was approximately derived under the assumption that there exists no observation noise. In this paper, we propose a probabilistic framework of SFA with rigorously derived marginal likelihood function. Here, we rigorously derive the marginal likelihood function of the probabilistic framework of SFA by using belief propagation. We show using numerical data that the proposed probabilistic framework of SFA can accurately extract the slow feature and underlying parameters for the latent dynamics simultaneously even under noisy environments.

Slow feature analysis based on online feature reordering and feature selection for dynamic chemical process monitoring

This study considers the insufficiency of traditional monitoring methods to eliminate dynamics, and proposes a novel online feature reordering- and feature selection-based slow feature analysis (SFA) algorithm. The SFA algorithm explores the process dynamics from the view of inner variation of data to extract the slowly varying features. The extracted SFs are considered as the representations of steady- and dynamic-state processes. Online feature reordering and feature selection strategies maximize online fault information and can be used to perform fault detection operation. The proposed method is applied to two simulated processes. Monitoring results show that the proposed method has better monitoring results than those of traditional methods.

A post-classification change detection method based on iterative slow feature analysis and Bayesian soft fusion

Post-classification with multi-temporal remote sensing images is one of the most popular change detection methods, providing the detailed “from-to” change information in real applications. However, due to the fact that it neglects the temporal correlation between corresponding pixels in multi-temporal images, the post-classification approach usually suffers from an accumulation of misclassification errors. In order to solve this problem, previous studies have separated the change and non-change candidates with change vector analysis, and they have only updated the classes of the changed pixels with the post-classification; however, this approach with thresholding loses the continuous change intensity information, where larger values indicate higher probability to be changed. Therefore, in this paper, a new post-classification method with iterative slow feature analysis (ISFA) and Bayesian soft fusion is proposed to obtain reliable and accurate change detection maps. The proposed method consists of three main steps: 1) independent classification is implemented to obtain the class probability for each image; 2) the ISFA algorithm is used to obtain the continuous change probability map of multi-temporal images, where the value of each pixel indicates the probability to be changed; and 3) based on Bayesian theory, the a posteriori probabilities for the class combinations of coupled pixels are calculated to integrate the class probability with the change probability, which is named as Bayesian soft fusion. The class combination with the maximum a posteriori probability is then determined as the change detection result. In addition, a class probability filter is proposed to avoid the false alarms caused by the spectral variation within the same class. Two experiments with multi-temporal Landsat Thematic Mapper (TM) images indicated that the proposed method achieves a clearly higher change detection accuracy than the current state-of-the-art methods. The proposed method based on Bayesian theory and ISFA was also verified to have the ability to improve the change detection rate and reduce the false alarms at the same time. Given its effectiveness and flexibility, the proposed method could be widely applied in land-use/land-cover change detection and monitoring at a large scale.

Causality of the Drought in the Southwestern United States Based on Observations

Slow feature analysis is used to extract driving forces from the monthly mean anomaly time series of the precipitation in the southwestern United States (1895–2015). Four major spectral scales pass the 95% confidence test after wavelet analysis of the derived driving forces. Further harmonic analysis indicates that only two fundamental frequencies are dominant in the spectral domain. The frequencies represent the influence of the Pacific decadal oscillation (PDO) and solar activity on the precipitation from the southwestern United States. In addition, solar activity has exerted a greater effect than the PDO on the precipitation in the southwestern United States over the past 120 years. By comparing the trend of droughts with the two fundamental frequencies, it is found that both the droughts in the 1900s and in the twenty-first century were affected by the PDO and solar activity, whereas the droughts from the 1950s to the 1970s were mainly affected by solar activity.

State Identification of Smart Devices Using Acoustic Fingerprinting

The widespread use of smart phones has brought the security and privacy problems. In the past research, fingerprinting of physical device can be used in authenticating system. However, using the fingerprinting to identify and track the user is useful in many legitimate scenes. In this paper, we propose a novel acoustic fingerprinting approach that uses the microphones and speakers of devices to assess the state of target device. While a person is walking or running with a smart phone, a certain amount of vibration is generated, which can be investigated as the basis of state identification. For mining the subtle vibration variations, firstly, a special high-frequency control simulation to speakers is introduced. Then an unsupervised learning method called Incremental Slow Feature Analysis (IncSFA) is applied to construct the state fingerprinting to represent the slow variation triggered by external driving. At last, two alternate approaches as support vector machine (SVM) and Bayes classifier are used to classify the different states from different smart phones.

Graph-based predictable feature analysis

We propose graph-based predictable feature analysis (GPFA), a new method for unsupervised learning of predictable features from high-dimensional time series, where high predictability is understood very generically as low variance in the distribution of the next data point given the previous ones. We show how this measure of predictability can be understood in terms of graph embedding as well as how it relates to the information-theoretic measure of predictive information in special cases. We confirm the effectiveness of GPFA on different datasets, comparing it to three existing algorithms with similar objectives---namely slow feature analysis, forecastable component analysis, and predictable feature analysis---to which GPFA shows very competitive results.

Multi-view feature extraction based on slow feature analysis

In this paper, we proposed to apply IncSFA to represent the feature of 3D model and employed graph matching to handle similarity measure problem between two different 3D model. First, we built the input data in order to guarantee it suitable for SFA mode according to structure information of 3D model. Second, SFA method utilizes iterations learning method to extract slow feature for each 2D views recorded from 3D model. Finally, weighted bipartite graph matching is leveraged to compute the similarity between query model and candidate model. Extensive comparison experiments were on the popular ETH dataset. The results demonstrate the superiority of the proposed method.

Identification of the driving forces of climate change using the longest instrumental temperature record

The identification of causal effects is a fundamental problem in climate change research. Here, a new perspective on climate change causality is presented using the central England temperature (CET) dataset, the longest instrumental temperature record, and a combination of slow feature analysis and wavelet analysis. The driving forces of climate change were investigated and the results showed two independent degrees of freedom —a 3.36-year cycle and a 22.6-year cycle, which seem to be connected to the El Niño–Southern Oscillation cycle and the Hale sunspot cycle, respectively. Moreover, these driving forces were modulated in amplitude by signals with millennial timescales.

Kernel Slow Feature Analysis for Scene Change Detection

Scene change detection between multitemporal image scenes can be used to interpret the variation of regional land use, and has significant potential in the application of urban development monitoring at the semantic level. The traditional methods directly comparing the independent semantic classes neglect the temporal correlation, and thus suffer from accumulated classification errors. In this paper, we propose a novel scene change detection method via kernel slow feature analysis (KSFA) and postclassification fusion, which integrates independent scene classification with scene change detection to accurately determine scene changes and identify the “from-to” transition type. After representation with the bag-of-visual-words model, KSFA is proposed to extract the nonlinear temporally invariant features, to better measure the change probability between corresponding multitemporal image scenes. Two postclassification fusion methods, which are based on Bayesian theory and predefined rules, respectively, are then employed to identify the optimal coupled class combinations of multitemporal scene pairs. Furthermore, in addition to identifying semantic changes, the proposed method can also improve the performance of scene classification, since the unchanged scenes are more likely to belong to the same class. Two experiments with high-resolution remote sensing image scene data sets confirm that the proposed method can increase the accuracy of scene change detection, scene transition identification, and scene classification.

Slow Feature Analysis for Mitotic Event Recognition

Slow Feature Analysis for Mitotic Event Recognition

Extracting the driving force signal from hierarchy system based on slow feature analysis

Extracting the signals from non-stationary time series is a difficult task in many fields such as physics, economics, and atmospheric sciences. The theory of hierarchy suggests that varying driving force leads to the non-stationary behavior, so extracting and analyzing the slowly varying features can help to study non-stationary dynamical system, which has become a compelling question recently. Slow feature analysis (SFA) is an effective technique for extracting slowly varying driving forces from quickly varying non-stationary time series. The basic idea of SFA is to nonlinearly extend the reconstructive signal into a combination form with one or higher order polynomials, and to apply the principal component analysis to this extended signal and its time derivatives. The algorithm is guaranteed to seek an optimal solution from a group of functions directly and can extract a lot of uncorrelated features that are ordered by slowness. A series of studies has shown its superiority in extracting the driving force of non-stationary time series. The extracted signal is found to be highly correlated with the real driving force. Results based on ideal models show that either the slow driving force itself or a slower subcomponent can be detected by SFA. Yet despite all that, the further investigating of SFA is still needed to reduce its uncertainty. In this study, we create two types of non-stationary models by the logistic map with time-varying parameters: one includes two varying driving forces with different time periods constraining the evolution of time series in a non-stationary way; and the other is a three-layer structure encompassing two superimposed signals in which the slower signal of driving force is modulated by the lowest one. According to the ideal model and SFA, we conduct the numerical experiments to develop corresponding analysis method and discuss its application prospect in extracting driving force signals. We find that for the system of first kind, either the slowest signal or the combination of two driving forces constructed by SFA contains some uncertain information. However, we can detect the two independent driving forces from the constructed signal by wavelet analysis. For the three-hierarchy system that includes two superimposed signals of driving force, successive applications through SFA on the original time series and the constructed SFA signal will in turn detect the slower varying driving force signal and the slowest varying driving forces signal. The successful application of SFA shows its promising prospect in analyzing the external driving forces in non-stationary system and understanding relevant dynamic mechanism.

Batch Process Monitoring Based on Multiway Global Preserving Kernel Slow Feature Analysis

As an effective nonlinear dynamic data analysis tool, kernel slow feature analysis (KSFA) has achieved great success in continuous process monitoring field during recent years. However, its application to batch process monitoring is unexploited, which is a more challenging task because of the complicated characteristics of batch process data. In this paper, we propose a novel batch process monitoring method based on the modified KSFA method, referred to as multiway global preserving kernel slow feature analysis (MGKSFA), to capture high nonlinearity and inherently time-varying dynamics of process data. In the proposed method, a two-step multiway data unfolding strategy is first utilized to convert the three-way batch process training data set into a two-way matrix. Then, the global structure preserving-based kernel slow feature analysis (GKSFA) is used to build the nonlinear statistical monitoring model, which not only explores the local dynamic data relationships but also considers the mining of global data structure information. Furthermore, a rule based on the cumulative slowness contribution is designed to determine the number of the retained slow features. Last, two monitoring statistics T <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> and SPE are built to detect the process faults. Two case studies, including one simple numerical nonlinear system and the benchmark fed-batch penicillin fermentation process, are used to demonstrate that the proposed MGKSFA method has the superior fault detection performance over the traditional batch process monitoring methods.