Probabilistic Finite State Automata Research Articles

State Identification Via Symbolic Time Series Analysis for Reinforcement Learning Control

Abstract This technical brief makes use of the concept of symbolic time-series analysis (STSA) for identifying discrete states from the nonlinear time response of a chaotic dynamical system for model-free reinforcement learning (RL) control. Along this line, a projection-based method is adopted to construct probabilistic finite state automata (PFSA) for identification of the current state (i.e., operational regime) of the Lorenz system; and a simple Q-map-based (and model-free) RL control strategy is formulated to reach the target state from the (identified) current state. A synergistic combination of PFSA-based state identification and RL control is demonstrated by the simulation of a numeric model of the Lorenz system, which yields very satisfactory performance to reach the target states from the current states in real-time.

Data Fusion and Pattern Classification in Dynamical Systems Via Symbolic Time Series Analysis

Abstract Symbolic time series analysis (STSA) plays an important role in the investigation of continuously evolving dynamical systems, where the capability to interpret the joint effects of multiple sensor signals is essential for adequate representation of the embedded knowledge. This technical brief develops and validates, by simulation, an STSA-based algorithm to make timely decisions on dynamical systems for information fusion and pattern classification from ensembles of multisensor time series data. In this context, one of the most commonly used methods has been neural networks (NN) in their various configurations; however, these NN-based methods may require large-volume data and prolonged computational time for training. An alternative feasible method is the STSA-based probabilistic finite state automata (PFSA), which has been shown in recent literature to require significantly less training data and to be much faster than NN for training and, to some extent, for testing. This technical brief reports a modification of the current PFSA methods to accommodate (possibly heterogeneous and not necessarily tightly synchronized) multisensor data fusion and (supervised learning-based) pattern classification in real-time. Efficacy of the proposed method is demonstrated by fusion of time series of position and velocity sensor data, generated from a simulation model of the forced Duffing equation.

Learning Phonotactics in a Differentiable Framework of Subregular Languages

Phonotactic constraints have been argued to beregular, meaning that they can be represented usingfinite-state automata (Heinz, 2018); furthermore, they have been argued to occupy a even more restrictedregion of the regular language class known as the subregular hierarchy (Rogers & Pullum, 2011). Ourcontribution is to present a simple model of phonotactic learning from positive evidence. Our approach isbased on probabilistic finite-state automata (Vidal et al., 2005a,b). We study the model’s ability to induce localand nonlocal phonotactics from wordlist data, both with and without formal constraints on the automaton.In particular, we evaluate the ability of our learner to induce nonlocal phonotactic constraints from data ofNavajo and Quechua. Our work provides a framework in which different formal models of phonotactics canbe compared, and sheds light on the structural nature of phonological acquisition (Dai, 2021; Shibata & Heinz,2019; Heinz & Rogers, 2010, 2013).

Predicting atrial fibrillation episodes with rapid ventricular rates associated with low levels of activity

BackgroundRapid and irregular ventricular rates (RVR) are an important consequence of atrial fibrillation (AF). Raw accelerometry data in combination with electrocardiogram (ECG) data have the potential to distinguish inappropriate from appropriate tachycardia in AF. This can allow for the development of a just-in-time intervention for clinical treatments of AF events. The objective of this study is to develop a machine learning algorithm that can distinguish episodes of AF with RVR that are associated with low levels of activity.MethodsThis study involves 45 patients with persistent or paroxysmal AF. The ECG and accelerometer data were recorded continuously for up to 3 weeks. The prediction of AF episodes with RVR and low activity was achieved using a deterministic probabilistic finite-state automata (DPFA)-based approach. Rapid and irregular ventricular rate (RVR) is defined as having heart rates (HR) greater than 110 beats per minute (BPM) and high activity is defined as greater than 0.75 quantile of the activity level. The AF events were annotated using the FDA-cleared BeatLogic algorithm. Various time intervals prior to the events were used to determine the longest prediction intervals for predicting AF with RVR episodes associated with low levels of activity.ResultsAmong the 961 annotated AF events, 292 met the criterion for RVR episode. There were 176 and 116 episodes with low and high activity levels respectively. Out of the 961 AF episodes, 770 (80.1%) were used in the training data set and the remaining 191 intervals were held out for testing. The model was able to predict AF with RVR and low activity up to 4.5 min before the events. The mean prediction performance gradually decreased as the time to events increased. The overall Area under the ROC Curve (AUC) for the model lies within the range of 0.67–0.78.ConclusionThe DPFA algorithm can predict AF with RVR associated with low levels of activity up to 4.5 min before the onset of the event. This would enable the development of just-in-time interventions that could reduce the morbidity and mortality associated with AF and other similar arrhythmias.

Thresholdless Classification of chaotic dynamics and combustion instability via probabilistic finite state automata

The objective of the work reported in this paper is to make decisions on the current state of a dynamical system for pattern classification and anomaly/fault detection, which is often achieved by time series analyses of pertinent measured signals. In this context, one of the most commonly used methods is hidden Markov model (HMM), while yet another popular method is neural networks (NN) in their various configurations; however, both of these methods may require large training data and computational time. An alternative feasible method is probabilistic finite state automata (PFSA), which is much faster for training and also for testing. In its current state-of-the-art, the standard PFSA, called s-PFSA, has certain shortcomings that this paper attempts to remedy. Therefore, s-PFSA is modified into the proposed projection-based PFSA, abbreviated as p-PFSA, to yield better classification accuracy and robustness. Efficacy of p-PFSA is first demonstrated on four different models of chaotic dynamical systems by comparison with s-PFSA, HMM and NN, which are used to serve as baseline methods for validation of classification performance; the NN models consist of two vanilla NNs and another NN with long short term memory (LSTM). Then, these results of comparison are extended to assess the relative performance of p-PFSA for a real-life application in terms of accuracy, robustness, and computational complexity on a laboratory-scale apparatus that emulates the essential characteristics of industrial-scale combustion systems.

Measure invariance of ergodic symbolic systems for low-delay detection of anomalous events

This paper investigates the spectral properties of ergodic measure of symbolic systems for applications to signal processing, pattern recognition, and anomaly detection in uncertain dynamical systems. The underlying algorithm is built upon the concept of ergodic sequences of measure-preserving transformations (MPT) on probability spaces, where non-stationary probabilistic finite state automata (PFSA) are constructed using short-length windows of time-series measurements. The resulting PFSA are non-homogeneous Markov, in which spectral properties of the MPT depend on several parameters that include the window length. The paper also develops an MPT-based metric to quantify the divergence of the evolving PFSA from that of the nominal PFSA; this information is then used for pattern classification and anomaly detection with low-delay tolerance. The MPT-based methodology has been validated with experimental data generated from a laboratory apparatus that deals with detection of thermoacoustic instabilities in combustion processes, which are known to have chaotic characteristics on a time scale of milliseconds. In this application, the concepts of MPT and ergodicity have been used to develop a novel symbolic time series analysis (STSA)-based detection method, whose performance is validated by comparison with two well-known techniques, namely, hidden Markov model (HMM) and cumulative sum (CUSUM), on the same experimental data. The results consistently show superior performance of the proposed MPT-based STSA.

Early Detection of Fatigue Crack Damage in Ductile Materials: A Projection-Based Probabilistic Finite State Automata Approach

Abstract Fatigue failure occurs ubiquitously in mechanical structures when they are subjected to cyclic loading well below the material’s yield stress. The tell-tale sign of a fatigue failure is the emergence of cracks at the internal or surface defects. In general, a machinery component has a finite fatigue life based on the number of cycles, it can sustain before a fracture occurs. However, the estimated life is generally conservative and often a large factor of safety is applied to make the component fail-safe. From the perspective of better utilization of a machinery component, it is, however, desirable to have maximum usage of the component without a catastrophic failure. It is, therefore, conducive to have a measure that can capture precursors to failure to facilitate active diagnosis of the machinery health. In this study, a precursor detection method is developed upon modifications of probabilistic finite state automata (PFSA). The efficacy of the proposed method is demonstrated on cold-rolled AL7075-T6 notched specimens in a computer-instrumented and computer-controlled fatigue testing apparatus. The results show that the proposed method is capable of detecting the emergence of cracks (at ∼95% accuracy) and also can capture precursors with good fidelity.

Identification of Long-Term Behavior of Natural Circulation Loops: A Thresholdless Approach from an Initial Response

Natural circulation loop (NCL) systems are buoyancy-driven heat exchangers that are used in various industrial applications. The concept of passive heat exchange in NCL systems is attractive, because there is no need for an externally driven equipment (e.g., a pump) to maintain the fluid circulation. However, relying on buoyancy as the sole driving force may lead to several potential difficulties, one of which is generation of (possibly) time-varying nonlinearities in the dynamical system, where a difference in the time scales of heat transfer and fluid flow causes the flow to change from a steady-state regime to either an oscillatory regime or a flow-reversal regime, both of which are undesirable. In this paper, an algorithm is developed using tools of symbolic time-series analysis (e.g., probabilistic finite state automata (PFSA)) for the purpose of identifying selected regimes of operation in NCL systems using only data from the early transient operation, where the underlying principle is built upon the concept of pattern classification from measurements of fluid-flow dynamics. The proposed method is shown to be capable of identifying the current regime of operation from the initial time response under a given set of operational parameters. The efficacy of regime classification is demonstrated by testing on two datasets, generated from numerical simulation of a MATLAB SimuLink model that has previously been validated with experimental data. The results of the proposed PFSA-based classification are compared with those of a hidden Markov model (HMM) that serves as the baseline.

Markov modeling of dynamical systems via clustering and graph minimization

Discrete dynamical systems are widely used in a variety of scientific and engineering applications. Modeling these systems involves performing statistical analysis of the system output to estimate the parameters of a model so it can behave statistically similar to the original system. These models can be used for simulation, performance analysis, fault detection, among other applications. This work presents a new algorithm to model discrete dynamical systems using probabilistic finite state automata (PFSA) by initially finding a special class of PFSA called D-Markov machines (a D-ary Markov process) and then applying machine learning algorithms and graph minimization techniques to obtain compact and precise PFSA models. Modeling examples are provided to discuss the trade-off between accuracy and the number of PFSA states. Results using simulated datasets show that the proposed FPSA construction achieves good sensitivity and precision for anomaly detection.

Human Activity Discovery and Recognition Using Probabilistic Finite-State Automata

Ambient assisted living and smart home technologies are a good way to take care of dependent people whose number will increase in the future. They allow the discovery and the recognition of human’s activities of daily living (ADLs) in order to take care of people by keeping them in their home. In order to consider the human behavior nondeterminism, probabilistic approaches are used despite difficulties encountered in model generation and probabilistic indicators computing. In this article, a global method based on probabilistic finite-state automata and the definition of the normalized likelihood and perplexity is proposed to manage ADLs discovery and recognition. In order to reduce the computational complexity, some results about a simplified normalized likelihood computation are proved. A real case study showing the efficiency of the proposed method is discussed. Note to Practitioners —This article is motivated by the problem of the automatic recognition of activities that are daily performed by elderly or disabled people in a smart dwelling. The set of activities to be recognized is defined by a medical staff (e.g., to prepare meal, to do housework, to take leisure, etc.) and correspond to pathologies that have to be monitored by doctors (e.g., loss of memory, loss of mobility, etc.). The proposed method is based on a systematic procedure of offline construction of a model for each activity to be monitored (the activity discovering step). The online recognition of activities actually performed (the activity recognition step) is afterward based on these models of activities. Since the human behavior is nondeterministic, and may even be irrational, probabilistic activity models are built from a learning database. In the same way, probabilistic indicators are used for determining online the most probable activities actually performed. The efficiency of the proposed approach is illustrated through a case study performed in a smart living lab.

Computation of moments for probabilistic finite-state automata

The computation of moments of probabilistic finite-state automata (PFA) is researched in this article. First, the computation of moments of the length of the paths is introduced for general PFA, and then, the computation of moments of the number of times that a symbol appears in the strings generated by the PFA is described. These computations require a matrix inversion. Acyclic PFA, such as word graphs, are quite common in many practical applications. Algorithms for the efficient computation of the moments for acyclic PFA are also presented in this paper.

Sensor fault detection in nuclear power plants using symbolic dynamic filter

This paper addresses an approach for small/minor fault detection of thermocouple sensor in a nuclear power plant. The small/minor fault means the faulty signal does not cross the system threshold values, i.e., the deviation from the normal signal is very less. The faults are detected by extracting the features of sensor signals using symbolic dynamic filter (SDF). A transformation function is proposed that maps the multidimensional heterogeneous sensor signals or trajectories into a homogeneous and approximately unique spectrum space. The transformed trajectories are partitioned by the partition functions and symbol sequences are generated. The probabilistic finite state automata (PFSA) is framed for each trajectory and features of the signals are compactly represented by the probability transition matrix. The faults are detected by analyse the transition matrix. The proposed method is demonstrated by real time data of the fast breeder test reactor (FBTR) with two different case studies.

Nonlinear Dynamics Tools for Offline Signature Verification Using One-class Gaussian Process

One of the major problems in biometrics and in document forensics is the offline mode of signature verification. This study aims to present a novel approach of verifying an individual’s signature through offline images of handwriting. The approach proposed here relies on a global method which considers signature images as waveforms. First, image decompositions are in terms of a series of wavelet sub-bands at some specific levels. Wavelet sub-bands are then extended so as to obtain waveforms. Each waveform is quantized by two Nonlinear Dynamics Tools in order to generate feature vectors. Multi-Resolution Box-Counting (MRBC) fractal dimension algorithm as well as probabilistic finite state automata (PFSA) are applied separately to signature waveforms. In the training and verification phase, we propose the one-class Gaussian process (GP) priors based on writer-independent approach. As one of the main parameters, optimal decision threshold is selected from False Accept Rate (FAR) and False Reject Rate (FRR) curves. The presented system was tested on two Persian databases (PHBC and UTSig) as well as on two Latin databases (MCYT-75 and CEDAR). In fact, the results produced by this method were generally better in terms of the four signature databases than the state-of-the-art results.

Automatic Extraction of Behavioral Patterns for Elderly Mobility and Daily Routine Analysis

The elderly living in smart homes can have their daily movement recorded and analyzed. As different elders can have their own living habits, a methodology that can automatically identify their daily activities and discover their daily routines will be useful for better elderly care and support. In this article, we focus on automatic detection of behavioral patterns from the trajectory data of an individual for activity identification as well as daily routine discovery. The underlying challenges lie in the need to consider longer-range dependency of the sensor triggering events and spatiotemporal variations of the behavioral patterns exhibited by humans. We propose to represent the trajectory data using a behavior-aware flow graph that is a probabilistic finite state automaton with its nodes and edges attributed with some local behavior-aware features. We identify the underlying subflows as the behavioral patterns using the kernel k -means algorithm. Given the identified activities, we propose a novel nominal matrix factorization method under a Bayesian framework with Lasso to extract highly interpretable daily routines. For empirical evaluation, the proposed methodology has been compared with a number of existing methods based on both synthetic and publicly available real smart home datasets with promising results obtained. We also discuss how the proposed unsupervised methodology can be used to support exploratory behavior analysis for elderly care.

Symbolic analysis-based reduced order Markov modeling of time series data

This paper presents reduced-order modeling of time-series data for a special class of Markov models using symbolic dynamics. These models are constructed from the time-series signal by partitioning the data and then inferring a probabilistic finite state automaton (PFSA) from the resulting symbol sequence, capturing a finite history (or memory) of symbol strings. In the proposed approach, the size of the temporal memory of a symbol sequence is estimated from spectral properties of the resulting stochastic matrix corresponding to a first-order Markov model of the symbol sequence. Then, agglomerative hierarchical clustering is used to cluster states of the corresponding full-order Markov model to construct a reduced-order Markov model based on information-theoretic criteria with a non-deterministic algebraic structure; the parameters of the reduced-order model are identified from the original model by making use of a Bayesian inference rule. The model size is inferred using an information-theoretic inspired criteria; the Markov parameters of the reduced-order model are identified from the original model by making use of a Bayesian inference rule. The paper also identifies theoretical bounds on the error induced in the reduced-size model in terms of expected Hamming distance between the sequences generated by the original and final reduced-size models. The proposed concept is elucidated and validated by two examples on different data sets. The first example analyzes a set of time series of pressure oscillations in a swirl-stabilized combustor, where controlled protocols are used to induce flame instabilities. Variations in the complexity of the derived Markov model represent how the system operating condition changes from stable to an unstable combustion regime. The second example is built upon a public data set of NASA’s repository for prognosis of rolling-element bearings. It is shown that: (i) even with a small state-space, the reduced-order models are able to achieve comparable performance, and (ii) the proposed approach provides flexibility in the selection of a reduced-order model for data representation and learning.

Inferring Extended Probabilistic Finite-State Automaton Models from Software Executions

Behavioral models are useful tools in understanding how programs work. Although several inference approaches have been introduced to generate extended finite-state automatons from software execution traces, they suffer from accuracy, flexibility, and decidability issues. In this article, we apply a hybrid technique to use both reinforcement learning and stochastic modeling to generate an extended probabilistic finite state automaton from software traces. Our approach—ReHMM (Reinforcement learning-based Hidden Markov Modelling)—is able to address the problems of inflexibility and un-decidability reported in other state-of-the-art approaches. Experimental results indicate that ReHMM outperforms other inference algorithms.

A Novel Approach in Adopting Finite State Automata for Image Processing Applications

This article describes how robust image processing application rely heavily on image descriptors extracted. Limited work is carried out in adopting probabilistic finite state automata (PFSA) models for image processing. A finite state automata for image processing (FSAFIP) method is presented here. Texture classification and content based image retrieval (CBIR) is considered. In FSAFIP, foreground and background regions of an image are identified and later split into patches. Using a tristate PFSA model, feature descriptors corresponding to background/foreground regions are constructed. A distance based large margin nearest neighbor (LMNN) classifier is considered in FSAFIP to impart intelligence. A performance and experimental study to evaluate performance of FSAFIP for CBIR and texture classification is presented. Comparison results in CBIR obtained prove superior performance of FSAFIP over existing methods on Corel-1K dataset. High texture classification accuracy of 99.2% is reported using FSAFIP on KHT-TIPS dataset. An improved texture classification accuracy is achieved using FSAFIP in comparison to former methods.

On the Derivational Entropy of Left-to-Right Probabilistic Finite-State Automata and Hidden Markov Models

Probabilistic finite-state automata are a formalism that is widely used in many problems of automatic speech recognition and natural language processing. Probabilistic finite-state automata are closely related to other finite-state models as weighted finite-state automata, word lattices, and hidden Markov models. Therefore, they share many similar properties and problems. Entropy measures of finite-state models have been investigated in the past in order to study the information capacity of these models. The derivational entropy quantifies the uncertainty that the model has about the probability distribution it represents. The derivational entropy in a finite-state automaton is computed from the probability that is accumulated in all of its individual state sequences. The computation of the entropy from a weighted finite-state automaton requires a normalized model. This article studies an efficient computation of the derivational entropy of left-to-right probabilistic finite-state automata, and it introduces an efficient algorithm for normalizing weighted finite-state automata. The efficient computation of the derivational entropy is also extended to continuous hidden Markov models.

Detection of Thermoacoustic Instabilities Via Nonparametric Bayesian Markov Modeling of Time-Series Data

Real-time detection and decision and control of thermoacoustic instabilities in confined combustors are challenging tasks due to the fast dynamics of the underlying physical process. The objective here is to develop a dynamic data-driven algorithm for detecting the onset of instabilities with short-length time-series data, acquired by available sensors (e.g., pressure and chemiluminescence), which will provide sufficient lead time for active decision and control. To this end, this paper proposes a Bayesian nonparametric method of Markov modeling for real-time detection of thermoacoustic instabilities in gas turbine engines; the underlying algorithms are formulated in the symbolic domain and the resulting patterns are constructed from symbolized pressure measurements as probabilistic finite state automata (PFSA). These PFSA models are built upon the framework of a (low-order) finite-memory Markov model, called the D-Markov machine, where a Bayesian nonparametric structure is adopted for: (i) automated selection of parameters in D-Markov machines and (ii) online sequential testing to provide dynamic data-driven and coherent statistical analyses of combustion instability phenomena without solely relying on computationally intensive (physics-based) models of combustion dynamics. The proposed method has been validated on an ensemble of pressure time series from a laboratory-scale combustion apparatus. The results of instability prediction have been compared with those of other existing techniques.

POSE: Prediction-Based Opportunistic Sensing for Energy Efficiency in Sensor Networks Using Distributed Supervisors.

This paper presents a distributed supervisory control algorithm that enables opportunistic sensing for energy-efficient target tracking in a sensor network. The algorithm called Prediction-based Opportunistic Sensing (POSE), is a distributed node-level energy management approach for minimizing energy usage. Distributed sensor nodes in the POSE network self-adapt to target trajectories by enabling high power consuming devices when they predict that a target is arriving in their coverage area, while enabling low power consuming devices when the target is absent. Each node has a Probabilistic Finite State Automaton which acts as a supervisor to dynamically control its various sensing and communication devices based on target's predicted position. The POSE algorithm is validated by extensive Monte Carlo simulations and compared with random scheduling schemes. The results show that the POSE algorithm provides significant energy savings while also improving track estimation via fusion-driven state initialization.