Variable Length Markov Models Research Articles

Hybrid machine learning for human action recognition and prediction in assembly

• Bi-stream convolutional neural network is developed for human action recognition. • Object information and transition class significantly improve recognition accuracy. • Variable-length Markov model captures causal dependency embedded in action sequences. • High action prediction accuracy is achieved with human-like prediction logic. As one of the critical elements for smart manufacturing, human-robot collaboration (HRC), which refers to goal-oriented joint activities of humans and collaborative robots in a shared workspace, has gained increasing attention in recent years. HRC is envisioned to break the traditional barrier that separates human workers from robots and greatly improve operational flexibility and productivity. To realize HRC, a robot needs to recognize and predict human actions in order to provide assistance in a safe and collaborative manner. This paper presents a hybrid approach to context-aware human action recognition and prediction, based on the integration of a convolutional neural network (CNN) and variable-length Markov modeling (VMM). Specifically, a bi-stream CNN structure parses human and object information embedded in video images as the spatial context for action recognition and collaboration context identification. The dependencies embedded in the action sequences are subsequently analyzed by a VMM, which adaptively determines the optimal number of current and past actions that need to be considered in order to maximize the probability of accurate future action prediction. The effectiveness of the developed method is evaluated experimentally on a testbed which simulates an assembly environment. High accuracy in both action recognition and prediction is demonstrated.

Blocking NK1 receptors disrupts the sequential and temporal organization of chain grooming in rats

The basal ganglia are a group of sub-cortical structures believed to play a critical role in action selection and sequencing. The striatum is the largest input structure of the basal ganglia and contains the neuropeptide substance P in abundance. Recent computational work has suggested that substance P could play a critical role in action sequence performance and acquisition, but this has not been tested experimentally before. The aim of the present study was to test how blocking substance P's main NK1-type receptors affected the sequential and temporal organization of spontaneous behavioral patterns. We did this in rats by focusing on the grooming chain, an innate and highly stereotyped ordered sequence. We performed an open field experiment in which the NK1 receptor antagonist L-733,060 was injected intraperitoneally in rats at two doses (2 and 4 mg/kg/ml), in a within-subject counterbalanced design. We used first order transition probabilities, Variable Length Markov Models, entropy metrics and T-pattern analysis to evaluate the effects of L-733,060 on sequential and temporal aspects of spontaneously ordered behavioral sequences. Our results suggest that blocking NK1 receptors made the transitions between the grooming chain elements significantly more variable, the transition structure of the grooming bouts simpler, and it increased the probability of transitioning from active to inactive states. Overall, this suggest that blocking substance P receptors led to a general break down in the fluency of spontaneous behavioral sequences, suggesting that substance P could be playing a key role in the implementation of sequential patterns.

Abnormal Crowd Traffic Detection for Crowdsourced Indoor Positioning in Heterogeneous Communications Networks

WiFi fingerprint-based indoor positioning system emerges to provide fundamental location-related service in heterogeneous communications networks. It relies on crowdsourcing technology in the collection of received signal strength (RSS) to dynamically update fingerprint database. However, this crowdsourced indoor positioning system is vulnerable to the intrusion of dishonest users (i.e., attackers). Attackers may manipulate the number of users who submit RSS fingerprints, and finally mislead the evaluation of crowd traffic evaluation. In this paper, we propose an abnormal crowd traffic detection (ACTD) scheme to identify attackers according to their abnormal RSS sensing behaviors. Specifically, a fog server is explored to serve as the crowdsourcing platform to perform data storage and detection. We first categorize attackers into three different levels according to their real geographical locations and collusion. Then, through the analysis of pseudonym changing behavior in RSS submission, we propose a rarity-based outlier detection to classify attackers of level-1. Furthermore, we propose a variable-length Markov model, i.e., probabilistic suffix tree (PST), to detect the colluded users who are not at the target point of interest (POI). In addition, a metric learning algorithm is developed to detect the collusion of AP organizer based on RSS fingerprint distance difference. The extensive simulation results show that the ACTD scheme can effectively resist attackers with high accuracy and appropriately deal with traffic evaluation from RSS fingerprint information.

KVLMM: A Trajectory Prediction Method Based on a Variable-Order Markov Model With Kernel Smoothing

With the dramatic proliferation of global positioning system (GPS) devices, a rich range of research has been conducted on the analysis of GPS trajectories. Research on trajectory prediction uses historical trajectory data to forecast future positions. The typical method is to use a statistical model based on the Markov chain. However, existing models are inefficient in two aspects. The methods of using lower-order Markov models use only current information and ignore historical information, degrading the prediction accuracy. In contrast, higher-order Markov models can improve the prediction accuracy but incur increased time and space complexity. Here, we propose the kernel variable length Markov model (KVLMM), a variable-order Markov model based on kernel smoothing, which combines sequence analysis with the Markov statistical model. The KVLMM can adaptively train trajectory data and learn rules from the training results. When training a large data sample, the KVLMM can rapidly execute training in linearly complex time and space. Moreover, this model uses kernel smoothing when training fewer data samples. In other words, the KVLMM improves the prediction accuracy and reduces the overhead of the data process. Our experimental results show that KVLMM has a lower algorithm complexity and a higher prediction accuracy.

A linear time algorithm for the [formula omitted]-neighbour Travelling Salesman Problem on a Halin graph and extensions

The Quadratic Travelling Salesman Problem (QTSP) is to find a least cost Hamiltonian cycle in an edge-weighted graph, where costs are defined for all pairs of edges contained in the Hamiltonian cycle. The problem is shown to be strongly NP-hard on a Halin graph. We also consider a variation of the QTSP, called the k-neighbour TSP (TSP(k)). Two edges e and f, e≠f, are k-neighbours on a tour τ if and only if a shortest path (with respect to the number of edges) between e and f along τ and containing both e and f, has exactly k edges, for k≥2. In (TSP(k)), a fixed nonzero cost is considered for a pair of distinct edges in the cost of a tour τ only when the edges are p-neighbours on τ for 2≤p≤k. We give a linear time algorithm to solve TSP(k) on a Halin graph for k=3, extending existing algorithms for the cases k=1,2. Our algorithm can be extended further to solve TSP(k) in polynomial time on a Halin graph with n nodes when k=O(logn). The possibility of extending our results to some fully reducible class of graphs is also discussed. TSP(k) can be used to model the Permuted Variable Length Markov Model in bioinformatics as well as an optimal routing problem for unmanned aerial vehicles (UAVs).

Computational Recognition of RNA Splice Sites by Exact Algorithms for the Quadratic Traveling Salesman Problem

One fundamental problem of bioinformatics is the computational recognition of DNA and RNA binding sites. Given a set of short DNA or RNA sequences of equal length such as transcription factor binding sites or RNA splice sites, the task is to learn a pattern from this set that allows the recognition of similar sites in another set of DNA or RNA sequences. Permuted Markov (PM) models and permuted variable length Markov (PVLM) models are two powerful models for this task, but the problem of finding an optimal PM model or PVLM model is NP-hard. While the problem of finding an optimal PM model or PVLM model of order one is equivalent to the traveling salesman problem (TSP), the problem of finding an optimal PM model or PVLM model of order two is equivalent to the quadratic TSP (QTSP). Several exact algorithms exist for solving the QTSP, but it is unclear if these algorithms are capable of solving QTSP instances resulting from RNA splice sites of at least 150 base pairs in a reasonable time frame. Here, we investigate the performance of three exact algorithms for solving the QTSP for ten datasets of splice acceptor sites and splice donor sites of five different species and find that one of these algorithms is capable of solving QTSP instances of up to 200 base pairs with a running time of less than two days.

Visual Speech Synthesis Using a Variable-Order Switching Shared Gaussian Process Dynamical Model

In this paper, we present a novel approach to speech- driven facial animation using a non-parametric switching state space model based on Gaussian processes. The model is an extension of the shared Gaussian process dynamical model, augmented with switching states. Two talking head corpora are processed by extracting visual and audio data from the sequences followed by a parameterization of both data streams. Phonetic labels are obtained by performing forced phonetic alignment on the audio. The switching states are found using a variable length Markov model trained on the labelled phonetic data. The audio and visual data corresponding to phonemes matching each switching state are extracted and modelled together using a shared Gaussian process dynamical model. We propose a synthesis method that takes into account both previous and future phonetic context, thus accounting for forward and backward coarticulation in speech. Both objective and subjective evaluation results are presented. The quantitative results demonstrate that the proposed method outperforms other state-of-the-art methods in visual speech synthesis and the qualitative results reveal that the synthetic videos are comparable to ground truth in terms of visual perception and intelligibility.

Prediction of moving object trajectory based on probabilistic suffix tree

In the prediction of moving object trajectory, concerning the low accuracy rate of low order Markov model and the expansion of state space in high order model, a dynamic adaptive Probabilistic Suffix Tree( PST) prediction method based on variable length Markov model was proposed. Firstly, moving object's trajectory path was serialized according to the time;then the probability characteristic of sequence context was trained and calculated from the historical trajectory data of moving objects, the probabilistic suffix tree model based path sequence was constructed, combined with the actual trajectory data,thus the future trajectory information could be predicted dynamically and adaptively. The experimental results show that the highest prediction accuracy was obtained in second order model, with the order of the model increasing, the prediction accuracy was maintained at about 82% and better prediction results were achieved. In the meantime, space complexity was decreased exponentially and storage space was reduced greatly. The proposed method made full use of historical data and current trajectory information to predict the future trajectory, and provided a more flexible and efficient location-based services.

Predictive Tabla Modelling Using Variable-length Markov and Hidden Markov Models

Tabla is a sophisticated, centuries-old percussion tradition from North India based on timbral sequences. We model these sequences in a predictive framework with Variable-length Markov Models (VLMMs). Using a database containing nearly 30,000 strokes in 35 compositions, we show that VLMMs have high predictive accuracy, with an average perplexity of 1.80, and median perplexity of 1.19, on a task with 42 distinct symbols. This basic framework is extended by the introduction of several new smoothing techniques that determine how to integrate predictions from the different order models. The model is then extended to include parallel representations of the sequence, a technique known as Multiple Viewpoint modelling. The work is then extended to the problem of recognizing strokes from audio. In this hidden context, the identity of the previous stroke is not revealed at each time step. A Variable-length Hidden Markov Model (VLHMM) is used to determine the next-symbol distribution that is used in computing the perplexity. We detail how the forward probabilities can be efficiently computed for the VLHMM by traversing a prediction suffix tree (PST) that is used to represent sequences. Using a VLHMM with a maximum order of 3, we obtain an average perplexity of 2.31, with a median of 1.16 on a nine-target task. To the best of our knowledge, this is the first use of Variable-length Hidden Markov Models for music modelling or prediction.

A COMPARISON OF SCORING METRICS FOR PREDICTING THE NEXT NAVIGATION STEP WITH MARKOV MODEL-BASED SYSTEMS

The problem of predicting the next request during a user's navigation session has been extensively studied. In this context, higher-order Markov models have been widely used to model navigation sessions and to predict the next navigation step, while prediction accuracy has been mainly evaluated with the hit and miss score. We claim that this score, although useful, is not sufficient for evaluating next link prediction models with the aim of finding a sufficient order of the model, the size of a recommendation set, and assessing the impact of unexpected events on the prediction accuracy. Herein, we make use of a variable length Markov model to compare the usefulness of three alternatives to the hit and miss score: the Mean Absolute Error, the Ignorance Score, and the Brier score. We present an extensive evaluation of the methods on real data sets and a comprehensive comparison of the scoring methods.

Rule generation for categorical time series with Markov assumptions

Several procedures of sequential pattern analysis are designed to detect frequently occurring patterns in a single categorical time series (episode mining). Based on these frequent patterns, rules are generated and evaluated, for example, in terms of their confidence. The confidence value is commonly interpreted as an estimate of a conditional probability, so some kind of stochastic model has to be assumed. The model is identified as a variable length Markov model. With this assumption, the usual confidences are maximum likelihood estimates of the transition probabilities of the Markov model. We discuss possibilities of how to efficiently fit an appropriate model to the data. Based on this model, rules are formulated. It is demonstrated that this new approach generates noticeably less and more reliable rules.

Learning Atomic Human Actions Using Variable-Length Markov Models

Visual analysis of human behavior has generated considerable interest in the field of computer vision because of its wide spectrum of potential applications. Human behavior can be segmented into atomic actions, each of which indicates a basic and complete movement. Learning and recognizing atomic human actions are essential to human behavior analysis. In this paper, we propose a framework for handling this task using variable-length Markov models (VLMMs). The framework is comprised of the following two modules: a posture labeling module and a VLMM atomic action learning and recognition module. First, a posture template selection algorithm, based on a modified shape context matching technique, is developed. The selected posture templates form a codebook that is used to convert input posture sequences into discrete symbol sequences for subsequent processing. Then, the VLMM technique is applied to learn the training symbol sequences of atomic actions. Finally, the constructed VLMMs are transformed into hidden Markov models (HMMs) for recognizing input atomic actions. This approach combines the advantages of the excellent learning function of a VLMM and the fault-tolerant recognition ability of an HMM. Experiments on realistic data demonstrate the efficacy of the proposed system.

Basing population genetic inferences and models of molecular evolution upon desired stationary distributions of DNA or protein sequences

Models of molecular evolution tend to be overly simplistic caricatures of biology that are prone to assigning high probabilities to biologically implausible DNA or protein sequences. Here, we explore how to construct time-reversible evolutionary models that yield stationary distributions of sequences that match given target distributions. By adopting comparatively realistic target distributions,evolutionary models can be improved. Instead of focusing on estimating parameters, we concentrate on the population genetic implications of these models. Specifically, we obtain estimates of the product of effective population size and relative fitness difference of alleles. The approach is illustrated with two applications to protein-coding DNA. In the first, a codon-based evolutionary model yields a stationary distribution of sequences, which, when the sequences are translated,matches a variable-length Markov model trained on human proteins. In the second, we introduce an insertion-deletion model that describes selectively neutral evolutionary changes to DNA. We then show how to modify the neutral model so that its stationary distribution at the amino acid level can match a profile hidden Markov model, such as the one associated with the Pfam database.

Projecting Cyberattacks Through Variable-Length Markov Models

Previous works in the area of network security have emphasized the creation of intrusion detection systems (IDSs) to flag malicious network traffic and computer usage, and the development of algorithms to analyze IDS alerts. One possible byproduct of correlating raw IDS data are attack tracks, which consist of ordered collections of alerts belonging to a single multistage attack. This paper presents a variable-length Markov model (VLMM) that captures the sequential properties of attack tracks, allowing for the prediction of likely future actions on ongoing attacks. The proposed approach is able to adapt to newly observed attack sequences without requiring specific network information. Simulation results are presented to demonstrate the performance of VLMM predictors and their adaptiveness to new attack scenarios.

Real-time 3-D human body tracking using learnt models of behaviour

In this paper, we introduce a 3-D human-body tracker capable of handling fast and complex motions in real-time. We build upon the Monte–Carlo Bayesian framework, and propose novel prediction and evaluation methods improving the robustness and efficiency of the tracker. The parameter space, augmented with first order derivatives, is automatically partitioned into Gaussian clusters each representing an elementary motion: hypothesis propagation inside each cluster is therefore accurate and efficient. The transitions between clusters use the predictions of a variable length Markov model which can explain high-level behaviours over a long history. Using Monte–Carlo methods, evaluation of model candidates is critical for both speed and robustness. We present a new evaluation scheme based on hierarchical 3-D reconstruction and blob-fitting, where appearance models and image evidences are represented by mixtures of Gaussian blobs. Our tracker is also capable of automatic-initialisation and self-recovery. We demonstrate the application of our tracker to long video sequences exhibiting rapid and diverse movements.

Evaluating Variable-Length Markov Chain Models for Analysis of User Web Navigation Sessions

Markov models have been widely used to represent and analyze user Web navigation data. In previous work, we have proposed a method to dynamically extend the order of a Markov chain model and a complimentary method for assessing the predictive power of such a variable-length Markov chain. Herein, we review these two methods and propose a novel method for measuring the ability of a variable-length Markov model to summarize user Web navigation sessions up to a given length. Although the summarization ability of a model is important to enable the identification of user navigation patterns, the ability to make predictions is important in order to foresee the next link choice of a user after following a given trail so as, for example, to personalize a Web site. We present an extensive experimental evaluation providing strong evidence that prediction accuracy increases linearly with summarization ability

A real-time hand tracker using variable-length Markov models of behaviour

We present a novel approach for visual tracking of structured behaviour as observed in human–computer interaction. An automatically acquired variable-length Markov model is used to represent the high-level structure and temporal ordering of gestures. Continuous estimation of hand posture is handled by combining the model with annealed particle filtering. The stochastic simulation updates and automatically switches between different model representations of hand posture that correspond to distinct gestures. The implementation executes in real time and demonstrates significant improvement in robustness over comparable methods. We provide a measurement of user performance when our method is applied to a Fitts’ law drag-and-drop task, and an analysis of the effects of latency that it introduces.

Optimized mixed Markov models for motif identification.

BackgroundIdentifying functional elements, such as transcriptional factor binding sites, is a fundamental step in reconstructing gene regulatory networks and remains a challenging issue, largely due to limited availability of training samples.ResultsWe introduce a novel and flexible model, the Optimized Mixture Markov model (OMiMa), and related methods to allow adjustment of model complexity for different motifs. In comparison with other leading methods, OMiMa can incorporate more than the NNSplice's pairwise dependencies; OMiMa avoids model over-fitting better than the Permuted Variable Length Markov Model (PVLMM); and OMiMa requires smaller training samples than the Maximum Entropy Model (MEM). Testing on both simulated and actual data (regulatory cis-elements and splice sites), we found OMiMa's performance superior to the other leading methods in terms of prediction accuracy, required size of training data or computational time. Our OMiMa system, to our knowledge, is the only motif finding tool that incorporates automatic selection of the best model. OMiMa is freely available at [1].ConclusionOur optimized mixture of Markov models represents an alternative to the existing methods for modeling dependent structures within a biological motif. Our model is conceptually simple and effective, and can improve prediction accuracy and/or computational speed over other leading methods.

Organization of the state space of a simple recurrent network before and after training on recursive linguistic structures

Recurrent neural networks are often employed in the cognitive science community to process symbol sequences that represent various natural language structures. The aim is to study possible neural mechanisms of language processing and aid in development of artificial language processing systems. We used data sets containing recursive linguistic structures and trained the Elman simple recurrent network (SRN) for the next-symbol prediction task. Concentrating on neuron activation clusters in the recurrent layer of SRN we investigate the network state space organization before and after training. Given a SRN and a training stream, we construct predictive models, called neural prediction machines, that directly employ the state space dynamics of the network. We demonstrate two important properties of representations of recursive symbol series in the SRN. First, the clusters of recurrent activations emerging before training are meaningful and correspond to Markov prediction contexts. We show that prediction states that naturally arise in the SRN initialized with small random weights approximately correspond to states of Variable Memory Length Markov Models (VLMM) based on individual symbols (i.e. words). Second, we demonstrate that during training, the SRN reorganizes its state space according to word categories and their grammatical subcategories, and the next-symbol prediction is again based on the VLMM strategy. However, after training, the prediction is based on word categories and their grammatical subcategories rather than individual words. Our conclusion holds for small depths of recursions that are comparable to human performances. The methods of SRN training and analysis of its state space introduced in this paper are of a general nature and can be used for investigation of processing of any other symbol time series by means of SRN.

Learning temporal structure for task based control

We present an extension for variable length Markov models (VLMMs) to allow for modelling of continuous input data and show that the generative properties of these VLMMs are a powerful tool for dealing with real world tracking issues. We explore methods for addressing the temporal correspondence problem in the context of a practical hand tracker, which is essential to support expectation in task-based control using these behavioural models. The hand tracker forms a part of a larger multi-component distributed system, providing 3-D hand position data to a gesture recogniser client. We show how the performance of such a hand tracker can be improved by using feedback from the gesture recogniser client. In particular, feedback based on the generative extrapolation of the recogniser's internal models is shown to help the tracker deal with mid-term occlusion. We also show that VLMMs can be used as a means to inform the prior in an expectation maximisation (EM) process used for joint spatial and temporal learning of image features.