Probabilistic Suffix Tree Research Articles

CapBad: Content-Agnostic, Payload-Based Anomaly Detector for Industrial Control Protocols

Efficient anomaly detection methods are urgently needed to prevent attacks in the application layer of the Industrial Internet of Things (IIoT). The existing intrusion detection systems have certain limitations in detecting abnormal packets exploited by the application-layer attacks. In this article, a content-agnostic-payload-based anomaly detector named the CapBad is proposed to detect malicious packets in the application layer of the IIoT system. Specifically, a phase-aware hidden semi-Markov model (pHSMM) is used to model the industrial control protocol packets and automatically learn the packets’ payload characteristics. The packet types are then inferred based on the packet likelihoods obtained by the pHSMM. In addition, the probabilistic suffix tree is employed to analyze the packets’ contextual similarity to the historical packets. The abnormal packets are finally detected by comparing their contextual similarity with that of the historical normal packets. The proposed algorithm is verified by simulations, and the results show that the CapBad has an excellent performance in detecting abnormal packets in the application layer.

Abnormal Crowd Traffic Detection for Crowdsourced Indoor Positioning in Heterogeneous Communications Networks

WiFi fingerprint-based indoor positioning system emerges to provide fundamental location-related service in heterogeneous communications networks. It relies on crowdsourcing technology in the collection of received signal strength (RSS) to dynamically update fingerprint database. However, this crowdsourced indoor positioning system is vulnerable to the intrusion of dishonest users (i.e., attackers). Attackers may manipulate the number of users who submit RSS fingerprints, and finally mislead the evaluation of crowd traffic evaluation. In this paper, we propose an abnormal crowd traffic detection (ACTD) scheme to identify attackers according to their abnormal RSS sensing behaviors. Specifically, a fog server is explored to serve as the crowdsourcing platform to perform data storage and detection. We first categorize attackers into three different levels according to their real geographical locations and collusion. Then, through the analysis of pseudonym changing behavior in RSS submission, we propose a rarity-based outlier detection to classify attackers of level-1. Furthermore, we propose a variable-length Markov model, i.e., probabilistic suffix tree (PST), to detect the colluded users who are not at the target point of interest (POI). In addition, a metric learning algorithm is developed to detect the collusion of AP organizer based on RSS fingerprint distance difference. The extensive simulation results show that the ACTD scheme can effectively resist attackers with high accuracy and appropriately deal with traffic evaluation from RSS fingerprint information.

Detecting Pattern Anomalies in Hydrological Time Series with Weighted Probabilistic Suffix Trees

Anomalous patterns are common phenomena in time series datasets. The presence of anomalous patterns in hydrological data may represent some anomalous hydrometeorological events that are significantly different from others and induce a bias in the decision-making process related to design, operation and management of water resources. Hence, it is necessary to extract those “anomalous” knowledge that can provide valuable and useful information for future hydrological analysis and forecasting from hydrological data. This paper focuses on the problem of detecting anomalous patterns from hydrological time series data, and proposes an effective and accurate anomalous pattern detection approach, TFSAX_wPST, which combines the advantages of the Trend Feature Symbolic Aggregate approximation (TFSAX) and weighted Probabilistic Suffix Tree (wPST). Experiments with different hydrological real-world time series are reported, and the results indicate that the proposed methods are fast and can correctly detect anomalous patterns for hydrological time series analysis, and thus promote the deep analysis and continuous utilization of hydrological time series data.

An Enhanced Human Activity Prediction at Rainy and Night Times

Human activity prediction aims to recognize an unfinished activity with limited motion and appearance information. A generalized activity prediction framework was proposed for human activity prediction where Probabilistic Suffix Tree (PST) was introduced to model casual relationships between constituent actions. Then, each kind of activity in videos was predicted by modeling interactive object information through Spatial Pattern Mining (SPM). This framework mined the temporal sequence patterns. For efficient human activity prediction a Spatio-Temporal Frequent Object Mining (STOM) was proposed in which the spatial, size and motion correlation among objects information were collected along with the temporal information. After the collection of this information, the objects were identified by using Modified Histogram Of Gradient (MHOG) and then the objects were tracked by particle filter technique. The frequent action of detected objects were identified by using Frequent Pattern-growth (FP-growth) which predicted the infrequent action as abnormal human activity in videos. However, MHOG based Object Detection and Tracking-STOM (MHOGODT-STFOM) based human activity prediction is not more effective at night time and rainy time. So in this paper, Enhanced Object Detection and Tracking- STFOM (EODT- STFOM) and Removing Rain Streaks-EODT-STFOM (RSR-EODT-STFOM) are proposed for human activity prediction even at night time and rainy time. In EODT, a modified Contrast Model is used which combined the contrast information and local entropy information to detect object contents present in the current image frame. Then, the objects are tracked by Kalman filter. In RSR-EODT, the rain streaks in the images are removed based on the deep Convolutional Neural Network (CNN). Then the objects are detected and tracked by modified Contrast Model and Kalman filter respectively. After the object detection and object tracking by EODT and RSR-EODT, the frequent actions are obtained by applying STFOM. The frequent actions are considered as normal activities and the infrequent actions are considered as abnormal activities. Thus the proposed EODT-STFOM and RSR-EODT- STFOM methods predict the human activities even at night time and rainy time.

FAAD: an unsupervised fast and accurate anomaly detection method for a multi-dimensional sequence over data stream

Recently, sequence anomaly detection has been widely used in many fields. Sequence data in these fields are usually multi-dimensional over the data stream. It is a challenge to design an anomaly detection method for a multi-dimensional sequence over the data stream to satisfy the requirements of accuracy and high speed. It is because: (1) Redundant dimensions in sequence data and large state space lead to a poor ability for sequence modeling; (2) Anomaly detection cannot adapt to the high-speed nature of the data stream, especially when concept drift occurs, and it will reduce the detection rate. On one hand, most existing methods of sequence anomaly detection focus on the single-dimension sequence. On the other hand, some studies concerning multi-dimensional sequence concentrate mainly on the static database rather than the data stream. To improve the performance of anomaly detection for a multi-dimensional sequence over the data stream, we propose a novel unsupervised fast and accurate anomaly detection (FAAD) method which includes three algorithms. First, a method called “information calculation and minimum spanning tree cluster” is adopted to reduce redundant dimensions. Second, to speed up model construction and ensure the detection rate for the sequence over the data stream, we propose a method called “random sampling and subsequence partitioning based on the index probabilistic suffix tree.” Last, the method called “anomaly buffer based on model dynamic adjustment” dramatically reduces the effects of concept drift in the data stream. FAAD is implemented on the streaming platform Storm to detect multi-dimensional log audit data. Compared with the existing anomaly detection methods, FAAD has a good performance in detection rate and speed without being affected by concept drift.

A Personal Location Prediction Method Based on Individual Trajectory and Group Trajectory

With the rapid development of communication technology, a large amount of spatiotemporal trajectory data has been produced. One of the critical applications of trajectory data is location prediction that is important for urban traffic planning and location-based services. Although many methods for personal location prediction have been proposed, to the best of our knowledge, some users always have sparse historical trajectory data in practical applications, resulting in poor location prediction precision. Targeting on this challenge, we propose an Individual Trajectory-Group Trajectory (ITGT) location prediction model by utilizing the pattern of group travels. First, the model performs the stay point extraction and conducts the spatial clustering to construct the clustering link. Second, Fano's inequality and clustering link are used to evaluate the predictability of location. Third, two variable order Markov models, named prediction by partial match (PPM) and probabilistic suffix tree (PST), are adopted to predict the clustering link. Finally, our approach is evaluated by using 608 712 points from 5000 volunteers at Shenzhen, China. The results show that: 1) when using individual trajectory, the PPM individual model is superior to the conventional N-order Markov model and PST individual model; 2) when all group trajectories are used, the PPM group model is not as accurate as the PPM individual model; and 3) when it classifies the group trajectory into different traffic zones, the PPM zone model is better than the PPM group model and the PPM individual model. The prediction precision of 1-3 order PPM zone model is 83.21%, 86.75%, and 87.35%, respectively, which introduces approximately 3% performance gains by utilizing the characters of traffic zone groups.

Execution anomaly detection in large-scale systems through console log analysis

Execution anomaly detection is important for development, maintenance and performance tuning in large-scale systems. System console logs are the significant source of troubleshooting and problem diagnosis. However, manually inspecting logs to detect anomalies is unfeasible due to the increasing volume and complexity of log files. Therefore, this is a substantial demand for automatic anomaly detection based on log analysis. In this paper, we propose a general method to mine console logs to detect system problems. We first give some formal definitions of the problem, and then extract the set of log statements in the source code and generate the reachability graph to reveal the reachable relations of log statements. After that, we parse the log files to create log messages by combining information about log statements with information retrieval techniques. These messages are grouped into execution traces according to their execution units. We propose a novel anomaly detection algorithm that considers traces as sequence data and uses a probabilistic suffix tree based method to organize and differentiate significant statistical properties possessed by the sequences. Experiments on a CloudStack testbed and a Hadoop production system show that our method can effectively detect running anomalies in comparison with existing four detection algorithms.

Log sequence clustering for workflow mining in multi-workflow systems

Current workflow mining efforts aim to discover process knowledge from user-system interaction logs and represent it as high-level workflow models. They assume there is one single workflow model in a system, or rely on the information that can explicitly link each log sequence to the underlying workflow model. Such assumptions may not be applicable to multi-workflow systems where the instances of different workflow models are mixed together without being differentiated. To address this issue, this paper proposes to apply sequence clustering methods to group similar log sequences together. Each sequence cluster corresponds to a workflow model and the log sequences in the cluster are the corresponding instances. This paper investigates different similarity measures, including structure-based and user-based, as well as different clustering algorithms, including one-side clustering and co-clustering. In order to incorporate user factors into sequence clustering, which is novel to the current sequence clustering methods, this paper proposes to model User Behavior Patterns (UBPs) as probabilistic distributions over sequences and learn it from the event log. We represent a UBP as a Probabilistic Suffix Tree and use it to measure sequence similarity. The co-clustering method leverages the dyad relationship between UBPs and log sequences to improve the clustering accuracy. An experimental study has been conducted and the result indicates that user-based methods outperform structure-based methods in terms of accuracy and they are more effective on dealing with noises in the log and the increase of log size. The UBP-sequence co-clustering method achieves the best performance which indicates the effectiveness of incorporating user factors and applying co-clustering.

Early Recognition of 3D Human Actions

Action recognition is an important research problem of human motion analysis (HMA). In recent years, 3D observation-based action recognition has been receiving increasing interest in the multimedia and computer vision communities, due to the recent advent of cost-effective sensors, such as depth camera Kinect. This work takes this one step further, focusing on early recognition of ongoing 3D human actions, which is beneficial for a large variety of time-critical applications, e.g., gesture-based human machine interaction, somatosensory games, and so forth. Our goal is to infer the class label information of 3D human actions with partial observation of temporally incomplete action executions. By considering 3D action data as multivariate time series (m.t.s.) synchronized to a shared common clock (frames), we propose a stochastic process called dynamic marked point process (DMP) to model the 3D action as temporal dynamic patterns, where both timing and strength information are captured. To achieve even more early and better accuracy of recognition, we also explore the temporal dependency patterns between feature dimensions. A probabilistic suffix tree is constructed to represent sequential patterns among features in terms of the variable-order Markov model (VMM). Our approach and several baselines are evaluated on five 3D human action datasets. Extensive results show that our approach achieves superior performance for early recognition of 3D human actions.

Horizontally scalable probabilistic generalized suffix tree (PGST) based route prediction using map data and GPS traces

Route prediction is an essential requirement for many intelligent transport systems (ITS) services like VANETS, traffic congestion estimation, resource prediction in grid computing etc. This work focuses on building an end-to-end horizontally scalable route prediction application based on statistical modeling of user travel data. Probabilistic suffix tree (PST) is one of widely used sequence indexing technique which serves a model for prediction. The probabilistic generalized suffix tree (PGST) is a variant of PST and is essentially a suffix tree built from a huge number of smaller sequences. We construct generalized suffix tree model from a large number of trips completed by the users. User trip raw GPS traces is mapped to the digitized road network by parallelizing map matching technique leveraging map reduce framework. PGST construction from the huge volume of data by processing sequentially is a bottleneck in the practical realization. Most of the existing works focused on time-space tradeoffs on a single machine. Proposed technique solves this problem by a two-step process which is intuitive to execute in the map-reduce framework. In the first step, computes all the suffixes along with their frequency of occurrences and in the second step, builds probabilistic generalized suffix tree. The probabilistic aspect of the tree is also taken care so that it can be used as a model for prediction application. Dataset used are road network spatial data and GPS traces of users. Experiments carried out on real datasets available in public domain.

Analyzing State Sequences with Probabilistic Suffix Trees: ThePSTRPackage

This article presents the PST R package for categorical sequence analysis with probabilistic suffix trees (PSTs), i.e., structures that store variable-length Markov chains (VLMCs). VLMCs allow to model high-order dependencies in categorical sequences with parsimonious models based on simple estimation procedures. The package is specifically adapted to the field of social sciences, as it allows for VLMC models to be learned from sets of individual sequences possibly containing missing values; in addition, the package is extended to account for case weights. This article describes how a VLMC model is learned from one or more categorical sequences and stored in a PST. The PST can then be used for sequence prediction, i.e., to assign a probability to whole observed or artificial sequences. This feature supports data mining applications such as the extraction of typical patterns and outliers. This article also introduces original visualization tools for both the model and the outcomes of sequence prediction. Other features such as functions for pattern mining and artificial sequence generation are described as well. The PST package also allows for the computation of probabilistic divergence between two models and the fitting of segmented VLMCs, where sub-models fitted to distinct strata of the learning sample are stored in a single PST.

Proactive channel access in cognitive radio networks using statistical radio environment maps

Inclusion of statistical knowledge of the primary user (PU) channel usage had shown to be beneficial in dynamic spectrum access. Motivated by this fact, this paper investigated the importance of collecting and using statistics on neighboring secondary users (SUs) in selecting channels in addition to the knowledge of PU channel usage. The paper assumed that PU traffic characteristics of the channels are included in the radio environment map in the form of probabilistic suffix trees, which is a sequence predictor based on Markov property. In the proposed method, an intelligent sequence hopping-based common control channel and a carrier sense multiple access (CSMA)/collision avoidance (CA)-based medium access control (MAC) protocol were introduced. As shown in the paper, selecting channels using statistics of both the neighboring SUs and PUs reduced the number of packet collisions compared to a scheme which only uses PU statistics. Furthermore, the simulation results showed that the scheme proposed had better throughput performance with respect to both the random channel selection scheme and the scheme which only uses PU statistics while having less training complexity.

Prediction of Indel flanking regions in protein sequences using a variable-order Markov model.

Insertion/deletion (indel) and amino acid substitution are two common events that lead to the evolution of and variations in protein sequences. Further, many of the human diseases and functional divergence between homologous proteins are more related to indel mutations, even though they occur less often than the substitution mutations do. A reliable identification of indels and their flanking regions is a major challenge in research related to protein evolution, structures and functions. In this article, we propose a novel scheme to predict indel flanking regions in a protein sequence for a given protein fold, based on a variable-order Markov model. The proposed indel flanking region (IndelFR) predictors are designed based on prediction by partial match (PPM) and probabilistic suffix tree (PST), which are referred to as the PPM IndelFR and PST IndelFR predictors, respectively. The overall performance evaluation results show that the proposed predictors are able to predict IndelFRs in the protein sequences with a high accuracy and F1 measure. In addition, the results show that if one is interested only in predicting IndelFRs in protein sequences, it would be preferable to use the proposed predictors instead of HMMER 3.0 in view of the substantially superior performance of the former.

Prediction of Human Activity by Discovering Temporal Sequence Patterns.

Early prediction of ongoing human activity has become more valuable in a large variety of time-critical applications. To build an effective representation for prediction, human activities can be characterized by a complex temporal composition of constituent simple actions and interacting objects. Different from early detection on short-duration simple actions, we propose a novel framework for long -duration complex activity prediction by discovering three key aspects of activity: Causality, Context-cue, and Predictability. The major contributions of our work include: (1) a general framework is proposed to systematically address the problem of complex activity prediction by mining temporal sequence patterns; (2) probabilistic suffix tree (PST) is introduced to model causal relationships between constituent actions, where both large and small order Markov dependencies between action units are captured; (3) the context-cue, especially interactive objects information, is modeled through sequential pattern mining (SPM), where a series of action and object co-occurrence are encoded as a complex symbolic sequence; (4) we also present a predictive accumulative function (PAF) to depict the predictability of each kind of activity. The effectiveness of our approach is evaluated on two experimental scenarios with two data sets for each: action-only prediction and context-aware prediction. Our method achieves superior performance for predicting global activity classes and local action units.

Pattern-based causal relationships discovery from event sequences for modeling behavioral user profile in ubiquitous environments

This paper presents a novel and practical model for behavioral user profile modeling using causal relationships. In this model, causal relationships, which represent the influence among variables, are discovered from event sequences representing users behaviors, and used for modeling behavioral user profiles. Our model first discovers significant patterns using probabilistic suffix trees, and then discovers pattern correlations using a new sequence clustering algorithm and a modified version of the normalized mutual information (NMI) measure. Causal relationships between the significant patterns are then discovered using the transfer entropy approach. These relationships are used to construct the causal graphs of activities to generate user profiles. Through extensive experiments over a variety of datasets, we empirically demonstrate that these causality-based profiles lead to significant improvement of performance in activity prediction and user identification. We also show that our proposed model is generic and effective in constructing individual user profiles and common profiles for groups of users, in indoor and outdoor environments.

Prediction of moving object trajectory based on probabilistic suffix tree

In the prediction of moving object trajectory, concerning the low accuracy rate of low order Markov model and the expansion of state space in high order model, a dynamic adaptive Probabilistic Suffix Tree( PST) prediction method based on variable length Markov model was proposed. Firstly, moving object's trajectory path was serialized according to the time;then the probability characteristic of sequence context was trained and calculated from the historical trajectory data of moving objects, the probabilistic suffix tree model based path sequence was constructed, combined with the actual trajectory data,thus the future trajectory information could be predicted dynamically and adaptively. The experimental results show that the highest prediction accuracy was obtained in second order model, with the order of the model increasing, the prediction accuracy was maintained at about 82% and better prediction results were achieved. In the meantime, space complexity was decreased exponentially and storage space was reduced greatly. The proposed method made full use of historical data and current trajectory information to predict the future trajectory, and provided a more flexible and efficient location-based services.

QS-STT: QuadSection clustering and spatial-temporal trajectory model for location prediction

Location prediction is a crucial need for location-aware services and applications. Given an object's recent movement and a future time, the goal of location prediction is to predict the location of the object at the future time specified. Different from traditional location prediction using motion function, some research works have elaborated on mining movement behavior from historical trajectories for location prediction. Without loss of generality, given a set of trajectories of an object, prior works on mining movement behaviors will first extract regions of popularity, in which the object frequently appears, and then discover the sequential relationships among regions. However, the quality of the frequent regions extracted affects the accuracy of the location prediction. Furthermore, trajectory data has both spatial and temporal information. To further enhance the accuracy of location prediction, one could utilize not only spatial information but also temporal information to predict the locations of objects. In this paper, we propose a framework QS-STT (standing for QuadSection clustering and Spatial-Temporal Trajectory model) to capture the movement behaviors of objects for location prediction. Specifically, we have developed QuadSection clustering to extract a reasonable and near-optimal set of frequent regions. Then, based on the set of frequent regions, we propose a spatial-temporal trajectory model to explore the object's movement behavior as a probabilistic suffix tree with both spatial and temporal information of movements. Note that STT is not only able to discover sequential relationships among regions but also derives the corresponding probabilities of time, indicating when the object appears in each region. Based on STT, we further propose an algorithm to traverse STT for location prediction. By enhancing the quality of the frequent region extracted and exploring both the spatial and temporal information of STT, the accuracy of location prediction in QS-STT is improved. QS-STT is designed for individual location prediction. For verifying the effectiveness of QS-STT for location prediction under the different spatial density, we have conducted experiments on four types of real trajectory datasets with different speed. The experimental results show that our proposed QS-STT is able to capture both spatial and temporal patterns of movement behaviors and by exploring QS-STT, our proposed prediction algorithm outperforms existing works.

Probabilistic suffix array: efficient modeling and prediction of protein families

Markov models are very popular for analyzing complex sequences such as protein sequences, whose sources are unknown, or whose underlying statistical characteristics are not well understood. A major problem is the computational complexity involved with using Markov models, especially the exponential growth of their size with the order of the model. The probabilistic suffix tree (PST) and its improved variant sparse probabilistic suffix tree (SPST) have been proposed to address some of the key problems with Markov models. The use of the suffix tree, however, implies that the space requirement for the PST/SPST could still be high. We present the probabilistic suffix array (PSA), a data structure for representing information in variable length Markov chains. The PSA essentially encodes information in a Markov model by providing a time and space-efficient alternative to the PST/SPST. Given a sequence of length N, construction and learning in the PSA is done in O(N) time and space, independent of the Markov order. Prediction using the PSA is performed in O(mlog{N is divided by /Σ/}) time, where m is the pattern length, and Σ is the symbol alphabet. In terms of modeling and prediction accuracy, using protein families from Pfam 25.0, SPST and PSA produced similar results (SPST 89.82%, PSA 89.56%), but slightly lower than HMMER3 (92.55%). A modified algorithm for PSA prediction improved the performance to 91.7%, or just 0.79% from HMMER3 results. The average (maximum) practical construction space for the protein families tested was 21.58±6.32N (41.11N) bytes using the PSA, 27.55±13.16N (63.01N) bytes using SPST and 47±24.95N (140.3N) bytes for HMMER3. The PSA was 255 times faster to construct than the SPST, and 11 times faster than HMMER3.

On Finite Memory Universal Data Compression and Classification of Individual Sequences

Consider the case where consecutive blocks of letters of a semi-infinite individual sequence over a finite-alphabet are being compressed into binary sequences by some one-to-one mapping. No a priori information about is available at the encoder, which must therefore adopt a universal data-compression algorithm. It is known that if the universal Lempel-Ziv (LZ) data compression algorithm is successively applied to -blocks then the best error-free compression, for the particular individual sequence is achieved as tends to infinity. The best possible compression that may be achieved by any universal data compression algorithm for finite -blocks is discussed. It is demonstrated that context tree coding essentially achieves it. Next, consider a device called classifier (or discriminator) that observes an individual training sequence . The classifier's task is to examine individual test sequences of length and decide whether the test -sequence has the same features as those that are captured by the training sequence , or is sufficiently different, according to some appropriate criterion. Here again, it is demonstrated that a particular universal context classifier with a storage-space complexity that is linear in , is essentially optimal. This may contribute a theoretical ldquoindividual sequencerdquo justification for the Probabilistic Suffix Tree (PST) approach in learning theory and in computational biology.

Probabilistic suffix models for API sequence analysis of Windows XP applications

Given the pervasive nature of malicious mobile code (viruses, worms, etc.), developing statistical/structural models of code execution is of considerable importance. We investigate using probabilistic suffix trees (PSTs) and associated suffix automata (PSAs) to build models of benign application behavior with the goal of subsequently being able to detect malicious applications as anything that deviates therefrom. We describe these probabilistic suffix models and present new generic analysis and manipulation algorithms. The models and the algorithms are then used in the context of API (i.e., system call) sequences realized by Windows XP applications. The analysis algorithms, when applied to traces (i.e., sequences of API calls) of benign and malicious applications, aid in choosing an appropriate modeling strategy in terms of distance metrics and consequently provide classification measures in terms of sequence-to-model matching. We give experimental results based on classification of unobserved traces of benign and malicious applications against a suffix model trained solely from traces generated by a small set of benign applications.