Traditional Sequential Pattern Mining Research Articles

Efficient weighted sequential pattern mining

In real-life applications, data mining task involves extracting valuable but hidden information from massive data. How to effectively find out interesting patterns from large databases is a current topic. Sequential pattern mining is the most popular approach in data mining domain. Traditional sequential pattern mining research generally focuses on discovering frequent sequential patterns. However, the account of occurrence times of patterns does not adequately indicate their importance. For instance, frequent patterns (e.g., pencil and eraser) are not profitable, whereas infrequent patterns (e.g., extreme weather) are high-risk. To extract more useful information, researchers study a weighted sequential pattern mining task. In this paper, an efficient algorithm for weighted sequential pattern mining task, called EWSPM, is proposed. Two new strict upper bounds, namely MWEbound and MSRIWbound, are designed based on the concepts of maximum weight estimation (simplified as MWE) and maximum sumation of remaining item weights (simplified as MSRIW), respectively. These upper bounds achieve better pruning effects and reduce the size of search space during the mining process, which significantly shortens execution time. In addition, a database-projection method is employed to optimize memory usage. It addresses potential memory explosion issues in a certain degree. Finally, we also conducted extensive experiments on nine datasets (including real and synthetic). The experimental results demonstrate that the EWSPM algorithm is capable of mining all interesting patterns efficiently, with the smallest size of search space. Additionally, the novel algorithm also exhibits superior performance in terms of execution time and memory consumption.

OPP-Miner: Order-Preserving Sequential Pattern Mining for Time Series.

Traditional sequential pattern mining methods were designed for symbolic sequence. As a collection of measurements in chronological order, a time series needs to be discretized into symbolic sequences, and then users can apply sequential pattern mining methods to discover interesting patterns in time series. The discretization will not only cause the loss of some important information, which partially destroys the continuity of time series, but also ignore the order relations between time-series values. Inspired by order-preserving matching, this article explores a new method called order-preserving sequential pattern (OPP) mining, which does not need to discretize time series into symbolic sequences and represents patterns based on the order relations of time series. An inherent advantage of such representation is that the trend of a time series can be represented by the relative order of the values underneath time series. We propose an OPP-Miner algorithm to mine frequent patterns in time series with the same relative order. OPP-Miner employs the filtration and verification strategies to calculate the support and uses the pattern fusion strategy to generate candidate patterns. To compress the result set, we also study to find the maximal OPPs. Experimental results validate that OPP-Miner is not only efficient but can also discover similar subsequences in time series. In addition, case studies show that our algorithms have high utility in analyzing the COVID-19 epidemic by identifying critical trends and improve the clustering performance. The algorithms and data can be downloaded from https://github.com/wuc567/Pattern-Mining/tree/master/OPP-Miner.

Status Set Sequential Pattern Mining Considering Time Windows and Periodic Analysis of Patterns.

The traditional sequential pattern mining method is carried out considering the whole time period and often ignores the sequential patterns that only occur in local time windows, as well as possible periodicity. Therefore, in order to overcome the limitations of traditional methods, this paper proposes status set sequential pattern mining with time windows (SSPMTW). In contrast to traditional methods, the item status is considered, and time windows, minimum confidence, minimum coverage, minimum factor set ratios and other constraints are added to mine more valuable rules in local time windows. The periodicity of these rules is also analyzed. According to the proposed method, this paper improves the Apriori algorithm, proposes the TW-Apriori algorithm, and explains the basic idea of the algorithm. Then, the feasibility, validity and efficiency of the proposed method and algorithm are verified by small-scale and large-scale examples. In a large-scale numerical example solution, the influence of various constraints on the mining results is analyzed. Finally, the solution results of SSPM and SSPMTW are compared and analyzed, and it is suggested that SSPMTW can excavate the laws existing in local time windows and analyze the periodicity of the laws, which solves the problem of SSPM ignoring the laws existing in local time windows and overcomes the limitations of traditional sequential pattern mining algorithms. In addition, the rules mined by SSPMTW reduce the entropy of the system.

Mining frequent pyramid patterns from time series transaction data with custom constraints

For the problem of mining pyramid scheme patterns, the traditional sequential pattern mining algorithm Prefixspan has many disadvantages such as poor timeliness, uniform threshold, etc. Therefore, we propose a timeliness variable threshold and increment Prefixspan algorithm, named TVI-Prefixspan, for mining the sequential patterns from time series transaction data. To be specific, TVI-Prefixspan aims to mine the patterns that co-occurrence in both an individual sequence and different sequences with high frequency. The most important challenges are how to define the thresholds of frequent one-item and pyramid patterns. We firstly analyze the attributes of the patterns which are hidden in the financial activities between different bank accounts. Secondly, the frequent threshold of each one-item is determined by its different frequency value in normal and pyramid related transaction sequences. We also consider the special relationships in both numerical values and time-series aspects between each pattern’s item. Therefore, TVI-Prefixspan produces the frequent one-item set based on its difference of the normal frequency, and then, mines the pyramid patterns with formulated relation constraints. For describing the correlation, we consider sequential, time interval and one-off constraints simultaneously. The experimental results, in real financial data containing pyramid transactions, show that TVI-Prefixspan algorithm succeeds in mining pyramid scheme patterns quickly and effectively. It is superior to traditional sequential pattern mining algorithms such as Prefixspan in efficiency and mining effect.

NetNCSP: Nonoverlapping closed sequential pattern mining

Sequential pattern mining (SPM) has been applied in many fields. However, traditional SPM neglects the pattern repetition in sequence. To solve this problem, gap constraint SPM was proposed and can avoid finding too many useless patterns. Nonoverlapping SPM, as a branch of gap constraint SPM, means that any two occurrences cannot use the same sequence letter in the same position as the occurrences. Nonoverlapping SPM can make a balance between efficiency and completeness. The frequent patterns discovered by existing methods normally contain redundant patterns. To reduce redundant patterns and improve the mining performance, this paper adopts the closed pattern mining strategy and proposes a complete algorithm, named Nettree for Nonoverlapping Closed Sequential Pattern (NetNCSP) based on the Nettree structure. NetNCSP is equipped with two key steps, support calculation and closeness determination. A backtracking strategy is employed to calculate the nonoverlapping support of a pattern on the corresponding Nettree, which reduces the time complexity. This paper also proposes three kinds of pruning strategies, inheriting, predicting, and determining. These pruning strategies are able to find the redundant patterns effectively since the strategies can predict the frequency and closeness of the patterns before the generation of the candidate patterns. Experimental results show that NetNCSP is not only more efficient but can also discover more closed patterns with good compressibility. Furtherly, in biological experiments NetNCSP mines the closed patterns in SARS-CoV-2 and SARS viruses. The results show that the two viruses are of similar pattern composition with different combinations.

Dramatically Reducing Search for High Utility Sequential Patterns by Maintaining Candidate Lists

A ubiquitous challenge throughout all areas of data mining, particularly in the mining of frequent patterns in large databases, is centered on the necessity to reduce the time and space required to perform the search. The extent of this reduction proportionally facilitates the ability to identify patterns of interest. High utility sequential pattern mining (HUSPM) seeks to identify frequent patterns that are (1) sequential in nature and (2) hold a significant magnitude of utility in a sequence database, by considering the aspect of item value or importance. While traditional sequential pattern mining relies on the downward closure property to significantly reduce the required search space, with HUSPM, this property does not hold. To address this drawback, an approach is proposed that establishes a tight upper bound on the utility of future candidate sequential patterns by maintaining a list of items that are deemed potential candidates for concatenation. Such candidates are provably the only items that are ever needed for any extension of a given sequential pattern or its descendants in the search tree. This list is then exploited to significantly further tighten the upper bound on the utilities of descendent patterns. An extension of this work is then proposed that significantly reduces the computational cost of updating database utilities each time a candidate item is removed from the list, resulting in a massive reduction in the number of candidate sequential patterns that need to be generated in the search. Sequential pattern mining methods implementing these new techniques for bound reduction and further candidate list reduction are demonstrated via the introduction of the CRUSP and CRUSPPivot algorithms, respectively. Validation of the techniques was conducted on six public datasets. Tests show that use of the CRUSP algorithm results in a significant reduction in the overall number of candidate sequential patterns that need to be considered, and subsequently a significant reduction in run time, when compared to the current state of the art in bounding techniques. When employing the CRUSPPivot algorithm, the further reduction in the size of the search space was found to be dramatic, with the reduction in run time found to be dramatic to moderate, depending on the dataset. Demonstrating the practical significance of the work, experiments showed that time required for one particularly complex dataset was reduced from many hours to less than one minute.

A Survey of Parallel Sequential Pattern Mining

With the growing popularity of shared resources, large volumes of complex data of different types are collected automatically. Traditional data mining algorithms generally have problems and challenges including huge memory cost, low processing speed, and inadequate hard disk space. As a fundamental task of data mining, sequential pattern mining (SPM) is used in a wide variety of real-life applications. However, it is more complex and challenging than other pattern mining tasks, i.e., frequent itemset mining and association rule mining, and also suffers from the above challenges when handling the large-scale data. To solve these problems, mining sequential patterns in a parallel or distributed computing environment has emerged as an important issue with many applications. In this article, an in-depth survey of the current status of parallel SPM (PSPM) is investigated and provided, including detailed categorization of traditional serial SPM approaches, and state-of-the art PSPM. We review the related work of PSPM in details including partition-based algorithms for PSPM, apriori-based PSPM, pattern-growth-based PSPM, and hybrid algorithms for PSPM, and provide deep description (i.e., characteristics, advantages, disadvantages, and summarization) of these parallel approaches of PSPM. Some advanced topics for PSPM, including parallel quantitative/weighted/utility SPM, PSPM from uncertain data and stream data, hardware acceleration for PSPM, are further reviewed in details. Besides, we review and provide some well-known open-source software of PSPM. Finally, we summarize some challenges and opportunities of PSPM in the big data era.

A MapReduce solution for incremental mining of sequential patterns from big data

Sequential Pattern Mining (SPM) is a popular data mining task with broad applications. With the advent of big data, traditional SPM algorithms are not scalable. Hence, many of the researchers have migrated to big data frameworks such as MapReduce and proposed distributed algorithms. However, the existing MapReduce algorithms assume the data as static and do not handle the incremental database updates. Moreover, they use to re-mine the updated database while new sequences are inserted. In this paper, we propose an efficient distributed algorithm for incremental sequential pattern mining (MR-INCSPM) using the MapReduce framework that can handle big data. The proposed algorithm incorporates the backward mining approach that efficiently makes use of the knowledge obtained during the previous mining process. Also, based on the study of item co-occurrences, we propose Co-occurrence Reverse Map (CRMAP) data structure. The issue of combinatorial explosion of candidate sequences is dealt using the proposed CRMAP data structure. Besides, a novel candidate generation and early prune mechanisms are designed using CRMAP to speed up the mining process. The proposed algorithm is evaluated on both the real and synthetic datasets. The experimental results prove the efficacy of MR-INCSPM with respect to processing time, memory and pruning efficiency.

Sequential Pattern Mining Algorithm Based on Text Data: Taking the Fault Text Records as an Example

Sequential pattern mining (SPM) is an effective and important method for analyzing time series. This paper proposed a SPM algorithm to mine fault sequential patterns in text data. Because the structure of text data is poor and there are many different forms of text expression for the same concept, the traditional SPM algorithm cannot be directly applied to text data. The proposed algorithm is designed to solve this problem. First, this study measured the similarity of fault text data and classified similar faults into one class. Next, this paper proposed a new text similarity measurement model based on the word embedding distance. Compared with the classic text similarity measurement method, this model can achieve good results in short text classification. Then, on the basis of fault classification, this paper proposed the SPM algorithm with an event window, which is a time soft constraint for obtaining a certain number of sequential patterns according to needs. Finally, this study used the fault text records of a certain aircraft as experimental data for mining fault sequential patterns. Experiment showed that this algorithm can effectively mine sequential patterns in text data. The proposed algorithm can be widely applied to text time series data in many fields such as industry, business, finance and so on.

CloFAST: closed sequential pattern mining using sparse and vertical id-lists

Sequential pattern mining is a computationally challenging task since algorithms have to generate and/or test a combinatorially explosive number of intermediate subsequences. In order to reduce complexity, some researchers focus on the task of mining closed sequential patterns. This not only results in increased efficiency, but also provides a way to compact results, while preserving the same expressive power of patterns extracted by means of traditional (non-closed) sequential pattern mining algorithms. In this paper, we present CloFAST, a novel algorithm for mining closed frequent sequences of itemsets. It combines a new data representation of the dataset, based on sparse id-lists and vertical id-lists, whose theoretical properties are studied in order to fast count the support of sequential patterns, with a novel one-step technique both to check sequence closure and to prune the search space. Contrary to almost all the existing algorithms, which iteratively alternate itemset extension and sequence extension, CloFAST proceeds in two steps. Initially, all closed frequent itemsets are mined in order to obtain an initial set of sequences of size 1. Then, new sequences are generated by directly working on the sequences, without mining additional frequent itemsets. A thorough performance study with both real-world and artificially generated datasets empirically proves that CloFAST outperforms the state-of-the-art algorithms, both in time and memory consumption, especially when mining long closed sequences.

A new approach for the deep order preserving submatrix problem based on sequential pattern mining

As an effective biclustering model, order-preserving submatrix (OPSM) has been widely applied to biological gene expression data mining, which can capture the general tendency of the gene expression under some experimental conditions. Recently, biologists hope to find deep OPSMs with long patterns and comparatively fewer support rows, which are important for the interpretation of gene regulatory networks. However, the traditional exact mining algorithms based on Apriori principle could not deal with the deep OPSM problem, which often take a large minimum support threshold for pattern pruning, and inevitably miss some significant deep OPSMs. In this paper, a new exact algorithm is proposed for mining deep OPSMs. Firstly all the common subsequences shared by every two rows are found out, then the row sets corresponding to the same common subsequences are formed. Finally all the deep OPSMs with support beyond the given threshold will be obtained. Experiments have been done in both real and synthetic data sets, and the results show that this new algorithm is capable of mining all the deep OPSMs over a small support. Under different thresholds of minimum support, this algorithm reveals better performance than the traditional sequential pattern mining algorithms.

An efficient approach for finding weighted sequential patterns from sequence databases

Weighted sequential pattern mining has recently been discussed in the field of data mining. Different from traditional sequential pattern mining, this kind of mining considers different significances of items in real applications, such as cost or profit. Most of the related studies adopt the maximum weighted upper-bound model to find weighted sequential patterns, but they generate a large number of unpromising candidate subsequences. In this study, we thus propose an efficient approach for finding weighted sequential patterns from sequence databases. In particular, a tightening strategy in the proposed approach is proposed to obtain more accurate weighted upper-bounds for subsequences in mining. Through the experimental evaluation, the results also show the proposed approach has good performance in terms of pruning effectiveness and execution efficiency.

Applying the maximum utility measure in high utility sequential pattern mining

Recently, high utility sequential pattern mining has been an emerging popular issue due to the consideration of quantities, profits and time orders of items. The utilities of subsequences in sequences in the existing approach are difficult to be calculated due to the three kinds of utility calculations. To simplify the utility calculation, this work then presents a maximum utility measure, which is derived from the principle of traditional sequential pattern mining that the count of a subsequence in the sequence is only regarded as one. Hence, the maximum measure is properly used to simplify the utility calculation for subsequences in mining. Meanwhile, an effective upper-bound model is designed to avoid information losing in mining, and also an effective projection-based pruning strategy is designed as well to cause more accurate sequence-utility upper-bounds of subsequences. The indexing strategy is also developed to quickly find the relevant sequences for prefixes in mining, and thus unnecessary search time can be reduced. Finally, the experimental results on several datasets show the proposed approach has good performance in both pruning effectiveness and execution efficiency.

An efficient method for mining cross-timepoint gene regulation sequential patterns from time course gene expression datasets

BackgroundObservation of gene expression changes implying gene regulations using a repetitive experiment in time course has become more and more important. However, there is no effective method which can handle such kind of data. For instance, in a clinical/biological progression like inflammatory response or cancer formation, a great number of differentially expressed genes at different time points could be identified through a large-scale microarray approach. For each repetitive experiment with different samples, converting the microarray datasets into transactional databases with significant singleton genes at each time point would allow sequential patterns implying gene regulations to be identified. Although traditional sequential pattern mining methods have been successfully proposed and widely used in different interesting topics, like mining customer purchasing sequences from a transactional database, to our knowledge, the methods are not suitable for such biological dataset because every transaction in the converted database may contain too many items/genes.ResultsIn this paper, we propose a new algorithm called CTGR-Span (Cross-Timepoint Gene Regulation Sequential pattern) to efficiently mine CTGR-SPs (Cross-Timepoint Gene Regulation Sequential Patterns) even on larger datasets where traditional algorithms are infeasible. The CTGR-Span includes several biologically designed parameters based on the characteristics of gene regulation. We perform an optimal parameter tuning process using a GO enrichment analysis to yield CTGR-SPs more meaningful biologically. The proposed method was evaluated with two publicly available human time course microarray datasets and it was shown that it outperformed the traditional methods in terms of execution efficiency. After evaluating with previous literature, the resulting patterns also strongly correlated with the experimental backgrounds of the datasets used in this study.ConclusionsWe propose an efficient CTGR-Span to mine several biologically meaningful CTGR-SPs. We postulate that the biologist can benefit from our new algorithm since the patterns implying gene regulations could provide further insights into the mechanisms of novel gene regulations during a biological or clinical progression. The Java source code, program tutorial and other related materials used in this program are available at http://websystem.csie.ncku.edu.tw/CTGR-Span.rar.

DFSP: a Depth-First SPelling algorithm for sequential pattern mining of biological sequences

Scientific progress in recent years has led to the generation of huge amounts of biological data, most of which remains unanalyzed. Mining the data may provide insights into various realms of biology, such as finding co-occurring biosequences, which are essential for biological data mining and analysis. Data mining techniques like sequential pattern mining may reveal implicitly meaningful patterns among the DNA or protein sequences. If biologists hope to unlock the potential of sequential pattern mining in their field, it is necessary to move away from traditional sequential pattern mining algorithms, because they have difficulty handling a small number of items and long sequences in biological data, such as gene and protein sequences. To address the problem, we propose an approach called Depth-First SPelling (DFSP) algorithm for mining sequential patterns in biological sequences. The algorithm’s processing speed is faster than that of PrefixSpan, its leading competitor, and it is superior to other sequential pattern mining algorithms for biological sequences.

Incremental Mining of Across-streams Sequential Patterns in Multiple Data Streams

Sequential pattern mining is the mining of data sequences for frequent sequential patterns with time sequence, which has a wide application. Data streams are streams of data that arrive at high speed. Due to the limitation of memory capacity and the need of real-time mining, the results of mining need to be updated in real time. Multiple data streams are the simultaneous arrival of a plurality of data streams, for which a much larger amount of data needs to be processed. Due to the inapplicability of traditional sequential pattern mining techniques, sequential pattern mining in multiple data streams has become an important research issue. Previous research can only handle a single item at a time and hence is incapable of coping with the changing environment of multiple data streams. In this paper, therefore, we propose the IAspam algorithm that not only can handle a set of items at a time but also can incrementally mine across-streams sequential patterns. In the process, stream data are converted into bitmap representation for mining. Experimental results show that the IAspam algorithm is effective in execution time when processing large amounts of stream data.

Mining sequential patterns across multiple sequence databases

In this paper, given a set of sequence databases across multiple domains, we aim at mining multi-domain sequential patterns, where a multi-domain sequential pattern is a sequence of events whose occurrence time is within a pre-defined time window. We first propose algorithm Naive in which multiple sequence databases are joined as one sequence database for utilizing traditional sequential pattern mining algorithms (e.g., PrefixSpan). Due to the nature of join operations, algorithm Naive is costly and is developed for comparison purposes. Thus, we propose two algorithms without any join operations for mining multi-domain sequential patterns. Explicitly, algorithm IndividualMine derives sequential patterns in each domain and then iteratively combines sequential patterns among sequence databases of multiple domains to derive candidate multi-domain sequential patterns. However, not all sequential patterns mined in the sequence database of each domain are able to form multi-domain sequential patterns. To avoid the mining cost incurred in algorithm IndividualMine, algorithm PropagatedMine is developed. Algorithm PropagatedMine first performs one sequential pattern mining from one sequence database. In light of sequential patterns mined, algorithm PropagatedMine propagates sequential patterns mined to other sequence databases. Furthermore, sequential patterns mined are represented as a lattice structure for further reducing the number of sequential patterns to be propagated. In addition, we develop some mechanisms to allow some empty sets in multi-domain sequential patterns. Performance of the proposed algorithms is comparatively analyzed and sensitivity analysis is conducted. Experimental results show that by exploring propagation and lattice structures, algorithm PropagatedMine outperforms algorithm IndividualMine in terms of efficiency (i.e., the execution time).

Constraint-based sequential pattern mining: The consideration of recency and compactness

Sequential pattern mining is an important data-mining method for determining time-related behavior in sequence databases. The information obtained from sequential pattern mining can be used in marketing, medical records, sales analysis, and so on. Existing methods only focus on the concept of frequency because of the assumption that sequences' behaviors do not change over time. The environment from which the data is generated is often dynamic, however, so the sequences' behaviors may change over time. To adapt the discovered patterns to these changes, two new concepts, recency and compactness, are incorporated into traditional sequential pattern mining. The concept of recency causes patterns to quickly adapt to the latest behaviors in sequence databases, while the concept of compactness ensures reasonable time spans for the discovered patterns. We named the new patterns CFR-patterns because three concepts (compactness, frequency, and recency) are simultaneously considered. An efficient method is presented to find CFR-patterns. Empirical evaluation shows that the proposed methods are computationally efficient and that they are more advantageous than traditional methods when sequences' behaviors change over time.