Frequent Pattern Mining Methods Research Articles

Local Search for Efficient Causal Effect Estimation

Causal effect estimation from observational data is a challenging problem, especially with high dimensional data and in the presence of unobserved variables. The available data-driven methods for tackling the problem either provide an estimation of the bounds of a causal effect (i.e. nonunique estimation) or have low efficiency. The major hurdle for achieving high efficiency while trying to obtain unique and unbiased causal effect estimation is how to find a proper adjustment set for confounding control in a fast way, given the huge covariate space and considering unobserved variables. In this paper, we approach the problem as a local search task for finding valid adjustment sets in data. We establish the theorems to support the local search for adjustment sets, and we show that unique and unbiased estimation can be achieved from observational data even when there exist unobserved variables. We then propose a data-driven algorithm that is fast and consistent under mild assumptions. We also make use of a frequent pattern mining method to further speed up the search of minimal adjustment sets for causal effect estimation. Experiments conducted on extensive synthetic and real-world datasets demonstrate that the proposed algorithm outperforms the state-of-the-art criteria/estimators in both accuracy and time-efficiency.

Exploring Crowd Travel Demands Based on the Characteristics of Spatiotemporal Interaction between Urban Functional Zones

As a hot research topic in urban geography, spatiotemporal interaction analysis has been used to detect the hotspot mobility patterns of crowds and urban structures based on the origin-destination (OD) flow data, which provide useful information for urban planning and traffic management applications. However, existing methods mainly focus on the detection of explicit spatial interaction patterns (such as spatial flow clusters) in OD flow data, with less attention to the discovery of underlying crowd travel demands. Therefore, this paper proposes a framework to discover the crowd travel demands by associating the dynamic spatiotemporal interaction patterns and the contextual semantic features of the geographical environment. With urban functional zones (UFZs) as the basic units of human mobility in urban spaces, this paper gives a case study in Wuhan, China, to detect and interpret the human mobility patterns based on the characteristics of spatiotemporal interaction between UFZs. Firstly, we build the spatiotemporal interaction matrix based on the OD flows of different UFZs and analyze the characteristics of the interaction matrix. Then, hotspot poles, defined as the local areas where people gather significantly, are extracted using the Gi-statistic-based spatial hotspot detection algorithm. Next, we develop a frequent interaction pattern mining method to detect the frequent interaction patterns of the hotspot poles. Finally, based on the detected frequent interaction patterns, we discover the travel demands of crowds with semantic features of corresponding urban functional zones. The characteristics of crowd travel distance and travel time are further discussed. Experiments with floating car data, road networks, and POIs in Wuhan were conducted, and results show that the underlying travel demands can be better discovered and interpreted by the proposed framework and methods in this paper. This study helps to understand the characteristics of human movement and can provide support for applications such as urban planning and facility optimization.

Frequent pattern-based parallel search approach for time-dependent agile earth observation satellite scheduling

To address the time-dependent agile earth observation satellite (AEOS) scheduling problem more effectively, the frequent pattern-based parallel search (FPBPS) algorithm is proposed, which is composed of a parallel local search procedure, a competition-based algorithm and operator adaptive selection procedure and the new solutions construction method based on frequent pattern. First, the algorithm and the operator are selected from the algorithm pool and operator pool and run in parallel. As a result, some high-quality and diverse solutions are obtained in a short time. Second, we update the probability of each algorithm and operator to strengthen the self-adaptive ability of the algorithm. Third, frequent pattern mining method is used to extract knowledge with respect to AEOS scheduling to construct new solutions, and parallel local search is applied to further improve these solutions. Finally, extensive experiments prove that the FPBPS algorithm has a better performance than other comparison algorithms in the quality of solution, computation time, and robustness.

Study on behaviour anomaly detection method of English online learning based on feature extraction

There are many problems in abnormal detection of online English learning behaviour, such as large error and high detection time. Therefore, a detection method based on feature extraction is proposed. Firstly, frequent pattern mining method is used to collect learners' behaviour data, and the data is collected and preprocessed. Then, the classification constraints are set by support vector machine to complete the data classification. Finally, the sequence minimum eigenvalue method is used to train the abnormal data, extract the high frequency features of the abnormal data, establish the anomaly detection model, and realise the anomaly detection. Experimental results show that the highest detection error of this method is 1.2%, and the highest time cost is 1.8 s. Therefore, this method can effectively reduce the detection error and time cost, and is feasible.

MapReduce Framework to Improve the Efficiency of Large Scale Item Sets in IoT Using Parallel Mining of Representative Patterns in Big Data

With the coming of the time of huge information, individuals can gather rich and various information from a wide assortment of assortment gadgets, like those of the Internet of Things. Information concealed in enormous information is extremely helpful and important. Frequent pattern mining, as a basic method of data mining, is applied to every aspect of society. However, the application of traditional frequent pattern mining methods to big data involves bottlenecks due to the large number of result sets. Such bottlenecks make it challenging to deliver commonsense worth underway and life. We proposed a new approach which involved representative patterns using mining technique. This framework can make the runtime difficult to evaluate in large data environments. Our approach gives best solution for the above gaps with the help of online representative pattern-set parallel-mining algorithm. We used MapReduce framework which performs horizontal segmentation over data bases. Finally, several performance optimization strategies are proposed. As shown by numerous experiments on the actual dataset, the algorithm proposed in this paper improves the time efficiency by one order of magnitude. Several optimization strategies reduce the execution time to varying degrees.

Multi-Objective Optimal Travel Route Recommendation for Tourists by Improved Ant Colony Optimization Algorithm

With the swift development of tourism all around the world, it has become vital to improve the recommendation of useful travel information to tourists to assure their convenience and satisfaction. In this paper, we propose a novel multi-objective optimal travel route recommendation framework, which collects tourists’ travel trajectories from their mobile phone signaling data. Then, the proposed framework preprocesses the mobile signaling data to transform raw trajectories into tourists’ travel sequences. Subsequently, the framework finds the popular attractions and frequent travel routes from the travel pattern sequences by using a frequent pattern mining method. Finally, an improved ant colony optimization (ACO) algorithm with a novel extensible heuristic factor approach is adopted to search the multi-objective optimal travel routes according to the popularity of attractions and travel time of tourists. The experimental results indicate that the proposed framework is efficient in recommending multi-objective optimal travel routes considering tourists’ travel time and attractions’ popularity while ensuring that the recommended travel route is suitable.

Semantic Trajectory Frequent Pattern Mining Method with Fuzzy Stay Time Constraint

In the security system, transforming a large number of collected target trajectories into semantic trajectories with a less volume and high quality and mining their frequent patterns are helpful to analyze the target behavior patterns, identify hazard sources, and enhance the internal prevention, and control of the security system. Aiming at the limitation of semantic trace frequent pattern mining method defined by precise stay time in practical application scenarios, a fuzzy semantic trace frequent pattern mining method is proposed. Firstly, the membership function of fuzzy stay time is defined, so the stay time of the target at the stay point is fuzzified, and the fuzzy semantic trajectory is obtained. Then, a fuzzy semantic trajectory frequent pattern mining algorithm FST-FPM (fuzzy semantic trajectory frequent pattern mining) is proposed. The FST-FPM algorithm is experimentally verified on the Geolife public dataset and the self-collected RFID positioning dataset. The experimental results show that FST-FPM algorithm can mine frequent patterns of fuzzy semantic trajectories on Geolife dataset and RFID positioning dataset, and the running time is reduced by more than 10% compared with classical PrefixSpan algorithm, PrefixSpan-x algorithm, and LFFT2 algorithm.

Apriori-roaring: frequent pattern mining method based on compressed bitmaps

Apriori-roaring: frequent pattern mining method based on compressed bitmaps

Apriori-Roaring: Frequent Pattern Mining Method Based on Compressed Bitmaps

Apriori-Roaring: Frequent Pattern Mining Method Based on Compressed Bitmaps

Fuzzy Association Rules Mining based on Type-2 Fuzzy Sets over Data Stream

Fuzzy set theory aims to solve the problem of mining more valuable human reasoning knowledge from quantitative databases, and to narrow the gap between computer and human understanding. Fuzzy Frequent Itemset (FFI) mining, which is very important in fuzzy association rule mining, converts quantitative values into language, and finds fuzzy frequency itemset according to the defined membership function. In the traditional fuzzy frequent pattern mining method, only the language item with the largest scalar base is considered, and the uncertainty of the membership function is not considered. At the same time, it is mainly considered for the fixed data set. This paper proposes a Fuzzy Frequent Pattern mining algorithm based on the Type-2 Fuzzy Set (T2FS) theory of the data stream. Under the T2FS theory, the data stream is dynamically divided based on the sliding window method, and the ambiguity is quickly found from the numerical data stream. In order to reduce the search space and improve over-mining, this paper designs a dynamic compressed lists structure and trees method. Under different minimum support degrees and sliding window sizes, many experiments have been carried out, and the experiments have verified the efficiency and effectiveness of the designed method in terms of running time and memory usage. The results can also contribute to exchange the information and preferences among the people who cooperate in decision-making.

A bit-object-based method for mining maximum frequent patterns in intensive cloud computing data

In order to overcome the problems of poor timeliness and low accuracy of mining existing in traditional methods, this paper designs a bit-object based maximum frequent pattern mining method for intensive cloud computing data. After judging the support number according to the bit object of the maximum frequent pattern, the intensive cloud computing data is accurately collected according to the difference between the load value of cloud data and the true value of load, so as to improve the accuracy of subsequent mining results, and then the maximum frequent pattern of data is accurately mined by combining the bit object. Experimental results show that the maximum time to generate mining results is only 4.6 s, the maximum bit error rate of output results is only 7%, and the maximum memory occupancy is only 3.90%. The above results show that this method is more suitable for practical excavation.

Extracting Frequent Sequential Patterns of Forest Landscape Dynamics in Fenhe River Basin, Northern China, from Landsat Time Series to Evaluate Landscape Stability

The forest landscape pattern evolution can reveal the intensity and mode of action of human–land relationships at different times and in different spaces, providing scientific support for regional ecological security, human settlement health, and sustainable development. In this study, we proposed a novel method for analyzing the dynamics of landscape patterns. First, patch density (PD), largest patch index (LPI), landscape shape index (LSI), and contiguity index (CI) were used to identify the types of forest spatial patterns. The frequent sequential pattern mining method was used to detect the frequent subsequences from the time series of landscape pattern types from 1991 to 2020 and further evaluate the forest landscape stability of the Fenhe River Basin in China. The results show that different frequent sequence patterns have conspicuous spatial and temporal differences, which describe the evolution processes and stability changes during a certain period of forest evolution and play an important role in the analysis of forest dynamics. The proportion of the disturbed regions to the total forest area exhibited a downward trend. The long-term evolution pattern indicates that there are many evolution processes and trends in the forest at the same time, showing an aggregation distribution law. Compared with 2016, the forest landscape has become complete in 2020, and the overall stability of the Fenhe River Basin has improved. This study can provide scientific support to land managers and policy implementers and offer a new perspective for studying forest landscape pattern changes and evaluating landscape stability.

FACER: An API usage-based code-example recommender for opportunistic reuse

To save time, developers often search for code examples that implement their desired software features. Existing code search techniques typically focus on finding code snippets for a single given query, which means that developers need to perform a separate search for each desired functionality. In this paper, we propose FACER (F eature-driven A PI usage-based C ode E xamples R ecommender), a technique that avoids repeated searches through opportunistic reuse. Specifically, given the selected code snippet that matches the initial search query, FACER finds and suggests related code snippets that represent features that the developer may want to implement next. FACER first constructs a code fact repository by parsing the source code of open-source Java projects to obtain methods’ textual information, call graphs, and Application Programming Interface (API) usages. It then detects unique features by clustering methods based on similar API usages, where each cluster represents a feature or functionality. Finally, it detects frequently co-occurring features across projects using frequent pattern mining and recommends related methods from the mined patterns. To evaluate FACER, we run it on 120 Java Android apps from GitHub. We first manually validate that the detected method clusters represent methods with similar functionality. We then perform an automated evaluation to determine the best parameters (e.g., similarity threshold) for FACER. We recruit 10 professional developers along with 39 experienced students to judge FACER’s recommendation of related methods. Our results show that, on average, FACER’s recommendations are 80% precise. We also survey a total of 20 professional Android and Java developers to understand their code search and reuse experiences, and also to obtain their feedback on the usability and usefulness of FACER. The survey results show that 95% of our surveyed professional developers find the idea of related method recommendations useful during code reuse.

Genotype Pattern Mining for Pairs of Interacting Variants Underlying Digenic Traits.

Some genetic diseases (“digenic traits”) are due to the interaction between two DNA variants, which presumably reflects biochemical interactions. For example, certain forms of Retinitis Pigmentosa, a type of blindness, occur in the presence of two mutant variants, one each in the ROM1 and RDS genes, while the occurrence of only one such variant results in a normal phenotype. Detecting variant pairs underlying digenic traits by standard genetic methods is difficult and is downright impossible when individual variants alone have minimal effects. Frequent pattern mining (FPM) methods are known to detect patterns of items. We make use of FPM approaches to find pairs of genotypes (from different variants) that can discriminate between cases and controls. Our method is based on genotype patterns of length two, and permutation testing allows assigning p-values to genotype patterns, where the null hypothesis refers to equal pattern frequencies in cases and controls. We compare different interaction search approaches and their properties on the basis of published datasets. Our implementation of FPM to case-control studies is freely available.

An Improved KNN-Based Efficient Log Anomaly Detection Method with Automatically Labeled Samples

Logs that record system abnormal states (anomaly logs) can be regarded as outliers, and the k-Nearest Neighbor (kNN) algorithm has relatively high accuracy in outlier detection methods. Therefore, we use the kNN algorithm to detect anomalies in the log data. However, there are some problems when using the kNN algorithm to detect anomalies, three of which are: excessive vector dimension leads to inefficient kNN algorithm, unlabeled log data cannot support the kNN algorithm, and the imbalance of the number of log data distorts the classification decision of kNN algorithm. In order to solve these three problems, we propose an efficient log anomaly detection method based on an improved kNN algorithm with an automatically labeled sample set. This method first proposes a log parsing method based on N-gram and frequent pattern mining (FPM) method, which reduces the dimension of the log vector converted with Term frequency.Inverse Document Frequency (TF-IDF) technology. Then we use clustering and self-training method to get labeled log data sample set from historical logs automatically. Finally, we improve the kNN algorithm using average weighting technology, which improves the accuracy of the kNN algorithm on unbalanced samples. The method in this article is validated on six log datasets with different types.

A Vibrational Signal Fault Diagnosis Rule Extraction Method Based on DST‐ACI Discriminant Criterion

A fault diagnosis rule extraction method oriented to machine foot signal based on dynamic support threshold and association coefficient interestingness (DST‐ACI) discriminant criterion is proposed in this paper. The new method includes three main innovations. First, the feature state coding method based onK‐means clustering fully takes into account the imbalanced distribution of signal feature values due to the noise interference, and divide the signal feature values into several range intervals to generate the feature state code. Second, the frequent feature pattern mining method based on dynamic support threshold (DST) discriminant criterion can dynamically adjust support threshold according to the frequency of the feature states in each candidate pattern. Third, the fault diagnosis rule extraction method based on the association coefficient interestingness (ACI) discriminant criterion introduces a new metrics called ACI to evaluate the correlation between the pattern and the fault. Four types of fault simulation experiments were carried out, and the performance of the DST‐ACI method was tested using the collected vibration signal. The results show that compared with the coding method based on equal‐width discretization or equal‐density discretization, the accuracy of the transactional dataset generated by the feature state coding method based on K‐means clustering is higher. Compared with the frequent feature pattern mining method based on the constant support threshold criterion, the pattern mined by the DST‐based criterion has generally higher support. Compared with the existing confidence‐lift‐based and confidence‐improint‐based fault diagnosis rule extraction frameworks, the positive correlation between the feature states and the fault type of the rules extracted based on the DST‐ACI framework is generally stronger.

Research on Distributed Vertical Frequent Pattern Mining Method Based on Metadata Integration

In order to better integrate the information industry into people’s life and work, and even in social development, how to mine data has become a hot issue. Metadata is a kind of data about data. Mining metadata helps data application and storage. Therefore, it is very important to find an efficient and intelligent data mining method. This paper introduces the metadata and its integration technology. On this basis, it introduces the distributed vertical frequent mode, and introduces its use in the process of mining metadata, providing a new working idea for the staff engaged in related industries.

A new weighted based frequent and infrequent pattern mining method on realtime E-commerce

A new weighted based frequent and infrequent pattern mining method on realtime E-commerce

Mining top-k approximate closed patterns in an imprecise database

Over the last few years, the growth of data is exponential, leading to colossal amounts of information being produced by computational systems. Meanwhile, the data in real-life applications are usually incomplete and imprecise, which poses big challenges for researchers to obtain exact and valid analytical results with traditional frequent pattern mining methods. Since the potential faults can break the original characteristics of data patterns into multiple small fragments, it is impossible to recover the long true patterns from these fragments. To explore the huge amount of imprecise data by means of frequent pattern mining, we propose a service-oriented model that enables a new way of service provisioning based on users' QoS (quality of service) requirements. The novel model is developed to solve the problem of mining top-k approximate closed patterns in imprecise databases and will be further applied to diagnosis and treatment of potential patients in online medical applications. We test the novel model in an imprecise medical database and the experimental results show that the new model can successfully improve the health services for online customers.

An Efficient Parallel Mining Algorithm Representative Pattern Set of Large-Scale Itemsets in IoT

With the advent of the age of big data, people can collect rich and diverse data from a wide variety of collection devices, such as those of the Internet of Things. Knowledge hidden in large data is very useful and valuable. Frequent pattern mining, as a basic method of data mining, is applied to every aspect of society. However, the application of traditional frequent pattern mining methods to big data involves bottlenecks due to the large number of result sets. Such bottlenecks make it difficult to produce practical value in production and life. Therefore, mining representative pattern sets has been proposed. However, most existing algorithms select representative patterns after mining frequent pattern sets. This framework can make the runtime difficult to evaluate in large data environments. To solve the above-mentioned problems, this paper presents an online representative pattern-set parallel-mining algorithm. Within the parallel MapReduce framework, this algorithm uses horizontal segmentation to process the database and then applies the online mining algorithm to mine the locally represented pattern sets on each small database. Finally, several performance optimization strategies are proposed. As shown by numerous experiments on the actual dataset, the algorithm proposed in this paper improves the time efficiency by one order of magnitude. Several optimization strategies reduce the execution time to varying degrees.