Pattern Growth Algorithm Research Articles

A heuristic approach for load balancing the FP-growth algorithm on MapReduce

Abstract Frequent itemset discovery is an important step in Association Rule Mining. The Frequent Pattern (FP) growth algorithm, often used for discovering frequent itemsets, cannot scale directly to today’s Big Data, especially for large sparse datasets. Hence there is a need to distribute and parallelize the FP-growth algorithm. Parallel FP-growth (PFP) is a parallel implementation of the FP-growth algorithm on Hadoop’s MapReduce execution framework. Though PFP scales to large datasets, it suffers from imbalanced load across processing units. In this paper we propose a heuristic based, lower order of complexity, load balancing strategy for the PFP algorithm, called Heuristic Based PFP (HBPFP). Our results show that HBPFP distributes the load more evenly across the Hadoop cluster nodes, runs faster than the PFP algorithm, and uses cluster resources more efficiently, especially for large sparse datasets.

Association rule mining for the ordered placement of traditional Chinese medicine containers: An experimental study.

In traditional Chinese medicine (TCM) clinics, the pharmacists responsible for dispensing the herbal medicine usually find the desired ingredients based on positions of the shelves (racks; frames; stands). Generally, these containers are arranged in an alphabetical order depending on the herbal medicine they contain. However, certain related ingredients tend to be used together in many prescriptions, even though the containers may be stored far away from each other. This can cause problems, especially when there are many patients and/or the limited number of pharmacists. If the dispensing time takes longer, it is likely to impact the satisfaction of the patients’ experience. Moreover, the stamina of the pharmacists will be consumed quickly.In this study, we investigate on an association rule mining technology to improve efficiency in TCM dispensing based on the frequent pattern growth algorithm and try to identify which 2 or 3 herbal medicines will match together frequently in prescriptions. Furthermore, 3 experimental studies are conducted based on a dataset collected from a traditional Chinese medicine hospital. The dataset includes information for an entire year (2014), including 4 seasons and doctors. Afterward, a questionnaire on the usefulness of the extracted rules was administered to the pharmacists in the case hospital. The responses showed the mining results to be very valuable as a reference for the placement and ordering of the frames in the TCM pharmacies and drug stores.

Optimizing the Feature Selection Process for Better Accuracy in Datasets with a Large Number of Features (Student Abstract)

Most feature selection methods only perform well on datasets with relatively small set of features. In the case of large feature sets and small number of data points, almost none of the existing feature selection methods help in achieving high accuracy. This paper proposes a novel approach to optimize the feature selection process through Frequent Pattern Growth algorithm to find sets of features that appear frequently among the top features selected by the main feature selection methods. Our experimental evaluation on two datasets containing a small and very large number of features shows that our approach significantly improves the accuracy results of the dataset with a very large number of features.

PENERAPAN DATA MINING UNTUK ANALISIS MARKET BASKET DENGAN ALGORITMA FP-GROWTH PADA PD PASAR TOHAGA

Sales data in 3 different shops (shop, Shop Maker Fernando and Son) at Tohaga Market in the form of PD book transactions are only seen in the absence of follow-up to determine the decision on who will come. Party owner only records the transactions of products sold and only see income per month. But with that data should be utilized to strategize on sales to come. By using the method of Frequent Pattern Growth Algorithm, the store can take decisions which require goods inventory more compared to other goods, and the placement of the goods in accordance with the relationship between the goods that are usually purchased a consumer can also be determined based on a Minimum Support and Minimum Confidence. Based on Market Basket Analysis obtained from the calculation of the Association by using the method of Frequent Pattern Growth Algorithm, then search for the value of the support and confidence to use Association Rules, Rules that are generated will be test by using Software RapidMiner. Then the placement of goods and inventory items in 3 different stores can be controlled with either the service so that the consumer will be increased, which in turn can increase the sales turnover. In this study Support is determined using threshold 40% and 83% Confidence. Having regard to the relationship of support and confidence the store owner can provide and put the items to be sold

Diagnosis of Type II Diabetes Based on Feed forward Neural Network Techniques

Diabetes is a disease caused by an increase in blood glucose levels due to insulin secretion deficiency (type I diabetes) or impaired insulin activity (type II diabetes). More than 90% of people with this condition are diagnosed with type II diabetes. Due to the sharply prevalence of type 2 diabetes in recent years, the prognosis and early diagnosis of the disease have become even more important. In this study, a model for diagnosis of type II diabetes was developed using Artificial Neural Network (ANN) method. The execution of the Frequent Pattern Growth algorithm on medical data is difficult. Association rule-based classification is an interesting area focused that can be utilized for early diagnosis. The discretization phase is necessary to transform numerical characteristics. Pima Indians Diabetes Data Set is taken as an input. The execution time, a number of rules generation and the detection of outlier percentage are analyzed. The CFP-growth algorithm utilizes for finding frequent patterns where constructing the Minimum Item Support (MIS)-tree, CFP-array and producing frequent patterns from the MIS-tree. From the set of frequent itemsets found, create all the association rules that have confidence exceeding the minimum confidence. In this study, we aim to build a model that helps Physicians in predicting Diabetes early and accurately. Data were collected from the PIMA Indian data set. Consisted of 768 samples (268 diabetic and 500 non-diabetics). Levenberg-Marquardt back-propagation algorithm was used to train the network, and the accuracy of the prediction of whether a person is diabetics or not was 88.8%.

RETRACTED ARTICLE: An improved memory adaptive up-growth to mine high utility itemsets from large transaction databases

High utility itemset (HUI) mining identifies an interesting pattern from transactional databases by taking into consider the utility of each in the transaction for instance, margins or profits. Many candidates are generated for HUIs which reduces the mining performance. Frequent pattern-growth (FP-Growth) algorithm was widely used to discover the frequent itemsets using FP-Tree. But, UP-Tree was constructed to store high utility item set. The mining of UP-Tree by FP-Growth extracts high utility itemset and generates too many candidates. So, UP-Growth and UP-Growth+ was proposed to shorten the candidate itemsets. In UP-Growth, two tactics such as discarding local unpromising items (DLU) and decreasing local node (DLN) were used in FP-Growth. In UP-Growth+, two strategies such as DLU and its estimated node utilities (DNU) and DLN tools for the nodes of internal UP-Tree by evaluated the descendant nodes (DNN) were incorporated in FP-Growth. However, the UP-Growth and UP-Growth+ has the problem of poor spatial and temporal localities. Initially, the UP-Tree is created in available main memory then extended to secondary memory when the transaction is large. The storage of data structure in secondary memory and accessibility is not clearly derived in existing algorithms. In this paper, the spatial locality issues of UP-Growth and UP-Growth+ is solved by rearranging nodes of UP-Tree in a depth first manner and temporal locality issues of UP-Growth and UP-Growth+ is solved by page blocking technique which reorganizes the execution part of UP-Growth and UP-Growth+. The computational part is rearranged. In addition to, the memory management strategy is introduced to minimize the space requirement of high utility itemsets. Thus the proposed Improved Adaptive UP-Growth (IAUP-Growth) and Improved Adaptive UP-Growth+ (IAUP-Growth+) overcomes the spatial and temporal locality problem and effectively reduces the memory usage.

Efficient Discovery of Weighted Frequent Neighborhood Itemsets in Very Large Spatiotemporal Databases

Weighted Frequent Itemset (WFI) mining is an important model in data mining. It aims to discover all itemsets whose weighted sum in a transactional database is no less than the user-specified threshold value. Most previous works focused on finding WFIs in a transactional database and did not recognize the spatiotemporal characteristics of an item within the data. This paper proposes a more flexible model of Weighted Frequent Neighborhood Itemsets (WFNI) that may exist in a spatiotemporal database. The recommended patterns may be found very useful in many real-world applications. For instance, an WFNI generated from an air pollution database indicates a geographical region where people have been exposed to high levels of an air pollutant, say $PM2.5$ . The generated WFNIs do not satisfy the anti-monotonic property. Two new measures have been presented to effectively reduce the search space and the computational cost of finding the desired patterns. A pattern-growth algorithm, called Spatial Weighted Frequent Pattern-growth , has also been presented to find all WFNIs in a spatiotemporal database. Experimental results demonstrate that the proposed algorithm is efficient. We also describe a case study in which our model has been used to find useful information in air pollution database.

Association Rule Mining Method Based on the Similarity Metric of Tuple-Relation in Indoor Environment

Association rules can detect the association pattern between POIs (point of interest) and serve the application of indoor location. In this paper, a new index, tuple-relation, is defined, which reflects the association strength between POI sets in indoor environment. This index considers the potential association information such as spatial and semantic information between indoor POI sets. On this basis, a new R-FP-growth (tuple-relation frequent pattern growth) algorithm for mining association rules in indoor environment is proposed, which makes comprehensive use of the co-occurrence probability, conditional probability, and multiple potential association information among POI sets, to form a new support-confidence-relation constraint framework and to improve the quality and application value of mining results. Experiments are performed, using real Wi-Fi positioning trajectory data from a shopping mall. Experimental results show that the tuple-relation calculation method based on cosine similarity has the best effect, with an accuracy of 87%, and 19% higher than that of the traditional FP-growth algorithm.

Badminton player scouting analysis using Frequent Pattern growth (FP-growth) algorithm

FP Growth algorithm is widely used to analyze patterns from a huge amount of data with (frequent) repeated items. The objective of this research is to analyze playing pattern a badminton player, one of a popular sport in Indonesia. The data set was generated from a technical stroke during the game. The model used in this study was Jonathan Christie a top Indonesian badminton player. The method of data collection was done by dividing the playing field into various areas of the game. Observations were made by using the software, to calculate and classify the types of stroke that carried out by the athlete. The result of this research; the tactical approach of Jonathan Christie during this match was described. The data obtained would be very useful for the coach to improve the athlete’s performance. Another advantage obtained was the analysis of the athlete’s performance can be done with a quantitative approach so that it can enrich the current methods. As the conclusion, the FP Growth algorithms were able to describe the game pattern of a badminton athlete, JC by using PHP and MySQL. Sport science has become a necessity to develop to increase athletes’ competitiveness.

An Improved Frequent Pattern Growth Based Approach to Intrusion Detection System Alert Aggregation

This paper introduces different approaches to intrusion detection system (IDS) alert aggregation and proposes an improved frequent pattern growth (FP-growth) algorithm for it. This approach can be divided into three parts, which are removal of noisy data, mining association rules and text similarity check. According to the experiment on Snort alarm dataset provided by an enterprise, all the association rules found by the proposed approach are valid. Therefore, compared with FP-growth algorithm, the proposed approach can increase the precision of the result and is useful for alert aggregation.

Association rules mining in parallel conditional tree based on grid computing inspired partition algorithm

Association rules have important applications in many fields, however, with the explosive growth of information technology in recent years, the mining efficiency of association rules has become a very serious problem. The parallel multi-swarm PSO frequent pattern (PMSPF) algorithm creatively combines the particle swarm optimisation (PSO) algorithm with the frequent pattern-growth (FP-growth) algorithm to greatly improve the mining efficiency of association rules. However, under the computing environment of the Spark cluster, the calculation load is not balanced. Therefore, large amount of data may lead to problems like memory overflow. In this paper, parallel conditional frequent pattern (PCFP) tree algorithm is proposed on the basis of PMSPF. First of all, through data grouping, the problem of too large a data volume to construct FP-tree is solved. Then, through parallel strategy of the condition tree, parallel computing is implemented. The experimental results show that although PCFP algorithm generates certain data redundancy in the process of data grouping, the efficiency of the algorithm is significantly higher than that of the PMSPF algorithm and traditional parallel frequent pattern (PFP) algorithm.

SISTEM REKOMENDASI PAKET MINUMAN BERDASARKAN PESANAN PELANGGAN DENGAN METODE FREQUENT PATTERN GROWTH

It is proposed that a beverage package is determined based on customer orders by looking for different frequent itemset patterns in the transaction data that has occurred. These data can be analyzed to be more useful. The method used is the association method. With the predicted beverage package the company can increase sales. With so many transaction data stored, the difficulty in managing data requires a method called the association method. Frequent Pattern Growth Algorithm is an alternative algorithm that is quite effective for finding the set of data that most often appears (frequent itemset) in a large data set. The test results from the Frequent Pattern Growth method can determine a number of packages that meet the minimum value of support and confidence with a combination of two itemset

A Combined Horizontal Parallel Apriori Algorithm and Adaptive Frequent Pattern Growth Algorithm for Big Data Mining

Due to the massive data size and complexness, big data mining using a sole computer is a problematic task. With the rapid increase in the database size, parallel and distributed computing systems can yield better benefits in the data mining applications. Parallelization of the Association Rule Mining (ARM) algorithms is a significant task in the data mining application for effectively mining the frequent itemsets from the large-size databases. These mining algorithms allocate the database in a horizontal manner or increase the number of processors to decrease the overall time necessary for mining the frequent itemsets. In this paper, a combined Horizontal Parallel-Apriori (HP-Apriori) and Adaptive Frequent Pattern (FP) Growth algorithm is proposed to divide the database both horizontally and vertically into four sub-processes, for parallel processing of all four tasks. The Horizontal Parallel-Apriori algorithm increases the speed of the mining process using an index file. Adaptive Binomial Distribution (ABD) is applied to the Frequent Pattern Growth Algorithm to find the minimum support for mining the optimal frequent itemsets. Experimental analysis established that the combined algorithm outperforms in terms of minimizing the overall execution time and increasing the computational speed in high scalability.

Designing an Application for Analyzing Consumer Spending Patterns Using the Frequent Pattern Growth Algorithm

In this modern era, the market has been growing rapidly which can be seen from the navel shopping that is lined up in the hearts of big cities such as supermarkets, grocery stores and others that are provided to meet people's needs for primary goods that are always needed at all times. One of them is Giant Express Tamalanrea, a supermarket in the city of Makassar that serves the sale of household goods and general needs. With the use of customer data analysis to determine the customers' purchasing patterns, Giant Express can optimize the collation of goods, by positioning goods at closer shelves based on the level of frequency of goods purchased together by customers. Therefore, this study suggests the creation of an application to analyze consumer spending patterns using the frequent pattern growth algorithm method to ensure appropriate placement of goods to increase sales at Giant Express Tamalanrea. The purpose of this study is to develop an application that can analyze consumer spending patterns to increase sales by positioning goods based on consumer shopping patterns, as well as implementing the Frequent Pattern Growth Algorithm method to determine customer spending patterns to increase sales. Stages of research methods conducted begin with data collection at the study site, system requirements analysis, system design with UML, and system testing with the Black Box method.

Designing an Application for Analyzing Consumer Spending Patterns Using the Frequent Pattern Growth Algorithm

<p class="JGI-AbstractIsi">In this modern era, the market has been growing rapidly which can be seen from the navel shopping that is lined up in the hearts of big cities such as supermarkets, grocery stores and others that are provided to meet people's needs for primary goods that are always needed at all times. One of them is Giant Express Tamalanrea, a supermarket in the city of Makassar that serves the sale of household goods and general needs. With the use of customer data analysis to determine the customers' purchasing patterns, Giant Express can optimize the collation of goods, by positioning goods at closer shelves based on the level of frequency of goods purchased together by customers. Therefore, this study suggests the creation of an application to analyze consumer spending patterns using the frequent pattern growth algorithm method to ensure appropriate placement of goods to increase sales at Giant Express Tamalanrea. The purpose of this study is to develop an application that can analyze consumer spending patterns to increase sales by positioning goods based on consumer shopping patterns, as well as implementing the Frequent Pattern Growth Algorithm method to determine customer spending patterns to increase sales. Stages of research methods conducted begin with data collection at the study site, system requirements analysis, system design with UML, and system testing with the Black Box method.</p>

A new MapReduce solution for associative classification to handle scalability and skewness in vertical data structure

Associative classification is a promising methodology in information mining that uses the association rule discovery procedures to build the classifier. But they have some limitations like: they are not able to handle big data as they have memory constraints, high time complexity, load imbalance and data skewness. Data skewness occurs invariably when big data analytics comes in picture and affects the efficiency of an approach. This paper presents the MapReduce solution for associative classification in respect of vertical data layout. To handle these problems we have proposed two algorithms MR-MCAR-F (MapReduce-Multi Class Associative Classifier-MapReduce fast algorithm) and MR-MCAR-L (MapReduce-Multi Class Associative Classifier Load parallel frequent pattern growth algorithm). Also in this paper, MapReduce solution of Tid List and Database coverage has been proposed. We have used three type of pruning techniques viz. database coverage, global and distributed pruning. The proposed approaches have been compared with latest approach from the literature survey in terms of accuracy, computation time and data skewness. The existing scalable approaches cannot handle skewness while, our proposed method handles it in a very effective manner. All the experiments have been performed on six datasets which have been extracted from UCI repositories on the Hadoop framework. Proposed algorithms are scalable solutions for associative classification to handle big data and data skewness.

TEKNIK DATA MINING UNTUK PENENTUAN PAKET HEMAT SEMBAKO DAN KEBUTUHAN HARIAN DENGAN MENGGUNAKAN ALGORITMA FP-GROWTH (STUDI KASUS DI ULFAMART LUBUK ALUNG)

In a business, it takes effort to maximize profits. Among them by doing promotions. Minimarket Ulfamart has not done innovation in the form of promotion of frugal packages to customers. The precision of promotion can be learned from the database of a retail company primarily shopping patterns on products commonly purchased simultaneously. Information about inaccurate customer spending patterns results in inaccurate and efficient promotional policies. One of the most common attempts to acquire and explore customer spending patterns is to use data mining known as Knowledge Discovery in Database (KDD). One of the data mining techniques is the Association Rule which is a procedure in Market Basket Analysis. Market basketball is defined as an itemset purchased simultaneously by the customer in a transaction. Market basket analysis is a powerful tool for cross-selling strategies. A pattern is determined by two parameters, namely support (value of support) and confidence (value of certainty). The Frequent Pattern Growth (FP-Growth) algorithm is used to help find some association rules from the database by applying a Tree Tree structure or called FP-Tree. Implementation using RapidMiner to help find accurate patterns to get a combination of items that can be used as a frugal package.

Identifying obesity-related multimorbidity combinations in the United States.

Interest in understanding the effects of multimorbidity on outcomes has increased in recent years. This paper presents the most common obesity-related groupings of multimorbidity in the United States. Using Cerner HealthFacts data, we applied the frequent pattern growth algorithm to identify prevalent multimorbidity groupings of 3 or more diseases (one being obesity) by race using a dataset of 574 172 patients with obesity from all over the United States. We set the minimum prevalence to 10% and identified groupings of ICD10-CM diagnoses that occur in our dataset at or above the minimum prevalence level. We provide binomial proportion confidence interval estimates to demonstrate the validity of the proportions. We performed g-test for independence to validate differences in prevalence by race. We found 18 multimorbidity combinations with prevalence higher than or equal to 10%. Our results indicate that there are multiple common multimorbidities groupings for patients with obesity. Each multimorbidity combination is composed of diseases from the following clinical categories: endocrine, nutritional and metabolic diseases; diseases of the circulatory system; diseases of the digestive system; diseases of the nervous system; and diseases of the musculoskeletal system and connective tissue. For each multimorbidity pattern, the prevalence was found to be significantly different by race according to the g-test with P-value < .001. Most frequent patterns include essential hypertension or disorder of lipid metabolism. This study identifies common groupings of multimorbidity. We believe our data can be useful for those developing integrated care plans, particularly for those serving diverse communities.

Identifying the Underlying Relationship Between Water Quality Parameters of the Groundwater Samples using Association and Clustering Algorithms in Coimbatore District

Water is a highly complex environmental system; its protection cannot be met by traditional methods. As a part of the process, it is mandatory to evaluate the parameters of ground water so as to pursue suitable treatment. These days’ data mining algorithms have been developed to handle various data-rich environmental problems. In data mining, several techniques such as complex non-linear science, soft computing techniques, clustering and association have been applied in the domain of ground water quality assessment and evaluation in and around Coimbatore District. In this work, the statistical cluster analysis methods and association rule mining techniques were used to identify the spatial distribution of different cluster of wells having similar characteristics and determine the relationship between different water quality variables. The water quality assessment in Coimbatore was done using 13 parameters, namely NO3 - , TDS, Mg2+, Ca2+, Na+ , Cl- , F- , SO4 2- , EC, pH and Hardness including location in different sites. The main objective of the present study is to assess the performance of various clustering algorithms of WEKA and identify the most suitable algorithm for clustering water quality samples. K-Mean algorithm and centroid method of Hierarchical clustering performed in the similar manner in clustering. In addition to that, this study focused on identifying the water quality parameters exceeding permissible limits that occur together (TDS, Mg2+, SO4 2- , EC, hardness) in the given samples using Association Algorithms. The performance and efficiency of different association algorithms like Apriori and Frequent Pattern Growth algorithm was evaluated by factors like support, confidence, lift, leverage and conviction values.

A Unified Framework for Behaviour Monitoring and Abnormality Detection for Smart Home

With the revolution in smart infrastructure in the recent past, the smart healthcare system has been paid more considerable attention. The continuous upgradation of electricity meters to smart electricity devices has probed into a new market of intelligent data analysis services, providing aid to the health care systems. This paper presents a unified framework for extracting user behaviour patterns from home-based smart electricity meter data. The structure allows exploration and integration of frequent pattern growth algorithm for pattern mining and application of a variety of machine learning algorithms for categorizing the activities into manually labelled classes along with the implementation of Local Outlier Factor method for detection of an abnormal pattern of the inhabitant of smart homes. To evaluate the proposed framework, the work is implemented on the smart electricity dataset from the United Kingdom by separating the data into four distinct data files meant for the morning, afternoon, evening, and night energy utilization records. The results show a remarkable performance of Support Vector Machine (SVM) and Multilayer Perceptron (MLP) classifiers with kappa statics greater than 0.95 for all time slots data. The resultant frequent device utilization patterns with anomaly score more than the threshold value, reflecting abnormal activity patterns, are found more in evening time data in comparison to other time slots, requiring the immediate attention of concerned healthcare authorities.