Market Basket Dataset Research Articles

On the big data processing algorithms for finding frequent sequences

AbstractSequential pattern mining algorithms extract trendy sequence appearances inside ordered transactional datasets such as market basket datasets. There is a lack of research employing big data processing techniques to locate frequent sequences on large‐scale datasets. Furthermore, there is a need for optimized sequential pattern mining algorithms that run on ordered one‐dimensional sequences. We also observe a lack of sequential pattern search studies in the literature, where the focus is centered around multi‐dimensional data sequences. Existing approaches that deal with ordered one‐dimensional datasets suffer from scalability issues as the amount of data to be analyzed is enormous. This research investigates the big data processing techniques used to find frequent sequences in large‐scale datasets. It also proposes a scalable sequence pattern mining algorithm called Sequential Pattern Acquisition by Reducing Search Space (SPARSS) designed for distributed data processing systems that efficiently handle large datasets containing sequential one‐element data. It introduces a prototype implementation of SPARSS and provides information on the SPARSS's memory and time requirements, which were calculated as part of experimental studies on a real‐world dataset. The results confirm our expectations and demonstrate SPARSS's superior scalability and run‐time efficiency compared to other distributed algorithms.

On Followers Search

Although followership has been widely studied in sociology and management, the problem of finding followers has not drawn attention in the field of artificial intelligence. We refer to the problem of finding followers of a given object as followers search. In sociology, followers are close to their leaders, and leaders are superior to their followers. In this article, aimed at finding followers of a given object, we formulate followers on the basis of both superiority and closeness. The former means that the given object should be superior to followers, and the latter means that followers should be close to the given object. We present a followers search algorithm to find the followers of the given object. Furthermore, we apply the ideas of followers to the market basket and recommender system datasets. The experimental results demonstrate the rationality of the discovered followers on the market basket dataset and the improved performance of the TrustPMF algorithm by adopting followership on recommender system datasets, which indicate a promising future for followers search.

Mining the Local Dependency Itemset in a Products Network

Many studies have been conducted on market basket analysis such as association rules and dependent patterns. These studies mainly focus on mining all significant patterns or patterns directly associated with a given item in a dataset. The problem that has not been addressed is how to mine patterns associated with a given item from the local view. This problem becomes very meaningful when the market basket dataset is huge. To address this problem, in this study, first, a new idea called “local dependency itemset” is put forward, which refers to patterns associated with the given item. Second, a framework of mining the local dependency itemset is presented. The framework has two steps, which are executed iteratively. One is expanding the local dependency itemset that initially consists of only the given item; the other is updating the local products network. Third, this framework is implemented by three different dependence indicators and a typical local community detection algorithm. The experimental results confirm that the local dependency itemset is meaningful.

Inventory prediction and management in Nigeria using market basket analysis associative rule mining: memetic algorithm based approach

<p class="Text">A key challenge in businesses today is determining inventory level for each product (to be) sold to clients. A pre-knowledge will suppress inventory stock-up and help avert unnecessary demurrage. It will also avoid stock out and avert loss of clients to competition. Study aims to unveil customer’s behavior in purchasing goods and thus, predict a next time purchase as well as serve as decision support to determine the required amount of each goods inventory. Study is conducted for Delta Mall (Asaba and Warri branches) department store. We adapt the memetic algorithm on market basket dataset to examine buying behavior of customers, their preference and frequency at which goods are purchased in common (basket). Result shows some items placed in basket allow customers to purchase items of similar value, or best combined with the selected items due to shelf-placement via concept of feature drift. Model yields 21-rules for eight items obtained from data transaction mining dataset acquired from Delta Mall.</p>

Analysis and Evaluation of Schemes for Secure Sum in Collaborative Frequent Itemset Mining across Horizontally Partitioned Data

Privacy preservation while undertaking collaborative distributed frequent itemset mining (PPDFIM) is an important research direction. The current state of the art for privacy preservation in distributed frequent itemset mining for secure sum in a horizontally partitioned data model comprises primarily public key based homomorphic schemes which are expensive in terms of the communication and computation cost. The nonpublic key based existing state-of-the-art scheme by Clifton et al. used for secure sum in PPDFIM is efficient but prone to security attacks. In this paper, we propose Shamir’s secret sharing based approaches and a symmetric key based scheme to calculate the secure sum in PPDFIM. These schemes are information theoretically secure under the standard assumptions. We further give a detailed theoretical and empirical evaluation of our proposed schemes for PPDFIM using a real market basket dataset. Our experimental analysis also shows that our schemes perform better in terms of the execution cost compared to the public key based scheme for secure sum in PPDFIM.

THE TABU SEARCH APPLICATION: AN APPROACH TO MINE MARKET BASKET DATA

ABSTRACT The economic value of data mining is well established. Market basket analysis is one of the most well-known data mining techniques that is used to discover the customer purchase patterns. However, as the size of the dataset increases, the mining process becomes increasingly cumbersome and time consuming. In this paper, we propose to apply Tabu search metaheuristic (TS) to find global optimal solutions for large scale market basket datasets. In preparation for Tabu search, a market basket dataset is converted to a graph-based network. The result of an experiment with a large scale supermarket dataset show that the proposed TS algorithm produces highly accurate solutions in reduced computational times comparing to the traditional algorithm used by SAS Enterprise Miner. Keywords Market basket, Tabu search, Meta-heuristic

A New Approach on Rare Association Rule Mining

Association rule mining is the process of finding some relations among the attributes/attribute values of huge database based on support value. Most existing association mining techniques are developed to generate frequent rules based on frequent itemsets generated on market basket datasets. A common property of these techniques is that they extract frequent itemsets and prune the infrequent itemsets. However, such infrequent or rare itemsets and consequently the rare rules may provide valuable information. So, many applications demand to mine such rare association rules which have low support but higher confidence. This paper presents a method to generate both frequent and rare itemsets and consequently the rules. The effectiveness of the rules has been validated over several real life datasets.

A parsimonious multivariate poisson model for market basket analysis

Intensified research on multivariate Poisson models offers new opportunities for the analysis of purchase quantities in market basket data. The investigation of positive or negative correlations in quantity decisions among product categories facilitates a deeper understanding of consumer purchase behavior. The applied multivariate log-normal Poisson model introduces interdependencies between categories with multivariate normal-distributed latent effects by means of a covariance matrix. As the size of this covariance matrix depends on the number of categories in the model, its ation may become tedious. Furthermore, we assume that quantity decisions do not interact for all pairs of categories. That is why we propose to use covariance selection to derive a parsimonious representation of the correlation structure. For two market basket data sets, we show that the vast majority of off-diagonal elements in the covariance matrix are irrelevant. For a data set with product categories, the model with a partly restricted covariance matrix achieves a better fit to the holdout data than the model with full covariance matrix. For a data set with subcategories of the broader category beverage, the proposed model with restricted covariance outperforms the model with full covariance matrix even on the calibration data. We conclude that interactions of quantity decisions are overall the exception, even for complements-in-use.

A new heuristic for learning Bayesian networks from limited datasets: a real-time recommendation system application with RFID systems in grocery stores

Bayesian networks (BNs) are a useful tool for applications where dynamic decision-making is involved. However, it is not easy to learn the structure and conditional probability tables of BNs from small datasets. There are many algorithms and heuristics for learning BNs from sparse datasets, but most of these are not concerned with the quality of the learned network in the context of a specific application. In this research, we develop a new heuristic on how to build BNs from sparse datasets in the context of its performance in a real-time recommendation system. This new heuristic is demonstrated using a market basket dataset and a real-time recommendation model where all items in the grocery store are RFID tagged and the carts are equipped with an RFID scanner. With this recommendation model, retailers are able to do real-time recommendations to customers based on the products placed in cart during a shopping event.

Parallel hierarchical clustering using weighted confidence affinity

There have been many attempts for clustering categorical data such as market basket dataset. However, most of categorical clustering approaches belong to partitional clustering which requires at least one input parameter (e.g., the minimum intra-cluster similarity or the desired number of clusters). In this paper, we propose a parallelised hierarchical clustering approach for categorical data (PH-clustering) using vertical data structures. In order to minimise the impact of low support items, we devise a weighted confidence (WC) affinity function to compute the similarity between clusters. Based on our analysis of the major clustering steps, we adopt a partial local and partial global approach to reduce computation time as well as to keep network communication at minimum. Load balance issues are addressed especially during the data partitioning phase. Our experimental results on standardised market basket data show that the proposed weighted confidence affinity measure is more accurate than other contemporary affinity measures in the literature and that our parallel clustering approach provides magnitudes of time improvements over sequential clustering especially over larger data sizes. Our results also indicate that the number of items/attributes in the dataset has a more drastic impact on performance than the number of transactions/tuples.

Techniques of acceleration for association rule induction with pseudo‐artificial life algorithm

AbstractFrequent pattern mining is an important problem in data mining. Generally, the number of potential rules grows rapidly as the size of the database increases. It is therefore hard for a user to extract association rules. To avoid such a difficulty, we propose a new method for association rule induction using a pseudo‐artificial life approach. The proposed method is to decide whether there exists an item set which contains N or more items in two transactions. If it exists, a series of item sets that are contained in the part of the transactions will be recorded. The iteration of this step contributes to the extraction of association rules. It is not necessary to calculate a huge number of candidate rules. In an evaluation test, the authors compared the extracted association rules using our method with rules obtained with other algorithms such as the a priori algorithm. In an evaluation using a huge retail market basket data set, our method was found to be approximately 10 to 20 times faster than the a priori algorithm and its variants. © 2010 Wiley Periodicals, Inc. Electron Comm Jpn, 93(2): 1–11, 2010; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecj.10225

Finding calendar-based periodic patterns

Mining patterns in a market-basket dataset is a well-stated problem. There are a number of approaches to deal with this problem. Different types of patterns may be present in a dataset. An interesting one is patterns that hold seasonally, which are called calendar-based patterns. Earlier methods require periods to be specified by the user. We present here a method which is able to extract different types of periodic patterns that may exist in a temporal market-basket dataset and it is not needed for the user to specify the periods in advance. We consider the time-stamps as a hierarchical data structure and then extract different types of patterns. The algorithm can detect both wholly and partially periodic patterns. Although we have applied our approach to market-basket dataset, the approach can be used for any event related dataset where the events are associated with time intervals.

A Projection Pursuit Method on the multidimensional squared Contingency Table

In this study a projection pursuit method is used to explore c d (square) contingency table data. The method operates on projection matrices constructed from the contingency tables using affine geometry and creates projections (or marginals) using a Radon transform. The projection matrices and the projections can be used to find the "interesting" (nonuniform structure), and to cluster and to order the, cases. This projection pursuit method is implemented with graph visualization of projection. It is similar to the discrete version of Andrews' curve. We demonstrate how this approach compares to association rules commonly used in data mining using a market basket data set and compare the PP results with the analysis of a data set from Wishart and Leach) (1970).