Distributed Data Mining Research Articles

Communication-efficient distributed mining of association rules

Mining for associations between items in large transactional databases is a central problem in the field of knowledge discovery. When the database is partitioned among several share-nothing machines, the problem can be addressed using distributed data mining algorithms. One such algorithm, called CD, was proposed by Agrawal and Shafer in [1] and was later enhanced by the FDM algorithm of Cheung, Han et al. [5]. The main problem with these algorithms is that they do not scale well with the number of partitions. They are thus impractical for use in modern distributed environments such as peer-to-peer systems, in which hundreds or thousands of computers may interact. In this paper we present a set of new algorithms that solve the Distributed Association Rule Mining problem using far less communication. In addition to being very efficient, the new algorithms are also extremely robust. Unlike existing algorithms, they continue to be efficient even when the data is skewed or the partition sizes are imbalanced. We present both experimental and theoretical results concerning the behavior of these algorithms and explain how they can be implemented in different settings.

Application of Parallel and Distributed Data Mining in e-Commerce

e-Commerce and Data Mining are inseparable. But Data Mining is a costly affair. So parallelization becomes inevitable. Also the web logs to be mined for revenue enhancement are distributed in nature. That brings in the distributed data mining. Here we discuss the sequential discovery of association rules first. Then we go on to see how that could be made parallel. We also see the application of the parallel and distributed approaches of data mining onto the web logs that are available due to the transactions taking place on the web.

10.1109/DSDE.2010.25

Data mining algorithm in electronic commerce environment must work. In fact, few analysts would disagree with the synthesis of distributed data mining algorithm in electronic commerce environment. Our focus in this position paper is not on whether A* search algorithm are continuously incompatible, but rather on introducing an distributed data mining algorithm (WaryDives) in electronic commerce environment.

Distributed data mining in credit card fraud detection

Credit card transactions continue to grow in number, taking an ever-larger share of the US payment system and leading to a higher rate of stolen account numbers and subsequent losses by banks. Improved fraud detection thus has become essential to maintain the viability of the US payment system. Banks have used early fraud warning systems for some years. Large scale data-mining techniques can improve the state of the art in commercial practice. Scalable techniques to analyze massive amounts of transaction data that efficiently compute fraud detectors in a timely manner is an important problem, especially for e-commerce. Besides scalability and efficiency, the fraud-detection task exhibits technical problems that include skewed distributions of training data and nonuniform per error, both of which have not been widely studied in the knowledge-discovery and data mining community. In this article, we survey and evaluate a number of techniques that address these three main issues concurrently. Our proposed methods of combining multiple learned fraud detectors under a cost model are general and demonstrably useful; our empirical results demonstrate that we can significantly reduce loss due to fraud through distributed data mining of fraud models.

The Distributed Data-Mining Worksho

The Workshop on Distributed Data Mining (DDM), held at the Fourth International Conference on Knowledge Discovery and Data Mining, brought interested researchers and practitioners together and created an environment for crystallizing the fast-growing field of DDM. The workshop focused on the state-of-the-art DDM algorithms, systems, and application-related issues.

A Distributed Framework for Parallel Data Mining Using HPJava

Java has become a language of choice for applications executing in heterogeneous environments utilising distributed objects and multithreading. To handle large data sets, scalable and efficient implementations of data mining approaches are required, generally employing computationally intensive algorithms. Conventional Java implementations do not directly provide support for the data structures often encountered in such algorithms, and they also lack repeatability in numerical precision across platforms. This paper describes a distributed framework employing task and data parallelism, and implemented in high performance Java (HPJava). Issues of interest for data mining algorithms are identified, and possible solutions discussed for overcoming limitations in the Java Virtual Machine. The framework supports parallelism across workstation clusters, using the message-passing interface as middleware, and can support different analysis algorithms, wrapped as Java objects, and linked to various databases using the Java database connectivity interface. Guidelines are provided for implementing parallel and distributed data mining on large data sets, and a proof-of-concept data mining application is analysed using a neural network.