Distributed Data Mining Approach Research Articles

An ensemble random forest algorithm for privacy preserving distributed medical data mining

As the voluminous amount of data is generated because of inexorably widespread proliferation of electronic data maintained using the Electronic Health Records (EHRs). Medical health facilities have great potential to discern the patterns from this data and utilize them in diagnosing a specific disease or predicting outbreak of an epidemic etc. This discern of patterns might reveal sensitive information about individuals and this information is vulnerable to misuse. This is, however, a challenging task to share such sensitive data as it compromises the privacy of patients. In this paper, a random forest-based distributed data mining approach is proposed. Performance of the proposed model is evaluated using accuracy, f-measure and appa statistics analysis. Experimental results reveal that the proposed model is efficient and scalable enough in both performance and accuracy within the imbalanced data and also in maintaining the privacy by sharing only useful healthcare knowledge in the form of local models without revealing and sharing of sensitive data.

An Ensemble Random Forest Algorithm for Privacy Preserving Distributed Medical Data Mining

As the voluminous amount of data is generated because of inexorably widespread proliferation of electronic data maintained using the Electronic Health Records (EHRs). Medical health facilities have great potential to discern the patterns from this data and utilize them in diagnosing a specific disease or predicting outbreak of an epidemic etc. This discern of patterns might reveal sensitive information about individuals and this information is vulnerable to misuse. This is, however, a challenging task to share such sensitive data as it compromises the privacy of patients. In this paper, a random forest-based distributed data mining approach is proposed. Performance of the proposed model is evaluated using accuracy, f-measure and appa statistics analysis. Experimental results reveal that the proposed model is efficient and scalable enough in both performance and accuracy within the imbalanced data and also in maintaining the privacy by sharing only useful healthcare knowledge in the form of local models without revealing and sharing of sensitive data.

Multi-Agent System Combined With Distributed Data Mining for Mutual Collaboration Classification

Distributed Data Mining (DDM) has been proposed as a means to deal with the analysis of distributed data, where DDM discovers patterns and implements prediction based on multiple distributed data sources. However, DDM faces several problems in terms of autonomy, privacy, performance and implementation. DDM requires homogeneity regarding environment, control, administration and the classification algorithm(s), and such that requirements are too strict and inflexible in many applications. In this paper, we propose the employment of a Multi-Agent System (MAS) to be combined with DDM (MAS-DDM). MAS is a mechanism for creating goal-oriented autonomous agents within shared environments with communication and coordination facilities. We shall show that MAS-DDM is both desirable and beneficial. In MAS-DDM, agents could communicate their beliefs (calculated classification) by covering private and non-sharable data, and other agents decide whether the use of such beliefs in classifying instances and adjusting their prior assumptions about each class of data. In MAS-DDM, we will develop and use a modified Naive Bayesian algorithm because (1) Naive Bayesian has been shown to be the most used algorithm to deal with uncertain data, and (2) to show that even if all agents in MAS-DDM use the same algorithm, MAS-DDM preforms better than DDM approaches with non-communicating processes. Point (2) provide an evidence that the exchange of information between agents helps in increasing the accuracy of the classification task significantly.

HDSM: A distributed data mining approach to classifying vertically distributed data streams

The rise in the Internet of Things (IoT) and other sensor networks has created many vertically-distributed and high-velocity data streams that require specialized algorithms for true distributed data mining. This paper proposes a novel Hierarchical Distributed Stream Miner (HDSM) that learns relationships between the features of separate data streams with minimal data transmission to central locations. Experimental evaluation demonstrates significant improvements in classification accuracy over previously proposed distributed stream-mining approaches while minimizing data transmission and computational costs. HDSM’s potential for dynamically trading off accuracy with computational resource costs is also demonstrated.

Multi-agent distributed data mining approach for classifying meteorology data: case study on Iran’s synoptic weather stations

Today’s organizations, industries and research centers are geographically distributed in different sites. To achieve true knowledge of business, mining massive amounts of data is necessary. In earth-related sciences such as meteorology, the date obtained from the various types of sensors is huge because of the high-frequency rate of data acquisition process and also the geographical distribution of weather stations. Therefore, the data mining and knowledge discovery process of this big and distributed data is a challenging work. In this paper, we propose a new distributed data mining approach called multi-agent hierarchical data mining to classify meteorology data, which has been collected from different sites widely distributed around the country (Iran). Our method utilizes a modified version of REPTree algorithm, which has been optimized to work in multi-agent system. To evaluate the proposed approach, it is implemented on 20 million of meteorology data record. Experimental results show that multi-agent hierarchical data mining approach can achieve significant performance improvement over centralized and parallel methods for knowledge discovery in large amounts of data.

Parallel and Distributed Code Clone Detection using Sequential Pattern Mining

This research presents a parallel and distributed data mining approach to code clone detection. It aims to prove the value and importance of deploying parallel and distributed computing for real-time large scale code clone detection. It is implemented this approach in a family of clone detectors, called PD EgyCD (Parallel and Distributed Egypt Clone Detector). In this approach, This research builds on an earlier work of the authors for code clone and plagiarism detection using sequential pattern mining by adding parallelism and distribution to our earlier tool EgyCD. Our approach uses data mining through a tailored Apriori-based algorithm for code clone detection. And it uses parallelization and distribution to achieve excellent performance to scale up to clone detection on very large systems. This approach has been implemented as a database application which leverages the capabilities of modern database tools. Two versions have been developed of this distributed technique. The first one uses client-server technique in which all clients and the server deal with only one database. The second one uses agents where each client acts as a separate agent and has its own database and after working on a sub-problem, it submits its partial solution to the server to finally get the complete solution (set of code clones). Experiments show that agents technique is faster than clientserver one. Distribution enhances performance very much. Speed improvement is a function of the number of clients/agents used. Our conclusion is that data mining, combined with parallel and distributed computing, can efficiently be deployed for code clone detection of very large systems.