Distributed Knowledge Discovery Research Articles

Data privacy-preserving distributed knowledge discovery based on the blockchain

Data are collected and regarded as valuable assets in many business domains. Their owner would not want to disclose them to the public due to their potential value. Distributed knowledge discovery techniques have been proposed which assume the cooperation of data owners even though they might not behave in a trustworthy manner. When a party decides to quit the cooperation in the distributed knowledge discovery, the other parties cannot continue the discovery task and hence they get some disadvantage due to the party???s betrayal. This paper is concerned with data privacy-preserving distributed knowledge discovery which gives penalty to the party who quits the cooperation in the discovery process. It proposes a blockchain-based distributed machine learning method which does not disclose the participating parties??? data and gives the penalty to betraying parties. The proposed method makes the participating parties communicate with each other via the smart contract on the blockchain network. It uses a blockchain-based incentive system to establish trust among parties and to improve the quality of discovery knowledge. The proposed method has been implemented with a smart contract on the blockchain and tested for a benchmark data.

EMGGR: an energy-efficient multipath grid-based geographic routing protocol for underwater wireless sensor networks

Routing in underwater wireless sensor networks (UWSN) is an important and a challenging activity due to the nature of acoustic channels and to the harsh environment. This paper extends our previous work [Al-Salti et al. in Proceedings of cyber-enabled distributed computing and knowledge discovery (CyberC), Shanghai, pp 331---336, 2014] that proposed a novel multipath grid-based geographical routing (MGGR) protocol for UWSNs. The extended work, EMGGR, viewed the network as logical 3D grids. Routing is performed in a grid-by-grid manner via gateways that use disjoint paths to relay data packets to the sink node. The algorithm consists of three main components: (1) a gateway election algorithm; responsible for electing gateways based on their locations and remaining energy level (2) a mechanism for updating neighboring gateways' information; allowing sensor nodes to memorize gateways in local and neighboring cells, and (3) a packet forwarding mechanism; in charge of constructing disjoint paths from source cells to destination cells, forwarding packets to the destination and dealing with holes (i.e. cells with no gateways) in the network. The performance of EMGGR has been assessed using Aqua-Sim, which is an NS2 based simulator for UWSNs. Results show that EMGGR is an energy efficient protocol in all simulation setups used in the study. Moreover, EMGGR can also maintain good delivery ratio and end-to-end delay.

A Random Decision Tree Framework for Privacy-Preserving Data Mining

Distributed data is ubiquitous in modern information driven applications. With multiple sources of data, the natural challenge is to determine how to collaborate effectively across proprietary organizational boundaries while maximizing the utility of collected information. Since using only local data gives suboptimal utility, techniques for privacy-preserving collaborative knowledge discovery must be developed. Existing cryptography-based work for privacy-preserving data mining is still too slow to be effective for large scale data sets to face today’s big data challenge. Previous work on random decision trees (RDT) shows that it is possible to generate equivalent and accurate models with much smaller cost. We exploit the fact that RDTs can naturally fit into a parallel and fully distributed architecture, and develop protocols to implement privacy-preserving RDTs that enable general and efficient distributed privacy-preserving knowledge discovery.

Fuzzy generalised classifier for distributed knowledge discovery

Classifier building form distributed data sources has been a fundamental computational problem to realise distributed knowledge discovery. The classification rule extraction from distributed databases suffers from the problems of high communication cost, lack of interpretability of rules and poor performance in handling high categorical data. The aim of this paper is to extend fuzzy generalised association rule extraction technique which is well proved in handling such issues to extract classification rules from distributed datasets. This paper presents a distributed data driven fuzzy generalised associative classifier D3FGAC framework for distributed knowledge discovery which extracts data driven fuzzy generalisation rules from horizontally fragmented datasets with minimum communication cost and builds global compact classifier using extracted rules. The experiments conducted on UCI datasets and their comparisons to other existing model shown in article to prove the efficiency of proposed framework.

Programming knowledge discovery workflows in service‐oriented distributed systems

SUMMARYIn several scientific and business domains, very large data repositories are generated. To find interesting and useful information in those repositories, efficient data mining techniques and knowledge discovery processes must be used. The exploitation of data mining techniques in science helps scientists in hypothesis formation and gives them a support on their scientific practices, whereas in industrial processes, data mining can exploit existing data sources as a real value for companies that can take advantage from the knowledge that can be extracted from their large data sources. Data mining tasks are often composed by multiple stages that may be linked to each other to form various execution flows. Moreover, data mining tasks are often distributed because they involve data and tools located over geographically distributed environments. Therefore, it is fundamental to exploit effective paradigms, such as services and workflows, to model data mining tasks that are both multi‐staged and distributed. This paper discusses data mining services and workflows for analyzing scientific data in high‐performance distributed environments such as Grids and Clouds. We discuss how it is possible to define basic and complex services for supporting distributed data mining tasks in Grids. We also present a workflow formalism and a service‐oriented programming framework, named DIS3GNO, for designing and running distributed knowledge discovery processes in the Knowledge Grid system. DIS3GNO supports all the phases of a knowledge discovery process, including composition, execution, and results visualization. After introducing DIS3GNO, some relevant use cases implemented by it and a performance evaluation of the system are discussed.Copyright © 2012 John Wiley & Sons, Ltd.

Privacy preserving distributed knowledge discovery: survey and future directions

The aim of privacy preserving data mining algorithms is to extract relevant knowledge from large amounts of data while protecting at the same time sensitive information. In the first part of this paper, we discuss privacy preserving distributed data mining techniques and provide a survey on the state–of–the–art methods in this field. We have considered randomisation, k–anonymisation, and cryptographic techniques. In the second part of the paper, we have described our proposed system for privacy preserving knowledge discovery over distributed databases which is still under development phase. The system is designed to perform local operations (local mining) in each site. This produces intermediate data that can be used to obtain the final result without revealing the private information of any site. Our proposal is mainly based on association rule mining and cryptographic techniques.

How distributed data mining tasks can thrive as knowledge services

Introduction Computer science applications are becoming more and more network centric, ubiquitous, knowledge intensive, and computing demanding. This trend will result soon in an ecosystem of pervasive applications and services that professionals and end-users can exploit everywhere. Recently, collections of IT services and applications, such as Web services and Cloud computing services, became available opening the way for accessing computing services as public utilities, like water, gas and electricity. Key technologies for implementing that perspective are Cloud computing and Web services, semantic Web and ontologies, pervasive computing, P2P systems, Grid computing, ambient intelligence architectures, data mining and knowledge discovery tools, Web 2.0 facilities, mashup tools, and decentralized programming models. In fact, it is mandatory to develop solutions that integrate some or many of those technologies to provide future knowledge-intensive software utilities. The Grid paradigm can represent a key component of the future Internet, a cyber infrastructure for efficiently supporting that scenario. Grid and Cloud computing are evolved models of distributed computing and parallel processing technologies. The Grid is a distributed computing infrastructure that enables coordinated resource sharing within dynamic organizations consisting of individuals, institutions, and resources. In the area of Grid computing a proposed approach in accordance with the trend outlined above is the Service-Oriented Knowledge Utilities (SOKU) model that envisions the integrated use of a set of technologies that are considered as a solution to information, knowledge and communication needs of many knowledge-based industrial and business applications. The SOKU approach stems from the necessity of providing knowledge and processing capabilities to everybody, thus supporting the advent of a competitive knowledge-based economy. Although the SOKU model is not yet implemented, Grids are increasingly equipped with data management tools, semantic technologies, complex work-flows, data mining features and other Web intelligence approaches. Similar efforts are currently devoted to develop knowledge and intelligent Clouds. These technologies can facilitate the process of having Grids and Clouds as strategic components for supporting pervasive knowledge intensive applications and utilities. Grids were originally designed for dealing with problems involving large amounts of data and/or compute-intensive applications. Today, however, Grids enlarged their horizon as they are going to run business applications supporting consumers and end-users. To face those new challenges, Grid environments must support adaptive knowledge management and data analysis applications by offering resources, services, and decentralized data access mechanisms. In particular, according to the service-oriented architecture (SOA) model, data mining tasks and knowledge discovery processes can be delivered as services in Grid-based infrastructures. Through a service-based approach we can define integrated services for supporting distributed business intelligence tasks in Grids. Those services can address all the aspects that must be considered in data mining and in knowledge discovery processes such as data selection and transport, data analysis, knowledge models representation and visualization. We worked in this direction for providing Grid-based architectures and services for distributed knowledge discovery such as the Knowledge Grid the Weka4WS toolkit, and mobile Grid services for data mining. Here we describe a strategy and a model based on the use of services for the design of distributed knowledge discovery services and discuss how Grid frameworks, such those mentioned above, can be developed as a collection of services and how they can be used to develop distributed data analysis tasks and knowledge discovery processes using the SOA model.

Using Grids for Exploiting the Abundance of Data in Science

Digital data volumes are growing exponentially in all sciences. To handle this abundance in data availability, scientists must use data analysis techniques in their scientific practices and solving environments to get the benefits coming from knowledge that can be extracted from large data sources. When data is maintained over geographically remote sites the computational power of distributed and parallel systems can be exploited for knowledge discovery in scientific data. In this scenario the Grid can provide an effective computational support for distributed knowledge discovery on large datasets. In particular, Grid services for data integration and analysis can represent a primary component for e-science applications involving distributed massive and complex data sets. This paper describes some research activities in data-intensive Grid computing. In particular we discuss the use of data mining models and services on Grid systems for the analysis of large data repositories.

A grid portal for solving geoscience problems using distributed knowledge discovery services

This paper describes our research effort to employ Grid technologies to enable the development of geoscience applications by integrating workflow technologies with data mining resources and a portal framework in unique work environment called MOSÈ. Using MOSÈ, a user can easily compose and execute geo-workflows for analyzing and managing natural disasters such as landslides, earthquakes, floods, wildfires, etc. MOSÈ is designed to be applicable both for the implementation of response strategies when emergencies occur and for disaster prevention. It takes advantage of the standardized resource access and workflow support for loosely coupled software components provided by web/grid services technologies. The integration of workflows with data mining services significantly improves data analysis. Geospatial data management and mining are critical areas of modern-day geosciences research. An important challenge for geospatial information mining is the distributed nature of the data. MOSÈ provides knowledge discovery services based on the WEKA data mining library and novel distributed data mining algorithms for spatial data analysis. A P2P bio-inspired algorithm for distributed spatial clustering as an example of distributed knowledge discovery service for intensive data analysis is presented. A real case application for the analysis of landslide hazard areas in the Campania Region near the Sarno area shows the advantages of using the portal.

Privacy-preserving decision trees over vertically partitioned data

Privacy and security concerns can prevent sharing of data, derailing data-mining projects. Distributed knowledge discovery, if done correctly, can alleviate this problem. We introduce a generalized privacy-preserving variant of the ID3 algorithm for vertically partitioned data distributed over two or more parties. Along with a proof of security, we discuss what would be necessary to make the protocols completely secure. We also provide experimental results, giving a first demonstration of the practical complexity of secure multiparty computation-based data mining.

Privacy-preserving SVM classification

Traditional Data Mining and Knowledge Discovery algorithms assume free access to data, either at a centralized location or in federated form. Increasingly, privacy and security concerns restrict this access, thus derailing data mining projects. What is required is distributed knowledge discovery that is sensitive to this problem. The key is to obtain valid results, while providing guarantees on the nondisclosure of data. Support vector machine classification is one of the most widely used classification methodologies in data mining and machine learning. It is based on solid theoretical foundations and has wide practical application. This paper proposes a privacy-preserving solution for support vector machine (SVM) classification, PP-SVM for short. Our solution constructs the global SVM classification model from data distributed at multiple parties, without disclosing the data of each party to others. Solutions are sketched out for data that is vertically, horizontally, or even arbitrarily partitioned. We quantify the security and efficiency of the proposed method, and highlight future challenges.

Agora

PurposeThe purpose of this paper is to propose an approach for describing and discovering knowledge resources in distributed environments.Design/methodology/approachThe paper proposes a solution that includes a framework model and a technological infrastructure to achieve distributed knowledge discovery. It provides the solution based on a combination of research from knowledge management and distributed processing areas.FindingsThe paper finds a faceted ontology and three discrete models that best achieve distributed processing in terms of knowledge items and knowledge objects. By these mechanisms are provided to: represent knowledge in a generic manner that will unchain knowledge from tight links to specific contexts, purposes and audiences; discover and retrieve knowledge stored in distributed sources; interpret different knowledge representations to enable comprehension of knowledge on the recipient side and tackle the heterogeneity problem; and simplify and accelerate knowledge exchange in distributed environments.Research limitations/implicationsThe utilization of knowledge in distributed environments (intra‐ or inter‐organizational) presupposes the existence of a representation of knowledge in a commonly understandable manner. Knowledge resources are distributed in the web and databases in an unstructured manner.Practical implicationsThe paper can be used as the basic consensus and infrastructure for the development of a coherent, manageable semantic web. The three models should be addressed in terms of commercial applications.Originality/valueKnowledge management and distributed processing research fields are combined in order to provide a solution that best addresses the problem of distributed knowledge.

Data mining ontology development for high user usability

This paper mainly introduces the development and implementation of the user-centered data mining service ontology on Universal Knowledge Grid (UKG), UKG is an ontology-based grid architecture model to build large-scale distributed knowledge discovery system on the grid. The data mining ontology services are the main service offering by UKG. It can meet the user requirements of knowledge discovery in different domains and different hierarchies and make the system exoteric, extensible and high usable. A data mining solution for money laundering is introduced.

Privacy-preserving clustering with distributed EM mixture modeling

Privacy and security considerations can prevent sharing of data, derailing data mining projects. Distributed knowledge discovery can alleviate this problem. We present a technique that uses EM mixture modeling to perform clustering on distributed data. This method controls data sharing, preventing disclosure of individual data items or any results that can be traced to an individual site.

Privacy-preserving clustering with distributed EM mixture modeling

Privacy and security considerations can prevent sharing of data, derailing data mining projects. Distributed knowledge discovery can alleviate this problem. We present a technique that uses EM mixt...

Distributed Data Mining on Grids: Services, Tools, and Applications

Data mining algorithms are widely used today for the analysis of large corporate and scientific datasets stored in databases and data archives. Industry, science, and commerce fields often need to analyze very large datasets maintained over geographically distributed sites by using the computational power of distributed and parallel systems. The grid can play a significant role in providing an effective computational support for distributed knowledge discovery applications. For the development of data mining applications on grids we designed a system called Knowledge Grid. This paper describes the Knowledge Grid framework and presents the toolset provided by the Knowledge Grid for implementing distributed knowledge discovery. The paper discusses how to design and implement data mining applications by using the Knowledge Grid tools starting from searching grid resources, composing software and data components, and executing the resulting data mining process on a grid. Some performance results are also discussed.

Metadata for Managing Grid Resources in Data Mining Applications

The Grid is an infrastructure for resource sharing and coordinated use of those resources in dynamic heterogeneous distributed environments. The effective use of a Grid requires the definition of metadata for managing the heterogeneity of involved resources that include computers, data, network facilities, and software tools provided by different organizations. Metadata management becomes a key issue when complex applications, such as data-intensive simulations and data mining applications, are executed on a Grid. This paper discusses metadata models for heterogeneous resource management in Grid-based data mining applications. In particular, it discusses how resources are represented and managed in the Knowledge Grid, a framework for Grid-enabled distributed data mining. The paper illustrates how XML-based metadata is used to describe data mining tools, data sources, mining models, and execution plans, and how metadata is used for the design and execution of distributed knowledge discovery applications on Grids.

The knowledge grid

Designing, building, and implementing an architecture for distributed knowledge discovery.

A data mining toolset for distributed high- performance platforms

Today a large number of scientific and commercial applications often require to analyse large data sets maintained over geographically distributed sites by using the computational power of distributed high-performance environments. Advances in networking technology and computational infrastructure made it possible to construct large-scale distributed computing platforms, called computational grids, that provide dependable, consistent, and pervasive access to high-end computational resources. Grids can play a significant role in providing an effective computational support for distributed data mining applications. Currently we are developing a software system for geographically distributed knowledge discovery applications called KNOWLEDGE GRID,which is designed on top of computational grid mechanisms, provided by grid environments such as Glob us. In this paper we present an integrated toolset named VEGA (Visual Environment for Grid Applications), which allows a Knowledge Grid user to develop and execute distributed data mining computations in a simple and effective way.

Distributed data mining on the grid

In many industrial, scientific and commercial applications, it is often necessary to analyze large data sets, maintained over geographically distributed sites, by using the computational power of distributed and parallel systems. The grid can play a significant role in providing an effective computational support for knowledge discovery applications. We describe a software architecture for geographically distributed high-performance knowledge discovery applications called K nowledge G rid , which is designed on top of computational grid mechanisms, provided by grid environments such as Globus. The K nowledge G rid uses the basic grid services such as communication, authentication, information, and resource management to build more specific parallel and distributed knowledge discovery tools and services. The paper discusses how the K nowledge G rid can be used to implement distributed data mining services.