Multiple Distributed Data Sources Research Articles

Multi-Agent System Combined With Distributed Data Mining for Mutual Collaboration Classification

Distributed Data Mining (DDM) has been proposed as a means to deal with the analysis of distributed data, where DDM discovers patterns and implements prediction based on multiple distributed data sources. However, DDM faces several problems in terms of autonomy, privacy, performance and implementation. DDM requires homogeneity regarding environment, control, administration and the classification algorithm(s), and such that requirements are too strict and inflexible in many applications. In this paper, we propose the employment of a Multi-Agent System (MAS) to be combined with DDM (MAS-DDM). MAS is a mechanism for creating goal-oriented autonomous agents within shared environments with communication and coordination facilities. We shall show that MAS-DDM is both desirable and beneficial. In MAS-DDM, agents could communicate their beliefs (calculated classification) by covering private and non-sharable data, and other agents decide whether the use of such beliefs in classifying instances and adjusting their prior assumptions about each class of data. In MAS-DDM, we will develop and use a modified Naive Bayesian algorithm because (1) Naive Bayesian has been shown to be the most used algorithm to deal with uncertain data, and (2) to show that even if all agents in MAS-DDM use the same algorithm, MAS-DDM preforms better than DDM approaches with non-communicating processes. Point (2) provide an evidence that the exchange of information between agents helps in increasing the accuracy of the classification task significantly.

How a BI-wise Responsible Integrated Management System May Support Food Traceability

Food manufacturers are required to meet certain traceability specifications. This research aims at underscoring the relevant needs and expectations of various stakeholders across the entire food supply chain. In this context, firms' decisions on resource allocation and risk mitigation overlap several domains, such as quality, safety, environment, social responsibility and information. Business Intelligence (BI) platforms are specifically conceived to support analytical decision making by providing a centralised view on multiple distributed data sources. However, BI solutions are usually deployed within a single organization, whilst traceability involves multiple actors with potentially divergent interests and diverse levels of willingness to participate. Along this line of thought, integration of management systems within a company and throughout the overall supply chain is suggested to meet the emerging managerial challenges. After a detailed survey of integrated management systems (IMSs) in food traceability contexts, this research proposes a BI-wise solution using the IMS overarching approach and investigates its success conditions.

Learning from Distributed Data Sources Using Random Vector Functional-Link Networks

One of the main characteristics in many real-world big data scenarios is their distributed nature. In a machine learning context, distributed data, together with the requirements of preserving privacy and scaling up to large networks, brings the challenge of designing fully decentralized training protocols. In this paper, the authors explore the problem of distributed learning when the features of every pattern are available throughout multiple agents (as is happening, for example, in a distributed database scenario). They propose an algorithm for a particular class of neural networks, known as Random Vector Functional-Link (RVFL), which is based on the Alternating Direction Method of Multipliers optimization algorithm. The proposed algorithm allows to learn an RVFL network from multiple distributed data sources, while restricting communication to the unique operation of computing a distributed average. The experimental simulations show that the algorithm is able to achieve a generalization accuracy comparable to a fully centralized solution, while at the same time being extremely efficient.

Assortment of Materialized View: A Comparative Survey in Data Warehouse Environment

warehouse is a repository of large amount of data collected from multiple heterogeneous and distributed data sources. Data warehouse stores lots of data in the form of views, referred as materialized views which provide a base for decision support or OLAP queries. Materialized views store the result of queries which improves the query performance. One of the most important aspect in data warehousing is the selection of materialized views which minimizes the query response time and maintenance cost, given a limited storage space. In this paper, analysis of various approaches of view selection in data warehousing environment is done that have been proposed in the recent past and also provided a comprehensive study of these approaches based on various parameters such as issues addressed, query language supported, comparison to benchmark etc.

Service-Oriented Collaborative Framework for High-Performance Data Transfer in Grids

As the number of data-intensive applications increases in various domains, scientists need to save, retrieve, and analyze increasingly large datasets. The huge volume of data and the long latency of data transfer on the Internet make it very difficult to ensure high-performance access to Data Grids. Thus, data replication techniques have been widely adopted to solve the latency problem. In this paper, we propose an efficient data replication algorithm for multi-source data transfer, whereby a data replica can be assembled in parallel from multiple distributed data sources and adapted to the fluctuation of network bandwidths. The experimental results show that the proposed algorithm can obtain more aggregated bandwidth, reduce connection overheads, and achieve superior load balance. We also develop a Service-oriented Collaborative Framework that not only implements the proposed data replication algorithm in a real Grid system, but also provides a virtual data management environment that integrates distributed and heterogeneous storage systems and enables users to share their data with group members for collaborative work.

Report on the 5 th international workshop on the design and management of data warehouses (DMDW'03)

Data Warehousing embraces technology and industrial practice to systematically integrate data from multiple distributed data sources and to use that data in annotated and aggregated form to support business decision-making and enterprise management. Although many database techniques have been revisited or newly developed in the context of data warehouses, such as view maintenance and OLAP, little attention has been paid to the design, management and high quality service of the management of a given enterprise. Little attention is also paid to aspects that are intrinsic to the functionality and usage of data warehouses, either in the back-stage (like data cleansing or data extraction and loading) or in the front-end (like similar and uncertain queries, or what-if analysis). The DMDW workshop is intended as a forum to fill this gap.

Performance comparison of three alternatives of distributed multidatabase systems: A global query perspective

Diversity and evolution in database applications often result in a multidatabase environment in which corporate data are stored in multiple distributed data sources, each managed by an independent database management system. One of the essential functions of a multidatabase system is to provide inter database access: the capability of evaluating global queries that require access to multiple data sources. The paper compares three common relational multidatabase approaches: the federated approach, the gateway approach, and the middleware approach from the perspective of global query performance. In particular, we examine their architectural impact on the applicability of pipelined query processing techniques and load balancing. We present a performance comparison based on a detailed simulation. The study suggests that the middleware approach, which is the most cost effective solution among the three, provides better or comparable performance to the other two approaches.

Efficient view maintenance at data warehouses

We present incremental view maintenance algorithms for a data warehouse derived from multiple distributed autonomous data sources. We begin with a detailed framework for analyzing view maintenance algorithms for multiple data sources with concurrent updates. Earlier approaches for view maintenance in the presence of concurrent updates typically require two types of messages: one to compute the view change due to the initial update and the other to compensate the view change due to interfering concurrent updates. The algorithms developed in this paper instead perform the compensation locally by using the information that is already available at the data warehouse. The first algorithm, termed SWEEP, ensures complete consistency of the view at the data warehouse in the presence of concurrent updates. Previous algorithms for incremental view maintenance either required a quiescent state at the data warehouse or required an exponential number of messages in terms of the data sources. In contrast, this algorithm does not require that the data warehouse be in a quiescent state for incorporating the new views and also the message complexity is linear in the number of data sources. The second algorithm, termed Nested SWEEP, attempts to compute a composite view change for multiple updates that occur concurrently while maintaining strong consistency.