Efficient Data Distribution Research Articles

Exact approaches for static data segment allocation problem in an information network

In a large distributed database, data are geographically distributed across several separate servers (or data centers). This helps in distributing load in the access network. It also helps to serve data locally where it is required. There are various approaches based on the granularity of data for efficient data distribution in a communication network. The file allocation problem (FAP) locates files to servers, the segment allocation problem (SAP) locates database segments, and the mirror location problem (MLP) locates replicas of the entire database. The placement of such data to multiple servers can be modeled as an optimization problem. The major decisions influencing optimization involves the location of servers, allocation of content and assignment of users. In this paper, we study the segment allocation problem (SAP), which is also known as the partial mirroring problem. This approach is more tractable than the file allocation problem in realistic cases and also eliminates the overhead of (constant) update costs that is incurred in the mirror location problem. Our contribution is two-fold: Firstly, earlier works on SAP assume pre-defined segments. We build a data partitioning method using well-known facility location models. We quantify the performance of the partitioning method. We show that the method partitions the database within a reasonable limit of error. Secondly, we introduce a new model for the segment allocation problem in which the segments are completely connected to each other by high-bandwidth links and contains a cost benefit for inter-segment traffic flows. We formulate this problem as an MILP and build exact solution approaches to solve large scale problems. We demonstrate some structural properties of the problem that make it solvable, using a Benders decomposition algorithm. Computational results validate the superiority of the decomposition approach.

TopFed: TCGA tailored federated query processing and linking to LOD.

BackgroudThe Cancer Genome Atlas (TCGA) is a multidisciplinary, multi-institutional effort to catalogue genetic mutations responsible for cancer using genome analysis techniques. One of the aims of this project is to create a comprehensive and open repository of cancer related molecular analysis, to be exploited by bioinformaticians towards advancing cancer knowledge. However, devising bioinformatics applications to analyse such large dataset is still challenging, as it often requires downloading large archives and parsing the relevant text files. Therefore, it is making it difficult to enable virtual data integration in order to collect the critical co-variates necessary for analysis.MethodsWe address these issues by transforming the TCGA data into the Semantic Web standard Resource Description Format (RDF), link it to relevant datasets in the Linked Open Data (LOD) cloud and further propose an efficient data distribution strategy to host the resulting 20.4 billion triples data via several SPARQL endpoints. Having the TCGA data distributed across multiple SPARQL endpoints, we enable biomedical scientists to query and retrieve information from these SPARQL endpoints by proposing a TCGA tailored federated SPARQL query processing engine named TopFed.ResultsWe compare TopFed with a well established federation engine FedX in terms of source selection and query execution time by using 10 different federated SPARQL queries with varying requirements. Our evaluation results show that TopFed selects on average less than half of the sources (with 100% recall) with query execution time equal to one third to that of FedX.ConclusionWith TopFed, we aim to offer biomedical scientists a single-point-of-access through which distributed TCGA data can be accessed in unison. We believe the proposed system can greatly help researchers in the biomedical domain to carry out their research effectively with TCGA as the amount and diversity of data exceeds the ability of local resources to handle its retrieval and parsing.

Toward content-centric privacy in ICN

We design a content-centric privacy scheme for Information-Centric Networking (ICN). We enhance ICN's ability to support data confidentiality by introducing attribute-based encryption into ICN and making it specific to the data attributes. Our approach is unusual in that it preserves ICN's goal to decouple publishers and subscribers for greater data accessibility, scalable multiparty communication and efficient data distribution. Inspired by application-layer publish-subscribe, we enable fine-grained access control with more expressive policies. Moreover, we propose an attribute-based routing scheme that offers interest confidentiality. A prototype system is implemented based on CCNx, a popular open source version of ICN, to showcase privacy preservation in Smart Neighborhood and Smart City applications.

A synchronized design technique for efficient data distribution

One of the important features of database fragmentation and allocation techniques is the fact that they depend not only on the entries of a database relation, but also on their empirical frequencies of use. Distributed processing is an effective way to improve performance of database systems. However, for a Distributed Database System (DDBS) to function efficiently, fragments of the database need to be allocated carefully at various sites across the relevant communications network. Therefore, fragmentation and proper allocation of fragments across network sites is considered as a key research area in distributed database environment. However, fragments allocation to the most appropriate sites is not an easy task to perform. This paper proposes a synchronized horizontal fragmentation, replication and allocation model that adopts a new approach to horizontally fragment a database relation based on attribute retrieval and update frequency to find an optimal solution for the allocation problem. A heuristic technique to satisfy horizontal fragmentation and allocation using a cost model to minimize the total cost of distribution is developed. Experimental results are consistent with the hypothesis and confirm that the proposed model can efficiently solve dynamic fragmentation and allocation problem in a distributed relational database environment.

Improved data distribution strategy for cloud storage system

Considering massive scale of cloud storage solutions,the traditional data distribution strategy confronts challenges to improve scalability and flexibility.This paper proposed an efficient data distribution strategy.Based on consistent hashing algorithm,the strategy introduced the virtualization technology,and employed virtual node to improve load balance.Moreover,the strategy used a new capacity-aware method to improve the performance of the cloud storage system.The evaluation experiments demonstrate that the proposed data distribution strategy improves system performance in both homogeneous and heterogeneous distributed storage architectures.

Performance evaluation of multisource Application Layer Multicast (ALM): Theoretical and simulative aspects

Multisource ALM routing protocol is an indispensable mechanism for efficient distribution of data from many sources to many receivers. The growing of Internet multimedia applications (e.g. videoconference and multiparty online game) which demand large network resources and call for multisource multicasting, has explicitly developed the necessity of an effective method to evaluate and compare the efficiency of various multisource multicast routings. We define a routing cost to be the total number of underlying hops traversed by multicast data over a multicast delivery tree. This study explores a multiplicative property to obtain the minimum cost of various multisource ALM routing protocols by simply multiplying the number of sources to the minimum cost in the single-source multicast. From theoretical and simulative studies, the multiplicative property has been proven through examining the minimum cost of multisource multicast delivery trees constructed by different multisource ALM routing protocols with source-specific tree (SST), core-based group-shared tree (GST-C) and bidirectional group-shared tree (GST-B), which are employed in the design of multisource ALM routing protocols.

MMT: An Emerging MPEG Standard for Multimedia Delivery over the Internet

Content-centric networking promises more efficient distribution of data through in-network caching and the propagation of content through the network. This networking paradigm poses a number of challenges and new opportunities for more efficient multimedia delivery. The MPEG Media Transport (MMT) standard is being developed to address these needs with specifications for encapsulation, delivery, and signaling, which enable fine-grained content access with uniquely identifiable names for optimized content delivery.

Combating Write Penalties Using Software Dispatch for On-Chip MRAM Integration

Recent advances in the emerging memory technology magnetic RAM (MRAM) enrich the opportunities to build high density and low power embedded systems. One common way of utilizing MRAM is integrating it with conventional memories and distributing data to the appropriate type of memory to mitigate the high write penalty of MRAM. In this paper, we propose a software-based approach to identify data access characteristics and guide hardware to perform efficient data distribution. We use our technique to build an on-chip MRAM-SRAM hybrid cache and demonstrate an 86.8% reduction in leakage power, a 9.8% reduction in total power, and a 5% memory performance improvement, compared to a traditional static RAM (SRAM)-only cache.

A survey of context data distribution for mobile ubiquitous systems

The capacity to gather and timely deliver to the service level any relevant information that can characterize the service-provisioning environment, such as computing resources/capabilities, physical device location, user preferences, and time constraints, usually defined as context-awareness, is widely recognized as a core function for the development of modern ubiquitous and mobile systems. Much work has been done to enable context-awareness and to ease the diffusion of context-aware services; at the same time, several middleware solutions have been designed to transparently implement context management and provisioning in the mobile system. However, to the best of our knowledge, an in-depth analysis of the context data distribution, namely, the function in charge of distributing context data to interested entities, is still missing. Starting from the core assumption that only effective and efficient context data distribution can pave the way to the deployment of truly context-aware services, this article aims at putting together current research efforts to derive an original and holistic view of the existing literature. We present a unified architectural model and a new taxonomy for context data distribution by considering and comparing a large number of solutions. Finally, based on our analysis, we draw some of the research challenges still unsolved and identify some possible directions for future work.

Towards a flexible middleware for context-aware pervasive and wearable systems

Ambient intelligence and wearable computing call for innovative hardware and software technologies, including a highly capable, flexible and efficient middleware, allowing for the reuse of existing pervasive applications when developing new ones. In the considered application domain, middleware should also support self-management, interoperability among different platforms, efficient communications, and context awareness. In the on-going "everything is networked" scenario scalability appears as a very important issue, for which the peer-to-peer (P2P) paradigm emerges as an appealing solution for connecting software components in an overlay network, allowing for efficient and balanced data distribution mechanisms. In this paper, we illustrate how all these concepts can be placed into a theoretical tool, called networked autonomic machine (NAM), implemented into a NAM-based middleware, and evaluated against practical problems of pervasive computing.

Multi-Threading and Suffix Grouping on Massive Multiple Pattern Matching Algorithm

The widely used multiple pattern matching algorithms experience severe performance degradation when the number of patterns to match increases. In light of this fact, this paper presents a multi-threaded multiple pattern matching algorithm to overcome the performance degradation; this algorithm presents two additional improvements on the original Wu–Manber algorithm. First, the proposed algorithm employs a multi-threaded execution model to parallelize the pattern matching operation on multi-core processors. Second, the patterns to be searched are distributed over multiple threads according to the pattern similarity. For this purpose, the proposed algorithm groups the target patterns on the basis of their suffixes and distributes the patterns over multiple threads. Through experiments and performance analysis, our algorithm shows a significant performance gain as compared with the original Wu–Manber algorithm and the previously proposed multi-threaded pattern matching on massive pattern sets of size exceeding 5000. The results obtained from the pattern matching operation using eight cores show much improved execution time, which is nearly 14.9 times faster on average than that of the conventional Wu–Manber algorithm. It is demonstrated that the proposed idea improves the overall performance by reducing the amount of workload on a single thread through multi-threading and an efficient data distribution policy.

An Efficient Method For Multichannel Wireless Mesh Networks With Pulse Coupled Neural Network

Multi cast communication is a key technology for wireless mesh networks. Multicast provides efficient data distribution among a group of nodes, Generally sensor networks and MANETs uses multicast algorithms which are designed to be energy efficient and to achieve optimal route discovery among mobile nodes whereas wireless mesh networks needs to maximize throughput. Here we propose two multicast algorithms: The Level Channel Assignment (LCA) algorithm and the Multi-Channel Multicast (MCM) algorithm to improve the throughput for multichannel sand multi interface mesh networks. The algorithm builds efficient multicast trees by minimizing the number of relay nodes and total hop count distance of the trees. Shortest path computation is a classical combinatorial optimization problem. Neural networks have been used for processing path optimization problem. Pulse Coupled Neural Networks (PCNNS) suffer from high computational cast for very long paths we propose a new PCNN modal called dual source PCNN (DSPCNN) which can improve the computational efficiency two auto waves are produced by DSPCNN one comes from source neuron and other from goal neuron when the auto waves from these two sources meet the DSPCNN stops and then the shortest path is found by backtracking the two auto waves.

An efficient technique for geocast region holes in underwater sensor networks and its performance evaluation

In this paper, we present a complete Geocasting solution for underwater sensor networks. The model named Routing and Multicast Tree based Geocasting ( RMTG) [1,2] has been designed for underwater sensor networks. In our previous work [1], we proposed and simulated the RMTG model in terms of routing, route maintenance, link stability and data dispersion inside the geocast region in static as well as in mobile network conditions. In [3] we provided the theoretical improvement over the RMTG in terms of Geocasting for underwater sensor network by providing solutions for hole/void area detection inside the geocast region and a solution to cover the scattered nodes in the geocast region for guaranteed message delivery by means of boundary routing (that is routing by traversing all the faces of a particular geocast region). Here, the work in [3] has been enhanced, the model is named as Routing and Multicast Tree based Geocasting with Hole Detection ( RMTG-HD). A detailed analysis and performance evaluation of the proposed model has also been provided. The simulation results show that the proposed model provides an efficient distribution of data in the geocast region in terms of packet delivery ratio, boundary routing delay, hole detection delay and handling of node mobility.

Providing reliable and link stability‐based geocasting model in underwater environment

SUMMARYThis paper presents a geocast technique for underwater sensor networks. The model named as Routing and Multicast Tree based Geocasting (RMTG) (TRACE'10, Bhopal, India, 2010; ICWET'10, Mumbai, India, 2010) has been designed for underwater sensor networks. In our previous work (TRACE'10, Bhopal, India, 2010; ICWET'10, Mumbai, India, 2010) we proposed only a theoretical model for underwater geocasting. The RMTG technique uses greedy forwarding and previous hop handshaking to route the packets towards the geocast region and further disseminates the data within the geocast region by creating a multicast shortest path tree. We also presented various link broken scenarios along with their solutions. The proposed model does not use the flooding technique to deliver the packets inside the geocast region that is used in most of the geocasting techniques. In this paper, we present the simulated RMTG model. The simulation results show that the proposed model provides an efficient distribution of data in the geocast region in terms of node mobility handling, packet delivery ratio and a better end‐to‐end latency. Most of the work on geocasting has been done for mobile and vehicular ad hoc networks; hence we present in this paper a novel simulated model for underwater environment. Copyright © 2011 John Wiley & Sons, Ltd.

PetaShare: A Reliable, Efficient and Transparent Distributed Storage Management System

Modern collaborative science has placed increasing burden on data management infrastructure to handle the increasingly large data archives generated. Beside functionality, reliability and availability are also key factors in delivering a data management system that can efficiently and effectively meet the challenges posed and compounded by the unbounded increase in the size of data generated by scientific applications. We have developed a reliable and efficient distributed data storage system, PetaShare, which spans multiple institutions across the state of Louisiana. At the back-end, PetaShare provides a unified name space and efficient data movement across geographically distributed storage sites. At the front-end, it provides light-weight clients the enable easy, transparent and scalable access. In PetaShare, we have designed and implemented an asynchronously replicated multi-master metadata system for enhanced reliability and availability, and an advanced buffering system for improved data transfer performance. In this paper, we present the details of our design and implementation, show performance results, and describe our experience in developing a reliable and efficient distributed data management system for data-intensive science.

An H.264/AVC Decoder with Reduced External Memory Access for Motion Compensation

The excessive memory access required to perform motion compensation when decoding compressed video is one of the main limitations in improving the performance of an H.264/AVC decoder. This paper proposes an H.264/AVC decoder that employs three techniques to reduce external memory access events: efficient distribution of reference frame data, on-chip cache memory, and frame memory recompression. The distribution of reference frame data is optimized to reduce the number of row activations during SDRAM access. The novel cache organization is proposed to simplify tag comparisons and ease the access to consecutive 4x4 blocks. A recompression algorithm is modified to improve compression efficiency by using unused storage space in neighboring blocks as well as the correlation with the neighboring pixels stored in the cache. Experimental results show that the three techniques together reduce external memory access time by an average of 90%, which is 16% better than the improvements achieved by previous work. Efficiency of the frame memory recompression algorithm is improved with a 32x32 cache, resulting in a PSNR improvement of 0.371dB. The H.264/AVC decoder with the three techniques is fabricated and implemented as an ASIC using 0.18µm technology.

Special Issue on the 9th International Symposium on Parallel and Distributed Computing

Welcome to this special issue on the ninth edition of International Symposium on Parallel and Distributed Computing (ISPDC) held in Istanbul in July 2010. ISPDC is dedicated to the advancement of state-of-theart in the area of parallel and distributed computing. The symposium program featured 28 full papers accepted out of 71 submitted papers. The program covered a wide spectrum of topics including parallel models and algorithms, interconnection topologies, GPU programming, grids, P2P, multicores and distributed systems. The three papers in this special issue were selected among the highest rated accepted papers and involve scientific programming on a wide spectrum of computational platforms, namely, Graphics Processing Units (GPUs), multi-cluster systems and grids. These three papers were extended and revised after a new round of review process before being published in Scientific Programming. The first paper titled “Improving accuracy for matrix multiplications on GPUs” by Badin et al. demonstrates the emergence of scientific programming research on Graphics Processing Units (GPU). The paper considers the problem of floating point arithmetic accuracy within the context of matrix multiplication on GPUs. On GPUs that lack double precision support, the problem of numerical accuracy becomes quite important. Hence, new algorithms that deliver high accuracy with little performance cost need to be developed. This paper addresses these issues and proposes doubly compensated summation and Kahan’s compensated summation based algorithms with little performance costs. The second paper titled “Performance models for the Spike banded linear system solver” by Manguoglu et al. investigates the problem of characterizing performance and scalability of banded preconditioners for linear system solvers. The authors develop parallel formulations of a banded solver, called the Truncated Spike solver. They present a highly accurate parallel performance model for the solver and establish the validity of the model on diverse heterogeneous multiclusters. Their performance model supported by actual runs demonstrates high scalability of their solver. The third paper “PetaShare: A reliable, efficient and transparent distributed storage management system” by Kosar et al. addresses the development of a distributed data management system for data-intensive e-science. The authors contribute a reliable and efficient distributed data storage system, called PetaShare, which spans multiple institutions over a wide geographic area. PetaShare provides a front-end with light-weight clients that enable easy, transparent and scalable access. A unified name space and efficient client data movement capability across geographically distributed storage sites are provided by PetaShare. The authors also present the details of their design and implementation and show performance results on reading/writing to remote PetaShare sites.

CMS offline conditions framework and services

Non-event data describing detector conditions change with time and come from different data sources. They are accessible by physicists within the offline event-processing applications for precise calibration of reconstructed data as well as for data-quality control purposes. Over the past years CMS has developed and deployed a software system managing such data. Object-relational mapping and the relational abstraction layer of the LHC persistency framework are the foundation; the offline condition framework updates and delivers C++ data objects according to their validity. A high-level tag versioning system allows production managers to organize data in hierarchical view. A scripting API in python, command-line tools and a web service serve physicists in daily work. A mini-framework is available for handling data coming from external sources. Efficient data distribution over the worldwide network is guaranteed by a system of hierarchical web caches. The system has been tested and used in all major productions, test-beams and cosmic runs.

Analyzing data distribution on disk pools for dCache

Most Tier-1 centers of LHC Computing Grid are using dCache as their storage system. dCache uses a cost model incorporating CPU and space costs for the distribution of data on its disk pools. Storage resources at Tier-1 centers are usually upgraded once or twice a year according to given milestones. One of the effects of this procedure is the accumulation of heterogeneous hardware resources. For a dCache system, a heterogeneous set of disk pools complicates the process of weighting CPU and space costs for an efficient distribution of data. In order to evaluate the data distribution on the disk pools, the distribution is simulated in Java. The results are discussed and suggestions for improving the weight scheme are given.

Efficient Multicast Algorithms for Multichannel Wireless Mesh Networks

The wireless mesh network is an emerging technology that provides high quality service to end users as the "last milerdquo of the Internet. Furthermore, multicast communication is a key technology for wireless mesh networks. Multicast provides efficient data distribution among a group of nodes. However, unlike other wireless networks, such as sensor networks and MANETs, where multicast algorithms are designed to be energy efficient and to achieve optimal route discovery among mobile nodes, wireless mesh networks need to maximize throughput. This paper proposes two multicast algorithms: the level channel assignment (LCA) algorithm and the multichannel multicast (MCM) to improve the throughput for multichannel and multi-interface mesh networks. The algorithms build efficient multicast trees by minimizing the number of relay nodes and total hop count distances of the trees. The algorithms use dedicated channel assignment strategies to reduce the interference to improve the network capacity. We also demonstrate that using partially overlapping channels can further diminish the interference. Furthermore, additional interfaces help to increase the bandwidth, and multiple gateways can further shorten the total hop count distance. Simulations show that those algorithms greatly outperform the single-channel multicast algorithm. We also observe that MCM achieves better throughput and shorter delay while LCA can be realized in distributed manner.