Homogeneous Network Environment Research Articles

SelectiveEC: Towards Balanced Recovery Load on Erasure-Coded Storage Systems

Erasure coding (EC) has been commonly used to offer high data reliability with low storage cost. Upon failures, the lost blocks are recovered in batches. Due to the limited number of stripes, the data layout within a batch is non-uniform. Together with the random selection of source and replacement nodes for recovery tasks, the recovery workload among live nodes is skewed within a batch, which severely slows down failure recovery. To solve this problem, We present SelectiveEC, a new recovery task scheduling module that provides provable network traffic and recovery load balancing for large-scale EC-based storage systems. It relies on bipartite graphs to model the recovery traffic among live nodes. Then, it intelligently selects tasks to form batches and carefully determines where to read source blocks or to store recovered ones, using theories such as a perfect or maximum matching and <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -regular spanning subgraph. SelectiveEC supports single-node failure and multi-node failure recovery, and can be deployed in both homogeneous and heterogeneous network environments. We implement SelectiveEC in HDFS, and evaluate its recovery performance in a local cluster of 18 nodes and AWS EC2 of 50 virtual machine instances. SelectiveEC increases the recovery throughput by up to <inline-formula><tex-math notation="LaTeX">$30.68\%$</tex-math></inline-formula> compared with state-of-the-art baselines in homogeneous network environments. It further achieves <inline-formula><tex-math notation="LaTeX">$1.32\times$</tex-math></inline-formula> recovery throughput and <inline-formula><tex-math notation="LaTeX">$1.23\times$</tex-math></inline-formula> benchmark throughput of HDFS on average in heterogeneous network environments, due to the straggler avoidance by the balanced scheduling.

LSBT-A NEW IMPEND FOR BIG DATA BROADCASTING

In contemporary period huge amount of data will be stored on cloud server, these types of data is referred as Big-Data. Big data computing is a complicated issue for broadcasters and engineers because they are dealing with datasets of petabytes in the cloud. To manage process and broadcast these massive data has increased drastically. Here we propose an approach to broadcast the data by large chunks to group of nodes that can reduce the maximum completion time. Here we model a Lock Step Broadcast Tree (LSBT) which will define an upload bandwidth(r) and each node capacity(C) at c/r per children. After these data sets divided into chunks, and are broadcasted in a pipeline manner. In homogeneous network environment the capabilities of each node is C, and uplink rate r which gives less maximum completion time. In heterogeneous networks capabilities C1, C2...Cn, so optimal uplink rate r will be calculated by O (nlog2n) algorithm there after we construct a LSBT. Finally our results show less computational complexity and less maximum completion time over other broadcasting techniques.

A Novel Pipeline Approach for Efficient Big Data Broadcasting

Big-data computing is a new critical challenge for the ICT industry. Engineers and researchers are dealing with data sets of petabyte scale in the cloud computing paradigm. Thus, the demand for building a service stack to distribute, manage, and process massive data sets has risen drastically. In this paper, we investigate the Big Data Broadcasting problem for a single source node to broadcast a big chunk of data to a set of nodes with the objective of minimizing the maximum completion time. These nodes may locate in the same datacenter or across geo-distributed datacenters. This problem is one of the fundamental problems in distributed computing and is known to be NP-hard in heterogeneous environments. We model the Big-data broadcasting problem into a LockStep Broadcast Tree (LSBT) problem. The main idea of the LSBT model is to define a basic unit of upload bandwidth, $r$ , such that a node with capacity $c$ broadcasts data to a set of $\lfloor c/r\rfloor$ children at the rate $r$ . Note that $r$ is a parameter to be optimized as part of the LSBT problem. We further divide the broadcast data into $m$ chunks. These data chunks can then be broadcast down the LSBT in a pipeline manner. In a homogeneous network environment in which each node has the same upload capacity $c$ , we show that the optimal uplink rate $r^*$ of LSBT is either $c/2$ or $c/3$ , whichever gives the smaller maximum completion time. For heterogeneous environments, we present an $O(n \text{log}^2 n)$ algorithm to select an optimal uplink rate $r^*$ and to construct an optimal LSBT. Numerical results show that our approach performs well with less maximum completion time and lower computational complexity than other efficient solutions in literature.

A novel vertical handoff algorithm based on fuzzy logic in aid of grey prediction theory in wireless heterogeneous networks

To coordinate the various access technologies in the 4G communication system, intelligent vertical handoff algorithms are required. This paper mainly deals with a novel vertical handoff decision algorithm based on fuzzy logic with the aid of grey theory and dynamic weights adaptation. The grey prediction theory (GPT) takes 4 sampled received signal strengths as input parameters, and calculates the predicted received signal strength in order to reduce the call dropping probability. The fuzzy logic theory based quantitative decision algorithm takes 3 quality of service (QoS) metric, received signal strength (RSS), available bandwidth (BW), and monetary cost (MC) of candidate networks as input parameters. The weight of each QoS metrics is adjusted along with the networks changing to trace the network condition. The final optimized vertical handoff decision is made by comparing the quantitative decision values of the candidate networks. Simulation results demonstrate that the proposed algorithm provides high performance in heterogeneous as well as homogeneous network environments.

The SIGIR peer-to-peer information retrieval workshop

Peer-to-peer (P2P) systems have emerged as a popular way to share huge volumes of data. The P2P paradigm holds many promises: it fits naturally with the Internet, the universal knowledge and service exchange medium; it favors scalability, by allowing the seamless plugging of data, services and computational resources into the global system; it increases system resilience, by avoiding unique points of failures; and it can speed up global access by distributing the indexing and query processing tasks to multiple computing nodes. However, retrieval methods for P2P systems are still in their infancy. P2P networks are prone to congestion when messages are not routed intelligently. Many of the most effective routing or data placement methods developed recently rely on relatively simple retrieval methods and homogeneous network environments.