Hierarchical Data Grid Research Articles

The Impact of Data Replication on Job Scheduling Performance in Hierarchical Data Grid

The Impact of Data Replication on Job Scheduling Performance in Hierarchical Data Grid

Data placement using Dewey Encoding in a hierarchical data grid

Nowadays, data grids can be seen as frameworks responding to the needs of large scale applications by affording so many resources. These are distributed on different geographically locations, but are organized to provide an integrated service. This paper studies replica placement in hierarchical data grid systems taking into account the following important issue: The hierarchical data grid can be presented as a tree structure. This structure will be labeled by a new labeling scheme (called Dewey Encoding) that lies at the core of query processing for many XML databases. This scheme avoids unnecessary replications and achieves good performance by balancing the load of replica servers. The good performance of the proposed algorithm is based on minimizing the Mean Job Execution time and Network utilization.

Network and data location aware approach for simultaneous job scheduling and data replication in large-scale data grid environments

Data Grid integrates graphically distributed resources for solving data intensive scientific applications. Effective scheduling in Grid can reduce the amount of data transferred among nodes by submitting a job to a node, where most of the requested data files are available. Scheduling is a traditional problem in parallel and distributed system. However, due to special issues and goals of Grid, traditional approach is not effective in this environment any more. Therefore, it is necessary to propose methods specialized for this kind of parallel and distributed system. Another solution is to use a data replication strategy to create multiple copies of files and store them in convenient locations to shorten file access times. To utilize the above two concepts, in this paper we develop a job scheduling policy, called hierarchical job scheduling strategy (HJSS), and a dynamic data replication strategy, called advanced dynamic hierarchical replication strategy (ADHRS), to improve the data access efficiencies in a hierarchical Data Grid. HJSS uses hierarchical scheduling to reduce the search time for an appropriate computing node. It considers network characteristics, number of jobs waiting in queue, file locations, and disk read speed of storage drive at data sources. Moreover, due to the limited storage capacity, a good replica replacement algorithm is needed. We present a novel replacement strategy which deletes files in two steps when free space is not enough for the new replica: first, it deletes those files with minimum time for transferring. Second, if space is still insufficient then it considers the last time the replica was requested, number of access, size of replica and file transfer time. The simulation results show that our proposed algorithm has better performance in comparison with other algorithms in terms of job execution time, number of intercommunications, number of replications, hit ratio, computing resource usage and storage usage.

Collaborative Services for Fault Tolerance in Hierarchical Data Grid

As fault tolerance is the ability of a system to perform its function correctly even in the presence of faults. Therefore, different fault tolerance techniques are critical for improving the efficient utilization of expensive resources in high performance data grid systems. One of the most popular strategies of fault tolerance is the replication, it creates multiple copies of resources in the system and it has been proved to be an effective way to achieve data availability and system reliability. In this paper the authors propose a new adaptive dynamic replication that combines between a replication based on availability and replication based on popularity. The authors' adaptive dynamic replication uses two types of replicas (primary and ordinary) and two types of placement nodes (best client and best responsible nodes) for the new replicas. In addition to the replication, we used other strategies such as fault detection, fault prediction, dynamicity management, self-stabilization. All these services are grouped in one fault tolerance box named Collaborative Services for Fault Tolerance (CSFT) that structure them in hierarchical services and organize the relationships between them.

A Threshold-based Dynamic Data Replication and Parallel Job Scheduling strategy to enhance Data Grid

Data Grids provide environment for huge, data-intensive applications that produce and process enormous data. Such environments are thus asked to manage data and schedule jobs at the same time. These two important operations have to be tightly coupled to achieve the best results. Replication techniques are widely used to increase the availability of data, improving performance of query latency and load balancing in Data Grid. Also effective resource scheduling is a challenging research issue. In this paper we propose a job scheduling policy, called Parallel Job Scheduling (PJS), and a dynamic data replication strategy, called Threshold-based Dynamic Data Replication (TDDR), to improve the data access efficiencies in a hierarchical Data Grid. The PJS uses hierarchical scheduling to reduce the search time for an appropriate computing node. It considers network characteristics, number of jobs waiting in queue, file locations, and disk read speed of storage drive at data sources. The main idea of TDDR strategy is using a threshold value to determine if the requested replica needs to be copied to the node. The TDDR determines this threshold dynamically based on data request arrival rates and available storage capacities. Then, in order to overcome the problem of limited storage space in each node, we design an efficient replica replacement strategy, which is developed as a two stages process. First, it deletes those files with minimum time for transferring. Second, if space is still insufficient then it considers the last time the replica was requested, number of access, size of replica and file transfer time. Results from the simulation show that our proposed algorithms have better performance in comparison with other algorithms in terms of Mean Job Time, Number of Intercommunications, Number of Replications, Computing Resource Usage, and Effective Network Usage.

Allocating replicas in large-scale data grids using a QoS-aware distributed technique with workload constraints

An important technique to speed access in data grids is replication, which provides nearby replicas. In a data grid environment, resource availability, network latency and user request patterns may change. In this paper, we introduce a new distributed replica placement algorithm for hierarchical data grids that determines the positions of a minimum number of replicas expected to satisfy certain quality requirements. Our placement algorithm computes replica locations by minimising overall replication cost (read and update) while maximising Quality of Service (QoS) satisfaction for a given traffic pattern. Our algorithm also assumes that the workload capacity of each replica server is bounded. The problem is formulated using dynamic programming. We assess our algorithm using OptorSim. A comparison of our algorithm to its QoS-unconstrained counterpart and to two other existing algorithms (Greedy Add and Greedy Remove) shows that our algorithm can shorten job execution time significantly while requiring only moderate network bandwidth.

Management and Placement of Replicas in a Hierarchical Data Grid

In large systems, data management is only possible through the use of replication techniques. In order to improve availability and ensure fast and efficient access, we propose in this paper a dynamic replication algorithm which takes into account also the placement of data in a hierarchical data grid. Experimental results show the effectiveness of our approach in terms of response time and availability of data in the system.

Adaptive Replica Placement in Hierarchical Data Grids

Data grids support distributed data-intensive applications that need to access massive (multi-terabyte or larger) datasets stored around the world. Ensuring efficient and fast access to such widely distributed datasets is hindered by the high latencies of wide-area networks. To speed up access, data files can be replicated so users can access nearby copies. Replication also provides high data availability, decreased bandwidth consumption, increased fault tolerance, and improved scalability. Since a grid environment is highly dynamic, resource availability, network latency, and users requests may change frequently. To address these issues a dynamic replica placement strategy that adapts to dynamic behavior in data grids is needed. In this paper, we extend our earlier work on popularity-based replica placement proposing a new adaptive algorithm for use in large-scale hierarchical data grids. Our algorithm dynamically adapts the frequency and degree of replication based on data access arrival rate and available storage capacities. We evaluate our algorithm using OptorSim. Our results show that our algorithm can shorten job execution time greatly and reduce bandwidth consumption compared to its non-adaptive counterpart which outperforms other existing replica placement methods.

The Impact of Data Replication on Job Scheduling Performance in Hierarchical Data Grid

In data-intensive applications data transfer is a primary cause of job execution delay. Data access time depends on bandwidth. The major bottleneck to supporting fast data access in Grids is the high latencies of Wide Area Networks and Internet. Effective scheduling can reduce the amount of data transferred across the internet by dispatching a job to where the needed data are present. Another solution is to use a data replication mechanism. Objective of dynamic replica strategies is reducing file access time which leads to reducing job runtime. In this paper we develop a job scheduling policy and a dynamic data replication strategy, called HRS (Hierarchical Replication Strategy), to improve the data access efficiencies. We study our approach and evaluate it through simulation. The results show that our algorithm has improved 12% over the current strategies.

Adaptive popularity-driven replica placement in hierarchical data grids

Data grids support access to widely distributed storage for large numbers of users accessing potentially many large files. Efficient access is hindered by the high latency of the Internet. To improve access time, replication at nearby sites may be used. Replication also provides high availability, decreased bandwidth use, enhanced fault tolerance, and improved scalability. Resource availability, network latency, and user requests in a grid environment may vary with time. Any replica placement strategy must be able to adapt to such dynamic behavior. In this paper, we describe a new dynamic replica placement algorithm, Popularity Based Replica Placement (PBRP), for hierarchical data grids which is guided by file "popularity". Our goal is to place replicas close to clients to reduce data access time while still using network and storage resources efficiently. The effectiveness of PBRP depends on the selection of a threshold value related to file popularity. We also present Adaptive-PBRP (APBRP) that determines this threshold dynamically based on data request arrival rates. We evaluate both algorithms using simulation. Results for a range of data access patterns show that our algorithms can shorten job execution time significantly and reduce bandwidth consumption compared to other dynamic replication methods.

A model to predict the optimal performance of the Hierarchical Data Grid

The Data Grid, as a specialization and extension of the Grid, has been adopted as the solution to solve data-intensive problems, such as high energy physics, life science and etc. Large data files are replicated across the Data Grid to improve the system performance. One form of the Data Grid studied is the Hierarchical Data Grid used in LHC experiments. There has been a lot of research work on replication algorithms for the Hierarchical Data Grid. It is important and useful to analyze the optimal performance, as it can be used as a benchmark to compare the performances of different replication algorithms and helps us to understand how good the replication algorithms are in approaching the optimal performance. In this paper, we construct a model to predict the optimal performance of the Hierarchical Data Grid. The performance metrics used for the model are average response time and average bandwidth cost. We propose the optimal replication algorithm and present a detailed analysis of its performance. The Multi-Tier Data Grid is used as a specific example of the Hierarchical Data Grid to verify our model. The performance of the proposed optimal replication algorithm is studied together with three well-known replication algorithms. The simulation results show that the optimal replication algorithm has the best performance and its performance is pretty close to the result obtained using our probabilistic model.

Optimal replica placement in hierarchical Data Grids with locality assurance

In this paper, we address three issues concerning data replica placement in hierarchical Data Grids that can be presented as tree structures. The first is how to ensure load balance among replicas. To achieve this, we propose a placement algorithm that finds the optimal locations for replicas so that their workload is balanced. The second issue is how to minimize the number of replicas. To solve this problem, we propose an algorithm that determines the minimum number of replicas required when the maximum workload capacity of each replica server is known. Finally, we address the issue of service quality by proposing a new model in which each request must be given a quality-of-service guarantee. We describe new algorithms that ensure both workload balance and quality of service simultaneously. We conduct extensive simulation experiments to evaluate the effectiveness of our algorithms. The comparison with the previous Affinity Replica Location Policy demonstrates that our algorithms consistently outperform the heuristic algorithm both in terms of minimum number of replicas used and the actual data transmission time.