Replica Placement Problem Research Articles

A hybrid chaos-based algorithm for data object replication in distributed systems

One of the primary challenges in distributed systems, such as cloud computing, lies in ensuring that data objects are accessible within a reasonable timeframe. To address this challenge, the data objects are replicated across multiple servers. Estimating the minimum quantity of data replicas and their optimal placement is considered an NP-complete optimization problem. The primary objectives of the current research include minimizing data processing costs, reducing the quantity of replicas, and maximizing the applied algorithms’ reliability in replica placement. This paper introduces a hybrid chaos-based swarm approach using the modified shuffle-frog leaping algorithm with a new local search strategy for replicating data in distributed systems. Taking into account the algorithm’s performance in static settings, the introduced method reduces the expenses associated with replica placement. The results of the experiment conducted on a standard data set indicate that the proposed approach can decrease data access time by about 33% when using approximately seven replicas. When executed several times, the suggested method yields a standard deviation of approximately 0.012 for the results, which is lower than the result existing algorithms produce. Additionally, the new approach’s success rate is higher in comparison with existing algorithms used in addressing the problem of replica placement.

DATA REPLICATION IN DISTRIBUTED SYSTEMS USING OLYMPIAD OPTIMIZATION ALGORITHM

Achieving timely access to data objects is a major challenge in big distributed systems like the Internet of Things (IoT) platforms. Therefore, minimizing the data read and write operation time in distributed systems has elevated to a higher priority for system designers and mechanical engineers. Replication and the appropriate placement of the replicas on the most accessible data servers is a problem of NP-complete optimization. The key objectives of the current study are minimizing the data access time, reducing the quantity of replicas, and improving the data availability. The current paper employs the Olympiad Optimization Algorithm (OOA) as a novel population-based and discrete heuristic algorithm to solve the replica placement problem which is also applicable to other fields such as mechanical and computer engineering design problems. This discrete algorithm was inspired by the learning process of student groups who are preparing for the Olympiad exams. The proposed algorithm, which is divide-and-conquer-based with local and global search strategies, was used in solving the replica placement problem in a standard simulated distributed system. The 'European Union Database' (EUData) was employed to evaluate the proposed algorithm, which contains 28 nodes as servers and a network architecture in the format of a complete graph. It was revealed that the proposed technique reduces data access time by 39% with around six replicas, which is vastly superior to the earlier methods. Moreover, the standard deviation of the results of the algorithm's different executions is approximately 0.0062, which is lower than the other techniques' standard deviation within the same experiments.

Scalable and Adaptive Data Replica Placement for Geo-Distributed Cloud Storages

In geo-distributed cloud storage systems, data replication has been widely used to serve the ever more users around the world for high data reliability and availability. How to optimize the data replica placement has become one of the fundamental problems to reduce the inter-node traffic and the system overhead of accessing associated data items. In the big data era, traditional solutions may face the challenges of long running time and large overheads to handle the increasing scale of data items with time-varying user requests. Therefore, novel offline community discovery and online community adjustment schemes are proposed to solve the replica placement problem in a scalable and adaptive way. The offline scheme can find a replica placement solution based on the average read/write rates for a certain period of time. The scalability can be achieved as 1) the computation complexity is linear to the amount of data items and 2) the data-node communities can evolve in parallel for a distributed replica placement. Furthermore, the online scheme is adaptive to handle the bursty data requests, without the need to completely override the existing replica placement. Driven by real-world data traces, extensive performance evaluations demonstrate the effectiveness of our design to handle large-scale datasets.

A novel replica placement algorithm for minimising communication cost in distributed storage platform

In large-scale distributed systems, replication service has been playing a critical role to improve the availability and reliability of user data. Conventionally, the existing replication services mainly concentrate on how many replicas are needed to maintain desirable availability and reliability rather than how to place replicas on the most suitable storage nodes. As a result, the communication-related costs when accessing data are significantly increased, which in turn degrades the execution performance of user applications. In this paper, we propose a novel replica placement algorithm which is designed to minimise the communication cost when accessing or managing replicas in a large-scale storage platform. In the proposed algorithm, the replica placement problem is formulised a classical multi-knapsack problem, and two heuristic metrics are introduced to obtain the sub-optimal solution of this problem. A lot of experiments are conducted to investigate the performance of the proposed algorithm. The experimental results indicate that our replica placement algorithm outperforms many existing approaches in terms of different performance metrics. In addition, the proposed algorithm can also significantly improve the execution efficiency for data-intensive applications, which are very common in nowadays large-scale distributed systems, such as grid and cloud.

A novel replica placement algorithm for minimising communication cost in distributed storage platform

In large-scale distributed systems, replication service has been playing a critical role to improve the availability and reliability of user data. Conventionally, the existing replication services mainly concentrate on how many replicas are needed to maintain desirable availability and reliability rather than how to place replicas on the most suitable storage nodes. As a result, the communication-related costs when accessing data are significantly increased, which in turn degrades the execution performance of user applications. In this paper, we propose a novel replica placement algorithm which is designed to minimise the communication cost when accessing or managing replicas in a large-scale storage platform. In the proposed algorithm, the replica placement problem is formulised a classical multi-knapsack problem, and two heuristic metrics are introduced to obtain the sub-optimal solution of this problem. A lot of experiments are conducted to investigate the performance of the proposed algorithm. The experimental results indicate that our replica placement algorithm outperforms many existing approaches in terms of different performance metrics. In addition, the proposed algorithm can also significantly improve the execution efficiency for data-intensive applications, which are very common in nowadays large-scale distributed systems, such as grid and cloud.

A QoS-enhanced data replication service in virtualised cloud environments

In recent cloud systems, data-intensive applications have been widely deployed for solving non-trivial problems, which often involve large amount of dataset. As a standard service for offering data availability and reliability, data replication service plays an important role and is proven to be effective in many real-world cloud environments. However, many existing replication services are designed for improving the system-oriented metrics instead of user-oriented metrics. In this paper, we first formulate the quality-of-service (QoS) constrained replication placing problem in different manners and then a novel replication service is proposed which uses multi-objective technique to solve the replica placement problem via a time-efficient approach. A set of experiments are conducted to evaluate the effectiveness and efficiency of the proposed service and the results indicate it can significantly reduce the average response time metric comparing with other existing replication services and therefore improve the QoS for user applications. In addition, it also exhibits a good robustness when the cloud storage system is facing intensive replication workloads.

A QoS-enhanced data replication service in virtualised cloud environments

In recent cloud systems, data-intensive applications have been widely deployed for solving non-trivial problems, which often involve large amount of dataset. As a standard service for offering data availability and reliability, data replication service plays an important role and is proven to be effective in many real-world cloud environments. However, many existing replication services are designed for improving the system-oriented metrics instead of user-oriented metrics. In this paper, we first formulate the quality-of-service (QoS) constrained replication placing problem in different manners and then a novel replication service is proposed which uses multi-objective technique to solve the replica placement problem via a time-efficient approach. A set of experiments are conducted to evaluate the effectiveness and efficiency of the proposed service and the results indicate it can significantly reduce the average response time metric comparing with other existing replication services and therefore improve the QoS for user applications. In addition, it also exhibits a good robustness when the cloud storage system is facing intensive replication workloads.

Dynamic multi-objective optimized replica placement and migration strategies for SaaS applications in edge cloud

Abstract The multi-replica strategy widely used in the traditional cloud makes storage costly and the replica cannot be accurately located. Due to many heterogeneous nodes in different locations of the edge cloud system, inefficient replica strategy will greatly affect the load balancing of the system. Therefore, the dynamic multi-objective optimized replica placement and migration strategies for SaaS applications in edge cloud are proposed in this paper. The multi-objective optimized replica placement problem is solved according to the fast non-dominated sorting genetic algorithm. In the experiment, the algorithm of replica placement will minimize the effective network utilization, reduce the response time and guarantee the load balancing of data nodes. Moreover, the replica migration model for access hotspots is proposed to obtain the pairing migration relationship from source node to target node. In the experiment, the replica migration algorithm will effectively reduce the migration time, minimize the response time and improve network resource utilization. Besides, the applicability of the proposed replica methods is shown in different scenarios such as the social media, intelligent agriculture, military field and transportation field.

Write-Aware Replica Placement for Cloud Computing

Cloud applications such as virtualized network functions usually rely on static or dynamic replication mechanisms to achieve high service reliability and satisfy the requirements of service level agreement. Because replicas are allocated in geographically distributed datacenters, their updates increase the difficulty of maintaining consistency among replicas in different datacenters. The consistency overheads among replicas increase bandwidth consumption and transmission latencies to degrade the quality of service (QoS) performance and service reliability. Although previous mechanisms attempt to shorten response time, reduce bandwidth consumption, or maintain high service reliability with minimized storage cost, they do not consider the overhead of data consistency. In this paper, we study the problem of replica placement with consistency overheads by analyzing the latency and reliability for different types of operations. A replication algorithm, write-aware replica placement (WARP), is then presented based on the analysis. The simulation results show that WARP can shorten response time and improve QoS without degrading reliability.

Multiobjective Glowworm Swarm Optimization-Based Dynamic Replication Algorithm for Real-Time Distributed Databases

Distributed systems offer resources to be accessed geographically for large-scale data requests of different users. In many cases, replication of the vital data files and storing their replica in multiple locations accessible to the requesting clients is vital in improving the data availability, reliability, security, and reduction of the execution time. It is important that real-time distributed databases maintain the consistency constraints and also guarantee the time constraints required by the client requests. However, when the size of the distributed system increases, the user access time also tends to increase, which in turn increases the vitality of the replica placement. Thus, the primary issues that emerge are deciding upon an optimal replication number and identifying perfect locations to store the replicated data. These open challenges have been considered in this study, which turns to develop a dynamic data replication algorithm for real-time distributed databases using a multiobjective glowworm swarm optimization (MGSO) strategy. The proposed algorithm adapts the random patterns of the read-write requests and employs a dynamic window mechanism for replication. It also models the replica number and placement problem as a multiobjective optimization problem and utilizes MGSO for resolving it. The cost models are presented to ensure the time constraint satisfaction in servicing user requests. The performance of the MGSO dynamic data replication algorithm has been studied using competitive analysis, and the results show the efficiency of the proposed algorithm for the distributed databases.

Block-secure: Blockchain based scheme for secure P2P cloud storage

With the development of Internet technology, the volume of data is increasing tremendously. To tackle with large-scale data, more and more applications choose to enlarge the storage capacity of users’ terminals with the help of cloud platforms. Before storing data to an untrusted cloud server, some measures should be adopted to guarantee the data security. However, the communication overhead will increase dramatically when users transmit files encrypted by a traditional encryption scheme. In this paper, we address the above problems by proposing a blockchain-based security architecture for distributed cloud storage, where users can divide their own files into encrypted data chunks, and upload those data chunks randomly into the P2P network nodes that provide free storage capacity. We customize a genetic algorithm to solve the file block replica placement problem between multiple users and multiple data centers in the distributed cloud storage environment. Numerical results show that the proposed architecture outperforms the traditional cloud storage architectures in terms of file security and network transmission delay. On average, the file loss rate based on the simulation assumptions utilized in this paper is close to 0% on our architecture while it’s nearly 100% and 71.66% on the architecture with single data center and the distributed architecture using genetic algorithm. Besides, with proposed scheme, the transmission delay on the proposed architecture is reduced by 39.28% and 76.47% on average on the user’s number and the number of file block replicas, respectively, in comparison to the architecture with single data center. Meanwhile, the transmission delay of file block replicas is also reduced by 41.36% on average than that on the distributed architecture using genetic algorithm.

A Genetic Algorithm Based Data Replica Placement Strategy for Scientific Applications in Clouds

Cloud computing is a promising distributed computing platform for big data applications, e.g., scientific applications, since excessive resources can be obtained from cloud services for processing and storing both existing and generated application datasets. However, when tasks process big data stored in distributed data centers, the inevitable data movements will cause huge bandwidth cost and execution delay. In this paper, we construct a tripartite graph based model to formulate the data replica placement problem and propose a genetic algorithm based data replica placement strategy for scientific applications to reduce data transmissions in cloud. Our approach can reduce 1) the size of moved data, 2) the time of data movement and 3) the number of movements. We conduct experiments to compare the proposed strategy with the random placement strategy used in Hadoop Distributed Files System (HDFS), which demonstrates that our strategy has better performance for scientific applications in clouds.

Manycast, Anycast, and Replica Placement in Optical Inter-Datacenter Networks

The expanding adoption of cloud-based services in recent years puts stringent requirements on datacenters (DCs) and their interconnection networks. Optical inter-datacenter networks represent the only viable option for satisfying the huge bandwidth required to replicate and update content for cloud-based services across geographically dispersed datacenters. In addition to content replication and synchronization, optical inter-datacenter networks must also support communication between datacenters and end-users. The resulting new traffic patterns and the enormous traffic volumes call for new capacityefficient approaches for inter-datacenter network designs that incorporate both transport and datacenter resource planning. This paper introduces an integrated approach to optimally place content replicas across DCs by concurrently solving the routing and wavelength assignment (RWA) problem for both inter-DC content replication and synchronization traffic following the manycast routing paradigm, and end-user-driven user-to-DC communication following the anycast routing paradigm, with the objective to reduce the overall network capacity usage. To attain this goal, the manycast, anycast, and replica placement (MARP) problem is formulated as an integer linear program to find optimal solutions for smaller problem instances. Due to the problem complexity, a scalable and efficient heuristic algorithm is developed to solve larger network scenarios. Simulation results demonstrate that the proposed integrated MARP strategy can significantly reduce the network capacity usage when compared to the benchmarking replica placement and RWA schemes aimed at minimizing the resources consumed by either of the two types of traffic independently.

OPTIMIZING THE STORAGE SPACE AND COST WITH RELIABILITY ASSURANCE BY REPLICA REDUCTION ON CLOUD STORAGE SYSTEM

Content Distribution Networks have been attracted a great deal of attraction in recent years on cloud computing. Replica placement problems (RPPs) as one of the key technologies in the Content Distribution Networks have been widely studied. The internet services are hosted by multiple geographically distributed datacenters. For the increasingly expanded utility of Cloud storage, the improvement of resources management and reduces the storage space, cost is complicated issues on data replication. So in order to reduce the data replication, this paper, proposed the concept of Reliability Assurance algorithm (RA). The RA is reducing the file replication from the server and improves the reliability of the server. The RA algorithm identified the data replication on un-accessing files. The unpredicted files called as replicated files, these files occupy a more space on cloud server. The low ranking prediction algorithm identified the unpredicted files on cloud server based on file accessing. Reduced data replication on cloud server it’s allows optimizing the storage space and cost. Using the RA and Ranking algorithm combined to reduce the data replication and storage space.

Mechanism design for Data Replica Placement (DRP) problem in strategic settings

ABSTRACTThis article addresses the problem of Data Replica Placement, an important technique used in storage-capable distributed networks to improve system availability, reliability, and fault-tolerance. The study focuses on the Data Replica Placement problem in strategic settings inspired by practical market-based data replication applications, such as content delivery networks. Multiple self-interested players with private preferences own data objects for replication. Players compete for storage space among replication servers for placing replicas with the objective to optimize their own profits. Using mechanism design approach, the authors consider the problem as a sequential composition of knapsack auctions and design an algorithmic mechanism DRPMECH to aggregate players’ preferences and approximate a social efficient allocation for the problem. This work analyzes both the economic and computational properties of DRPMECH, validates the properties using experiments, and compares its performance against...

Effect of selective replication strategy and dynamic admission control to QoS management in video on demand systems

Abstract Several solutions have been proposed till date to achieve effective replication policies and admission control methods that can ensure a high Quality of Service (QoS) to Video on Demand (VoD) systems. In this paper, we have chosen to study admission control and a replication strategy in a video on demand system called Feedback Control Architecture for Distributed Multimedia Systems (FCA-DMS). The proposed approaches are founded on (i) the supervision and auto-adaptation of the system load, (ii) continued knowledge of the network and of the video servers’ workload, and (iii) the development of a dynamic replication strategy of segments for popular video, that remade to replica-placement problem by discussing the problem of the replicas number to create (Replication Degree). The workload of video servers and of network are calculated by a QoS controller. The proposed approach takes into account of replica detail and offers a replica removal policy to avoid waste and for a good management of storage space. The goal is to maintain an updated broad view of the system’s behaviour from the collected measurements to databases of available videos and databases of available video servers. To demonstrate the efficiency and feasibility of our approaches and validate the results obtained, we conducted a series of experiments based on a simulator that we implemented.

Power-aware replica placement in tree networks with multiple servers per client

In this paper, we revisit the well-studied problem of replica placement in tree networks. Rather than minimizing the number of servers needed to serve all client requests, we aim at minimizing the total power consumed by these servers. In addition, we use the most general (and powerful) server assignment policy, where the requests of a client can be served by multiple servers located in the (unique) path from this client to the root of the tree. We consider multi-modal servers that can operate at a set of discrete speeds, using the dynamic voltage and frequency scaling (DVFS) technique. The optimization problem is to determine an optimal location of the servers in the tree, as well as the speed at which each server is operated. A major result is the NP-completeness of this problem, to be contrasted with the minimization of the number of servers, which has polynomial complexity. Another important contribution is the formulation of a Mixed Integer Linear Program (MILP) for the problem, together with the design of several polynomial-time heuristics. We assess the efficiency of these heuristics by simulation. For mid-size instances (up to 30 nodes in the tree), we evaluate their absolute performance by comparison with the optimal solution (obtained via the MILP). The most efficient heuristics provide satisfactory results, within 20% of the optimal solution.

A new hybrid heuristic for replica placement and request distribution in content distribution networks

In Content Distribution Networks (CDN), in order to better serve clients, it is necessary to replicate contents at surrogate servers and distribute requests using such servers. The decisions of where to place replicated contents and how to distribute clients can be modeled as an optimization problem known as Replica Placement and Request Distribution Problem (RPRDP). In this paper we use a model that regards several realistic details that are not treated simultaneously in the literature, such as: constraints in server disk space and bandwidth, QoS requirements of requests and changes in the network conditions. Also, a new hybrid method, that uses exact and heuristic concepts simultaneously, is presented to solve the RPRDP. We compared the results obtained by the proposed algorithm with: a bound obtained by an exact offline approach, a solution of a real CDN provider and other hybrid heuristics. Results show that the proposed method outperforms the solution used in real CDNs and also in all other studied methods.

The dynamic replica placement problem with service levels in content delivery networks: a model and a simulated annealing heuristic

Dynamically assigning copies (replicas) of internet files to regionally placed servers is a crucial planning problem for content delivery network providers. Their goal is to minimize placement, storage and delivery costs while fulfilling service-level agreements. Although there is considerable work on simplified versions of this problem such as the static case, the dynamic problem with all three cost types and service-level constraints has yet not been considered in the literature. This paper provides a mixed integer programming (MIP) formulation for the problem as well as a simulated annealing metaheuristic. A computational study is employed to assess the simulated annealing heuristic and the MIP employing a state-of-the-art solver.

스마트 네트워크 환경에서의 자원 및 경로 최적화 연구

본 연구에서는 네트워크 총 비용 최소화의 관점에서 스마트 노드 자원사용과 네트워크 트래픽 경로를 함께 결정하는 최적화 문제를 고려하였다. 이를 위해, 스마트 노드의 기술 추세와 자원 최적화에 대한 분석방법, 스마트 네트워크의 경제적 효과와 CDN에 대해 살펴보았다. CDN에 대한 분석을 기반으로 혼합정수계획법 모형을 제안하였으며, 이는 기존에 알려진 복제위치선정과 고객요청 분배문제 (RPRDP)와 경로 결정문제가 결합된 형태로 볼 수 있다. 제안된 혼합정수계획법 모형을 구현하고 그 결과를 소개함으로써 제안된 모형의 유효성을 밝혔다. In this research, we dealt with an optimization problem which aims to minimize total cost of resources usage and network routing in a smart node environment. Toward this, we briefly introduced technology trends in smart nodes, methods in resource optimization fields, economic effects of smart network and content delivery (or distribution) network (CDN). Moreover, based on CDN we proposed and implemented a mathematical (mixed integer) programming model to combine replica placement and requests distribution problem (RPRDP) and routing problem. Computational result of an example on RPRDP+Routing problem is also provided.