Scalable Resource Management Research Articles

Topology-aware scalable resource management in multi-hop dense networks

The current society is becoming increasingly interconnected and hyper-connected. Communication networks are advancing, as well as logistics networks, or even networks for the transportation and distribution of natural resources. One of the key benefits of the evolution of these networks is to bring consumers closer to the source of a resource or service. However, this is not a straightforward task, particularly since networks near final users are usually shaped by heterogeneous nodes, sometimes even in very dense scenarios, which may demand or offer a resource at any given moment. In this paper, we present DEN2NE, a novel algorithm designed for the automatic distribution and reallocation of resources in distributed environments. The algorithm has been implemented with six different criteria in order to adapt it to the specific use case under consideration. The results obtained from DEN2DE are promising, owing to its adaptability and its average execution time, which follows a linear distribution in relation to the topology size.

Learning Decentralized and Scalable Resource Management for Wireless Powered Communication Networks

Learning Decentralized and Scalable Resource Management for Wireless Powered Communication Networks

A rack-aware scalable resource management system for Hadoop YARN

A rack-aware scalable resource management system for Hadoop YARN

A rack-aware scalable resource management system for Hadoop YARN

Big data have brought in an era of data exploration and utilisation with MapReduce computational paradigm as its major enabler. Though great efforts through the implementation of Hadoop have made computation scale to tens of thousands of commodity cluster processors, the centralised architecture of resource manager has adversely affected response time in large data centres. The developed model decouples the responsibilities of resource manager by providing another layer where each daemon called rack unit resource manager (RU_RM) carries out the responsibility of allocating resources to compute nodes within its local rack to ensure low latency for large files. The application was developed and tested with Hadoop workload benchmarks used for analysis. Two performance evaluation metrics (efficiency and average task-delay ratio) were used for comparison. Efficiency quantifies average cluster utilisation while average task-delay ratio measures average delay time. Results obtained showed that as file size increases, the developed model outperforms the existing framework.

Moving average fuzzy resource scheduling for virtualized cloud data services

Cloud computing offers simplified system maintenance and scalable resource management with Virtual Machines (VMs). Users access resources of data centers by allocating VMs to hosts. Therefore, to improve the quality of cloud computing environment, not only the conventional multi Quality of Service (QoS) be satisfied, but also specific importance has to be made on certain metrics such as the system accessibility and resource scheduling in a cooperative and dynamic manner. This paper proposes a method called, Moving Average-based Fuzzy Resource Scheduling (MV-FRS) for virtualized cloud environment to optimize the scheduling of resources through virtual machines. Initially, the MV-FRS method starts by predicting the resource (i.e. bandwidth, memory and processing cycle) requirements. Then a measure of relationships between availability of resources and the requirements of resources are made. Finally, a fuzzy control theory is designed to accomplish system accessibility between user cloud requirements and cloud users resources availability. The simulations results demonstrate that the MV-FRS method is able to reduce the total waiting time of cloud user resource requirements and also ensure the feasibility and effectiveness of the overall system accessibility in terms of average success rate and resource usage when running in a cloud computing environment.

Resource Management in Clouds: Survey and Research Challenges

Resource management in a cloud environment is a hard problem, due to: the scale of modern data centers; the heterogeneity of resource types and their interdependencies; the variability and unpredictability of the load; as well as the range of objectives of the different actors in a cloud ecosystem. Consequently, both academia and industry began significant research efforts in this area. In this paper, we survey the recent literature, covering 250+ publications, and highlighting key results. We outline a conceptual framework for cloud resource management and use it to structure the state-of-the-art review. Based on our analysis, we identify five challenges for future investigation. These relate to: providing predictable performance for cloud-hosted applications; achieving global manageability for cloud systems; engineering scalable resource management systems; understanding economic behavior and cloud pricing; and developing solutions for the mobile cloud paradigm .

HiCOO: Hierarchical cooperation for scalable communication in Global Address Space programming models on Cray XT systems

Global Address Space (GAS) programming models enable a convenient, shared-memory style addressing model. Typically this is realized through one-sided operations that can enable asynchronous communication and data movement. With the size of petascale systems reaching 10,000s of nodes and 100,000s of cores, the underlying runtime systems face critical challenges in (1) scalably managing resources (such as memory for communication buffers), and (2) gracefully handling unpredictable communication patterns and any associated contention. For any solution that addresses these resource scalability challenges, equally important is the need to maintain the performance of GAS programming models. In this paper, we describe a Hierarchical COOperation (HiCOO) architecture for scalable communication in GAS programming models. HiCOO formulates a cooperative communication architecture: with inter-node cooperation amongst multiple nodes (a.k.a multinode) and hierarchical cooperation among multinodes that are arranged in various virtual topologies. We have implemented HiCOO for a popular GAS runtime library, Aggregate Remote Memory Copy Interface (ARMCI). By extensively evaluating different virtual topologies in HiCOO in terms of their impact to memory scalability, network contention, and application performance, we identify MFCG as the most suitable virtual topology. The resulting HiCOO architecture is able to realize scalable resource management and achieve resilience to network contention, while at the same time maintaining or enhancing the performance of scientific applications. In one case, it reduces the total execution time of an NWChem application by 52%.

Service Middleware for Self-Managing Large-Scale Systems

Resource management poses particular challenges in large-scale systems, such as server clusters that simultaneously process requests from a large number of clients. A resource management scheme for such systems must scale both in the in the number of cluster nodes and the number of applications the cluster supports. Current solutions do not exhibit both of these properties at the same time. Many are centralized, which limits their scalability in terms of the number of nodes, or they are decentralized but rely on replicated directories, which also reduces their ability to scale. In this paper, we propose novel solutions to request routing and application placement- two key mechanisms in a scalable resource management scheme. Our solution to request routing is based on selective update propagation, which ensures that the control load on a cluster node is independent of the system size. Application placement is approached in a decentralized manner, by using a distributed algorithm that maximizes resource utilization and allows for service differentiation under overload. The paper demonstrates how the above solutions can be integrated into an overall design for a peer-to-peer management middleware that exhibits properties of self-organization. Through complexity analysis and simulation, we show to which extent the system design is scalable. We have built a prototype using accepted technologies and have evaluated it using a standard benchmark. The testbed measurements show that the implementation, within the parameter range tested, operates efficiently, quickly adapts to a changing environment and allows for effective service differentiation by a system administrator.

A scalable intra-domain resource management architecture for DiffServ networks

With the rapid growth of the Internet into a global communication and commercial infrastructure, the need for Quality of Services (QoS) in the Internet becomes more and more important. With a Bandwidth Broker (BB) support in each administrative domain Differentiated Services (DiffServ) is seen as a key technology for achieving QoS guarantees in a scalable, efficient, and deployable manner in the Internet.This paper presents the design and implementation of a new Bandwidth Broker (BB) model to achieve QoS across a DiffServ domain. Our BB uses centralized network state maintenance and pipe-based intra-domain resource management schemes. The proposed model significantly reduces admission control time and minimizes scalability problems present in prior research while optimizing network resource utilization. The experimental results verify the achievements of our model.

Scalable resource management for high‐performance Web servers

AbstractAlthough many research efforts have been devoted to network congestion in the face of an increase in the Internet traffic, there is little recent discussion on performance improvements for endhosts. In this paper, we propose a new architecture, called scalable socket buffer tuning (SSBT), to provide high‐performance and fair service for many TCP connections at Internet endhosts. SSBT has two major features. One is to reduce the number of memory accesses at the sender host by using some new system calls, called Simple Memory‐copy Reduction (SMR) scheme. The other is equation‐based automatic TCP buffer tuning (E‐ATBT), where the sender host estimates ‘expected’ throughput of the TCP connections through a simple mathematical equation, and assigns a send socket buffer to them according to the estimated throughput. If the socket buffer is short, max–min fairness policy is used. We confirm the effectiveness of our proposed algorithm through both simulation technique and an experimental system. From the experimental results, we have found that our SSBT can achieve up to a 30% gain for Web server throughput, and a fair and effective usage of the sender socket buffer can be achieved. Copyright © 2004 John Wiley & Sons, Ltd.