Abstract
Servers in a data center are underutilized due to over-provisioning, which contributes heavily toward the high-power consumption of the data centers. Recent research in optimizing the energy consumption of High Performance Computing (HPC) data centers mostly focuses on consolidation of Virtual Machines (VMs) and using dynamic voltage and frequency scaling (DVFS). These approaches are inherently hardware-based, are frequently unique to individual systems, and often use simulation due to lack of access to HPC data centers. Other approaches require profiling information on the jobs in the HPC system to be available before run-time. In this paper, we propose a reinforcement learning based approach, which jointly optimizes profit and energy in the allocation of jobs to available resources, without the need for such prior information. The approach is implemented in a software scheduler used to allocate real applications from the Princeton Application Repository for Shared-Memory Computers (PARSEC) benchmark suite to a number of hardware nodes realized with Odroid-XU3 boards. Experiments show that the proposed approach increases the profit earned by 40% while simultaneously reducing energy consumption by 20% when compared to a heuristic-based approach. We also present a network-aware server consolidation algorithm called Bandwidth-Constrained Consolidation (BCC), for HPC data centers which can address the under-utilization problem of the servers. Our experiments show that the BCC consolidation technique can reduce the power consumption of a data center by up-to 37%.
Highlights
High Performance Computing (HPC) data centers typically contain a large number of computing nodes each consisting of multiple processing cores
We evaluate the network-level performance with the consolidation algorithm in an HPC data center with network-level simulations
Instead of Bandwidth-Constrained Consolidation (BCC), if Clustered Exhaustive Search (CES) or greedy approach based consolidation (GRD) consolidation was implemented, at lower injection rates, there was no significant difference in achieved throughput for both S2S-WiDCN and fat-tree networks compared to BCC
Summary
High Performance Computing (HPC) data centers typically contain a large number of computing nodes each consisting of multiple processing cores. In HPC systems, the scheduling of jobs is influenced by their value; typically a resource management system will attempt to maximize its profits by allocating its limited resources to the highest-value jobs in the queue This is especially true when jobs arrive at a rate higher than the rate at which the system can process and execute them. Server-centric wireless DCNs where direct wireless links are used for server-to-server communication have been designed [24,25] These wireless data center architectures can be considered as viable alternate for traditional wired architecture for HPC computing for reducing even more power consumption.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have