Profit: <underline>Pr</underline>iority and P<underline>o</underline>wer/Per<underline>f</underline>ormance Opt<underline>i</underline>miza<underline>t</underline>ion for Many-Core Systems

Abstract

As power density emerges as the main constraint for many-core systems, controlling power consumption under the thermal design power while maximizing the performance becomes increasingly critical. To dynamically save power, dynamic voltage frequency scaling techniques have proved to be effective and are widely available commercially. Meanwhile, systems have certain performance constraints that the applications should satisfy to ensure quality of service. In this paper, we present an online distributed reinforcement learning (OD-RL)-based DVFS control algorithm for many-core system performance improvement under both power and performance constraints. At the finer grain, a per-core RL method is used to learn the optimal control policy of the voltage/frequency (VF) levels in a model-free manner. At the coarser grain, an efficient global power budget reallocation algorithm is used to maximize the overall performance. The experiments show that compared to the state-of-the-art algorithms: 1) OD-RL produces up to 98% less budget overshoot; 2) up to 23% higher energy efficiency; and 3) two orders of magnitude speedup over state-of-the-art techniques for systems with hundreds of cores. Furthermore, priority-aware OD-RL can better satisfy performance constraints than OD-RL with: 1) $17.8\boldsymbol {\times }$ more epochs satisfying the performance constraints; 2) $5.6\boldsymbol {\times }$ better performance gain; and 3) $20.0\boldsymbol {\times }$ better performance-power tradeoffs under similar efficiency and scalability.