Simulated Annealing for Timeliness and Energy aware Deep Learning Job Assignment

Abstract

In this paper, we propose a simulated annealing method for timeliness and energy aware deep learning (DL) job assignment. In the proposed method, we make a decision of server position for DL training jobs in order to consider both the timeliness of deep neural network (DNN) model updating and the associated energy consumption. For timeliness management, we design three penalty functions; step, linear, and exponential functions, so as to practically penalize the service quality degradation of DL inference due to delay of DL training. For energy management, we formulate the operation cost optimization problem considering energy consumption for DL training. By using these management schemes, our method is able to find the trade-off of the timeliness and energy consumption of DL training. Especially, in order to overcome the non-smoothness of the defined problem, we design a simulated annealing (SA) based metaheuristic which finds approximated optimal solution for DL training job assignment. For the performance evaluation, we show the preliminary experimental results of DL training jobs with AlexNet, Inception-V3, and ResNet on NVIDIA GPU devices.