Optimized Look-Ahead Offloading Decisions Using Deep Reinforcement Learning for Battery Constrained Mobile IoT Devices

Abstract

Mobile Internet of things (MIoT) applications serving smart cities bring the promise of innovative and enhanced public services such as air pollution monitoring, enhanced road safety and city resources metering and management. These applications rely on a number of energy-constrained MIoT units (MUs) (e.g., robots and drones) to continuously sense, capture and process data and images from their environments to produce immediate adaptive actions. In this paper, we consider a scenario where a battery constrained MU executes a number of time- sensitive data processing tasks whose arrival times and sizes are stochastic in nature. We first formulate the problem of making optimal offloading decisions that minimize the cost of current and future tasks as a constrained Markov decision process (CMDP) that accounts for the constraints of the MU battery and the limited reserved resources on the MEC infrastructure by the application providers. Next, we show how the CMDP problem can be relaxed using Lagrangian primal-dual optimization. We then develop a novel deep reinforcement learning (DRL) based algorithm to train the MU to solve the relaxed problem. Simulation results validate the proposed algorithm and demonstrate the achieved performance improvement.