Optimization of Single-user Task Migration based on Improved DDPG

Abstract

Aiming at the problems of slow convergence and unstable convergence of traditional reinforcement learning algorithms in minimizing computational cost on edge servers with random task arrivals and time-varying wireless channels, an improved DDPG algorithm (IDDPG) was proposed. The Critic network structure of DDPG was replaced by the Dueling structure, which converged faster by splitting the state value function into an advantage function and a value function. The update frequency of the Critic network was adjusted to be higher than that of the Actor-network to make the overall training more stable. The Ornstein- Uhlenbeck noise was added to the actions selected through the Actor-network to improve the algorithm exploration ability, and the action noise size was set in segments to ensure the stability of convergence. Experimental results show that, compared with other algorithms, the IDDPG algorithm can better minimize the computational cost and has a certain improvement in the convergence speed and convergence stability.