DeepSensing: A Novel Mobile Crowdsensing Framework With Double Deep Q-Network and Prioritized Experience Replay

Abstract

Mobile crowdsensing (MCS) is a new and promising paradigm of data collection due to the growing number of mobile smart devices. It can be utilized in applications of large-scale sensing by employing a group of mobile users with their smart devices. Since a large number of mobile users are recruited, the allocation of sensing tasks to mobile users has a critical influence on the performance of MCS applications. To efficiently assign sensing tasks to mobile users, we propose a novel MCS framework named DeepSensing. This framework consists of six executive phases, i.e., registration of sensing tasks, the announcement of reward rule, collection of users' information, task allocation, execution of sensing activities, and distribution of data and rewards. Here, the phase of task allocation is a key component, which directly determines the performance of DeepSensing, e.g., the platform's profit. DeepSensing aims to maximize the platform's profit by taking into account the various constraints of sensing tasks and mobile users. Therefore, we propose a deep reinforcement learning (DRL) method to optimally assign sensing tasks to mobile users. Specifically, we employ a double deep Q-network with prioritized experience replay (DDQN-PER) to address the task allocation problem, which is also formulated as a path planning problem with time windows. To evaluate our proposed DDQN-PER solution, three baseline solutions are provided, i.e., the ant colony system (ACS), E-greedy, and random solutions. Finally, the results of numerical simulations show that our proposed DDQN-PER solution outperforms the baseline solutions in terms of the platform's profit and it plans better organized traveling paths for mobile users.