Computation offloading in blockchain-enabled MCS systems: A scalable deep reinforcement learning approach

Abstract

In Mobile Crowdsensing (MCS) systems, cloud service providers (CSPs) pay for and analyze the sensing data collected by mobile devices (MDs) to enhance the Quality-of-Service (QoS). Therefore, it is necessary to guarantee security when CSPs and users conduct transactions. Blockchain can secure transactions between two parties by using the Proof-of-Work (PoW) to confirm transactions and add new blocks to the chain. Nevertheless, the complex PoW seriously hinders applying Blockchain into MCS since MDs are equipped with limited resources. To address these challenges, we first design a new consortium blockchain framework for MCS, aiming to assure high reliability in complex environments, where a novel Credit-based Proof-of-Work (C-PoW) algorithm is developed to relieve the complexity of PoW while keeping the reliability of blockchain. Next, we propose a new scalable Deep Reinforcement learning based Computation Offloading (DRCO) method to handle the computation-intensive tasks of C-PoW. By combining Proximal Policy Optimization (PPO) and Differentiable Neural Computer (DNC), the DRCO can efficiently make the optimal/near-optimal offloading decisions for C-PoW tasks in blockchain-enabled MCS systems. Extensive experiments demonstrate that the DRCO reaches a lower total cost (weighted sum of latency and power consumption) than state-of-the-art methods under various scenarios.