Multiagent Cooperative Caching Policy in Industrial Internet of Things

Abstract

To handle the impact of the explosive growth of data traffic generated by smart industrial applications in the Industrial Internet of Things (IIoT) scenario, edge caching is commonly used in the IIoT to reduce the content access delay and to release the backhaul load. However, due to the mobility of IIoT devices and temporal dependence of popularity, traditional content caching policies, such as least frequently used (LFU) and least recently used (LRU), cannot cache the required contents accurately within the coverage of edge server. Therefore, by exploiting the moving trajectory of the IIoT devices and the temporal dependence of the content popularity, we propose a multiagent cooperative caching policy, in which each edge server acts as an agent to cooperatively learn the optimal caching decision, then each edge server caches the corresponding contents to reduce the content access delay. Especially, we first apply the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> -order Markov chain to predict the moving trajectory of the IIoT devices to get the IIoT device set within the coverage of each edge server. Second, we use long short-term memory (LSTM) to get the prior knowledge of the content requests by using the moving trajectory prediction results. Finally, we obtain the optimal caching decision by deep reinforcement learning to improve the Quality of Service (QoS) of the IIoT applications. Experimental results demonstrate that the proposed caching policy can efficiently improve the cache-hit ratio and content access delay.