NVM-enhanced Machine Learning Inference in 6G Edge Computing

Abstract

With the increasing popularization of smart terminals and real-time interactive applications, fast growing technical requirements push both academia and industry to look beyond 5G and conceptualize the sixth generation (6G) mobile network. Artificial intelligence (AI) with machine learning capacities at the edge is one crucial component of a 6G mobile network which makes various time-sensitive and high-stake services possible, e.g., smart security, virtual reality, self-driving vehicles. However, resource constraints, especially the memory limitation of edge servers, become major obstacles to deploying machine learning services at the edge. Fortunately, the new generation of non-volatile memory (NVM) provides new affordable memory resources that can be easily attached to existing edge servers. In this paper, we propose a novel machine learning application placement scheme using the NVM technology at the edge to reduce the end-to-end latency. Specifically, the proposed NVM-enhanced placement scheme takes into consideration the latency of various machine learning applications over NVM devices and the network. The corresponding optimization problem is exceedingly challenging, i.e., NP-hard. Therefore, we developed a novel approximation algorithm with both low computational complexity and theoretical guarantees. Experiments and extensive simulations using real-world applications highlight that our scheme provides significantly lower end-to-end latency compared with existing baselines.