Sampling, Communication, and Prediction Co-Design for Synchronizing the Real-World Device and Digital Model in Metaverse

Abstract

The metaverse has the potential to revolutionize the next generation of the Internet by supporting highly interactive services with satisfactory user experience. The synchronization between devices in the physical world and their digital models in the metaverse is crucial. This work proposes a sampling, communication and prediction co-design framework to minimize the communication load subject to a constraint on the tracking error. To optimize the sampling rate and the prediction horizon, we exploit expert knowledge and develop a constrained deep reinforcement learning algorithm. We validate our framework on a prototype composed of a real-world robotic arm and its digital model. The results show that our framework achieves a better trade-off between the average tracking error and the average communication load compared with a communication system without sampling and prediction. For example, the average communication load can be reduced up to 87% when the average track error constraint is 0.007°. In addition, our policy outperforms the benchmark with the static sampling rate and prediction horizon optimized by exhaustive search, in terms of the tail probability of the tracking error. Furthermore, with the assistance of expert knowledge, the proposed algorithm achieves better convergence time, stability, communication load, and average tacking error.