Edge Intelligence Empowered Cross-Modal Streaming Transmission

Abstract

Cross-Modal services, including audio, video, and haptic signals, have gradually been the core components of multimedia applications. Unfortunately, owing to stringent transmission requirements of haptic signals and varying, even conflicting, communication qualities among these heterogeneous streams, how to ensure concurrent cross-modal streaming transmission has been the significant technical challenge. To get over this dilemma, this work proposes an edge intelli-gence-empowered cross-modal streaming transmission architecture, which takes full advantage of communication, caching, computation, and control capabilities (4C). In this architecture, we first introduce artificial intelligence (AI) into 4C for further performance improvement, including secure communication, efficient caching, and collaborative computation. Then, the highlight of this work lies in deriving a control model for the joint optimization problem formulation of communication, caching, and computation, which aims to enable the architecture to be adaptive to dynamic network conditions, various service scenarios, and heterogeneous streams. Finally, we explore the autonomous transmission decision for this problem through attention-based deep reinforcement learning (A-DRL). Importantly, experimental results validate the efficiency of the proposed cross-modal streaming transmission architecture.