Partial Expert-Based Adversarial Relay Learning Strategy for Underwater Acoustic Sensor Networks

Abstract

Cooperative transmission is a promising solution for reliable communication in underwater acoustic sensor networks. To fully exploit the benefits of cooperative system, efficient management of relay resources is required. However, the challenging working conditions, including imprecise channel information, limited observation feedback and dynamic underwater environment, call for an adaptive relay selection strategy for improving system capacity performance. In this paper, we present an innovative Partial Expert-based Reinforcement Adversarial Learning (PER-AL) strategy that unifies expert assisted learning and self-adjusting learning in a integral adversarial multi-armed bandit framework. The user constantly interacts with the environment and optimizes the relay decision by analyzing all gathered observation rewards, rather than relying on prior channel information. By introducing a novel expert learning mechanism into the initial learning stage, expert generator and expert learner are proposed to control the performance of experts and provide optimal expert advice to reinforce the initial relay learning for efficiency. With the deepening learning, the relay quality is then evaluated by analyzing the historical observable rewards directly through a Exp3 policy without engaging complexity too much. The proposed algorithm could automatically adapt to the underwater conditions and detect the optimal relay within a short time, significantly improving system capacity without jeopardizing the adversarial regret guarantee. Finally, extensive simulation results demonstrated the superior of PER-AL algorithm compared with recent existing CSI-based and CSI-free solution benchmarks. Particularly, PER-AL algorithm has a 28.3% improvement of the total network capacity and achieves a 44.5% decrease of implementation time.