Adaptive switching for communication profiles in underwater acoustic modems based on reinforcement learning

Abstract

The underwater acoustic (UWA) channel is a complex and stochastic process with large spatial and temporal dynamics. This work studies the adaptation of the communication strategy to channel dynamics. Specifically, a set of communication strategies are considered, including Frequency-Hopped Binary Frequency Shift Keying (FH-BFSK), Single-Carrier (SC) communication, and multicarrier communication. Based on the channel condition, a reinforcement learning (RL) algorithm, the Deep Deterministic Policy Gradient (DDPG) method along with a Gumbel-softmax scheme is employed for intelligent and adaptive switching among those communication strategies. The adaptive switching is performed on a transmission block-by-block basis, with the goal of maximizing long-term system performance. The reward function is defined based on the energy efficiency (EE) and the spectral efficiency (SE) of the communication strategies. Simulation results and experimental data processing results reveal that the proposed method outperforms a random selection method and a direct feedback method in time-varying channels.