Model-Based Thompson Sampling for Frequency and Rate Selection in Underwater Acoustic Communications

Abstract

Due to the harsh propagation environment, limited bandwidth, and constrained battery life, transmission efficiency is a crucial issue for underwater acoustic (UWA) communications. This paper studies the link adaptation problem of a single UWA link by jointly selecting the transmission frequency and data rate. Since the current UWA channel lacks a universal model, we formulate this joint selection problem as a model-based stochastic multi-armed bandit (SMAB) problem. Thereafter, we propose three algorithms to solve this model-based SMAB problem under the settings of the stationary channel, non-stationary channel, and large arm (i.e., frequency and rate pair) space. For the stationary channel, we propose a unimodal objective-based Thompson sampling (UO-TS) algorithm by exploiting the unimodal feature of the objective function. For the non-stationary channel, we put forth a hybrid change detection UO-TS (HCD-UO-TS) algorithm based on the features of the unimodal objective function and non-stationary channel. For the large arm space, we propose an iterative boundary-shrinking TS (IBS-TS) algorithm by using the logistic regression-based arm classification model. These algorithms are all data-driven and have low complexity and a fast convergence rate. In addition, we derive an upper regret bound for the UO-TS algorithm. Numerical results show that the proposed algorithms outperform the state-of-the-art bandit algorithms and are not sensitive to the arm space.