TSOR: Thompson Sampling-Based Opportunistic Routing

Abstract

Routing is a fundamental problem and has been extensively studied in various networks. However, in highly dynamic networks (e.g., wireless ad hoc networks), nodes have limited transmission opportunities due to high mobility, noise and interference, where traditional routing is often not the best approach. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Opportunistic routing (OR)</i> , on the other hand, can effectively minimize the routing cost (e.g., the number of hops) and improve the success of routing by utilizing link metrics. However, the link metrics are usually unknown in advance and changing. In this paper, we design an adaptive algorithm called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Thompson sampling-based opportunistic routing (TSOR)</i> motivated by the distributed Bellman-Ford algorithms. TSOR is able to learn the link metrics and route packets simultaneously to reduce the overall cost. Theoretically, we show a lower bound and an upper bound of the cumulative regret (i.e., performance gap) between TSOR and the optimal routing algorithm that knows all link metrics in advance. The regret increases sublinearly with respect to the number of packets, and has a lower order in terms of the network size than the best-known results. Furthermore, we compare TSOR with the state-of-the-art algorithms, and the evaluation results show that TSOR has a lower regret and a faster convergence rate to the optimal policy than the state-of-the-art algorithms.