Smart Congestion Control for Delay- and Disruption Tolerant Networks

Abstract

In this paper, we propose a novel congestion control framework for delay- and disruption tolerant networks (DTNs). The proposed framework, called Smart-DTN-CC, adjusts its operation automatically as a function of the dynamics of the underlying net- work. It employs reinforcement learning, a machine learning technique known to be well suited to problems in which the environment, in this case the network, plays a crucial role; yet, no prior knowledge about the target environment can be assumed, i.e., the only way to acquire information about the environment is to interact with it through continuous online learning. Smart-DTN-CC nodes get input from the environment (e.g., its buffer occupancy, set of neighbors, etc), and, based on that information, choose an action to take from a set of possible actions. Depending on an action's effectiveness in controlling congestion, it will be given a reward. Smart-DTN-CC's goal is to maximize the over- all reward which translates to minimizing congestion. To our knowledge, Smart-DTN-CC is the first DTN congestion control framework that has the ability to automatically and continuously adapt to the dynamics of the target environment. As demonstrated by our experimental evaluation, Smart-DTN- CC is able to consistently outperform existing DTN congestion control mechanisms under a wide range of network conditions and characteristics.