Towards autonomous data ferry route design through reinforcement learning

Abstract

Communication in delay tolerant networks can be facilitated by the use of dedicated mobile ldquoferriesrdquo which physically transport data packets between network nodes. The goal is for the ferry to autonomously find routes which minimize the average packet delay in the network. We prove that paths which visit all nodes in a round-trip fashion, i.e., solutions to the traveling salesman problem, do not yield the lowest average packet delay. We propose two novel ferry path planning algorithms based on stochastic modeling and machine learning. We model the path planning task as a Markov decision process with the ferry acting as an independent agent. We apply reinforcement learning to enable the ferry to make optimal decisions. Simulation experiments show the resulting routes have lower average packet delay than solutions known to date.