Graph-Based MDP to Mobile Source with Energy Harvesting in Delay Tolerant Networks System

Abstract

We consider a mobile delay tolerant networks (MDTNs) with energy harvesting capabilities. In order to determine energy management policies that will improve network capacity, packet delivery ratio and maximize the system throughput. we consider that a source node seeks to send packets to a destination node. The optimal policy for the source varies according to its system state, which allows it to guarantee a maximum delivery probability rate. Our problem is modeled by decision theory; as a start, We are interested in the MDP, which are used to model and solve such sequential decision problems. Our goal is to optimize, for each node, a utility depending on a random environment and decisions made by the node. As the MDP formalism reaches its limits when it is necessary to take into account the interactions between the different several nodes, we will start using the Graph-based MDP where the state and action spaces are factorizable by variables. The transition functions and rewards are then decomposed into local functions and the dependency relations between the nodes are represented by a graph. To calculate the optimal policy, we propose Mean Field Approximation (MFA) and Approximate linear-programming (ALP) algorithms for solving GMDP problem.