Inter-carrier SLA negotiation using Q-learning

Abstract

Inter-domain high performance services (e.g. telepresence) are not sustainable over the current Internet architecture. The Quality of Service (QoS) guarantees they demand require to settle on end-to-end Service Level Agreements (SLAs) among providers (aka. carriers) and across different networks. This process is critical since it must provide the most benefits while dealing with heterogeneous operators' business interests and confidentiality constraints. In this paper, we propose, in the frame of a cooperative organizational model called federation, a composition technique for inter-carrier SLAs that respects end-user's QoS requirements while maximizing network operators' long-term benefits. We formulate the dynamic optimization problem as a Markov Decision process (MDP). This latter allows to provide an iterative near-optimal solution through reinforcement learning (more precisely, Q-learning). The SLA composition is thus performed taking into account customers and network providers' utilities. We also propose a version including several negotiation rounds and observe how it affects the results.