Distributed cross-layer resource allocation using Correlated Equilibrium based stochastic learning

Abstract

In recent years, cross-layer Resource Management (RM) has been widely considered as an efficient approach for improving the network performance. However, due to the factors such as limited knowledge about the wireless environment and spontaneous characteristics of device behaviors, designing an efficient distributed RM scheme using the cross-layer paradigm becomes extremely difficult. In this paper, a distributed, cross-layer RM scheme for multi-users transmitting scalable video in an ad-hoc CDMA network is presented. For the cross-layer RM design, an end-to-end performance criterion at the APP layer is applied and the resource allocations in the PHY, LINK and APP layers are unified into a single decision process. Due to the dynamic property of the wireless channels and the uncoordinated user interactions, the distributed RM decision is modeled in the framework of stochastic game. In the game, each device gradually learns its policy using a distributed, Correlated Equilibrium (CE) based Q-Learning algorithm. In order to calculate the CE in a distributed way, the local devices apply the regret-matching based mechanism for policy and state-action value updating. Simulation experiments show that with the proposed resource management scheme, the system performance can be improved by about 15% compared to the layered RM scheme and 10% compared to the RM scheme purely based on the local-level learning.