Quadric Lyapunov Algorithm for Stochastic Networks Optimization with Q-learning Perspective

Abstract

In this article, we investigate stochastic networks optimization using Quadric Lyapunov Algorithm (QLA) with Q-learning perspective. We proposed firstly a model of stochastic queueing networks with power constraints. QLA is then proposed aiming at minimizing an expression containing Lyapunov drift. Based on the analysed similarity between QLA and Q-learning, we show the possibility and feasibility of Q-learning. Simulation of a simple queue network model is carried out, and results using both QLA and Q-learning are compared.