A Fast Algorithm for Large-Scale MDP-Based Systems in Smart Grid

Hua Xiao,Qicong Peng,Yingjie Zhou,Fan Yang,Huai-Zong Shao

doi:10.19026/rjaset.5.5107

Abstract

In this study, we investigate the fast algorithms for the Large-Scale Markov Decision Process (LSMDP) problem in smart gird. Markov decision process is one of the efficient mathematical tools to solve the control and optimization problems in wireless smart grid systems. However, the complexity and the memory requirements exponentially increase when the number of system state grows in. Moreover, the limited computational ability and small size of memory on board constraint the application of wireless smart grid systems. As a result, it is impractical to implement those LSMDP-based approaches in such systems. Therefore, we propose the fast algorithm with low computational overhead and good performance in this study. We first derive the factored MDP representation, which substitutes LSMDP in a compact way. Based on the factored MDP, we propose the fast algorithm, which considerably reduces the size of state space and remains reasonable performance compared to the optimal solution.

Highlights

Smart grid (Li et al, 2010; Moghe et al, 2012; Moslehi et al, 2010) is a promising technology to improve the efficiency and reliability of the generation, transmission and distribution of electricity services
We focus on the MDP-based control and optimization problems in wireless smart grid systems
We derive the fast algorithm based on the factored model, which remains good performance compared to the optimal solution

Summary

Introduction

Smart grid (Li et al, 2010; Moghe et al, 2012; Moslehi et al, 2010) is a promising technology to improve the efficiency and reliability of the generation, transmission and distribution of electricity services. Due to the time-varying nature of wireless channel and the complicated actions in smart grid, Markov decision process becomes an ideal mathematical tool to model and solve related problems in such scenarios. We focus on the MDP-based control and optimization problems in wireless smart grid systems. The optimal transmission power and rate policies are proposed base on the semi-Markov decision process.

Results

Conclusion