Dynamic Routing for Military Network Based on Semi-Markov Prediction and Q-Learning

Abstract

AbstractDue to the constantly moving of the communication nodes in the combat environment, the network topology is highly dynamic and uncertain. However, combat missions require the real-time and accurate information transmission through the military network. Therefore, it is of great practical interest to design a dynamic routing algorithm to accommodate to the dynamic and uncertain network topology. We consider this important problem in this paper and make the following contributions. First, a semi-Markov prediction model is proposed to predict the uncertainty of the network topology. Second, a dynamic resource table is updated by the formation transition probability which can describe the dynamic of the network topology. Third, the Q-learning method is applied to find an optimal routing in the uncertain and dynamic network topology. Numerical experiments demonstrate that the proposed method can predict the network topology more accurate and obtain an optimal routing strategy with improved transmission efficiency and reduced transmission delay.KeywordsMilitary networksSemi-MarkovTopology predictionQ-learningDynamic routing