Learning Intelligent Controls in High Speed Networks: Synergies of Computational Intelligence with Control and Q-Learning Theories

Abstract

The phenomena of congestion and packet-drops in high-speed communications and computer networks do affect the quality-of-service and overall performance more than ever existing time-delays with uncertain variations. Their control and, possibly, prevention are subject to extensive research ever since the internet is available. Because of the uncertainties and time-varying phenomena, obtaining the accurate and complete information on the network traffic patterns, especially for the multi-bottleneck case, is rather difficult hence learning intelligent controls are needed. One such control is a multi-agent flow controller (MAFC) based on Q-learning algorithm in conjunction with the theory of Nash equilibrium of opponents’ strategies. The other is a model-independent Q-learning control (MIQL) scheme having focus on the flow with higher priority, which also does not need prior-knowledge on communication traffic and congestion. The competition of communication flows with different priorities is considered as a two-player non-cooperative game. The Nash Q-learning algorithm control obtains the Nash Q-values through trial-and-error and interaction with the network environment so as to improve its behaviour policy. The MAFC can learn to take the best actions in order to regulate source flows that guarantee high throughput and low packet-loss ratio. The MIQL control, through a specific learning processing, does achieve the optimum sending rate for the sources with lower priority while observing the sources with higher priority. Designed intelligent controls achieve superior performances in controlling the flows in high-speed networks in comparison to the standard ones and avoid communications congestion.