Learning-Based Real-Time Transmission Control for Multi-Path TCP Networks

Abstract

The evolution of the Internet of Things and 5G/6G networks is ushered in the exponential proliferation of mobile devices with multiple wireless interfaces. Multi-Path TCP (MPTCP) is a widely used protocol that provides reliable connections with multiple parallel paths for these devices. To cope with the increasingly complex multi-path network environments, some learning-based methods have been proposed to achieve better control than existing heuristic methods in congestion control and multi-path scheduling. However, most learning-based efforts use Deep Reinforcement Learning (DRL) requiring the agent and the environment to evolve alternatively, which cannot adapt to the constantly active MPTCP networks. In this paper, we propose Real-time Multi-path Transmission Control (RMTC), a cognitive framework that achieves real-time congestion control and multi-path scheduling in MPTCP networks, allowing the agent and the environment to evolve simultaneously. It adopts the Multi-Agent DRL (MADRL) to improve the granularity of the control and handle the change in the number of sub-flows. Besides, RMTC develops a pacing mechanism in MPTCP connections for all congestion control algorithms to alleviate network fluctuations, which helps the agent make accurate decisions. Extensive experimental results show that RMTC obtains better network performance than existing learning-based or heuristic methods and considers TCP-friendliness and the Head-of-Line (HoL) blocking problems.