Abstract
Urban network traffic congestion can be caused by disturbances, such as fluctuation and disequilibrium of traffic demand. This paper designs a distributed control method for preventing disturbance-based urban network traffic congestion by integrating Multi-Agent Reinforcement Learning (MARL) and regional Mixed Strategy Nash-Equilibrium (MSNE). To enhance the disturbance-rejection performance of Urban Network Traffic Control (UNTC), a regional MSNE concept is integrated, which models the competitive relationship between each agent and its neighboring agents in order to improve the decision-making process of MARL. The learning rate is enhanced with a self-adaptive ability to avoid a local optimal dilemma; Jensen-Shannon (JS) divergence is utilized to define the learning rate of the modified MARL. A two-way rectangular grid network with nine intersections is modeled via a Cell Transmission Model (CTM). A probability distribution mechanism, which can update the turn ratio of each approach dynamically and discretely, is established to represent the segmented route-decision process of the vehicles. The effectiveness of the proposed control method is evaluated through simulations in the grid network. The results show the influence of major disturbances, such as fluctuation of vehicle arrival rate, fluctuation of traffic demand (e.g. a rapidly rising flow and extreme changes in origin-destination distribution), and disequilibrium of traffic demand (e.g. different arrival flows at each boundary of the urban network), on the performance of the suggested control method. The results can be used to improve the state of the art in order to reduce urban network traffic congestion due to these disturbances.
Highlights
In urban networks, traffic congestion can occur for various reasons [1], including traffic incidents, constraints on network capacity or stochastic fluctuations in demand
In this article, a distributed control method has been presented for Urban Network Traffic Control (UNTC) based on the principle of Multi-Agent Reinforcement Learning (MARL) and Game Theory (GT) to prevent disturbance-based traffic congestion in an urban network
We have focused on improving the learning rate, as this can affect the global search ability of Mixed Strategy Nash-Equilibrium (MSNE)-MARL and the effectiveness of MARL in facing stochastic fluctuations
Summary
Traffic congestion can occur for various reasons [1], including traffic incidents, constraints on network capacity or stochastic fluctuations in demand. (3) The Jensen-Shannon (JS) divergence is introduced to define the learning rate of MARL to provide self-adaptive ability to manage new scenarios generated in the urban network This method enhances the sensitivity of MARL after convergence and improves the accumulation of new experiences when faced with state transitions. The learning rate was given a gradually decreasing form which considers both the rapid replacement of incorrect experiences in the early stage of learning and stability due to convergence in the later stage of learning with the iterative process revised by an embedding algorithm, such as Simulated Annealing (SA) [53] The Q-learning algorithm performed by each agent can be summarized in ALGORITHM 1 which is O (n × T )
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
