Autonomous intersection driving with neuro-evolution

Abstract

Recently there has been increasing research attention focused on producing adaptive control systems for autonomous vehicles. To accommodate such autonomous vehicles there have been proposals that current road and highway infrastructure undergo significant changes. For example, replacing traffic lights and stop signs and allowing autonomous vehicles to coordinate their interactions so as to avoid collisions and safely navigate through intersections [8]. In the context of Intelligent Transportation Systems, Dresner and Stone [4] proposed a new automated intersection management system called Autonomous Intersection Management (AIM) for autonomous vehicles. AIM used a First Come, First Served (FCFS) policy for directing vehicles through intersections. Intersection management simulations demonstrated AIM as out-performing current intersection control including traffic lights and stop signs, in terms of increased traffic throughput and decreased delays. A key limitation of AIM and the FCFS protocol, is that perfect traffic flow conditions and vehicle sensory information is assumed. AIM does not generally account for uncertain and unpredictable traffic conditions or dynamic obstacles [5], such as pedestrians. Such unpredictable behavior, incomplete information and noisy sensory environments must be appropriately handled if autonomous vehicles are to be successfully implemented. Another approach to automated intersection management that potentially handles such problems is to use decentralized control where each vehicle's controller automatically adapts as vehicles interact with their environment. That is, to automate the synthesis of vehicle controllers such that when vehicles interact a desired collective behavior emerges for any given road environment. Neuro-Evolution (NE) [6] has been used to evolve controllers in land-based vehicles that accomplish various tasks [3], [9], [11]. However, there has been little work on evolving coordinated movement for maximizing traffic flow through intersections. This study used NE to synthesize collective driving behaviors for given road networks (interconnected intersections), where there were no traffic signals to assist with vehicle coordination and navigation. Rather, NE automates controller design where collective driving behavior emerges in response to the task of maximizing traffic throughput and minimizing delays at intersections. The first research objective was to demonstrate the efficacy of NE for collective driving behavior synthesis, where task performance is average vehicle throughput and idle time on road networks of interconnected intersections. The second objective evaluated the efficacy of NE versus centralized heuristic controllers for autonomous intersection management. Neuro-Evolution of Augmenting Topologies (NEAT) [10] was used for controller evolution. Groups of vehicles were behaviorally and morphologically homogenous in that one evolved ANN controller and one sensor configuration was used by all vehicles. As a benchmark for the intersection management task, a modified version of AIM and the FCFS protocol [4] was comparatively tested and evaluated on the same road networks. Vehicles followed pre-planned routes through intersections, where vehicles continuously circled a given road network1 and average vehicle throughput, speed and idle time was calculated. Experiments tested 48 autonomous vehicles in 3D simulations2 of traffic passing through 10 road networks of interconnected intersections (modeled after real traffic intersections major metropolitan areas). NEAT and AIM were evaluated on increasingly difficult road networks. Task difficulty was equated with the number of start and end points, the number of lanes per road and hence the number of vehicles that could concurrently enter an intersection. For each road network, the task was to automate the coordination of N vehicles, each following their own preset path through a road network. The goal of NEAT and AIM was to maximize average vehicle throughput and thus minimize average vehicle idle time. Methods for automated intersection management were evaluated and compared as follows. The AIM controller was run on each of the 10 road networks and an average vehicle throughput (over 20 runs) calculated. For NEAT, the fittest controller was selected from 20 evolutionary runs and set as the vehicles' controller on each of the 10 tracks, where the same evaluation procedure was used. Figure 1 presents average (normalized) vehicle throughput across all road networks tested. Overall results indicate that AIM using the FCFS protocol for intersection traffic management yields a high average vehicle throughput. However, NEAT out-performed AIM on specific types of road networks and otherwise yielded comparable vehicle throughput. Statistical tests indicated that the fittest NEAT evolved controllers yielded a significantly higher average vehicle throughput on road networks 3 (traffic-circle), 6 (eight-way intersection) and 10 (double lane merge and one-way intersection exits)1. Thus, AIM did not always produce optimal vehicle throughput. Average vehicle throughput results (figure 1) indicate that AIM (using the FCFS protocol) is not as well suited to handling networks of intersections that include road features such as in road networks 3, 6 and 10. Such road networks are conducive to high traffic flow meaning that vehicles entering intersections will on average wait for longer periods before there is a clear path. In the case of intersections in seven of tested road networks, NEAT evolved controllers yielded no advantage over the centralized controller of AIM. However in the case of intersections with many entry and exit points and connecting one-way roads, the NEAT evolved controller was better able to handle increased traffic flow and traffic congestion in the intersections. That is, NEAT controllers evolved sensory-motor correlations such that all vehicles moved collectively and in close proximity to each other when passing through intersections. In these simulations, NEAT was able to leverage few of the benefits associated with using NE to adapt vehicle controllers. That is, NE is best suited to evolve controllers to adapt to dynamic, noisy task environments, where controllers must process incomplete sensory information [6], into appropriate motor outputs. Importantly, such conditions were not present in the task environments (road networks) tested in this study. That is, the intersection management task assumed that there was no vehicle sensor noise or sensor failures, no uncertainty in vehicle operations (such as mechanical failures [1]), and no unpredictability in traffic conditions (such as pedestrians). Intersection management tasks with these types of conditions favor an AIM controller. This study's results corroborate the benefits of using AIM with the FCFS protocol for specific types of intersections [4], [7], [2], but also demonstrate the efficacy of using NE to automate intersection management. NEAT evolved controllers yielded significantly higher average vehicle speed for nine often tested road networks, higher vehicle throughput on three road networks, and comparable vehicle throughput on other road networks. To the best of the authors' knowledge this is the first study that has compared AIM (with the FCFS protocol) as a centralized heuristic based approach, with NE evolved controllers, as a decentralized evolutionary approach. An important caveat to this study was that it assumed the vehicles operated in perfect traffic conditions. Current work on this topic is investigating the efficacy of NE for evolving controllers given increasing levels of unpredictable behavior on road networks. For example, uncertainty will be introduced as sensor noise, pedestrians crossing roads and intersections at random locations as well as obstacles appearing on the roads.