Extending the learning and communicating shout-ahead architecture with user-defined exception rules -- a case study for traffic light controls

Abstract

We present an extension of the shout-ahead agent architecture that allows for adding human user-defined exception rules to the rules created by the hybrid learning approach for this architecture. The user-defined rules can be added after learning as reaction to weaknesses of the learned rules or learning can be performed with the user-defined rules already in place. We applied the extended shout-ahead architecture and the associated learning to a new application area, cooperating controllers for the traffic lights of intersections. In our experimental evaluations, adding user-defined exception rules to the learned rules for several traffic flow instances increased the efficiency of the resulting controllers substantially compared to just using the learned rules. Performing learning with user-defined exception rules already in place decreased the learning time substantially for all flows, but had mixed results with respect to efficiency. We also evaluated user-defined exception rules for a variant of the architecture that is not using communication and saw similar effects as for the variant with communication. For the communicating version, both variants of adding user-defined exception rules create controllers that are much more flexible than what using the original shout-ahead architecture with its learning is able to create as indicated by experiments with variations of flows.