Exploiting spatio-temporal patterns using partial-state reinforcement learning in a synthetically augmented environment

Abstract

Responding to or anticipating a sequence of events caused by adversarial human actors, such as crimes, can be a difficult task. Reinforcement learning has not been highly utilized as a method for positioning agents to respond to such events. In our earlier work, which was applied to positioning naval vessel agents to respond to Somali maritime piracy attacks, we developed a method to synthetically augment the information in the events’ environment with digital pheromones and other information augmenters, used the resulting augmenter signatures as states that agents could react to, and applied reinforcement learning to exploit regularities in the timing and location of events to position agents in spatio-temporal proximity of anticipated events. This work extends that methodology with a new learning boosting method wherein learning is improved as partial augmenter signatures are reinforced, which is not possible when learning is based only on the aggregated state. The enhanced methodology is applied to positioning police patrols in response to a sequence of business robberies in Denver, Colorado and its effectiveness is analyzed.