ST$^{2}$: Spatial-Temporal State Transformer for Crowd-Aware Autonomous Navigation

Abstract

Empowering an intelligent agent with the ability of autonomous navigation in complex and dynamic environments is an important and active research topic in embodied artificial intelligence. In this letter, we address this challenging task from the view of exploiting both the spatial and temporal states of a mobile robot interacting with the crowded environment. Specifically, we propose a Spatial-Temporal State Transformer (ST <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> ) to encode the states while leveraging the deep reinforcement learning method to find the optimal navigation policy accordingly. Technically, the proposed ST <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> model consists of a global spatial state encoder and a temporal state encoder, which are built upon the Transformer structure. The spatial state encoder is devised to extract the global spatial features and capture the spatial interaction between pedestrians and the robot. The temporal state encoder is designed to model the temporal correlation among consecutive frames and infer the dynamic relationship of the spatial position transformation. Based on the comprehensive spatial-temporal state representation, the value-based reinforcement learning method is leveraged to obtain the optimal navigation policy. Extensive experiments demonstrate the superiority of the proposed ST <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> over representative state-of-the-art methods. The source code will be made publicly available.