Information Elevation Network for Online Action Detection and Anticipation

Abstract

Given a partially observed video segment, online action detection and anticipation aim to identify a current action and forecast future actions, respectively. To detect actions in a streaming video for monitoring applications including surveillance, robot assistants, and autonomous driving, online action detection methods have been proposed. Considering the importance of current action in online action detection, we introduce a novel information elevation unit (IEU) that lifts and accumulates the past information relevant to the current action, to compensate for forgotten essential information. Using the IEUs, we propose an information elevation network (IEN) that effectively identifies a current action and anticipates future actions through the dense prediction of past and current action classes within the video segment. For its practical use in online monitoring applications, our IEN takes visual features extracted from a fast action recognition using only RGB frames because extracting optical flows requires heavy computation overhead. On THUMOS-14 and TVSeries, our IEN out-performs state-of-the-art methods using only RGB frames. Furthermore, on the THUMOS-14 dataset, our IEN outperforms the state-of-the-art methods.