SAE-PPL: Self-guided attention encoder with prior knowledge-guided pseudo labels for weakly supervised video anomaly detection

Abstract

Recently, many weakly supervised video anomaly detection (WS-VAD) methods focus on generating reasonable pseudo-labels for frames by using video-level annotations. To achieve this goal, some existing label-noise cleaning techniques and pseudo-label generators are used by them, but their performance is limited. The main reason is that the useful prior knowledge, i.e., the graduality of anomaly events, is ignored. Here, we propose a Self-Training Framework, which assumes flexible soft boundaries between abnormal and normal clips. The framework consists of: (1) A prior knowledge guided pseudo-label generator that incorporates prior knowledge of video segment distributions into the MIL framework to generate high-confidence pseudo labels; (2) An improved self-guided attention encoder is developed to capture multiscale long-term spatiotemporal features, where dependencies among anomaly frames are preserved. Moreover, a pseudo-label based self-training scheme is adopted to supervise the encoder. Experimental results verify the superiority of our method over baseline approaches.