A Two-branch Network for Video Anomaly Detection with Spatio-temporal Feature Learning

Video anomaly detection is very challenging, as most anomalies are rare and inconclusive. Previous weakly supervised learning approaches utilize the classifier trained with video-level labels to locate anomalous clips from the video. However, the anomalous clips often contain both anomalies and numerous irrelevant background behaviors, increasing the difficulty of localization. In this work, we propose a two-branch network to obtain the global and each local objects' action information of the clip respectively, where the local objects are extracted by a pre-trained object detector. This local-cum-global perception highlights the anomalous features from the background noise. We further propose a spatio-temporal relationship network, which is based on the attention mechanism to model the spatial relations of different objects and the temporal correlations among different clips to efficiently capture the spatio-temporal distribution of anomalies in the video. Extensive experiments on two benchmarks show that our method achieves significant performance gains.