Background-Weakening Consistency Regularization for Semi-Supervised Video Action Detection

Consistency-based techniques have produced state-of-the-art results in semi-supervised action detection. When the model false detects the dynamic information in the background as an action, spatio-temporal consistency calculations can hardly reflect this misdetection result. We consider weakening the dynamic information in the augmented video background to reduce its spatio-temporal consistency with the dynamic information in the original video background. We propose a Background-Weakening with Calibration Constraint (BWCC) framework, which highlights the negative impact of information in the background of misdetection on detection by calculating the consistency of the predictions of the background weakened video and the original video. Specifically, Background Weaken (BW) module judges the foreground and background of the video based on the initial predictions of the model and makes adjustments to the video background. Misjudgements may result in weakened action pixels. We additionally introduce a model that does not undergo background weakening to aid training through Calibration Constraint (CC) module. We demonstrate the effectiveness of the proposed approach on two action detection datasets, UCF101-24 and JHMDB-21.