Self-Supervised Adversarial Training

Recent work has demonstrated that neural networks are vulnerable to adversarial examples. To escape from the predicament, many works try to harden the model in various ways, in which adversarial training is an effective way which learns robust feature representation so as to resist adversarial attacks. With this view, we find that self-supervised learning is very suitable for it exploiting internal structures of data. In this paper, we introduce self-supervised learning for defense. Specifically, the self-supervised representation coupled with k-Nearest Neighbour is used for classification. To further strengthen the defense ability, a general framework of self-supervised adversarial training is proposed, which maximizes the mutual information between the representations of original examples and adversarial examples. Experimental results show that the self-supervised representation outperforms its supervised version in respect of robustness and self-supervised adversarial training does improve the defense ability efficiently.