Filter Pruning Via Softmax Attention

In this paper, we propose a novel network pruning method using the proposed relative depth-wise separable convolutions and softmax attention channel pruning. The relative depthwise separable convolution enhances conventional depth-wise separable convolutions by enabling the channel interaction, which can prevent accuracy drops even after severe pruning. The softmax attention channel pruning probabilistically expresses the importance of filters and removes unimportant channels efficiently. Experimental results demonstrate that our pruning method outperforms other state-of-the-art pruning methods in terms of Flops, parameters, and top-1 classification accuracy.