CHANNEL PRUNING VIA ATTENTION MODULE AND MEMORY CURVE

As an effective pruning method, dynamic pruning introduces gate modules that allow different input data to choose different channels. This shows that the choice of channels strongly depends on data. However, this creates an additional computational burden because of the gate modules. In this paper, we propose a simple, efficient and transferable channel pruning method via attention module and memory curve, dubbed as CPAM, which not only takes advantage of the strong correlation between data and channels, but also does not impose any additional computational burden on the model. Inspired by the memory curve, we use a progressive method without any sparse operation. Moreover, our method has been demonstrated effective for many advanced CNN architectures. Notably, on CIFAR-10, CPAM reduces 50% FLOPs on ResNet-56 with 0.31% relative accuracy improvement, which has advanced the state-of-the-art.