STACKED MULTI-SCALE ATTENTION NETWORK FOR IMAGE COLORIZATION

Deep convolutional networks (CNNs) show their potential in image colorization for producing plausible results. Recently, the attention mechanism further boosts the performances of CNNs by constructing channel and spatial interactions. However, existing attention methods are performed in a single-scale manner, which is hard to capture multi-scale information interactions in a limited computational cost. This can limit the performance of the network to reconstruct color channels. In this paper, we propose a stacked multi-scale attention network (SMSANet) for image colorization. The core idea is to perform the attentions of a feature map in a multi-scale manner so that sufficient interactions are conducted to efficiently and adaptively capture multi-scale and long-range dependencies. By stacking the multi-scale attention layers in different convolutional layers, the SMSANet can focus on more discriminative features to reconstruct color channels. Moreover, a salient loss is designed to further refine the generated image at both pixel-level and object-level. Extensive experiments on the ImageNet dataset have demonstrated that SMSANet outperforms the state-of-the-art automatic colorization methods.