Pan-sharpening based on parallel pyramid convolutional neural network

Existing deep learning-based pan-sharpening methods mainly learn spatial information from a high-resolution (HR) panchromatic (PAN) image for each spectral channel. However, due to the own characteristics of remote sensing image data, the spatial information of PAN image often shows weak correlation with some spectral channel, especially for channels non-overlapped by PAN channel. In this paper, we propose a parallel pyramid network (PPN) for pan-sharpening. First, a three-branch parallel structure is proposed for dealing with PAN image detail, multispectral (MS) images detail and spectral property respectively. Second, pyramid network structure is introduced in two detail branches to solve the problem of weak correlation due to scale difference. Third, the feature level fusion in two detail branches is implemented, which utilizes redundancy between channels to solve detail representation of channels non-overlapped by PAN channel. The qualitative and quantitative experimental results on various data sets demonstrate the superiority of our proposed method over the state-of-the-art methods.