PERCEPTION-ORIENTED OMNIDIRECTIONAL IMAGE SUPER-RESOLUTION BASED ON TRANSFORMER NETWORK

Omnidirectional image (ODI) super-resolution (SR) is an important technique in augmented reality and virtual reality applications to address the low-resolution problem caused by limitations in capturing devices or bandwidth. The ODI projection distortion makes it challenging to apply existing SR methods. In this paper, we propose an ODI SR method by leveraging the characteristics of ODIs and human visual characteristics. Specifically, we firstly design a perception-orientated adaptive loss function by jointly utilizing saliency map and latitude map. In our proposed ODI-SR network, we introduce an attention module to aggregate multi-scale information and leverage spherical convolution to adapt to the spheric format of ODIs. Furthermore, we design a data augmentation strategy for ODIs according to viewpoint distribution to further improve the visual quality of SR images. Extensive experimental results demonstrate that the proposed method achieves state-of-the-art performance according to both qualitative and quantitative evaluations.