TEXT-GUIDED FACIAL IMAGE MANIPULATION FOR WILD IMAGES VIA MANIPULATION DIRECTION-BASED LOSS

This paper proposes a novel text-guided facial image manipulation approach to improve robustness against the diversity of input images. Conventional text-guided facial image manipulation methods have achieved highly accurate image manipulation by restricting the aspect ratio and composition of the input image. In practical face image manipulation, images provided by the user often include regions other than the face, such as the upper body, and thus conventional methods have limitations in their application. To solve these problems, we tackle a new task of text-guided facial image manipulation with no restrictions on the composition and aspect ratio of the input image. As a base structure, we employ the face swapping method, which is robust to the diversity of input images and allows to swap faces. To achieve text-guided image manipulation based on the face swapping method, we also employ the manipulation direction, which focuses on the direction of changes between the image and its corresponding text that occurs before and after image manipulation. Experimental results show the effectiveness of the proposed method on wild images that include regions other than faces and have different aspect ratios.