Low-frequency Guided Self-supervised Learning for High-fidelity 3D Face Reconstruction in the Wild

In this paper, we propose a low-frequency guided self-supervised learning method for high-fidelity 3D face reconstruction from an in-the-wild image.
Unlike other self-supervised methods only using the color difference between the original image and the estimated image, we add low-frequency albedo information to enhance the self-supervised learning for more realistic albedo while insensitive to the non-skin regions.
Specifically, based on a PCA albedo model, we first train a Boosting Network (B-Net) to provide illumination and intact albedo distribution. Then with the those information, we learn an image-to-image non-linear Facial Albedo Network (FAN) by self-supervision to form a high-fidelity albedo.
We further propose a Detail Recovering Network (DRN) to recover geometric details such as wrinkles.
FAN and DRN permit to reconstruct 3D faces with high-fidelity albedo and geometry details.
Finally, experimental results demonstrate the effectiveness of the proposed method.