3D Facial Expression Generator Based on Transformer VAE

We present a generative model for the 3D facial expression mesh sequences, from onset to the termination of a desired expression. We tailor a Transformer VAE architecture: The encoder compresses a sequence of facial landmarks into an expression-aware regularized latent space, while the decoder generates a new sequence from the sampled latent variable, conditioned on a desired expression. After a landmark-guided mesh deformation, a given 3D neutral face is driven to an animated mesh sequence with the expected expression. The generated sequences are consistent, of quality, and exhibit a good level of diversity, improving over state-of-the-art methods. We validate our model by conducting extensive experiments on two representative datasets. The supplementary video and code are available on a GitHub page (https://github.com/ZOUKaifeng/FacialExpressionGeneration).