MASK-BASED ATTENTION PARALLEL NETWORK FOR IN-THE-WILD FACIAL EXPRESSION RECOGNITION

Facial expression recognition suffers big pose and occlusion in real world and attention mechanism is deployed widely to cope with these challenges. But most previous attention-based methods are inadequate in locating crucial expression-related regions precisely and capturing useful facial expression features comprehensively. For these reasons, we present a novel mask-based attention parallel network (MAPNet). Firstly, mask-based attention module that locates expression-related regions is constructed from binary mask extracted by key landmark detection. Secondly, the designed parallel network embeds mask-based attention modules into its different layers to acquire comprehensive facial expression features. Thirdly, the extracted parallel features are divided into several detached blocks from spatial dimension to predict facial expression independently. Finally, the expression label is acquired by combining two predictions of the parallel network and a new loss function is designed to weigh unbalanced facial expression distribution. We validate our method on three popular in-the-wild datasets and the results demonstrate that our MANPnet outperforms previous state-of-the-art methods among RAFDB, AffectNet and FEDRO.