Efficient Fitness Action Analysis Based on Spatio-temporal Feature Encoding

Human action analysis has been an active research area in computer vision. Most of existing approaches are data-driven and focus on general actions. In this paper, we aim to recognize fitness actions from image sequences and propose an action evaluation method, which can be applied in artificial intelligence (AI) fitness system. Firstly, we extract human skeleton information from the captured fitness video with a simplified skeleton model. Secondly, the extracted skeleton images of an action sequence are transformed to an uniform two-dimensional plane with the proposed spatial-temporal skeleton encoding method, which describes a global action feature. Finally, an action classifier and a geometrical registration metric are constructed respectively to analyze the fitness actions. In addition, we build a dataset for fitness actions recognition and evaluation. Experimental results demonstrate that our method has a good performance both on the fitness action dataset and small-scale dataset.