Automatic Key Moment Extraction and Highlights Generation based on Comprehensive Soccer Video understanding

The massive growth of sports video has resulted in a need for automatic generation of sports highlights that are comparable in quality to the hand-edited highlights produced by editors. Many methods have been applied to this task and have achieved some positive results. Unlike previous works ignoring the multi-modality information, we propose a novel system that leverages visual and audio information derived from the soccer video. The proposed system involves three crucial tasks which can be jointly used to produce highlights automatically, i.e. play-back detection, soccer event recognition and commentator emotion classification. We introduce a new dataset of 460 soccer games totaling 700 hours with a benchmark for three tasks. Making use of recent progress in deep learning, we further provide strong baselines on three tasks. The experiments on the proposed dataset demonstrate state-of-the-art performance on
each independent task. The real-world deployment shows that this system can be useful for soccer games to find and extract soccer video highlights.