Scaling DNN-based Video Analysis by Coarse-grained and Fine-grained Parallelism

Deep neural networks have been widely used in video analysis applications such as automatic metadata generation, action recognition, and video summarization. A fundamental module in the pipeline of such DNN-based applications is feature extraction. However, extracting features for videos is a major bottleneck since it is performed on every frame of each video sequentially. In addition, the long training time of these complex networks also hinders their usability. In this work, we identify fine-grained and coarse-grained parallelism techniques to speed up vital components in video analysis applications through inter-frame and intra-video parallelism. We demonstrate these techniques on the feature extraction and summarization modules. We leverage frame-level parallelism in feature extraction and intra-video parallelism to speed up video summarization and implement them in a distributed environment using Hadoop Map-Reduce framework to get a speed up of 2.67X on a 4-node setup. Furthermore, we show in our results that our approach has similar accuracy to the sequential applications.