Non-Uniform Video Time-Lapse Method Based On Motion Scenario And Stabilization Constraint

Time-lapse of user captured video becomes popular in many applications recently, non-uniform sampling and digital video stabilization (VS) are usually two independent steps to keep meaningful contents and provide stabilized output. However, non-uniform sampling may produce large sampling interval and then result in larger motion, this would beyond the stabilization ability of VS and produce unpleasant output. To address this problem, we propose a new auto time-lapse framework, which selects frames not only based on camera motion scenarios, but also refer to the smoothed camera trajectory and considering VS ability according to Field of View (FOV) loss constraint. More specific, we introduce an advanced Markov chain (MC) model, in which smoothed camera trajectory, FOV loss constraint of VS, camera motion scenario, and sampling interval similarity between consecutive frames are encoded as potential functions. Finally, dynamic programming (DP) is employed to find the best non-uniform sampling. Experimental results demonstrate that the proposed method not only achieves pleasant and meaningful non-uniform sampling, but also provides satisfactory stabilization performance. The whole algorithm can work in real time during video recording on mobile device.