Deformable cross attention for learning optical flow

Optical flow is the process of estimating motion in scenes. Each object in the scene has a homogeneous motion, i.e., moves in the same direction with the same velocity. There-fore, connecting the parts of an image globally provides essential cues for learning an accurate motion. Convolution-based methods estimate the motion features from the local regions, which miss this important cue. Recently, some methods used Transformers to model global dependencies in order to improve optical flow. However, Transformers suffer excessive attention computations and still bring irrelevant parts into the region of interest. Therefore, we proposed a deformable cross-attention for optical flow estimation, which provides two important advantages: connecting the parts of the image globally while deforming the attention to the objects’ shapes in the image and reducing the memory consumption. Our pro-posed method achieved superior performance on the Sintel and KITTI 2015 datasets in terms of accuracy and efficiency.