IMAGE-COUPLED VOLUME PROPAGATION FOR STEREO MATCHING

Several leading depth-from-stereo methods rely on memory-intensive 4D cost volumes and computationally intensive 3D convolutions for feature matching. We propose to integrate two independent concepts into a new 4D cost volume to achieve a symbiotic relationship. A feature matching part is responsible for identifying matching pixel pairs along the baseline, while a concurrent image volume part is inspired by depth-from-mono CNNs. More technically, the processing of the 4D cost volume is divided into a 2D propagation and a 3D propagation part. Starting from the feature maps of the left image, the 2D propagation, instead of directly predicting depth, supports the 3D propagation part of the cost volume at different levels by adding visual features to the geometric context. Combining the two parts reduces the number of the 3D convolution layers without sacrificing accuracy. Experiments show that our end-to-end trained CNN ranks 2nd on the KITTI2012 and ETH3D benchmarks, while being significantly faster than the 1st ranked method. The source code is available at https://github.com/ohkwon718/icvp.