N2MVSNet: Non-local Neighbors Aware Multi-View Stereo Network

Learning-based multi-view stereo (MVS) methods have been widely studied recently. However, current works are limited to using fixed-size convolution kernels, leading to suboptimal features that lack anisotropy in low-textured regions and tend to produce invalid depth blending at the edge of the foreground and background. In this paper, we propose N2MVSNet, which learns adaptive non-local neighbors matching (ANNM) and their spatial impact to overcome these deficiencies. Furthermore, we explore the ability of spatial perception to depth dimension and propose 3D ANNM. Besides, following the coarse-to-fine scheme, severe mismatchings in coarse stages will result in error accumulation and propagation in finer stages. To this end, we adopt the pre-trained RGB guided depth refinement for depth hypothesis repolish. The robustness of the training process is further elevated by the energy aggregation loss. Extensive experiments on the DTU and Tanks and Temples datasets demonstrate that the proposed network achieves state-of-the-art results.