Regularizing Neural Radiance Fields from Sparse RGB-D Inputs

This paper aims to improve neural radiance fields (NeRF) from sparse inputs. NeRF achieves photo-realistic renderings when given dense inputs, while its' performance drops dramatically with the decrease of training views' number. Our insight is that the standard volumetric rendering of NeRF is prone to over-fitting due to the lack of geometry regularization and neighborhood information from limited inputs. To address this issue, we propose a global sampling strategy together with a geometry regularization utilizing warped images as augmented pseudo-views to encourage geometry consistency across multi-views. In addition, we introduce a local patch sampling scheme with a patch-based regularization for appearance consistency. Furthermore, our method exploits depth information for explicit geometry regularization and faster training. Compared with existing baselines, our approach outperforms other methods on real benchmarks DTU dataset from sparse inputs and achieves the state of art results.