Cross-Modality Depth Estimation via Unsupervised Stereo RGB-to-Infrared Translation

Existing depth estimation methods infer depth only from stereo RGB images. Since RGB is sensitive to changes in light, it's difficult to estimate disparities accurately in degraded visibility conditions. In contrast, IR imaging is not affected by brightness changing, providing extra clues for estimation. However, most datasets used do not have IR images paired with RGB-D data. Therefore, how to obtain the paired IR images and exploit the advantages of RGB and IR is of vital importance. Our core idea is to develop an unsupervised RGB-to-IR translation network with proposed Fourier domain adaptation and multi-space warping regularization to synthesize stereo IR images from their corresponding RGB images. Then modified depth estimation backbones can be used as the cross-modality depth estimation network to infer disparities from RGB-IR stereo pairs. We obtain superior performance just by deploying our framework to several off-the-shelf depth estimation backbones of RGB based methods.