Single-Shot Real-Time Multiple-Path Time-Of-Flight Depth Imaging For Multi-Aperture And Macro-Pixel Sensors

Multiple-Path Interference (MPI) is a major drawback of Time-of-Flight (ToF) sensors. MPI occurs when a ToF pixel receives more than a single light bounce from the scene. Current methods resolving more than a single return per pixel rely on the sequential acquisition of large amounts of data and are too computationally expensive to deliver depth images in real time. These factors have precluded the development of a multiple-path ToF camera to date. In this work we consider two hardware alternatives that can be used to acquire all necessary raw data in a single shot. We introduce a unified signal processing pipeline that takes the single-shot raw data from any of these sensors as input and delivers a set of depth images, one per interfering path. We use the measurements to estimate the lowest-frequency Fourier coefficients of the scene response function, which can be ideally modeled as a train of Dirac delta functions. The use of a parametric spectral estimation method allows us recovering the unknown parameters (distance and reflectivity of the targets) for all pixels in real time.