Joint Unmixing and Demosaicing Methods for Snapshot Spectral Images

Recent technological advances in design and processing speed have successfully demonstrated a new snapshot mosaic imaging sensor architecture (SSI), allowing miniaturized platforms to efficiently acquire the spatio-spectral content of the dynamic scenes from a single exposure. However, SSI systems have a fundamental trade-off between spatial and spectral resolution because they associate each pixel with a specific spectral band. In this paper, we introduce the problem of joint "demosaicing" and "unmixing" for the hyperspectral images acquired by the SSI camera that we formulate as a low-rank matrix factorization and completion problem. For that reason—and in addition to a "naive" approach—we extend the "pure pixel" framework to the SSI sensor patch level and propose a dedicated method which (i) assumes the observed data to be locally rank-1 in some SSI "patches" to find, (ii) estimates endmembers in these patches which are (iii) clustered to derive the actual spectra. The abundances are then recovered using nonnegative least squares in each patch. The experiments show that our proposed scheme provides a slightly better demosaicing performance than state-of-the-art methods and a much higher unmixing enhancement.