Deep-Fusion: An End-To-End Approach For Compressive Spectral Image Fusion

This paper presents an end-to-end (E2E) deep learning approach for the fusion of the data from two compressive spectral imaging systems, where a single neural network is developed to simultaneously optimize the sensing matrices and the decoder operator. The proposed E2E method models the sensing operator of the systems to fuse as optical layers, where the learnable parameters are the coded apertures of these CSI systems. These optical layers are then concatenated to an inspired unrolled deep neural network, where after training, these sensing matrices remain non-trainable along the optimization stages. Finally, a loss function is proposed. Simulation results show an improvement of the proposed coupled method compared with previous work and an enhancement due to the training of the sensing matrices and the proposed loss function.