Devising Transformers as an Autoencoder for Unsupervised Multivariate Time Series Imputation

Time-series data processing is essential across various fields, including healthcare, transportation, and weather forecasting. Multivariate time-series data, in particular, exhibit a correlation pattern over a common independent variable. This is illustrated by concurrent sensor readings in applications like autonomous driving or multiple channels in data collection devices used in medical diagnoses. However, the increasing incidence of data acquisition failures, including sensor malfunctions and human errors, results in gaps and substantial loss of information. The presenter will propose a novel method called Multivariate Time-Series Imputation with Transformers (MTSIT) to tackle these challenges. This method employs an unsupervised autoencoder model with a transformer encoder to leverage unlabeled observed data for simultaneous reconstruction and imputation of multivariate time series. The MTSIT strategy presents an input sequence with gaps (missing patterns) to the transformer encoder. The final encoder block produces an output sequence that is linearly transformed into the imputed sequence. The Mean Squared Error (MSE) is subsequently computed between the missing values and their predicted imputations, guiding the network’s training toward minimizing the MSE.