Incremental Semi-Supervised Learning For Multi-Genre Speech Recognition

In this work, an effective data scheduling strategy for semi- supervised learning (SSL) for acoustic modeling in automatic speech recognition is explored. The conventional approach uses a seed model trained with supervised data to automatically recognize the entire set of unlabeled (auxiliary) data to generate new labels for subsequent acoustic model training. In this paper, we propose an approach in which the unlabelled set (typically audio from web) is split into multiple equal- sized subsets. These subsets are processed in an incremental fashion: for each iteration a new subset is added to the training list used for SSL (starting from only one subset in the first iteration). The acoustic model from the previous iteration becomes the seed model for the next one. The proposed scheduling strategy is compared to the approach employing all unlabeled data in one-shot for training. Experiments using lattice-free maximum mutual information based acoustic model training on Fisher English gives 80% word error recovery rate. On the multi-genre evaluation sets on Lithuanian and Bulgarian relative improvements of upto 17.2% in word error rate are observed.