Enhancing Deep Paraphrase Identification Via Leveraging Word Alignment Information

Recent deep learning based methods have achieved impressive performance on paraphrase identification (PI), a fundamental NLP task, judging whether two sentences are semantically equivalent or not. However, their success heavily relies on massive labeled samples, which are time-consuming and expensive to obtain. To alleviate this problem, this study explores the effect of word alignment information (WAI), extracted by existing monolingual alignment tools, on deep PI baseline models. Apart from directly encoding WAI into fixed-size embeddings, we propose a novel auxiliary task so that the baselines can be pre-trained using a large amount of unlabeled in-domain data. Moreover, our proposed auxiliary task can also jointly train with the baselines, aiming to eliminate the overheads of preprocessing WAI at the test period. Experimental results verify that our methods can significantly outperform the deep PI baseline model.