Nus Auto Lyrix Align

NUS Auto Lyrix Align is a system that automatically provides word-level alignment of a given lyrics text to a given polyphonic song. Automatic lyrics alignment in polyphonic music is a challenging task because the singing vocals are corrupted by the background music. In this system, we use an acoustic model that has been trained on music genre-specific characteristics of polyphonic music. With such a genre-based approach, we explicitly model the music without removing it during acoustic modeling. This algorithm has been submitted to ICASSP 2020, also has outperformed all other systems in the International Music Information Retrieval Evaluation eXchange platform MIREX 2019, with mean absolute word alignment error of less than 200 ms across all test datasets. For the first time, we present our algorithm in the form of an interactive web interface, where a user can upload a song as an mp3 file or a youtube link along with its lyrics text file. The system processes the inputs and outputs word-level time-aligned lyrics. The aligned lyrics are displayed on the screen in karaoke fashion, i.e. scrolling highlighting of the words as the audio plays. The user may download the word-aligned output file in json or txt format, which is compatible with Audacity. A video demonstrating our system is shown here: https://drive.google.com/open?id=1oGdXQ9d3SfecPu8R3TBhY8kufFfXsd8_ The system is online now, please try it out! https://autolyrixalign.hltnus.org/ Our Relevant Publications [1] C. Gupta, E. Y?lmaz, H. Li, “Automatic Lyrics Alignment and Transcription in Polyphonic Music: Does Background Music Help?,” arXiv preprint:1909.10200v2 [eess.AS], 2019 (submitted to ICASSP 2020). [2] C. Gupta, E. Y?lmaz, and H. Li, “Acoustic modeling for automatic lyrics-to-audio alignment,” in Proc. Interspeech, 2019. [3] C. Gupta?, B. Sharma?, H. Li, and Y. Wang, “Automatic lyrics-to-audio alignment on polyphonic music using singing-adapted acoustic models,” in Proc. ICASSP, 2019. (*equal contributors). [4] C. Gupta, H. Li, and Y. Wang, "Automatic Pronunciation Evaluation of Singing," in Proc. Interspeech, 2018