Universal Speech Model and Language Foundational Models

The topic of the tutorial includes one of the most advanced in-class models for ASR, the Universal Speech Model (USM) as well as for language, the PaLM 2 language model. Based on the frozen foundational speech and language models, we will also present a joint Speech and Language Model (SLU) , a versatile and high-performing speech-language understanding model that performs unseen generation tasks including contextual ASR, dialog generation, speech continuation and question answering given a speech input and text instruction as a prompt.