Perspectives Talk II - Academia: Toward Sign Language Understanding in the Real World

Sign languages are used by millions of deaf and hard of hearing individuals around the world. Research on sign language video processing is needed to make all of the technologies that are now available for spoken and written languages available also for sign languages. Sign languages are under-resourced and unwritten, so research in this area shares many of the same challenges faced by research on other low-resource languages. However, there are also sign language-specific challenges, such as the difficulty of analyzing fast, coarticulated body pose changes. Sign languages also include certain linguistic properties that are specific to this modality. There has been some encouraging progress, including on tasks like isolated sign recognition and sign-to-written language translation. However, the current state of the art is far from handling arbitrary linguistic domains and visual environments. This talk will provide a perspective on research in this area, including work in my group and others aimed at a broad range of real-world domains and conditions. Along the way, I will present recently collected datasets as well as technical strategies that have been developed for handling the challenges of natural sign language video.