Tutorial: Learning with Multiple Objectives Beyond Bilevel Optimization - New Foundations and Speech Applications

Part I - Introduction and Background New challenges of learning under multiple objectives Two optimization toolboxes to address those challenges History of bilevel and multi-objective optimization Part II - Bilevel Optimization for Learning with Ordered Objectives Solution concepts and metrics of optimality Implicit gradient-based methods for bilevel optimization Value function-based methods for bilevel optimization Part III - Multi-objective Optimization for Learning with Competing Objectives Solution concepts and metrics of optimality Dynamic weighting-based methods for multi-objective optimization Generalization bounds on multi-objective optimization algorithms Part IV - Applications to Automatic Speech Recognition Automatic Speech Recognition Opportunities and Challenges Recursive pre-training and fine-tuning with limited labels Multilingual training for low-resource speech recognition Part V - Open Research Directions