IKN-1: Industry Keynote I

Foundation models play a crucial role in the AI community and are considered essential for the success of AGI. In this talk, I will discuss three types of foundations for foundation models. 1) Parameter Optimizer: I will introduce a new optimizer called Adan from a theoretical perspective and demonstrate its effectiveness in improving the convergence rate of various foundation models by 1.5-2x. 2) Network Architecture: I will showcase how insights from neuroscience have influenced and will continue to drive the development of new deep learning network architectures. 3) Representative/Barrier Data: I will showcase the exemplar prototypes of Skywork AI to demonstrate the importance of representative data.