Timestamps: 0:00 Introduction 0:45 Google DeepMind's vision of distributed training 2:57 Building blocks towards DiPaCo 11:33 Building blocks towards DiPaCo: DiLoCo - a low communication distributed training optimization 18:41 Introducing DiPaCo: Distributed Paths Composition 22:37 Q&A session
Timestamps:
0:00 Introduction
0:45 Google DeepMind's vision of distributed training
2:57 Building blocks towards DiPaCo
11:33 Building blocks towards DiPaCo: DiLoCo - a low communication distributed training optimization
18:41 Introducing DiPaCo: Distributed Paths Composition
22:37 Q&A session
Join BuzzRobot community on Slack: join.slack.com/t/buzzrobot/shared_invite/zt-1zsh7k8pd-iMu_M8bUxIK3pOJgqJgCRQ