CoRL 2024 MRM-D Workshop: Ted Xiao - What's Missing for Robotics Foundation Models?

This video is part of our CoRL 2024 Workshop on Robot Manipulation in a World of Abundant Data
Overview:
Manipulation is a crucial skill for fully autonomous robots operating in complex, real-world environments. As robots move into dynamic, human-centric spaces, it is increasingly important to develop reliable and versatile manipulation abilities. With the availability of large datasets (e.g., RT-X) and recent advances in robot learning and perception (e.g., deep RL, diffusion, and language-conditioned methods), there has been significant progress in acquiring new skills, understanding common sense, and enabling natural interaction in human-centric environments. These advances spark new questions about (i) the learning methods that best utilize abundant data to learn versatile and reliable manipulation policies and (ii) the modalities (e.g., visual, tactile) and sources (e.g., real-world, high-fidelity contact simulations) of training data for acquiring general-purpose skills. In this workshop, we aim to facilitate an interdisciplinary exchange between the communities in robot learning, computer vision, manipulation, and control. Our goal is to map out further potential and limitations of current large-scale data-driven methods for the community and discuss pressing challenges and opportunities in diversifying data modalities and sources for mastering robot manipulation in real-world applications.
Webpage: www.dynsyslab....
Invited Speakers and Panelists:
Sergey Levine, UC Berkeley
Jens Lundell, KTH Stockholm
Ankur Handa, NVIDIA
Carlo Sferrazza, UC Berkeley
Ted Xiao, Google DeepMind
Christian Gehring, ANYbotics
Mohsen Kaboli, BMW and TU/e
Katerina Fragkiadaki, CMU
Shuran Song, Stanford University
Organizing Team:
Angela Schoellig, TUM and University of Toronto
Animesh Garg, Georgia Tech and NVIDIA
Karime Pereida, Kindred
Oier Mees, UC Berkeley
Ralf Römer, TUM
Martin Schuck, TUM
Siqi Zhou, TUM