ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning

ConceptGraphs builds open-vocabulary 3D scenegraphs that enable a broad range of perception and task planning capabilities.
For more details, please take a look at the project webpage:
concept-graphs...
🔵 Author Information:
Qiao Gu†* (University of Toronto)
Alihusein Kuwajerwala†* (Université de Montréal)
Sacha Morin* (Université de Montréal)
Krishna Murthy Jatavallabhula* (MIT)
Bipasha Sen (Université de Montréal)
Aditya Agarwal (Université de Montréal)
Corban Rivera (JHU APL)
William Paul (MIT)
Kirsty Ellis (Université de Montréal)
Rama Chellappa (JHU)
Chuang Gan (UMass Amherst)
Celso Miguel de Melo (DEVCOM Army Research Laboratory)
Joshua B. Tenenbaum (MIT)
Antonio Torralba (MIT)
Florian Shkurti (University of Toronto)
Liam Paull (Université de Montréal)
*Equal contribution
† Project Lead
🏫 Affiliations:
MIT
Université de Montréal
University of Toronto
IIIT Hyderabad
JHU APL
JHU
UMass Amherst
DEVCOM Army Research Laboratory

[@TomBielecki]

So many possibilities with your work. With ARkit RoomPlan you can scan an entire house interior. What if you could create an entire home inventory automatically categorized for insurance purposes? Cluster items to get tips how to tidy/organize? Show in AR where things belong? Tag items in AR with metadata and pointers to related items?

[@alihkw]

These are exactly the kind of use cases we're excited about! As you put it so well, ConceptGraphs can be used for more than just robotics. All the code is available on the website if you'd like to mess around with it!