[MERL Seminar Series Fall 2023] Visual Programming - A compositional approach to building General...

[MERL Seminar Series Fall 2023] Visual Programming - A compositional approach to building General Purpose Vision Systems
Tanmay Gupta, Allen Institute for Artificial Intelligence, presented a talk in the MERL Seminar Series on October 31, 2023.
Abstract:
Building General Purpose Vision Systems (GPVs) that can perform a huge variety of tasks has been a long-standing goal for the computer vision community. However, end-to-end training of these systems to handle different modalities and tasks has proven to be extremely challenging. In this talk, I will describe a lucrative neuro-symbolic alternative to the common end-to-end learning paradigm called Visual Programming. Visual Programming is a general framework that leverages the code-generation abilities of LLMs, existing neural models, and non-differentiable programs to enable powerful applications. Some of these applications continue to remain elusive for the current generation of end-to-end trained GPVs.