Yoshua Bengio: From System 1 Deep Learning to System 2 Deep Learning (NeurIPS 2019)
HTML-код
- Опубликовано: 10 июл 2024
- This is a combined slide/speaker video of Yoshua Bengio's talk at NeurIPS 2019. Slide-synced non-RUclips version is here: slideslive.com/neurips/neurip...
This is a clip on the Lex Clips channel that I mostly use to post video clips from the Artificial Intelligence podcast, but occasionally I post clips from other lectures by me or others. Hope you find these interesting, thought-provoking, and inspiring. If you do, please subscribe, click bell icon, and share.
Lex Clips channel:
/ lexclips
Lex Fridman channel:
/ lexfridman
Artificial Intelligence podcast website:
lexfridman.com/ai
Apple Podcasts:
apple.co/2lwqZIr
Spotify:
spoti.fi/2nEwCF8
RSS:
lexfridman.com/category/ai/feed/
Connect with on social media:
- Twitter: / lexfridman
- LinkedIn: / lexfridman
- Facebook: / lexfridman
- Instagram: / lexfridman
- Medium: / lexfridman 
Наука
Outline of the talk:
0:00 - Introduction
0:56 - The State of Deep Learning
2:20 - System 1 and System 2
4:09 - What's currently missing in deep learning
8:56 - Out-of-distribution generalization
12:16 - Compositionality
16:08 - Contrast with Symbolic AI
18:11 - Attention and Consciousness
27:07 - Consciousness prior
32:17 - Meta-learning
39:09 - Operating on sets of objects
41:21 - Recap
44:48 - Question: moral implications of building machines that are conscious
46:18 - Question: Integrated information theory
47:17 - Question: Spatial prior
48:28 - Question: Symbolic AI
50:51 - Question: What is a data distribution?
52:05 - Question: measuring progress
52:52 - Question: causality
53:34 - Question: relation and associative memory
I am so glad I found this video. The presenter gives rise to various useful points!
I was there, and after 8 months I am still studying what he said...
Excellent talk Bengio
THANK YOU
Great talk
Great man..
What is the main argument professor Bengio has against hybrid systems (DL + Symbolic AI)?
Thank you very much Professor for a so bright talk.
Just a question : the System 2 Deep Learning architecture showed is very similar to the Bayesian network Open-universe Probability Model - OUPM (section 14.6.3 of the book of Russell and Norvig: Artificial Intelligence, a modern approach). Is possible to elaborate about the difference, please?
Bayesian network Open-universe Probability Model - OUPM is just one possible way to instantiate a System 2, but it is not neural network-based. Its main capability is to perform reasoning in the presence of probabilities, without making an assumption that there is only a fixed number of objects in the world.
This is fine and useful, but Prof Bengio is also talking about the problem of creating a System 2 that integrates smoothly with a System 1. It seems that he suggests creating a neural network-based System 2 so that it is more easily integrable with System 1 neural networks (having both systems be neural network-based allows for joint training with gradient descent). This is one possible approach but there are other proposals for integrating non-neural network based Systems 2 with neural network-based Systems 1 (see for example Luc De Raedt's DeepProbLog).
@@rodrigobraz2 Thank you very much for your answer!
What I got before your reply is that OUPM is a Deductive learning approach while the System 2 is Inductive (indeed, Professor Bengio does not want to come back to Symbolic AI). The possible mistake I think you could have made is when you talk about collaboration between system 1 and system 2. Indeed, System 2 is an extension of System 1, and when it will be created, System 1 as we know it now will disappear. We will therefore only talk about the System 2 as AGI.
Anyway, also thank you for informing me about the DeepProbLog approach, the book of Russell and Norvig: Artificial Intelligence, a modern approach (that I cited) recommends these initiatives in section 27.1, page 1062, just before the last paragraph. And as I can read on the DeepProbLog paper (arxiv.org/abs/1805.10872), it is the first initiative of this kind.
Thank you again!
@@kkjc You're welcome! Just a few more points: I do not think that the intention is to get rid of System 1 once we have System 2. This nomenclature came from the book "Thinking, Fast and Slow" by Daniel Kahneman. From its Wikipedia page: "The main thesis is that of a dichotomy between two modes of thought: "System 1" is fast, instinctive and emotional; "System 2" is slower, more deliberative, and more logical." So, the idea is that intelligent agents keep both, just like we humans do. For example, an AI system may have a low-level, perceptual level System 1 that instantly classifies an image while also having a System 2 that reasons logically about it (possibly even realizing incorrect perceptions as it sometimes happens in optical illusions).
It seems to me like there should be a way to run proof searches where you are constrained to converge to logical proofs but you allow a pre-trained intuitive neural network to make the decisions about which paths to explore first.
Another idea is some non-symmetric adversarial system where one half of the system is optimizing to imitate human creativity and the other system is optimizing for logical correctness. A left brain/right brain system.
I was part of the #NeurIPS2019
Dual-system theories are popular in psychology but highly disputed. Kahneman may have received recognition for research related to System 1 (heuristics and habitual responses). What is disputed, however, is System 2. Is a System 2 necessary to explain human "higher" cognition. It is probably a bad idea to base your AI ideas on this flawed idea from psychology.
ML research by brain research ?
As it has been since the first ML systems, and as it should be until we have implemented in ML systems all the principles we have discovered in brains