I really enjoyed this conversation with John. Here's the outline: 0:00 - Introduction 2:35 - Difference between biological and artificial neural networks 8:49 - Adaptation 13:45 - Physics view of the mind 23:03 - Hopfield networks and associative memory 35:22 - Boltzmann machines 37:29 - Learning 39:53 - Consciousness 48:45 - Attractor networks and dynamical systems 53:14 - How do we build intelligent systems? 57:11 - Deep thinking as the way to arrive at breakthroughs 59:12 - Brain-computer interfaces 1:06:10 - Mortality 1:08:12 - Meaning of life
I once called Dr. Hopfield uninvited as an undergrad at Oklahoma State University. Dr. Hopfield was at CalTech at the time I think. Dr. Hopfied was very gracious and talked to me for a good 30 minutes explaining his work carefully, focusing on an aspect of his papers that was not making sense to me at the time. I am a life long fan.
Listening to these conversations gives me a deep ache in my chest, and brings tears to my eyes. The purity of yearning for the knowledge of the yearning for knowledge is such a beautiful thing.
@@Nintendo_fanboy87 nah, idk what I was talking about haha. This happens every so often, I get a reply to an old comment I made and I have no clue why tf I made it lol
Wonderful to listen to John Hopfield. I had implemented his Hopfield network on silicon in 1988 🙂. I moved to other things but kept a deep interest in brain biology and artificial intelligence.
I listened this many times, pausing, going back and forth and it always amazes me how Lex is so prompt in asking the best questions and follow the responses so wisely. HATS OFF to both of you.
Lex, I can't thank you enough nor tell you how amazingly educational and useful your podcast has become in my life. Thank you for everything you do, have done and will do. This is a field in which I'm truly passionate about and the information in this conversation is priceless.
Thank you so much Lex. I had a lot of these thoughts as a high school seniors. Now I’m 69. Just got me all excited. And I’m on lesson two of deep learning Ai Andrew Ng class thanks to you.
.... you are such a lucky man to meet all these people and have such long, deep conversations with them. (Yes, I know, you are working super hard to get 'so lucky'...!) - You are so young and when you keep on doing this for the next decades you will turn into a legend yourself... (Silly me thought 'Lex-Icon of AI' ...)
Beautiful interview. I’m a neuroscientist neuroethologist and yesterday I was teaching Hopfield landscapes to my psychology and medicine students… next semester I’ll include some of his appreciations from your conversations… for me personally, it gives me motivation to pursue that Postdoc; I’ll be defending my PhD thesis within a month on multimodal integration and neuromodulation in bees
Took me a while to get into this one, but again very good indeed. I like the quote from Marvin Minski that "consciousness is basically overrated.... all the hard computations are done non-consciously" and "consciousness is your effort to explain to yourself what you have already done" This is very topical because I just was listening to an analyst saying Tesla would not get full autonomy because AI cannot do the sort reasoning we do - I think this is completely wrong - we drive our cars unconsciously most of the time.
99% of my brain: a great conversation, great to hear from the man himself and grateful to lex for the content 1% of my brain: this dude's tongue has a mind of its own
I dont ever feel like commenting as I enjoy your podcasts and that's all I'm here for. But your podcasts are special. We've built an economy of content but not enough context. It's important that these interviews have highlighted not only the search for purpose but their ability to be actionable about it. I hope these podcasts inspire the general public one day.
Lex! Watched you a couple of times at Roagans channel! After you expressed your intent behind building your business I got interested in your work. I am now mesmerised by the quality of questions you directed to John. Inspiring, intriguing and humbling. I will be back for sure. Keep it up brother
Enjoyed this interview. The question "What does it mean to understand something?" which Hopfield asks multiple times during this interview is my key takeaway. Thanks, Lex.
As someone else wrote, wonderful indeed to listen to J. Hopfield. But I was most impressed with Lex's talent as an "interviewer". While J.H. is definately still "all there" in terms of his intellect, etc (my opinion) ... many "mature" scientists provide explanations that leave a fair amount of "white space", perhaps b/c the nuances of their ideas & discoveries start to seem trivial (to them) over time. Lex brilliantly keeps the flow in this conversation with a great scientist/human by steering the conversation in the subtle manner that befits speaking with a brilliant mind that prefers to take longer "strides" with respect to sharing his insights & knowledge. Simply put, Lex prompts JH to connect certain dots but allows for a certain amount of "drift" ... perhaps out of respect, deference, love. That talent (on the part of Lex) is uplifting (and effective). Bottom line ... Lex has orchestrated a wonderful journey into a great mind & person ... yet again. Thx Lex
This feels like an important interview doesn't it? I particularly enjoyed his take on neural interfacing. He used so many wonderful metaphors that actually helped redirect my imagination and this... is about as good as it gets for sharing with humanity. Thanks Lex
There seemed to be a grinding of gears going on for these two not far under the surface. Lex seemed impatient at times. Not like I’ve seen him in other interviews. Perhaps the difficulties JH had getting some of the words out made the pace difficult for Lex. The analogies that JH was drawing were well worth the wait - utterly brilliant guy. He seemed able to express the gist of a lifetime of insights on how to think clearly about nature and science into short pithy comparisons. JH joked at the end that Lex might have difficulty getting 5 minutes of coherent conversation out of the recording. It all seemed very sensical. Abstract at times but deeply insightful.
The "mystery of consciousness" was discussed, albeit indirectly, by Ludwig Wittgenstein in his posthumously published book Philosophical Investigations. I spent many hours, night after night, when I was 17 years old, reading that book line by line. The book is very difficult to understand, partly because Wittgenstein's language was German, which I do not speak or read, and also because Wittgenstein's manner of expressing his ideas was odd and seemingly archaic. Wittgenstein did not discuss consciousness directly in anything I have read, but the content of his work bears strongly on the present discussion. His attitude toward the question, "What is consciousness?" might have been that there is no point discussing something we cannot even define. If you describe a world In which human beings do everything of which we are capable, without mentioning consciousness, what would be lost or incorrect as a result? And the only answer would seem to be that nothing was lost.
Excellent, thank you Lex for interviewing Hopfield. Many insights on NLD and its omissions in the ANN world. I wonder if Hopfield has read “Beyond Boundaries” and the research work of Miguel Nicolelis. It sounded like he hadn’t and that you haven’t. Nicolelis has done some beautiful work on measuring collective phase locking behavior of actual networks of various sizes. I realize his research is controversial, since the probes are invasive and the test subjects are laboratory animals. There is no doubt in my mind that the neuroscience world would benefit from improved diagnostic test methods, non-invasive if possible. FMRI is too qualitative and disconnected with the actual NLD.
Incredible insights on neural dynamics and how understanding of dynamical systems can give new ideas to improve artificial neural networks. It's good to learn a bit about "neural coding" and "chaotic attractors" before listening.
Please, please interview Barbara Arrowsmith-Young who wrote ‘the woman who changed her brain’. Also a conversation between Barbara Arrowsmith -Young and John Hopfield would be fascinating and enlightening. They could perhaps lead you to your starting point in your new AI project…
Now that I have been listening to your podcasts, the chats are serving as a brain mechanism, the propeller of my discourse, a response mechanism of these words, the other day, last year!, just came to my mind that people are like rocks. In the sense that brains, neurology people act always the same, they have to be tought, trained, and seems they never learn well, we came back everyday to do the same, even though it seems really different for all the days, years in our lives. Analyzing this past sentence: this has some of Descartes rationalism, blank slate, maybe that's where the idea comes from: people are like rocks/slates, as I am introspecting, past knowledge, to understand where I would like to go. Knowing my human experience, existensialism or meta-existence has limits. My learned knowledge in phisics is null, which is impossible, but true is that I have no conscience of phisics in my life, I cook, I go shopping, on the bus, train, subway, but phisics never crosses my mind. I have to get some fundamental book in phisics, I forgot everything from high school phisics! Thank You both 😀
well, this HTM theory didn't pick up. Same as it didn't pick up for Imagination Engines with its STANNOs developed by Steven Thaler. Makes me think all these folks who "reinvent the wheel" are just marketing gradient descent or similar another well known techniques under their own name, and get banned by AI community. One vivid example of such a ban is "Extreme Machine Learning" which is nothing more than random projection and a linear regression on top of it.
Very profound discussion. Hopfields observation that he cannot yet find a smoking gun that links physical systems to consciousness rings true to me - how do electronic signals in the form of action potentials make a leap into a sense of “I”?
What? No white shirt? I barely recognized you. I'm looking forward to an experimenter's explanation of how to implement emergent properties in computer simulations of neurobiology. Excellent podcast as always.
I infer that if a living system is a seemingly messy coalescence of physics, biology, chemistry, computation and what not, any semblance of true artificial general intelligence may well require an unified application of natural general intelligence.
I don't know if my memory is caught in a loop (and I'm weaving my own narrative) or the John Dean example had already been used by another guest in the podcast
I added a transcript to the interview in LinkedIn and GitHub: LinkedIn: www.linkedin.com/posts/alfonsorreyes_artificialintelligence-physics-neuralnetworks-activity-6655541541651734528-ScRj GitHub: github.com/f0nzie/transcript_interview_john_hopfield_by_lex_fridman
llms are recurrent - each token generated is fed back to the network to generate the next, so the wisdom of the network is in a sense being unpacked. Recurrence is a kind of compression.
I really enjoyed this conversation with John. Here's the outline:
0:00 - Introduction
2:35 - Difference between biological and artificial neural networks
8:49 - Adaptation
13:45 - Physics view of the mind
23:03 - Hopfield networks and associative memory
35:22 - Boltzmann machines
37:29 - Learning
39:53 - Consciousness
48:45 - Attractor networks and dynamical systems
53:14 - How do we build intelligent systems?
57:11 - Deep thinking as the way to arrive at breakthroughs
59:12 - Brain-computer interfaces
1:06:10 - Mortality
1:08:12 - Meaning of life
is there a place to get a transcript of the conversation?
Bring David Deutsch please! :)
Lex, could you add subtitles for the hearing impaired people?
Lex Fridman to Interview this man is a great honor. I have never seen someone describe the loneliest place so perfect!
@@wizardstein3153 www.linkedin.com/pulse/transcript-interview-john-hopfield-lex-fridman-alfonso-r-reyes/?published=t
github.com/f0nzie/transcript_interview_john_hopfield_by_lex_fridman
Congratulations for his Nobel prize in physics
I once called Dr. Hopfield uninvited as an undergrad at Oklahoma State University. Dr. Hopfield was at CalTech at the time I think. Dr. Hopfied was very gracious and talked to me for a good 30 minutes explaining his work carefully, focusing on an aspect of his papers that was not making sense to me at the time. I am a life long fan.
And no wonder , it must have been super inspiring .
And now he is Nobel laureate
This man deserves a noble prize!
Listening to these conversations gives me a deep ache in my chest, and brings tears to my eyes. The purity of yearning for the knowledge of the yearning for knowledge is such a beautiful thing.
True. People around me think I am weird for listening to these videos.
Nerd
@@Adam-st8ys haha, yeah
@@eddieharding6788 do you know him/her personally?
@@Nintendo_fanboy87 nah, idk what I was talking about haha. This happens every so often, I get a reply to an old comment I made and I have no clue why tf I made it lol
Wonderful to listen to John Hopfield. I had implemented his Hopfield network on silicon in 1988 🙂. I moved to other things but kept a deep interest in brain biology and artificial intelligence.
I listened this many times, pausing, going back and forth and it always amazes me how Lex is so prompt in asking the best questions and follow the responses so wisely. HATS OFF to both of you.
17.37 "but the rhythms what??" Damn Hopfield goes #savage on Lex... what amazing insights ... one of the best podcasts. Thanks!
Lex, I can't thank you enough nor tell you how amazingly educational and useful your podcast has become in my life. Thank you for everything you do, have done and will do. This is a field in which I'm truly passionate about and the information in this conversation is priceless.
Thank you so much Lex. I had a lot of these thoughts as a high school seniors. Now I’m 69. Just got me all excited. And I’m on lesson two of deep learning Ai Andrew Ng class thanks to you.
.... you are such a lucky man to meet all these people and have such long, deep conversations with them. (Yes, I know, you are working super hard to get 'so lucky'...!) - You are so young and when you keep on doing this for the next decades you will turn into a legend yourself... (Silly me thought 'Lex-Icon of AI' ...)
Indeed.
Lex’s laugh when Dr. Hopfield responded with “there’s a question of what do you mean by understand” :’)
Beautiful interview. I’m a neuroscientist neuroethologist and yesterday I was teaching Hopfield landscapes to my psychology and medicine students… next semester I’ll include some of his appreciations from your conversations… for me personally, it gives me motivation to pursue that Postdoc; I’ll be defending my PhD thesis within a month on multimodal integration and neuromodulation in bees
Took me a while to get into this one, but again very good indeed. I like the quote from Marvin Minski that "consciousness is basically overrated.... all the hard computations are done non-consciously" and "consciousness is your effort to explain to yourself what you have already done" This is very topical because I just was listening to an analyst saying Tesla would not get full autonomy because AI cannot do the sort reasoning we do - I think this is completely wrong - we drive our cars unconsciously most of the time.
He won Nobel prize in Physics today
This is very much amongst my favorite discussions you've had. So much honest wisdom.
99% of my brain: a great conversation, great to hear from the man himself and grateful to lex for the content
1% of my brain: this dude's tongue has a mind of its own
I dont ever feel like commenting as I enjoy your podcasts and that's all I'm here for.
But your podcasts are special.
We've built an economy of content but not enough context. It's important that these interviews have highlighted not only the search for purpose but their ability to be actionable about it.
I hope these podcasts inspire the general public one day.
At this specific date - november 14, 2020 - I predict that John Hopfield will one day win the Nobel Prize in physics.
This guy just won a nobel prize in physics
now he got nobel in physics, congrats
He has just won the Nobel Prize!
Mr. Hopfield has a beautiful and elegant mind. This was such a pleasure to watch. Thank you Lex.
He won nobel prize 🏆
Lex! Watched you a couple of times at Roagans channel! After you expressed your intent behind building your business I got interested in your work.
I am now mesmerised by the quality of questions you directed to John. Inspiring, intriguing and humbling. I will be back for sure. Keep it up brother
Maybe it's just me but I'm enormously jealous Lex. These guests are incredible.
This was a lovely conversation. JH's analogy about the adaptive speedometer was so simple and so profound.
Congrats for the Nobel Prize
Enjoyed this interview. The question "What does it mean to understand something?" which Hopfield asks multiple times during this interview is my key takeaway. Thanks, Lex.
As someone else wrote, wonderful indeed to listen to J. Hopfield. But I was most impressed with Lex's talent as an "interviewer". While J.H. is definately still "all there" in terms of his intellect, etc (my opinion) ... many "mature" scientists provide explanations that leave a fair amount of "white space", perhaps b/c the nuances of their ideas & discoveries start to seem trivial (to them) over time. Lex brilliantly keeps the flow in this conversation with a great scientist/human by steering the conversation in the subtle manner that befits speaking with a brilliant mind that prefers to take longer "strides" with respect to sharing his insights & knowledge. Simply put, Lex prompts JH to connect certain dots but allows for a certain amount of "drift" ... perhaps out of respect, deference, love. That talent (on the part of Lex) is uplifting (and effective). Bottom line ... Lex has orchestrated a wonderful journey into a great mind & person ... yet again. Thx Lex
I hope I still have my whits about me when I'm 87 like this guy!
Watching after he awarded with nobel prize for ai ml♥️
Single best channel on yt I'd recommend to everyone in the world right here. Your work is amazing Lex.
Great guy. Please interview John Hopfield again.
This feels like an important interview doesn't it? I particularly enjoyed his take on neural interfacing. He used so many wonderful metaphors that actually helped redirect my imagination and this... is about as good as it gets for sharing with humanity. Thanks Lex
Another killer podcast! Thanks.
Thank you Hopfield, it was all a coherent sense to me throughout the video.
Amazed at this Hopfield interview, If Hinton isn't returning your calls you should ask Mike Spivey.
Thanks for making these kinds of videos. Fascinating stuff.
Lex "Let's linger a bit on that" Friedman
Lex "Beautiful" Friedman
*Fridman
There seemed to be a grinding of gears going on for these two not far under the surface. Lex seemed impatient at times. Not like I’ve seen him in other interviews. Perhaps the difficulties JH had getting some of the words out made the pace difficult for Lex. The analogies that JH was drawing were well worth the wait - utterly brilliant guy. He seemed able to express the gist of a lifetime of insights on how to think clearly about nature and science into short pithy comparisons. JH joked at the end that Lex might have difficulty getting 5 minutes of coherent conversation out of the recording. It all seemed very sensical. Abstract at times but deeply insightful.
I really love how you bring these diverse brilliant conversation to everyday people!
I really liked this deep slo-mo interview! Gives thoughts room for wandering... Thanks Lex!
The "mystery of consciousness" was discussed, albeit indirectly, by Ludwig Wittgenstein in his posthumously published book Philosophical Investigations. I spent many hours, night after night, when I was 17 years old, reading that book line by line. The book is very difficult to understand, partly because Wittgenstein's language was German, which I do not speak or read, and also because Wittgenstein's manner of expressing his ideas was odd and seemingly archaic.
Wittgenstein did not discuss consciousness directly in anything I have read, but the content of his work bears strongly on the present discussion. His attitude toward the question, "What is consciousness?" might have been that there is no point discussing something we cannot even define. If you describe a world In which human beings do everything of which we are capable, without mentioning consciousness, what would be lost or incorrect as a result? And the only answer would seem to be that nothing was lost.
The best of his interviews I’ve seen
You ask good questions. Thanks for your podcast.
Congratulations on the Nobel prize in physics 👏
Some good analogies used in this discussion. Interesting to hear how models to approximate how the brain works are evolving.
Thank you both.
I've great respect for Hopfield.
Lex you are doing very good job you can be proud of it and also very lucky. Keep doing!
Hello Mr. Nobel Prize Winner! 🙌
Excellent, thank you Lex for interviewing Hopfield. Many insights on NLD and its omissions in the ANN world. I wonder if Hopfield has read “Beyond Boundaries” and the research work of Miguel Nicolelis. It sounded like he hadn’t and that you haven’t. Nicolelis has done some beautiful work on measuring collective phase locking behavior of actual networks of various sizes. I realize his research is controversial, since the probes are invasive and the test subjects are laboratory animals. There is no doubt in my mind that the neuroscience world would benefit from improved diagnostic test methods, non-invasive if possible. FMRI is too qualitative and disconnected with the actual NLD.
This was a wonderful interview - Hopfield is a great thinker/scientist and Lex, you asked some really good questions!
Incredible insights on neural dynamics and how understanding of dynamical systems can give new ideas to improve artificial neural networks. It's good to learn a bit about "neural coding" and "chaotic attractors" before listening.
Lex you are on fire 🔥 with these podcasts👌
Thank you so much 🙏
Thank you man
My favorite episode so far
I'm a simple man, I see John I click like.
The rhythms... Look at central pattern generator circuits in the spine and how they solve the motor control problem...
such an inspiring talk .. kudos to both of you.
What a podcast, thank you Lex!
Another great interview. Thank you for a job well done.
New favorite RUclipsr.
Today Geoffrey Hinton and John Hopfield got Nobel price for their contribution in AI
Now days lex don't bring guest like this❤
Who's here after the Nobel prize!
Me
Please, please interview Barbara Arrowsmith-Young who wrote ‘the woman who changed her brain’. Also a conversation between Barbara Arrowsmith -Young and John Hopfield would be fascinating and enlightening. They could perhaps lead you to your starting point in your new AI project…
Great work Lex, thank you 🙏🏻
Very nice work. Could you please also interview synthetic biologists?
Great episode! Please have Alan Kay on!
I love your podcast Lex. I’m a huge fan. Your conversations are just amazing. Keep up the great work.
congrats on the nobel
This should be a good one 🙏🌿💚
coming from the nobel prize 2024
Who is watching that after he got nobel prize
Now that I have been listening to your podcasts, the chats are serving as a brain mechanism, the propeller of my discourse, a response mechanism of these words, the other day, last year!, just came to my mind that people are like rocks. In the sense that brains, neurology people act always the same, they have to be tought, trained, and seems they never learn well, we came back everyday to do the same, even though it seems really different for all the days, years in our lives. Analyzing this past sentence: this has some of Descartes rationalism, blank slate, maybe that's where the idea comes from: people are like rocks/slates, as I am introspecting, past knowledge, to understand where I would like to go. Knowing my human experience, existensialism or meta-existence has limits.
My learned knowledge in phisics is null, which is impossible, but true is that I have no conscience of phisics in my life, I cook, I go shopping, on the bus, train, subway, but phisics never crosses my mind.
I have to get some fundamental book in phisics, I forgot everything from high school phisics!
Thank You both 😀
Absolutely amazing interview, thank you
very poetic intro
I still have quite a bit of hope for approaches like Numenta’s HTM theory.
well, this HTM theory didn't pick up. Same as it didn't pick up for Imagination Engines with its STANNOs developed by Steven Thaler. Makes me think all these folks who "reinvent the wheel" are just marketing gradient descent or similar another well known techniques under their own name, and get banned by AI community. One vivid example of such a ban is "Extreme Machine Learning" which is nothing more than random projection and a linear regression on top of it.
Absolute Zero I don’t think you understand how HTM works. No gradient descent at all. Go learn about instead of armchair criticizing.
loved this podcast. its not everyday Vision from MCU comes to explain how human brain works
Well done both, thnx
Brilliant interview. You should have Professor Leroy Hulsey of UAF on at some point.
Thanks for it
one of the most overdue Nobel prizes in physics (the other overdue one being Alain Aspect's)
Wish I could upvote 10x
27:27 Great conversation. . Might this with hindsight be alluding to generative processes .
Thank You
Very profound discussion. Hopfields observation that he cannot yet find a smoking gun that links physical systems to consciousness rings true to me - how do electronic signals in the form of action potentials make a leap into a sense of “I”?
1.5 playback speed is the best for this video
Loved this!
So excellent! Thanks lex
That was very interesting
Very nice
An amazing man! I wonder what he thinks of hierarchical temporal networks
What? No white shirt? I barely recognized you. I'm looking forward to an experimenter's explanation of how to implement emergent properties in computer simulations of neurobiology. Excellent podcast as always.
I infer that if a living system is a seemingly messy coalescence of physics, biology, chemistry, computation and what not, any semblance of true artificial general intelligence may well require an unified application of natural general intelligence.
That's a smart guy!
I don't know if my memory is caught in a loop (and I'm weaving my own narrative) or the John Dean example had already been used by another guest in the podcast
This is a Noble Prize 🏆🏆🏆 Winner? What causes issues like that ?
I added a transcript to the interview in LinkedIn and GitHub:
LinkedIn: www.linkedin.com/posts/alfonsorreyes_artificialintelligence-physics-neuralnetworks-activity-6655541541651734528-ScRj
GitHub: github.com/f0nzie/transcript_interview_john_hopfield_by_lex_fridman
llms are recurrent - each token generated is fed back to the network to generate the next, so the wisdom of the network is in a sense being unpacked. Recurrence is a kind of compression.