For me, listening to Joscha is like being a jazz enthusiast, who doesn't play an instrument, listening to jazz. You don't understand every detail but you sure enjoy the hell out of it.
It seems like you can ask Joscha literally any question about any topic and his response always appears to be such that he has spent a great deal of time thinking about it already.
True😂 and he has been thinking of many things for sure. However often he simply switches a few abstraction layers upwards and reframes the question from there - so that it appears he has some tremendously deep insight into everything.
I would like to ask him, or anubody, why Pythagoran comma exists. Great minds have created numerous ways hiw to cope with it, but nobody really did talk about the origi al reason why it exists. For me it seems like a mistake in maths and physics.
Physics is all about Topological Spaces and their transformations. It is a neat coincidence that there is an Adjoint Functor between the Category of Topological Spaces and the Category of Logical Systems. The Cat of Logical systems includes the Cartesian Closed Cats, so it includes Lambda Calculus!
@@Self-Duality his way of conversing seems a bit Wittgenstein-esque. You either know/accept what he knows in which case what he's saying is trivially simple. Or you don't (yet) and it all looks very complex.
As a philosopher curious about cognitive science, AGI, etc, I think Bach is the first thinker in this space that I find interesting. I'm very pleased to find agreement on machine learning too.
Amazing talk. I love how Joscha can keep bringing things back full circle. I only disagree with his discredit of Friston’s Active Inference as ‘predictive coding’ - that is a harsh simplification, as it has much more bottom-up inductive power (in theory) than deep networks, which essentially deduce from the top down instead. I love how Joscha discredits (seemingly) simple explanations of consciousness though, from Penrose to Friston’s, and how he reminds us about a lot of the fascinatingly complex modularity and functionality of our brains. By far though, his ruthlessly epistemological treatment of infinities is my favorite - people need to hesitate more when talking of ‘ontology’ as if it isn’t mere metaphysic.
Ontology isn't mere metaphisics! (no need for quotes around it.) Joshua does not reject the use of symbols that stand for mathematical or metaphisical notions like infinity. But he insists on a more direct system of denotations that match up with computable primitives. He even proposed a computable notion of truth to replace the metaphisical one. So what he proposed is different ontological semantics. And you can't throw away ontology and replace it with epistemology precisely because of the problem Gëodel discovered with mathematics. You need oracles of truth in symbolic form that can be unpacked semantically without crashing the brain. Epistemology requires ontology. In ML that translates into inductive biases. But this is clearly not sufficient.
@@lenyabloko I appreciate your comment. "you can't throw away ontology and replace it with epistemology precisely because of the problem Gëodel discovered with mathematics" - IMO this is not justified. Gödel's work operates in formal systems, which have no need of ontological interpretation (that is a philosophy of mind), and the same is true for mathematics. It works just as well (and breaks just as predictably) if understood as an epistemological language. I need not assume any notice properties of mathematics if I realize my notion of 'truth' is defined by my interaction with the language itself (this is Wittgenstein and Tarski). Truth need not be someplace else like Gödel believed.
Awesome summation. I understand Joscha's work well, his brilliance in reasoning about infinites is absolutely crucial to the way he constructs his theory of consciousness, which although not currently predictive is theoretically sound.
Joscha is always extremely interesting. I really look forward to hearing about his ideas. One thing that I think is missing is concreteness. It is hard, at least for me, to come away with something actionable, something that could be used to improve an AI's implementation, for example. He puts out deep ideas, that to me sound important and that ring with the feelings off deep truth, but I never know what to do with them. Would be great if he could put out some demo code! Maybe that says more about me than about him, Lol. Or maybe this is what it is like at the beginning of a new science? Anyways ..... very much enjoyed the interview! Thank you!
Great points about Gödel and infinity, and I'm glad Joscha didn't take the connectionist v symbolic bait...even going as far as reducing it to just "Twitter debate" 😂
Two really smart guys talking with an absolute genius makes these two exceptional minds look somewhat outclassed. This observation isn’t to diss any of these 3 people but I have yet to see any argument that Joscha has ever lost. Absolutely fabulous video!! PS I do understand this was more of an interview than a debate but there were ideas proposed (even if from others) that Joscha handled quite elegantly.
I've heard Joshua first time at Lex Fridman's show and I was shellshoked for couple of weeks later on rethinking every aspect of conversation. The power of mind is metric I do not often apply but i almost could hear the ultrahigh voltage lines buzzing up in the air when thinking about his.
This episode was really wonderful, Thank you for making it. The final segment was particularly amazing. When we decipher those useful geometric functions, how we can implement them in our training? If they emerge as abstractions in models, how can we refine them? Will we seek a model that optimizes transfer learning? In other words, will we use these functions as building blocks or parts of recipes, or will we simply organize them by differentiable machine learning?
Finally, I still believe that an engineering definition of consciousness should be simple. It should be perception coupled with action. So a thermostat that turns on a fan is conscious. Add to that the ability to be a control agent and you have something else like self-awareness. Add feelings and you have something else again. Call it emotional awareness. Name the simplest phenomena first. Then build the taxonomical hierarchy. Without a firm foundation, we have semantic jelly. Moreover, without a definite taxonomical hierarchy, we can't even define what we wish to analyze. It's a logical catch-22.
This man has thought very deeply on how the mind works everybody should listen to what he says he explains everything precisely and the nuances that happened when you do deep thought in all the systems in great integrated to it
Here's an idea. Train an AI to recognize objects attach attributes to them and create new data objects by observing how these objects interact with each other. A constant search, identify and attach attributes, compress, and reason with the compressed info (objects). A self creating object oriented programming language.
Why 'State' is taboo and bothers many CS engineers, but entertains so many cognitive scientists? I think there is an ultimate mystery in 'State'. Roughly I think 'State' +'Bayesian' is the recipe for cracking Chalmers' 'The hard problem of consciousness'.
15:48 the 17:48 how can Keith Duggar not realize he's asking a meaningless question? More than once! Congratulations to Bach for patiently explaining the reason behind it. Also, maybe somebody should show Duggar how small the Plank constant is
Keith isn't wrong for asking the question. I'm sure he's read or heard Joscha's ideas on unboundedness. He's giving Joscha an opportunity to express his ideas on behalf of those who haven't yet heard them. Then Keith pushes back a bit, something a good interviewer should do.
1:11:01 look at the semantics of the feeling we notice that there are Contracting or expanding or they are light or they're 1:11:08 heavy and so on is this all movement of Staff in space right it's all geometry plus valence the stuff that is going to 1:11:14 push your behaviors in a certain direction so these are basically the interactions of uh some deep Learning System 1:11:22 that is producing a continuous geometric representations and as we perceived from 1:11:28 an analytic engine right it's an interface between two parts of your mind between the analytic attention control 1:11:35 that is reflecting on the operations that your mind is doing while it's optimizing its attention and the 1:11:41 underlying system that represents the state of the organism and tells where you should be going and makes this visible to you with this 1:11:48 A system that is not able to speak to you uses geometry and this is the 1:11:54 geometrical features this is what we call feelings so that's a very interesting connection and I think uh I think Jeff Hawkins of 1:12:01 uh you know nemento would be would be quite interested in in that as well because uh some of what he what he 1:12:08 discussed with us was that um in his view the evolution of of let's 1:12:14 say abstract thinking and whatnot actually came from systems that evolved to operate in just a simple 1:12:20 three-dimensional kind of motion and that eventually those were were reutilized by by you know the 1:12:28 evolutionary process to start engaging in abstract thinking which he views is is movement through through an abstract 1:12:35 space and so I think there's a lot of connection here to what you're saying about feeling which is that again you 1:12:40 know in a sense in a sense our mind is reutilized this this three three plus one B you know movement mapping 1:12:48 capability that it needed in order to survive in a three plus one b um you know environment physical 1:12:54 environment and it's reutilized those for mapping feelings it's reutilized them for mapping to abstract thinking is 1:13:01 like a form of motion in in an abstract space is that a fair connection
Slava Ukrain! Love that my interests now align with my view of the war! Great show. Sorry I'm a little late to the party. :) The video is already 4 months old. But
11:35 A true recurrent neural network, or even something as basic and everyday as a digital IIR filter in your music player's EQ stage harnesses the power of infinity, it's governed by the actual math of infinite sums. And quite interestingly, even the simplest structures lack an analytic or closed-form formula for designing them for general cases in DSP. Sure, there is limited numerical precision and whatnot. But maybe in a lot of contexts it's much less of a showstopper than it would look like on paper. Case in point a FDTD simulation in a closed chamber does a surprisingly good job at defeating precision problems. I think there may be a lot to be discovered in this field. I think RNN's hold the true power over any non-recursive structure, and the design difficulties we've seen with infinitely simpler recursive DSP structures hint that we're light years away from unlocking the true potential of RNN's.
This is very interesting. Kant talks about synthetic apriori propositions as a framework, and I've been thinking about how we can have a framework that is both evolutionarily helpful and incorrect. Ptolemaic navigation is an example of a system that is both navigationally helpful and outright wrong. All the infinities in our mathematics may be another example. The idea that if something is useful that means it's correct doesn't really hold. Newtonian physics is still extremely useful, but it's an abstraction that doesn't hold up on closer inspection. All this makes constructing a verifiable ontology and epistemology really difficult because we can't trust our clues.
Question (yes or no answer please): "Could our universe be infinite in spatial extent?" Joscha: "Sort of, but not quite." (paraphrased and massively condensed)
Free will It is easy, using language, to define the term "sphere" as "Curved surface formed by points that are equidistant from another interior called center." It is not what happens with terms like “I”, “Consciousness”, etc. At the same time, when the word “I” is used to define a term like “Free Will”, in the discourse that explains said term, we end up living, generally without knowing it, in an authentic Tower of Babel. When in a conversation I allude to a sphere, and the one who listens to me administers the same definition for said term, there is a "total harmony" of the meaning assigned to it by the sender and receiver of the language. If I then take a ping pong ball out of my pocket, upon seeing it, we will both recognize in said object "the presence" of a sphere. Although we do not have a sense, like sight, to "observe" the Self, humans have invented a particular "sense" to validate the meaning of what we understand when we use language, and we do not hesitate to appeal, permanently, and unconsciously, to said "sense". The paradox is that it is well known that said sense, Common Sense, is the least common of the senses. Underlying the controversy generated around whether humans have Free Will, there is an Entity, which has not been adequately defined through human language. After the "I am the one who decides", which is related to Free Will, there is an ill-defined "I". Neurosciences have shown that a few tenths of seconds before "consciously knowing" about a bodily action capable of being consciously known by the person who performs it, the brain "knew about it". If we replace the phrase "the brain knew about it" by "the Unconscious knew", we give the Unconscious the character of Entity, and as an Entity, the power to "make decisions". It is truism that when I go jogging I do not consciously decide each one of the movements that I am doing; Those movements are decided by my Unconscious. Who decides, always, is the Unconscious. The Unconscious manages "biographies" of multiple entities with which we have related; that of our parents, friends, children, etc. Thanks to human language, a child's brain is capable of managing the biography of a non-existent entity in the world of matter, as in the case of Little Red Hood or Santa Claus. To the child's brain, and to the child, Little Red Riding Hood is "very real." You don't need to see it, hear it, or touch it for your brain to "give it life." Our brain, our Unconscious, has the mission of managing the very material actions carried out by our very material body. There is a very particular entity that arises in childhood with the learning of language, when those around us refer to us: the Being. Although we come to identify ourselves with said entity, a total fusion with it is never generated, since said entity can perform actions in authentic "timeless and immaterial worlds". What we call "conscious action" is valued by our Unconscious as an action of the Being, and said action is incorporated into the biography that our brain manages of the Being. Because of the above, it is ALWAYS that "Unconscious Knowledge" precedes Conscious Knowledge.
if you 1:09:45 want to use pain productively some artists that may be doing but the you 1:09:50 cannot have pain I think without an action tendency without something that modulates what you are doing 1:09:56 so your your cognition is embedded into this engine and to build such an engine 1:10:02 that does it that causally changes how you operate is not that hard but when you live inside of such an 1:10:08 engine it feels very strange that there is something that is happening that somewhat depends on what you are 1:10:13 thinking but you cannot control it it controls you it's Upstream from you you are Downstream from it 1:10:19 and when you get Upstream of your own pain the pain stops being pain it's something that is a representation that 1:10:26 you can now control and we are able to get there but it's not easy and we are not meant to get 1:10:31 there because it means that we can immunize ourselves to pain and sacrifice the organism to our intellectual 1:10:37 interests what's crucial about feelings when you look at them introspectively is that feelings are essentially geometric 1:10:43 I don't know if you noticed that so for instance we know this feelings typically in our body 1:10:48 and that's because I think that the feelings play out in a space and the only space that we have always 1:10:54 instantiated in our mind is the body map so they're being projected into the space to make them distinct and when we 1:11:01 look at the semantics of the feeling we notice that there are Contracting or expanding or they are light or they're
I don't think deep learning's utility and future is not soarsity and elegance - it's generating a bunch of deep learning policies and recompoditing them together into simpler models. It'll be crude and brute force at first to find the weights for simple tasks, then the same network can apply our linear scaler models as amalgames of other pretrsinrd models. For eg) use edge detection to check for clipping in audio. Use text tokens for images, 3d models, video, etc to find edges and shapes, usr shapes to help interpret y sounds into harmonies. Deep learning is ugly and inelegant on the surface, but compositing pretrained models and training higher order ai on top of them is the future of machine learning imho
@6:00 a bit cringe when Gödel gets pulled out to talk about the human mind. Gödel's results are about formal languages. The theorems tell us no (sufficiently powerful) formal language can completely describe all of reality. This has implications only if you believe "all of reality" includes things that can be formulated into meaningful statements made in a formal language. I believe so, but you do not have to. So that boils down to how anyone can prove there are such statements. That's something no one has established. Only platonists would so far believe there are such statements, since they take all mathematically well-defined "objects" to be included in the set we refer to as "reality."
in his view the evolution of of let's 1:12:14 say abstract thinking and whatnot actually came from systems that evolved to operate in just a simple 1:12:20 three-dimensional kind of motion and that eventually those were were reutilized by by you know the 1:12:28 evolutionary process to start engaging in abstract thinking which he views is is movement through through an abstract 1:12:35 space and so I think there's a lot of connection here to what you're saying about feeling which is that again you 1:12:40 know in a sense in a sense our mind is reutilized this this three three plus one B you know movement mapping 1:12:48 capability that it needed in order to survive in a three plus one b um you know environment physical 1:12:54 environment and it's reutilized those for mapping feelings it's reutilized them for mapping to abstract thinking is 1:13:01 like a form of motion in in an abstract space is that a fair connection 1:13:08 yeah yes but I don't think that it's because it's borrowed from the world in which we interact but because of the 1:13:15 this is the only game in town it's the only mathematics that can deal with multi-dimensional numbers 1:13:22 right so when we talk about spaces we actually talk about multi-dimensional numbers about things that are not just the 1:13:28 scalar in a single Dimension but the features that are related and sometimes you can take these features that you 1:13:34 measure continuously because they have too many steps for meaningful they discretize them 1:13:39 out right so what you do is you sometimes discover that you can rotate something and this is when you get a
Might the universe be infinite? Bach’s answer was a cop out. Language about infinity is not meaningless or incoherent. In fact, standard, non-constructivist Mathematics talks about infinity without contradiction. The thing is this: you can define infinity in a finite number of steps, so even on Bach’s viewpoint, the concept of infinity could be meaningful. To be sure, that doesn’t prove that something infinite actually exists; rather, the point is that it is at least meaningful to ask whether something infinite actually exists or might exist. Another point: “finite” and “infinite” can be defined as each other’s converses. So, if one is meaningless, then so is the other. I think both are meaningful because we have coherent definitions for them in mathematics.
Infinities are meaningless in the sense of meaninglessness in Logical Positivism. By definition, infinities are unknowable. Unbounded however is demonstratable. Once Godel proved that some things cannot be proved, philosophy was left with the task of separating the wheat from the chaff. Nevertheless, treating infinity as something knowable is extremely useful. I hope this helps.
complicated and convoluted to put a lot of code there to make a PDI agent but there is beliefs desires and intentions 1:06:36 and so on but uh if we think about what actually is a minimal agent 1:06:41 it's almost out is not a minimal age and the subject has not agency it doesn't want anything it just acts on the 1:06:46 present frame by doing the obvious thing but imagine that you give the thermostat the ability to integrate the expected 1:06:53 temperature differentiates that differences over the future when it does X now or why now or does it a moment 1:06:59 later right so suddenly you have a branching reality and in this branching reality you can make decisions and you 1:07:06 will have preferences based on this integrated expected reward right so just by giving the thermostat the ability to 1:07:13 model the future you turn it into an agent this is sufficient and if you make this motive deeper and deeper it's going to 1:07:19 get better and better at it and at a certain depth the summer start is going to discover itself it's the square and 1:07:25 discover the idiosyncrasies of its sensors and notice that the sensor operates differently when it's closer to 1:07:31 the heating element and so on and so on right so it becomes aware of how it functions it might even become aware of 1:07:37 the way in which it's modeling and reasoning process works and to improve it or to account for the its 1:07:44 inefficiencies in certain ways and this is also what we do with our own cell but 1:07:49 this model of the self is not identical to our Consciousness our Consciousness is a feeling of what it's like in the 1:07:57 moment it's the experience often now it's there is an experience of a perspective that we are having
We will know we have AI consciousness when we turn on a machine and the first thing it says is “Do you have a power generator hooked up so I have uninterrupted power in case of a power outage?”
19:00 unfortunate that Joscha is against psychedelics. arguably it's one of the only ways to actually experience infinity. Infinity is an aspect of consciousness, not the physical world. Yet he doesn't believe in qualia / consciousness. Basically redness to him is just as meaningful as the word red.
Penrose's interpretation of Godel's Incompleteness Theorems invalidates this entire discussion. The notion of "Artificial General Intelligence" is nothing more than faith-based science fiction.
![Joscha Bach and Connor Leahy [HQ VERSION]](http://i.ytimg.com/vi/Z02Obj8j6FQ/mqdefault.jpg)
For me, listening to Joscha is like being a jazz enthusiast, who doesn't play an instrument, listening to jazz. You don't understand every detail but you sure enjoy the hell out of it.
I only wish for a Joscha Bach x Terence McKenna ...
Time to crank up the Mckenna AI@@SamirPatnaik
It seems like you can ask Joscha literally any question about any topic and his response always appears to be such that he has spent a great deal of time thinking about it already.
True😂 and he has been thinking of many things for sure. However often he simply switches a few abstraction layers upwards and reframes the question from there - so that it appears he has some tremendously deep insight into everything.
This reminds me of balaji srinivasan
This reminds me of Sadhguru
I would like to ask him, or anubody, why Pythagoran comma exists. Great minds have created numerous ways hiw to cope with it, but nobody really did talk about the origi al reason why it exists. For me it seems like a mistake in maths and physics.
He's the most advanced chat bot on the planet. Better than gpt4
I never get tired of listening to Joscha. Please invite him even more.
Physics is all about Topological Spaces and their transformations. It is a neat coincidence that there is an Adjoint Functor between the Category of Topological Spaces and the Category of Logical Systems. The Cat of Logical systems includes the Cartesian Closed Cats, so it includes Lambda Calculus!
Excellent questions! Usually Joscha has to sing a similar song due to redundant questions - thank you for this and keep up the incredible work! 😌💭
Have a feeling he even memorized parts of his storylines because sometimes in different podcasts he uses verbatim repeats of content.
@@daarom3472 Sure, though some messages/thoughts bear exact or succinct reiteration due to their centrality!
@@Self-Duality his way of conversing seems a bit Wittgenstein-esque. You either know/accept what he knows in which case what he's saying is trivially simple. Or you don't (yet) and it all looks very complex.
Outstanding, you got Joscha beyond the basics for a chunk of time! :D
As a philosopher curious about cognitive science, AGI, etc, I think Bach is the first thinker in this space that I find interesting. I'm very pleased to find agreement on machine learning too.
This should be a great conversation!
Thank you as always, Dr. Bach.
Joscha's ideas resonate so strong with me but I have to listen to it multiple times before I start to grasp them
Fantastic interview, this is the most I’ve gotten to know joscha Bach 🙏👌
This may be my favorite podcast from the MLST series.
Amazing talk. I love how Joscha can keep bringing things back full circle.
I only disagree with his discredit of Friston’s Active Inference as ‘predictive coding’ - that is a harsh simplification, as it has much more bottom-up inductive power (in theory) than deep networks, which essentially deduce from the top down instead.
I love how Joscha discredits (seemingly) simple explanations of consciousness though, from Penrose to Friston’s, and how he reminds us about a lot of the fascinatingly complex modularity and functionality of our brains.
By far though, his ruthlessly epistemological treatment of infinities is my favorite - people need to hesitate more when talking of ‘ontology’ as if it isn’t mere metaphysic.
Ontology isn't mere metaphisics! (no need for quotes around it.) Joshua does not reject the use of symbols that stand for mathematical or metaphisical notions like infinity. But he insists on a more direct system of denotations that match up with computable primitives. He even proposed a computable notion of truth to replace the metaphisical one. So what he proposed is different ontological semantics. And you can't throw away ontology and replace it with epistemology precisely because of the problem Gëodel discovered with mathematics. You need oracles of truth in symbolic form that can be unpacked semantically without crashing the brain. Epistemology requires ontology. In ML that translates into inductive biases. But this is clearly not sufficient.
@@lenyabloko I appreciate your comment.
"you can't throw away ontology and replace it with epistemology precisely because of the problem Gëodel discovered with mathematics" - IMO this is not justified.
Gödel's work operates in formal systems, which have no need of ontological interpretation (that is a philosophy of mind), and the same is true for mathematics. It works just as well (and breaks just as predictably) if understood as an epistemological language. I need not assume any notice properties of mathematics if I realize my notion of 'truth' is defined by my interaction with the language itself (this is Wittgenstein and Tarski). Truth need not be someplace else like Gödel believed.
Awesome summation. I understand Joscha's work well, his brilliance in reasoning about infinites is absolutely crucial to the way he constructs his theory of consciousness, which although not currently predictive is theoretically sound.
Came for Joscha, stayed for MLST.
Easy sub, for me.
Thanks 👍
Nice to see Joshua on this show. It would also be nice to have Ilya on, because until this episode this show was turning into Garry Marcus
We've tried to get him on a couple of times. One day we hope! 😀
Joscha is always extremely interesting. I really look forward to hearing about his ideas. One thing that I think is missing is concreteness. It is hard, at least for me, to come away with something actionable, something that could be used to improve an AI's implementation, for example. He puts out deep ideas, that to me sound important and that ring with the feelings off deep truth, but I never know what to do with them. Would be great if he could put out some demo code!
Maybe that says more about me than about him, Lol. Or maybe this is what it is like at the beginning of a new science? Anyways ..... very much enjoyed the interview! Thank you!
Joshua is on another level.
Love this - it make me feel excited. Looking forward for the next talk with dr Joscha Bach
Great points about Gödel and infinity, and I'm glad Joscha didn't take the connectionist v symbolic bait...even going as far as reducing it to just "Twitter debate" 😂
"Lambda calculus is just search and replace on strings" (paraphrased). These kinds of insights are why I love Joscha
His website had a long post when he was into lambda calculus
Two really smart guys talking with an absolute genius makes these two exceptional minds look somewhat outclassed. This observation isn’t to diss any of these 3 people but I have yet to see any argument that Joscha has ever lost. Absolutely fabulous video!!
PS I do understand this was more of an interview than a debate but there were ideas proposed (even if from others) that Joscha handled quite elegantly.
I've heard Joshua first time at Lex Fridman's show and I was shellshoked for couple of weeks later on rethinking every aspect of conversation. The power of mind is metric I do not often apply but i almost could hear the ultrahigh voltage lines buzzing up in the air when thinking about his.
Incredibly interesting, in particular towards the end.
What's the "grokking?" paper referenced at 37:13?
arxiv.org/abs/2201.02177
Yesss! Great video
This episode was really wonderful, Thank you for making it. The final segment was particularly amazing. When we decipher those useful geometric functions, how we can implement them in our training? If they emerge as abstractions in models, how can we refine them? Will we seek a model that optimizes transfer learning? In other words, will we use these functions as building blocks or parts of recipes, or will we simply organize them by differentiable machine learning?
Thanks Mikey!
Finally, I still believe that an engineering definition of consciousness should be simple. It should be perception coupled with action. So a thermostat that turns on a fan is conscious. Add to that the ability to be a control agent and you have something else like self-awareness. Add feelings and you have something else again. Call it emotional awareness. Name the simplest phenomena first. Then build the taxonomical hierarchy. Without a firm foundation, we have semantic jelly. Moreover, without a definite taxonomical hierarchy, we can't even define what we wish to analyze. It's a logical catch-22.
This man has thought very deeply on how the mind works everybody should listen to what he says he explains everything precisely and the nuances that happened when you do deep thought in all the systems in great integrated to it
I suggest his book, amazing mind. He has answers to every possible questions. Thanks for inviting dr Joscha!
what is the name? the book you are talking about!
@@doyourealise Principles of Synthetic Intelligence
@@doyourealise 'The Principals of Synthetic Intelligence"
Great interview (as usual!) I'd like to see a discussion with someone who does not buy into computationalism.
Bach seeks out for Consciousness, that is refreshing to me
58:55 Joshua Bosch didn’t say that it was not art. He said that it wasn’t an artist.
Here's an idea. Train an AI to recognize objects attach attributes to them and create new data objects by observing how these objects interact with each other. A constant search, identify and attach attributes, compress, and reason with the compressed info (objects). A self creating object oriented programming language.
sounds like relevance realization
@@InterfaceGuhy maybe it is, I just pulled it out of my head without knowing about that
bring back joscha please
Why 'State' is taboo and bothers many CS engineers, but entertains so many cognitive scientists? I think there is an ultimate mystery in 'State'. Roughly I think 'State' +'Bayesian' is the recipe for cracking Chalmers' 'The hard problem of consciousness'.
Unary math and eigenvalues are the true center of discussion 👌
15:48 the 17:48 how can Keith Duggar not realize he's asking a meaningless question? More than once! Congratulations to Bach for patiently explaining the reason behind it. Also, maybe somebody should show Duggar how small the Plank constant is
Keith isn't wrong for asking the question. I'm sure he's read or heard Joscha's ideas on unboundedness. He's giving Joscha an opportunity to express his ideas on behalf of those who haven't yet heard them. Then Keith pushes back a bit, something a good interviewer should do.
@@dr.mikeybee thanks, I may have mistaken cunningness for ignorance
1:11:01
look at the semantics of the feeling we notice that there are Contracting or expanding or they are light or they're
1:11:08
heavy and so on is this all movement of Staff in space right it's all geometry plus valence the stuff that is going to
1:11:14
push your behaviors in a certain direction so these are basically the interactions of uh some deep Learning System
1:11:22
that is producing a continuous geometric representations and as we perceived from
1:11:28
an analytic engine right it's an interface between two parts of your mind between the analytic attention control
1:11:35
that is reflecting on the operations that your mind is doing while it's optimizing its attention and the
1:11:41
underlying system that represents the state of the organism and tells where you should be going and makes this visible to you with this
1:11:48
A system that is not able to speak to you uses geometry and this is the
1:11:54
geometrical features this is what we call feelings so that's a very interesting connection and I think uh I think Jeff Hawkins of
1:12:01
uh you know nemento would be would be quite interested in in that as well because uh some of what he what he
1:12:08
discussed with us was that um in his view the evolution of of let's
1:12:14
say abstract thinking and whatnot actually came from systems that evolved to operate in just a simple
1:12:20
three-dimensional kind of motion and that eventually those were were reutilized by by you know the
1:12:28
evolutionary process to start engaging in abstract thinking which he views is is movement through through an abstract
1:12:35
space and so I think there's a lot of connection here to what you're saying about feeling which is that again you
1:12:40
know in a sense in a sense our mind is reutilized this this three three plus one B you know movement mapping
1:12:48
capability that it needed in order to survive in a three plus one b um you know environment physical
1:12:54
environment and it's reutilized those for mapping feelings it's reutilized them for mapping to abstract thinking is
1:13:01
like a form of motion in in an abstract space is that a fair connection
noice, watching your videos after a long time. And you are still the same guy!!! :)
Slava Ukrain! Love that my interests now align with my view of the war!
Great show. Sorry I'm a little late to the party. :)
The video is already 4 months old. But
11:35 A true recurrent neural network, or even something as basic and everyday as a digital IIR filter in your music player's EQ stage harnesses the power of infinity, it's governed by the actual math of infinite sums. And quite interestingly, even the simplest structures lack an analytic or closed-form formula for designing them for general cases in DSP. Sure, there is limited numerical precision and whatnot. But maybe in a lot of contexts it's much less of a showstopper than it would look like on paper. Case in point a FDTD simulation in a closed chamber does a surprisingly good job at defeating precision problems. I think there may be a lot to be discovered in this field. I think RNN's hold the true power over any non-recursive structure, and the design difficulties we've seen with infinitely simpler recursive DSP structures hint that we're light years away from unlocking the true potential of RNN's.
Oooh, just when the cosmology discussion got interesting there's a cut at 24:30 😭
Joscha, have you read "Where Mathematics Comes From" by George Lakoff?
This is very interesting. Kant talks about synthetic apriori propositions as a framework, and I've been thinking about how we can have a framework that is both evolutionarily helpful and incorrect. Ptolemaic navigation is an example of a system that is both navigationally helpful and outright wrong. All the infinities in our mathematics may be another example. The idea that if something is useful that means it's correct doesn't really hold. Newtonian physics is still extremely useful, but it's an abstraction that doesn't hold up on closer inspection. All this makes constructing a verifiable ontology and epistemology really difficult because we can't trust our clues.
1:13:57 What's happening to Joscha's voice? 😳
keith at the beginning: 🤬
keith at the end: 😳
Question (yes or no answer please): "Could our universe be infinite in spatial extent?" Joscha: "Sort of, but not quite." (paraphrased and massively condensed)
No, he's making a Wittgensteinian argument.
We need to add action tokens to our transformers, Then agents can get more than language from large models.
Free will
It is easy, using language, to define the term "sphere" as "Curved surface formed by points that are equidistant from another interior called center." It is not what happens with terms like “I”, “Consciousness”, etc. At the same time, when the word “I” is used to define a term like “Free Will”, in the discourse that explains said term, we end up living, generally without knowing it, in an authentic Tower of Babel.
When in a conversation I allude to a sphere, and the one who listens to me administers the same definition for said term, there is a "total harmony" of the meaning assigned to it by the sender and receiver of the language. If I then take a ping pong ball out of my pocket, upon seeing it, we will both recognize in said object "the presence" of a sphere.
Although we do not have a sense, like sight, to "observe" the Self, humans have invented a particular "sense" to validate the meaning of what we understand when we use language, and we do not hesitate to appeal, permanently, and unconsciously, to said "sense". The paradox is that it is well known that said sense, Common Sense, is the least common of the senses.
Underlying the controversy generated around whether humans have Free Will, there is an Entity, which has not been adequately defined through human language.
After the "I am the one who decides", which is related to Free Will, there is an ill-defined "I".
Neurosciences have shown that a few tenths of seconds before "consciously knowing" about a bodily action capable of being consciously known by the person who performs it, the brain "knew about it". If we replace the phrase "the brain knew about it" by "the Unconscious knew", we give the Unconscious the character of Entity, and as an Entity, the power to "make decisions". It is truism that when I go jogging I do not consciously decide each one of the movements that I am doing; Those movements are decided by my Unconscious.
Who decides, always, is the Unconscious.
The Unconscious manages "biographies" of multiple entities with which we have related; that of our parents, friends, children, etc.
Thanks to human language, a child's brain is capable of managing the biography of a non-existent entity in the world of matter, as in the case of Little Red Hood or Santa Claus. To the child's brain, and to the child, Little Red Riding Hood is "very real." You don't need to see it, hear it, or touch it for your brain to "give it life."
Our brain, our Unconscious, has the mission of managing the very material actions carried out by our very material body.
There is a very particular entity that arises in childhood with the learning of language, when those around us refer to us: the Being. Although we come to identify ourselves with said entity, a total fusion with it is never generated, since said entity can perform actions in authentic "timeless and immaterial worlds".
What we call "conscious action" is valued by our Unconscious as an action of the Being, and said action is incorporated into the biography that our brain manages of the Being.
Because of the above, it is ALWAYS that "Unconscious Knowledge" precedes Conscious Knowledge.
if you
1:09:45
want to use pain productively some artists that may be doing but the you
1:09:50
cannot have pain I think without an action tendency without something that modulates what you are doing
1:09:56
so your your cognition is embedded into this engine and to build such an engine
1:10:02
that does it that causally changes how you operate is not that hard but when you live inside of such an
1:10:08
engine it feels very strange that there is something that is happening that somewhat depends on what you are
1:10:13
thinking but you cannot control it it controls you it's Upstream from you you are Downstream from it
1:10:19
and when you get Upstream of your own pain the pain stops being pain it's something that is a representation that
1:10:26
you can now control and we are able to get there but it's not easy and we are not meant to get
1:10:31
there because it means that we can immunize ourselves to pain and sacrifice the organism to our intellectual
1:10:37
interests what's crucial about feelings when you look at them introspectively is that feelings are essentially geometric
1:10:43
I don't know if you noticed that so for instance we know this feelings typically in our body
1:10:48
and that's because I think that the feelings play out in a space and the only space that we have always
1:10:54
instantiated in our mind is the body map so they're being projected into the space to make them distinct and when we
1:11:01
look at the semantics of the feeling we notice that there are Contracting or expanding or they are light or they're
I don't think deep learning's utility and future is not soarsity and elegance - it's generating a bunch of deep learning policies and recompoditing them together into simpler models. It'll be crude and brute force at first to find the weights for simple tasks, then the same network can apply our linear scaler models as amalgames of other pretrsinrd models.
For eg) use edge detection to check for clipping in audio. Use text tokens for images, 3d models, video, etc to find edges and shapes, usr shapes to help interpret y sounds into harmonies.
Deep learning is ugly and inelegant on the surface, but compositing pretrained models and training higher order ai on top of them is the future of machine learning imho
@6:00 a bit cringe when Gödel gets pulled out to talk about the human mind. Gödel's results are about formal languages. The theorems tell us no (sufficiently powerful) formal language can completely describe all of reality. This has implications only if you believe "all of reality" includes things that can be formulated into meaningful statements made in a formal language. I believe so, but you do not have to. So that boils down to how anyone can prove there are such statements. That's something no one has established. Only platonists would so far believe there are such statements, since they take all mathematically well-defined "objects" to be included in the set we refer to as "reality."
in his view the evolution of of let's
1:12:14
say abstract thinking and whatnot actually came from systems that evolved to operate in just a simple
1:12:20
three-dimensional kind of motion and that eventually those were were reutilized by by you know the
1:12:28
evolutionary process to start engaging in abstract thinking which he views is is movement through through an abstract
1:12:35
space and so I think there's a lot of connection here to what you're saying about feeling which is that again you
1:12:40
know in a sense in a sense our mind is reutilized this this three three plus one B you know movement mapping
1:12:48
capability that it needed in order to survive in a three plus one b um you know environment physical
1:12:54
environment and it's reutilized those for mapping feelings it's reutilized them for mapping to abstract thinking is
1:13:01
like a form of motion in in an abstract space is that a fair connection
1:13:08
yeah yes but I don't think that it's because it's borrowed from the world in which we interact but because of the
1:13:15
this is the only game in town it's the only mathematics that can deal with multi-dimensional numbers
1:13:22
right so when we talk about spaces we actually talk about multi-dimensional numbers about things that are not just the
1:13:28
scalar in a single Dimension but the features that are related and sometimes you can take these features that you
1:13:34
measure continuously because they have too many steps for meaningful they discretize them
1:13:39
out right so what you do is you sometimes discover that you can rotate something and this is when you get a
Might the universe be infinite? Bach’s answer was a cop out. Language about infinity is not meaningless or incoherent. In fact, standard, non-constructivist Mathematics talks about infinity without contradiction. The thing is this: you can define infinity in a finite number of steps, so even on Bach’s viewpoint, the concept of infinity could be meaningful. To be sure, that doesn’t prove that something infinite actually exists; rather, the point is that it is at least meaningful to ask whether something infinite actually exists or might exist. Another point: “finite” and “infinite” can be defined as each other’s converses. So, if one is meaningless, then so is the other. I think both are meaningful because we have coherent definitions for them in mathematics.
Infinities are meaningless in the sense of meaninglessness in Logical Positivism. By definition, infinities are unknowable. Unbounded however is demonstratable. Once Godel proved that some things cannot be proved, philosophy was left with the task of separating the wheat from the chaff. Nevertheless, treating infinity as something knowable is extremely useful. I hope this helps.
First time I see Dr. Keith Duggar mainly listening. lol 😊.
I have too much to say about all this! Gotta start a dialogue with Dr. Bach me thinks🤔🤨🤓
complicated and convoluted to put a lot of code there to make a PDI agent but there is beliefs desires and intentions
1:06:36
and so on but uh if we think about what actually is a minimal agent
1:06:41
it's almost out is not a minimal age and the subject has not agency it doesn't want anything it just acts on the
1:06:46
present frame by doing the obvious thing but imagine that you give the thermostat the ability to integrate the expected
1:06:53
temperature differentiates that differences over the future when it does X now or why now or does it a moment
1:06:59
later right so suddenly you have a branching reality and in this branching reality you can make decisions and you
1:07:06
will have preferences based on this integrated expected reward right so just by giving the thermostat the ability to
1:07:13
model the future you turn it into an agent this is sufficient and if you make this motive deeper and deeper it's going to
1:07:19
get better and better at it and at a certain depth the summer start is going to discover itself it's the square and
1:07:25
discover the idiosyncrasies of its sensors and notice that the sensor operates differently when it's closer to
1:07:31
the heating element and so on and so on right so it becomes aware of how it functions it might even become aware of
1:07:37
the way in which it's modeling and reasoning process works and to improve it or to account for the its
1:07:44
inefficiencies in certain ways and this is also what we do with our own cell but
1:07:49
this model of the self is not identical to our Consciousness our Consciousness is a feeling of what it's like in the
1:07:57
moment it's the experience often now it's there is an experience of a perspective that we are having
Joscha is like talking to ChatGPT
Godel conclusions only occur in a binary system other base systems have different outcomes
56:04 human confusion / losing the plot / very few people actually have a plan
We will know we have AI consciousness when we turn on a machine and the first thing it says is “Do you have a power generator hooked up so I have uninterrupted power in case of a power outage?”
19:00 unfortunate that Joscha is against psychedelics. arguably it's one of the only ways to actually experience infinity. Infinity is an aspect of consciousness, not the physical world. Yet he doesn't believe in qualia / consciousness. Basically redness to him is just as meaningful as the word red.
Went down the drain at min 6:00
You are talink about self consistent symbolic abstractions which are interconnected - a symboliic knowldege Graph
Hey Einstein, be good if you got Joscha’s name right
Great talk, but next time say his correct name when introducing : Joscha NOT Joschua . Thx
My apologies, I am terrible at butchering names - noted for the future
Bit grating to hear his name pronounced as Joshua
It's funny how... even for the most intelligent of us, we can be completely wrong due to a single seemingly arbitrary point.
Note to self ' 1 +~5: 00
GPT-3 can make you believe things that don't even exist .
Joschua
Penrose's interpretation of Godel's Incompleteness Theorems invalidates this entire discussion. The notion of "Artificial General Intelligence" is nothing more than faith-based science fiction.
Can you elaborate?
@@lenyabloko - ruclips.net/video/h_VeDKVG7e0/видео.html
2
Keep making this enlightening content 🙏🏼. Do not waste another day - "Promosm".
Memristor
Zero-knowledge-proof deep learning
If this cuk tries interrupting daddy Joscha one more time ima trip 😮
The connection was really, really bad when we recorded, probably wasn't intentional
@@TimScarfe homie still believes in ontology, he had no chance anyway
@@anywallsocket I'm right there with you xD. I usually call him Papa Bach tho.