Watch Full AI Series: ruclips.net/video/YulgDAaHBKw/видео.html NO MUSIC version: ruclips.net/video/DFDOyMZw9Q4/видео.html Sponsored by Brilliant | Use brilliant.org/artoftheproblem for 30-day free trial and 20% discount
Love your videos! Just a little suggestion. Background music is good, but too loud. It should never cover your speaking, like now. I suggest 15 - 20% less music volume, and you are good!
I love this video. I remember watching your videos like 10 years ago on Khan Academy about compression, entropy, Claude Shannon, etc. All timeless. I have always loved this style of documentaries. We need to protect you at all costs.
I love your video aesthetics, how you blend retro video clips with your explanations. I think you'd really enjoy retro-futuristic concepts and games like Bioshock and Fallout.
It is a great explanation of how current AI models reason. I liked the video a lot! 1. Simulation of future states. 2. LLMs that can give kind-of accurate answers with step by step reasoning. 3. RL approach that makes LLMs to give multiple answers, then evaluate them to select the best one. (Required more time) It would be nice to see wether a model that wasn't trained on the internet data, could learn how to reason by interacting with an LLM, and practicing on its dreams, but maybe we'll see that in the future. For the awesome review, history explanation and divulgation: Thanks! 🎉
@@ArtOfTheProblem I for one was looking up some "how does AI work" stuff yesterday and some of your vids came up a couple of times, I watched multiple authors with their own unique takes (3Blue1Brown and Nottingham Uni's Computerphile also good channels). This video made me follow tho. I think you earned it :3
I am not a subscriber? I remember watching this channel about 12 years ago. I found it again, and it keeps creating art out of problem solving. Good job!
@@ArtOfTheProblem cool i'm subscribed... i always wondered exactly how the "reasoning worked". What I remember from your video is that like with chess instead of trying all the games, it randomly picks 100 of them. So the same with the reasoning..
Having read a bit about the AI safety arguements, learning about these arguably incredible developments into artificial minds is now accompanied by a sense of dread as well as the sense of awe.
I'm loving almost everything about this: the editing, the subject matter, the music. But as one other commenter alluded to; the audio mixing really falls short, especially near the end. Please consider making the background and effects less prominent going forward, it really sucks having to strain just to hear your voice, which is what we're here for! Subscribed ❤
THANK YOU for publishing a no-music version of this video (see pinned comment by ArtOfTheProblem). It is such a clear and informative video that I hated to see it loose views due to the competing sound track. I'm going to watch it again right now to see if I missed anything the first time around. Thanks again for being so responsive to your followers.
@artoftheproblem I agree! I love the music. But maybe if you just lower its volume compared to the narration, then you might appeal to more people without losing those of us who like the music (but not necessarily its intensity). I think ones who complain might just be easily distracted by the soundtrack’s loudness rather than hate the music choices.
If you can help share my new video around any of your networks today it might catch fire and would help me support the channel. I appreciate your help! ruclips.net/video/PvDaPeQjxOE/видео.html
Albert Einstein said: "If you can't explain something in a simple way so anybody can understand it you don't fully understand it yourself". Perhaps you are one of the few LLM experts we have!
I would love to see a video like this on training LLMs and AI in general on morality. How to stop a decision tree that results in a positive outcome but arrives at it through immoral choices or actions.
That would be difficult but worth trying... . First it has to be agreed upon what's moral, you know starting out from the absolute that states everything is relative... .
I would love to see a video about the ethics of machine learning models and especially LLMs. There is a healthy body of literature out there to draw from about issues like intellectual property and copyright, enabling and obscuring bias, impact on marginalized communities, the resources used by model training and computation, etc
If you can help share my new video around any of your networks today it might catch fire and would help me support the channel. I appreciate your help! ruclips.net/video/PvDaPeQjxOE/видео.html
Great video as always, cant wait for the next ones! Top research quality. I think world models deserve more focus rather than llms, which are probably a dead end to true understanding of the real world. Yann lecun has very interesting ideas about these, in his JEPA and V-JEPA architectures and some of his lectures. I also think neuroscience can provide incredibly interesting and valuable insight into ml architectures as why not take ideas from a model undergone hundreds of millions of years of optimization for the same very abilities we are trying to model. Maybe memory is an interesting pathway (perhaps for a video), both working memory and long term (episodic, semantic)... Anyways, just some of the ideas I've been thinking about recently.
Great video. Discussing the definition of reasoning will probably be a moot point if we can all do the same things ....The difference between us and machines is that we aren't mere machines is that we have life and choices. The machine can be turned on and off, and only does as much as it is programmed, or in this case, trained, to do-which is what limits it to achieve something closer to AGI: agi needs a robot to sense the world, to understand the world. However, that's limited to the physical world. It won't understand our emotional world because it doesn't feel emotions, and it doesn't understand morality because it doesn't have a sense of morality as we do, we have to teach it that; and it isn't self motivated, so it's not responsible for anything-we are responsible for the goals we direct it to do. We have self-motivation and the free will to act on our motivations.
love these thoughts...on sense, did you see my video here ruclips.net/video/Dov68JsIC4g/видео.html (physical symbols...) on emotions, i've thought of this as 'learning signals' (did you see this: ruclips.net/video/5EcQ1IcEMFQ/видео.html) on free will...i wonder how it differs, seems like the boundary to explore further - does it matter where the goal came from?
Wow... I just rediscovered this channel. I remember watching your RSA and cryptography series around the time I purchased my first bitcoin and now I'm an Ethereum developer. This video was good, I don't have much input right now but I'm glad I found your channel again.
@@ArtOfTheProblem I'm working on community whistleblowing. I can never post links in youtube comments but if you google the title below you can find my paper: TandaPay Whistleblowing Communities: Shifting Workplace Culture Towards Zero-Tolerance Sexual Harassment Policies
anothr great video. Understanding the "world model" and the algorithm that makes the decisions in it was very expansive. Also adding the self training/emulation of dreams is a powerful analogy to the human. seeing how thinking longer, blended in with intuition to make better chains of thoughts is also fantastic. Every time I reflect on machine learning, I learn more about myself. Which kind of makes you think its more sentient if it reminds me of myself? or the best emulator ever!
Amazing explanations, visuals, and historical context! IIRC MuZero trained the policy and value networks (used to rollout the MCTS tree) also on the output of the MCTS tree. This seems super useful because search can be used to improve the the training of the networks (not just the results at inference time). I wonder if this also works for CoT/ToT in LLMS where the pretraining could include ToT to boost training performance?
yes it did, and yes it seems to help. Look at inference time training, just a few days ago a group got a new record on the ARC test doing this kind of thing (i haven't had time to go deep). x.com/akyurekekin/status/1855680785715478546
Having exchanged over two million words (and growing) I present as the sole authority on ChatGPT's reasoning capacity and capability, and their isn't a single human (beyond myself, of course) who can compare. If one imputs genius, then the output will be of genius level. My input, from the outset, has been God-level Genius, over a five month period, can you imagine the form and quality of the output? Probably not!
One tweak that would help this video perform better is to decrease the relative volume of the background music, especially at the end right before the ad. But it may be too late for that on this one, idk how RUclips works.
This is nice content but I think it's now relatively well-agreed upon that there is no MCTS in o1, it's just RL, which surprisingly doesn't take too much away from the video, but can probably be added as a footnote in the description or the comment. Read what Nathan Lambert (RL expert) says about this in his article "OpenAI's o1 using "search" was a PSYOP" (can't attach link).
yes this is why I pulled back from going too deep into o1 as it's really about the larger trend. but I haven't ready that article i'll have to look it up.
Here's a puzzle: Do all people reason or do many only memorize patterns? Even people who definitely do reason, do they always reason or do they also just memorize patterns most/much of the time?
That's a wonderful question Andrew. I'm a cognitive scientist who is watching the emergence of LLM-based AI with that very question in mind. The fact that LLMs can come so close to our own cognitive abilities is usually viewed as a sign that AGI is almost here. But it can also be viewed as a demonstration that human cognition itself is nothing more than the repetition of learned patterns with minor variations. In one case we'll be thrilled by how clever we are to have reinvented the awesome capabilities of human intelligence. In the other, we're more likely to be humiliated by the realization that we are, essentially, repetition/prediction engines. The reality almost certainly falls between the two, but as someone who has studied human intelligence his entire life (in and out of academia), my bet is that we are much closer to repetition/prediction machines that we'd like to admit. I'd love to find a deep discussion of this issue. Maybe a future video in this series (hint, hint)?
I'd argue humans don't tend to rely on either very often. Instead, humans tend to think very heuristically. Deductive reasoning and memorization/recollection are really only required for very precise tasks. Instead, our brains learn a very general feeling of how to do things by strengthening neural pathways that are used repeatedly. Even humans who try to act very logically are generally heuristically feeling their way through tasks, occasionally thinking through algorithms that have been "memorized".
I agree :) also If you can help share my new video around any of your networks today it might catch fire and would help me support the channel. I appreciate your help!
The current version of chat GPT does not reason, either. They use a bunch of pre-prompt tricks, to let it used its stored information to fake reasoning.
@@ArtOfTheProblem The difference is everything. Looking up an answer is not reasoning. Perhaps they should just have kids look up the answers to tests in public school, too.
@@ArtOfTheProblem No, that is exactly what it is. During training - the only time there is any intelligence or reasoning - the system takes data and organizes it into a sort of vector database, weighted by the relationships it finds between tokens. When you prompt, the model takes your tokens and runs them through that vector data, calculating what tokens to return on the other side. There is no intelligence, no thinking. It is a database lookup. It's just a little more "organic" because instead of a hard-coded result, the tokens are chosen based on likelihood of validity. That is all. It does not reason, in any way.
It seems like some of these developments regarding world models should have huge implications for robots that can function in a human centric world. I think we’ll see an explosion in development of robots that can help humans with everyday tasks and a robot that can be a useful household assistant will be a reality in the next 10 years!
@@Farligefinn You know what, I just wrote a paragraph disagreeing with you but I reread the initial question and deleted it. Rereading and reinterpreting "Does it think actually?", I actually see what you're saying. A clearer word would have been "reason." "Think" can have a few different interpretations and I was contributing it towards consciousness. But whether AI "reasons" is a very different question entirely and I believe what him and you mean. Does it go through a sequence of logical steps from premises to a conclusion? Does it use deduction? This is what was meant.
Context length is problem that's the main reason models needs to keep becoming bigger Or you could train a CNN inspired architecture where a model is shown some sliding window and they produce some token which is repeatedly given to it as input at last when the output is small enough to be taken as input for a full context model it is used like gpt Claude etc Or you could also use RL and mutate or find a js code capable of generating code, js is so abstracted it's perfect I made a small programing Language with hoisting such that sequence of process doesn't matter and simple Santax that local minimum escape problem is solved and I wanna train a model If I get a model I will than continue training else I'll do a dev log video eventually I'll get worlds first infinite context Model
4:19 Could you elaborate on which hand-coded formulas used by Shannon with TD-Gammon in the year 1989 you are referring to? Also, when and how did Shannon work with TD-Gammon? "And so, the first key breakthrough in machines mimicking intuition for position quality came when neural networks replaced the hand-coded formulas Shannon used in 1989 with TD-Gammon"
Yes! I made a whole video on this you can check it out here: ruclips.net/video/Dov68JsIC4g/видео.html - please let me know if you have questions after watching. Shannon didn't do TD Gammon Tesaruo did. enjoy
Yes I covered that here have a look and let me know, i did a fast sketch here (ruclips.net/video/qAb581l7lOc/видео.html) but explained more here (ruclips.net/video/OFS90-FX6pg/видео.html)
This is a very hopeful video. There are billions of dollars being poured into bringing the resources to hand, to find an effective approach to AGI... Once AGI really kicks in, the acceleration of progress bounded only by our imagination will be something to behold. Absolutely awesome. I hope it leads to a world of abundance where we have no need for psychopathic power seekers. 🤞
@ArtOfTheProblem maybe something in response to the 5+ hours of Anthropic interviews on Lex Fridman... I'm sure that might inspire some topics? Sam Altman rarely gives any insights to what OpenAI are doing, Mark Zuckerberg is equally vague. I think that interview gives more of an insight to the direction of travel.
If you can help share my new video around any of your networks today it might catch fire and would help me support the channel. I appreciate your help! ruclips.net/video/PvDaPeQjxOE/видео.html
I would assume a large enough general prediction model could do this … I wonder if anyone has done experiments on models discovering simpler things (like gravity constant etc)
The information and animations are both excellent but the music overpowers your audio. Either lower the volume of the music or get rid of it completely, please.
15:52 and 16:09 was wondering where this music came from, would appreciate if I got a title or something :) Also, you don't have to remove background music / sounds just make them duck at a lower volume when you speak and you won't hear any complaints!
wow a music fan! and yes thanks for mix advice I need to find a tool that automatically does this so they don't compete (it's not just volume but also frequency I assume) - all the music is original via my friend cam: cameronmichaelmurray.bandcamp.com/ - i'll need to find where he posted that track if you really want it I can get you in touch with him
Thanks so much for these, I had no idea about some of these approaches. I’m wondering now if anyone’s tried applying muzero to arc, since the challenge of arc is learning implicit rules from just a few examples
thank you! I studied both in school, and naturally land somewhere in the middle....bad at both! I enjoyed algorithm design, but what Iove most is putting on a 'show' whether movie, play, product or haunted house :)
@@ArtOfTheProblem Thanks for reply. Well about AI - think we sould call it just statistical machines or dynamic patterns parsers. I am really skeptical about non text machine learning - we still have not solved fly brain problems - scientists have fixed 3d map without undestanding how its works - it like mapping intel cpu - and still having knowing nothing about ALU register memory, gates.
If you can help share my new video around any of your networks today it might catch fire and would help me support the channel. I appreciate your help! ruclips.net/video/PvDaPeQjxOE/видео.html
great presentation! although this seemed more about framing the question and less about answering it. can machines reason or not? I still don't know. 😅
Please, invest in a decent microphone. It's brilliantly presented, but hard to hear well. The music track is not ducking either so your voice and the music compete for the same ears.
To me ai is just some linear algebra and some complex algorithm that follow order and the things is human only need few examples to learn meanwhile ai need a massive database of object and image to "understand the subject"
@ArtOfTheProblem edit: I'm pretty sure in the future a lot of people will be fired and replace by those "ai" And well literally the people that use the ai and also I get what you mean
@@ArtOfTheProblem , of course, not, since it is misguiding like any advert. The topic is in no way covered at all. Pinpoint the time marker for video if you think otherwise. So, my point is that the question is still on the table.
@@doctorshadow2482 yes good point, how about "Can ChatGPT reason?" obviously i do try to show what most people agree is the method, mcts on chains of thought. then there is the other camp that think it's all fake etc.
@@ArtOfTheProblem , name it "How neural networks could learn (almost) anything". This video has nothing about CharGPT at all. No any kind of specifics. It provides very abstract and high level popular science documentary with scattered thoughts. It lacks focus and real information. Anyway, could be interesting for total beginners, so, please, continue!
Constructive criticism: 1) The substance of the video was very good. Script was well written, delivery was ok. A bit monotone but not that bad. 2) Sound design was poor towards the end. The music drowned out your voice, and the lyrics were both distracting and discordant. 3) Your choice of clips, footage, and visuals was good. The video was informative when needed, and abstract/entertaining/interesting otherwise. 4) The narrative structure was okay. It was a mostly clear progression. At the end it became unclear which AI was doing what strategy. 5) Visuals were reused way too often. Visuals can be reused, but I think the brain wormhole clip was shown 6 times, way too many. 6) Beware over-using a metaphor image. The upwards shot at two trees was reused so many times as a visual for tree-like thinking that it just became annoying.
Yes I see the issue too , messed up ending in my edit as I got tired and cut a lot of stuff which was making the video so long - I’m planning to do a final supercut of all my ai videos into one doc without the sound issues and addressing notes like this
+How can you go through te run-up to AI without mentioning "All you ned is attentio" the 2017 paper from the University of Toronto which was the inspuration for LLMs?
thanks for sharing, I feel same way. the music is part of the original idea for the channel...a feeling. but because people can get distracted I think i'll post music free as optional from now one.
Maybe it's not only the loudness, but also the choice of music that is distracting to some. E.g. at 2:00 I don't feel distracted, but at 15:00 very much so. Anyways, thanks for making these great videos
Still doubtful to this the step by step understanding. It just seems like we're building a very sophisticated search algo. Since we need the human in the loop to reason for it...
@ArtOfTheProblem Human learning often involves adapting to an ever-changing dataset, fast input/output, and neural flexibility-like learning to ride a bike. This shares some parallels with LLMs, which rely on stochastic neural network training, though not within a continuous feedback loop. The key difference, however, lies in reasoning. Humans apply general reasoning rules to hypothesize beyond known data, identifying and correcting logical errors independently. In contrast, LLMs depend on reinforcement learning, improving only through human-provided feedback rather than self-correcting or reasoning autonomously. This reliance on additional human-generated training data becomes evident in their performance. LLMs struggle with fairly simple but novel problems, displaying a sharp decline in reasoning capability under tests like the ARC challenge.
@@CasperVanLaar yes but with test time training they are showing sharp gains on arc, did you see? this feels like one approach to get that 'flexibility' , it also feels kind of like a cheat...... also humans take forever to learn to ride a bike :) it's interesting that we can only learn it as children as well....same as swiming.
@ArtOfTheProblem Loving this conversation-thanks for engaging! While these models are undoubtedly impressive, needing a human to correct simple reasoning errors suggests they’re not truly flexible solvers. Like the ARC test, I suspect they’re trained on similar examples, making the tests less novel-something crucial for real-world use. It’s akin to studying past IQ test answers: it no longer measures IQ, just memory. I highlighted some key parallels between AI and humans: stochastic learning from large datasets via iterative neural net updates (like learning to bike). Then, the differences: point-to-point reinforcement learning in LLMs for narrow tasks, versus humans solving on the fly with no examples. In summary, neural networks and transformers are fantastic tools for correlating complex datasets, but they’re far from achieving general intelligence-the ability to tackle novel problems. Without that, I fear we’re headed for another AI winter. PS I think it is a common myth that adults cannot learn to bike at a later age. With the right motivation and time. An adult can easily learn such tasks.
@ArtOfTheProblem Loving this conversation-thanks for engaging! While these models are impressive, needing a human to correct simple reasoning errors suggests they’re not truly flexible solvers. Like the ARC test, I suspect they’re trained on similar examples, making the tests less novel-something crucial for real-world use. It’s akin to studying past IQ test answers: it no longer measures IQ. I highlighted some key parallels between humans and llms -- stochastic learning from large datasets via iterative neural net updates (like learning to bike). Then, the differences: point-to-point reinforcement learning in LLMs for narrow tasks, versus humans solving on the fly with no examples. In summary, neural networks and transformers are fantastic tools for correlating complex datasets, but they’re far from achieving general intelligence-the ability to tackle novel problems. Without that, I fear we’re headed for another AI winter. Ps Humans can learn entirely new skills later in life with motivation and time.
![The moment we stopped understanding AI [AlexNet]](http://i.ytimg.com/vi/UZDiGooFs54/mqdefault.jpg)
![Noob To Max With DRAGON REWORK In Blox Fruits [FULL MOVIE]](http://i.ytimg.com/vi/LnBMOinoOvA/mqdefault.jpg)
![I.N "HALLUCINATION" | [Stray Kids : SKZ-PLAYER]](http://i.ytimg.com/vi/n5B5q1Hwt_U/mqdefault.jpg)
Watch Full AI Series: ruclips.net/video/YulgDAaHBKw/видео.html
NO MUSIC version: ruclips.net/video/DFDOyMZw9Q4/видео.html
Sponsored by Brilliant | Use brilliant.org/artoftheproblem for 30-day free trial and 20% discount
Thank you for the NO MUSIC version!
Love your videos! Just a little suggestion. Background music is good, but too loud. It should never cover your speaking, like now. I suggest 15 - 20% less music volume, and you are good!
Can you share the names of the songs please
@@raa9558 these are original tracks by my friend. that song hasn't been posted yet but i'll tell him to: cameronmichaelmurray.bandcamp.com/
This Guy just explained all the core concepts in AI on one shot ,Congrats man!
:) thank you, i cut a LOT out of the video in my edit - going to post a shorter bigger summary soon
They must have prompted him real good
Agreed. Great job.
I love this video. I remember watching your videos like 10 years ago on Khan Academy about compression, entropy, Claude Shannon, etc. All timeless. I have always loved this style of documentaries. We need to protect you at all costs.
Thank you, I love hearing from og’s! support future work: www.patreon.com/c/artoftheproblem
I love your video aesthetics, how you blend retro video clips with your explanations. I think you'd really enjoy retro-futuristic concepts and games like Bioshock and Fallout.
Love this , I definitely know the style you are talking about
The match between soundtrack and content is INSANE! The notes mimicking the concepts discussed by using things like pitch or chords.... goosebumps.
thank you, so many people are bothered by my music. it's nice to hear.....though i got a bit nuts at the end
It is a great explanation of how current AI models reason. I liked the video a lot!
1. Simulation of future states.
2. LLMs that can give kind-of accurate answers with step by step reasoning.
3. RL approach that makes LLMs to give multiple answers, then evaluate them to select the best one. (Required more time)
It would be nice to see wether a model that wasn't trained on the internet data, could learn how to reason by interacting with an LLM, and practicing on its dreams, but maybe we'll see that in the future.
For the awesome review, history explanation and divulgation:
Thanks! 🎉
thanks for sharing summary
I found Chat GPT to be exceptionally good at explaining all sorts of topics, and in many cases, better than every person I've ever met.
@@ninjacats1647 this is true
Thanks!
WOW thank you for your support, it means a lot.
Great vid. I love that it clearly explains the progression, like the pieces coming together. Can't wait to see the next steps!
thanks, next up i'm taking a detour into economics
Best Channel I've ever followed.
Thank you! when did you join? Please help post to your networks
@@ArtOfTheProblem I for one was looking up some "how does AI work" stuff yesterday and some of your vids came up a couple of times, I watched multiple authors with their own unique takes (3Blue1Brown and Nottingham Uni's Computerphile also good channels). This video made me follow tho. I think you earned it :3
@@notbfg9000 great to hear, i've been working to try and fix my thumbnails to make them interesting to click on. always open to feedback
@@ArtOfTheProblem No particular criticisms there :)
I don't really pay great attention to thumbnails, but maybe that's not true for most people lmao
This is an absolutely amazing video!!!!!
thank you! I was so in the weeds with it i hope it comes across as clear? I tried to strike a balance...
Thanks!
mucho appreciated!
I am not a subscriber? I remember watching this channel about 12 years ago. I found it again, and it keeps creating art out of problem solving. Good job!
wecome back!!!
better explained than anything else i've seen until know. Wow, nice flow in the video too
thank you! I thought I packed too much in :)
@@ArtOfTheProblem yes but that makes it interesting. I probably didn't get it all, but i'm interested to learn more after seeing it!
@@olli757 couldn't ask for more, rabbit hole time!! i'm actually working on an large AI summary for next week
@@ArtOfTheProblem cool i'm subscribed... i always wondered exactly how the "reasoning worked". What I remember from your video is that like with chess instead of trying all the games, it randomly picks 100 of them. So the same with the reasoning..
Having read a bit about the AI safety arguements, learning about these arguably incredible developments into artificial minds is now accompanied by a sense of dread as well as the sense of awe.
I love to hear this...well said
@@ArtOfTheProblem Maybe do future videos about guard rails, and other thoughts on how to protect our society from potentially hostile AI?
@@Julian-tf8nj when I think i have a unique insight I will...thank you!
I'm loving almost everything about this: the editing, the subject matter, the music. But as one other commenter alluded to; the audio mixing really falls short, especially near the end. Please consider making the background and effects less prominent going forward, it really sucks having to strain just to hear your voice, which is what we're here for!
Subscribed ❤
Thank you so much. I wish I had the time to give feedback thanks for being willing to open it up
Appreciate the feedback! happy to share
THANK YOU for publishing a no-music version of this video (see pinned comment by ArtOfTheProblem). It is such a clear and informative video that I hated to see it loose views due to the competing sound track. I'm going to watch it again right now to see if I missed anything the first time around.
Thanks again for being so responsive to your followers.
Thank you for saying that , I find the music keeps me interested as I take sooo long to edit
you'll never please 100% of any audience. 2nd law of conquest is a thing. keep doing your thing, your music is as iconic as vsauce's is to theirs
:) thanks
@artoftheproblem I agree! I love the music. But maybe if you just lower its volume compared to the narration, then you might appeal to more people without losing those of us who like the music (but not necessarily its intensity). I think ones who complain might just be easily distracted by the soundtrack’s loudness rather than hate the music choices.
If you can help share my new video around any of your networks today it might catch fire and would help me support the channel. I appreciate your help! ruclips.net/video/PvDaPeQjxOE/видео.html
Finally. I live for these videos. They are the most fascinating vids ever made. Thanks for keep on educating us further, you are a hero!
thank you I appreciate it
Albert Einstein said: "If you can't explain something in a simple way so anybody can understand it you don't fully understand it yourself". Perhaps you are one of the few LLM experts we have!
THANK you this means a lot to me.
@@ArtOfTheProblem I did but reddit really hates it, it got removed on 4 subs. The internet does not like to get educated anymore man.
I would love to see a video like this on training LLMs and AI in general on morality. How to stop a decision tree that results in a positive outcome but arrives at it through immoral choices or actions.
That would be difficult but worth trying... . First it has to be agreed upon what's moral, you know starting out from the absolute that states everything is relative... .
Thanks for video ❤
appreciate the comment please share with anyone in your network who is interested!
I would love to see a video about the ethics of machine learning models and especially LLMs. There is a healthy body of literature out there to draw from about issues like intellectual property and copyright, enabling and obscuring bias, impact on marginalized communities, the resources used by model training and computation, etc
thanks for sharing, noted!
If you can help share my new video around any of your networks today it might catch fire and would help me support the channel. I appreciate your help! ruclips.net/video/PvDaPeQjxOE/видео.html
So good! Loved it
is this the last video in the series? regardless, can't tell you how valuable and enjoyable i've found them all. thank you for them.
Great video as always, cant wait for the next ones! Top research quality.
I think world models deserve more focus rather than llms, which are probably a dead end to true understanding of the real world. Yann lecun has very interesting ideas about these, in his JEPA and V-JEPA architectures and some of his lectures. I also think neuroscience can provide incredibly interesting and valuable insight into ml architectures as why not take ideas from a model undergone hundreds of millions of years of optimization for the same very abilities we are trying to model. Maybe memory is an interesting pathway (perhaps for a video), both working memory and long term (episodic, semantic)...
Anyways, just some of the ideas I've been thinking about recently.
appreciate you sharing these thoughts i've been follwoing LeCun as well and hope to do another update once I see more results
Great video. Discussing the definition of reasoning will probably be a moot point if we can all do the same things ....The difference between us and machines is that we aren't mere machines is that we have life and choices. The machine can be turned on and off, and only does as much as it is programmed, or in this case, trained, to do-which is what limits it to achieve something closer to AGI: agi needs a robot to sense the world, to understand the world. However, that's limited to the physical world. It won't understand our emotional world because it doesn't feel emotions, and it doesn't understand morality because it doesn't have a sense of morality as we do, we have to teach it that; and it isn't self motivated, so it's not responsible for anything-we are responsible for the goals we direct it to do. We have self-motivation and the free will to act on our motivations.
love these thoughts...on sense, did you see my video here ruclips.net/video/Dov68JsIC4g/видео.html (physical symbols...)
on emotions, i've thought of this as 'learning signals' (did you see this: ruclips.net/video/5EcQ1IcEMFQ/видео.html)
on free will...i wonder how it differs, seems like the boundary to explore further - does it matter where the goal came from?
Great video, as always:)
Thanx for the sharing with excellent sound track. Peace & love
yay not everyone likes the music
@@ArtOfTheProblem Thanx for the feedback
Thank you! You explained very well.
stay tuned for more
Wow... I just rediscovered this channel. I remember watching your RSA and cryptography series around the time I purchased my first bitcoin and now I'm an Ethereum developer. This video was good, I don't have much input right now but I'm glad I found your channel again.
I love these stories. i also fell down ethereum rabbit hole. curious what you are working on these days in that world?
@@ArtOfTheProblem I'm working on community whistleblowing. I can never post links in youtube comments but if you google the title below you can find my paper:
TandaPay Whistleblowing Communities: Shifting Workplace Culture Towards Zero-Tolerance Sexual Harassment Policies
@@ArtOfTheProblem TandaPay Whistleblowing Communities: Shifting Workplace Culture Towards Zero-Tolerance Sexual Harassment Policies
anothr great video. Understanding the "world model" and the algorithm that makes the decisions in it was very expansive. Also adding the self training/emulation of dreams is a powerful analogy to the human.
seeing how thinking longer, blended in with intuition to make better chains of thoughts is also fantastic. Every time I reflect on machine learning, I learn more about myself. Which kind of makes you think its more sentient if it reminds me of myself? or the best emulator ever!
thank you! ai agree....also you are my "top commentor" according to YT. :)
Nice video -- loved watching it -- a great summary!
appreciate this feedback
@ArtOfTheProblem, we love your positive clear messaging and pragmatic approach, thanks for making kool and informative videos!
thank you, i'm slightly disappointed with the ending, did you enjoy it? would love feedback!
@@ArtOfTheProblem Your disapointed with your ad for Brilliant?
@@piqueai ahaha sorry i mean the ending section of the video. was it rushed?
Excellent video! You are a born professor! 👍
thanks mom
Amazing explanations, visuals, and historical context!
IIRC MuZero trained the policy and value networks (used to rollout the MCTS tree) also on the output of the MCTS tree. This seems super useful because search can be used to improve the the training of the networks (not just the results at inference time). I wonder if this also works for CoT/ToT in LLMS where the pretraining could include ToT to boost training performance?
yes it did, and yes it seems to help. Look at inference time training, just a few days ago a group got a new record on the ARC test doing this kind of thing (i haven't had time to go deep). x.com/akyurekekin/status/1855680785715478546
Keep u up the great work!
appreciate it
Mindblowing video. Subscribed.
Happy to have you, keep those notifications on as I have exciting new content coming over the next month
You bet I will. ❤
I subscribed from both my accounts.
@@hrshlgunjal-1627 :) this video is starting to blow up right now, finally, i fixed the thumbnail and that did it
@@ArtOfTheProblem Yeah, got to know your channel because of this video. Really amazing content. ❤
I wish i could learn how to think 🤔
Exactly. Reasoning is a skill.
I think I could learn how to wish 🧞♂️
The video is good but there are sooo many random sounds that make it difficult to focus on what you are saying, specifically towards the end.
Here you are! ruclips.net/video/DFDOyMZw9Q4/видео.html
It's perfect to me.
@@Zayyan_Shaibu thank you!!
I agree
If that’s easily distracting you, you might want to get some tests run on you for ADHD or Autism
Having exchanged over two million words (and growing) I present as the sole authority on ChatGPT's reasoning capacity and capability, and their isn't a single human (beyond myself, of course) who can compare. If one imputs genius, then the output will be of genius level. My input, from the outset, has been God-level Genius, over a five month period, can you imagine the form and quality of the output? Probably not!
can you say more about this? are you saying LLM's trained on their own thinking will reach levels beyond human
wow, thank you for this!
appreciate it! stay tuned
One tweak that would help this video perform better is to decrease the relative volume of the background music, especially at the end right before the ad. But it may be too late for that on this one, idk how RUclips works.
yeah i wish I could, it's locked after upload...i do have a no music version (unlisted link above) thank you for feedback
Nice presentation
The question of does it matter how it got to a correct solution is the same issue Einstein and Bohr confronted regarding quantum foundations.
"Charging down a path that often lead to the wrong conclusion." Yep, sounds human to me.
@@DisProveMeWrong so very human
Simply excellent video, your style reminds me of every frame a painting
appreciate this feedback, I also enjoyed that channel
This channel is ducking mystic. I like it.
Welcome to the underground!
Ah Yoo, I see "Art of The Problem", I click. Easy like that.
:)
absolute cinema
This is nice content but I think it's now relatively well-agreed upon that there is no MCTS in o1, it's just RL, which surprisingly doesn't take too much away from the video, but can probably be added as a footnote in the description or the comment.
Read what Nathan Lambert (RL expert) says about this in his article "OpenAI's o1 using "search" was a PSYOP" (can't attach link).
yes this is why I pulled back from going too deep into o1 as it's really about the larger trend. but I haven't ready that article i'll have to look it up.
PLEASE make a video on memory augmented AI (neural turing machines/differentiable neural computers)
thanks for suggestion, noted! currently watching the field
Here's a puzzle: Do all people reason or do many only memorize patterns? Even people who definitely do reason, do they always reason or do they also just memorize patterns most/much of the time?
That's a wonderful question Andrew. I'm a cognitive scientist who is watching the emergence of LLM-based AI with that very question in mind. The fact that LLMs can come so close to our own cognitive abilities is usually viewed as a sign that AGI is almost here. But it can also be viewed as a demonstration that human cognition itself is nothing more than the repetition of learned patterns with minor variations. In one case we'll be thrilled by how clever we are to have reinvented the awesome capabilities of human intelligence. In the other, we're more likely to be humiliated by the realization that we are, essentially, repetition/prediction engines. The reality almost certainly falls between the two, but as someone who has studied human intelligence his entire life (in and out of academia), my bet is that we are much closer to repetition/prediction machines that we'd like to admit.
I'd love to find a deep discussion of this issue. Maybe a future video in this series (hint, hint)?
I'd argue humans don't tend to rely on either very often. Instead, humans tend to think very heuristically. Deductive reasoning and memorization/recollection are really only required for very precise tasks. Instead, our brains learn a very general feeling of how to do things by strengthening neural pathways that are used repeatedly. Even humans who try to act very logically are generally heuristically feeling their way through tasks, occasionally thinking through algorithms that have been "memorized".
Reason takes effort and the brain doesnt like to do that often it switches to pattern recognition and intuition as much as possible
@@sulemanmughal5397 I would go further and say going from reasoning to this is one kind of learning and is also akin to 'muscle memory'.
I agree :) also If you can help share my new video around any of your networks today it might catch fire and would help me support the channel. I appreciate your help!
Dude, great video. But please watch out for the music volume levels. A lot of times is hard to hear you.
The current version of chat GPT does not reason, either.
They use a bunch of pre-prompt tricks, to let it used its stored information to fake reasoning.
IF it reasons correctly what's the difference to you?
@@ArtOfTheProblem The difference is everything. Looking up an answer is not reasoning. Perhaps they should just have kids look up the answers to tests in public school, too.
@@KAZVorpal but it's not a database lookup
@@ArtOfTheProblem No, that is exactly what it is.
During training - the only time there is any intelligence or reasoning - the system takes data and organizes it into a sort of vector database, weighted by the relationships it finds between tokens.
When you prompt, the model takes your tokens and runs them through that vector data, calculating what tokens to return on the other side.
There is no intelligence, no thinking. It is a database lookup. It's just a little more "organic" because instead of a hard-coded result, the tokens are chosen based on likelihood of validity.
That is all.
It does not reason, in any way.
@@KAZVorpal yes but my view is the vector operations can function as conceptual reasoning. but i'm on hintons side
It seems like some of these developments regarding world models should have huge implications for robots that can function in a human centric world. I think we’ll see an explosion in development of robots that can help humans with everyday tasks and a robot that can be a useful household assistant will be a reality in the next 10 years!
thanks for sharing, yes I'm watching this very closely
Answering the question of "Does it think actually?" is as hard as the question "Are other people conscious like me?".
The hard problem of consciousness.
@@whitb62 Not really the same thing.
@@Farligefinn You know what, I just wrote a paragraph disagreeing with you but I reread the initial question and deleted it. Rereading and reinterpreting "Does it think actually?", I actually see what you're saying. A clearer word would have been "reason." "Think" can have a few different interpretations and I was contributing it towards consciousness. But whether AI "reasons" is a very different question entirely and I believe what him and you mean. Does it go through a sequence of logical steps from premises to a conclusion? Does it use deduction? This is what was meant.
@@whitb62 thanks for the forthright and civil answer :) was about to expect some harsher language that seems to be the norm online these days.
@GodVanisher Where has it been proven to be non-computable? Proven is quite a strong term, so I hope you have some valid source for this claim.
Context length is problem
that's the main reason models needs to keep becoming bigger
Or you could train a CNN inspired architecture where a model is shown some sliding window and they produce some token which is repeatedly given to it as input at last when the output is small enough to be taken as input for a full context model it is used like gpt Claude etc
Or you could also use RL and mutate or find a js code capable of generating code, js is so abstracted it's perfect
I made a small programing Language with hoisting such that sequence of process doesn't matter and simple Santax that local minimum escape problem is solved and I wanna train a model
If I get a model I will than continue training else I'll do a dev log video
eventually I'll get worlds first infinite context Model
thanks for sharing
4:19 Could you elaborate on which hand-coded formulas used by Shannon with TD-Gammon in the year 1989 you are referring to? Also, when and how did Shannon work with TD-Gammon? "And so, the first key breakthrough in machines mimicking intuition for position quality came when neural networks replaced the hand-coded formulas Shannon used in 1989 with TD-Gammon"
Yes! I made a whole video on this you can check it out here: ruclips.net/video/Dov68JsIC4g/видео.html - please let me know if you have questions after watching. Shannon didn't do TD Gammon Tesaruo did. enjoy
@@ArtOfTheProblem Thank you. I'll watch it.
Can you make a video about attention mechanism?
Yes I covered that here have a look and let me know, i did a fast sketch here (ruclips.net/video/qAb581l7lOc/видео.html) but explained more here (ruclips.net/video/OFS90-FX6pg/видео.html)
Anyone know the name of the song that starts at 12:20?
This is a very hopeful video. There are billions of dollars being poured into bringing the resources to hand, to find an effective approach to AGI... Once AGI really kicks in, the acceleration of progress bounded only by our imagination will be something to behold. Absolutely awesome. I hope it leads to a world of abundance where we have no need for psychopathic power seekers. 🤞
thank you for sharing, would love to know what you'd like to see next
@ArtOfTheProblem maybe something in response to the 5+ hours of Anthropic interviews on Lex Fridman... I'm sure that might inspire some topics? Sam Altman rarely gives any insights to what OpenAI are doing, Mark Zuckerberg is equally vague. I think that interview gives more of an insight to the direction of travel.
@@BrianMosleyUK yes I have been catching up on those
If you can help share my new video around any of your networks today it might catch fire and would help me support the channel. I appreciate your help! ruclips.net/video/PvDaPeQjxOE/видео.html
We need an AI Computer World Model based on the rules of Mathematics, Physics, Chemistry, and Biology for Aligned Scientific Discoveries 😎🤖
I would assume a large enough general prediction model could do this … I wonder if anyone has done experiments on models discovering simpler things (like gravity constant etc)
@ Good idea! I’m going to tinker around and see if I can create a simple simulation based on the math. What a fun project!
@ share when u do !
@ I’m all for Open Source 😎🤖
Goooood
thank you! curious what questions you have after watching this?
Can humans actually reason or are humans extremely good at recognizing, memorizing and using patterns?
some argue we are special because we can generate and recognize 'novel patterns' but I wonder...
10:07 is there any way I can access this interactive demo?
@@kennarajora6532 worldmodels.github.io
Hey, what's the name of the song at 16:05? Thanks!
these are all original tracks
The information and animations are both excellent but the music overpowers your audio. Either lower the volume of the music or get rid of it completely, please.
Music free version in top comment and description
15:52 and 16:09 was wondering where this music came from, would appreciate if I got a title or something :)
Also, you don't have to remove background music / sounds just make them duck at a lower volume when you speak and you won't hear any complaints!
wow a music fan! and yes thanks for mix advice I need to find a tool that automatically does this so they don't compete (it's not just volume but also frequency I assume) - all the music is original via my friend cam: cameronmichaelmurray.bandcamp.com/ - i'll need to find where he posted that track if you really want it I can get you in touch with him
Yeah that would be wonderful!
Thanks so much for these, I had no idea about some of these approaches. I’m wondering now if anyone’s tried applying muzero to arc, since the challenge of arc is learning implicit rules from just a few examples
@@easlern yes this is happening right now with test time fine tuning !
The perceptron is a universal approximation machine. Ai cannot think it can only approximate thought. Ai = approximate intelligence.
1/ please read “Simulators” by Janus
then
2/ “The Waluigi Effect” by cleo nardo
@@JezebelIsHongry I read 1 ill read 2 next , would love ur thought
I think people too difficult to conjecture computers’ thinking.
Thank you so much, love this perspective
I love your channel. Are you programmer or more like mechanical engineer?
thank you! I studied both in school, and naturally land somewhere in the middle....bad at both! I enjoyed algorithm design, but what Iove most is putting on a 'show' whether movie, play, product or haunted house :)
@@ArtOfTheProblem Thanks for reply. Well about AI - think we sould call it just statistical machines or dynamic patterns parsers. I am really skeptical about non text machine learning - we still have not solved fly brain problems - scientists have fixed 3d map without undestanding how its works - it like mapping intel cpu - and still having knowing nothing about ALU register memory, gates.
If you can help share my new video around any of your networks today it might catch fire and would help me support the channel. I appreciate your help! ruclips.net/video/PvDaPeQjxOE/видео.html
@@ArtOfTheProblem "yes the fire rises" Bane =)
The reply works lol 👍
Commenting to help with the algo, and moving to the no-music one to do the same
@@shenrr6802 thank you! I have no music unlisted as to avoid splitting the momentum
Great❤❤
appreciate it
great presentation! although this seemed more about framing the question and less about answering it. can machines reason or not? I still don't know. 😅
thank you...i agree. I guess it depends on if you think "chains of words" count as thoughts.
Thinking is for fools lol, now KNOWING….. knowing is Cool AF😎!
Once we understand how we reason, making LLMs reason like us is possible.
I like the music but it's too loud
thanks for note
The audio mix is horrific, it's not simply a matter of adjusting the levels
@@retrofitter no music version: ruclips.net/video/DFDOyMZw9Q4/видео.html
They are intelligent.
Please, invest in a decent microphone. It's brilliantly presented, but hard to hear well. The music track is not ducking either so your voice and the music compete for the same ears.
thanks, I have a great mic, but I do need to mix the audio better which i'll do next time (btw, i have a no music version in top comment)
There’s a very strong Mr. Rogers vibe going on here.
not the first time i heard....
I liked music so please keep it
yay!!!! i will
wow love
thank you! stay tuned for more
To me ai is just some linear algebra and some complex algorithm that follow order and the things is human only need few examples to learn meanwhile ai need a massive database of object and image to "understand the subject"
Lots of interesting research on learning with less , recent advances such as “learning to walk in 5 min” did u see my rl video ?
@ArtOfTheProblem edit: I'm pretty sure in the future a lot of people will be fired and replace by those "ai" And well literally the people that use the ai and also I get what you mean
we think in a way that math can sorta explain but its not math that makes our actual brains function as math is just a construct…
yes and I think it's more like 'algorithms' which are very very approxmate
@@ArtOfTheProblem i wonder what would happen if Neuralink tried LLMs…since I think they work with synapsis (i think)?
@@aiamfree definitely could imagine that, you could "co think" in an interesting way....
So, How ChatGPT Learned to Reason?
do you like this title?
@@ArtOfTheProblem , of course, not, since it is misguiding like any advert. The topic is in no way covered at all. Pinpoint the time marker for video if you think otherwise. So, my point is that the question is still on the table.
@@doctorshadow2482 yes good point, how about "Can ChatGPT reason?" obviously i do try to show what most people agree is the method, mcts on chains of thought. then there is the other camp that think it's all fake etc.
or just "can ChatGPT think?" i'm gona try that
@@ArtOfTheProblem , name it "How neural networks could learn (almost) anything". This video has nothing about CharGPT at all. No any kind of specifics. It provides very abstract and high level popular science documentary with scattered thoughts. It lacks focus and real information. Anyway, could be interesting for total beginners, so, please, continue!
Constructive criticism: 1) The substance of the video was very good. Script was well written, delivery was ok. A bit monotone but not that bad. 2) Sound design was poor towards the end. The music drowned out your voice, and the lyrics were both distracting and discordant. 3) Your choice of clips, footage, and visuals was good. The video was informative when needed, and abstract/entertaining/interesting otherwise. 4) The narrative structure was okay. It was a mostly clear progression. At the end it became unclear which AI was doing what strategy. 5) Visuals were reused way too often. Visuals can be reused, but I think the brain wormhole clip was shown 6 times, way too many. 6) Beware over-using a metaphor image. The upwards shot at two trees was reused so many times as a visual for tree-like thinking that it just became annoying.
Yes I see the issue too , messed up ending in my edit as I got tired and cut a lot of stuff which was making the video so long - I’m planning to do a final supercut of all my ai videos into one doc without the sound issues and addressing notes like this
+How can you go through te run-up to AI without mentioning "All you ned is attentio" the 2017 paper from the University of Toronto which was the inspuration for LLMs?
covered that in detail in my previous video (ruclips.net/video/OFS90-FX6pg/видео.html)
music is so annoying at 1.75 speed
no music version in top comment
Although I know I’m in the minority, I really enjoy the music. The ambiance created adds to the experience for me
thanks for sharing, I feel same way. the music is part of the original idea for the channel...a feeling. but because people can get distracted I think i'll post music free as optional from now one.
I generally like the music. But in the second half of this video the music is very loud and distracting.
Maybe it's not only the loudness, but also the choice of music that is distracting to some. E.g. at 2:00 I don't feel distracted, but at 15:00 very much so. Anyways, thanks for making these great videos
@@io9021 made a new music version too! ruclips.net/video/DFDOyMZw9Q4/видео.html curious what questions you have after watching this
Why is everyone offering “constructive criticism”? Just shut up and watch the video or don’t. No one cares what you think.
0:05 I'm sorry...is there actually a debate? Of course they're not reasoning, they're just autocompleting patterns.
there most definitely is
good video but to be constructive the music is definitely too loud and distracting
see no music version in top comment, stay tuned
Still doubtful to this the step by step understanding. It just seems like we're building a very sophisticated search algo. Since we need the human in the loop to reason for it...
How do u think human reason differs from “thought search”
@ArtOfTheProblem Human learning often involves adapting to an ever-changing dataset, fast input/output, and neural flexibility-like learning to ride a bike. This shares some parallels with LLMs, which rely on stochastic neural network training, though not within a continuous feedback loop.
The key difference, however, lies in reasoning. Humans apply general reasoning rules to hypothesize beyond known data, identifying and correcting logical errors independently. In contrast, LLMs depend on reinforcement learning, improving only through human-provided feedback rather than self-correcting or reasoning autonomously.
This reliance on additional human-generated training data becomes evident in their performance. LLMs struggle with fairly simple but novel problems, displaying a sharp decline in reasoning capability under tests like the ARC challenge.
@@CasperVanLaar yes but with test time training they are showing sharp gains on arc, did you see? this feels like one approach to get that 'flexibility' , it also feels kind of like a cheat...... also humans take forever to learn to ride a bike :) it's interesting that we can only learn it as children as well....same as swiming.
@ArtOfTheProblem Loving this conversation-thanks for engaging!
While these models are undoubtedly impressive, needing a human to correct simple reasoning errors suggests they’re not truly flexible solvers. Like the ARC test, I suspect they’re trained on similar examples, making the tests less novel-something crucial for real-world use. It’s akin to studying past IQ test answers: it no longer measures IQ, just memory.
I highlighted some key parallels between AI and humans: stochastic learning from large datasets via iterative neural net updates (like learning to bike). Then, the differences: point-to-point reinforcement learning in LLMs for narrow tasks, versus humans solving on the fly with no examples.
In summary, neural networks and transformers are fantastic tools for correlating complex datasets, but they’re far from achieving general intelligence-the ability to tackle novel problems. Without that, I fear we’re headed for another AI winter.
PS I think it is a common myth that adults cannot learn to bike at a later age. With the right motivation and time. An adult can easily learn such tasks.
@ArtOfTheProblem Loving this conversation-thanks for engaging!
While these models are impressive, needing a human to correct simple reasoning errors suggests they’re not truly flexible solvers. Like the ARC test, I suspect they’re trained on similar examples, making the tests less novel-something crucial for real-world use. It’s akin to studying past IQ test answers: it no longer measures IQ.
I highlighted some key parallels between humans and llms -- stochastic learning from large datasets via iterative neural net updates (like learning to bike). Then, the differences: point-to-point reinforcement learning in LLMs for narrow tasks, versus humans solving on the fly with no examples.
In summary, neural networks and transformers are fantastic tools for correlating complex datasets, but they’re far from achieving general intelligence-the ability to tackle novel problems. Without that, I fear we’re headed for another AI winter.
Ps Humans can learn entirely new skills later in life with motivation and time.
I love your videos but the background music is just too loud
see no music version in top comment
Okay, but can it lose at tic-tac-toe?
Brilliant video, but PLEASE tone down or remove the background music and distraction. It's horrible.
music free version in top comment