Chapters (Powered by ChapterMe) - 0:00 Coming Up 0:54 What models get unlocked with the biggest venture round ever? 5:35 Some discoveries take a long time to actually be felt by regular people 9:53 Distillation may be how most of us benefit 14:26 o1 making previously impossible things possible 21:17 The new Googles 23:47 o1 makes the GPU needs even bigger 25:44 Voice apps are fast growing 27:05 Incumbents aren’t taking these innovations seriously 31:52 Ten trillion parameters 33:15 Outro
One order of magnitude higher is referring to compute, not parameter size. The current frontier models (4o/pro/sonnet) are all much smaller than their previous sota (4/ultra/opus). GPT4-1.8T MoE might continue to be the largest model people train for a while.
I'm a VoIP (Voice Over Internet Protocol) developer at a small company, I want to start a startup in the comming 5 months and now I'm looking for co-founders, my previous startup (in generative AI space) got accepted into Microsoft for founders hub, if you are a technical, innovative or market person reply me here to discuss more. 😊
Winning AI company capturing all the value is speculative, therefore an answer can also be speculative that the more any AI becomes powerful, the more complex society becomes, hence the value of being human to cope with new contexts to feed the "hungry AI" proportionately increases and hopefully it's fun.
Fact Check: Terence Tao's parents are first-generation immigrants from Hong Kong to Australia. Which makes him Australian-American after he naturalised for duo citizenship.
Even if AI doesn't improve from here companies like Microsoft, palantir etc. are building out the AI eco systems that you will start seeing big companies using for all their backend systems. The average Joe will see the changes pretty fast I would think.
Perhaps as the world becomes more corporatized and people get fed up with Tech overreach there will be great opportunities for entirely new OS that don't support AI?
I love the runtime paradigm of o1 but I sure do struggle to find a use case where its better than a solid prompt template and 4o, quite the opposite infact o1 gives me worse results most of the time
never, unless you think we can fit a couple tbs of ram on a USB stick, we'd have to figure out an entirely different method of computing that doesn't rely on silicon, or even semi-conductors...
@@yesyes-om1po we already have 1tb micro sd cards which wasn’t possible a few years back trust me eventually we’ll figure it out. Tech is still developing ,we haven’t even scratched the surface
How do you solve quantum coherence multibody physics problems. I think they said a practically useful quantum computer has 200,000 to 1,000,000 qubits. They will probably need AI's help to solve such hard problems.
I really get bugged by the name OpenAI now when we know its so closed that they dont even disclose any model related information in their technical reports.
I feel bad for you. It seems like such a pointless thing to obsess over. It takes money to do this stuff, and unless you are ready to fund their requirements, you should probably just get over it and focus on something that actually matters. One of their goals was to provide AI to everyone, and considering anyone with an Internet connection and smartphone or computer can use it for free, they are following through. But they said!!! Blah blah blah.... Move on and be thankful they put this technology out into the world so that it's not being hoarded by the elite.
@@tracy419 Seriously. It's very frustrating when these free loaders want to use the state of the art AI but refuse to acknowledge the insane cost required to get it. Another reason I never understood the ridiculous criticisms of Sam Altman.
AI agent can listen to you describing the symptoms, run through every combination of symptom and all diseases know to humanity in seconds and suggest the doctor which direction he should go next or propose most accurate diagnosis.
more parameters means more better-er, people often confuse parameters with neurons, but its actually more like synapses, and humans have 100 trillion synapses, GPT-3.5 is only 305b parameters or so, and GPT-4 is just an MoE model which runs 11 GPT-3.5s in parallel with one master model choosing which model to use based on the prompt, GPT-4o is likely smaller since its cheaper.
Yeah this whole thing was odd. We didn’t have o1 then maybe, but we do have it now and it’s not even as good as claude. Certainly not the leap they seem to claim it is.
problem is that while compute poring over data can progressively build more and better the 'routine' apps and systems we know and use today you will be stuck with the 60pt system with no inhouse ability to reach 61points and you have to wait for AI to be able to generate code that can get to 61 and nobody of even average talent would care to, understand or work on that pile of code that was conveniently generated in minutes, even bother to push it to 61 - and that is a latent threat to the system development ecosystem
sounds like a load of baloney, the only reason nobody would understand the code is because AI generally writes hard to read code, even if they put copious amounts of "documentation" with it. Sounds like the solution to this is just to write code with minimal AI intervention, unless you plan to completely replace the programmer.
Can you have a ten trillion parameter model? Can you extract that many parameters from the dataset/language? I honestly don't know, but at some point you end up with a larger portion of parameters that have very few (if any?) instances, no? (ie most columns are all NA).
Yes, but how you interface with that ai will be important. So its important to build a platform or network or processing company now. People will need platforms & how they interface will be different in the future. Device startups realize this, as Steve Jobs said, the best UI is no UI .
@ikleveland Temporarily. Soon the AI will build the required interface on the fly, and will manage all networking tasks. The era of software is coming to an end.
Btw, a little thing. I think you guys may (or not) want to hide your feet. They sometimes dangling take a little of the attention of the content... And thanks for the content.
Open AI 4.0+ cannot work effectively at the same time successfully in multiple windows. It cannot retain a topic if you come back later, it's like talking to a new person. I'm ADHD this feels like my failure when any request cannot be met. It frequently cuts me off even though I pay for plus to come back in two hours that's some bs. I'm creating gold, shot down by you used too much come back in two hours 🤣lame
@ says random bot on the internet. Do your research, everything points to scaling as a logarithmic curve, particularly for CNN and Large language model networks.
what's the use of your AI, when you accept Arab funding, add policies that hide the any islamic things from GPT, AI. Literally, GPT dont talk directly but give excuses like Mullah of ISLAM to legtitimise the atrocities by founder
Chapters (Powered by ChapterMe) -
0:00 Coming Up
0:54 What models get unlocked with the biggest venture round ever?
5:35 Some discoveries take a long time to actually be felt by regular people
9:53 Distillation may be how most of us benefit
14:26 o1 making previously impossible things possible
21:17 The new Googles
23:47 o1 makes the GPU needs even bigger
25:44 Voice apps are fast growing
27:05 Incumbents aren’t taking these innovations seriously
31:52 Ten trillion parameters
33:15 Outro
Lol did sam altman tell yall to avoid talking about claude and comouter use update
They talked about developers in their batch switching from OpenAI's models to Claude's models at around the 12:50 mark.
who's Sam Altman?
There is literally nothing in this podcast that make it worth spending the time to listen to it
Was looking for this comment. It's like a kindergarten class.
Need a 3x button on yt
😭😭
My thoughts exactly
Accurate
One order of magnitude higher is referring to compute, not parameter size. The current frontier models (4o/pro/sonnet) are all much smaller than their previous sota (4/ultra/opus).
GPT4-1.8T MoE might continue to be the largest model people train for a while.
but can it code crysis?
I'm a VoIP (Voice Over Internet Protocol) developer at a small company, I want to start a startup in the comming 5 months and now I'm looking for co-founders, my previous startup (in generative AI space) got accepted into Microsoft for founders hub, if you are a technical, innovative or market person reply me here to discuss more. 😊
Winning AI company capturing all the value is speculative, therefore an answer can also be speculative that the more any AI becomes powerful, the more complex society becomes, hence the value of being human to cope with new contexts to feed the "hungry AI" proportionately increases and hopefully it's fun.
Fact Check: Terence Tao's parents are first-generation immigrants from Hong Kong to Australia. Which makes him Australian-American after he naturalised for duo citizenship.
Oops. Sorry about this. -Garry
Thanks for the inspiring sharing! Let's leverage the power of AI to improve human lives and building a more sustainable future!
Even if AI doesn't improve from here companies like Microsoft, palantir etc. are building out the AI eco systems that you will start seeing big companies using for all their backend systems. The average Joe will see the changes pretty fast I would think.
Is o1 a significant upgrade over o1 preview
Perhaps as the world becomes more corporatized and people get fed up with Tech overreach there will be great opportunities for entirely new OS that don't support AI?
the CFO telling us how their tech works is hillarious
Hardly, but whatever.
It wouldn’t be that slow if you use the new probablistic computing which is 100 million times more efficient than the state of the art
I'm hoping we hit a local maximum and hang out there a while.
0:35 I heard that as "then Windows will just be whoever builds the best...".
_Well..._
I love the runtime paradigm of o1 but I sure do struggle to find a use case where its better than a solid prompt template and 4o, quite the opposite infact o1 gives me worse results most of the time
I do like it better for coding tougher problems and debugging problems.
This has me thinking when we’ll be able to run these large models on a usb drive size computer ….
Good thinking but not likely until “AI” helps human invent the sub atomic circuits to replace today’s semiconductor technology.
never, unless you think we can fit a couple tbs of ram on a USB stick, we'd have to figure out an entirely different method of computing that doesn't rely on silicon, or even semi-conductors...
@@yesyes-om1po we already have 1tb micro sd cards which wasn’t possible a few years back trust me eventually we’ll figure it out. Tech is still developing ,we haven’t even scratched the surface
How do you solve quantum coherence multibody physics problems. I think they said a practically useful quantum computer has 200,000 to 1,000,000 qubits. They will probably need AI's help to solve such hard problems.
Ant Financial's fundraise was larger?
A great Notification
Omg why do they taaaalk like thIIIIs?
I really get bugged by the name OpenAI now when we know its so closed that they dont even disclose any model related information in their technical reports.
I feel bad for you. It seems like such a pointless thing to obsess over.
It takes money to do this stuff, and unless you are ready to fund their requirements, you should probably just get over it and focus on something that actually matters.
One of their goals was to provide AI to everyone, and considering anyone with an Internet connection and smartphone or computer can use it for free, they are following through.
But they said!!! Blah blah blah....
Move on and be thankful they put this technology out into the world so that it's not being hoarded by the elite.
@@tracy419 Seriously. It's very frustrating when these free loaders want to use the state of the art AI but refuse to acknowledge the insane cost required to get it. Another reason I never understood the ridiculous criticisms of Sam Altman.
What does it mean for our healthcare system? Perhaps an AI doctor assistant to every human one?
AI agent can listen to you describing the symptoms, run through every combination of symptom and all diseases know to humanity in seconds and suggest the doctor which direction he should go next or propose most accurate diagnosis.
@@Digi4DD doctors aren't glorified look-up tables though, you don't need AI to do that.
What are we expecting to gain from more parameters? I’m not feeling a lack for parameters.
Scale. The bigger the model, the smarter and the more well it generalize
"Feel" - There is nothing to "feel", mathematically they require more parameters to enhance the model output.
more parameters means more better-er, people often confuse parameters with neurons, but its actually more like synapses, and humans have 100 trillion synapses, GPT-3.5 is only 305b parameters or so, and GPT-4 is just an MoE model which runs 11 GPT-3.5s in parallel with one master model choosing which model to use based on the prompt, GPT-4o is likely smaller since its cheaper.
LOVE ✊🏿
It can't do 90% of knowledge work today. It cant even do basic second year accounting journals right.
Skill issue
It can, you just don’t know how
Yeah this whole thing was odd. We didn’t have o1 then maybe, but we do have it now and it’s not even as good as claude. Certainly not the leap they seem to claim it is.
@@Brain4Brain What do you mean?
@@AaronBlox-h2t RAG
problem is that
while compute poring over data can progressively build more and better the 'routine' apps and systems we know and use today
you will be stuck with the 60pt system
with no inhouse ability to reach 61points and
you have to wait for AI to be able to generate code that can get to 61
and nobody of even average talent would care to, understand or work on that pile of code that was conveniently generated in minutes, even bother
to push it to 61 - and that is a latent threat to the system development ecosystem
sounds like a load of baloney, the only reason nobody would understand the code is because AI generally writes hard to read code, even if they put copious amounts of "documentation" with it.
Sounds like the solution to this is just to write code with minimal AI intervention, unless you plan to completely replace the programmer.
Can you have a ten trillion parameter model? Can you extract that many parameters from the dataset/language? I honestly don't know, but at some point you end up with a larger portion of parameters that have very few (if any?) instances, no? (ie most columns are all NA).
@@WearyTimeTraveler That is total bullshit.
Wattupp Garry
Excellent comparison with Fourier transform!
Wow.. Fourier Transform & an average person/ Joe..
Laplace Transform :V
It's quite obvious that in a very few years there will be no software industry. Your AI will be your only application.
The only people who say this build neither software nor AI.
Yes, but how you interface with that ai will be important. So its important to build a platform or network or processing company now. People will need platforms & how they interface will be different in the future. Device startups realize this, as Steve Jobs said, the best UI is no UI .
@ikleveland Temporarily. Soon the AI will build the required interface on the fly, and will manage all networking tasks. The era of software is coming to an end.
@@kangaroomax8198 but I do, my friend. That's why I KNOW.
It's quite obvious that you are not in the software industry.
Btw, a little thing. I think you guys may (or not) want to hide your feet. They sometimes dangling take a little of the attention of the content... And thanks for the content.
Take your eyes off them feet and focus on the content. I don’t even know they were there. I would have never seen them, focus!!!
Hahahahahaha!!!
@@Enedee007 I love to know the brand of their shoes, don't you?
Sure, I do! 😝
@@Enedee007no I completely agree with him, now he mentions it, the feet are incredibly distracting.
Open AI 4.0+ cannot work effectively at the same time successfully in multiple windows. It cannot retain a topic if you come back later, it's like talking to a new person. I'm ADHD this feels like my failure when any request cannot be met. It frequently cuts me off even though I pay for plus to come back in two hours that's some bs. I'm creating gold, shot down by you used too much come back in two hours 🤣lame
It's really frustrating to me that no one mentions new memory-centric computing paradigms that already have (low-scale) prototypes in labs.
so in other words, cache? VRAM and Sys memory does that already no?
Is this mono audio 😭
❤❤❤❤❤
LLama 70B makes a lot mistakes. Nope not true.
Incredible video and discussion. High quality debates. Congrats for the work and study
❤️☺️🍓
Scaling like this doesn’t work.
Says random guy on internet…
@ says random bot on the internet.
Do your research, everything points to scaling as a logarithmic curve, particularly for CNN and Large language model networks.
what's the use of your AI, when you accept Arab funding, add policies that hide the any islamic things from GPT, AI. Literally, GPT dont talk directly but give excuses like Mullah of ISLAM to legtitimise the atrocities by founder
What you are on