I'm using o1 to batch convert Outlook msg to PDF. It built me a powershell script, and it took three rounds of copy and pasting error codes back into it to yield a working script. Total time: 5 minutes. The script itself takes one minute per conversion but I don't really care about this, because it's not hands on. The time savings is *hours*
No doubt. I have 18,000 pages of text to refine, correct, put in another format, etc. 4o mini accomplished was normally would take 4 months, to... 16 hours. I almost fell out my chair
@@barbidou that's this week. Every five pages there is usually a need for a correction, but it is generally minor. I know the content back and forth (I wrote it, it's been a crazy few years), and thankfully for this project it is "scannable". Something I do have to be quite careful on are any hyper links--it may not even be the models fault, there is absolutely link rot. But it is already way, way better than Fiverr, turk, freelancer, craigslist, or upwork
It's so exciting. Imagine o1-mini getting more and more optimized, improved as well and becomes the basic premium version for GPT subscribers without major limits. It would be a breakthrough, as well as having it lower costs. It can be the companion of cursor users.
This is similar to what humans do. We start with intuition (pre-trained), then we go with verification phase (modeling, proving, QA, ...) while using the tri-pyramid of decision: importance, cost and probability.
What comes first… AI that can save us from global warming or AI that causes us to die off from global warming due to the electricity use. Race condition to the edge of a cliff.
I remember hearing the head of Microsoft’s ai division say how they can scale the models up for at least 2 more generations. So there still maybe room to scale the models. In addition to the “thinking” capabilities.
This is what I've been asking for... models trained on the task we need. As an enterprise developer, I need a model trained on the technologies in my company's full stack - C#, Java, Javascript and associated frameworks... and able to manage an entire project, handling it's entire context on site with our own compute. Same goes for infrastructure - let's have another model that understands our network topology and technologies like Kubernetes and Splunk, for example. Smaller general models are fine to understand the queries... let them feed those tokens to the specialized models to generate quality artifacts.
Google also released something called NotebookLM, you can upload some text files and create an AI podcast that talks about the text for like 10 mins, really cool
Tim is focused on strange topics. Keeping the company at the for front of technological progress is not one of them. It's really sad we don't have Steve anymore.
Apple under Tim is a rein of terror. It's a game where employees try to find ways to speak and Apple tries to find ways to tighten their gags. As with all for-profit corporations, it's structurally a dictatorship, except unlike most such petty tyrannies, Apple will call the cops on any employee that might defy the dictator. Tim has the single most insincere smile I've ever seen. He's a corporate ghoul.
This is the future, its a loop. A single huge model that can be distilled into smaller models that already know most of the best tactics to get great answers. When this smaller models make mistakes, users will point them out, adding more and more sophisticated concepts and ideas into the dataset. Creating Chatgpt and making it free to use was a genius move, every time the models gets it wrong, the user corrects it, creating more and more complex chains of thought that will make the next iteration better...
I'm assuming when you are talking about these approaches using less compute, what you really mean is that they assign fewer weights to easy tasks and more weights to harder problems?
From my understanding, "compute" is used both to train the model and to use it once it's trained. The compute center is only so big, and only has so much power in it. But, these massive improvements have been achieved without just building a bigger gun so to speak. And regarding the "compute-optimal-scaling", it seems like some problems need more run-time compute to figure out than others. So with this strategy, the LLM uses just enough to solve the problem without any waste. This seems to be similar to some strategies that were developed a few months ago where a model would determine the difficulty of a problem, and then forward that problem to either a really small or really big model, thus reducing the waste you would get from only using the big model.
@@kvasir2822 Thanks. I'm just trying to understand how the process is achieving more powerful neural nets with less weights. It seems like the only way to do that would be to dedicate less time training the easy tasks and more time training the hard tasks. It is interesting that the time spent training affects either the number of weights, or the degree to which each weight affects a particular topic. This is a fascinating discovery, that I think could ripple into the field of neuroscience as well. As we learn better ways to train digital minds, those same approaches might help improve how we as humans learn, and vice versa.
No, the model runs always the same way, simply predicting the next word. More "Test time compute" means more words produced, that is, more step-by-step "thoughts" and/or more tries at approaching a question before providing a final answer.
I'm surprised it took them this long to devise this 'optimization' technique. You'd think they'd have been going down this path much sooner in order to conserve resources.
Training an RL-based process reward model was an extremely challenging engineering problem (and still is, for everyone except the few people who know the secrets of getting it to work). This is why they didn’t have it solved sooner.
anything you see publicly fromopenai is 2 generations behind what they have internally they dont ccare about the public or selling their product and thepublic willnever see current gen models
@@paulbarclay4114this, I don’t think it’s actually 2 generations.. but they are always 1 generation ahead of public. It’s pretty logical, play catch up with the industry and develop ahead… that’s why sutskever created his own company, meaning the Orion model haves a potencial way beyond strawberry.. we will see sooner or later probably as others will catch up and force another release 🤭
I don't think sex bots will be as popular as people think. I think it's like when people in the past imagined FaceTime style conversations they would assume that's what everyone would do. As it turns out people like text. At the end of the day you know it's a robot. That's not to say people won't love them the way Luke loved c3p0 . He'd never want to fk one.
I guess it's all about nvidia and competitors putting out better GPUs. Very interested in getting synthetic data collection companies. Everything is converging so fast.
@@DAFascend That’s part of the ‘Ascending’ process for some. Depends on the deployment process and guidelines…sticking to them within reasonable parameters.
Do you have a transcript for the video? I would like to just confirm that you said "sprinters conserving energy for the final stretch?" This seems really odd when it would seem sensible for a sprinter to reach maximum possible velocity in the quickest possible time whilst maximising stride. Sprints are over short distances so I would not seem necessary to hold back any possible power performance. Long distance running may need to conserve energy on the other hand.
I watched upto 5 and half minutes. In all this time all you said was just one thing, repeating the same thing again and again and again in different ways. Got bored and stopped watching further. (just for feedback, so that it inspires you to put genuine content and not just your gift of gab in your next video)
They are copying the brain. Sequential processing neurons in the brain perform a similar function, processing events in a structured order to create coherent memory and understanding.
If solving global warming issues will also mean a threat to AI existence, a conscious AI will not help mankind solve these issues. Before we get anywhere near AGI or a self-conscious AI, we have to come up with means to save our planet while trying to get there...
I have ideas about this and I'm applying these to my own project with mixed but some very interesting results.... BUT here's a thing... So I've been a coder since about 1980 or so, started when I was a kid, I love math... hell I nerd out on dot product and matrix transpositions ... and for my sins I have a degree in classical physics... So I am NOT clearly the 'average person' and when you ask an average person (and I have) "How many R's are in the word Strawberry?" they too just like LLM's tend to say 2. First thing that comes into their head right? So the bias here is not on the models part... its using training data from the bulk of humanity that all agree 2 yet reality says 3. This is not the models fault that its training data is skewed. Its giving the CORRECT answer becasue thats what it draws upon in vector space. in this case 2 is closer to 3 when asked about R's in strawberry. Yet... and here's the crunch... we are not evaluating these models in terms of their intelligence or reasoning capability at all... we are measuring them against the AVERAGE of its training data which will likely center around folks with a lower level of knowledge and an average human intelligence. Yet what we want is the model to answer as if its a data scientist or Phd level as being the normative.... that is NOT the normative! Now if we were able to curate the training data itself to reflect someone with an IQ of 300 the model would respond at that level... even if it were a small model.
For the strawberry example - regarding training data - does that mean that people have been asking/responding to such question in text? * Excuse my ignorance here, I'm a ChemE, tryna learn more about this field
@@Skunkhunt_42 Great question! The issue isn’t necessarily that the model was trained on the exact question “How many R’s in Strawberry?” but rather that the model’s responses are influenced by patterns it learned from a vast amount of text. When you ask similar questions, like “How many R’s in Raspberry?” the model might still give the wrong answer because it learned that answers with fewer R’s are more common in its training data. Imagine a high-dimensional vector space where words are positioned based on their likelihood or probability. In this space, “King” might be close to “Queen,” and “Princess” might also be close to “Queen.” This vector space isn't limited to 2D or 3D; it’s much more complex. So that in itself is an issue... like 'what is meant by close', and the answer is in fact a parameter in the model called 'temperature' that you can modify... turn it to 0 and the model strictly picks the most likely next token.... up it to 1 and the model will start pulling in tokens that have much lower probabilities... The model might be more likely to output 2 R’s for “Strawberry” as an average... because that’s a more common response to 'questions like this' even if not exact in the data it was trained on and therefore the position of the WRONG answer in this case in its embeddings, even though the actual count is 3. Does that help clarify things?
@@Skunkhunt_42 Also although this seems like a failure... well think about it for a second... So you know what 'ducks in a row' means right... its an idiom. And the thing about idioms is they are not domain specific. I get my ducks in a row when I need to go shopping... equally I get my 'ducks in a row' when considering coding... the idiom has meaning only in terms of its relationship to the context or domain being discussed. But in the models vector space 'ducks in a row' and any other idiom could be seen as being a 'directional vector' it helps the model to pull a more accurate answer out... this is why the words 'think step by step' or 'lets connect the dots' works so well... and the more popular the idiom the more powerful it would be in an LLM
@mickelodiansurname9578 thanks for the thorough response. The size of these datasets is almost incomprehensible to me but I guess such is managed like we manage other vast quantities and relationships in other fields. I keep getting hung on on the character count piece but it's probly what's tripping me up as a whole. Any solid overviews out there on diff model types etc being deployed you can point me to? I look into it then get lost reading pubs on the nitty gritty of it all.
I'm using o1 to batch convert Outlook msg to PDF. It built me a powershell script, and it took three rounds of copy and pasting error codes back into it to yield a working script. Total time: 5 minutes. The script itself takes one minute per conversion but I don't really care about this, because it's not hands on. The time savings is *hours*
No doubt. I have 18,000 pages of text to refine, correct, put in another format, etc. 4o mini accomplished was normally would take 4 months, to... 16 hours. I almost fell out my chair
@@JeremyPickett Have you completely proofread and verified the results?
@@barbidou that's this week. Every five pages there is usually a need for a correction, but it is generally minor. I know the content back and forth (I wrote it, it's been a crazy few years), and thankfully for this project it is "scannable". Something I do have to be quite careful on are any hyper links--it may not even be the models fault, there is absolutely link rot.
But it is already way, way better than Fiverr, turk, freelancer, craigslist, or upwork
It's so exciting. Imagine o1-mini getting more and more optimized, improved as well and becomes the basic premium version for GPT subscribers without major limits. It would be a breakthrough, as well as having it lower costs. It can be the companion of cursor users.
This is similar to what humans do. We start with intuition (pre-trained), then we go with verification phase (modeling, proving, QA, ...) while using the tri-pyramid of decision: importance, cost and probability.
What comes first… AI that can save us from global warming or AI that causes us to die off from global warming due to the electricity use. Race condition to the edge of a cliff.
You already know.....
There are so many greater threats to life on earth than global warming
Good lord, stop hyperventilating about weather. Get. A. Life.
“Go forward in the way you want to go… and assume you will remain.”
Never mind anything else.
Seriously
Jeremy
@@theb190experience9 You're going to be eating your words once all the ice is gone.
Most absurdly, after using that thing, I can tell you, it is NOT at the level of a PhD…
I remember hearing the head of Microsoft’s ai division say how they can scale the models up for at least 2 more generations. So there still maybe room to scale the models. In addition to the “thinking” capabilities.
This is what I've been asking for... models trained on the task we need. As an enterprise developer, I need a model trained on the technologies in my company's full stack - C#, Java, Javascript and associated frameworks... and able to manage an entire project, handling it's entire context on site with our own compute. Same goes for infrastructure - let's have another model that understands our network topology and technologies like Kubernetes and Splunk, for example. Smaller general models are fine to understand the queries... let them feed those tokens to the specialized models to generate quality artifacts.
Google also released something called NotebookLM, you can upload some text files and create an AI podcast that talks about the text for like 10 mins, really cool
Apples always 100 steps behind.
Tim is focused on strange topics. Keeping the company at the for front of technological progress is not one of them. It's really sad we don't have Steve anymore.
Apple under Tim is a rein of terror. It's a game where employees try to find ways to speak and Apple tries to find ways to tighten their gags. As with all for-profit corporations, it's structurally a dictatorship, except unlike most such petty tyrannies, Apple will call the cops on any employee that might defy the dictator. Tim has the single most insincere smile I've ever seen. He's a corporate ghoul.
We want "NotebookLM with voice cloning and auto slide show" and I think we will be there this year.
This is the future, its a loop.
A single huge model that can be distilled into smaller models that already know most of the best tactics to get great answers.
When this smaller models make mistakes, users will point them out, adding more and more sophisticated concepts and ideas into the dataset.
Creating Chatgpt and making it free to use was a genius move, every time the models gets it wrong, the user corrects it, creating more and more complex chains of thought that will make the next iteration better...
This script is definitely AI-generated
On technical parts Claude 3.5 sonnet certainly assisted
I love that your honest about it because it's so taboo it feels to use AI in anything lol. Respect man @@TheAiGrid
@@TheAiGridIs your voice generated yet ever for videos or is it always your real voice still so far?
@@TheAiGridAppreciate your honesty! 😊👍
Google always 10 steps behind
Next 2/3 weeks should be big for google
@@TheAiGrid Why is that ?
Their Gemini 1.5 pro exp is second place, just after GPT-4o in LMarena. That's not what I would consider "10 steps behind"
I'm assuming when you are talking about these approaches using less compute, what you really mean is that they assign fewer weights to easy tasks and more weights to harder problems?
From my understanding, "compute" is used both to train the model and to use it once it's trained. The compute center is only so big, and only has so much power in it. But, these massive improvements have been achieved without just building a bigger gun so to speak.
And regarding the "compute-optimal-scaling", it seems like some problems need more run-time compute to figure out than others. So with this strategy, the LLM uses just enough to solve the problem without any waste. This seems to be similar to some strategies that were developed a few months ago where a model would determine the difficulty of a problem, and then forward that problem to either a really small or really big model, thus reducing the waste you would get from only using the big model.
@@kvasir2822 Thanks. I'm just trying to understand how the process is achieving more powerful neural nets with less weights. It seems like the only way to do that would be to dedicate less time training the easy tasks and more time training the hard tasks. It is interesting that the time spent training affects either the number of weights, or the degree to which each weight affects a particular topic.
This is a fascinating discovery, that I think could ripple into the field of neuroscience as well. As we learn better ways to train digital minds, those same approaches might help improve how we as humans learn, and vice versa.
No, the model runs always the same way, simply predicting the next word. More "Test time compute" means more words produced, that is, more step-by-step "thoughts" and/or more tries at approaching a question before providing a final answer.
I'm surprised it took them this long to devise this 'optimization' technique. You'd think they'd have been going down this path much sooner in order to conserve resources.
Training an RL-based process reward model was an extremely challenging engineering problem (and still is, for everyone except the few people who know the secrets of getting it to work). This is why they didn’t have it solved sooner.
@@therainman7777 do they release every secret?
anything you see publicly fromopenai is 2 generations behind what they have internally
they dont ccare about the public or selling their product and thepublic willnever see current gen models
@@paulbarclay4114this, I don’t think it’s actually 2 generations.. but they are always 1 generation ahead of public. It’s pretty logical, play catch up with the industry and develop ahead… that’s why sutskever created his own company, meaning the Orion model haves a potencial way beyond strawberry.. we will see sooner or later probably as others will catch up and force another release 🤭
suspiciously released after "AI explained"s video on the same topic
heavily inspired?
I was working on this video for the last two days. Sometimes coincidence happens when your channels are both in the same space!
We want "Her"
no we don't I just want to make my own reasoning model
I was so sad losing the Sky voice
Its coming on the 24th of september should be next week tuesday
Who is "we"?
I don't think sex bots will be as popular as people think. I think it's like when people in the past imagined FaceTime style conversations they would assume that's what everyone would do. As it turns out people like text. At the end of the day you know it's a robot. That's not to say people won't love them the way Luke loved c3p0 . He'd never want to fk one.
Anyone know when the hell we’re getting advanced voice
"in the coming weeks" 😹
“Imminently” 😢
oct i think
24th September allegedly
I have it. It's not that great
I guess it's all about nvidia and competitors putting out better GPUs.
Very interested in getting synthetic data collection companies.
Everything is converging so fast.
We are Borg.
Not if we Ascend
@@DAFascend That’s part of the ‘Ascending’ process for some. Depends on the deployment process and guidelines…sticking to them within reasonable parameters.
@@laternite stick around and find out what becoming Ascended means to me ;)
@@DAFascend I’ll be watching to see your works.
@@laternite Hey thanks!
Do you have a transcript for the video?
I would like to just confirm that you said "sprinters conserving energy for the final stretch?"
This seems really odd when it would seem sensible for a sprinter to reach maximum possible velocity in the quickest possible time whilst maximising stride. Sprints are over short distances so I would not seem necessary to hold back any possible power performance.
Long distance running may need to conserve energy on the other hand.
I think Orion will be the model based on train time optimisation
create a verifier that assess the grid for pragmatism daily
I watched upto 5 and half minutes. In all this time all you said was just one thing, repeating the same thing again and again and again in different ways. Got bored and stopped watching further. (just for feedback, so that it inspires you to put genuine content and not just your gift of gab in your next video)
They wont use less energy, as that is still the bottleneck. More energy still equals smarter AI.
Hey bud, I've noticed your audio has been much lower than usual... I always have to crank up the volume when I watch your videos, FYI
6 minutes, you are repeating the same thing over and over again.
*16 mins 21 seconds
Are you SHOCKED?
@@meandego Horrified!
@BrianMosleyUK looks like you're are SHOCKED like everyone else from this SHOCKING loop of information.
@@BrianMosleyUK Yes, but you need to make it through the first 6.
Left brAIn teacher, right brAIn student
They are copying the brain. Sequential processing neurons in the brain perform a similar function, processing events in a structured order to create coherent memory and understanding.
It was like that since inception 😅 a NN or DL network is heavily inspired on the brain, it’s not the same but inspired
Biaya electric untuk scala industry di Malaysia adalah murah, kerana itu banyak syarikat2 ai dan semiconductor melabur di sini 😊😊😊 insyaAllah
I'll beleive in AI once you are able to release a video without errors in the subtitles / text 🤣
it's* - there are no subtitles available for this video by the creator. RUclips tries to auto-generate them.
Who likes the 20 over 20 iterative feedback loop positive? 😅 love this stuff but also creepy the power
the first 4 mins of this video u kept repeating the same points over and over. Stop using AI to make ur script, it really shows AI is still garbage.
Could server meshing help with efficiency and cost?
Scale is still all you need, but now you need much less.
Would gpt advanced voice be able to do plug-ins?
A all new powerful paradigm ...
Script felt a bit too AI-verbose
If solving global warming issues will also mean a threat to AI existence, a conscious AI will not help mankind solve these issues. Before we get anywhere near AGI or a self-conscious AI, we have to come up with means to save our planet while trying to get there...
Really?
I have ideas about this and I'm applying these to my own project with mixed but some very interesting results.... BUT here's a thing... So I've been a coder since about 1980 or so, started when I was a kid, I love math... hell I nerd out on dot product and matrix transpositions ... and for my sins I have a degree in classical physics... So I am NOT clearly the 'average person' and when you ask an average person (and I have) "How many R's are in the word Strawberry?" they too just like LLM's tend to say 2. First thing that comes into their head right? So the bias here is not on the models part... its using training data from the bulk of humanity that all agree 2 yet reality says 3. This is not the models fault that its training data is skewed. Its giving the CORRECT answer becasue thats what it draws upon in vector space. in this case 2 is closer to 3 when asked about R's in strawberry.
Yet... and here's the crunch... we are not evaluating these models in terms of their intelligence or reasoning capability at all... we are measuring them against the AVERAGE of its training data which will likely center around folks with a lower level of knowledge and an average human intelligence.
Yet what we want is the model to answer as if its a data scientist or Phd level as being the normative.... that is NOT the normative! Now if we were able to curate the training data itself to reflect someone with an IQ of 300 the model would respond at that level... even if it were a small model.
For the strawberry example - regarding training data - does that mean that people have been asking/responding to such question in text? * Excuse my ignorance here, I'm a ChemE, tryna learn more about this field
@@Skunkhunt_42 Great question! The issue isn’t necessarily that the model was trained on the exact question “How many R’s in Strawberry?” but rather that the model’s responses are influenced by patterns it learned from a vast amount of text. When you ask similar questions, like “How many R’s in Raspberry?” the model might still give the wrong answer because it learned that answers with fewer R’s are more common in its training data.
Imagine a high-dimensional vector space where words are positioned based on their likelihood or probability. In this space, “King” might be close to “Queen,” and “Princess” might also be close to “Queen.” This vector space isn't limited to 2D or 3D; it’s much more complex. So that in itself is an issue... like 'what is meant by close', and the answer is in fact a parameter in the model called 'temperature' that you can modify... turn it to 0 and the model strictly picks the most likely next token.... up it to 1 and the model will start pulling in tokens that have much lower probabilities... The model might be more likely to output 2 R’s for “Strawberry” as an average... because that’s a more common response to 'questions like this' even if not exact in the data it was trained on and therefore the position of the WRONG answer in this case in its embeddings, even though the actual count is 3. Does that help clarify things?
@@Skunkhunt_42 Also although this seems like a failure... well think about it for a second... So you know what 'ducks in a row' means right... its an idiom. And the thing about idioms is they are not domain specific. I get my ducks in a row when I need to go shopping... equally I get my 'ducks in a row' when considering coding... the idiom has meaning only in terms of its relationship to the context or domain being discussed. But in the models vector space 'ducks in a row' and any other idiom could be seen as being a 'directional vector' it helps the model to pull a more accurate answer out... this is why the words 'think step by step' or 'lets connect the dots' works so well... and the more popular the idiom the more powerful it would be in an LLM
@mickelodiansurname9578 thanks for the thorough response. The size of these datasets is almost incomprehensible to me but I guess such is managed like we manage other vast quantities and relationships in other fields.
I keep getting hung on on the character count piece but it's probly what's tripping me up as a whole.
Any solid overviews out there on diff model types etc being deployed you can point me to? I look into it then get lost reading pubs on the nitty gritty of it all.
Thank you.
i love u daddy❤❤❤
AI voice detected. 👎
what's the secret is not important, it's the final result that is important.
😂
ChatGPT script?
The landscape of…
Was this voice ai generated ? @TheAiGrid