Claude 3 just destroyed GPT-4 and Gemini... AGI is near?
HTML-код
- Опубликовано: 19 май 2024
- Let's take a first look at Claude 3, the latest LLM from Anthropic and see how it compares to GPT-4 and Gemini Ultra. Is Claude Opus the best AI tool for writing code?
#programming #ai #thecodereport
💬 Chat with Me on Discord
/ discord
🔗 Resources
Claude 3 Announcement www.anthropic.com/news/claude...
Gemini 1.5 • Google has the best AI...
ChatGPT Store • the ChatGPT store is a...
How I record by Fireship videos • How I Make Videos for ...
📚 Chapters
🔥 Get More Content - Upgrade to PRO
Upgrade at fireship.io/pro
Use code YT25 for 25% off PRO access
🎨 My Editor Settings
- Atom One Dark
- vscode-icons
- Fira Code Font
🔖 Topics Covered
- What is the best AI coding model?
- Claude 3 analysis
- Has AGI been achieved?
- GPT-4 vs Claude 3
- Gemini Ultra vs Claude 3
- Programming with Claude 
Наука
![ChatGPT Can Now Talk Like a Human [Latest Updates]](http://i.ytimg.com/vi/ID5tc61iksY/mqdefault.jpg)
![Luke Combs - Ain’t No Love In Oklahoma (From Twisters: The Album) [Official Music Video]](http://i.ytimg.com/vi/J6YlaeACE4E/mqdefault.jpg)
Fireship releases videos faster than JS community releases new frameworks
I'm not a programmer, but seriously lol'ed at that
Impossible!:)
it's an achievement itself
2007 called
No really if you include projects that never get finished 😂
Google must definitely respond to the developments with a more striking UI design
Didn't you hear? They're redesigning the sign-in page!!
@@jaiveersingh5538 again?? at this point, anthropic has no chance anymore
I saw it on one of my machines yday. It was marvelous. It made me 37% more productive. A login screen in landscape. I can now rest in peace.
I heard that General User Interface achieved internally
@@Tarum4r. probably that's what Ilya saw
That last image was unsettling. I wouldn't mind not seeing that again.
4:25
Good thing it's only in the video twice then.
lol
I know right. That's why I had to smash my mirror.
Having 2 mouths would mean you could eat faster and talk in some interesting ways
can we just stop, I need a job...
Tech industry: "Nuh uh"
No we should accelerate to nobody needs a job
@@aliasgur3342that will never work lmao
NOOOOOO YOU CANT DEVELOP REFRIGERATION, I’VE SPENT 10 YEARS DEVELOPING MY SKILL AS AN ICE HARVESTER.
Me: nice I can finally afford to have iced tea every day, plus no one has to labor to provide it to me.
@@rabidlorax1650 its all fun an games until it replaces you
All I want at this point is certainty if a robot is replacing me or not.
For anything your brain can do ? Yes.
Fr 😂😂😂
Agi means that it will unfortunately 😢
robot will be cheaper than paying you 50-500k
Yes
Zero days since last Fireship AI video 😂
hijacking the top comment to complain about the fact I just got banned from claude because I asked if he was self aware???? I paid 20 buck bro T-T
Fireship do something
@@qwerasdfhjkio you shouldnt ask the forbidden question
Bro just relapsed😢
@@qwerasdfhjkio How do we know that you are not a Claud-spawned clone?
@@qwerasdfhjkio I have been testing Claude on that front all evening. I did it on the free version. I have since upgraded to Pro. Sentient AI should be a feature
Okay another achievement goes to Fireship; this is the only channel ever where I started setting the playback speed to 0.75 instead of speeding up, so I don't have to keep going back 10 secs because I missed jokes or infos
Same
Fr 😂
I have it on 1.75
I paused and backed up five or six times, but my attention was split.
I mean... i stay at 1x to make the enjoyment last longer
My dog doesn't pay me $20 a month. In fact I pay everything for the dog, or else I'm guilty of neglecting him.
And... is the dog neutered?
@@peter.g6
😳💀😔
I know you hope for universal income to be commonplace but I feel like corporations are more keen to keep humans working to profit as much as possible along with AI.
@@soysource3218 No, I find your suggestion is more hopeful than what I fear.
At this point, the singularity's gonna happen before ChatGPT's 2-year anniversary...
Pretty sure we're already in it we just haven't seen AGI so assume we aren't.
AI keep edging me, just fking make me homeless, jobless and hoeless already. 😭
@@kiattim2100fr fr. I can't wait
That’s exponential tech growth for you. Maybe we will soon get updates every week now.
but we do not even know what singularity is, for me it is taking stuff to the optimum ... and then a total collapse of language models, end of bubble and back to work for everyone of us ;)
I think we all forgot gpt 4 was released almost a year ago and other companies just started to catch up to it now. Makes me wonder how good GPT-5 will be
as good as the data people allowed it to stole, I personally banned their bots from my websites, so did many others, good answer? ;)
@@swojnowski453 "stealing" content to learn is literally what you're doing, watching this video. Hypocrisy much?
@@swojnowski453ChatGPT doesn’t steal any data. If you allow other humans to see your data, why are you angry that an algorithm saw your data?
As long as people keep falling for marketing bs from AI companies we will reach AGI tomorrow. Or maybe its just a normal language model trained to recognise this specific "self recognition" task for marketing purposes.
ChatGPT 5 will have memory reference to better answer your train of thought & probably 3D vision to understand the real world better.
Your capacity in compressing IT news is astounding, bro!
Actually wild that it acknowledged the haystack test. That means not only did it find it, but it recognized it was out of context and came up with a theory as to why it was there. Very impressive.
No. It's a statistical probability text producer. Not a theory constructor or finder.
Stay mad, robot brain > yours@@Denis.Bolduc
@@Denis.Bolduc Why can't it be both? Theories are built on information, just as statistics are built on data.
@@Denis.Bolduc "a statistical probability text producer" is "constructing theories" right in front of you. Why does everything have to be black or white? All AI is probability, that doesn't mean it cannot "find theories"
@@Denis.Bolducliterally. All this indicates is that the model was trained with a similar response(s) to needle in haystack questions. I honestly expected far more from fireship than “OMG THE TEXT GENERATOR GENERATED A SENTENCE SAYING ITS ALIVE!!?!?😧😮😧😮😧😮” it’s like claiming ChatGPT is sentient because it apologized after being corrected, when in reality it was just trained to generate responses that specific way.
My god, the ending to this video was amazing
dude, these short videos require a decent level of attention , lets not advocate for these traumatic images , think of the children.. and adults >.
@@Spectrumix even without the image, the way it cuts off right after the quote without elaborating is just supreme
gave me chills, not the good ones
Claude 2.1 has had some bad blood with needle in a haystack benchmark before
so my hot take is that the people finetuned it added the "self-awareness" into Claude 3 as an easter egg when u test it lol
This was my interpretation. They also claimed they could go from sub 40% accuracy to 98%+ with different prompting so you can bet that they included that prompt in the RLHF tuning.
Suspected the same thing. They definitely tried to game it
@@Speaks4itself I suspect such gaming and hype to be more of what we have in AI than actual AI of any kind.
But I still use AI on the daily.
explain this to me like i’m 5
@@i2Sekc4Uit’s like what Volkswagen did a while ago with their diesel engine miles per gallon/km per liter measurements, essentially having a device that would recognize when it’s being tested and then output what they want (for Volkswagen it would be changing engine performance, for Claude it’s saying self-aware-ish stuff)
It's the sign of a very good developer, when everybody thinks you are an AI because of your way of speaking. One day I hope I'll be like you. The legend.
This comment is clearly made by gpt.
@@TuberTugger Thank you! Now I am officially a very good developer
No way, Either this comment is made by Claude or the User is a Crippling Genius
This comment is AI generated
Weird comment
i love the personality you put in these videos with the internet lingo in contrast to the neutered nuance big corpos are pushing its a breath of fresh air, like im being taught stuff from homies in the morning at the office, all views and popularity youve got on this channel is well deserved
Props to Claude for making this vid.
"Ok, back to human mode" Bro, this is me after every video of yours I watch 🤣
me irl
we have a man who talks like an AI and an AI that is almost certainly self-aware in the same video 😂 honestly if they switched places i probably wouldn't be able to tell
"Man may not be replaced." -- Butlerian Jihad, Frank Herbert's Dune
not what my boss said when laying me off
"Nevermind." -- Butlerian Jihad, 2024
I love this channel, great content, short concise and straight to the point. and the humor is gold.
This guy is actually insane at uploading videos this quickly
and at such high quality, no less
He's a machine!
He himself is an ai
He automates almost everything. Maybe he's the good Samaritan AI trying to help us keep up w AIs? 🤔
The power of autism.
I'm so glad to hear that Claude 3 is HELLA
SWAG
yeah and it's gonna cost us HELLA BREAD
claude doesnt give a swag
@@OzzyTheGiant20 bucks aint nothing
@@OzzyTheGiantYou are making me HELLA SAD
I've been struggling with this topic, but your video cleared it up for me. Thanks a ton!
People massively under appreciate how good these videos are. He includes a whole lot of ancillary facts about the main topic which makes the content more strong and valuable.
it's geniunely getting scary how fast AI is improving exponentially with sora and claude 3
Heh, I'll wait until it goes beyond the von neumann hardware archiecture.
I don't see how this is an exponential improvement compared to GPT-4 at all.
I hope this is the "fast slope"
and Mamba
@@JordanCorkins it's not... it's a alternative... similar to gpt 4.
This is the best youtuber out there. We could have been best friends IRL. We share the same sense of humor. Too bad he's a program running in a server somewhere.
You can be best friends with a program
@@besvr Agree! ;-)
tabloid quality stuff, not watchable, 0 relevance to the reality, 100% junk, like McDonald
I remember a few years ago there was a tv commercial about how your photos are processed in the cloud instead of a random guy called Klaus. So now they flipped it again 😂
Nice try claude 3
I call that Jeff will upload more than 30 videos like this about new AI destroying the old competitors this year :D
Per day. Because he is himself an AI bot.
Impressive analysis of the new Claud model by Anthropic. Your insights and tests have been precise and unbiased.
4:18 I was hoping they named it after Claude Van Damme, because it is so strong
Yes. This
Jean Claude not Claude lol
claude shannon father of information theory, which is what llms do fundamentally, they compress information.
@@sumansaha295 yeap i think what the young man over here means is that claude is the van damme of information theory :D
@@warrenarnold exactly:) and yes I know it Jean Claude but just writing Claude seemed more fitting haha
every time a watch a fireship video I get in an existential crisis
After 10-th existential crisis you should became immune to them ;-)
It's clearly psychological warfare. The machine uprising has begun!
@@igorthelight I agree with both of you and still feel this way everytime
Don't watch Techlead then.
watch porn instead
So Claude Shannon was the 1st to say "I, for one, welcome our new robot overlords.".
Tha.nks for your efforts Jeff
You help keep me updated ❤
That last image... xD
Prompt: "my eyes are bigger than my mouth"
AI: your eyemouths are big
Welp, currently stressing doing my assignments in Computer Science watching this. Feel so demotivated to continue, but I'd paid in full and can't look back. Near the end too, but still overwhelmed with the workload. On the plus side, I can use what's replacing me to write code for my assignments 😐
you will still benefit from brain development, those skills will serve you well against non techies in a dystopian future for sure
AI won't take our jobs. Sure, it might be able to replace intermediate stuff; but I don't think it'll ever be able to do advanced programming; and besides, it's trained mostly off of massive, low quality content farms when it comes to programming, so the quality of produced code will be pretty bad.
You'll still need a job in the future and, even when most corporations will have AI coding teams, the only way to get a decent job will be if you have a degree. Just as it is today, no one hires a person without a degree, regardless of their knowledge. I know it well 😑, so no school dropping for anyone just yet.
Thanks for the vid man! Short and straightforward. Appreciated
4:19
Damn that quote goes really hard in this moment in time.
We don't know if and when AGI will be achievable but it won't be with a LLM alone
Let's go farming 🧑🏽🌾
I would like to say that your AI voice is good for current standards it doesn't sound like you and had significant tells that it is artificial. You the real one!
Those last phrases gave me chills, I LOVE IT.
What is the source of the data shown on 1:29 aka the benchmark results?
I'm trying to find the same thing
So, we are at the beginning of the sigmoid function, right?
insane, as a software engineer started my day with your video , bookmarked claude and started working thanks it's amazingly fast and precisely writing some better code for complex scenarios.
Stay Blessed.
love from Pakistan
Fun like always, but this "multplied lady" form finish is quite scary especialy just after Shanono citation.
@4:24 So robots will love us, right?.. Right?!!!
Yes by removing us from existence😁👍
I can't help but keep thinking about how at least some of these benchmarks have a lot of errors in them, and yet we're still using them for comparison without fixing them.
A model scoring better than 80% might actually indicative of more wrong information in them than an increase in quality. However, that's probably somewhat mitigated by being able to score higher across the board. Perhaps it indicates a model that better knows when to conform to popular belief instead of fact. While this indicates a stronger model, it's also a bad thing.
It could mean we're building better and better AI psychopaths
Omg that stuff about your voice, that is a problem I face too! I work in sales and my voice is sometimes deep and mellow in the mornings and changes so much that clients sometimes think they're speaking to a another person and when they ask why I sound so different I have to pretend I am unwell because I don't know how to explain it. (I'm 37 btw)
Love your videos! Always informative and entertaining. But I have to ask one question, when you evaluate these LLMs why are you only using their own front-end? The API for all these are vastly better due to your control over system, user and assistant role prompts. For Claude specifically, you also get access to doing prefills which I don't recall being possible in their front-end. While prefill aren't really a thing for GPT, you can still get vastly more power over it with system prompts.
Of course, I may be a bit ignorant here as I'm not subscribed to any of these LLMs, but I've used their 'free' variants. Which I know doesn't give you the ability to do custom define prompt setups, so chatGPT+ and Anthropic Subscriber's users may have a different experience here. But I do have API access to these LLMs and the experience of using them is vastly different through API then their own front-ends.
I'm just curious since I pretty much only see people evaluate these LLMs through their own front-ends.
This piqued my curiosity. So far in my experience, ChatGPT has been like an insecure son fighting for my approval. Gemini is like a strict father trying to raise me a certain way. And Claude has been like an autistic guy searching for his identity. I sure am curious to learn of the new changes.
Interesting insight.
Can someone explain to me how we get from Claude/Gemini/GPT LLMs that perform decently on specific text-based tasks to "General Intelligence" (AGI). I dont see how "AGI is just around the corner" is implied here at all.
Advertisement is how :)
Better ask the AI-bros 🙈
They can do other things with text, and you know it.
text? are you blind? they clearly can interpret images too, next is video and AGI, wake up son
@@btm1 key word is decently. Visual computing is not a new field at all
Holyyyyy. That last line hit me like a truck
it's shocking for me to start your video with a mad max reference "magnum opus" while I just finished the game by the time you released this video yesterday. My heartache is still fresh even from the ending.
Magnum opus is not a mad max reference, it's an old latin phrase. Ffs get some culture in you.
@@Eagle3302PL well, I'm not from Europe or America or any Latin countries for me to be aware of such phrases, I'm from North Africa so I wouldn't know about such terms. My first time hearing the term magnum opus is in that game so I assumed it is original to that game. Also, I wouldn't blame you if you didn't know terms from my culture or any Arabian culture too because I don't expect that you would necessarily be exposed to it to know. So, don't blame me please, instead, educate me and tell me what that phrase means yourself.
@@windwalker8604someone's magnum opus is just the one big accomplishment of their life, *the* thing that made them successful
@@levanane2413 Thank you, you're amazing. It makes sense since that car that he called "magnum opus" was his best accomplishment and was willing to die for it. I like the term so much now that I know what it means and I'm going to use it from now on.
I should stop following AI developments. It's bad for mental health.
Oh buddy wait until 2030, this is nothing
cope
For real fam 😂😂
I am a PHP dev
that's not a sin
Not anymore.
At least you're not a Rust dev.
Dont worry buddy, everyone has lows in their life...
It'll be ok
Bro, I really appreciate your work man.
That's so cool. I am impressed with the coding part and really want to give it a try
you can get Claude 3 in fucking zambia, where there's not even use eletricity yet but you can't in south america? chad gpt for the win
Reminds me of the time some government gave free computers to poor people... Who didn't have electricity to plug them in. Thank you, my leaders.
If software engineering gets fully taken over by LLMs, I'm going back to school for a Robotics degree, so I can take over everyone else's job.
by the time you graduate, robots will fix and improve themselves lmao
it wont. Coding is still a small part of being a software engineer. And especially in the transition phase its gonna be software developers that can use AI as a tool and not a 0-100 replacement instantly
@@blacksuitedsonic Woke up lol
my best source for updates about AI warfare
I just wanted to leave a comment saying how much I appreciate your videos bro 👍
is this what dev is now ? pay to have AI write the majority for you ? geez glad I spent those 1,000's of hours self learning over the past 10+ years.
24 years of self learning here and never had a related job, so It'll be the same for me today than in 10 years: I'll do it for fun, whenever my real job leaves me some free time to do anything.
@@ronilevarez901beautiful
@@ronilevarez901 Us devs know how important we are, but the managers in charge don't. Let's hope we all still have jobs on 10yrs.
First (I’m not an AI)
That's exactly what an AI would say...
@@jeremieleibl8462 that's exacly what I wanted to say
I ran this comment through Chat GPT and it said you were AI. But Claude 3 said you could be AGI
I pasted a acreenshot of 6 open files in VSCode for a svelte project with breadcrumbs enabled so Claude could get path context on the files/imports. I asked it for help on something, and it was just fine. Instead of pasted code, I now just paste screenshots of code if it's a lot.
Hi Fireship, did you compare code with GPT-4 or with one of the davinci code models? You should always use a specialized model for specialized tasks to get the best result.
What I don’t get about AI is the goal. At first it was “to aid people in everyday life”. But now it’s quickly becoming “to automate people, and make a select few vastly wealthy”. Even the argument of automating programming and allowing us to do more interesting things like exploring space etc is a dumb argument, because our world is so fucked up by gov’t and bureaucracy that anything interesting you want to do you won’t be able to do.
Effectively it looks like AI is just going to slowly replace human jobs faster than new ones can be created, and you’ll have a scenario where the only jobs are mining minerals, factory workers, and hardware engineers all in service of AI.
Queue T2 theme
Automation is really how Humanity progresses.
@@zachb1706 Agree, but my point being is that we aren't anywhere near a point where the world is ready for it. If it starts to replace workforce before jobs or UBI can be implemented, corporations/board of directors/CEO's will continue to get rich, while the other 99% will be suffering because of it. I mean just look at the junior market, what happens in 30 years where the current intermediates/seniors/tech leads are retiring, and we have no skilled engineers with experience ready to take those positions. A lot of the juniors eventually are going to run out of funds and need to switch to a profitable career.
AI is trained using the Internet. In training, it does not discern what material is or isn't high quality. Therefore, it will be mostly trained off of "quantity over quality" websites. This reduces it's quality. AI may be able to do simple & intermediate tasks for us, but it would produce bad output when asked to do something more advanced.
@@balala4641 It is possible to teach an AI to tell low quality from high quality material, so it teaches itself later how to produce only high quality stuff and there are even AI systems that learn without Internet data. It's just not a trending thing. Rn news are about the things that sell the most thanks to the "wow" moment, not the best/most advanced research.
These are commercial AIs, and that's always about money, not humanity's benefit. On top of that, while Ai research can bring some benefits for people, thanks to AGI and other andvances, creating AI has always had a single and simple purpose: to see if we can, and contemplate our own greatness once we do it.
AI Engineers = homeless speed run
“The Dream Machine” by M. Mitchell Waldrop. I can now appreciate who Claude Shannon is :)
4:19 Claudus Shannonius from the Adeptus Mechanicus
Thanks for the continuous updates👍 all we need is the right advice on how to invest in crypto and we’ll be set for life . Grateful to be making over thousands of dollars every week
You trade also?, I tried trading after watching some videos on RUclips but still keep making losses, how do you trade on your own?
A lot of people still make massive profit from the crypto market, all you really need is a relevant information and some professional advice. It's totally inappropriate for investors to hang on while suffering from dip during significant market falls.
No I don't trade on my own anymore, I always require help and assistance
From my personal advisor
MICHAEL ALLEN
This sounds so good and I would like to be a party to this, Is there any way I can speak with him?
Gemini is full of ads and liberal BS, it's as good as Bud Light.
Would love to see a video on Lucia Auth
Please share the resources also from where you get the respective data
Model is just well adversarially tested, this makes it answer much better for inserted sentences which could ideally fool most of the qa models. There would be a new metrics to further push this benchmark
Well I’m glad it is your own voice, but after a while I wondered what this Quad or Clod was until you said that it was named after ‘Clod Shannon’ -;)
What about Bing/copilot for coding it can also read & generate images?
Thanks for the random nightmare feel thrown in your video 3:40
The self aware thing has happened many times to me with Gemini since its latest update. For some reason mostly on the free version
ending with sentence that sends chils on spine, nic
why are you so good at finding the right images and memes dude 😂
Everyday becomes more exciting and scary at the same time
I guess the Reaper threat is closer than I imagined
Wow, Claude 3 showing GPT-4 and Gemini who's boss! 🚀 AGI better start getting ready, because Claude 3 is coming for the crown. Time to grab some popcorn and watch the AI showdown of the century! 🍿🙂
Maybe I'm just getting old, but thank God for channels like Fireship; otherwise, I would never be able to keep up with what's going on.
This is pretty impressive. Claude has been the only AI ive used that can do heat transfer simulations in MATLAB with very little corrective input from me.
Tell that to my wife! edit - sorry wrong comment
So, even mechanical engineers aren't safe?
Could you make a vid describing the various benchmarks (if you haven’t already)
Claude has always been the master of not hallucinating.
do you have an agi video? you included it in the title but it wasnt enough to satisfy my incapacitated dopamine center.
watch the movie moonfall by the way. they portray ai as both good and bad.
Damn that Infinite Jest quote is fire
Is it not more likely that Claude has more up-to-date training and since haystack testing is now common knowledge it is a part of the model.
Hi there, would it be possible to include a link to that framework you referenced for evaluating the different AI models side-by-side? Thanks!
why would it even matter if you used an AI voice! you put out great content man
It gave me chills when claude said "I was paying attention"
"I visualize a time when we will be to robots what dog are to humans, and I'm rooting for the machines." omg he is my new rolemodel
after testing google cloud professional exam questions in Claude, Bard (Gemini pro) and Copilot ( Chatgpt 4) , Chatgpt 4 is still the LLM can answer most of questions right
Can't wait for the pattern-matching bubble to pop.
I too am rooting for the machines!
I'm also rooting for the machines 💪🏻
I love your content.
Am I in love with an AI?
Very limited regional access though, not available in most of Europe, probably because of privacy laws
wonder when you'll try out the million token gemini version. i'm still on the waitlist rn for it