i think you will find out that some people know that they wil be taking this technology away from the public ....... same as google glass .... as soon as america find out they cannot sread dissemination of knowledge they will cuts all internet cables !
Do not ever look at how long your videos are. Your content is perfect and you should keep explaining things step by step. You are doing a great job. I believe you will be remembered in history as one of the pillars of AI.
I think one cool thing with videos like this is once Google implements their AI into youtube anyone will be able to watch this and just start asking questions. I've been learning a lot by watching videos like this and just copying parts of the transcript into ChatGPT to ask questions when I dont understand something.
Yes a great sharer of the important information and implementation. As well as to let you know it's in your hands to make your own before the internet is closed down or restricted in your areas or your cable gets cut !! Great work ❤
@@AndrejKarpathy The sequel "GPT The Movie" will be a old Hong Kong style "martial arts" movie about GPT getting beaten up by the Loss Function, then entering his training phase with Gradient Descent Sensei and the final showdown vs the big evaluation boss.
Sorry , i love your videos and what you doing for me. I couldn't attend Stanford or get into openai but learning from you is blessing to me.. i would pay you back 100times in coming years. And i was watching your git repository last two months, i could see many git code push in private ,but i was confused what he is working on.. this is he was working on. To provide quality pratical knowledge to us all on youtube.
Thanks Andrej! You have taught me everything I know about the theory and practice of neural networks, starting with CS231n till now. I love how you explain things starting with simple examples to build intuitions (template matching for CV, bigram/table look up for sequence modelling), and then build to state of the art. Your lessons have had a profound impact on my learning, and I can imagine there are 1000s of engineers out there just like me.
I rarely comment on videos, but I had to here I had to. Your in-depth high-quaity resources are something to talk about. You make very complicated topics easy and engaging, your provide the knowledge for anyone to learn these highly-regarded concepts. Fruthermore, you are truly advancing the general knowledge of the public by providing these powerful videos. I would just like to express my gratitude for your videos, and how they really are making a positive impact. Thank you for dedicating many hours of work to upload these videos.
I am an undergraduate student. This is the lost lecture that professors never touched upon but absolutely crucial, thank you!! I especially love how you start from the basics for so many notions, and I really learned a lot.
I think your teaching projects may be the most impactful ever existed. First CS231n the course that inspired a generation of students to pursue Deep Learning, then micrograd and nanogpt that gives us the power to recreate billion dollar research for basically free. Not to mention that many companies like tiny grad and suno were inspired by your projects. Thanks for sharing your knowledge in such a clean and elegant way!
These kinds of videos is what the world needs more. Long form content with real time thinking. You see the master at his craft in real time. So much to learn from this!
We have one of the best people on the planet to walk you through it step by step (I haven't seen it yet, but I believe so). I am eager to learn this and want to master it. Thank you, thank you, and thank you very much!
Many Thanks to Andrej for making this tutorial available to everyone! I have never seen a clearer explanation of a nn before stumbling upon this zero to hero series. This will help all the people articulate the inner workings of neural net and help people understand deeper concepts, that is hard to understand. Looking forward to learning more with Andrej!
Finally, finished watching such a long video. Thank Andrej for sharing so many details of your knowledge. Like your teaching style so much since Tesla AI day. You are the best AI teacher!
The longer your videos are, the better it is for humanity. I think you are such a wonderful person and providing this stuff for everyone for free, can't thank you enough.
Tieto videá sú naozaj skvelé. Snažil som sa hlbšie pochopiť ako fungujú LLM čítaním vedeckých publikácií. Keďže som vyštudoval odbor softvérové inžinierstvo a nie umelá inteligencia a tiež som neabsolvoval postgraduálne štúdium, “papers” sa mi, kvôli medzerám v mojich znalostiach, čítajú veľmi ťažko. Keď mi ukazuješ kód, krok za krokom, všetko do dnes zapadá a dáva mi to zmyslel. Asi nebudem na LambdaLabs node robiť pre-training vlastného modelu, ale ten pocit, že s tým čo si ma tu naučil by som to teoreticky dokázal je neskutočne silný. Si perfektný učiteľ. Ďakujem ti, Andrej!
What an amazing video! Inspiring and so learningful. You (together with Jeremy Howard and Andrew Ng) are true gems for the AI community and master educators! Please keep the great videos coming and more than happy to pay!
Haven't been this excited about a RUclips video since makemore! Your videos are like an antidepressant. Such a joy to watch and follow and completely send contained. It's like having Mozart explain his art note by note
I wonder how long will it take for even GPT4 to be trainable on our own rigs. Nvidia said that in 8 years, computation will be reduced 350x times. (i.e. a gpu you will buy in 8 years, will be 350x better at training, if I understood correctly). Though is this enough? And 8 years is a loooong time in the present AI world.
@@Katatonya I think it's going to happen even faster. with the current rate of advancements, in 8 years training something like GPT-4 locally would be trivial to say the least. but i'm just guessing. who knows what will happen.
@@Person-hb3dv I think it purely depends on if the models will get much much more efficient and cheaper to run. They most likely will. Hardware-wise though, Nvidia didn't guess what will happen in 8 years, they know for sure as they have to plan in advance.
Thanks to your previous videos, I watched this one at 1.5x speed. During the nearly 3-hour runtime, I found myself clapping alone in my room and even crying because of how amazing this content is! We are so lucky to have your next-level expertise captured in a RUclips video!
Hi Andrej, just wanted to thank you. You are a truly inspiration. Thanks for all the effort you put in this videos and all the tremendous value they offer when being publicly spread
The best part of Dr. Karpathy’s videos is that he explains “WHY” than just “HOW”. Moreover, he has the humility to say “I don’t know why” or “This is too long to read” (as rare as they are). I’m curious why you don’t use “assert” statements? The #1 thing I learnt from my mentor is the use of assert statements (makes me more cocky and confident) 😁🤣
These videos are the best resource on modern neural networks I’ve found. Based on earlier videos, I built my own GPT with PyTorch. Now I’m doing a bunch of big projects based on what I’ve learnt. Thank you!
Hi Andrej, your content is incredibly inspiring and motivating. Watching you build something as complex as GPT-2 from scratch pushes me to improve my own tutorials, even if they're on simpler topics. Thanks for sharing your knowledge and for being such a positive influence on the community. Keep up the amazing work!
This is an unbelievably high quality lecture! I always learn a ton of new things from Andrej Karpathy. Actually I believe if I have to rank the amount of knowledge I learned from a single person, Andrej would easily rank as #1.
To anyone looking for the possible cause of the error in the parameter count: It is due to the vocabulary size. In GPT-1 it was around 40000, whereas in GPT-2 the vocab_size is around 50000. So, with 40K we will get 117M and with 50K we will get 124M.
Truly amazing. Many thanks for the generosity with which you share your deep knowledge. I personally struggle following the code with its many details and idiosyncracies, but the "high-level intuition" comes across perfectly and it's deeply satisfying to get a glimpse of the nature of a technology that seems "indistinguishable from magic". Bravo Andrej.
Thank you ser, this is priceless. Felt sad that it had to end at some point. Please cover more topics like mech interp, fine-tuning, mixture models, etc.
I thoroughly appreciate how you continue to give back to the community. This helps raise the water level for everyone on the way to building mastery! Thank you!
We are living in great times, where geniuses like Karpathy offers their invaluable knowledge for free, and people are rewarding him with the sum of money they can afford 🎉
Thanks for this incredible resource. For anyone wondering about the transposes in the parameter copying from HF GPT2 model to implemented one. HF model uses nn.Conv1d for qkv projection while Andrej uses nn.Linear. The weights dimensions in Conv1d are transposed. Hence, we need to transpose some of these weights before copying them over to Andrej's model.
The spelled-out intro to neural networks and backpropagation: building micrograd -> Iron Man The spelled-out intro to language modeling: building makemore -> The Avengers Building makemore Part 2: MLP -> Avengers: Age of Ultron Building makemore Part 3: Activations & Gradients, BatchNorm -> Captain America: Civil War Building makemore Part 4: Becoming a Backprop Ninja -> Doctor Strange Building makemore Part 5: Building a WaveNet -> Guardians of the Galaxy Let's build GPT: from scratch, in code, spelled out -> Thor: Ragnarok State of GPT | BRK216HFS -> Avengers: Infinity War Let's build the GPT Tokenizer -> Ant-Man and the Wasp Let's reproduce GPT-2 (124M) -> Avengers: Endgame Long movies, series of consecutive movies, requires multiple viewings to grasp all the details. Thank you for enriching my weekend.👏
Thanks for the video. I like your intuition for weight decay. Since the decay is proportional to the value, it encourages the contributions to the residual stream to be spread over more neurons.
you are such an amazing teacher... it took me quite a while to acquire all the knowledge you communicated so concisely and understandably in this video from other sources
you're probably the best teacher for ML I've had, none of my professors had your level of clarity, or ability to express concepts with simplicity, I'm still watching your cs231n course. One day when I make it as a successful guy in ML/DL/AI I'd love to have an opportunity to meet you. thanks man.
Andrej I cannot thank you enough for these videos. Your ability to explain deep learning concepts in a simple manner is unparalleled. I’ve always been hugely interested in ML since my early teens. Now I’m currently doing my Masters project comparing Transformers to Mamba and xLSTMs and doing a PhD in deep learning next year. I’ve been following your work since I first heard about PixelCNN++ and have been inspired ever since. Keep up the great work!
Thank you for doing this Andrej❤ As an undergraduate student I really appreciate having access to such incredible contents. Best of luck to your startup! Also looking forward to some computer vision related videos.
This is one of the best overviews I've seen not just on LLMs, but on the entire Deep Learning process. Thank you for going into so much detail, you're expertise really shows through your explanations. Would I watch another 4 hour video from you? Absolutely, any day!
![The moment we stopped understanding AI [AlexNet]](http://i.ytimg.com/vi/UZDiGooFs54/mqdefault.jpg)
![[1hr Talk] Intro to Large Language Models](/img/1.gif)
It’s rare to find such high-quality, free resources that make complex topics accessible and engaging!
i think you will find out that some people know that they wil be taking this technology away from the public ....... same as google glass .... as soon as america find out they cannot sread dissemination of knowledge they will cuts all internet cables !
Jesus - that's some tip!
❤
@@Handlebrake2 :)) Jesus - That's some 4hrs of brilliant content!
Dude this guy's net worth is $50M, you could've bought yourself 10 burritos at chipotle.
Do not ever look at how long your videos are. Your content is perfect and you should keep explaining things step by step. You are doing a great job. I believe you will be remembered in history as one of the pillars of AI.
I think one cool thing with videos like this is once Google implements their AI into youtube anyone will be able to watch this and just start asking questions. I've been learning a lot by watching videos like this and just copying parts of the transcript into ChatGPT to ask questions when I dont understand something.
I was excited by how long it was instead 😂😂
@@pin65371 jesus christ what a thought...and Andrej just starts talking back to you, answering your exact questions...
@@pin65371 What other videos do you recommend that have helped you?
Yes a great sharer of the important information and implementation. As well as to let you know it's in your hands to make your own before the internet is closed down or restricted in your areas or your cable gets cut !!
Great work ❤
My life is simple;
Andrej drops GPT-2 The Movie, I watch.
"GPT-2 The Movie" 😅
The movie and the sequel. I had to force myself to stop watching after I realized an hour had passed.
@@AndrejKarpathy The sequel "GPT The Movie" will be a old Hong Kong style "martial arts" movie about GPT getting beaten up by the Loss Function, then entering his training phase with Gradient Descent Sensei and the final showdown vs the big evaluation boss.
You know you gotta bring the popcorn
Andrej poster video on your biography@@AndrejKarpathy
The fact this video is free is incredible.
Sorry , i love your videos and what you doing for me. I couldn't attend Stanford or get into openai but learning from you is blessing to me.. i would pay you back 100times in coming years. And i was watching your git repository last two months, i could see many git code push in private ,but i was confused what he is working on.. this is he was working on. To provide quality pratical knowledge to us all on youtube.
Hey, the webside for the GPT2 is down, is there anyway to dowload it ?
Sorry i will have look up to it ..i will do today night will reply to you after that @@shutup1209
hf@@shutup1209
@@shutup1209 you should follow the video and make it from scratch! 😀
Those are generous tips :). I wanna learn basics so I can understand this first.
Thanks for spreading the knowledge! Happy to see a 4hr workout session 😅
:) Workout 🏋️♂️ is very much the right way to think about it imo!
Heavy weight training
more like a 3-day session to me, but I'm happy :)
Andrej is doing himself what OpenAi was supposed to do in the early days - make AI open. Thank you, Andrej!
Thanks Andrej! You have taught me everything I know about the theory and practice of neural networks, starting with CS231n till now. I love how you explain things starting with simple examples to build intuitions (template matching for CV, bigram/table look up for sequence modelling), and then build to state of the art. Your lessons have had a profound impact on my learning, and I can imagine there are 1000s of engineers out there just like me.
You're a legend! Andrej will light up when he see's this 🤍
you're too thankful ty! :)
Thanks!
Thanks! 4 hours of decoding a "Decoder-Transformer", Kudos and appreciate your existence in this field.
My anterior mid-cingulate cortex is getting bigger just watching this video because it’s hard! Thank you for your lessons, master Kaparthy.
lmao you david goggins gagger
Even the legend, himself, reads the generations of a half-trained LLM to find the giggle-worthy lines!
You're awesome. Thank you for this!
It's such a privilege to watch high-quality content from leading experts for free
true
I rarely comment on videos, but I had to here I had to. Your in-depth high-quaity resources are something to talk about. You make very complicated topics easy and engaging, your provide the knowledge for anyone to learn these highly-regarded concepts. Fruthermore, you are truly advancing the general knowledge of the public by providing these powerful videos. I would just like to express my gratitude for your videos, and how they really are making a positive impact. Thank you for dedicating many hours of work to upload these videos.
Thank you Andrej! from zero to hero boosted my professional career!
You are the Excalibur of cutting through the hype. Thank you so much. Your ethics are inspiring, and your educational materials priceless.
This guy is "the one" in the industry, who has helped me understand the LLMs. I respectfully love this man. Hats off.
Thanks AK, appreciate you sharing your knowledge with the world!
I am an undergraduate student. This is the lost lecture that professors never touched upon but absolutely crucial, thank you!!
I especially love how you start from the basics for so many notions, and I really learned a lot.
Which year in you are and which country
@@ppyogesh7394 I am at the University of Toronto, going to the third year this September
Perfect to watch while on a plane
and then running gpt 2 on planes computer
I think your teaching projects may be the most impactful ever existed. First CS231n the course that inspired a generation of students to pursue Deep Learning, then micrograd and nanogpt that gives us the power to recreate billion dollar research for basically free. Not to mention that many companies like tiny grad and suno were inspired by your projects. Thanks for sharing your knowledge in such a clean and elegant way!
I like when you add comments/metaphors about your intuition for how and why it works. Thanks you.
Checkout this man here, he got the best Ai news
Hello Andrej, thank you so much for the sharing and effort! Really appreciate it!
These kinds of videos is what the world needs more. Long form content with real time thinking. You see the master at his craft in real time. So much to learn from this!
We have one of the best people on the planet to walk you through it step by step (I haven't seen it yet, but I believe so). I am eager to learn this and want to master it. Thank you, thank you, and thank you very much!
The intellectual generosity of this man is of the highest standard.
I've learned a lot from your Neural Network video playlist. Thank you
Your guidance is inspiring.
Many Thanks to Andrej for making this tutorial available to everyone! I have never seen a clearer explanation of a nn before stumbling upon this zero to hero series. This will help all the people articulate the inner workings of neural net and help people understand deeper concepts, that is hard to understand. Looking forward to learning more with Andrej!
Finally, finished watching such a long video. Thank Andrej for sharing so many details of your knowledge. Like your teaching style so much since Tesla AI day. You are the best AI teacher!
Thanks for your Zero-to-hero series!
wow you're very thankful ty! :)
The longer your videos are, the better it is for humanity. I think you are such a wonderful person and providing this stuff for everyone for free, can't thank you enough.
Simply the best deep learning and LLM series online! Please keep making more videos and I'd love to be part of the journey!
🤗What an absolutely fantastic explanation! Every minute is filled with nuggets of deep insights!
Thanks! Really appreciate the effort you put into making these high quality educational videos!
Tieto videá sú naozaj skvelé. Snažil som sa hlbšie pochopiť ako fungujú LLM čítaním vedeckých publikácií. Keďže som vyštudoval odbor softvérové inžinierstvo a nie umelá inteligencia a tiež som neabsolvoval postgraduálne štúdium, “papers” sa mi, kvôli medzerám v mojich znalostiach, čítajú veľmi ťažko. Keď mi ukazuješ kód, krok za krokom, všetko do dnes zapadá a dáva mi to zmyslel. Asi nebudem na LambdaLabs node robiť pre-training vlastného modelu, ale ten pocit, že s tým čo si ma tu naučil by som to teoreticky dokázal je neskutočne silný. Si perfektný učiteľ. Ďakujem ti, Andrej!
super :) skvele je to pocut a dakujem!
Thanks very much Andrej! Your videos are real gifts to the whole world.
Thank you Andre for all these amazing in-depth, high quality tutorials!!!
What an amazing video! Inspiring and so learningful. You (together with Jeremy Howard and Andrew Ng) are true gems for the AI community and master educators! Please keep the great videos coming and more than happy to pay!
Haven't been this excited about a RUclips video since makemore! Your videos are like an antidepressant. Such a joy to watch and follow and completely send contained. It's like having Mozart explain his art note by note
Andrej in a few years: Lets reproduce GPT-5 (124T)
we can only hope
and it's gonna cost 3$ to reproduce
I wonder how long will it take for even GPT4 to be trainable on our own rigs. Nvidia said that in 8 years, computation will be reduced 350x times. (i.e. a gpu you will buy in 8 years, will be 350x better at training, if I understood correctly). Though is this enough? And 8 years is a loooong time in the present AI world.
@@Katatonya I think it's going to happen even faster. with the current rate of advancements, in 8 years training something like GPT-4 locally would be trivial to say the least. but i'm just guessing. who knows what will happen.
@@Person-hb3dv I think it purely depends on if the models will get much much more efficient and cheaper to run. They most likely will. Hardware-wise though, Nvidia didn't guess what will happen in 8 years, they know for sure as they have to plan in advance.
Thanks to your previous videos, I watched this one at 1.5x speed.
During the nearly 3-hour runtime, I found myself clapping alone in my room and even crying because of how amazing this content is!
We are so lucky to have your next-level expertise captured in a RUclips video!
Hi Andrej, just wanted to thank you. You are a truly inspiration. Thanks for all the effort you put in this videos and all the tremendous value they offer when being publicly spread
one of the best teachers I ever had
Super useful! Looking forward to any in-depth "anatomy" video on Mamba Architecture as well.
Yeah, mamba would be interesting to see... I hope Andrej hears us
The best part of Dr. Karpathy’s videos is that he explains “WHY” than just “HOW”. Moreover, he has the humility to say “I don’t know why” or “This is too long to read” (as rare as they are).
I’m curious why you don’t use “assert” statements? The #1 thing I learnt from my mentor is the use of assert statements (makes me more cocky and confident) 😁🤣
Seeing an Andrej upload in my recommended is like a mini holiday.
These videos are the best resource on modern neural networks I’ve found. Based on earlier videos, I built my own GPT with PyTorch. Now I’m doing a bunch of big projects based on what I’ve learnt. Thank you!
That’s cool how Andrej explains right after releasing code
Hi Andrej, your content is incredibly inspiring and motivating. Watching you build something as complex as GPT-2 from scratch pushes me to improve my own tutorials, even if they're on simpler topics. Thanks for sharing your knowledge and for being such a positive influence on the community. Keep up the amazing work!
You are doing great job and teaching which many top 50 universities combined couldn't manage in years
This is the best learning resource on language models bar none.
Thanks Andrej!
This is an unbelievably high quality lecture! I always learn a ton of new things from Andrej Karpathy. Actually I believe if I have to rank the amount of knowledge I learned from a single person, Andrej would easily rank as #1.
To anyone looking for the possible cause of the error in the parameter count: It is due to the vocabulary size. In GPT-1 it was around 40000, whereas in GPT-2 the vocab_size is around 50000. So, with 40K we will get 117M and with 50K we will get 124M.
Truly amazing. Many thanks for the generosity with which you share your deep knowledge. I personally struggle following the code with its many details and idiosyncracies, but the "high-level intuition" comes across perfectly and it's deeply satisfying to get a glimpse of the nature of a technology that seems "indistinguishable from magic". Bravo Andrej.
Thanks for all these videos Andrej!
Thank you ser, this is priceless. Felt sad that it had to end at some point. Please cover more topics like mech interp, fine-tuning, mixture models, etc.
Thanks Andrej! Tons of stuff in the video explained in simple terms, I learned a lot from it.
Thanks You Andrej!
I thoroughly appreciate how you continue to give back to the community. This helps raise the water level for everyone on the way to building mastery! Thank you!
I found GPT4 Here!
We are living in great times, where geniuses like Karpathy offers their invaluable knowledge for free, and people are rewarding him with the sum of money they can afford 🎉
This video is absolutely brilliant! Thank you so much Andrej for taking the time to share your knowledge with us!
You are the reason I got interested in neural networks, thank you for being a great teacher.
Hii bro, can i get roadmap for ML an deep learning please
Thanks for this incredible resource.
For anyone wondering about the transposes in the parameter copying from HF GPT2 model to implemented one.
HF model uses nn.Conv1d for qkv projection while Andrej uses nn.Linear. The weights dimensions in Conv1d are transposed. Hence, we need to transpose some of these weights before copying them over to Andrej's model.
if i know everthing taught in that tutorial with details and i am also able to apply them by myself, can i count myself as an advanced AI developer?
Currently, I'm just watching your videos. They makes me calm and happy. Hope to continue studying later this year.
4hrs while being 4k quality is chef kiss
you see 4k quality option? I don't see it :(
Andrej you are doing so much for the community! Really want to express my gratitude here.
Thanks for sharing your knowledge!
The spelled-out intro to neural networks and backpropagation: building micrograd -> Iron Man
The spelled-out intro to language modeling: building makemore -> The Avengers
Building makemore Part 2: MLP -> Avengers: Age of Ultron
Building makemore Part 3: Activations & Gradients, BatchNorm -> Captain America: Civil War
Building makemore Part 4: Becoming a Backprop Ninja -> Doctor Strange
Building makemore Part 5: Building a WaveNet -> Guardians of the Galaxy
Let's build GPT: from scratch, in code, spelled out -> Thor: Ragnarok
State of GPT | BRK216HFS -> Avengers: Infinity War
Let's build the GPT Tokenizer -> Ant-Man and the Wasp
Let's reproduce GPT-2 (124M) -> Avengers: Endgame
Long movies, series of consecutive movies, requires multiple viewings to grasp all the details. Thank you for enriching my weekend.👏
Thanks for the video. I like your intuition for weight decay. Since the decay is proportional to the value, it encourages the contributions to the residual stream to be spread over more neurons.
Thank you for contributing to Open Source models! And not just open weights!
you are such an amazing teacher... it took me quite a while to acquire all the knowledge you communicated so concisely and understandably in this video from other sources
Another piece of art 🔥
Another epic tutorial to build models from scratch. Thank you, Andrey!
Having to work when you just want to watch Andrej's videos is like being invited to an open buffet but you're on a diet :)
Andrej explains everything so clear, so logical and knows the "why" to every choice. You can't find a better tutorial than this. Brillant video!
That's crazy
I've been waiting for a new video of yours for a while so when I see this I'm just excited
I'll learn again a lot 😁
YOU are Awesome, Andrej!! 🥂🤖
you're probably the best teacher for ML I've had, none of my professors had your level of clarity, or ability to express concepts with simplicity, I'm still watching your cs231n course. One day when I make it as a successful guy in ML/DL/AI I'd love to have an opportunity to meet you. thanks man.
Better trilogy than lord of the rings
I was about to implement the GPT2 starting today but had no idea where to start. This video is made for me. Thank you so much 🙌
OMG a 4hr movie!
I can't believe we have access to this content for free from one of the brightest minds in the field! What a privilege; thanks, Andrej.
the GOAT has returned!
thank you for yet another amazing video Andrej!
Love your content, thanks!
Andrej I cannot thank you enough for these videos. Your ability to explain deep learning concepts in a simple manner is unparalleled. I’ve always been hugely interested in ML since my early teens. Now I’m currently doing my Masters project comparing Transformers to Mamba and xLSTMs and doing a PhD in deep learning next year. I’ve been following your work since I first heard about PixelCNN++ and have been inspired ever since. Keep up the great work!
Yayyyy thank you Andrej
Kind remainder that GOOD videos are NEVER too long
So thrilled to have you making stuff on here. It's the best version of what RUclips can be.
Huge respect for Andrej🤗. Sharing knowledge for free is incredible.
your explanation is really amazing, please keep fulfilling the gap >>>> build things from scratch
This is a really great video of explaining training GPT-2 from scratch. I learned a lot from it. Thanks a lot!
Thank you for doing this Andrej❤ As an undergraduate student I really appreciate having access to such incredible contents. Best of luck to your startup! Also looking forward to some computer vision related videos.
I've never seen and experienced like you teaching me making me think i can learn everything with your teaching
Thanks a million, Andrej. This is what you meant by "being the 'Coral Reef' for the ecosystem"-!!
Huge Love from South Korea.
Hi Andrej, what's the difference between this one and your "Let's build GPT" video? Which one should one learn first/which one is preferred?
Was wondering the same 😅
Use the "lets build" first, then this one; it goes over the understanding of it first, like the tokenization one as well.
This is one of the best overviews I've seen not just on LLMs, but on the entire Deep Learning process. Thank you for going into so much detail, you're expertise really shows through your explanations.
Would I watch another 4 hour video from you? Absolutely, any day!