How to self-host and hyperscale AI with Nvidia NIM
HTML-код
- Опубликовано: 8 июл 2024
- Try out Nvidia NIM in the free playground nvda.ws/4avifod
Learn how to build a futuristic workforce of AI agents, then self-host and scale them for any workload thanks to Nvidia NIM - a platform that containerizes AI models and runs them on any GPU with Kubernetes.
#programming #ai #nvidia
💬 Chat with Me on Discord
/ discord
🔗 Resources
Nvidia NIM nvda.ws/4avifod
CUDA in 100 Seconds • Nvidia CUDA in 100 Sec...
Docker 101 • 100+ Docker Concepts y...
🔥 Get More Content - Upgrade to PRO
Upgrade at fireship.io/pro
Use code YT25 for 25% off PRO access
🎨 My Editor Settings
- Atom One Dark
- vscode-icons
- Fira Code Font
🔖 Topics Covered
- How to scale AI models
- How to self-host an AI model
- Nvidia H100 GPU
- Will AI replace humans in the workforce? 
Наука
![The moment we stopped understanding AI [AlexNet]](http://i.ytimg.com/vi/UZDiGooFs54/mqdefault.jpg)
Try all the NIMs for free right now in the playground nvda.ws/4avifod
I like your magic world magic ( ai code) men
I didn't even watch the video cause I was engaging with the sponsor so much
NO, nvidia is evil
is this a ad?
no thanks
And here we are, Nvidia trying its best to take the whole monopoly doing it Nord VPN style
All thanks to CloseAI and other big tech using their GPUs for training dumb AI models that can't even solve a simple mental ability problem, like really, I asked GPT-4o to solve a simple class 9 mental ability problem and it failed, even with options it failed and kept giving the wrong calculations, thus approved that AI is dumb, and non tech people are dumber. Humans, am I right?
At the end of the day it's all about the money: Even for the loser behind fireship.
@@jurycould4275 so? you want him to work for free? lol
@@jurycould4275 what other thing can equip you with access to resources for staying alive other than money?
What is Nord VPN style?
Day 0 for not talking about ai
Such day would in most cases would never come , ai has fascinated a large amount of non coders too and these non coders have become a large part of viewership of such videos even though they basically understand nothing about coding , so channels now have started to talk on ai more and more and they gain more views and they are not wrong who doesn't want more views and more money to buy rtx 6090 , thanks to listening to my yap session which wasted 34 seconds of your life and if you are still reading it wasted 38 secs of your life
I listened to a cutie for 38 seconds
gotta sell those shovels
@@AwindowBest use of 38 sec 🙌
@@AwindowYeah as if that 44 sec of me writting this reply and reading your comment will make any difference. Lol know your obsoleteness you pleasent bot.
Fireship just casually switched from Web Dev channel to AI channel
I think his account is now being run by a sentient AI
Fr
Yea i lowkey miss the frontend stuff, but the “100 second of X” gave him the platform to be knowledgeable on all things CS. I loved “The Code Report”, its just all there is to report on these days is AI, despite the fact im sure meaningful computer science is still being done in other subfields
Remember when this channel was called Angular Firebase?
market forces
Now it's an informerrcial channel
sponsored by nvidia? this man both made it and lost his soul
IKR. How is he supposed to give us a good review when he is literally being paid by them? I know this channel is more of a news channel, but still kinda sucks
Guess what? Everything he discussed and showcased in this video still applies to FOSS frameworks like pytorch-rocm.
Get that bag, Jeff.
@@hardikgupta8496When did Fireship ever do reviews of Nvidia products?
@@hardikgupta8496 This is not meant to be a review, he's just explaining how it works. Where did he say he was doing a review?
As does everyone, thats how you actually get paid, this is a LIVING like anything else. Thats kind of the point of trying to become a big youtuber, especially a tech one where a majority probably know what ad block is. He isn't giving you news out of the kindness of his heart cause he is your older brother. He is growing his channel for the sake of getting more deals and sponsors. No one tries to grow their user base to millions as a hobby to continue working a 9-5.
Pretty sure the first words out of AGI will be "Kubernetes? Are you shitting me?"
it'll be a real giveaway when the audio synthesis engine outputs "K 8 S are you shitting me?" in a sexy flirty voice
I'm pretty sure it will be something like "Guys, we need to get rid of that JavaScript garbage."
@@DdDd-pl3nt*immediately rewrites self in holy C*
😂😂😂😂😂
I still cant belive that Nvidia somehow became the wealthiest company in the world.
Shovels
hopefully they are wealthy enough to provide good linux gpu drivers :(
Can't wait to see their reaction once the bubble pops
Isn't it obvious? The somehow is by selling the product that you need hundreds or thousands of in order to run an AI efficiently at scale...
@VoidHuskie
Didn't people used to say that about Ryzen ten years ago?
Fireship is big now, it's even doing 6 minute ads.
Selling shovels never looked this lucrative!
Did NVIDIA pay you per mention of the "H100"?
Probably not, but probably just required a minimum 🙂
Probably AI generated script/talking points. Most LLMs were trained on SEO derived text, so it outputs content like that a lot.
He should've just said it the entire video then. Just one word "H100" repeated over and over again.
@@darkspace5762 he doesnt get the check if the manager doesnt approve it
He said it already! The video is *SPONSORED*
It's ok guys, this is AI Jeff. The real Jeff is hanging out with Jensen on the beach
No better bandwagon to jump on than NVIDIA.
Ironically , your comment got stolen by AI , i just saw a bot posted it few minutes ago...
@@user-fr2jc8xb9g lmao, I thought you were joking. I had to look just to confirm...what a world lol
me 6 minutes ago: Oh a Fireship video great!
Me now: AI is a disease on this planet and must be expunged, also I just watched a 6 minute ad.
Another infiltrator from Dinosaur Enterprises, I see! 🦖🦕
Completely agree, it manages to have more fkn downsides than good sides for society
@@FriedRice3519 You can say that about most of modern technology though. The internet and its contribution to social media, computers leading to sedentary office jobs, smart phones, tv, video games, military and any mass destruction weapons (nukes come to mind), and the concept of modern finance, banking and money as a whole. We wouldn't have a capitalist system if society as a whole "mattered." Yet everyone is defending that system because they are trying to save their 9-5 jobs from AI instead of having a discussion how we should get politicians who are more open about reducing what the work week looks like thanks to AI. Things don't exist because it's "better for all of society", things exist because there are multiple ruling classes competing with each other for power and AI is the new tech that the ruling class is trying to capitalize on. It doesn't matter if its corporations, or government. OpenAI is funded by the United States DoD now and has a former member of the NSA on board while the Chinese government is pouring billions into AI this year alone. This wont go away and be "expunged" just because people don't like it. No more than the internet and smartphones will be "expunged" cause people are complaining how people these days are glued to their phones.
@@FriedRice3519You sure you're not just downplaying the very real possibility of AI technologies destabilizing the tech industry as we know it? Maybe the downside is that you need to learn more stuff lmao
Sponsored by Nvidia, good for you bro get that bag.
These days that the first key to the kingdom
How is he getting sponsored by Nvidia? He was sponsored by Docker last time?? What's next? He's going to be sponsored by Internet 💀💀💀
@@jiwachhetri7317 next is him sponsoring himself
@@Thewhiteandorange 😂😂
@@jiwachhetri7317 The European Union 💀
Guys, i think that AI killed Jeff and now runs this channel.
well DUH!
finally i can host my own custom ai girlfriend
And for 30k that's kind of a steal.
Blade runner style
@@RottenMuLoT real steal
The Nim (formerly nimrod) language team now bites their ass so hard that they didn't protect the acronym.
6 minutes ad...
Yes
yes
Yes
yes
we can make it a minute extra for getting excited to checkout the Nvidia crap and see what we are up against. Of course an extra 2 minutes for yelling at Jeff for doing this to us. How could he?
Finally nvidia bought fireship before NIM could replaced him
now nvidia is gonna eat all the search results for my favorite programming language
I would hope that the creators of Nim will sue the shit out of nvidia. Unfortunately I remember when Google stole the name “Go” from another programming language and got away with it :(
Hopefully
Welcome to C
That studering got me 😂😂😂
Discharge the old patient: Web dev
Admit the new old patient: AI
Good work getting a sponsor from them while throwing shade at the same time, it’s a fine line but you did it well . I would have blown it with prompts like “Nvidia.. generate me.. a bag”
adult site link bot stealing DarrenReidAu comment from 14 minutes ago.
Super cool, congrats on the sponsorship!
I have yet to see anything LLM related that will replace something like a Lawyer, a Programmer or a Doctor. At best they will be helpful tools to MAYBE boost their productivity in specific scenarios.
You haven't looked very hard. I'd list links but I'm off the clock
"[Most people] underestimate what they can do in ten years."
Fusion power startups: *Allow us to introduce ourselves*
I mean. It's definitely the funniest ad I've seen today. Well done!
So Nvidia now wants to compete with AWS and Azure? Are they sitting on their own inventory or what?
They manufacturer things. I would hope they have inventory.
@@DoctorMandible They design things
Host the NIM on AWS or Azure, no competition
Can't believe i just watched a 6 minute ad. Bravo Vincent!!
Your videos are works of art. Concepts, Wording, Jokes, Knowledge, Memes, just everything incredibly on point.
Fireship: what's the best js library?
LLaMA: React
Meta: 😅
fireship:
fireship AI channel :
my mand sold his soul to GPUs god.
The video was interesting but the comments about the sponsored content was fascinating too.
so basically It's nvidia's version of google's Vertex AI ?
Nvidia sponsoring you... Damn!!! Congratulations buddy
my man sold his soul for gpus
30k gpu = 6 minute video..
I mean... I wouldn't mind trading my kidney for an h100 👀
You know you like a channel, when you first like the video and then start watching.
And then unlike the video, when you realized what the video was about.
@@andrevvv Or liked it again because you want to not make the AI overlords angry.
I wonder how YT weighs like and dislikes at different quarters of the video... Or even more fine grained like first 15 seconds...
Well that's nothing to be proud of.
You are hysterical dude. I love the sarcastic dry humor.
finally a worthy oponent to run "hello world" in
Skip this video, unless you want to watch a 6:43 minute NVIDIA ad?
I do.
Calm down a bit, would ya
shovel sellers... gj on securing that bag Jeff 👏
You were the chosen one, destroy the sith not join them. You were my brother Jeff I loved you
The subtlety of the voice stutter to continue the joke that he is an AI is 🤌🏻
So when the kubernettes auto-scale, where do they auto-scale to? Do you need multiple servers of your own for the auto scaling to work? Do you rent servers from Nvidia for the auto scaling?
It seems like you can use it in the cloud or on your own hardware. In the cloud auto scaling will probably rival the biggest AWS surprise bills. On prem you'll probably need to have enough hardware for it to scale.
Its not meant for individuals, it is an enterprise solution. He is showing it because he got paid. You'll never be able to use this unless you like really expensive side hobbies.
What I am missing here is the context: I mean this is the tool and it can do this and that but how to integrate it in a real application? What about the customer's data running through these services hosted on other companies servers, will be data safe from exposure? Whould be nice a 100 seconds orientation tour :). thanks
"AGI is highly speculative and a dumb image of the future"
**proceeds to propose a slightly less speculative, slightly less dumb image of the future**
Humans will always be preferred over AI, I know this breaks your heart as a programmer, but that's just the truth of it. God made us, we made AI, it's a pretty simple concept to grasp.
one minute is criminal
Comedy with news as usual, solid gold.
man, i sure wish i wasn't so fucking slow.
i would love to be cutting edge.
you guys live in a fascinating world!
How much $ did they give you
at least $30k in tech parts
my bet is at least 100k
Can't blame him
I am amazed how all this lead to asking the model 2 questions and both answers were completely wrong! This sums up the situation with LLM's so far.
The only job that can be easily replaced by an AI is the CEO, but I wonder why we haven't seen that yet
Ai can't burn through money that fast yet
Commercial AI has too many safety checks and effectively too much of a good conscience to be a CEO.
tech CEOs are dumb idiots who have no idea how AI models work, and they are the only ones who now how to burn corporate money to rob humans and conduct big layoffs.
Thereby guaranteeing all incompetent ceo's, instead of merely 99% incompetence.
@@ibendover4817😂😂😂
that thrrii-thrrii-thrive is on point 🤣
This is the most sarcastic ad I've ever seen
This is better than some ads you've done. The real difference in non-sponsored videos is that the jokes are relentless and borderline NSFW. In sponsored content, it's like using your library-voice when at XMAS dinner with your partner's family. Thanks for all the videos!
I love this channel. This is the first video that feels like an ad and somewhat phoned in though.
I am happy for Jeff to get a paycheck though, he deserves it.
Just hope he will go back to his more funny and critical content.
Recently i "somehow" got access to H100 GPU, Nvidia is a sponsor of today video. 😁
Fireship getting more views than some JRE episodes is something
This seems pretty awesome tbh.
I love 7 minute ads from my tech "news" channel
This video did not make me laugh. Is this an Nvidia commercial channel now or was this just a one-off to pay the bills?
"Yes."
Let the man make money, and someone might actually find this useful
Let Jeff get that bag. Literally everything in this video can be implemented with completely open stacks. It's no different than SageMaker except that with NIM there are a few less middlemen between you and Jensen's jacket supplier.
@@GSBarlevthis is true of almost everything Amazon does. Yet they're filthy rich. Open source "tools" existing is NOT THE SAME as a PRODUCT.
Interesting, how much the sentiment is shared..! We all need bread.
Awesome tools! Is there a way to access the talking avatar showcased not to long ago at NVIDIA's annual conference?
I love the part where you followed FTC guidelines and clearly stated that this was an ad
Finally I can make a bot that plays my steam catalog for me
I'm just here to listen to the wizard speak his magical spells
bro u are inspiring people to take our jobs
We can’t escape Kubernetes can we?
Its built directly into docker. There could be alternatives to a service mesh level system like Kubernetes, but there are very few options. Most infact rely on kubernetes. No one wants to make all of those services from scratch.
Congratulations on the Nvidia spo!!!!!
The audio at 3:26 might be unintentional but it's a nice touch
The perfect shovel
The costs ???
would love to see your setup on system monitoring with Grafana
Happy Tuesday!
So NiM's is like AWS Lambda, but for AI specific use cases instead of general app services?
Nice, fella got a NVIDIA sponsorship going.
Grab for the stars.
I was just toying with codestral on ollama running in my local machine, very impressive to run a model locally
Today was death of a star
3:20 that sound effect... genius
Sounds interesting.
How's the performance with a "normal" GPU & large amount of system memory instead of a GPU you have to pay a kidney or two for?
This
2:42 pure gold
The gliche in his speech almost had me 😅
10 years from now:
“hi there my child got access to the keypad of my Apple house and I’m locked out for 13 days. Can you help me?”
“I can absolutely help you! First off try adding Elmer‘s glue to the keypad!”
The bag on this video must have been huge.
This channel amazed me with its content🎉
Jeff is my fav AI youtuber
Mozilla has a similarly cool technology called Llamafile which bundles cross-platform LLMs into a single portable package that exposes an OpenAI compatible API.
Can I use Nim to program NIM?
selling your soul to nvidia before being replaced by nvidia
Holy crap, Nvidia is very-very serious about making the pickaxes.
I can't wait to start working with my AI agent team:) I just need 15-20% incraese in quality of agent's output - at the moment accuracy is to low to scale it. In pharmaceutical industry we need 95-99% accuracy.
does it still count as a sponsored video if it's just a straight-up ad?
Good work getting a sponsor from them while throwing shade at the same time, it’s a fine line but you did it well 👏. I would have blown it with prompts like “Nvidia.. generate me.. a bag” 😂
So the adult site link bot stole your comment and got all the likes.
@@mc1993lol good catch, they must be running Nvidia NIM to steal content so quickly.. 🥁 🙄
What kind a customer will be for company which are operated by CEO and AI?
The S&P 500's 8 day at the money implied volatility it sounds super complex and fancy but basically all it represents is the relative price of options so when this is low does it means options are cheap? and when this is high does it means options are pretty expensive?
I’m pretty sure this is an AI voice now. Jeff finally automated himself completely. Sad days.
seems fun!
Did you mine on it? We need a video about bevy
That GPU is more than my net worth 👀
Sam IS the AGI...wake up sheeples
Got that backwards-the AGI is _Sam._ Every request sent to the ChatGPT API is answered by Alt-Man personally. He's the ultimate Mechanical Turk.
@@GSBarlev hey I like that chess reference. But really what was Sam thinking when he started working on CloseAI? Really tech is turning into a marketing industry and less of an innovation industry, I wonder what ever happened to innovation? Did he went on a long vacation or something? Or did Sammy boy stabbed him?
6:15 What is this code editor theme?
Ugly
you forgot to mention discreet computing limitation. either need to discretize down to the atomic level, or have analogue AI. good luck having people manage literal infinity