Its quiet interesting how efficient our brains are. I can make imaginary images in my head all day with the energy that can be stored in a burger or two.
Efficient doesn't equal smart, otherwise you'd know that a typical burger contains about 600wh of energy, which would be enough to charge about 10 laptops.
You are right. AI is in the hype. Like all other topic in history, it will soon go into an adjustment where people start to change their way of using it.
Very nice job. I have been been in the data center industry a while (maybe too long?!), and I like your approach to show and explain things. Very very well done. I share videos like this to new team members to help them wrap their head around what's going on.
That's good to know people in the field like the video. I find data centres fascinating are there any other videos or sources you'd recommend? Many thanks
Every Google search provides an AI overview even without requesting. Google AI states that 99,000 Google searches are executed per second i.e. 1800 rare steaks per seconds.
Something that would be helpful, maybe in a future video, is to explain what is physically happening when these data centers are at work. What are the physical requirements that result in the energy use and why is so much of it turned into waste heat?
I have been researching it all day, how exactly ai and these large language models work, and even as a guy who knows computers and tech pretty well, it's very complex. Something fundamental has changed drastically with software, coding, and the like, so much so with this a.i. stuff that it's completely different, like a whole new world has opened up with software capabilities and how it actually runs, it's not even coding and algorithms anymore like we used to think about it. It involves "training" by "scraping" so that people don't even tell the programs what to do anymore with code, it teaches it self and we just feed it data, this and so many other things it's all very mind blowing what they can do now. All I can liken it to is, we were at this level, and we just broke through it. A huge fundamental breakthrough happened. Or at least it seems.
@W4RZD i mean, it turns out it might be :) i'm aware of some weapon made by the US that strikes objects from a distance with a laser, but i really didn't know they could measure as well.. but seriously, why did you mention that exact period ?? were there any local wars or events where they've used such devices ?? here's what i knew about ruclips.net/video/eFiDYFnlp7s/video.html
2:56, your numbers are from running a local model, where you only really run 1 input at a time, because its only for 1 person. but in reality when in production, they pause for a set amount of time (say 5 seconds) to gather hundreds of inputs, and process all of them at the same time as a batch, which is FAR more efficient than doing one at a time.
It's surprising how much these data centers look like the computer rooms from the old mainframe days. Those machines were nowhere near as powerful but demanded just as much cooling. I go back all the way to the IBM punch card era and worked in one of those rooms with tape drives humming and all sorts of lights blinking. It feels like we've come full circle.
Interesting analogy and comparison. Think about this: the average SUV uses 1,685 watt-hours for every mile driven. That is equivalent to 5,000 prompts of the ChatGPT model
Great video - I get tired about being shamed for using electricity though. From my power company to some of the “Think Tanks” - electric power is a modern marvel to be enjoyed!
Great video, I think the solution to this problem is figuring out more efficient power generation. All the big tech companies are looking into nuclear. This could be the kick start our aging power grid needs
Aside from the obvious gains mentioned and "hotly" pursued on more efficient architectures and algorithms, the role of quantum computers in AI including much more advanced AI is likely to be huge. Definitely a space that needs disruption.
In a few decade, we will have smaller decides that can handle such volume of workload without needing thee facilities. Just as we had big mainframe computers that were stored in large houses to process information and now we have laptops that can handle the same work.
I think the AI tools should display the amount of energy consumed for each response (similar to how many flight bookings sites now show amount of CO2 emitted by a selected flight). That way, people would know the real cost of their prompts.
1:01 The electrical grid can't provide enough power to these data centers so they have to truck in shipping container sized power generators. These probably run off natural gas.
@SV-mc1jq Nope the NATURAL GAS generators are not just for backup. They are PEAKER generators to compensate for high cost periods when the grid has high load. This is all about reducing COST. $$$ is always the motivation. The Diesel Generators with sets of diesel tanks is for backup.
How much electricity is wasted on heat dissipation in traditional data centers? The pressure of environmental protection is too great. Broadwatt’s vertical design seems to have made a breakthrough in heat dissipation, and it also optimizes energy consumption together with Broad Group. It’s worth a try.
Would have been more informative to compare the energy use to other uses we are very familiar with (the time to charge a high end smartphone, the equivalent amount of energy in gallons of gas, etc.). Energy used in cooking is very unreliable because cooking equipment is wildly inefficient.
Just completely shot by the target, what would be interesting instead is to calculate how many prompts are made today, and then calculate the power usage for one day. What do I want to do with a steak when we’re talking about AI
I don't use AI-based suggestions on a daily basis, but companies like Google do AI-based summaries for my search queries without my asking, dozens of them every day. So they should be blamed, not the consumer, it's hard to imagine how many AI results are thrown in the trash, generated just to promote these hallucinations.
It's likely Google is caching SOME level of AI responses to search qureries or doing partial AI generation on top of existing cached answers to cut down on cost to themselves.
It's likely Google is caching SOME level of AI responses to search qureries or doing partial AI generation on top of existing cached answers to cut down on cost to themselves.
I feel like as a British vegetarian I might not have been the target audience for this! 😅 Great video though and refreshing to have insight from real experts in the field, rather than guys with baseball caps on podcasts.
great content but i feel this was a bit manipulative to focus on video generation. the majority of generative ai usage is text based, which by definition does not consume the same energy per prompt
NVidia tech is based on classical digital design of the past several decades, and inefficient. The chip design toolchain and the CAD tools have not made a leap other than able to handle lots and lots of digital logic gates, that are shrunk to operate at low voltage. We need a paradigm shift to make orders of magnitude progress from here.❤👍
Why not compare to hours with a HD TV on. Or how many times you can changer your iPhone Grilling sounds silly measure of electricity. Better maybe how many average washing machine cycle
compared to actually making a film physically the traditional way, AI generated movies will be so much more efficient, provided the right ideas are fed into it. Sooner or later some talented small team of creative geniuses are going to make a fully AI generated feature film and win an Oscar award.
So this it actually a pretty interesting video but it relate the title for like 35 secs ? Literally the 2nd and shorter chapter which in it self is a sooooooooooooooo much cut down version of the explanation. Nice
Great video as always! But the conclusion that people should think twice about the model they use is a bad take IMO. Reducing demand is not the path towards sustainable energy, increasing supply is. The question should be “how do we get these data centers on 100% clean energy and closed loop water systems in 3 years”…not “make fewer silly videos”
In "AI" I accept the "A", but not the "I". It is just extra large searches of existing stuff, and it is the "large" part that needs all the energy. we are are almost in a "post I" economy where most energy is wasted on completely useless outcomes.
Read more about how much energy AI prompts use here: bit.ly/4esbmrk
Its quiet interesting how efficient our brains are. I can make imaginary images in my head all day with the energy that can be stored in a burger or two.
Efficient doesn't equal smart, otherwise you'd know that a typical burger contains about 600wh of energy, which would be enough to charge about 10 laptops.
Ok, but can anyone else see the imaginary dancing steak images in your head? They can not, so the comparison is stupid.
@Djamonja I just did reading the comment
God can't be matched 🌈✝️🕊️👑🙏😇🇺🇸🦅🇮🇱🕊️
You are right. AI is in the hype. Like all other topic in history, it will soon go into an adjustment where people start to change their way of using it.
Can we file "how cooked is my steak" under "anything but metric"? 😁
Rest of the world:
Joules per second
USA:
Medium rare tenderloins per Real Housewives episode
do yuros have anything to do but write these comments? does each one help power the fan on the AC you don't have?
@frydegz Whole worlds using metric mate.
Joanna is really the perfect person to do these videos.
💯
@electrology she got that Aunt May energy
She is extremely cringe.
@Johndoe-hf5fb You, sir, have no sense of humor.
her voice is grating
Very nice job. I have been been in the data center industry a while (maybe too long?!), and I like your approach to show and explain things. Very very well done. I share videos like this to new team members to help them wrap their head around what's going on.
That's good to know people in the field like the video. I find data centres fascinating are there any other videos or sources you'd recommend? Many thanks
this video barely showed anything
Great journalist! 👍 Love her videos (like grilling Apple's execs 😂)
Joanna, you and your team are amazing. This is such a great work and something I have always wanted to learn about!
I used to cook eggs on my Pentium 66 megahertz processor. Oh how I miss the good old days.
Fascinating, and great way to present it !
Whatever this journalist does is really cool
*cringe
Super cool
Every Google search provides an AI overview even without requesting. Google AI states that 99,000 Google searches are executed per second i.e. 1800 rare steaks per seconds.
Thanks, Joanna for looking into this. Hopefully more usage stats on these data centers can be released in the future as well.
i live in ashburn. every other non residential building is a data center. its so ugly 😭
Would be cool if the chat apps could give you a running estimate of your usage for each conversation... in tokens or teraflops if not in WH.
it would also require energy to compute and store your running usage tally. bit of a conundrum.
I loved ❤this. Way to go WSG for this awesome reporting
Thanks Joanna..
Something that would be helpful, maybe in a future video, is to explain what is physically happening when these data centers are at work. What are the physical requirements that result in the energy use and why is so much of it turned into waste heat?
Yeah
I have been researching it all day, how exactly ai and these large language models work, and even as a guy who knows computers and tech pretty well, it's very complex. Something fundamental has changed drastically with software, coding, and the like, so much so with this a.i. stuff that it's completely different, like a whole new world has opened up with software capabilities and how it actually runs, it's not even coding and algorithms anymore like we used to think about it. It involves "training" by "scraping" so that people don't even tell the programs what to do anymore with code, it teaches it self and we just feed it data, this and so many other things it's all very mind blowing what they can do now.
All I can liken it to is, we were at this level, and we just broke through it. A huge fundamental breakthrough happened. Or at least it seems.
@RubbersPVP bcs ai isnt about coding but math
Kettles and toasters are the preferred kitchen comparison for power use as they're easy to comprehend and draw a round 1kW.
btw GPUs shut down at 90 degrees Celsius, not farenheit. So that's more like 200 F.
😮 didn't know there were such devices for measuring the temperature 5:28
@W4RZD i mean, it turns out it might be :) i'm aware of some weapon made by the US that strikes objects from a distance with a laser, but i really didn't know they could measure as well.. but seriously, why did you mention that exact period ?? were there any local wars or events where they've used such devices ??
here's what i knew about
ruclips.net/video/eFiDYFnlp7s/video.html
5:50 surely this guy is telling the truth in good faith
We need another model to tell us which model is the most efficient to use for each specific scenario.
2:56, your numbers are from running a local model, where you only really run 1 input at a time, because its only for 1 person.
but in reality when in production, they pause for a set amount of time (say 5 seconds) to gather hundreds of inputs, and process all of them at the same time as a batch, which is FAR more efficient than doing one at a time.
How much time to create 1000 video clips by human?
It's surprising how much these data centers look like the computer rooms from the old mainframe days. Those machines were nowhere near as powerful but demanded just as much cooling. I go back all the way to the IBM punch card era and worked in one of those rooms with tape drives humming and all sorts of lights blinking. It feels like we've come full circle.
Majority ai result are trash means increase in power losses
Nobody at the american media company remembered to season or marinate the steak
Interesting analogy and comparison. Think about this: the average SUV uses 1,685 watt-hours for every mile driven. That is equivalent to 5,000 prompts of the ChatGPT model
Agrees Joanna's dancing steak video was totally worth the kWhs . --------->
Honestly, I was surprised at how efficient AI generation is!
really neat
Now show us deepseek since its 5times smarter
You should have compared it to a normal query as well
Couldn't we just make them in places further up north like Iceland where it's colder?
This shows you just how efficient our brains are.
Do the AI on the inter-NETS
wow
Joanna rocks
Amen!
❤❤❤
Great video - I get tired about being shamed for using electricity though. From my power company to some of the “Think Tanks” - electric power is a modern marvel to be enjoyed!
Great video, I think the solution to this problem is figuring out more efficient power generation. All the big tech companies are looking into nuclear. This could be the kick start our aging power grid needs
Aside from the obvious gains mentioned and "hotly" pursued on more efficient architectures and algorithms, the role of quantum computers in AI including much more advanced AI is likely to be huge. Definitely a space that needs disruption.
This is awesome!, let's cook ourselves to let AI create a video of us cooking ourselves.
Dancing steak? Dinner and a show!
Makes tons of high quality videos. OMG they consume a lot of power.. *Shocked Pikachu face*
In a few decade, we will have smaller decides that can handle such volume of workload without needing thee facilities.
Just as we had big mainframe computers that were stored in large houses to process information and now we have laptops that can handle the same work.
Imagine if they put this in the Artic and Artic.
How much brain matter is lost with each AI usage is probably equally as important to discover as how much electric usage occurs. .
Cooling them is a science on its own
Joana❤😊
That's a lot 9f energy 😮
I think the AI tools should display the amount of energy consumed for each response (similar to how many flight bookings sites now show amount of CO2 emitted by a selected flight). That way, people would know the real cost of their prompts.
I’ve been in this game long enough to recognize the good setups. $BEL is one.
Bellarium is flying under the radar right now, but not for long.
5:28, dosen't he mean celsius?
1:01 The electrical grid can't provide enough power to these data centers so they have to truck in shipping container sized power generators. These probably run off natural gas.
Just love how corporations live beyond their means.
That's not true in this case. Yes the grid is reaching its limit with datacenters but those generators are for backup
@SV-mc1jq Nope the NATURAL GAS generators are not just for backup. They are PEAKER generators to compensate for high cost periods when the grid has high load. This is all about reducing COST. $$$ is always the motivation. The Diesel Generators with sets of diesel tanks is for backup.
Correct. This was exposed for the Tesla / xAI data center. They have a bunch of dirty natural gas generators
Well you got to save it not just make it .. I mean the video not the steak
These facilities can be built in cold regions but they wouldn’t talk about it
92 degrees F seems not that hot for the thermal limit
How much electricity is wasted on heat dissipation in traditional data centers? The pressure of environmental protection is too great. Broadwatt’s vertical design seems to have made a breakthrough in heat dissipation, and it also optimizes energy consumption together with Broad Group. It’s worth a try.
can it run cyberpunk in 8k?
Lol the equipment there can only get to 92F?
220 for the win
So asking ChatGBT to calculate PI to infinity is a bad idea.. got it.
There’s something special happening with Bellarium. Keep watching.
❤🔥❤🔥❤🔥❤🔥❤🔥❤🔥❤🔥❤🔥❤🔥❤🔥❤🔥
We need more and more energy..
90 degrees F or Celsius ?
What I learn - We have to be smart to use LLMs. Sadly people don’t think deep. It’s hard to do that. So more GPUs melting.. lol
We are powering intelligence over electricity that would eventually lead to more electricity
$BEL might be the most promising alt launching this month.
“Google, Microsoft and other Generative AI companies” while showing the Meta logo…oooh, the shade! 😂
A new Marvel supervillain will create a billion simultaneous prompts and cause a global meltdown.
Would have been more informative to compare the energy use to other uses we are very familiar with (the time to charge a high end smartphone, the equivalent amount of energy in gallons of gas, etc.). Energy used in cooking is very unreliable because cooking equipment is wildly inefficient.
Just completely shot by the target, what would be interesting instead is to calculate how many prompts are made today, and then calculate the power usage for one day. What do I want to do with a steak when we’re talking about AI
I don't use AI-based suggestions on a daily basis, but companies like Google do AI-based summaries for my search queries without my asking, dozens of them every day. So they should be blamed, not the consumer, it's hard to imagine how many AI results are thrown in the trash, generated just to promote these hallucinations.
@AB-wf8ekI will never get used to this. Keywords till I die
You are so woke
It's likely Google is caching SOME level of AI responses to search qureries or doing partial AI generation on top of existing cached answers to cut down on cost to themselves.
It's likely Google is caching SOME level of AI responses to search qureries or doing partial AI generation on top of existing cached answers to cut down on cost to themselves.
I feel like as a British vegetarian I might not have been the target audience for this! 😅 Great video though and refreshing to have insight from real experts in the field, rather than guys with baseball caps on podcasts.
It would actually make more sense to focus on all the resources it took to create that steak..
it would be more efficient for A.I to do teh task they were given if american ask it in Metric System
Fahrenheit?
Philippines
great content but i feel this was a bit manipulative to focus on video generation. the majority of generative ai usage is text based, which by definition does not consume the same energy per prompt
Although I appreciate the information, more specifics would have been nice.
she's gonna have her own TV show or something like that
We need more energy, fusion, battery, solar and everything
6:25 nope
90-92 F as max Temperature? I'd guess, it's rather 90-92 C. (That's at least what I run my computer on.)
NVidia tech is based on classical digital design of the past several decades, and inefficient. The chip design toolchain and the CAD tools have not made a leap other than able to handle lots and lots of digital logic gates, that are shrunk to operate at low voltage. We need a paradigm shift to make orders of magnitude progress from here.❤👍
Why not compare to hours with a HD TV on. Or how many times you can changer your iPhone
Grilling sounds silly measure of electricity. Better maybe how many average washing machine cycle
Next time, please use the metric system as well for measurements.
I built these
compared to actually making a film physically the traditional way, AI generated movies will be so much more efficient, provided the right ideas are fed into it.
Sooner or later some talented small team of creative geniuses are going to make a fully AI generated feature film and win an Oscar award.
So this it actually a pretty interesting video but it relate the title for like 35 secs ? Literally the 2nd and shorter chapter which in it self is a sooooooooooooooo much cut down version of the explanation. Nice
Great video as always! But the conclusion that people should think twice about the model they use is a bad take IMO. Reducing demand is not the path towards sustainable energy, increasing supply is. The question should be “how do we get these data centers on 100% clean energy and closed loop water systems in 3 years”…not “make fewer silly videos”
Too hot for TV 😂
BELLARIUM!!!
Evaporation is partial, not total loss
In "AI" I accept the "A", but not the "I". It is just extra large searches of existing stuff, and it is the "large" part that needs all the energy.
we are are almost in a "post I" economy where most energy is wasted on completely useless outcomes.
Well, yeah, adapt get’r done.
In the future, energy will be currency