deju vu. this feels like the FUD with crypto mining since 2017-18 or the Intel Pentium 4 FUD with skyrocketing heat from CPUs. I wouldn't be surprised in a few years that writers will be thanking the AI boom for better energy solutions due to steadily growing number of EVs.
That is very true, but it's easy to not recognize how much information ChatGPT knows. For example... my brain is energy efficient, but I just used ChatGPt to create a python script with 100s of lines. I didn't have to learn the syntax (this is my first python script). I didn't have to look up and learn about different modules that it uses. To be honest, it probably saved me 20 to 40 hours and was completed and tested in about 1 hour. I'm not convinced it's less efficient.
@@TwoBitDaVinci So half joking...is the next step in computing an organic computer or some kind of human brain farm (like in the Matrix but for harvesting computational power instead of heat energy)? Lol, it would at least make for an interesting plot for a scifi movie.
It seems like my comment got deleted... I just used a LLM to help me to program a python script. It's up to 3500 lines of code, but more than half was copy and paste from the LLM. Before this, I have never used Python and i didn't have to get a book and learn the syntax, etc. I estimate that it's saved me 80% of the time it would take for me to write it from scratch. Yes it takes power and yes I can write a 10 page essay, but a LLM can write it better and in 1 min. Brains take less power, but they I'm not convinced they are more energy efficient (energy consumed per output unit)
@mccue2439 When talking about energy efficiency, speed is not what we're looking at. What is amazing about the brain is that it could write that script at the energy cost of a cup of coffee.
Happy to hear that analog computation is finding new life. Sixty years ago I had the pleasure of programming the Electronic Associated PACE Model 231R, the last of the large analog computers. It regularly calculated results faster than our department's digital computers. Then we installed a Philco 2000 with 10k core memory! Analog still won the timed races, but lost out on precision with "only" four-digit resolution.
@@TwoBitDaVinci Sure. Residing San Diego now, but relocating to Phoenix in August. BTW, imagine the power requirements if AI only had the vacuum tubes we relied on!
I live in the area where the Cedar Rapids data center is to be built. The local nuclear power plant has reached its end of life and is now being replaced by 3,500 acres of solar panels on prime farmland. This video just put the pieces together for me as to why. I can see a lot more expansion of these panels to more prime farmland. Looks like this would have been better served in a location where less productive land is abundant.
40% of the corn grown in the country is for Ethanol. PV solar produces 100 times more energy per acre than corn ethanol. In addition, many are also using the land with PV solar for specialized agriculture, like sheep grazing and part shade crops.
Sorry but commercial nuclear power is not the answer. In part due to its ultra-expensive cost and ultra-long build times, commercial nuclear power will be gone from the grid in about 50 years. Not just because of that but renewables are at historical low cost per MWh generation (and still falling) while nuclear keeps getting more expensive. Furthermore, "firm renewables" at scale are just around the corner with massive government, scientific and industrial interest in grid-scale storage technologies. When firm renewables arrive at scale, all other energy production methods like fossil fuel and nuclear are done on the grid. Nuclear will still be used in small-scale scenarios like military, scientific and deep-space exploration. Commercial nuclear is just the wrong tool for the job on the grid. Commercial nuclear power's own history bears that out with it powering only 10% of the world's grids after a long 70-years of history and its percentage is falling. Nuclear failed us from its promises of abundant energy "too cheap to meter" promised in the 1950s. If nuclear had come close its promises, we wouldn't be so far down the climate crisis hole we're currently in. Nuclear failed us. At 99.9% the mass of the solar system, the sun provides a cosmic 173,000 terawatts to the Earth non-stop. Or about 1000W per square meter peak at the ground and is always peak somewhere on Earth. All the combined fossil and fissile material on Earth would amount to a bucket or two of water in an ocean compared to the daily amounts of energy provided to us from the free clean *fusion* power we get from the sky. Renewables harness this energy directly, and the raw inputs of sunlight and wind cannot be taxed, embargoed, blockaded or sanctioned. With respect to land, much less than 1% of the world's total land surface alone in current generation photovoltaics is enough to power all the world's grids. But why stop there? Imagine a full 1% or 2%? Solar can also produce power 'in situ' on-premises which greatly reduces load on the grid because rooftop solar "unloads" the grid. It helps that datacenters have massive rooftop and parking lot areas. Also, a lot of the biggest datacenters generally have a lot of open land around them as they're built outside of urban areas. A solar farm is actually more environmentally friendly than an actual agricultural farm. Current agricultural methods are ruinous to the environment with animal run-off, pesticide run-off, and soil erosion. So, it's not surprising that with a solar farm on it, the land would live a much more peaceful life than it ever did as an agricultural farm. New agricultural methods are needed. One example is using "agrivoltaics", one can reap two crops off the land: food and energy which humans need a constant supply of.
@@MichaelF350 Soybeans are a thing. Also, I am watching these solar panels go up just 3 miles from my home and they are not planning on raising animals under them-they are too low for that.
you bet, and thanks for being such an active voice in our videos! I see you and appreciate you! don't even know you're real first name, but thank you budgetAudiophile :) great name btw
District heating has higher infrastructure costs up front, but once the infrastructure is built there are lots of inputs. Even waste heat from nuclear plants can go into district heat grids.
@@TwoBitDaVinciplease do and then tell us why this isn’t happening everywhere. Thanks for your awesome videos and for being such a generally cool guy. Not at all surprised why Mrs. Llewellyn liked you so much the instant she saw one of your videos.
Hmmm--I wonder has people thought about putting up server farms in the "safe" areas of Iceland, server farms all powered by plentiful geothermal power.
Yes! Make a video on heat redistribution (at least at the data-center level) and how we are not taking full use of this topic. Recycled electricity at the time of use!!!
They should move to Australia. They have some serious issues on how to use their excessive solar energy coming from residential houses. The electric company even stated that the residents have to pay if they send their extra energy back to the grid.
Well... In Alberta Canada we have flared excess natural gas beyond what the government has set as a limit. For zero usage, not even as heat. It is of course to maximize sales of oil to the USA.
It's just crazy. We've been doing the same here in Venezuela for decades, even when there's no supply of natural gas for households, you would see the flares (called mechurrios here) burning high on the top of tall towers. So sad to see.
Popular models of neural networks that are used in AI models have such problem as catastrophic forgetting. A neural network trained in new tasks dramatically loses the ability to perform previously developed tasks. And what we’re seeing now is NVidia crystallize this drawback. It's a waste of our energy for a while.
I manage a SCADA system for one of the greenest utilities in the country. Data centers have moved in, and now their baseload is 4x their previous all-time peak. The data centers are eating up all their green energy, and the remaining is coming from firing up old coal plants. AI and Cryptomining are totally negating and alternative energy benefits. Ironic Tesla started all of this. Kind of contrary to their mission statement. Utilities I work with are planning on a 3x increase in electrical demand for upcoming data farms.
not only are AI data centers chugging energy, but the demand is only increasing. One thing people miss is these companies are getting huge discounts on energy and tax breaks.
While i agree to some point with you, you would want what, that the corporate media feed them their "truth"'? Only way to get the truth right now IS the internet, if you know how to search and have the mental ability and will to analyse and interpret data. AI is no different that any other tool that you need to learn how to use it, and no diferent then any other info source in that you need to analyse and use critical thinking to validate if it is indeed garbage or not. These day you cannot afford to trust anything blindly.
Firstly, this is the actual problem with this tech, and a far more relevant topic that what the artists are trying to push. Thanks for making this video. Also, anyone else had the name Ricky ruined by Weird Al? Every time he says his name I think" Hey Ricky..." and have to stop and refocus on the video.
DEEP DIVE? HECK YES--on everything you asked if we (i) want to see. Love your explanations and the thoroughness with which you cover each subject you take on. As the Nike ads opine--DO IT!!
What?? How exactly do GPUs degrade in computation efficiency over time??? They’ll flat out break way before you’re going to measure any type of efficiency decline
@@antonvoltchok7794 Maybe I didn't say that right, but GPUs get more efficient each generation, so they are more efficient power wise, time wise, and usually space wise also for each successive generation. So if don't use them 24/7, you loose that opportunity cost.
I would like to see a deep dive into the techniques for implementing neural networks without matrix multiplication. I did some published research on NNs in the early 1990's but haven't really been involved with them since.
Yes @TwoBitDavinci this is frigging important. As a comsumer you should be (easily) able to see where your app-energy comes from, and how much it draws. End-to-end... Super complex tho.
simple solution: Make the big data companies pay for the construction of nuclear power plants, to even out the energy it progressively takes out the regular grid from people. Clearly they got the money for it
I've had that thought for a while. Residential power use can be met with grid-tied or off-grid renewables/storage, while industry can run off nuclear/hydro/geothermal if it needs to be so consistently supplied.
Exactly. Tell them to build solar plants and use them. If they don't like it, fine them. We don't even have real AI. It's just google 2.0. it's all marketing and we see through it.
@@stephenbell9324 Hence why industry can use nuclear. Metal refining and AI training are 24/7/365 processes, and any changes to the load would be slow.
nuclear power plants take decades to build, due to high safety and regulation. so there's o practical way we could do this, especially if other countries dont
Future video topic suggestion. The discussion can be split between training and inference (and deployment). Training is more batch-able, and can utilize renewable energy, esp. when there's plenty. (There are also cap-ex and capacity factors to consider.) Inference is more real-time, and requires 24/7 power. But it's also a space ripe for innovation, such as quantization, compressed models, new hardware such as analog, or elimination of scheduling and frequent memory transfers in favor of optimized deterministic pipeline via compiler (e.g. Groq). There's also edge devices, consumer AI hardware, such as what Apple is doing, in conjunction with private external servers, and services.
Glad you did this video. Its long past time to shine the light on this massive energy consumer, not just on efforts to force those of us that like ice cars to regulatorily mandate evs. Ill make a deal, let me keep my gas car, and i will agree to stay as far away from ai as possible.
To me this just indicates a greater concentration in specialization and humans relying upon this specialization for their existence. There is a huge danger for any life form to rely upon more and more specialization as usually it ends up in their extinction.
@@anthonycarbone3826 Well the nice things about humans is we aren't really that specialized like an animal. It isn't like we are born to do a few things, and rely on a single diet. Or a single climate just to breed. If you think about it. Humans are super generalized, and adaptable. General intelligence allows us to change with our conditions or change those conditions entirely. Specialization won't be our downfall. Our downfall will be our greed.
An interesting irony is that it may be AI tech that gets us to a better place in regards to energy and energy usage. It can design better chips, that can design better more power efficient chips - that go on to develop better power grids, that go to unlock genetic secrets to help medicine, etc.
At 1:15 Brad Smith of Microsoft compares the energy consumption problem of AI to a moonshot where the distance from the earth to the moon increasing by a factor of 5 in a very short period of time: “So in many ways the moon is 5 times away than it was in 2020.” I don’t know about you but that analogy suggests that you have not just a very difficult challenge but now a nearly impossible one, certainly in the short term where all of these companies are racing to control AI. For perspective, that meant a moonshot to a moon went from a distance of 238,900 miles to 1,194,500 miles. There isn’t going to be a manned moonshot at that distance, all other aspects of astrophysics and current engineering capabilities held constant. Incidentally, we are not just talking about electric power grids being insufficient; green energy is absolutely unsuitable to scale for realistic energy demand of AI, because of cost, reliability and even material sourcing/supply chains. Perhaps nuclear fission reactors will work, or the adaptation of neuromorphic chip architectures will mitigate the rapacious energy demand of current GPUs. Bottom line, MS, along with everyone else, is way too optimistic about their growth in this area without fully appreciating that physical infrastructure has to be in place for their plans to have any chance to work.
What people do not understand about AI is that a gaming console becomes an AI when you place a game CD in it. Game programs have become so complex as to, at many levels, outperform most larger server farms in terms of running complex programs. Because of this, no new gaming platforms can be sold outside the U.S. because our government is afraid that terrorists will use the technology within the console to stage attacks. With that in mind, imagine 3 things. First, one building per city block is fitted with enough servers as to be hooked into all other buildings within the same city block as to monitor electrical consumption; building security features; etc.; Second, each apartment, house, business, etc. is also hooked into the system as well; Lastly, because of this, an AI can monitor everything within your residence 24/7 as to reduce the amount of energy consumed during the day when no one is home or otherwise present. Imagine an AI cutting the power to your refrigerator after you left for work because the AI knows that the refrigerator will remain cool enough to keep everything cold because no one will be in your home opening and closing your refrigerator during the day. It can then monitor the temperature to keep it cool, or turn it back on when it knows that you are on your way home. It will know that your on your way home because it has seen you leaving your work from a nearby security camera as well as a traffic camera along the road to your home. Imagine if it can monitor your activities as to make suggestions which will ultimately save you money. There are many cities which uses delivery services for a various number of things. Imagine you coming home to find a box of food just delivered to your front door from Walmart via FedEx. Upon close inspection you ended up saving because the AI used various methods/sales/online coupons/etc. to save you money on the food you eat on a regular basis. Even if you wanted to try new things, it will be able to read various actions you do as to guess what food(s) you are in the mood for eating. Imagine that it figures out that you are in the mood for steak, and reminds you before you leave for work that you have a reservation at a steak house for that evening? Can you afford not to live that way in the current economy?
Your 100 percent right with AI for searching. I am soooo much more efficient in time just asking ai about coding questions. Also recipes, you just get a tiny write up and thats it no more story and ads their either. I'm not going back to search engines if I can help it
A few years ago I though about if we could use the excess heat of these server farms and convert it into motion energy using a sterling motor. Then you could use the motion energy of the motor and power an electric motor, generating electric energy. We could regain some of the energy we used on these server farms and since the heat supply is steady, it would even be base line power with no fluctuations (idk the english term, in Germany it is grundlastfähig)
Rick, we have an impass about the second law of thermodymamics. You are probably as tired of me thinking we can escape it as I am about you asserting it is inescapible. Here is one of my attempts at being too simple to fail but not too simple: A simple rectifier crystal can, iust short of a replicatable long term demonstration of a powerful prototype, almost certainly filter the random thermal motion of electrons or discrete positiive charged voids called holes so the electric current flowing in one direction predominates. At low system voltage a filtrate of one polarity predominates only a little but there is always usable electrical power derived from the source, which is Johnson Nyquest thermal electrical noise. This net electrical filtrate can be aggregated in a group of separate diodes in consistent alignment parallel creating widely scalable electrical power. As the polarity filtered electrical energy is exported, the amount of thermal energy in the group of diodes decreases. This group cooling will draw heat in from the surrounding ambient heat at a rate depending on the filtering rate and thermal resistance between the group and ambient gas, liquid, or solid warmer than absolute zero. There is a lot of ambient heat on our planet, more in equatorial dry desert summer days and less in polar desert winter nights. Refrigeration by the principle that energy is conserved should produce electricity instead of consuming it. Focusing on explaining the electronic behavior of one composition of simple diode, a near flawless crystal of silicon is modified by implanting a small amount of phosphorus on one side from a ohmic contact end to a junction where the additive is suddenly and completely changed to boron with minimal disturbance of the crystal pattern. The crystal then continues to another ohmic contact. A region of high electrical resistance forms at the junction in this type of diode when the phosphorous near the ĵunction donates electrons that are free to move elsewhere while leaving phosphorus ions held in the crystal while the boron donates a hole which is similalarly free to move. The two types of mobile charges mutually clear each other away near the junction leaving little electrical conductivity. An equlibrium width of this region is settled between the phosphorus, boron, electrons, and holes. Thermal noise is beyond steady state equlibrium. Thermal noise transients where mobile electrons move from the phosphorus added side to the boron added side ride transient extra conductivity so the forward moving electrons are preferentally filtered into the external circuit. Electrons are units of electric current. They lose their thermal energy of motion and gain electromotive force, another name for voltage, as they transition between the junction and the array electrical tap. Inside the diode, heat is absorbed: outside the diode, an attached electrical circuit is energized. The net energy in diodes connected in consistent alignment parallel is aggregated. The maximum energy is converted from ambient heat to productive electricity when the electrical load is matched to the array impeadence. Matched impeadence output (watts) is k (Boltzman's constant, ~1.38^-23) times T (tempeature Kelvin) times bandwidth (0 Hz to a natural limit ~2 THz @ 290 K) times rectification efficiency times the number of diodes in the array Order is imposed on the random thermal motion of electrons by the rigerous structual orderlyness of a diode array made of diodes made in a slab: v v v v v v v v v v v v v v v v v v All the boron doped anodes abut the top face of the slab; all the phosphorous doped cathodes abut the bottom face. Simple. Aloha
I imagine that if quantum computers truly become a thing, they'll make training even larger models much more accessible, easier, faster, and more energy efficient. This begs the question, how many parameters are enough? Will we simply continue scaling AI models indefinitely? I don't think they will and this is why I think that: When we scale the models and the number of parameters, we need to scale the training data accordingly. With all the issues around AI and copyright, intellectual property, and whatnot, there won't be enough real-world, human-generated data to train the models on. What will happen then is that the models will be trained on synthetic data output by previous models. For me, this points to a future where models will probably start degrading at first and then eventually plateauing, or developers will simply put a cap on how many parameters they add, or the "real" data they scrape from the web will all be AI-generated, in which case it'll be the exact same scenario as using mostly synthetic data. I wonder how AI will evolve after that? The only thing that I can imagine is that they improve the training algorithms to get a better output on the same training data, and one way they'll probably do that is by asking an AI to improve its own training protocol. After that, it's either it's either Skynet, the Architect, or the Oracle. 🤪
The comment at 9:15 isn't correct. corporate companies adding their own data to AI's use RAG and not usually fine tuning (eg training the base model) because it's too expensive. I'm not saying that the point you are making isn't correct over all but there are now ways to give AI's access to data you want them to be aware of without actually training them further. MS sells access to openAI APIs on locally run models but you don't usually change the base model. If you read up on RAG, Embeddings and Vector DBs you will see what I mean
There is an AI jevon's paradox, where limitations of energy supply will drive efficiency gains out of necessity, which will boost the pace of AI and therefore increase demand. There's many orders of magnitude gains to be had from algrorithimic effciency alone, not considering hardware design, other optimisations. We've already seen AI specific hardware dropping from FP32 FP8 precisions for wicked speed ups. Lots of low hanging fruit in these areas to be picked.
@@AZOffRoadster He has an unhealthy obsession with Elon Musk, and not in a good way. Apparently Elon is the most evil crooked dumbest snake oil salesman in living history LOL
Plot twist: AI already has access to the internet and is using everyone's devices for computing. Granted it still takes a lot of energy, but now it doesn't need to build the infrastructure. ChaosGPT 2024
I’m always up for heat recovery videos. I’m not sure the temperatures are high enough to economically support waste heat recovery. You should visit Ormat and learn about their geothermal and waste heat recovery methods using a low temperature organic working fluid like butane.
9:22 No, that is LoRA (Low rank adapter), it's training is a tiny fraction of a training a foundational model from scratch. In fact that can be done on the desktop PC.
Incredible graphics. Great title, though a lot of competition on that data point. Can you guys a/b test with titles? I noticed this one changed. Did graduate work in 1992 on 'designing' analog neural networks. SWRI did this before me. (In case anyone is looking for IP precedence ;-)
interesting, so you're saying get the cooling benefits of vaporizing water, then find a way to collect it free of the contaminants? issue is salt would gunk up the meshes... but its an interesting thought!
I am a career data center guy. Every time I see big data centers claim to be ‘green’ by purchasing all the power from a hydro power source I call “BullSH**!” Locking in that power does exactly what you suggest, make utilities find other sources for their other customers!! I also agree that Data Center heat needs to be USED not discarded. But here in the US it’s cheaper to burn new gas than to recycle DC heat, so very few do. It is low grade but it can easily heat space and domestic level hot water. We just need to Do It.
"I am a career data center guy. Every time I see big data centers claim to be ‘green’ by purchasing all the power from a hydro power source I call “BullSH**!” Locking in that power does exactly what you suggest, make utilities find other sources for their other customers!! " If they purchase "green" power they can make that claim. And the more demand for "green" electricity the more it will be built out. This is not a big issue.
"There is so much we could do if we got a little more clever." Making electricity from waste heat requires additional processes, equipment, materials, costs, space, maintenance, etc. At some point, chasing incremental efficiency gains isn't worth the additional expenses. Getting clever doesn't do much good when it isn't economically viable or runs into sustainability implications, like Canada coming to the conclusion that most of its hydro potential is tapped out and will need something else, likely new natural gas power plants, to guarantee baseline load through the electrification demand curve. Can't use nuclear for this due to its 15-25 years lead time from proposal to production when you include the endless environmental studies and NIMBY lawsuits.
Investing in solar + batteries makes perfect sense for AI companies, after the initial cost all that energy is free! Why waste money on buying electricity from the grid?
Even after built its not really „free“ as these systems still have upkeep and replacements. Though even with that renewables are just incredibly cheap, the only reason they are not more around already is that the profit margins are bad.
The way it’s going, the AI led energy gold rush is definitely underway and it brings the desperate need for professional energy engineers, a brand new technology and discipline that will provide jobs for everyone in future.
In your comparison between Google and chat gpt, you forgot to factor in the fact that Google returns such bad results you have to keep trying several times and scroll around in your own computer for 10 minutes before you find any results as useful.
Step 1. Invent google search Step 2. Ruin it by funding it with ads. Step 3. Build an AI that can parse through all the ads that you put there. Step 4. Invent a meta-AI search algorithm that optimizes all of the competing AIs to provide the best answer... Rather than simply building public infrastructure, we keep shooting ourselves in the foot by insisting on preserving a parasitic investor class, and we have to invent increasingly baroque ways to do it.
What about Jevons principle, in which the more efficient something is the more the demand for it goes up? James Watt’s more efficient steam engine ushered in the industrial revolution.
As many countries no longer have a sustainable replacement birth rate, I wonder what impact a 20-40% reduction in population will have on the overall energy consumption? Korea was at 0.72 birth rate in 2023.
As long as the energy is properly priced according to supply and demand, it won't be an issue. Let these AI companies have the energy, but make sure they pay the price for it.
Commercial power rates are typically a fraction of the cost of residential. Same power sold at different rates... surprising? Nah, our system is designed this way. 😅😮.
"As long as the energy is properly priced according to supply and demand, it won't be an issue. Let these AI companies have the energy, but make sure they pay the price for it." Yeah. Market forces will fix everything in the long term.
Thank you for sounding the alarm. I’ve been screaming about this without a platform. I hold a masters in a sort of MBA Sustainability Leadership from Arizona State and work in deep tech and am loosing sleep over it. If you see in your head data regeneration from AI, it’s horrifying. I use it sparingly, but could use it all day long if I didn’t know what it costs. We have a solution. Reducing data processing by 7/8, especially AI because it runs so much data bloat. Clean out all the extraneous and it becomes more efficient. It does look like the gold rush is abating because it frequently proves hallucinogenic or just plain wrong.
Stop worrying about it. here s why:-1966: oil gone in 10yrs 1967: dire famine forecast by 1975 1968: overpopulation will spread worldwide 1969: everyone will disappear in a cloud of blue steam by 1989 1970: the world will use up all its natural resources by 2000, urban citizens will require gas masks by 1985, nitrogen build-up will make all land unusable, decaying pollution will kill all the fish, killer bees, ice age by 2000 and America will be subject to water rationing by 1974 and food rationing by 1980 1971: new ice age coming by 2020 or 2030 1972: new ice age by 2070 and oil depleted in 20yrs 1974: space satellites show new ice age coming fast, ozone depletion and "Great peril to life" 1976: scientific consensus planet cooling and famines imminent 1977: department of energy says oil will peak in the 90s 1978: no end in sight to 30yr cooling trend 1980: acid rain kills life in lakes and peak oil in 2000 1988: regional droughts in the 90s, temperatures in DC will hit record highs and Maldives will be underwater by 2018 1989: rising sea levels will obliterate nations if nothing is done by 2000 and New York City's West Side Highway will be underwater by 2019 1996: peak oil in 2020 2000: children won't know what snow is 2002: famine in 10yrs if we don't give up eating meat, fish and dairy and peak oil in 2010 2004: Britain will be Siberia by 2024 2005: Manhattan will be underwater by 2015 2006: super hurricanes 2008: the Arctic will be ice free by 2018 and Climate Genius Al Gore predicts an ice free Arctic by 2013 2009: Climate Genius Prince Charles says we have 96 months to save the world, UK Prime Minister Gordon Brown says we have 50 days to "save the planet from catastrophe" and Climate Genius Al Gore moves his 2013 prediction of an ice free Arctic to 2014 2013: the Arctic will be ice free by 2015 2014: only 500 days before "Climate chaos" 2019: Hey Greta, we need you to convince them it's really going to happen this time 2020 Greta Thunderberg
yeah early days focus on shear volume of data and results, and fine tuning and optimization often comes later... but its very important as you noted. how do we reduce the processing overhead exactly?
"Thank you for sounding the alarm. I’ve been screaming about this without a platform. I hold a masters in a sort of MBA Sustainability Leadership from Arizona State and work in deep tech and am loosing sleep over it. " That's a you problem I'm afraid.
Great video as usual, but please fix sound. Listen and compare with several other videos on youtube for side by side comparison. Of the hundreds of videos I watch a month, these consistently have the worst sound.
I think in the long run, A.I. won't consume as much energy as we think it will do. I suspect as A.I. becomes more capable, more of us and more businesses will want to run A.I. at a local level for privacy and security reasons, especially as A.I. becomes more capable and useful, which will likely push more consumers and businesses to not want to use online A.I. services, and that will only get a lot worse as A.I. becomes more useful and even worse once A.I. can have a long term memory so it can change, adapt and learn of our habits. A.I. running at a local level will likely get absorbed into the energy our computers use, and likely just use a little more than what we do now, A.I. isn't what we should be worried about when it comes to power use, it's robotics that could use a heck of a lot more energy than we use today as that won't be absorbed into our current system of computers and will be thrown onto our electrical grid like EV cars are. With that said, there's no real way to know, reason being is that corporations and governments might go crazy with A.I. at such a level that needs an insane amount of power, but I think for personal and corporate use, I think the energy use will be minimum and just a little higher than what we already use, basically, it will be absorbed into what we use as tech gets better and able to do A.I. more effectively, but there is a lot of unknowns in all this as we don't know the limits of how far we'll push A.I. and how far we can push it before there's a drop-off in quality that pushing harder isn't giving us much better results but ends up needing far more energy that it's not worth it. With all that said, we humans will always live within the means of how much energy we produce and will likely use more if we produce more, which renewable energy over the long run could produce far more energy than we use over the coming decades. In any case, we should remember that unlike computers or the internet, which was adding a lot of energy use didn't exist before because we didn't have computers or the internet, A.I. is just another service being thrown onto what is already there, so yes there will be a spike in energy use, especially early on, but I think long term it will be absorbed into our computer use and not make that much of a difference, but robotics on the other hand is another story entirely and is a bit like EV cars, so EV cars and robotics is what's really going to have a massive spike on our energy use.
I'm interested to find out how much energy we have saved over the past 50 years by switching from cathode ray tubes and vacuum tube technology to flat panel IC based technology for the item that we buy hundreds of millions of and that is TVs. Let alone the cost of the energy used to ship all of those heavy glass objects around everywhere. So we have saved countless amounts of money from one sector and put it into this centralized sector which is capable of doing things we only dreamed of 50 years ago
most of the things people think they want to use AI for is like asking a chef to make toast with a Volcano its a massive waste of power for most things We already have efficient methods of doing those things. AI should not be used for most things
I asked that microshite copilot why we are using all this processing power on AI instead of, say, cracking the nuclear launch codes. It said that was an innapropriate question, I promptly deactivated it off my computer...
Thanks DeleteMe for sponsoring this video! Protect your online Info Today! joindeleteme.com/TwoBitDavinci
Shows how efficient quarternary is compared to binary or quantum.
deju vu. this feels like the FUD with crypto mining since 2017-18 or the Intel Pentium 4 FUD with skyrocketing heat from CPUs. I wouldn't be surprised in a few years that writers will be thanking the AI boom for better energy solutions due to steadily growing number of EVs.
AI is unsustainable.
Knowing how much power A.I. consumes makes you truly appreciate how efficiently the human brain operates.
That is very true, but it's easy to not recognize how much information ChatGPT knows.
For example... my brain is energy efficient, but I just used ChatGPt to create a python script with 100s of lines.
I didn't have to learn the syntax (this is my first python script).
I didn't have to look up and learn about different modules that it uses.
To be honest, it probably saved me 20 to 40 hours and was completed and tested in about 1 hour.
I'm not convinced it's less efficient.
well said! pretty wild that thing between our ears!
@@TwoBitDaVinci So half joking...is the next step in computing an organic computer or some kind of human brain farm (like in the Matrix but for harvesting computational power instead of heat energy)?
Lol, it would at least make for an interesting plot for a scifi movie.
It seems like my comment got deleted...
I just used a LLM to help me to program a python script. It's up to 3500 lines of code, but more than half was copy and paste from the LLM.
Before this, I have never used Python and i didn't have to get a book and learn the syntax, etc. I estimate that it's saved me 80% of the time it would take for me to write it from scratch.
Yes it takes power and yes I can write a 10 page essay, but a LLM can write it better and in 1 min.
Brains take less power, but they I'm not convinced they are more energy efficient (energy consumed per output unit)
@mccue2439
When talking about energy efficiency, speed is not what we're looking at. What is amazing about the brain is that it could write that script at the energy cost of a cup of coffee.
Me: ChatGPT, can you tell me how much electricity you're actually using?
ChatGPT: Eh, can't we talk about something else?
Happy to hear that analog computation is finding new life. Sixty years ago I had the pleasure of programming the Electronic Associated PACE Model 231R, the last of the large analog computers. It regularly calculated results faster than our department's digital computers. Then we installed a Philco 2000 with 10k core memory! Analog still won the timed races, but lost out on precision with "only" four-digit resolution.
wow that's so cool, i'd love to interview you for something if i cover it again!
@@TwoBitDaVinci Sure. Residing San Diego now, but relocating to Phoenix in August. BTW, imagine the power requirements if AI only had the vacuum tubes we relied on!
I live in the area where the Cedar Rapids data center is to be built. The local nuclear power plant has reached its end of life and is now being replaced by 3,500 acres of solar panels on prime farmland. This video just put the pieces together for me as to why. I can see a lot more expansion of these panels to more prime farmland. Looks like this would have been better served in a location where less productive land is abundant.
Knowing they use farmland for this when there are so many flat topped buildings with fully unused space drives me bonkers, too.
40% of the corn grown in the country is for Ethanol. PV solar produces 100 times more energy per acre than corn ethanol. In addition, many are also using the land with PV solar for specialized agriculture, like sheep grazing and part shade crops.
Sorry but commercial nuclear power is not the answer. In part due to its ultra-expensive cost and ultra-long build times, commercial nuclear power will be gone from the grid in about 50 years. Not just because of that but renewables are at historical low cost per MWh generation (and still falling) while nuclear keeps getting more expensive.
Furthermore, "firm renewables" at scale are just around the corner with massive government, scientific and industrial interest in grid-scale storage technologies. When firm renewables arrive at scale, all other energy production methods like fossil fuel and nuclear are done on the grid. Nuclear will still be used in small-scale scenarios like military, scientific and deep-space exploration. Commercial nuclear is just the wrong tool for the job on the grid. Commercial nuclear power's own history bears that out with it powering only 10% of the world's grids after a long 70-years of history and its percentage is falling. Nuclear failed us from its promises of abundant energy "too cheap to meter" promised in the 1950s. If nuclear had come close its promises, we wouldn't be so far down the climate crisis hole we're currently in. Nuclear failed us.
At 99.9% the mass of the solar system, the sun provides a cosmic 173,000 terawatts to the Earth non-stop. Or about 1000W per square meter peak at the ground and is always peak somewhere on Earth. All the combined fossil and fissile material on Earth would amount to a bucket or two of water in an ocean compared to the daily amounts of energy provided to us from the free clean *fusion* power we get from the sky. Renewables harness this energy directly, and the raw inputs of sunlight and wind cannot be taxed, embargoed, blockaded or sanctioned.
With respect to land, much less than 1% of the world's total land surface alone in current generation photovoltaics is enough to power all the world's grids. But why stop there? Imagine a full 1% or 2%? Solar can also produce power 'in situ' on-premises which greatly reduces load on the grid because rooftop solar "unloads" the grid. It helps that datacenters have massive rooftop and parking lot areas. Also, a lot of the biggest datacenters generally have a lot of open land around them as they're built outside of urban areas.
A solar farm is actually more environmentally friendly than an actual agricultural farm. Current agricultural methods are ruinous to the environment with animal run-off, pesticide run-off, and soil erosion. So, it's not surprising that with a solar farm on it, the land would live a much more peaceful life than it ever did as an agricultural farm. New agricultural methods are needed. One example is using "agrivoltaics", one can reap two crops off the land: food and energy which humans need a constant supply of.
@@MichaelF350 Soybeans are a thing. Also, I am watching these solar panels go up just 3 miles from my home and they are not planning on raising animals under them-they are too low for that.
@@beyondfossil people cause most of the problems you’re associating with nuclear power generation.
If we built more nuclear power plants instead of shutting them down. That would help with energy needs.
Amen
Put data centers along the coasts and use waste heat to desalinate ocean water. Would be especially useful on the west coast
🤗 THANKS RICKY,FOR SHARING THIS 🤯INFO, AND THE POSSIBILITIES FOR THE FUTURE 💚💚💚
you bet, and thanks for being such an active voice in our videos! I see you and appreciate you! don't even know you're real first name, but thank you budgetAudiophile :) great name btw
🤗HAPPY TO SUPPORT YOU,AND I WATCHED YOUR LIVE STREAM, I AM RETIRED ( on a budget,hence the name) BUT MY NAME IS RICHARD AND I GO BY RICK 💚💚💚
All of this AI development, and yet, it's not helping to stop major issues like war, crime, inflation, crazy housing and medical costs.
Can somebody please tell me what AI has achieved so far after using all of this precious resource? Like a basic cost analysis. Thankyou
In Denmark Facebook is already using heat from their serverpark to heat homes. It's already happening.
I’ll have to look into it!
District heating has higher infrastructure costs up front, but once the infrastructure is built there are lots of inputs. Even waste heat from nuclear plants can go into district heat grids.
@@TwoBitDaVinciplease do and then tell us why this isn’t happening everywhere. Thanks for your awesome videos and for being such a generally cool guy. Not at all surprised why Mrs. Llewellyn liked you so much the instant she saw one of your videos.
Hmmm--I wonder has people thought about putting up server farms in the "safe" areas of Iceland, server farms all powered by plentiful geothermal power.
I was wondering about that the other day. You could eliminate air and water heating costs for residential commercial and public sector.
All of us who were using NVIDIA GPUs to mine Ethereum could have told you this.
In my case, I was mining using rooftop solar electricity.
Womp womp, Ethereum go brr wanna cry.
Yes! Make a video on heat redistribution (at least at the data-center level) and how we are not taking full use of this topic. Recycled electricity at the time of use!!!
Yessss
I wanna see more options for residual server closet heat recycling or a heat dump into a sunamp thermal battery or a sand battery
heat is so often disregarded, but its got so much application.
Oh yeah does this mean like using heat pumps to convert that heat into electricity?
@@benmcreynolds8581 I think it makes much more sense to use the heat as it is, to heat something else.
They should move to Australia. They have some serious issues on how to use their excessive solar energy coming from residential houses. The electric company even stated that the residents have to pay if they send their extra energy back to the grid.
Well... In Alberta Canada we have flared excess natural gas beyond what the government has set as a limit. For zero usage, not even as heat.
It is of course to maximize sales of oil to the USA.
It's just crazy. We've been doing the same here in Venezuela for decades, even when there's no supply of natural gas for households, you would see the flares (called mechurrios here) burning high on the top of tall towers. So sad to see.
The gas has to be flared if there isn't a pipeline network to take it away.
Popular models of neural networks that are used in AI models have such problem as catastrophic forgetting. A neural network trained in new tasks dramatically loses the ability to perform previously developed tasks. And what we’re seeing now is NVidia crystallize this drawback. It's a waste of our energy for a while.
I just found your channel today and I love it! Thanks so much for all the knowledge 😊
I manage a SCADA system for one of the greenest utilities in the country. Data centers have moved in, and now their baseload is 4x their previous all-time peak. The data centers are eating up all their green energy, and the remaining is coming from firing up old coal plants. AI and Cryptomining are totally negating and alternative energy benefits. Ironic Tesla started all of this. Kind of contrary to their mission statement. Utilities I work with are planning on a 3x increase in electrical demand for upcoming data farms.
not only are AI data centers chugging energy, but the demand is only increasing. One thing people miss is these companies are getting huge discounts on energy and tax breaks.
AI a working example of Garbage In Garbage Out. If it is relying on the internet to 'inform' itself, AI is doomed to fail.
While i agree to some point with you, you would want what, that the corporate media feed them their "truth"'? Only way to get the truth right now IS the internet, if you know how to search and have the mental ability and will to analyse and interpret data. AI is no different that any other tool that you need to learn how to use it, and no diferent then any other info source in that you need to analyse and use critical thinking to validate if it is indeed garbage or not. These day you cannot afford to trust anything blindly.
Firstly, this is the actual problem with this tech, and a far more relevant topic that what the artists are trying to push. Thanks for making this video.
Also, anyone else had the name Ricky ruined by Weird Al? Every time he says his name I think" Hey Ricky..." and have to stop and refocus on the video.
DEEP DIVE? HECK YES--on everything you asked if we (i) want to see. Love your explanations and the thoroughness with which you cover each subject you take on. As the Nike ads opine--DO IT!!
Off peak ai training sounds good until you think about how fast the hardware depreciates, not just in expense, but also in efficiency.
What?? How exactly do GPUs degrade in computation efficiency over time??? They’ll flat out break way before you’re going to measure any type of efficiency decline
@@antonvoltchok7794 Maybe I didn't say that right, but GPUs get more efficient each generation, so they are more efficient power wise, time wise, and usually space wise also for each successive generation. So if don't use them 24/7, you loose that opportunity cost.
I would like to see a deep dive into the techniques for implementing neural networks without matrix multiplication. I did some published research on NNs in the early 1990's but haven't really been involved with them since.
you got it, and you're not alone... it's pretty interesting stuff
Yes @TwoBitDavinci this is frigging important. As a comsumer you should be (easily) able to see where your app-energy comes from, and how much it draws. End-to-end... Super complex tho.
Thumbnail/title suggestion: Something along the lines of "This unstoppable technology will suck up MASSIVE amounts of energy"
They should invest in green energy and small portable nuclear reactors to offset used energy.
Great video
The dirtier secret is all that energy are probably not well spent in terms of the outcome it generates.
Great video. love to see a video on waste heat from data centers.
Please DO produce a video on analog computing! Thank you!!
we have here it is! ruclips.net/video/VWn6Ixh2eDg/видео.html
Stick all the humans into cells as human batteries.
...no wait!😮
simple solution: Make the big data companies pay for the construction of nuclear power plants, to even out the energy it progressively takes out the regular grid from people.
Clearly they got the money for it
I've had that thought for a while. Residential power use can be met with grid-tied or off-grid renewables/storage, while industry can run off nuclear/hydro/geothermal if it needs to be so consistently supplied.
Exactly. Tell them to build solar plants and use them. If they don't like it, fine them.
We don't even have real AI. It's just google 2.0. it's all marketing and we see through it.
Nuclear is too slow but the fracking geothermal tech looks promising
@@stephenbell9324 Hence why industry can use nuclear. Metal refining and AI training are 24/7/365 processes, and any changes to the load would be slow.
nuclear power plants take decades to build, due to high safety and regulation. so there's o practical way we could do this, especially if other countries dont
Yes, please do a video on that research paper
I would say yes to a new video. Thank you for your work.
you got it, noted!
Wow - great insight. Well done, Ricky!
thank you, it's hard to see sometimes because we're so far removed from our decisions
This video just turned into a great Nvidia commercial
AMD is ready to start shipping their A.I. Chips. They have huge orders coming in now. What a time to be alive !
Future video topic suggestion. The discussion can be split between training and inference (and deployment).
Training is more batch-able, and can utilize renewable energy, esp. when there's plenty. (There are also cap-ex and capacity factors to consider.)
Inference is more real-time, and requires 24/7 power. But it's also a space ripe for innovation, such as quantization, compressed models, new hardware such as analog, or elimination of scheduling and frequent memory transfers in favor of optimized deterministic pipeline via compiler (e.g. Groq). There's also edge devices, consumer AI hardware, such as what Apple is doing, in conjunction with private external servers, and services.
This is not dark side, it is bright side. When initial rush is over it will be elastic consumer than can stop taking power in peak price moments.
Glad you did this video. Its long past time to shine the light on this massive energy consumer, not just on efforts to force those of us that like ice cars to regulatorily mandate evs. Ill make a deal, let me keep my gas car, and i will agree to stay as far away from ai as possible.
To me this just indicates a greater concentration in specialization and humans relying upon this specialization for their existence. There is a huge danger for any life form to rely upon more and more specialization as usually it ends up in their extinction.
@@anthonycarbone3826 Well the nice things about humans is we aren't really that specialized like an animal.
It isn't like we are born to do a few things, and rely on a single diet. Or a single climate just to breed.
If you think about it. Humans are super generalized, and adaptable. General intelligence allows us to change with our conditions or change those conditions entirely.
Specialization won't be our downfall. Our downfall will be our greed.
An interesting irony is that it may be AI tech that gets us to a better place in regards to energy and energy usage. It can design better chips, that can design better more power efficient chips - that go on to develop better power grids, that go to unlock genetic secrets to help medicine, etc.
4:45 Actually THIS sentence here would be a perfect title for this video
At 1:15 Brad Smith of Microsoft compares the energy consumption problem of AI to a moonshot where the distance from the earth to the moon increasing by a factor of 5 in a very short period of time:
“So in many ways the moon is 5 times away than it was in 2020.”
I don’t know about you but that analogy suggests that you have not just a very difficult challenge but now a nearly impossible one, certainly in the short term where all of these companies are racing to control AI. For perspective, that meant a moonshot to a moon went from a distance of 238,900 miles to 1,194,500 miles. There isn’t going to be a manned moonshot at that distance, all other aspects of astrophysics and current engineering capabilities held constant.
Incidentally, we are not just talking about electric power grids being insufficient; green energy is absolutely unsuitable to scale for realistic energy demand of AI, because of cost, reliability and even material sourcing/supply chains. Perhaps nuclear fission reactors will work, or the adaptation of neuromorphic chip architectures will mitigate the rapacious energy demand of current GPUs. Bottom line, MS, along with everyone else, is way too optimistic about their growth in this area without fully appreciating that physical infrastructure has to be in place for their plans to have any chance to work.
What people do not understand about AI is that a gaming console becomes an AI when you place a game CD in it. Game programs have become so complex as to, at many levels, outperform most larger server farms in terms of running complex programs. Because of this, no new gaming platforms can be sold outside the U.S. because our government is afraid that terrorists will use the technology within the console to stage attacks.
With that in mind, imagine 3 things. First, one building per city block is fitted with enough servers as to be hooked into all other buildings within the same city block as to monitor electrical consumption; building security features; etc.; Second, each apartment, house, business, etc. is also hooked into the system as well; Lastly, because of this, an AI can monitor everything within your residence 24/7 as to reduce the amount of energy consumed during the day when no one is home or otherwise present.
Imagine an AI cutting the power to your refrigerator after you left for work because the AI knows that the refrigerator will remain cool enough to keep everything cold because no one will be in your home opening and closing your refrigerator during the day. It can then monitor the temperature to keep it cool, or turn it back on when it knows that you are on your way home. It will know that your on your way home because it has seen you leaving your work from a nearby security camera as well as a traffic camera along the road to your home.
Imagine if it can monitor your activities as to make suggestions which will ultimately save you money. There are many cities which uses delivery services for a various number of things. Imagine you coming home to find a box of food just delivered to your front door from Walmart via FedEx. Upon close inspection you ended up saving because the AI used various methods/sales/online coupons/etc. to save you money on the food you eat on a regular basis. Even if you wanted to try new things, it will be able to read various actions you do as to guess what food(s) you are in the mood for eating. Imagine that it figures out that you are in the mood for steak, and reminds you before you leave for work that you have a reservation at a steak house for that evening? Can you afford not to live that way in the current economy?
Your 100 percent right with AI for searching. I am soooo much more efficient in time just asking ai about coding questions. Also recipes, you just get a tiny write up and thats it no more story and ads their either. I'm not going back to search engines if I can help it
I'm with you all the way! But I feel it's just a matter of time until they start monetizing AI and adding ads to results.
@@Israel_Two_Bit I know, the enshitification of the future AI is going to be sad
A few years ago I though about if we could use the excess heat of these server farms and convert it into motion energy using a sterling motor. Then you could use the motion energy of the motor and power an electric motor, generating electric energy. We could regain some of the energy we used on these server farms and since the heat supply is steady, it would even be base line power with no fluctuations (idk the english term, in Germany it is grundlastfähig)
I just spent 1.21 Gigawatts of energy!
Did you use your flux capacitor at 88mph?
No! Better! I have a PowerPoint slide of cats!
Scotty:” I’M GIVING ALL SHE’S GOT CAPTAIN!!!”
lol
Every time this stuff comes up the lore of rainworld rings in my mind.
@@turboprint3d truely a rainworld moment
Rick, we have an impass about the second law of thermodymamics. You are probably as tired of me thinking we can escape it as I am about you asserting it is inescapible.
Here is one of my attempts at being too simple to fail but not too simple:
A simple rectifier crystal can, iust short of a replicatable long term demonstration of a powerful prototype, almost certainly filter the random thermal motion of electrons or discrete positiive charged voids called holes so the electric current flowing in one direction predominates. At low system voltage a filtrate of one polarity predominates only a little but there is always usable electrical power derived from the source, which is Johnson Nyquest thermal electrical noise. This net electrical filtrate can be aggregated in a group of separate diodes in consistent alignment parallel creating widely scalable electrical power. As the polarity filtered electrical energy is exported, the amount of thermal energy in the group of diodes decreases. This group cooling will draw heat in from the surrounding ambient heat at a rate depending on the filtering rate and thermal resistance between the group and ambient gas, liquid, or solid warmer than absolute zero. There is a lot of ambient heat on our planet, more in equatorial dry desert summer days and less in polar desert winter nights.
Refrigeration by the principle that energy is conserved should produce electricity instead of consuming it.
Focusing on explaining the electronic behavior of one composition of simple diode, a near flawless crystal of silicon is modified by implanting a small amount of phosphorus on one side from a ohmic contact end to a junction where the additive is suddenly and completely changed to boron with minimal disturbance of the crystal pattern. The crystal then continues to another ohmic contact.
A region of high electrical resistance forms at the junction in this type of diode when the phosphorous near the ĵunction donates electrons that are free to move elsewhere while leaving phosphorus ions held in the crystal while the boron donates a hole which is similalarly free to move. The two types of mobile charges mutually clear each other away near the junction leaving little electrical conductivity. An equlibrium width of this region is settled between the phosphorus, boron, electrons, and holes. Thermal noise is beyond steady state equlibrium. Thermal noise transients where mobile electrons move from the phosphorus added side to the boron added side ride transient extra conductivity so the forward moving electrons are preferentally filtered into the external circuit. Electrons are units of electric current. They lose their thermal energy of motion and gain electromotive force, another name for voltage, as they transition between the junction and the array electrical tap. Inside the diode, heat is absorbed: outside the diode, an attached electrical circuit is energized. The net energy in diodes connected in consistent alignment parallel is aggregated. The maximum energy is converted from ambient heat to productive electricity when the electrical load is matched to the array impeadence.
Matched impeadence output (watts) is k (Boltzman's constant, ~1.38^-23) times T (tempeature Kelvin) times bandwidth (0 Hz to a natural limit ~2 THz @ 290 K) times rectification efficiency times the number of diodes in the array
Order is imposed on the random thermal motion of electrons by the rigerous structual orderlyness of a diode array made of diodes made in a slab:
v v v v v v v v v v v v v v v v v v
All the boron doped anodes abut the top face of the slab; all the phosphorous doped cathodes abut the bottom face. Simple.
Aloha
I imagine that if quantum computers truly become a thing, they'll make training even larger models much more accessible, easier, faster, and more energy efficient. This begs the question, how many parameters are enough? Will we simply continue scaling AI models indefinitely?
I don't think they will and this is why I think that:
When we scale the models and the number of parameters, we need to scale the training data accordingly. With all the issues around AI and copyright, intellectual property, and whatnot, there won't be enough real-world, human-generated data to train the models on. What will happen then is that the models will be trained on synthetic data output by previous models.
For me, this points to a future where models will probably start degrading at first and then eventually plateauing, or developers will simply put a cap on how many parameters they add, or the "real" data they scrape from the web will all be AI-generated, in which case it'll be the exact same scenario as using mostly synthetic data.
I wonder how AI will evolve after that? The only thing that I can imagine is that they improve the training algorithms to get a better output on the same training data, and one way they'll probably do that is by asking an AI to improve its own training protocol. After that, it's either it's either Skynet, the Architect, or the Oracle.
🤪
I guess no one realizes this dirty little secret: EVERYTHING HAS THE SAME SECRET, we live in an 'advanced civilization'. Moving on.
Who made the background music
The comment at 9:15 isn't correct.
corporate companies adding their own data to AI's use RAG and not usually fine tuning (eg training the base model) because it's too expensive.
I'm not saying that the point you are making isn't correct over all but there are now ways to give AI's access to data you want them to be aware of without actually training them further.
MS sells access to openAI APIs on locally run models but you don't usually change the base model. If you read up on RAG, Embeddings and Vector DBs you will see what I mean
There is an AI jevon's paradox, where limitations of energy supply will drive efficiency gains out of necessity, which will boost the pace of AI and therefore increase demand. There's many orders of magnitude gains to be had from algrorithimic effciency alone, not considering hardware design, other optimisations. We've already seen AI specific hardware dropping from FP32 FP8 precisions for wicked speed ups. Lots of low hanging fruit in these areas to be picked.
You just got slapped by thunderf00t LOL 😂
Yes he did ,, So ..DUCE BITZ please do a retraction video cuz MAN you got it WRONG
He's still around? Is he still forever looping video of what happens when you pull a vacuum on a container designed for pressure? (Hyperloop bashing)
@@AZOffRoadster He has an unhealthy obsession with Elon Musk, and not in a good way. Apparently Elon is the most evil crooked dumbest snake oil salesman in living history LOL
Plot twist: AI already has access to the internet and is using everyone's devices for computing. Granted it still takes a lot of energy, but now it doesn't need to build the infrastructure. ChaosGPT 2024
Always interesting, thanks. Can't wait to discover what AS (Artificial Stupidity) will bring us.
I’m always up for heat recovery videos. I’m not sure the temperatures are high enough to economically support waste heat recovery.
You should visit Ormat and learn about their geothermal and waste heat recovery methods using a low temperature organic working fluid like butane.
9:22 No, that is LoRA (Low rank adapter), it's training is a tiny fraction of a training a foundational model from scratch. In fact that can be done on the desktop PC.
This is why nuclear will need to be developed and deployed at scale in the next decade.
Incredible graphics. Great title, though a lot of competition on that data point. Can you guys a/b test with titles? I noticed this one changed. Did graduate work in 1992 on 'designing' analog neural networks. SWRI did this before me. (In case anyone is looking for IP precedence ;-)
Is anybody using evaporative cooling for desalination? This could be a win-win.
interesting, so you're saying get the cooling benefits of vaporizing water, then find a way to collect it free of the contaminants? issue is salt would gunk up the meshes... but its an interesting thought!
Surely though, that is just an engineering problem, and not any worse than any other heat-based desalination program.
Interesting , Thank you.
I am a career data center guy. Every time I see big data centers claim to be ‘green’ by purchasing all the power from a hydro power source I call “BullSH**!” Locking in that power does exactly what you suggest, make utilities find other sources for their other customers!!
I also agree that Data Center heat needs to be USED not discarded. But here in the US it’s cheaper to burn new gas than to recycle DC heat, so very few do. It is low grade but it can easily heat space and domestic level hot water. We just need to Do It.
"I am a career data center guy. Every time I see big data centers claim to be ‘green’ by purchasing all the power from a hydro power source I call “BullSH**!” Locking in that power does exactly what you suggest, make utilities find other sources for their other customers!! "
If they purchase "green" power they can make that claim. And the more demand for "green" electricity the more it will be built out. This is not a big issue.
There is no way to create energy with all that extra heat?
"There is so much we could do if we got a little more clever."
Making electricity from waste heat requires additional processes, equipment, materials, costs, space, maintenance, etc. At some point, chasing incremental efficiency gains isn't worth the additional expenses.
Getting clever doesn't do much good when it isn't economically viable or runs into sustainability implications, like Canada coming to the conclusion that most of its hydro potential is tapped out and will need something else, likely new natural gas power plants, to guarantee baseline load through the electrification demand curve. Can't use nuclear for this due to its 15-25 years lead time from proposal to production when you include the endless environmental studies and NIMBY lawsuits.
Maybe clarify how more data centers helps to solve Google's problem of having so many ads pop up in search results.
Yes. Definitely interested in that paper. How on earth can you train or operate an LLM without matrix multiplication? That would be huge. 💚
This is why Apple wants to do much of the AI processing at the client level whether it's iPhone, iPad or one of the various Mac models.
Excellent.
Investing in solar + batteries makes perfect sense for AI companies, after the initial cost all that energy is free! Why waste money on buying electricity from the grid?
Even after built its not really „free“ as these systems still have upkeep and replacements. Though even with that renewables are just incredibly cheap, the only reason they are not more around already is that the profit margins are bad.
The way it’s going, the AI led energy gold rush is definitely underway and it brings the desperate need for professional energy engineers, a brand new technology and discipline that will provide jobs for everyone in future.
Keep going down this rabbit hole please 🙏 🍿
In your comparison between Google and chat gpt, you forgot to factor in the fact that Google returns such bad results you have to keep trying several times and scroll around in your own computer for 10 minutes before you find any results as useful.
Yeah we really need to make the leap to being a type one civilization. Will we ever be able to crack Fusion reactors?
Step 1. Invent google search
Step 2. Ruin it by funding it with ads.
Step 3. Build an AI that can parse through all the ads that you put there.
Step 4. Invent a meta-AI search algorithm that optimizes all of the competing AIs to provide the best answer...
Rather than simply building public infrastructure, we keep shooting ourselves in the foot by insisting on preserving a parasitic investor class, and we have to invent increasingly baroque ways to do it.
Hopefully quantum computers become commercially viable like your recent video explores, that could help!
Has anyone else noticed the sound quality tank a few months ago? it sounds like he is talking through a wet sponge.
What about Jevons principle, in which the more efficient something is the more the demand for it goes up? James Watt’s more efficient steam engine ushered in the industrial revolution.
Whats up with the audio? Seems HIGHLY compressed.
NPU's should also bridge the gap
As many countries no longer have a sustainable replacement birth rate, I wonder what impact a 20-40% reduction in population will have on the overall energy consumption? Korea was at 0.72 birth rate in 2023.
I would like to see data centres built in space where power and cooling is not an issue
As long as the energy is properly priced according to supply and demand, it won't be an issue. Let these AI companies have the energy, but make sure they pay the price for it.
Commercial power rates are typically a fraction of the cost of residential. Same power sold at different rates... surprising? Nah, our system is designed this way. 😅😮.
@@ryandgarlandyeah but there are many reasons it is that way. Just keep in mind there’s nuance to it
"As long as the energy is properly priced according to supply and demand, it won't be an issue. Let these AI companies have the energy, but make sure they pay the price for it."
Yeah. Market forces will fix everything in the long term.
@@yourcrazybear why the fuck did you copy the entire comment you replied to?
@@bob-km4uq "@yourcrazybear why the fuck did you copy the entire comment you replied to?"
So you never heard of a quote before?
Eventually something will have to go...
They aren't building any new power stations.
The water supplies are getting stretched to
The limit .
Thank you for sounding the alarm. I’ve been screaming about this without a platform. I hold a masters in a sort of MBA Sustainability Leadership from Arizona State and work in deep tech and am loosing sleep over it. If you see in your head data regeneration from AI, it’s horrifying. I use it sparingly, but could use it all day long if I didn’t know what it costs.
We have a solution. Reducing data processing by 7/8, especially AI because it runs so much data bloat. Clean out all the extraneous and it becomes more efficient.
It does look like the gold rush is abating because it frequently proves hallucinogenic or just plain wrong.
Stop worrying about it. here s why:-1966: oil gone in 10yrs
1967: dire famine forecast by 1975
1968: overpopulation will spread worldwide
1969: everyone will disappear in a cloud of blue steam by 1989
1970: the world will use up all its natural resources by 2000, urban citizens will require gas masks by 1985, nitrogen build-up will make all land unusable, decaying pollution will kill all the fish, killer bees, ice age by 2000 and America will be subject to water rationing by 1974 and food rationing by 1980
1971: new ice age coming by 2020 or 2030
1972: new ice age by 2070 and oil depleted in 20yrs
1974: space satellites show new ice age coming fast, ozone depletion and "Great peril to life"
1976: scientific consensus planet cooling and famines imminent
1977: department of energy says oil will peak in the 90s
1978: no end in sight to 30yr cooling trend
1980: acid rain kills life in lakes and peak oil in 2000
1988: regional droughts in the 90s, temperatures in DC will hit record highs and Maldives will be underwater by 2018
1989: rising sea levels will obliterate nations if nothing is done by 2000 and New York City's West Side Highway will be underwater by 2019
1996: peak oil in 2020
2000: children won't know what snow is
2002: famine in 10yrs if we don't give up eating meat, fish and dairy and peak oil in 2010
2004: Britain will be Siberia by 2024
2005: Manhattan will be underwater by 2015
2006: super hurricanes
2008: the Arctic will be ice free by 2018 and Climate Genius Al Gore predicts an ice free Arctic by 2013
2009: Climate Genius Prince Charles says we have 96 months to save the world, UK Prime Minister Gordon Brown says we have 50 days to "save the planet from catastrophe" and Climate Genius Al Gore moves his 2013 prediction of an ice free Arctic to 2014
2013: the Arctic will be ice free by 2015
2014: only 500 days before "Climate chaos"
2019: Hey Greta, we need you to convince them it's really going to happen this time
2020 Greta Thunderberg
yeah early days focus on shear volume of data and results, and fine tuning and optimization often comes later... but its very important as you noted. how do we reduce the processing overhead exactly?
"Thank you for sounding the alarm. I’ve been screaming about this without a platform. I hold a masters in a sort of MBA Sustainability Leadership from Arizona State and work in deep tech and am loosing sleep over it. "
That's a you problem I'm afraid.
@@esecallum Yeah. People worry too much.
Great job.
Great video!
There’s lots of custom models, but there’s a lot more RAG/vector db stuff
Great video as usual, but please fix sound. Listen and compare with several other videos on youtube for side by side comparison. Of the hundreds of videos I watch a month, these consistently have the worst sound.
It might make sense to mandate by regulation that all new data centers install solar and wind with batteries
I think in the long run, A.I. won't consume as much energy as we think it will do.
I suspect as A.I. becomes more capable, more of us and more businesses will want to run A.I. at a local level for privacy and security reasons, especially as A.I. becomes more capable and useful, which will likely push more consumers and businesses to not want to use online A.I. services, and that will only get a lot worse as A.I. becomes more useful and even worse once A.I. can have a long term memory so it can change, adapt and learn of our habits.
A.I. running at a local level will likely get absorbed into the energy our computers use, and likely just use a little more than what we do now, A.I. isn't what we should be worried about when it comes to power use, it's robotics that could use a heck of a lot more energy than we use today as that won't be absorbed into our current system of computers and will be thrown onto our electrical grid like EV cars are.
With that said, there's no real way to know, reason being is that corporations and governments might go crazy with A.I. at such a level that needs an insane amount of power, but I think for personal and corporate use, I think the energy use will be minimum and just a little higher than what we already use, basically, it will be absorbed into what we use as tech gets better and able to do A.I. more effectively, but there is a lot of unknowns in all this as we don't know the limits of how far we'll push A.I. and how far we can push it before there's a drop-off in quality that pushing harder isn't giving us much better results but ends up needing far more energy that it's not worth it.
With all that said, we humans will always live within the means of how much energy we produce and will likely use more if we produce more, which renewable energy over the long run could produce far more energy than we use over the coming decades.
In any case, we should remember that unlike computers or the internet, which was adding a lot of energy use didn't exist before because we didn't have computers or the internet, A.I. is just another service being thrown onto what is already there, so yes there will be a spike in energy use, especially early on, but I think long term it will be absorbed into our computer use and not make that much of a difference, but robotics on the other hand is another story entirely and is a bit like EV cars, so EV cars and robotics is what's really going to have a massive spike on our energy use.
brilliant analysis, a very valid exploration. 28k terra watt hrs. Aye Eye -
energy use is the sign of a developed nation. The more energy a society uses the more advanced it is. Energy use only goes up, never down.
I'm interested to find out how much energy we have saved over the past 50 years by switching from cathode ray tubes and vacuum tube technology to flat panel IC based technology for the item that we buy hundreds of millions of and that is TVs. Let alone the cost of the energy used to ship all of those heavy glass objects around everywhere. So we have saved countless amounts of money from one sector and put it into this centralized sector which is capable of doing things we only dreamed of 50 years ago
Id love to see you do a deep dive into Musk smashing through the coherence wall and how he built it so FAST!
most of the things people think they want to use AI for is like asking a chef to make toast with a Volcano
its a massive waste of power for most things
We already have efficient methods of doing those things. AI should not be used for most things
Yes, I want to know what we can do with the waste heat. Maybe a lot of thermoelectric devices in the cooling towers I used in my research paper.
I would love to see a full video on non matrix multiplication ai, if it's not just a pipe dream.
I asked that microshite copilot why we are using all this processing power on AI instead of, say, cracking the nuclear launch codes. It said that was an innapropriate question, I promptly deactivated it off my computer...