Thank you for watching! We’re writing all the time at work whether it’s emails, drafting up video scripts, etc. but having a tool like Grammarly will help improve your productivity and work more efficiently! It’s FREE, why not? Sign up for a FREE account and get 20% off Grammarly Premium: grammarly.com/LTT
Are kitten/cat claws hard enough to scratch LCD screens? For SCIENCE! you must do a video on this! plus the internet loves cat videos so think of that sweet sweet ad revenue :)
Hey! I work in the aerospace engineering field and we recently purchased about 12 xenowulf servers each with 5 GPUs for simulation work. They cut down our fluid works simulation times from HOURS to single digit minutes. We used to have kind of like a whiteboard schedule/queue for when someone wants to use the servers, now it’s just hit send to queue, even with a team of 50 or so people using it, you can go make a coffee, come back and the data is ready to be analysed and looked at. Absolutely absurd
Thank you very much for the insight from someone that actually works with these machines. I'm just the guy that puts them in the rack and gets them ready so it's great to hear some user experience instead of only being able to appreciate the hardware.
Why would you admit to that???? LOL After all, If you were to have told your bosses it only cut the time by 1/2, and had some Spare time to do other stuff... Or, at the very least, you could start looking like Miracle workers... Boss : How long to finish this? Tech : Looks like about 3 hours... Boss : Damn it.. I need it for the conference call in 2 hours. >1 Hour 50 Minutes later< Boss, We did some >insert Mumbo Jumbo< and got that thing done for you just now.... {in reality, you had it done long enough ago to have a coffee and donut break, as well as go to the bathroom.....}😋
23:53 So I actually work for an animation studio. We do use gaming components for individual machines simply because its cost-effective. When it comes to studio-wide rendering though, Epycs are the way to go
Similar here, we have a big mix of systems that were built at different times, but most of the artist workstations are single xeon + Quadro with a ton of RAM. Render farm is some of those, including idle workstations, and a smaller number of beefier systems for the really heavy renders and sims. Realistically, most renders can be completed on a normal workstation in a reasonable amount of time, so the path to speed is parallelization, not faster individual systems. Some of the fastest servers are actually split into multiple virtual render nodes to run multiple renders in parallel (especially for 2d comp renders) As they noticed here in some of the benchmarks, startup time can take longer than the render time on a very powerful system, as it can't be scaled across very large numbers of cores as effectively as the sampling process. So beyond a certain point, it's faster to stack multiple renders than it is to dedicate the whole machine to a single render at a time.
Amazon web services. Some or most VFX companies do have their own data centres/servers but cloud is used where bandwidth permits. For example I believe Weta sent the data for Avatar 2 to a number of Australian based AWS centres to be rendered as Weta's own render server wasn't up to task.... Or at least not while working on other projects at the same time. I have heard it was about 18PB of data.
@Linus, if you wanna check the pure render without the assets loading time (on Blender), got to Render properties> Performance> Final Rendrer and enable Persistent Data, so the next time you ll run the benchmark, they will be loaded already (hurts the ram a liitle bit)
For machines like this you should really consider benchmarking Neural Network training, that's one of their major use cases and a place where you can truly see the scaling
9:17 In the automotive worls, we actually "stack" "heat exchangers" all the time. Radiators, AC condensers, oil coolers and even front-mount intercoolers all get stacked quite often. Works just fine.
@@Pippy1 Yeah, no. I just can't possibly believe that when you're driving down the highway with 75mph of airflow passing through your radiator, that stacking them would change the efficiency in any meaningful way. Prove me wrong.
@@jonny6702 air seeks the path of the least resistance. Whether at 10mph or 200mph, if the resistance of all the stacked heat-exchangers is too big, the air will simply go around and find another way. ;)
In Animation, when dealing with these kinds of rendering speeds you'd probably want to optimize the scene and make sure that the 3D assets are cached in memory, so they don't need to be loaded again and again for every single frame. Once the assets are loaded and cached, you I only need to wait for the actual rendering to finish, so even the 60s gooseberry render would drop to 15sec l
@@Kitty-oy5nj no, it's correctly designed. 3d models, shaders, particle systems, simulations , lighting conditions, those are all things that CAN change from a frame to the next. Part of the science of optimization is knowing what can be baked and what can't. An example are simulation files. You would usually simulate on houdini, import as a sequence of .vdb files and for every frame, there's a new file that needs to be loaded. Maybe you can cache the whole sequence on memory, if you are dealing with servers like this, but if you are a peasant, one of the biggest factors for render times besides the render itself is the reading speed of your storage
@@Kitty-oy5nj There is an option to load the full scene to VRAM but the cost is increase in VRAM usage. VRAM is mission critical in GPU rendering and hence is in the interest of the artist to conserve it.
The fan controller likely looks at component temperatures and adjusts pump speed as well. When components get hot you need to increase flow rates but may not need to increase fan speed. Fan speed would be entirely dependent on the coolant temperature.
Now that'd be interesting. I do know they have someone in The Lab who's working on a really intence GPU test, but that's more for testing singular GPUs for gaming.
I absolutely love watching you guys discover enterprise grade hardware/ways of doing things in recent videos. Something to consider (talking about that fan controller not being able to interact with the system directly) - management connections/items are usually handled in a completely separate network section (considered "Out Of Band", whether it be separate switches or just separate vlans). Definitely helps separate data traffic from management traffic and potential vectors. Please keep up the amazing content!
Quite! The guys and gals using it don't care about the actual hardware just that it does the job they're using it for, whereas The Admin' actually looking after the hardware don't care what it's used for.
@@RoryStarr Completely wrong. It doesn't matter how many amps the power supply is capable of providing, you need voltage to drive current through a given load. A car battery can easily deliver 800A while cranking, but it will never give you even the slightest shock, in the same way that it will not only _not obliterate_ a 230V light bulb (which can only handle 500mA at most), but in fact won't even get anywhere close to _turning it on._ Something like a capacitive dropper connected to a 230V supply might only be able to deliver 50mA, but it will still kill you because 30mA are enough to stop your heart.
You won't get a shock, but Linus was right - heating is current squared times the resistance, so you drop a screw in the wrong place and there will be a lot of hot. I have seen a spanner melt like a fuse on a big 24v truck battery when it hit bodywork while tightening a bolt on the busbar - welded itself to the bodywork then just melted. Not great for the battery.
@@demoniack81 I don't see how your description equals "completely wrong". A MOSFET can have extraordinarily high voltage and do nothing to a person. I'll agree that it's an interplay--both matter.
23:52 The reason gaming hardware mostly doesn't cut it is VRAM. These pro GPUs can hit hundreds of Gigabytes Nvilinked while gaming hardware will max out at 48GB with 3090s. VRAM is critical for GPU rendering as production scenes can hit 150GB + in VRAM.
luckily many renders support out-of-core now, where they can partially render scenes and rely on regular ram or even disk memory for scene storage. it comes with a substantial performance penalty though.
@@mitlanderson They did not specify but i know they got sponsored by either Google or Nvidia, they maybe provided acces to a render farm etc. But i dont think they used rtx quadro.
That small 90 degree tube is probably an automotive part, that comes bent 90 degrees from the factory. (if so it's synthetic rubber reinforced with nylon string) I was disappointed that you guys didn't raid the cooling system parts bin at Princess Auto or Canadian Tire before your car radiator PC video, there would have been parts there that could have joined your hoses up without leaks,
Stacking radiators doesn't work because at some point the air temperature in the stack equalizes with the water temperature. Especially if the radiators are running in serial. These 3 appear to be running in parallel so the water temp won't drop off and those fans are probably in the 3000 rpm range which means there is so much airflow screaming through the radiators that the air has no chance to get heat soaked.
well i think its "it doesn't make it any cooler" but if you need more surface area in that configuration its not any hotter and takes up far less space.
When rendering in blender you can press "render animation" and all the frames after first one are gonna be showing pure gpu perfomance without waiting for assets to load.
Might have missed something, but as far as i can see the design of the machine forces air in from the front and pushes it to the back. Benchmarking it with the side panel off seems to me to mess up the cooling performance? Edit: Nevermind, saw the side panel go back on at 26:50
"I can't believe you've done this", how dare you make my dreams look small 😆. Love the vid, industry hardware is my thing and this is one of the few times you guys make a vid on that topic that doesn't directly relate to LMG.
I just LOOOOVE these kind of uber crazy computers. That 46 degree value is still wigging me out. What a piece of engineering. Thanks for the video; I really enjoyed it! Big thumbs up.
6:50 I've been using EK's "old" aluminum 240mm fluid gaming kit since like 2017, and upgraded it to 240mm + 120mm, and even upgraded to ryzen from my old core i5 2320 that i started overclocking using this kit. It's been amazing all these years, and i change out the fluid once a year, havent had a problem with leaks, overheating, or any other problems you could have with old watercooling setups
I've been in enterprise for a long time, and what everyone needs to remember is in a Data Center facility you have cold and hot rows so temp management is even better. Ecstatic to see water-cooling becoming mainstream for servers.
23:56 the reason why they wouldn't deploy gaming PCs is (apart from the reliability thing) that Gaming PCs typically don't have the memory capacity to hold the rendering assets in system memory. Iirc just the texture size of the Millenium Falcon that was used to render the Star Wars Sequels was about 400 GB. The performance of the PC would be abysmal if it had to render something like that, because it would have to swap to the SSD.
you could just build a thing with 1 gpu and lot of ram, but that true they must be a ratio on "cost of one machine" versus "cost of mega system with lot of gpu but shared CPU motherboard ram etc".
@@Kitty-oy5nj No! Actually not. You still have to model and texture things realistically and believable even when you blur them. If you don't and then blur, it can look uncanny, fake, bad quality. So yea, making it actually look real is necessary. (I got that from a Corridor Crew video where they had a VFX artist from Iron Man 2(?). There was an underwater scene with tons of rubble and that rubble was all very blurry because of the murky water, but they still had realistic rubble, even if you couldn't see it)
the whole "frames per second" thing for animation studios is mind boggling. for reference, it took on average 29 hours to render each frame of monsters inc. 29 hours. to get 1/30th of a second of footage.
@@DistrosProjects they can be done in parallel. Similar to how you'd set up this machine that can do something like 400 frames per minute, you'd have multiple of them to reduce the total render time down.
I don't think your numbers are quite right. Monster Inc is 1 hour 36 min long. That is 96 min or 5760 seconds long. Assuming it was animated a 30 fps then that is 172800 frames to animate. If each one took on average 29 hours to render, then it would've taken 5011200 hours to animate fully, and that's over 572 years.
i have one of those vga to hdmi cables, the usb is both for power, and audio. you need to install the drivers to get it to work properly, and the actual connection inside the cable is weak as heck, so keep the cable itself immobilized, and crap unless you want your video to cut out all the time.
@@josephjoyce2833 and after it frizzes out a few times windows joins in and starts having errors with it, disabling it until you completely unplug it, uninstall its drivers, and re install them, then plug it back in.....like its loads of hastle, i had to get a new one, to replace the old one, and i just kept it wire wrapped up, and protected it further with a few layers of duct tape, so now it less of a short cable, and more of a fat, squat brick, and when i clean, i just turn everything off for a few minutes.
About the aluminum fins. Copper is a much better conductor of heat, but aluminum has better abilities to wick heat away into the air. So in theory the best possible heat sink or radiators would be either aluminum or aluminum coated copper.
12:00 in the walls of this tubing is fabrik wich helps it with noch stretching and ripping. It also is used in Automotive uses and sometimes as tubing for presurised air on trucks/lories.
The look on Linus' face when he found out that Nvidia can be easy to work with hahaha (I wouldn't believe it either). Also, Ah EK knows Linus too well, sending him it with Windows 10.
@@vicmac3513 this isn't a server they're using, they sent it to him with Windows on it because that's what they run all their benchmarks on. All of their file servers are Linux AFAIK
@@vicmac3513 They also directly state in the video that it can be loaded with linux or windows server depending on consumer choice. They just wanted their hardware back in one piece after ltt was done.
Nice. I always come back if i have the feeling i did something good. This brings me down to earth and reminds me my PC is utter crap and i have to be ashamed of it
yknow when something is so good at a thing that it seems unimpressive how good it is. that's how this felt, no issues, just performance. good on ek for making their perfect server boring. no, really.
About the flexible tubing that holds its shape in tight bends, it's not a new concept at all, though applying it to computer water cooling might be, and it makes me think Alex wasn't joking about using automotive parts. That's actually pretty typical of automotive coolant hoses, and sometimes other fluid hoses, which is why you often need to buy a specific part rather than just get some appropriately-sized tubing and cut and bend it to fit if the hoses in your car needs replacing. Just ask me how much fun that is when a transmission cooler line springs a leak... Anyway, they tend to be multilayered, with synthetic rubber on the outside and polyester fiber reinforcement pretty much in the middle, but I'm not entirely certain what goes into the other layers.
You should go to a production studio that uses these kind of machines and ask them how they are improving their work and lives (less time spent rendering = more free time)
The answer is not at all. Visual artists in movie, just like game studios are whipped into 24/7 crunch time. Having zero downtime between renders sure as hell doesn't make their life more comfortable.
I can give you a rough benchmark. Even five years ago, studios were still committed to pure CPU render farms utilizing (on average) 10,000 cores to do runs. When the GPU Render Boxes started coming along, usually five units daisy chained together, we did a prior sequence that previously took 4 months to do on pure CPU and completed it over a weekend! It was still a lot of work in scaling texture maps and lots of PBR light rigs to fit within the confines of available GPU ram, but the Massive increase in rendering efficiency blew my mind. This kind of setup would be run by several render wranglers, each tasked with doing a set of runs with each utilizing a central rack. Larger studios would have 10 of these with dedicated cooling rooms that hold the ambient air temp below 50*F, even in the max heat of a Texas summer for instance. You would have to wear a winter coat to do your setups.
“It’s only 12 volts…” My dude, back in the ancient early days of digital watches, I’d use a 6v lantern battery to power two sewing pins… Shorting them out would generate enough heat at the tips to do the tiny soldering required to fix the buttons. With the right load, you can start a fire with just about any power source. :)
Yup. I don't know much about electricity, but I know that volts mean very little. It's amps that kill. That's why things like static shock, which can generate up to 25k-35k volts, just pops a little. That being said you do need both to be lethal, and 12v isn't enough to really conduct through skin effectively, even with high amps.
Copper/brass coolant systems are actually the middle budget option. Aluminum fins can be much thinner which results a much higher surface area for the radiators. You can also use a much thinner layer of metal on the bottom of the water block. Copper is used because it allows for cruder manufacturing. What you often see in consumer PC parts is thick machined aluminum which is worse than copper.
I love the way Linus says "check this out" like he knows he's about to blow your mind with some cool thing he just noticed about whatever project he's working on
People don't understand how much of a difference this is for VFX peeps. This is one render of one frame. In TV they usually use 24 frames/sec. So being 1/3 - 1/4 faster is a HUGE deal. I am super excited that GPUs are starting to come down. I have been waiting forever to get one so I can speed my renders up.
9:50 If you think about a pump pressure-flow curve, they are more or less a straight line between the max pressure 0 flow, and max flow 0 pressure. When you put them in series, you scale it vertically, and when you put them parallel you scale it horizontally. The system will also have a pressure-flow curve, but tilted the other direction, where it starts at 0 pressure 0 flow, and as you increase flow the required pressure increases. If you superimpose the pump and system curves, where they intersect is the flow you get. It would depend on how restrictive the system is as to whether parallel or series would result in higher flow. Another consideration is with series, if one fails, it can pump through the other one, but in parallel, you need to add check valves so the flow doesn't short circuit if one fails. And as for stacking radiators, it all has to do with the airflow. If you are limited by the airflow, ie, using normal PC fans, the air will reach near water temps overt the first rad, and then beyond that it can't pick up much more heat, but if you got the industrial finger chopper fans, then there is sufficient mass flow of air that it doesn't heat up too much as it passes the first rad, and therefor the 2nd rad still has a good TD and can transfer a decent amount of heat.
Alex- "You can't hurt yourself with 12 volts." True, provided you have no cuts or anything, but breach the epidermis and you can indeed hurt yourself with 12 volts.
LOL no. You could hook this up to your heart directly and it wouldn't kill you. The lowest voltage that has ever stopped a heart in clinical trials is 42 volts. Skin is a major defense against electricity but such a low voltage is safe even if you have a major breach in your skin. Need at least 30 volts to cause damage. Edit: I do want to add that such a thing as a capacitor exists and not all 12v things are safe to poke around in. A capacitor can store energy it is receiving and output it in higher voltages.
@@Nilboggen no mate, the pwoer supplies are rated in the KW range, that's 150 amps, at 12 volts it wont stop your heart it will boil it in the sac in short order
@@AnIdiotAboard_ ssshhhhh Darwin would be ashamed. Allow the bolted egos to kill themselves. They may take your comment and learn voltage is irrelevant. Amps are the problem
Well, putting pumps in series does result the sum of pump heads if the flow remains the same. However, to get the performance in a real-life system, you need to follow the second stage pump curve to the point were it meets the system curve. That will yield a higher flow and lower head.
I know Linus owns the company, but his team needs to have a tech handling intervention for him. His handling of expensive tech gives his viewers hives.
It’s what happens when your surrounded around that hardware constantly. I mean honest, watch a skilled mechanic handle tools and parts on a regular basis. A more bad example is how police officers seem to be worse at handling guns than most civilians. With constant exposure comes more mistakes
@@goldie6961 It's true. The more "second nature" a task becomes or more "muscle memory" you get from doing the task, the lazier your brain gets. When you barely ever do something, you have to think about doing it.
FYI, direct soldering and thus bypassing connectors are much better for the signal integrity as well as getting rid of potential connector-to-board oxidation issues in the future. That is also one of the reasons professional boards use tinned pads instead of gold plated: copper and tin is less prone to cause issues like micro cracks etc. (complex metallurgy). So really, it is good.
@@napnapnop Electronegativity - Cu (copper): 1,9. Sn (Tin): 1,96. Au (Gold): 2,54. and silver (Ag): 1,93. The more dissimilar materials are the more prone to issues. This was observed by an engineer who worked with audio circuits and boards for decades, the gold plated boards had a higher tendency to develop oxidation between the copper and gold vs the copper and tinned pads. In general. Differences between hard gold and ENIG could exist but is unknown. We are most likely not talking absolutes here, more rule of thumb, and besides, tinned boards have a tendency to last for a loooong time :)
You really need to show more interesting tests/demos with these monster type of machines. Renders of blender scenes, or someone modeling and rendering something else than these default 3D scenes, or deepfacelabs, or how about the new open source Stable Diffusion AI. Anything that can leverage this type of hardware. Real world entertaining stuff. I still enjoy these videos anyway. But they need more "flair" imo.
We wanted to use Topaz but getting it running well is a huge pain. I found when I tried to use more than 2 GPUs it would change to CPU only. Simply didn't have the time to get it working well enough to compare.
@@alexanderclark9719 lol yeah Topaz Video Enhance AI is, unfortunately, not very optimized for more than one GPU with one instance of the application. Hopefully the v.3 will be better and will be able to leverage all the gpus, for "real world" testing. Runing 2 or more instance (if the hardware is there) is often something people suggest to do. The video was interesting anyway :)
Coming from the automotive world, seeing aluminum radiators and formed rubber hoses is completely normal haha. Aluminum conducts heat very well and dissipates it very fast too, basically all high performance automotive radiators are aluminum. They have to keep an internal combustion engine cool at temps above the boiling point of water, meaning the hoses have to hold at least 1 bar of pressure as well. So basically all cooling hoses or heater hoses are reinforced with either steel, nylon or kevlar mesh. That hose at @11:30 looks like any old automotive heater hose you'd see coming off a thermostat housing on most japanese cars from the 80s on up, kinda neat to see!
I would love to watch one of the engineers reacting to this video. With video like these I allways keep imagining the pain in the face of the engineers
@@LainK1978 As one of the engineers, when he started touching the Fan Controller with no ESD strap I had to bite my lip and not weep a little. Looked like he touched right on the aluminum electrolytic capacitor for the 12V rail.... ok everything has ESD protection but really ... really ... This video was great and alot of fun also the Xenowulf machines across all the product lines are amazing. The Xenowulf lab is GEEK HEAVEN.
Can't answer the first question, but they do not currently use solar panels for any of their offices(Linus does have it for his own home though). They've expressed interest on the WAN show podcast about installing some, but it did not sound like they had any active plans to actually do it yet.
@@RobinSylveoff I'm not sure if solar panels would be the best for them. Electricity in BC is mostly hydro generated, meaning that it's relatively cheap, so it doesn't make much financial sense considering how long it would take for it to "pay itself".
@@hueanao Do you really think they'd turn their roof into a solar farm without it being sponsored? It would end up costing them very little, and they now own a fairly large, flat-roofed industrial building they can use.
@@megachonk9440 true, but still, the cost for the partner company would be A LOT higher than just paying for a sponsor spot or even a sponsored showcase, so I'm not sure if it makes financial sense, even more when you consider that the solar industry isn't that distance-friendly, meaning that it would be too expensive for a viewer from another province to use the services of the sponsor, so the cost of sponsorship would be a lot while the relevant viewership would be relatively small. Maybe it would make sense for a B2B-focused company, since they can charge a reliability premium, but then again, hydroelectricity isn't expensive enough to make sense. But oh well, enough of my rambling. Would be fun seeing a solar panels-focused LTT video, specially if it was a bit more in the technical side, but I don't think we'll be seeing it anytime soon.
11:57 Thats a molded 90 degree elbow. that's an automotive product, used on coolant systems where they need to make a tight bend you can probably find some if you look around
As someone who does regularly use Blender, it's absolutely MIND BLOWING how quick those renders are. I'd loooove to finally do some big physics animations and complex stuff - but man, my 1080Ti / 1070 combo can only do so much hah.
When you feel heat from something it's because it's transferring energy to you... So the fact that copper feels hotter may actually be _because_ it's better...
it might be better but in the use case, none of the temperatures got anywhere near hot and maintained that cool level regardless of how hard they hit it. so using anything "better" would be excessive and yield no useful change while increasing cost.
Stacking radiators does in fact work just fine as long as there's enough airflow. I have a system that runs on 2 single-120 radiators sandwiched around a single noctua ippc-3000 fan. It holds temp just fine but does get a bit noisy if it's running hard.
that's totally insane that it can render multiple frames of animation per second. on toy story 1, each frame took more than 24 hours to render. with this machine, that whole movie could render in under 6 hours. that's nuts.
I think in recent movies it takes a bit longer, like 1 frame = 25 years on a average computer longer ruclips.net/video/-TQb-6vZ9h4/видео.html 12min mark
the psu setup is called a power distribution board, and its actually not new at all. if you look at some dell workstations like the precision t7600 from 2012, it has a dual psu redudant setup kind of like this server and has the same kind of power distribution board to supply cpu power, 24pin atx, and vga cables for pcie
Track/Race Cars use aluminum radiators, and a good brand radiator for cars is MISHIMOTO. Kell Radiator Service says, "Aluminum is strong and durable, which makes the radiator less likely to crack or bend, even when subject to high temperatures. Aluminum is less susceptible to corrosion and rust. Aluminum conducts heat well and absorbs it faster, which allows the hot coolant going through the tubes to cool more quickly". A normal operating temperture for a car engine is 195-250 Fahrenheit. A high performance CPU runs up to 205 Fahrenheit. It's crazy how the heat coming from the CPU alone is almost as hot as a car engine when you're flooring it at high speeds.
Hey LTT, if you like that controller for the water cooling, check out the Aquaero 6 LT ... same idea (onboard SoC running things) and Aquasuite allows you to do some funky tuning of fans and pump based on sensor inputs.
This video is so amazing. It peek into what computer / computational works business operations spend around 50-100k USD. I can ballpark mechanical manufacturing machine. But tech beast, is another beast I never know!
I neeeeed this for Redshift rendering. FYI “hybrid” rendering, CPU & GPU is still very new, much slower, and not really supported, yet. Those render times are about right, not 300 per/sec like stated in the beginning. You could get better render times if you were able to adjust the samples and run a denoiser and other settings. Benchmarks just slam the settings, something you wouldn’t do in the real world.
I'm pretty sure he said something about renders per minute, not seconds. I reacted to that number as it would mean 5 renders per second. But to be fair it all depends on what you are rendering and that was not specified. The complexity of the scene makes a huge difference so unless you are comparing the same software rendering the same scenes the numbers aren't really saying much. It's like stating that a PC runs games at 200 fps and not mention resolution or what game was tested.
There is some wisdom in not building your entire heatsink out of copper, you want to use the copper as an interface between the GPU/CPU and your heat pipes or fins, not to the area around hot part on the circuit board
Thank you for watching! We’re writing all the time at work whether it’s emails, drafting up video scripts, etc. but having a tool like Grammarly will help improve your productivity and work more efficiently! It’s FREE, why not? Sign up for a FREE account and get 20% off Grammarly Premium: grammarly.com/LTT
Great watch as always.
By the way are we going to get Linus Lift Tips anytime soon? His arms look ripped and I'd love to know his workout routine
@@spookwagen-thegreat1350 badminton
@@spookwagen-thegreat1350 Linus Workout Tips
Are kitten/cat claws hard enough to scratch LCD screens?
For SCIENCE! you must do a video on this!
plus the internet loves cat videos so think of that sweet sweet ad revenue :)
But can it run Crysis
Hey! I work in the aerospace engineering field and we recently purchased about 12 xenowulf servers each with 5 GPUs for simulation work. They cut down our fluid works simulation times from HOURS to single digit minutes. We used to have kind of like a whiteboard schedule/queue for when someone wants to use the servers, now it’s just hit send to queue, even with a team of 50 or so people using it, you can go make a coffee, come back and the data is ready to be analysed and looked at. Absolutely absurd
Thank you very much for the insight from someone that actually works with these machines.
I'm just the guy that puts them in the rack and gets them ready so it's great to hear some user experience instead of only being able to appreciate the hardware.
HOURS TO MINUTES??? HOLY SHIT THAT'S FAST
Why would you admit to that???? LOL After all, If you were to have told your bosses it only cut the time by 1/2, and had some Spare time to do other stuff... Or, at the very least, you could start looking like Miracle workers... Boss : How long to finish this? Tech : Looks like about 3 hours... Boss : Damn it.. I need it for the conference call in 2 hours. >1 Hour 50 Minutes later< Boss, We did some >insert Mumbo Jumbo< and got that thing done for you just now.... {in reality, you had it done long enough ago to have a coffee and donut break, as well as go to the bathroom.....}😋
@@scottstewart3884 and then the boss finds out and fires you........
@@scottstewart3884 smart people dont lie to boss, they enjoy the work they do and are happy to do it :)
I think that 90 degree tubing is made as a 90 degree, like you find in a lot of automotive. straighening it out would be bending it.
Was gonna say that.
Same here was also going to say that
Yep, molded hose.
I was gonna say it looks like a radiator hose
Literally just came down here to comment this XD
1:01 That countdown before Linus knocks something off the desk... Great editing work there.
23:53
So I actually work for an animation studio. We do use gaming components for individual machines simply because its cost-effective. When it comes to studio-wide rendering though, Epycs are the way to go
@1 Million With 0 Videos Challenge (Day 20) I didn't lolll
Similar here, we have a big mix of systems that were built at different times, but most of the artist workstations are single xeon + Quadro with a ton of RAM. Render farm is some of those, including idle workstations, and a smaller number of beefier systems for the really heavy renders and sims. Realistically, most renders can be completed on a normal workstation in a reasonable amount of time, so the path to speed is parallelization, not faster individual systems. Some of the fastest servers are actually split into multiple virtual render nodes to run multiple renders in parallel (especially for 2d comp renders)
As they noticed here in some of the benchmarks, startup time can take longer than the render time on a very powerful system, as it can't be scaled across very large numbers of cores as effectively as the sampling process. So beyond a certain point, it's faster to stack multiple renders than it is to dedicate the whole machine to a single render at a time.
What stuff do you animate ?
What stuff do you animate ?
@1 Million With 0 Videos Challenge (Day 20) Ok, I won't!
I remember when EK started and were advertising their water blocks on a local Slovenian forum. Crazy to think how far they've come.
Yeah, with cellphone camera images of brass fittings and stuff :).
Absolutely amazing seeing them today.
At the time the general public perception around PC water cooling wasn't even "enthusiast", it was more "black magic toxic fumes inhaling crazy".
And now their patent of a superbasic watercooling parts blocks all competitors from competing with them in NA.
@@therodyman700 Are you sure you’re not talking about Asetek? They’re the one with the patent.
@@domchu4897 Asetek has the patent on AIO pumps, I don't know about patents on other water-cooling parts so he could be talking about something else.
Any chance y'all could do a follow up with like a dreamworks or other studio that works with machines like this? This is insane!
Dreamworks etc don't generally use many GPUs. The scenes are too large to fit into VRAM.
Amazon web services. Some or most VFX companies do have their own data centres/servers but cloud is used where bandwidth permits. For example I believe Weta sent the data for Avatar 2 to a number of Australian based AWS centres to be rendered as Weta's own render server wasn't up to task.... Or at least not while working on other projects at the same time.
I have heard it was about 18PB of data.
@@Outland9000 Genuinely at the scale were "sending a truck full of HDDs" might be faster
@Linus, if you wanna check the pure render without the assets loading time (on Blender), got to Render properties> Performance> Final Rendrer and enable Persistent Data, so the next time you ll run the benchmark, they will be loaded already (hurts the ram a liitle bit)
Not that they don’t have enough RAM
@@kuttispielt7801 XD
Since taht comment got some friction, IF you want to stretch the render, go render animation and not render image (ctrl+f12)
Yes, most of the GPU render time will be transferring data in to VRAM. The actual CUDA calculation will be a fraction of that.
@@BurnsRubber also studios use a6000 for many reasons including use optix and not cuda, but that's another story
11:18 These tubes are "molded hoses" and often used in the automotive area. They are produced in this shape and keep it forever.
Was about to say, looks a lot like molded automotive hose for heater cores.
For machines like this you should really consider benchmarking Neural Network training, that's one of their major use cases and a place where you can truly see the scaling
9:17 In the automotive worls, we actually "stack" "heat exchangers" all the time. Radiators, AC condensers, oil coolers and even front-mount intercoolers all get stacked quite often. Works just fine.
Yes and no, u tend to lose a lot of efficiency because air doesnt want to flow through all of them but there isnt much of a choice with cars
@@Pippy1 Yeah, no. I just can't possibly believe that when you're driving down the highway with 75mph of airflow passing through your radiator, that stacking them would change the efficiency in any meaningful way.
Prove me wrong.
@@jonny6702 air seeks the path of the least resistance. Whether at 10mph or 200mph, if the resistance of all the stacked heat-exchangers is too big, the air will simply go around and find another way. ;)
just don't integrate radiators like ford australia did in 2003 falcon range...
Heavily depends on air flow and the desired temperature of each radiator.
In Animation, when dealing with these kinds of rendering speeds you'd probably want to optimize the scene and make sure that the 3D assets are cached in memory, so they don't need to be loaded again and again for every single frame. Once the assets are loaded and cached, you I only need to wait for the actual rendering to finish, so even the 60s gooseberry render would drop to 15sec l
Also baking textures should give a massive boost in performance
reload per frame ? is it that badly coded
@@Kitty-oy5nj no, it's correctly designed. 3d models, shaders, particle systems, simulations , lighting conditions, those are all things that CAN change from a frame to the next. Part of the science of optimization is knowing what can be baked and what can't.
An example are simulation files. You would usually simulate on houdini, import as a sequence of .vdb files and for every frame, there's a new file that needs to be loaded. Maybe you can cache the whole sequence on memory, if you are dealing with servers like this, but if you are a peasant, one of the biggest factors for render times besides the render itself is the reading speed of your storage
I think the animation studios know what they are doing…
@@Kitty-oy5nj There is an option to load the full scene to VRAM but the cost is increase in VRAM usage. VRAM is mission critical in GPU rendering and hence is in the interest of the artist to conserve it.
The fan controller likely looks at component temperatures and adjusts pump speed as well. When components get hot you need to increase flow rates but may not need to increase fan speed. Fan speed would be entirely dependent on the coolant temperature.
LTT actually can get some projects to render VFX from corridor digital if they down to share or make collab to create some heavy renders.
Now that'd be interesting. I do know they have someone in The Lab who's working on a really intence GPU test, but that's more for testing singular GPUs for gaming.
@@toseltreps1101 Look, if there is anything I learned from programming, it's that you don't have to spell well, just consistantly.
Anything expensive this man touches shakes me to my core
Thumbnail is like
Ek Gold pc
I come to LTT specifically for the tech gore.
Bot nxjdbhh Bot
And Many of us watch for those moments of things been dropped
24:40
We must have got lucky, we had 20 CPUs to pick from... Want to try the 7763 with water cooling in the future. Awesome video as always Linus !
I absolutely love watching you guys discover enterprise grade hardware/ways of doing things in recent videos. Something to consider (talking about that fan controller not being able to interact with the system directly) - management connections/items are usually handled in a completely separate network section (considered "Out Of Band", whether it be separate switches or just separate vlans). Definitely helps separate data traffic from management traffic and potential vectors. Please keep up the amazing content!
Quite!
The guys and gals using it don't care about the actual hardware just that it does the job they're using it for, whereas
The Admin' actually looking after the hardware don't care what it's used for.
I specifically said *do not* cut your fingers.
20 likes 💀
Yeah, thats weird 😅
6:03 Alex: Its just 12V, it can't hurt you.
Linus: You could probably start a fire?
Alex: You gotta be trying pretty hard.
NZXT: *sweating* 👀
Also, 12 V isn't necessarily safe. It depends on the amperage and there's no way this thing isn't beefy with amps.
@@RoryStarr Completely wrong. It doesn't matter how many amps the power supply is capable of providing, you need voltage to drive current through a given load. A car battery can easily deliver 800A while cranking, but it will never give you even the slightest shock, in the same way that it will not only _not obliterate_ a 230V light bulb (which can only handle 500mA at most), but in fact won't even get anywhere close to _turning it on._
Something like a capacitive dropper connected to a 230V supply might only be able to deliver 50mA, but it will still kill you because 30mA are enough to stop your heart.
You won't get a shock, but Linus was right - heating is current squared times the resistance, so you drop a screw in the wrong place and there will be a lot of hot.
I have seen a spanner melt like a fuse on a big 24v truck battery when it hit bodywork while tightening a bolt on the busbar - welded itself to the bodywork then just melted. Not great for the battery.
@@demoniack81 I don't see how your description equals "completely wrong". A MOSFET can have extraordinarily high voltage and do nothing to a person. I'll agree that it's an interplay--both matter.
@@RoryStarr wasn't it the amount of power + enough voltage to get through the skin? Not sure
23:52 The reason gaming hardware mostly doesn't cut it is VRAM. These pro GPUs can hit hundreds of Gigabytes Nvilinked while gaming hardware will max out at 48GB with 3090s. VRAM is critical for GPU rendering as production scenes can hit 150GB + in VRAM.
Yeah i knew one viz artist needed tb of vram for project (and they actually did the project).
luckily many renders support out-of-core now, where they can partially render scenes and rely on regular ram or even disk memory for scene storage. it comes with a substantial performance penalty though.
@@msavasacar how many A6000s was that?
@@mitlanderson all of them
@@mitlanderson They did not specify but i know they got sponsored by either Google or Nvidia, they maybe provided acces to a render farm etc. But i dont think they used rtx quadro.
im looking forward to seeing this on LGR in like 25 years
This is automotive grade tubing, where tubes are molded in particular shape when needed, like that short 90° angle.
Dang, I'm very happy that the "HOLY $H!T" series are back and are released quite often. Thanks, LTT!
That small 90 degree tube is probably an automotive part, that comes bent 90 degrees from the factory. (if so it's synthetic rubber reinforced with nylon string) I was disappointed that you guys didn't raid the cooling system parts bin at Princess Auto or Canadian Tire before your car radiator PC video, there would have been parts there that could have joined your hoses up without leaks,
Corsair: *“Stacking up radiators like that doesn't work!”*
EK: *“Challenge accepted!”*
It's all a question of airflow - and Linus did say that those radiators are low fin density, so they're high airflow optimized.
Stacking radiators doesn't work because at some point the air temperature in the stack equalizes with the water temperature. Especially if the radiators are running in serial. These 3 appear to be running in parallel so the water temp won't drop off and those fans are probably in the 3000 rpm range which means there is so much airflow screaming through the radiators that the air has no chance to get heat soaked.
EK: hold my beer! 😀
well i think its "it doesn't make it any cooler" but if you need more surface area in that configuration its not any hotter and takes up far less space.
@@kevinmarsh8835 in the video he said they were running at 1600, but could max at 6000rpm
When rendering in blender you can press "render animation" and all the frames after first one are gonna be showing pure gpu perfomance without waiting for assets to load.
Might have missed something, but as far as i can see the design of the machine forces air in from the front and pushes it to the back. Benchmarking it with the side panel off seems to me to mess up the cooling performance? Edit: Nevermind, saw the side panel go back on at 26:50
"I can't believe you've done this", how dare you make my dreams look small 😆. Love the vid, industry hardware is my thing and this is one of the few times you guys make a vid on that topic that doesn't directly relate to LMG.
@1 Million With 0 Videos Challenge (Day 20) bot
Getting a similar one to this at work soon (not water cooled unfortunately :( ) but 2 Epycs and 8 40GB A100s! What a time to be alive
how much is it?
What field are you working in?
Lead times are soooo long, we sell these but not with watercooling… hope you‘ll get it very soon :)
@@dazaaaa.lolllfn way too much 😢they could hire 2 of me for a year for the price
@@gusztavferenczi4773 Deep Learning
Linus: "Come with me and check out this super expensive, highly-cooled piece of server infrastructure"
Alex:
Me: "Oh boy! "
"This is like, perfect cat bed temperature right now."
The thought of using a $70,000 server as a cat bed just cracked me up!
yeah and the thought of a cat sleeping on a server this noisy
@@thekevinager5294 noise ain't stopping no pussy
@@kylel8605 unless it’s from a vacuum
Npc ahh comment
npc
I just LOOOOVE these kind of uber crazy computers. That 46 degree value is still wigging me out. What a piece of engineering. Thanks for the video; I really enjoyed it! Big thumbs up.
6:50 I've been using EK's "old" aluminum 240mm fluid gaming kit since like 2017, and upgraded it to 240mm + 120mm, and even upgraded to ryzen from my old core i5 2320 that i started overclocking using this kit. It's been amazing all these years, and i change out the fluid once a year, havent had a problem with leaks, overheating, or any other problems you could have with old watercooling setups
I've been in enterprise for a long time, and what everyone needs to remember is in a Data Center facility you have cold and hot rows so temp management is even better. Ecstatic to see water-cooling becoming mainstream for servers.
23:56 the reason why they wouldn't deploy gaming PCs is (apart from the reliability thing) that Gaming PCs typically don't have the memory capacity to hold the rendering assets in system memory. Iirc just the texture size of the Millenium Falcon that was used to render the Star Wars Sequels was about 400 GB. The performance of the PC would be abysmal if it had to render something like that, because it would have to swap to the SSD.
you could just build a thing with 1 gpu and lot of ram, but that true they must be a ratio on "cost of one machine" versus "cost of mega system with lot of gpu but shared CPU motherboard ram etc".
and then they add motion blur to the final cut, making it pointless
@@StargateurBis well.... if the gpu takes the data from ram it's also gonna be slower
@@Kitty-oy5nj No! Actually not.
You still have to model and texture things realistically and believable even when you blur them.
If you don't and then blur, it can look uncanny, fake, bad quality. So yea, making it actually look real is necessary.
(I got that from a Corridor Crew video where they had a VFX artist from Iron Man 2(?). There was an underwater scene with tons of rubble and that rubble was all very blurry because of the murky water, but they still had realistic rubble, even if you couldn't see it)
Dreamworks released their MoonRay renderer for public use. Renderman is also avaliable since you mentioned wanting to test production rendering.
the whole "frames per second" thing for animation studios is mind boggling. for reference, it took on average 29 hours to render each frame of monsters inc. 29 hours. to get 1/30th of a second of footage.
You sure?? That’s over 500 years for the whole movie.
@@DistrosProjects they can be done in parallel.
Similar to how you'd set up this machine that can do something like 400 frames per minute, you'd have multiple of them to reduce the total render time down.
I don't think your numbers are quite right.
Monster Inc is 1 hour 36 min long.
That is 96 min or 5760 seconds long.
Assuming it was animated a 30 fps then that is 172800 frames to animate. If each one took on average 29 hours to render, then it would've taken 5011200 hours to animate fully, and that's over 572 years.
@@jeratzel they would have had entire server farms working in parallel. They wouldn’t have had to do one frame then the next.
@@Aikano9 they would run multiple servers run parallel to each other to reduce time
i have one of those vga to hdmi cables, the usb is both for power, and audio. you need to install the drivers to get it to work properly, and the actual connection inside the cable is weak as heck, so keep the cable itself immobilized, and crap unless you want your video to cut out all the time.
Booting up a system after cleaning with this cable is soo damn frustrating due to constant display error.
@@josephjoyce2833 and after it frizzes out a few times windows joins in and starts having errors with it, disabling it until you completely unplug it, uninstall its drivers, and re install them, then plug it back in.....like its loads of hastle, i had to get a new one, to replace the old one, and i just kept it wire wrapped up, and protected it further with a few layers of duct tape, so now it less of a short cable, and more of a fat, squat brick, and when i clean, i just turn everything off for a few minutes.
About the aluminum fins. Copper is a much better conductor of heat, but aluminum has better abilities to wick heat away into the air. So in theory the best possible heat sink or radiators would be either aluminum or aluminum coated copper.
12:00 in the walls of this tubing is fabrik wich helps it with noch stretching and ripping. It also is used in Automotive uses and sometimes as tubing for presurised air on trucks/lories.
I can never get enough of the Linus x Alex combo! The videos are always top tier... in a janky, but somehow it works, way - I LOVE I
the 90 degree bend is automotive coolant hose, usually used for heater core/heater valve/bypass situations
The look on Linus' face when he found out that Nvidia can be easy to work with hahaha (I wouldn't believe it either).
Also, Ah EK knows Linus too well, sending him it with Windows 10.
They promised long time ago that every LMG server will be Linux-based, but it was bs.
@@vicmac3513 this isn't a server they're using, they sent it to him with Windows on it because that's what they run all their benchmarks on.
All of their file servers are Linux AFAIK
*Cue Vietnam flashbacks
@@vicmac3513 They also directly state in the video that it can be loaded with linux or windows server depending on consumer choice. They just wanted their hardware back in one piece after ltt was done.
That has to be the most beautiful most organized backplate view I've ever seen on an OEM. Even my OCD approves
Thanks.
Nice. I always come back if i have the feeling i did something good.
This brings me down to earth and reminds me my PC is utter crap and i have to be ashamed of it
yknow when something is so good at a thing that it seems unimpressive how good it is. that's how this felt, no issues, just performance. good on ek for making their perfect server boring. no, really.
and the screaming loud volume of the cooling system, you know something is truely built for power when it doesn't care how loud it gets
everyone : uses lbs or kg to measure weight
Linus : "its really heavy" *drops really expensive device*, *thud* "see it makes that sound. "
👀👀👀
He has to give it the Linus test too though.
@@whitekong316 yeah, it wouldn't be a Linus review without a drop or 2. Lol.
About the flexible tubing that holds its shape in tight bends, it's not a new concept at all, though applying it to computer water cooling might be, and it makes me think Alex wasn't joking about using automotive parts. That's actually pretty typical of automotive coolant hoses, and sometimes other fluid hoses, which is why you often need to buy a specific part rather than just get some appropriately-sized tubing and cut and bend it to fit if the hoses in your car needs replacing. Just ask me how much fun that is when a transmission cooler line springs a leak... Anyway, they tend to be multilayered, with synthetic rubber on the outside and polyester fiber reinforcement pretty much in the middle, but I'm not entirely certain what goes into the other layers.
They don't need to run it that fast, that's just what they're getting"
Well that is basically describing every PC enthusiast ever.
Glad this series is more active again. Love the crazy stuff y'all can get
8:27 Alex frustration level = 10000
You should go to a production studio that uses these kind of machines and ask them how they are improving their work and lives (less time spent rendering = more free time)
The answer is not at all. Visual artists in movie, just like game studios are whipped into 24/7 crunch time. Having zero downtime between renders sure as hell doesn't make their life more comfortable.
I can give you a rough benchmark. Even five years ago, studios were still committed to pure CPU render farms utilizing (on average) 10,000 cores to do runs. When the GPU Render Boxes started coming along, usually five units daisy chained together, we did a prior sequence that previously took 4 months to do on pure CPU and completed it over a weekend! It was still a lot of work in scaling texture maps and lots of PBR light rigs to fit within the confines of available GPU ram, but the Massive increase in rendering efficiency blew my mind.
This kind of setup would be run by several render wranglers, each tasked with doing a set of runs with each utilizing a central rack. Larger studios would have 10 of these with dedicated cooling rooms that hold the ambient air temp below 50*F, even in the max heat of a Texas summer for instance. You would have to wear a winter coat to do your setups.
@@C-M-E why would anyone need to be in the server room to do any work? That's why we have a farm and submit tasks to it...
“It’s only 12 volts…” My dude, back in the ancient early days of digital watches, I’d use a 6v lantern battery to power two sewing pins… Shorting them out would generate enough heat at the tips to do the tiny soldering required to fix the buttons. With the right load, you can start a fire with just about any power source. :)
Yup. I don't know much about electricity, but I know that volts mean very little. It's amps that kill. That's why things like static shock, which can generate up to 25k-35k volts, just pops a little. That being said you do need both to be lethal, and 12v isn't enough to really conduct through skin effectively, even with high amps.
Love those 9volt batteries. easy to short with your tounge.
Copper/brass coolant systems are actually the middle budget option. Aluminum fins can be much thinner which results a much higher surface area for the radiators. You can also use a much thinner layer of metal on the bottom of the water block. Copper is used because it allows for cruder manufacturing.
What you often see in consumer PC parts is thick machined aluminum which is worse than copper.
Alex: *"It's only 12 volt. You can't hurt yourself with 12 volt DC"*
A wrench across car battery terminals: *"Let me introduce myself!"*
Yeah i think thesame. 4kW on 12v it alot of amps, without protection in those power suply, molten magma could Flow in seconds.
@@gacikpl It’s just 1kW per connector though, right?
@@DanKaschel dunno whats inside, but Alex pulled one power this probably sugest one 12V rail.
Remember the NZXT H1 pcie riser caught fire when its 12v line shorted only just enough to get hot but not trip most PSU short circut protections.
It is actually the current that hurts and kills....
I love the way Linus says "check this out" like he knows he's about to blow your mind with some cool thing he just noticed about whatever project he's working on
People don't understand how much of a difference this is for VFX peeps. This is one render of one frame. In TV they usually use 24 frames/sec. So being 1/3 - 1/4 faster is a HUGE deal. I am super excited that GPUs are starting to come down. I have been waiting forever to get one so I can speed my renders up.
9:50 If you think about a pump pressure-flow curve, they are more or less a straight line between the max pressure 0 flow, and max flow 0 pressure. When you put them in series, you scale it vertically, and when you put them parallel you scale it horizontally. The system will also have a pressure-flow curve, but tilted the other direction, where it starts at 0 pressure 0 flow, and as you increase flow the required pressure increases. If you superimpose the pump and system curves, where they intersect is the flow you get. It would depend on how restrictive the system is as to whether parallel or series would result in higher flow.
Another consideration is with series, if one fails, it can pump through the other one, but in parallel, you need to add check valves so the flow doesn't short circuit if one fails.
And as for stacking radiators, it all has to do with the airflow. If you are limited by the airflow, ie, using normal PC fans, the air will reach near water temps overt the first rad, and then beyond that it can't pick up much more heat, but if you got the industrial finger chopper fans, then there is sufficient mass flow of air that it doesn't heat up too much as it passes the first rad, and therefor the 2nd rad still has a good TD and can transfer a decent amount of heat.
Alex- "You can't hurt yourself with 12 volts."
True, provided you have no cuts or anything, but breach the epidermis and you can indeed hurt yourself with 12 volts.
Yes but typically the low voltage doesn’t have enough “push” to kill a person. Plenty of amperage tho. Just not enough voltage.
LOL no. You could hook this up to your heart directly and it wouldn't kill you. The lowest voltage that has ever stopped a heart in clinical trials is 42 volts. Skin is a major defense against electricity but such a low voltage is safe even if you have a major breach in your skin. Need at least 30 volts to cause damage. Edit: I do want to add that such a thing as a capacitor exists and not all 12v things are safe to poke around in. A capacitor can store energy it is receiving and output it in higher voltages.
@@Nilboggen no mate, the pwoer supplies are rated in the KW range, that's 150 amps, at 12 volts it wont stop your heart it will boil it in the sac in short order
@@AnIdiotAboard_ ssshhhhh Darwin would be ashamed. Allow the bolted egos to kill themselves. They may take your comment and learn voltage is irrelevant. Amps are the problem
@@AnIdiotAboard_ amps dont boil you lmao, you can touch 2 cables with 1 kiloamp running thruh it and you dont even feel anything
9:04 It's amazing how Canadian you get when you guys are excited. I was half expecting an "oh, fuck yeah bud!"
Well, putting pumps in series does result the sum of pump heads if the flow remains the same. However, to get the performance in a real-life system, you need to follow the second stage pump curve to the point were it meets the system curve. That will yield a higher flow and lower head.
I loved this review of this machine. You guys work so nicely together.
I have an Eheim 1046. It's been running multiple hours for 20 years in my watercooled PCs. Never had a problem, still running strong.
I know Linus owns the company, but his team needs to have a tech handling intervention for him. His handling of expensive tech gives his viewers hives.
Linus drop tips
i feel like its more of a running joke at this point rather than clumsiness. i wouldnt be that surprised if its written into his script what to drop
Speak for yourself lol
It’s what happens when your surrounded around that hardware constantly. I mean honest, watch a skilled mechanic handle tools and parts on a regular basis. A more bad example is how police officers seem to be worse at handling guns than most civilians. With constant exposure comes more mistakes
@@goldie6961 It's true. The more "second nature" a task becomes or more "muscle memory" you get from doing the task, the lazier your brain gets. When you barely ever do something, you have to think about doing it.
FYI, direct soldering and thus bypassing connectors are much better for the signal integrity as well as getting rid of potential connector-to-board oxidation issues in the future. That is also one of the reasons professional boards use tinned pads instead of gold plated: copper and tin is less prone to cause issues like micro cracks etc. (complex metallurgy). So really, it is good.
Hmm, but copper and tin are way more prone to oxidate... Apart from that, directly soldering cables instead of connectors is way better yes.
@@napnapnop Electronegativity - Cu (copper): 1,9. Sn (Tin): 1,96. Au (Gold): 2,54. and silver (Ag): 1,93. The more dissimilar materials are the more prone to issues. This was observed by an engineer who worked with audio circuits and boards for decades, the gold plated boards had a higher tendency to develop oxidation between the copper and gold vs the copper and tinned pads. In general. Differences between hard gold and ENIG could exist but is unknown. We are most likely not talking absolutes here, more rule of thumb, and besides, tinned boards have a tendency to last for a loooong time :)
For those that are curious, the usb on that vga to hdmi is for audio. VGA does not transmit audio.
EK beast server: "Do you want to see me render?"
Also EK beast server: "Want to see me do it again?"
It took Linus 15 minutes (that we've seen) to break something. Pretty impressive
Watching Linus benchmark products is akin to how I imagine gear heads feel at the drag strip watching new fast cars take some runs
6:00 Supermicro also has conversion boards for all hot-swappable PSUs in their servers
You really need to show more interesting tests/demos with these monster type of machines.
Renders of blender scenes, or someone modeling and rendering something else than these default 3D scenes, or deepfacelabs, or how about the new open source Stable Diffusion AI. Anything that can leverage this type of hardware.
Real world entertaining stuff. I still enjoy these videos anyway. But they need more "flair" imo.
I'd like to see it Topaz Lab Video AI upscale something in real time.
We wanted to use Topaz but getting it running well is a huge pain. I found when I tried to use more than 2 GPUs it would change to CPU only. Simply didn't have the time to get it working well enough to compare.
@@alexanderclark9719 lol yeah Topaz Video Enhance AI is, unfortunately, not very optimized for more than one GPU with one instance of the application. Hopefully the v.3 will be better and will be able to leverage all the gpus, for "real world" testing.
Runing 2 or more instance (if the hardware is there) is often something people suggest to do.
The video was interesting anyway :)
Coming from the automotive world, seeing aluminum radiators and formed rubber hoses is completely normal haha. Aluminum conducts heat very well and dissipates it very fast too, basically all high performance automotive radiators are aluminum. They have to keep an internal combustion engine cool at temps above the boiling point of water, meaning the hoses have to hold at least 1 bar of pressure as well. So basically all cooling hoses or heater hoses are reinforced with either steel, nylon or kevlar mesh. That hose at @11:30 looks like any old automotive heater hose you'd see coming off a thermostat housing on most japanese cars from the 80s on up, kinda neat to see!
Can't wait to see one of these kind of videos again, with a zen 4 server chip and 4000 series render cards. Then you could drag race the two systems.
I would love to watch one of the engineers reacting to this video. With video like these I allways keep imagining the pain in the face of the engineers
Eh, I think that they wouldn't be that worried about it, they have likely done worse to it already.
@@LainK1978 What they do inside the office no one cares, what Linus does in front of a camera, the world will see. It's different.
@@LainK1978 As one of the engineers, when he started touching the Fan Controller with no ESD strap I had to bite my lip and not weep a little. Looked like he touched right on the aluminum electrolytic capacitor for the 12V rail.... ok everything has ESD protection but really ... really ... This video was great and alot of fun also the Xenowulf machines across all the product lines are amazing. The Xenowulf lab is GEEK HEAVEN.
1:33 I think that USB cable is for audio. VGA doesn't carry audio but HDMI does, so the USB is for injecting audio into the HDMI cable.
Hey LTT, for our curiosity, how much do you pay for monthly power consumption between the office and the lab and do you offset that with solar panels?
Can't answer the first question, but they do not currently use solar panels for any of their offices(Linus does have it for his own home though). They've expressed interest on the WAN show podcast about installing some, but it did not sound like they had any active plans to actually do it yet.
I would LOVE to see a video of them installing solar panels
@@RobinSylveoff I'm not sure if solar panels would be the best for them. Electricity in BC is mostly hydro generated, meaning that it's relatively cheap, so it doesn't make much financial sense considering how long it would take for it to "pay itself".
@@hueanao Do you really think they'd turn their roof into a solar farm without it being sponsored? It would end up costing them very little, and they now own a fairly large, flat-roofed industrial building they can use.
@@megachonk9440 true, but still, the cost for the partner company would be A LOT higher than just paying for a sponsor spot or even a sponsored showcase, so I'm not sure if it makes financial sense, even more when you consider that the solar industry isn't that distance-friendly, meaning that it would be too expensive for a viewer from another province to use the services of the sponsor, so the cost of sponsorship would be a lot while the relevant viewership would be relatively small.
Maybe it would make sense for a B2B-focused company, since they can charge a reliability premium, but then again, hydroelectricity isn't expensive enough to make sense.
But oh well, enough of my rambling. Would be fun seeing a solar panels-focused LTT video, specially if it was a bit more in the technical side, but I don't think we'll be seeing it anytime soon.
I love alex cohost videos. He has so much passion for server grade stuff and just nerds the fuck out
11:57 Thats a molded 90 degree elbow. that's an automotive product, used on coolant systems where they need to make a tight bend
you can probably find some if you look around
As someone who does regularly use Blender, it's absolutely MIND BLOWING how quick those renders are. I'd loooove to finally do some big physics animations and complex stuff - but man, my 1080Ti / 1070 combo can only do so much hah.
If you can, get a regular 3060 for your budget because the 12 GB of VRAM would really help.
@@flintfrommother3gaming To be fair, the 1080Ti has 11GB and Blender combines the two cards for rendering for 19GB total
@@BBROPHOTO 23 is bigger than 19, hehehehe.
Watching Linus fiddle with anything, raises my anxiety level to Nine. Linus = slow motion accident.
8:18 linus about to have the drop test on something worth more than my car
When you feel heat from something it's because it's transferring energy to you... So the fact that copper feels hotter may actually be _because_ it's better...
it might be better but in the use case, none of the temperatures got anywhere near hot and maintained that cool level regardless of how hard they hit it. so using anything "better" would be excessive and yield no useful change while increasing cost.
This man knows alot about computers
And inspired me to learn about them and your videos taught me alot
Oh you got jokes.
@BrandoRando Imagine imagining 🤓
@BrandoRando imagine being an asshole just because somebody is younger than you.
Stacking radiators does in fact work just fine as long as there's enough airflow. I have a system that runs on 2 single-120 radiators sandwiched around a single noctua ippc-3000 fan. It holds temp just fine but does get a bit noisy if it's running hard.
that's totally insane that it can render multiple frames of animation per second. on toy story 1, each frame took more than 24 hours to render. with this machine, that whole movie could render in under 6 hours. that's nuts.
I think in recent movies it takes a bit longer, like 1 frame = 25 years on a average computer longer
ruclips.net/video/-TQb-6vZ9h4/видео.html
12min mark
I'd love to see them just use that cooling system on a gaming desktop to see how good it is outside of cooling servers
At the end of the day it is just normal water-cooling but on a larger scale
loud
the psu setup is called a power distribution board, and its actually not new at all. if you look at some dell workstations like the precision t7600 from 2012, it has a dual psu redudant setup kind of like this server and has the same kind of power distribution board to supply cpu power, 24pin atx, and vga cables for pcie
Every future Trainee ever in their Office: I want to be like Linus. Everybody else in the Office:Please don't!
I actually would love to see a tour in one of the big animation studios. What kind of monsters they are running.
They just use clustered servers.
Track/Race Cars use aluminum radiators, and a good brand radiator for cars is MISHIMOTO. Kell Radiator Service says, "Aluminum is strong and durable, which makes the radiator less likely to crack or bend, even when subject to high temperatures. Aluminum is less susceptible to corrosion and rust. Aluminum conducts heat well and absorbs it faster, which allows the hot coolant going through the tubes to cool more quickly".
A normal operating temperture for a car engine is 195-250 Fahrenheit. A high performance CPU runs up to 205 Fahrenheit. It's crazy how the heat coming from the CPU alone is almost as hot as a car engine when you're flooring it at high speeds.
i think a lot of these expensive PC's are dumb ... But this one here is dope AF. best I've seen thus far
Reminds me that DreamWorks opened the source code of their try tracing renderer. Maybe it could be interesting testing it out with this monster?
This thing is ripping through these tests like the grim reaper in the middle of a plague, DEAR GOD IS IT FAST!
I'd hope a $70,000 machine wouldn't be using acrylic for cooling lol
Hey LTT, if you like that controller for the water cooling, check out the Aquaero 6 LT ... same idea (onboard SoC running things) and Aquasuite allows you to do some funky tuning of fans and pump based on sensor inputs.
This video is so amazing. It peek into what computer / computational works business operations spend around 50-100k USD. I can ballpark mechanical manufacturing machine. But tech beast, is another beast I never know!
I neeeeed this for Redshift rendering. FYI “hybrid” rendering, CPU & GPU is still very new, much slower, and not really supported, yet. Those render times are about right, not 300 per/sec like stated in the beginning. You could get better render times if you were able to adjust the samples and run a denoiser and other settings. Benchmarks just slam the settings, something you wouldn’t do in the real world.
I'm pretty sure he said something about renders per minute, not seconds. I reacted to that number as it would mean 5 renders per second. But to be fair it all depends on what you are rendering and that was not specified. The complexity of the scene makes a huge difference so unless you are comparing the same software rendering the same scenes the numbers aren't really saying much. It's like stating that a PC runs games at 200 fps and not mention resolution or what game was tested.
I'm so sorry for the company that colabed with ek on this one
that transition to the sponsor was so smooth i almost slipped
There is some wisdom in not building your entire heatsink out of copper, you want to use the copper as an interface between the GPU/CPU and your heat pipes or fins, not to the area around hot part on the circuit board