I love that kioxia said challenge accepted. They weren't " that's not we were thinking" or "here's a script for this." But that's an interesting engineering project.
Kioxia had a a mishap a month or two ago and lost 600,000 exabytes of flash. I guess they are pretty good at catching up. So they lost 600 million times more than what is used here..... they are supplying the planet of flash.
@@danieljensen2626 why would they start talking about setting it up to their systems? Can you send a link of the ep of the wan show where they talk about it?
The excitement is infectious! The fact that they just unboxed this at a pokey part of their warehouse giggling like kids on Christmas morning was really fun to watch. This is going to be a great series!
"Building a data center is the most expensive part about building a data center" - LS 2022 Never in my life have my ears been blessed with such wisdom.
been looking for a invisible character, would you be willing to give me yours? edit: actually visiting your page i can see it, im looking for one thats invisible when viewing the channel and have tried dozens of invisible characters but none have worked so far.
Honestly the best part of this is Kioxia playing ball. "Yeah, that would be an interesting engineering challenge." This is the kind of stuff you like to see.
Linus 2022 "we may never see performance like this again in our lifetime" Linus 2040 "Today we are reviewing the Samsung Universe XXII with 50 petabytes of quantum memory onboard"
I'm sure some will. And I'm pretty sure taking a 1% cut on even one single 100K item is better than 1% cut on 1000 pcs of 50$ each. I've no clue how much they get from their affiliate links, plus the links are also just a good way, for people interested, to check out the further specs on items shown.
This is so fucking cool. You have to realise it's only because Linus and co have been so diligent about building their legitimacy, brand, reputation, and connections for YEARS that we got to this point. It's only because the industry trusts him so much that we got to see such amazing tech like this.
Also that he did some pretty poorly done reviews of enterprise cards a few months back and nvidia didn't like the bad press and wanted to see him do it right.
At LTT we usually see consumer PCs that most enthusiasts can only dream of. This is enterprise level hardware that's so cutting edge and over the top that people who spec servers in data centers dream of this.
@@morosis82 I actually expect one of them, if not both, might appear cause I bet Wendell is as excited as Linus and Jake if not both of them together bout this much power... Just my thoughts
I can just imagine the folks and nvidia and kioxia covering their faces in fear as linus handles their $1,000,000 worth of hardware in such a care free way lmao.
Dont worry , they dont care about them. They would not even feel the loss of one. I acctually used to support a competior version of this. This ladies and gents is called "High availablity cloud". I've actually seen a dataceter made out of those and yes they cost arm and leg (just to look at them).
8:33 the "flappers" are to prevent backflow if that unit goes bad and the rotors go to 0 rpm. The internal pressure is so high from the other fans that the air would go the wrong way without those.
When Linus and Jake can't actually believe what it is they're looking at, even though they already know what's coming, you know it's going to be something special.
8:34 "They've got flappers!" Those louvers are there to close of the air path if the fans dies. When the air pressure get higher on the back the louvers will close so air cant leak back to the front through the fans. Pretty slick and low tech which makes them very reliable.
The amount of computing power available today is absolutely insane. I work in a big data company and one of our hadoop clusters has 104TB of RAM and over 13000 CPU cores. When I first started working there about 3 years ago, I would sometimes navigate to the resourcemanager just to take another look to make sure my eyes weren't betraying me. I must admit that I have never seen 1PB of flash storage though so this will be a freaking crazy project and I can't wait to see what the result benchmarks are.
I saw him or his shirt slip around a box before. He had just reached "fuck it". I gotta be over there so I'm gonna be in shot. They can always re-shoot it.
I work at a datacenter, so I'm fairly used to high performance enterprise hardware, but I've never got to work with something this powerful. I'm very excited to see what's next with this project.
Yeah I get what you are saying, those numbers are a definitely high for a medium enterprise but for larger customers, $1M IT orders are no big deal. I have clients whose cloud monthly bill are close to that number. BUT I have to say, this is still next level performance that I am certain is an overkill for all but a very specific use cases where u need that kind of fast storage. I honestly can't think what you could possible need that kind of storage for, because basically anything that needs that fast, you load it (& run it in memory - which is still much faster than any NVME), Anything lower and u cache into NVME and then tier the storage like any other sane person based on usage.
@@AmrXcellent Agree - this is a "because we can and got it for free" project for what they are going to use it for (presumably video editing?)- in the real world of real budgets you would certainly tier it and size the fastest tier to your normal working dataset. I've been an Enterprise IT consultant in storage and backup for 15 years, now also part of the high performance computing team at my employer and I get to play with this stuff every week. Good times! I will say that some of our HPC clients do have this sort of storage in performance and amount for HPC work. Just absolutely enormous datasets, so the time to even load the dataset into RAM becomes a real issue, and many multiple datasets flying in and out of the GPUs for processing. I have to admit, I'm the hardware dude - I don't have that much of a clue about how any of the software they run on these things actually works.
@@Kevin_Rhodes They're not keeping it, they have to send it back when they're done. Genuinely the amount of power draw this thing is pulling would probably make it too expensive for them in and of itself lmao.
enterprise storage gear is bonkers, I've deployed some multi-million dollar solutions, but I don't think I've ever had my hands on anything with that much potential performance.
I’ve worked with some 11+ figure projects that would blow your mind, the density is impressive but the speeds are last gen. Can’t wait to see what his power delivery solution will be.
I work in the Enterprise space, and I deal with Datacenters with massive SAN's, compute racks, etc....we have 4 DC's ourselves, let alone the 3rd Party DC's we colocate racks into. But the pure insanity of this sort of power, and the configuration with the NVMe-oF to a central GPU accelerated central compute, absolutely blows my mind...and the absolute density of this setup to fit in what...about 10U of space, so a 16RU rack holding everything....insane.
The engineers who worked on this must either HATE them and their lives right now or squealed like children when they were handed this project. Possibly both. This couldn't have been easy. Probably required some innovation and development to make work.
@@avroarchitect1793 I kinda doubt it. Looks more like they get to test the next gen product lines. None of this looks custom, and it seems to be a future standard configuration.
I still work in DCs. The density nowadays is so insane. 4kva racks are standard, customer trying to push like 6-8 kva per rack at just a colo facility the cooling is having a rough time. this kind of hardware is just so badass and fun.
i love the fact the intro has never changed, always updated but never a complete rework. classic man i love this channel so much & have been following for around 7yrs now. so proud of Linus & all the hardworking folks at LTT making all this possible. shoot past the stars ❤️
@@profblack And still, having all running through a Pi and then check the capacity of that array/pool with 1PiB would be crazy impressive for that small board.
As long as that friend isn't one of those people who immediately turn around and go back to an off-the-shelf product because either A. it works very so slightly better with their mac because macs, or B. it looks more aesthetically pleasing on a desktop. ( valid yet vapid reasons )
It would be interesting to know if it would beat HashiCorp's Nomad 2 Million Container Challenge but this time around running Kubernetes. Linus and Jeff please make it happen 🙏
God, I barely know any of what you're talking about when it comes to this avenue of techy talk, but it's so satisfying to hear excited people talk about what they love
@@20blog28 did they buy this? Was it offered in exchange for a shouting? Sounds insane but you never know. Is this really a million dollars? Also do we know what it's for? They have a specific need or is it just because why not? Do they keep it or have to send it back? Thanks.
Future: This old IBM PC AT went on evolve into mind blowingly powerful technology. Sadly this little IBM is all that remains and we no longer have the technology to make one even this simple.
As a former amazon data center employee I’m super super super excited to see the benchmarks done with this unit. I’ve always only just fixed these types of servers. Replaced those dac cables seen upwards of 400 gbps dac’s and 15.6 terra-byte drives I believe but never ever once got to benchmark or see just how amazing they are. I am extremely excited for the entire series
One does not get good at flipping and catching random assorted shit by being careful and not dropping things, they get good by dropping things trying to flip them. He's just an expert at dropping things such that he has gotten to the point that he just drops them intentionally with style back into his own hands. You're merely witnessing a REFINEMENT of pure dropping technique.
Part of his attention is focused on the teleprompter, while handling pieces of hardware, presenting it to the right angle for the camera, making sure A roll and B roll shot are made. In the contrary, flipping his screwdriver is something that requires less attention and something he's used to handle
I work in manufacturing and assembly for these machines and watching these two casually take components out and handle them with such "care" is the reason I have work nightmares
I swear to god I've never been so nervous watching an LTT video.. you guys are killing me! Setting that thing on two edges of an open cardboard box.. I was actually clenched the whole time.
As a enterprise IT guy, the MSRP for that server with the A100s is roughly 150-200k USD. I've looked into it before for some projects that we did. Edit, my vendor tells me each 12TB ssd from Kioxia is about 5k. That's another 360k on top of the servers.
So here in Australia that would easily be AUD$1,000,000 allowing for exchange rate, GST and 'the Australia Tax' which is a random markup put on because of 'distance etc.'.
@@ozgruntsyd4281 'random markup for distance ' yeah it's called shipping costs. If you look at shipping lanes they basically make a path East to West meaning that Europe and North and South America are all along a roughly straight line so it's efficient and cheaper to ship whereas Australia they have to head south from China just for Australia and New Zealand which is a lot less efficient so the shipping costs are higher.
@@pepperroni6252 We usually get charged a higher base price and shipping on top of that. As for inconvenient shipping the ship can be back at Singapore Japan Taiwan or China before it could have made it to the USA or Europe and no pirates to dodge.
@@johnmccallum9106 the base cost is usually to do with manufacturing costs, Europe Asia and America all have their cheap labour countries. There's many more people in Europe, and the American continents than South Asia and Oceania so it's still more economical for them to ship to those places, also ships will return with products from those countries whereas there'll be less to bring back from South Asia and Oceania so another loss. It's all about minimising loss and increasing efficiency.
I sometimes forget how insane some of this stuff is because I deal with it every day. It's always fun to see someone not from the enterprise space see these things and be reminded how cool they are
@@Abbrahan can confirm; I work for a financial firms enterprise data center, and our virtual workstations are allocated 8 threads of a Xeon Gold 6xxx (I forgot the exact chip), 16gb of memory and around 512gb of storage and even then they are atrociously slow.. when I WFH, I try my best to just remote into my office pc 99% of the time lol
@@RamyWarda A big part of it is network congestion too. I work in NYC and the upper floors of my building with fewer employees do just fine. The 2nd and 3rd floors (which are much larger, with many more users) are slow as hell. Doesn't matter which pool we put them in. Perhaps the 2960x switches are the bottleneck. (edit: 2960, not 2690)
18:00 It's a whole lotta fun to see these guys be this excited over such exceptional hardware. A few moments though where I couldn't not be aware of just how much Linus was breathing directly onto $100,000 of ultra-high-end electronics 7:17 🙊 10:38 😅
@@user-rd3jw7pv7i I helped install a *dozen* nVidia DGX GPU servers a couple weeks ago. They are $500K each or so, and are basically nVidias own version of this. HPC in the enterprise (or in that case a large research university) is both fun and mind-bendingly expensive. Which is a great gig to have. I have to admit, *I* got excited seeing those giant 450lb monster servers racked together. They certainly looked every bit of $10M bucks. I've been doing Enterprise storage, backup, and virtualization consulting and implantation engineering for 15 years, just getting into the high performance computing stuff. Looked to me like they have the wrong PDUs for those high-amp plugs on the GPU server though. And those DGXs use *6* of those EACH. A mind-boggling amount of power.
💯 I just order some new 400Gb network cores and servers with CX6 nics. As you are tossing them around and installing them... you forget they are worth a corvette each.
This is exactly why I love seeing enterprise stuff on the channel, and especially the more exotic stuff, cuz you get to see things like this power supply 5:05 and this weirdo connector for all that massive data 5:39 and the insane amounts of RAM with all the drives. I love it and I seriously can't wait for the next video to come out about this, which doesn't happen often for me lol
I remember those days... Retired from IT Infrastructure now. I was the SAN guy at the office and the day the two+ million dollars of equipment and a couple of petabytes of storage (SAN storage racks, controllers, Brocade switches and boxes of fibre to connect it all) came in, well... I remember that feeling. Christmas does come early some times.
He Mark, I just left them a similar message. I was a network admin back in the 90's and early 2000's and was just like they are today. I miss getting switches, routers, storage, wiring, tape backups, learning to work with fiber, even making hundreds of patch cables back when I first started.
They're laughing out of excitement like Sponge Bob and Patrick and honestly, I'm laughing along. These crazy projects and the adventure sense they always have about them are one of the top reasons I love LTT so much.
@@NemoConsequentae Well the guidance computer was built with reliability in mind, taking a phone through the van allen belt would instantly corrupt the contents of it's RAM, whereas the guidance computer went to the moon and back going through the van allen belt twice without crashing
How else is make going to maintain his minecraft server. Clearly this is what hes been working on since he dropped work on the server last year. Coincidence I think not lol. 1mil mc server hahaha
This is like... HPC level hardware. No enterprise or datacenter is running this level of hardware for normal compute... If you're running this in production you've got a seriously niche need.
I've never been so excited to see someone unbox something before. This is going to be an Epyc series! Thanks Xioxia, Nvidia, and SuperMicro for making this happen!
I’ve been so excited for this! After hearing about it from the WANShow! This kind of content is always super entertaining, and the dynamic of these two is so fun!
Never in the history of the internet has an unboxing won my sub... until today. I NEED to see how this turns out. I've been in the industry since computers didn't come with display outputs.. when storage media was on magnetic tape... this is just absolutely amazing.
Then again, in 15 years people will look back at this video and chuckle because their phone has more speed, storage and has a higher bandwidth through a single thin cable... But yep, these specs are really mental.
Hearing about how this unboxing came about on WAN Show had me even more excited for this video. Glad to see it finally come to fruition, & I guess you’re really going full tilt with this LMG server overhaul (albeit not with this gear specifically since they’re returning it later).
Oh yeah. This is the kind of content I can get behind. Also, it's brilliant to have Linus next to all those boxes and crates in the intro. It really makes them look super large.
I work at a data center for a cloud provider and we've been deploying rows and rows of this hardware. It's been pretty fun to deploy and play with the hardware. You will definitely need to rethink your power and cooling solutions to support this stuff.
When I see stuff like this, I have to wonder about the absolutely insane amount of bandwidth that AWS or Google has in total. It has to be at least exabits per second. I don't even know what's past exa but that might be a reality in terms of flops (maybe?)
i'm woking in a data center as well, i would love to see something see to install something like that for a customer in my queue... until now only had smaller netapp shelf or hp vault setups... just like linus, i giggle to myself when see such rediculus brilliant setups!
@@Pussalia Yeah, I hope LTT revamps their server room infrastructure specifically to support this hardware. I don't think they've thought that far ahead yet haha. It'll be fun to watch them figure this out.
I have a SuperMicro 200TB storage with some processing servers in a 10Gbps switch at work, and it is sweet. This monster that they are mounting is just mind boggling. I am very pumped to watch this series unfold.
Well if an average gaming computer has or had 1 GBit at max speed, which barely ever gets fully utilized, then his server rack had a capacity of 12 800 GBit networking. - That means it could handle almost 13 thousand gaming computers at once, all at max speed which they ever can utilize. If you drop this value down to lets say FullHD (1080p) content streaming which is something like 3MBit per computer / client. That'll make it capable of serving approximately 4 MILLION 267 THOUSAND customers at once! (Tho can the rest of that server handle that many clients?) Almost 4,3 Million computers running FullHD video at once? Is this crazy enough? Just one server can serve almost every citizen in my country at the same time, streaming some 1080p content.
Holy sh*t has always been one of the most entertaining series on this channel. Love every installment no matter how long it takes between two 'episodes' haha. Keep pushing the boundaries!
Beautiful, something really awesome about the raw magnitude of this build. A note on the power, hopefully these two power connections are in sync, or being run through a central UPS/building or otherwise to clean it up, but running extra long cable to sensitive equipment of differing lengths can cause a dirty power input, not something to problematic but not ideal and in brown out scenarios can cause a fault down one side of the rack, not sure on the tolerances on the kit. On oneside it should be bullet proof as it's DC equipment, on the the side it's DC equipment and may expect incoming facilities to be within DC tolerance. Can't wait for the follow up videos
@@kirkanos3968 Seriously, they repeated how long this project has been in the works. Was Brian the Electrician not available to extend the service? Oof!
I hope this comment gets some attention, they aren't really going to know how serious that could be so hopefully there can be some input from the few who might be able to answer this.
Seriously… watching this hardware being handled this way makes me cringe. Casual box dropping, tossing and torquing chassis, forcing mechanicals, no regard for static. I’d fire Linus if he worked for me
It's actually one of the reasons why I don't watch his videos much anymore. He always handles very expensive hardware extremely carelessly for no reason whatsoever. He clearly isn't even doing it for views or anything, he just does it instinctively. He may take a multi-thousand-dollar heavy piece of hardware and flail it around with one hand like it were a coctail shaker (heck, even cocktail shakers are handled more carefully because they are usually shaken with both hands, not just one). This video is no exception (eg. he does that at 8:55). And, of course, every time he unboxes some really expensive server hardware, instead of doing it safely on a table in a clean environment, he insists in doing it in some dirty storeroom, on the floor, placing it precariously on some cardboard box. I'm surprised he doesn't do the unboxings on a piece of plywood propped up onto some rickety garden chairs... on the roof during a windstorm.
As someone who used to install enterprise gear, I wince every time you guys handle that kit. I'd be in a prepared room with almost zero dust, with anti-static strapping, adequate room to set everything up, have the rack ready to go and in the correct spot, have tested the power and network links, and a thousand other things before that first box got opened. I love the way you guys do this stuff, but it makes my teeth itch and my eyes lose focus when I see a cool mil of kit get thrown around! Can't wait to see exactly how over-powered this is compared to your needs ;-)
Ah there’s a difference, normal people lose jobs when they drop things … Linus dropping things be like “ok let’s send him more shit to break for fun” then he makes another 100k dollars for the added twist …
@@simonjo1984 Normal people definitely don't lose jobs* when they drop things unless there's gross negligence involved on the part of the person doing the work (not in company procedure planning). * = assumes the country has worker protection laws AND the workplace isn't the kind where you get fired for taking a week of sick leave.
I spent the last two years as a systems engineer for a large saas company working with gear this big or bigger and it wasn't a big deal. I feel bad for you guys with ridiculous procedures and bosses
I'm so jealous of someone getting to use this kind of stuff. Where I work we have tons of very expensive servers and network devices, but I don't really get to mess around with it too much. I just fiddle with my server at home on a dell workstation.
At this part I'm halfway convinced sponsors provide Linus with hardware, let's say, 50% for the advertisement and 50% because some engineers somewhere are wondering "What's gonna happen if we hook up a week of merchandise together?"
The huge heatsinks next to the GPU coolers, are the NVLink heatsinks, they are the internal network switches between the GPUs :) Also, please don't remove them, they are not your average heatsink installation (patric from serve the home did a video on why its a bad idea with the previous gen Nvidia GPU node, TL;DR you need like a 1thousand dollar screwdriver to do it without breaking something) I cant wait for the performance video from you guys, this is a really cool project, really goes to show how far the enterprise segment has come vs gaming hardware
Yeah, pretty sure they are bare die chips without integrated heat spreaders. Really easy to crack the die if you aren't extremely careful or are using calibrated torque screwdrivers.
i'm just an ol'school guy... got started in PCs back when Commodore 64/128 were the 'woo woo' tabletop/home PC... these numbers... woosh! so fast over my head i need an air-traffic control tower on my skull! but a PETABYTE i had to look that one up... 1024 terabytes, or a million gigabytes... whew... tha hell! :O and that is just in solid state drives... i had a Seagate 4TB HDD go tits up on me a few years back that i was using for 'backup in-house data storage' instead of relying on DSL up n download speeds to 'cloud saves'... and i was crying into my hanky for hours trying to revive the bugger... and these guys are hauling around the equivalency of 256 of those Seagates... gah... mindblowing! aight! where's part 2! i wanna see the rest of the loot! :D
I work in IT for a university, this configuration alone nearly matches or heavily outclasses (depending on what you measure) the entire server environment we have. Not to mention the other servers LMG wield...
Yes, I've considered Kioxia last week. Not the best option this time but they get my consideration now. I know that sounds big headed but my customers would know Western Digital but not Kioxia.
@@wayland7150 exactly that! A while ago I'd thought the same as your customers. Now I've changed my mind and would shortlist them in my comparison against drives by other manufacturers. I guess that's what they try to achieve with these videos. I won't autobuy them, but I will consider them as I would Samsung, WD or Kingston.
"performance we may well never see again" I'm willing to bet that it will be just a couple years before LTT does the next upgrade which will make this rig look just ok.
I can see it now... 2028. LTT is about to unveil their new "Editing Server".... And Linus goes... Yes ladies and gents... This thing will have 2 Zettabytes of storage. What happened to our Petabyte server you ask? It's in the media/gaming lounge. It has just enough storage to have COD 22 - The Venusian Wars and expansion packs installed.
@@5eeeeeb5 I know this is a joke, but for those who don't realize, yeah, this is a joke. There's no way a single game will be a petabyte in just 6 years.
I thought that too. Go back 10 years and tell Linus that he'll have a petabyte of nvme flash in his hands and he'd either scoff or be absolutely floored.
I was in Student Cluster Competitions in uni and I can vividly remember the feeling of finally assembing a server when all the gear arrived from sponsors. No biggie, just a $100k chunk of metal dented because someone at the airport couldn't care less
Of all the server hardware that I never seen it before, I think it was the most beautiful and the most insane server hardware period. NVIDIA has the design appeal on the A100s plus the memory, NVMe etc. This is the datacenter's wet dream
You're going to need really robust and well-protected and filtered C20 PDU's for those compute units. You'll also need a minimum of L6-30 30A supplies with 50A preferable. Just FYI
SuperMicro, good call. Love their hardware. On a side note, as crazy as that architecture setup might seem, it's got a 3 to 5 year shelf life in the enterprise space before it's decommissioned and replaced by something that's probably twice the performance. The companies that can afford that level of gear outright are constantly seeking higher performance/density. Your bottleneck will likely be the CPU core interconnects depending on how you segment the compute and storage and the software you overlay on top. Definitely going to need Linux and some creative heavy hitting to really get the 'full spec' out of this setup.
"Performance which we have never seen and may well never see again." I'm not a fortune teller but I strongly feel the second half of that sentence will age really bad really fast.
@@satellitereigns Yeah, I heard that. They are really young tho. I'd consider 5 years really fast compared their remaining life and I'd be shocked if today's existing tech would be still the fastest at that time.
Exactly my thoughts. If we are looking at how computers evolved in the last 30 years we can safely assume that even in their lifetime (Linus is around 35) there will be grander builds than this. I might gamble to say that even in the consumer space, not just enterprise level stuff.
Make sure this server and your welder (when you plug it back in) dont share any breakers/lines. The voltage drop created by welding can cause the server to crash.
Near the end of the video, they said they were, and that they just wouldn't run the welder, since this thing needs so much power. They're using another circuit in addition to the welder's.
This had better be on their massive UPS. I'm so paranoid about power fluctuations and noise that I am wondering if I need to EM shield my garage from my welder, as I have a TV on the other side of the wall from my garage. (And I am considering a whole-house UPS for everything NOT in the garage)
Back in 2010-2016, I was working in a server software test lab as the sysadmin, and I had the pleasure of building several test units for new storage. In one case, I think 2013 or 2014, we built a whole rack OBS system with the storage units being 12X4TB drive 1U Asus servers (I think there were twenty) and the 3 head units being all SSD 1U Dell servers. The servers interconnected internally to the storage in a local dedicated network in all 10Gb, and then served out to the rest of the world with 6 10Gb connections. It was $1.5 million for the test unit, and I had the pleasure of assembling it, with the instructions from the developers. It was insanely powerful at the time. I know how fun it is to open and assemble such things.
I don't know anything about any of this hardware, but judging by their reactions and how giddy they got every time they opened a box or a cover, I could tell it was good stuff.
Its the stuff of supercomputers. The most powerful CPUs, insane amounts of RAM, high performance interconnects. It seriously doent get higher end than this, the only way up is just more of the same (and the hardware is made to scale near infinitely, add more hardware get more performance).
I love that kioxia said challenge accepted. They weren't " that's not we were thinking" or "here's a script for this." But that's an interesting engineering project.
I can imagine Kioxia enjoying blowing LTT minds with what is normal to themselves.
Why is human technology so depressing? Especially Gpus producing heat like an air fryer.
The thing is... everyone who heard about this was just like "sure, this seems like fun!" and partnered with the project. thats so cool!
Kioxia had a a mishap a month or two ago and lost 600,000 exabytes of flash. I guess they are pretty good at catching up. So they lost 600 million times more than what is used here..... they are supplying the planet of flash.
The engineers were like:
"*He wants to do WHAT?!*"
"Not saying we can, but assuming we COULD what would it take? You know for -science- LTT"
Kioxia was like 'That sounds like fun, let's make this happen!'
For reference, each of those CD6 drive costs $3100.
How many were there? (sorry I don't remember them saying it in the video)
I think they had 12 per server and said they gonna have 6 servers so a total of 72… so… 220000$ of drives
He discussed on WAN show that all this hardware is a loan, not a gift. After the project is done they have to give it all back.
@@danieljensen2626 why would they start talking about setting it up to their systems? Can you send a link of the ep of the wan show where they talk about it?
@@Jack-rq7ll because to test a machine like this you have to hook it up to a network to even test the speed/reliability of the unit
Every time Linus moves a part I sweat bullets lol
We were nervous too
@@LinusTechTips i guess so I'm nervous and Frikin Excited
Bread
@@LinusTechTips You should have been
Same here. I want to add that college did not equip me to understand much, if any, of what is being said. But I like the excitement.
The excitement is infectious! The fact that they just unboxed this at a pokey part of their warehouse giggling like kids on Christmas morning was really fun to watch. This is going to be a great series!
It is funny how Linus didn't know what part was the server.
"Building a data center is the most expensive part about building a data center"
- LS 2022
Never in my life have my ears been blessed with such wisdom.
been looking for a invisible character, would you be willing to give me yours?
edit: actually visiting your page i can see it, im looking for one thats invisible when viewing the channel and have tried dozens of invisible characters but none have worked so far.
we'll just use the welder and the mill plugs for this, we got it with extension cables. sounds about right.
What did he mean with that?
@@william41017 Linus meant that the cost of the servers in a DC is less of a factor than the cost of the actual building and infrastructure.
@@jkmicha thanks
Honestly the best part of this is Kioxia playing ball. "Yeah, that would be an interesting engineering challenge." This is the kind of stuff you like to see.
Kioxia really said fuck around and find out
this is the attitude that keeps tech improving further and further… you love to see it
I'm so excited for the part where he has his editors use it and respond with "yeah its ok"
Taryn, paging Taryn!
"Is this supposed to be better?" 🤣
Lol
"I'm getting some stuttering on video editing"
Linus panics.
Especially Hoffman.
“It’s still laggy and shitty”
Linus 2022 "we may never see performance like this again in our lifetime"
Linus 2040 "Today we are reviewing the Samsung Universe XXII with 50 petabytes of quantum memory onboard"
ikr. It's like they haven't learned at all how fast this shit improves.
2040 continued... Why not, everyone needs a starter system...
I love how LMG linked everything they used in the description like the viewers would just snag some of it for themselves.
I'm sure some will. And I'm pretty sure taking a 1% cut on even one single 100K item is better than 1% cut on 1000 pcs of 50$ each.
I've no clue how much they get from their affiliate links, plus the links are also just a good way, for people interested, to check out the further specs on items shown.
Corporate IT folks watch LTT too. What Linus showed today could likely replace our entire Itanium data center.
@@BobSentell I had a feeling. Just thought it'd be a chuckle anyways.
I mean some crazy son of a gun may use an epyc processor in a gaming rig, I don't doubt it's possible, even practical.
@@BobSentell as a datacenter tech, I can only say they've hit the nail on the head for things that cannot be discussed.
very easy to recreate at home, affordable as well 👍
yes :/
Yes you could buy this and totally dont break the bank
Rofl
I've been looking for this tutorial for so long, and finally found a simple, well explained one.
He only runs his own servers because he can farm content and he gets the hardware for free.
I love that linus has a “pull out animation” for the screwdriver, every time he takes it out he flips it
And surprisingly he doesn't drop it either
@@zachylimaki2167 Maybe that's the only skill he has. otherwise it's Linux DropTips
Marketing and Practice
When you spend a few million on a screwdriver you damn well cherish it and this just shows in how he prefers to use it himself.
Like the "pull out" animation I had with your mom
This is so fucking cool. You have to realise it's only because Linus and co have been so diligent about building their legitimacy, brand, reputation, and connections for YEARS that we got to this point. It's only because the industry trusts him so much that we got to see such amazing tech like this.
Also that he did some pretty poorly done reviews of enterprise
cards a few months back and nvidia didn't like the bad press and wanted to see him do it right.
At LTT we usually see consumer PCs that most enthusiasts can only dream of. This is enterprise level hardware that's so cutting edge and over the top that people who spec servers in data centers dream of this.
As a Systems Admin, I approve..
@@wongperson4909 just wish we could see more supercomputers
I'd love to see Wendell or Patrick make a showing in this series.
@@morosis82 I actually expect one of them, if not both, might appear cause I bet Wendell is as excited as Linus and Jake if not both of them together bout this much power... Just my thoughts
People who spec servers in datacenters stopped caring long ago.
I can just imagine the folks and nvidia and kioxia covering their faces in fear as linus handles their $1,000,000 worth of hardware in such a care free way lmao.
@DJ idk how strong?
Let's be honest. If they were that worried, they wouldn't have sent it.
Nah they probably wipe their ass with that amount of money lol.
Dont worry , they dont care about them. They would not even feel the loss of one. I acctually used to support a competior version of this. This ladies and gents is called "High availablity cloud". I've actually seen a dataceter made out of those and yes they cost arm and leg (just to look at them).
@@psedog idk man, Linus is known for dropping things. Unless they had a backup plan like Linus pays compensation for dropping I'm guessing.
8:33 the "flappers" are to prevent backflow if that unit goes bad and the rotors go to 0 rpm. The internal pressure is so high from the other fans that the air would go the wrong way without those.
Ohhh that makes sense! Thank you :v
When a case has so much air pressure inside, you have to start adding airlocks...
Nice
Ooh it’s like a bomb lol
@@PureRushXevus "gas escape valve went out on my case, gotta stick it in the freezer now"
I love when people are genuinely excited about their passion like this, its amazing.
i feel they were not professional and were horsing around throughout the whole video.
@@anthonyhernandez157 who cares lmao
Gotta hand it to Kioxia, they really have some pull in the industry, and they went in this full send.
They did literally invent Flash Memory. XD
@@goodiesohhi oh forreally?
@@goodiesohhi oh didnt know that
They heard Linus's challenge and were like "Alright, bet"
@@hxd9321 kioxia (formerly known as Toshiba Semiconductor) is one of the first company to made and commercialize flash memory.
The amount of times Jake hands Linus a piece of hardware then says “don’t drop this” is priceless.
And then Jake proceeds to keep 1 hand on it. 🤣
Wher'd Linus find "Giggles the Clown" at?? I recommend 40cc of Euthasol .......STAT!! {0.o}
😆😂🤣
And then Jake dropped on purpose one PSU.
wysi
and usually that piece is worth more than your hole life
When Linus and Jake can't actually believe what it is they're looking at, even though they already know what's coming, you know it's going to be something special.
wow fr?
@@inflammatorycommentswithno2407 fr
I can already tell that this is going to be my favorite series on this channel.
8:34 "They've got flappers!"
Those louvers are there to close of the air path if the fans dies. When the air pressure get higher on the back the louvers will close so air cant leak back to the front through the fans. Pretty slick and low tech which makes them very reliable.
300 watt* fans, 0 watt protections.
*or wattever ;)
The amount of computing power available today is absolutely insane. I work in a big data company and one of our hadoop clusters has 104TB of RAM and over 13000 CPU cores. When I first started working there about 3 years ago, I would sometimes navigate to the resourcemanager just to take another look to make sure my eyes weren't betraying me. I must admit that I have never seen 1PB of flash storage though so this will be a freaking crazy project and I can't wait to see what the result benchmarks are.
How many racks was that, too? Probably not just the half-rack they had there!
@@kaukospots I have no clue as I don't work in the infrastructure side but we have multiple data centers so probably lots of racks!
can you imagine what google or amazon has?!?!!?
@@skmetal7 Google, Amazon, and Microsoft probably have more than we would even think on a high estimate in their cloud datacenters for sure.
What it's for exactly?
Alex walking into frame:
"Nothing new here, we do this everyday"
I like that short "camouflage appearance" in the middle of the video. :-D
@@michael7738 Wait, Alex appear on the video?!
@@Braiam A wild Alex appeared for a second.
I saw him or his shirt slip around a box before. He had just reached "fuck it". I gotta be over there so I'm gonna be in shot. They can always re-shoot it.
9:48 "Building a data center, is the most expensive thing about building a data center"
I work at a datacenter, so I'm fairly used to high performance enterprise hardware, but I've never got to work with something this powerful. I'm very excited to see what's next with this project.
This is already outdated if only you could see it
Yeah I get what you are saying, those numbers are a definitely high for a medium enterprise but for larger customers, $1M IT orders are no big deal. I have clients whose cloud monthly bill are close to that number.
BUT I have to say, this is still next level performance that I am certain is an overkill for all but a very specific use cases where u need that kind of fast storage. I honestly can't think what you could possible need that kind of storage for, because basically anything that needs that fast, you load it (& run it in memory - which is still much faster than any NVME), Anything lower and u cache into NVME and then tier the storage like any other sane person based on usage.
@@AmrXcellent Basically Powering the Starships next.
@@AmrXcellent Agree - this is a "because we can and got it for free" project for what they are going to use it for (presumably video editing?)- in the real world of real budgets you would certainly tier it and size the fastest tier to your normal working dataset. I've been an Enterprise IT consultant in storage and backup for 15 years, now also part of the high performance computing team at my employer and I get to play with this stuff every week. Good times! I will say that some of our HPC clients do have this sort of storage in performance and amount for HPC work. Just absolutely enormous datasets, so the time to even load the dataset into RAM becomes a real issue, and many multiple datasets flying in and out of the GPUs for processing. I have to admit, I'm the hardware dude - I don't have that much of a clue about how any of the software they run on these things actually works.
@@Kevin_Rhodes They're not keeping it, they have to send it back when they're done. Genuinely the amount of power draw this thing is pulling would probably make it too expensive for them in and of itself lmao.
enterprise storage gear is bonkers, I've deployed some multi-million dollar solutions, but I don't think I've ever had my hands on anything with that much potential performance.
you have, probably took 8x the space of this however
@@justbob333 or more in some cases, but some of the newest stuff I deployed fit into a rack or two, especially the flash solutions
I’ve worked with some 11+ figure projects that would blow your mind, the density is impressive but the speeds are last gen. Can’t wait to see what his power delivery solution will be.
I work in the Enterprise space, and I deal with Datacenters with massive SAN's, compute racks, etc....we have 4 DC's ourselves, let alone the 3rd Party DC's we colocate racks into. But the pure insanity of this sort of power, and the configuration with the NVMe-oF to a central GPU accelerated central compute, absolutely blows my mind...and the absolute density of this setup to fit in what...about 10U of space, so a 16RU rack holding everything....insane.
Can we get the Yottabyte Project next?
The engineers who worked on this must either HATE them and their lives right now or squealed like children when they were handed this project. Possibly both. This couldn't have been easy. Probably required some innovation and development to make work.
@@avroarchitect1793 I kinda doubt it. Looks more like they get to test the next gen product lines. None of this looks custom, and it seems to be a future standard configuration.
I still work in DCs. The density nowadays is so insane. 4kva racks are standard, customer trying to push like 6-8 kva per rack at just a colo facility the cooling is having a rough time. this kind of hardware is just so badass and fun.
And at science databases there could be racks after racks full of servers like this one column after column side by side.
i love the fact the intro has never changed, always updated but never a complete rework. classic man i love this channel so much & have been following for around 7yrs now. so proud of Linus & all the hardworking folks at LTT making all this possible. shoot past the stars ❤️
So... will you start installing petabytes of flash storage in collabs with other RUclipsrs now? A friend would like to know.
Try doing it with a raspberry pi ;)
@@erdragh Imagine having all of these servers and equipment and then routing all of that traffic through a Pi.
@@profblack And still, having all running through a Pi and then check the capacity of that array/pool with 1PiB would be crazy impressive for that small board.
As long as that friend isn't one of those people who immediately turn around and go back to an off-the-shelf product because either A. it works very so slightly better with their mac because macs, or B. it looks more aesthetically pleasing on a desktop. ( valid yet vapid reasons )
It would be interesting to know if it would beat HashiCorp's Nomad 2 Million Container Challenge but this time around running Kubernetes. Linus and Jeff please make it happen 🙏
"Everyone you can possibly name in the server space" - RIP Intel.
):
doesn't already matter :D everyone has new cool "home stuff" :D
marketing is business ...
I really want to see this running a Minecraft server.
@@IM2awsme lol 1 petabyte minecraft world
@@IM2awsme finally a server that can run modded minecraft worlds that don't crash when more than 2 people are exploring lmao
God, I barely know any of what you're talking about when it comes to this avenue of techy talk, but it's so satisfying to hear excited people talk about what they love
same
Basically, a small company got hold of the kind of hardware that you would find in a google data center
@@20blog28 did they buy this? Was it offered in exchange for a shouting? Sounds insane but you never know. Is this really a million dollars?
Also do we know what it's for? They have a specific need or is it just because why not? Do they keep it or have to send it back?
Thanks.
@@Andrei5656 Given the specs of what they've shown - Yes - This is easily a million dollars worth of hardware.
@@Andrei5656 They do have to give it back, yeah.
18:04 is Linus in a candy store where he was told he could get whatever he wanted.
Linus: "we may never see this again in our lifetime."
Technology: "come see this cell phone in 20 years"
He's underestimating Moore's law xD
@@spicybaguette7706 unless process node density slows down as gate features approach fundamental atomic limits.
@@flinx Dennard scaling seems to have hit a wall, just wait until you see the TDP of next generation server CPUs.
Future: “remember cellphones”
Future: This old IBM PC AT went on evolve into mind blowingly powerful technology. Sadly this little IBM is all that remains and we no longer have the technology to make one even this simple.
Favorite thing about LTT was when we'd get to see some absolutely bonkers over-the-top hardware. Glad that is back.
I’ve been watching since Whole room watercooling.
@@wesrihn I came for the tips, but Whole Room Water Cooling is why I stayed.
It was either that or one of the Compensator builds.
As a former amazon data center employee I’m super super super excited to see the benchmarks done with this unit. I’ve always only just fixed these types of servers. Replaced those dac cables seen upwards of 400 gbps dac’s and 15.6 terra-byte drives I believe but never ever once got to benchmark or see just how amazing they are.
I am extremely excited for the entire series
As a Datacenter Technician I find it very cute how you are excited so much about such small things :D
lol mate working with like 1000 of those and call it a SMALL system
2:17
can’t wait for people to comment 25 years later and say “we have 1tb vram nowdays”
yes
It's 2047 and 1TB of memory is less than my breakfast subscription.
Niktek boi
They did 640gb vram in the video
25yrs later and we no longer have pc's at home. Just VR interfaces and everything is owned by someone else.
5:30 it's incredible that Linus can flip a screwdriver so nonchalantly, while also being known for dropping thousands worth of equipment
Cause it's his own, when it comes to anything not owned it's a drop
One does not get good at flipping and catching random assorted shit by being careful and not dropping things, they get good by dropping things trying to flip them. He's just an expert at dropping things such that he has gotten to the point that he just drops them intentionally with style back into his own hands. You're merely witnessing a REFINEMENT of pure dropping technique.
Part of his attention is focused on the teleprompter, while handling pieces of hardware, presenting it to the right angle for the camera, making sure A roll and B roll shot are made. In the contrary, flipping his screwdriver is something that requires less attention and something he's used to handle
I work in manufacturing and assembly for these machines and watching these two casually take components out and handle them with such "care" is the reason I have work nightmares
Lol
this video must have given you nightmares D:
I am sure they can get a replacement part!
I don't work in manufacturing or assembly for these machines and I was still sweating
14:50 here you go
I swear to god I've never been so nervous watching an LTT video.. you guys are killing me! Setting that thing on two edges of an open cardboard box.. I was actually clenched the whole time.
As a enterprise IT guy, the MSRP for that server with the A100s is roughly 150-200k USD. I've looked into it before for some projects that we did.
Edit, my vendor tells me each 12TB ssd from Kioxia is about 5k. That's another 360k on top of the servers.
So here in Australia that would easily be AUD$1,000,000 allowing for exchange rate, GST and 'the Australia Tax' which is a random markup put on because of 'distance etc.'.
@@ozgruntsyd4281 'random markup for distance ' yeah it's called shipping costs. If you look at shipping lanes they basically make a path East to West meaning that Europe and North and South America are all along a roughly straight line so it's efficient and cheaper to ship whereas Australia they have to head south from China just for Australia and New Zealand which is a lot less efficient so the shipping costs are higher.
@@pepperroni6252 We usually get charged a higher base price and shipping on top of that. As for inconvenient shipping the ship can be back at Singapore Japan Taiwan or China before it could have made it to the USA or Europe and no pirates to dodge.
@@johnmccallum9106 the base cost is usually to do with manufacturing costs, Europe Asia and America all have their cheap labour countries. There's many more people in Europe, and the American continents than South Asia and Oceania so it's still more economical for them to ship to those places, also ships will return with products from those countries whereas there'll be less to bring back from South Asia and Oceania so another loss. It's all about minimising loss and increasing efficiency.
What would you say in your estimation the engineering cost of the whole thing would have been? Let's say, another million across all partners?
I sometimes forget how insane some of this stuff is because I deal with it every day. It's always fun to see someone not from the enterprise space see these things and be reminded how cool they are
Yeah it loses a lot of it's appeal when that insane hardware is powering slow as shit VDI workstations.
@@boltinabottle6307 why are they slow?
@@boltinabottle6307 ahhhh a man of culture I see. Best part is when you have to manage those POS VDI's....
@@Abbrahan can confirm; I work for a financial firms enterprise data center, and our virtual workstations are allocated 8 threads of a Xeon Gold 6xxx (I forgot the exact chip), 16gb of memory and around 512gb of storage and even then they are atrociously slow.. when I WFH, I try my best to just remote into my office pc 99% of the time lol
@@RamyWarda A big part of it is network congestion too. I work in NYC and the upper floors of my building with fewer employees do just fine. The 2nd and 3rd floors (which are much larger, with many more users) are slow as hell. Doesn't matter which pool we put them in. Perhaps the 2960x switches are the bottleneck.
(edit: 2960, not 2690)
Mad props to Kioxia... this is by far the coolest tech video I've watched in a long time
18:00 It's a whole lotta fun to see these guys be this excited over such exceptional hardware.
A few moments though where I couldn't not be aware of just how much Linus was breathing directly onto $100,000 of ultra-high-end electronics 7:17 🙊 10:38 😅
I forget how much working in a datacenter desensitizes you to seeing awesome hardware like this, I love how excited they were
Mood
Do you see these insane tech on the daily basis? That's insanneeee
@@user-rd3jw7pv7i I helped install a *dozen* nVidia DGX GPU servers a couple weeks ago. They are $500K each or so, and are basically nVidias own version of this. HPC in the enterprise (or in that case a large research university) is both fun and mind-bendingly expensive. Which is a great gig to have. I have to admit, *I* got excited seeing those giant 450lb monster servers racked together. They certainly looked every bit of $10M bucks. I've been doing Enterprise storage, backup, and virtualization consulting and implantation engineering for 15 years, just getting into the high performance computing stuff. Looked to me like they have the wrong PDUs for those high-amp plugs on the GPU server though. And those DGXs use *6* of those EACH. A mind-boggling amount of power.
@@Kevin_Rhodes I don't even know what to say. I'm speechless.
💯 I just order some new 400Gb network cores and servers with CX6 nics. As you are tossing them around and installing them... you forget they are worth a corvette each.
This is exactly why I love seeing enterprise stuff on the channel, and especially the more exotic stuff, cuz you get to see things like this power supply 5:05 and this weirdo connector for all that massive data 5:39 and the insane amounts of RAM with all the drives. I love it and I seriously can't wait for the next video to come out about this, which doesn't happen often for me lol
Yep. That's stuff i can't get my hands on, and i work in a data center...
I remember those days... Retired from IT Infrastructure now. I was the SAN guy at the office and the day the two+ million dollars of equipment and a couple of petabytes of storage (SAN storage racks, controllers, Brocade switches and boxes of fibre to connect it all) came in, well... I remember that feeling. Christmas does come early some times.
He Mark, I just left them a similar message. I was a network admin back in the 90's and early 2000's and was just like they are today. I miss getting switches, routers, storage, wiring, tape backups, learning to work with fiber, even making hundreds of patch cables back when I first started.
"Warranty void if removed" Jake and Linus: Who?
They're laughing out of excitement like Sponge Bob and Patrick and honestly, I'm laughing along. These crazy projects and the adventure sense they always have about them are one of the top reasons I love LTT so much.
Imagine a day like 30 years later where this is the equivalent to a portable ssd at that time...
One day in 30 years: _Remember when that phone you're holding took up a full 16U rack & needed more power than a house?_
I mean, probably. 40 years ago the Cray Y-MP was as powerful as it got.
It barely meets the requirements for Vista.
@@SaddisticSpeller And right now, that phone is far more powerful than the Apollo guidance computer. And probably has more sensitive accelerometers.
@@NemoConsequentae Well the guidance computer was built with reliability in mind, taking a phone through the van allen belt would instantly corrupt the contents of it's RAM, whereas the guidance computer went to the moon and back going through the van allen belt twice without crashing
@@talibong9518 the amount of gold insulating it is not comparable.
The amount of high pitched laughs just shows how excited these gamer bois are about the new servers! Gaming days at work just got alot more real lol
How else is make going to maintain his minecraft server. Clearly this is what hes been working on since he dropped work on the server last year. Coincidence I think not lol. 1mil mc server hahaha
Except for they don't get to keep
Not for gaming....
@@joshse8709 almost as if you didn't read sarcasm. Nobody would use this for gaming it's over kill and stupid.
@@shanebluebutterfly no shit Sherlock when did you become a detective
The modularity of that hardware is wonderful and I hope we get some of that sort of thing in lower-end/DIY servers in the future.
There's enterprise level hardware, and then there's data center level hardware.
There's data center level hardware, then there's your mom's beside drawer level hardware.
@@FlyboyHelosim your mom out there blowing circuit breakers using over 12 kW.
This is like... HPC level hardware. No enterprise or datacenter is running this level of hardware for normal compute... If you're running this in production you've got a seriously niche need.
@@VexingRaven yup. I mean Linus talks about density, but there has to be a limit. It's hard to believe this is worth the money just for the density.
And then there are research clusters/HPC
I've never been so excited to see someone unbox something before. This is going to be an Epyc series! Thanks Xioxia, Nvidia, and SuperMicro for making this happen!
Kioxia
@@SphinxKingStone must’ve auto corrected or I’m just an idiot, either scenario is likely.
I’ve been so excited for this! After hearing about it from the WANShow! This kind of content is always super entertaining, and the dynamic of these two is so fun!
free stuff its always good
you need a life, man..
Never in the history of the internet has an unboxing won my sub... until today. I NEED to see how this turns out. I've been in the industry since computers didn't come with display outputs.. when storage media was on magnetic tape... this is just absolutely amazing.
10:03 this made me laugh harder than it should’ve 😂
that shit got me so hard i swear, tbh i thought about it and when he said it, i just fkin died
The fact that it came outta nowhere really caught me off guard
@@pyromaniacpenguin I also thought about it at the same time he said it haha that was amazing
Same here bro I almost literally rolled on the floor
These specs are mental! I didn't even know these existed until now
OK noob.
piss off bot
BOT ACCOUNT ALERT 🚨
Then again, in 15 years people will look back at this video and chuckle because their phone has more speed, storage and has a higher bandwidth through a single thin cable...
But yep, these specs are really mental.
@@jbdvries bold of you to assume phones will still use cables, apple will drop it because muh courage then everyone will copy them
Hearing about how this unboxing came about on WAN Show had me even more excited for this video. Glad to see it finally come to fruition, & I guess you’re really going full tilt with this LMG server overhaul (albeit not with this gear specifically since they’re returning it later).
If you followed on WAN Show you also remember they said they have to return most of it sadly
It's not an overhaul lol theyre not keeping
G u haven't even watched the video, it was released 9 mins ago 😂
I believe they are returning all of it and they are definitely returning most of it
@@poletooke4691 True, but the other server stuff that they’re getting is an overhaul.
this is the coolest thing I've seen, cant wait to see it all together!
Oh yeah. This is the kind of content I can get behind.
Also, it's brilliant to have Linus next to all those boxes and crates in the intro. It really makes them look super large.
I work at a data center for a cloud provider and we've been deploying rows and rows of this hardware. It's been pretty fun to deploy and play with the hardware. You will definitely need to rethink your power and cooling solutions to support this stuff.
When I see stuff like this, I have to wonder about the absolutely insane amount of bandwidth that AWS or Google has in total. It has to be at least exabits per second. I don't even know what's past exa but that might be a reality in terms of flops (maybe?)
@@blackbriarmead1966
Giga Byte
Tera Byte
Peta Byte
Exa Byte
Zetta Byte
Yotta Byte
Yeah, they're pulling power from wherever they can but there was no mention of cooling at all. They better have something special lined up.
i'm woking in a data center as well, i would love to see something see to install something like that for a customer in my queue... until now only had smaller netapp shelf or hp vault setups... just like linus, i giggle to myself when see such rediculus brilliant setups!
@@Pussalia Yeah, I hope LTT revamps their server room infrastructure specifically to support this hardware. I don't think they've thought that far ahead yet haha. It'll be fun to watch them figure this out.
I'm super jacked for this series! Can't wait to see it going all together and up and running!! Freaking awesome!!
hihi
I have a SuperMicro 200TB storage with some processing servers in a 10Gbps switch at work, and it is sweet. This monster that they are mounting is just mind boggling. I am very pumped to watch this series unfold.
man i LOVE server / enterprise content. This is next level
The amount of giggling in this video tells me everything I need to know about just how insane this all is.
Well if an average gaming computer has or had 1 GBit at max speed, which barely ever gets fully utilized, then his server rack had a capacity of 12 800 GBit networking.
- That means it could handle almost 13 thousand gaming computers at once, all at max speed which they ever can utilize.
If you drop this value down to lets say FullHD (1080p) content streaming which is something like 3MBit per computer / client. That'll make it capable of serving approximately 4 MILLION 267 THOUSAND customers at once! (Tho can the rest of that server handle that many clients?)
Almost 4,3 Million computers running FullHD video at once? Is this crazy enough? Just one server can serve almost every citizen in my country at the same time, streaming some 1080p content.
The little editing subtitles are absolutely hysterical and I appreciate every single one.
Holy sh*t has always been one of the most entertaining series on this channel. Love every installment no matter how long it takes between two 'episodes' haha. Keep pushing the boundaries!
I have never seen them so giddy. They are like kids on Christmas unpacking best-freakin-gift-eeveeeeeeer
Beautiful, something really awesome about the raw magnitude of this build. A note on the power, hopefully these two power connections are in sync, or being run through a central UPS/building or otherwise to clean it up, but running extra long cable to sensitive equipment of differing lengths can cause a dirty power input, not something to problematic but not ideal and in brown out scenarios can cause a fault down one side of the rack, not sure on the tolerances on the kit. On oneside it should be bullet proof as it's DC equipment, on the the side it's DC equipment and may expect incoming facilities to be within DC tolerance. Can't wait for the follow up videos
With what they are plugging in i would have thought they would get it a direct line of power
@@kirkanos3968 Seriously, they repeated how long this project has been in the works. Was Brian the Electrician not available to extend the service? Oof!
I hope this comment gets some attention, they aren't really going to know how serious that could be so hopefully there can be some input from the few who might be able to answer this.
Agree - those extensions need to be the same length.
Agree, with a million dollars of high end equipment why wouldnt you run dedicated power from a panel to it? No extension cables.
I hold my breath every time Linus is holding things.
And I love how Jake gets to tell his boss "Don't do that!"
Seriously… watching this hardware being handled this way makes me cringe. Casual box dropping, tossing and torquing chassis, forcing mechanicals, no regard for static. I’d fire Linus if he worked for me
@@nicholasmitchell8184 dam linus sweating bruh u rlly got him
It's actually one of the reasons why I don't watch his videos much anymore. He always handles very expensive hardware extremely carelessly for no reason whatsoever. He clearly isn't even doing it for views or anything, he just does it instinctively. He may take a multi-thousand-dollar heavy piece of hardware and flail it around with one hand like it were a coctail shaker (heck, even cocktail shakers are handled more carefully because they are usually shaken with both hands, not just one). This video is no exception (eg. he does that at 8:55).
And, of course, every time he unboxes some really expensive server hardware, instead of doing it safely on a table in a clean environment, he insists in doing it in some dirty storeroom, on the floor, placing it precariously on some cardboard box. I'm surprised he doesn't do the unboxings on a piece of plywood propped up onto some rickety garden chairs... on the roof during a windstorm.
As someone who used to install enterprise gear, I wince every time you guys handle that kit. I'd be in a prepared room with almost zero dust, with anti-static strapping, adequate room to set everything up, have the rack ready to go and in the correct spot, have tested the power and network links, and a thousand other things before that first box got opened. I love the way you guys do this stuff, but it makes my teeth itch and my eyes lose focus when I see a cool mil of kit get thrown around!
Can't wait to see exactly how over-powered this is compared to your needs ;-)
Ah there’s a difference, normal people lose jobs when they drop things … Linus dropping things be like “ok let’s send him more shit to break for fun” then he makes another 100k dollars for the added twist …
@@simonjo1984 Normal people definitely don't lose jobs* when they drop things unless there's gross negligence involved on the part of the person doing the work (not in company procedure planning).
* = assumes the country has worker protection laws AND the workplace isn't the kind where you get fired for taking a week of sick leave.
I spent the last two years as a systems engineer for a large saas company working with gear this big or bigger and it wasn't a big deal. I feel bad for you guys with ridiculous procedures and bosses
I was nervous through this entire video lol
Yeah it's kinda pissing me off how they manhandle that gear..
I'm so jealous of someone getting to use this kind of stuff. Where I work we have tons of very expensive servers and network devices, but I don't really get to mess around with it too much. I just fiddle with my server at home on a dell workstation.
At this part I'm halfway convinced sponsors provide Linus with hardware, let's say, 50% for the advertisement and 50% because some engineers somewhere are wondering "What's gonna happen if we hook up a week of merchandise together?"
Engineers:
"There is a point where we needed to stop and we have clearly passed it but let's keep going and see what happens"
The huge heatsinks next to the GPU coolers, are the NVLink heatsinks, they are the internal network switches between the GPUs :)
Also, please don't remove them, they are not your average heatsink installation (patric from serve the home did a video on why its a bad idea with the previous gen Nvidia GPU node, TL;DR you need like a 1thousand dollar screwdriver to do it without breaking something)
I cant wait for the performance video from you guys, this is a really cool project, really goes to show how far the enterprise segment has come vs gaming hardware
Yeah, pretty sure they are bare die chips without integrated heat spreaders. Really easy to crack the die if you aren't extremely careful or are using calibrated torque screwdrivers.
They are direct on the silicon, and it's die is massive.
6:28 A wild Alex appears. He uses annoyed power.
i'm just an ol'school guy... got started in PCs back when Commodore 64/128 were the 'woo woo' tabletop/home PC... these numbers... woosh! so fast over my head i need an air-traffic control tower on my skull!
but a PETABYTE i had to look that one up... 1024 terabytes, or a million gigabytes... whew... tha hell! :O and that is just in solid state drives... i had a Seagate 4TB HDD go tits up on me a few years back that i was using for 'backup in-house data storage' instead of relying on DSL up n download speeds to 'cloud saves'... and i was crying into my hanky for hours trying to revive the bugger... and these guys are hauling around the equivalency of 256 of those Seagates... gah... mindblowing!
aight! where's part 2! i wanna see the rest of the loot! :D
Thanks Linus for this video! Ive ordered mine and it arrives next week 👍🏻
wow I'm so jealous, I hope one day I can afford it so I finally can run minecraft effortlessly
This for mining?
I work in IT for a university, this configuration alone nearly matches or heavily outclasses (depending on what you measure) the entire server environment we have. Not to mention the other servers LMG wield...
It might be a good idea for LMG to donates servers to Unis
this is a completely insane level of hardware and I'm so excited to see what it can do. Thanks for sponsoring this madness Kioxia!
I work for Supermicro (although I am not replying as a company representative) and this makes me really proud! Thanks for a great video!
HOLY FUCKING SHIT I LOVE THIS. Kudos to Kioxia for balling out so hard, I won't hesitate to buy one of their drives if I ever need an SSD.
@@Lebon19 maybe just a small NVME consumer one... XD
Yes, I've considered Kioxia last week. Not the best option this time but they get my consideration now. I know that sounds big headed but my customers would know Western Digital but not Kioxia.
@@wayland7150 exactly that! A while ago I'd thought the same as your customers. Now I've changed my mind and would shortlist them in my comparison against drives by other manufacturers. I guess that's what they try to achieve with these videos. I won't autobuy them, but I will consider them as I would Samsung, WD or Kingston.
This is INSANE, love seeing crazy stuff like this.. just.. insane to see the progress tech is making
"performance we may well never see again"
I'm willing to bet that it will be just a couple years before LTT does the next upgrade which will make this rig look just ok.
I can see it now...
2028.
LTT is about to unveil their new "Editing Server"....
And Linus goes... Yes ladies and gents... This thing will have 2 Zettabytes of storage.
What happened to our Petabyte server you ask? It's in the media/gaming lounge. It has just enough storage to have COD 22 - The Venusian Wars and expansion packs installed.
More than just a couple years probably, as they aren't actually getting to keep this.
@@5eeeeeb5 I know this is a joke, but for those who don't realize, yeah, this is a joke.
There's no way a single game will be a petabyte in just 6 years.
I doubt they will be able to use fraction of this setup's performance
I thought that too. Go back 10 years and tell Linus that he'll have a petabyte of nvme flash in his hands and he'd either scoff or be absolutely floored.
I need one of these so i can install my whole steam library finally
I was in Student Cluster Competitions in uni and I can vividly remember the feeling of finally assembing a server when all the gear arrived from sponsors. No biggie, just a $100k chunk of metal dented because someone at the airport couldn't care less
Of all the server hardware that I never seen it before, I think it was the most beautiful and the most insane server hardware period. NVIDIA has the design appeal on the A100s plus the memory, NVMe etc.
This is the datacenter's wet dream
You're going to need really robust and well-protected and filtered C20 PDU's for those compute units. You'll also need a minimum of L6-30 30A supplies with 50A preferable. Just FYI
I can imagine the surge spikes when a load hits the server. You're going to need some kind of battery backup buffer for sure.
They have extension cables and their data centre is a room next to CNC machines and welding equipment! This hurts my data centre soul.
@@marc0523 Today we find out RUclips Video channel is not a data center.
@@marc0523 the room is seperated by a concrete wall
@@bisbis2.0 It's good that they're keeping that server safe from the server room. I've never seen their shop catch on fire, but their server room has.
8:11 Lol Jake, the Heat Sinks 😄. You guys are Awesome in this one, Love the Energy & Tempo!
SuperMicro, good call. Love their hardware. On a side note, as crazy as that architecture setup might seem, it's got a 3 to 5 year shelf life in the enterprise space before it's decommissioned and replaced by something that's probably twice the performance. The companies that can afford that level of gear outright are constantly seeking higher performance/density. Your bottleneck will likely be the CPU core interconnects depending on how you segment the compute and storage and the software you overlay on top. Definitely going to need Linux and some creative heavy hitting to really get the 'full spec' out of this setup.
3 to 5 years is about right with the gear decommissioned from main data handling to backup pools.
Supermicro servers are great, theyre so easy to work on compared to the lenovo servers we have at work.
Linux go burr
Windows go 😱
I just love the "chaotic smart" energy of Linus & Jake. Whenever I see a video is with both of them, I gotta watch it right away.
"Performance which we have never seen and may well never see again."
I'm not a fortune teller but I strongly feel the second half of that sentence will age really bad really fast.
He did self correct and say their lifetimes. But I guess we'll see
@@satellitereigns Yeah, I heard that. They are really young tho. I'd consider 5 years really fast compared their remaining life and I'd be shocked if today's existing tech would be still the fastest at that time.
Exactly my thoughts. If we are looking at how computers evolved in the last 30 years we can safely assume that even in their lifetime (Linus is around 35) there will be grander builds than this. I might gamble to say that even in the consumer space, not just enterprise level stuff.
@6:38 "So do we slide it in?" - "Uhhhh, I think we should put it in the rack first"
She's a gorgeous piece of kit. Love to the sponsors for this project, can't wait for the rest of the series.
Make sure this server and your welder (when you plug it back in) dont share any breakers/lines. The voltage drop created by welding can cause the server to crash.
Near the end of the video, they said they were, and that they just wouldn't run the welder, since this thing needs so much power. They're using another circuit in addition to the welder's.
if they run the server and welder at the same time, the neighbourhood looses power
@@kitsunekaze93 they might need to establish LTTown.
This had better be on their massive UPS. I'm so paranoid about power fluctuations and noise that I am wondering if I need to EM shield my garage from my welder, as I have a TV on the other side of the wall from my garage. (And I am considering a whole-house UPS for everything NOT in the garage)
Back in 2010-2016, I was working in a server software test lab as the sysadmin, and I had the pleasure of building several test units for new storage.
In one case, I think 2013 or 2014, we built a whole rack OBS system with the storage units being 12X4TB drive 1U Asus servers (I think there were twenty) and the 3 head units being all SSD 1U Dell servers. The servers interconnected internally to the storage in a local dedicated network in all 10Gb, and then served out to the rest of the world with 6 10Gb connections. It was $1.5 million for the test unit, and I had the pleasure of assembling it, with the instructions from the developers. It was insanely powerful at the time.
I know how fun it is to open and assemble such things.
Kioxia's response just proves to me that the people LTT were talking to were engineers. Engineers love an interesting challenge like this
Seeing Linus and Jake's tangible excitement has me looking forward to this even more. Lookin like kids in a candy shop lol.
I don't know anything about any of this hardware, but judging by their reactions and how giddy they got every time they opened a box or a cover, I could tell it was good stuff.
You can tell it's expensive when linus is being careful not to drop it 🤣
This is the type of stuff that runs RUclips.
Its the stuff of supercomputers. The most powerful CPUs, insane amounts of RAM, high performance interconnects. It seriously doent get higher end than this, the only way up is just more of the same (and the hardware is made to scale near infinitely, add more hardware get more performance).
12:11 so that's why LTT water bottles do not have necks?
Linus: *mentions silicon storage*
Linus in the same five minutes: *threatens every piece of silicon within a 5 foot radius*