Really appreciate all the advice in the comments on this one. Some changes I made: Switched to 5x12 drive vdevs. Link agg off. Jumbo frames on for the whole network. Data Scrubbing scheduled. I haven't enabled snapshots because I think the max size for that is 200TB. Let me know if there's anything else you recommend!
You guys need proper storage devices that also considers cold storage, read time, malware, ups etc. You need a professional storage solution and relevant network to make use of it. Maybe I can hook you up from worlds largest ICT vendor.
Still can't get over Linus' face when Gavin told him his server would last him "until the end of summer", when Linus thought he'd be good for at least a year or two lmao
The format definitely supports that (I go back and forth on "cine is a wonderfully flexible format well-suited to scientific video" and "it's just a stack of BMP files and some timing information"), but I'm not sure there's good tooling for it and I don't think Gavin wants to add a couple of hours of swearing at Matlab or Python to his editing workflow. If there is good tooling for it lmk, my data is about to get at least 50% better-looking.
Nope, just brightness recorded for each pixel, like the raw output of a camera. The main customer for the high-end cameras is research, and the whole field has spent too much time going "Compression artefact or interesting physics?" to trust most video compression.
@@nicholastaylor3128 I mean, there are lossless compression options, and slow motion video has many not changing pixels. I guess just a compression and deleting frames from speedramped sections, could mean saving 90% of the storage space.
Personally the speed-ramping, iconic background music and added sfx is what makes Slow Mo Guys. Well worth the added storage requirements and associated cost. Great stuff Gav, keep it up!
i only ever understand about half of what gavin says in these types of videos, but it's always fun to hear people much smarter than me talk about stuff they're interested in
I understand all of it and that's what mortifies me. This dude probably has more HDD space in his living room than I ever encountered across my whole life.
And with that much data storage available, hopefully they'll have enough for the next few years, or *at the very least* until the end of the coming summer (and that assumes they don't get an even better ultra-high-speed camera that generates even more data per second).
As a sysadmin it's so satisfying to hear you talking about LACP and SMB and expandable storage and having a really practical use for it all. Most people myself included either have the skills but not the use case or vice versa. Your situation is super unique in that you both have the need and the expertise 😀
"Most people myself included either have the skills but not the use case" Everyone has the use case but not the money to spend. Like just think about saving Porn-h-u-b to local storage, that takes a few TB.
@@Varangian_af_Scaniae Yea, lol. I play a lot of msfs2020 and I dream of downloading a local copy of their world map. But that about 50 times what Gavin has here and costs as much as a house....
Last time I counted, I think I have around 30TB, but I've gotten rid of a few computers here and there. I use it because I archive a lot of online stuff. So if a RUclipsr deletes a video, I can still watch it / reupload it for archive reasons.
Well there is a camera named after Vera Rubin. Which captures 3.2 GIGApixel photos and it's meant to use 24/7. The write speed of that is 60 petabytes in a day. That is roughly 4x more than the Slow Mo Guys best camera can capture.
I get the feeling that the technical explanations you're giving are 95% explaining to the audience, and 5% re-convincing yourself that, yes, you do need this capability. :)
I love this channel. The behind the scenes stuff for all of this content is so interesting. Especially, since there's another video on this channel of you just shoving hard drives into a massive PC case and calling it a day. Nice to see that you finally have a professional storage system that can grow with you and Dan and your antics :)
I LOVE these 2nd channel videos. There's just something so soothing about listening to someone talk about something they're passionate about. I could watch all day and never get bored.
I was worried when you did the tape backup video you wouldn't also supplement that with an online storage system, so I'm glad you're adding this petabyte. Now I'm counting down the days until we see you become the first RUclipsr with an exabyte!
We’ve actually had this synology longer than the tape. Tapes are currently the backup of this server in two locations. And all the data was put on by our older NAS units that combined are another complete backup. Sounds like I should use vdevs with less drives though.
Honestly, I really appreciate the technical level that you were explaining this video in. I've been working my first IT job and am going into the Networking field, and being able to keep up with the protocols you were talking about just made realize how much more talented you and Dan are outside of your perceived field.
I just wanted to say, since you did that collaboration with Linus, I have fallen in love with his content and RUclips channels and have been watching essentially every upload of his since. It's so funny how one collaboration can greatly impact new viewers, using myself as a prime example. Of course, I still love The Slow Mo Guys' stuff, too!
Out of the entire video, the one thing I appreciated most was right at the start, you straight up saying that you got the system for free. No trying to hide it or be discrete, just straight up "Hi, here's all this stuff. It came from this company and they gave it to us for free."
I have been binge rewatching @howridiculous for the last day or so, I wonder what their data storage looks like considering they do slow mo too! Would love to see a collab between you lot! The banter would be off the charts!
@@CK-ceekay 10 cameras with 3 hours 4k footage would generate 500GB or so. If they don't discard any footage (and they probably do), they still wouldn't be close to a Petabyte.
The howridiculous guys dont use a phantom that can take 90k frames a second right so there files are probably big but not as big as a single 2.5TB video thats arround 5 to 8 minutes of slowmo footage
Yeah I never thought about how much data slow mo footage was taking up. It’s awesome and now I also want to know what those aussies are packing in their editing dept.
Hey Gav, You have two PSU, you really should really also go with two UPS. And if possible two different breakers. One failure mode of psu is to fail short. This mean that the UPS connected to it will fail, and may trip the breaker. If the breaker trip, then you lose power, and possibly lose data. Also, make sure that both UPS are sending signal to your server so if the power fail the server knows about and will shutdown safelly. Also, you really should get a spare hard drive!
To add to this, rack-mounted UPS would be the way to go. They should be on your network for remote monitoring. The rack probably needs at least 3x 20A circuits (each with a UPS): 2x for the server and a third for the network equipment.
+1 to the spares. with that many drives you don't want to try searching for same model replacements years later. Buy 4-5 spares now and be set for years to come.
@@BenjaminKramerI left a comment telling him to get a large window A/C to help keep the room specified for the server rack to cool it down to 60*F. I’m a mechanical engineer for 30 years. I’ve designed my fair share of them and that’s my set point on a separate roof top unit. If he really wants to get fancy he can duct part of the discharge from the A/C directly into bottom portion of the server cabinet. Open the top for the heat to escape. There’s a lot of ways to get it done. It just needs to be done.
Sound cabinet still needs to have some form of proper external ventilation connection, otherwise the system will cook itself off or experience heat related failures under load. If the fans are just recycling the same hot air in a confined space it will rapidly create a thermal feedback loop which risks significant damage.
I grateful for the content you and the guys create, surely one of the top prime content on the tube! I always look forward to yet another slomo video, with great commenting and humour! Cheers!
I'm thinking about getting one but I can't decide between the DS418 / DS420j / DS420+ ... What justifies the cost of going for the + ones over just the j?
@@gargaj oh, I just found out that only the + models (or more technically correct: the Intel CPU models) can run docker! So there's a big differentiator right there, if you care about that.
See, i think i know a real good bit about building PCs and other tech, then hearing Gavin talking about cameras and servers makes me realize I can definitely learn more. Also some of Gavins files are larger than both my SSDs, Jesus.
Mechanical Engineer here. If I were you, the room that you store that server and cabinet in your house, I would get a window air conditioner big enough to cool that room to 60*F for the simple fact that server DO NOT like heat and tend to melt hard drives. I know this is late to the party but this is the cheapest way to cool the server itself while not killing your house A/C or your wallet. I’ve designed several casino’s around me and other server room systems. I always allocate a separate unit or multiple units for those rooms depending on how big they are and what the heat load calculations show me. Like I said, I know I’m late but I hope you get it figured out. Love what you do and I sub to both channels.
You put 24 drives into a raid 6. You're going to lose everything, guaranteed. The likelihood of two additional drives failing during the very first rebuild you have to do is effectively 100%. The way you've set this up is insane. If you don't want to take my word for it, then please, for the sake of all your video, send Linus a message and ask him if a 24 drive raid 6 array is bad idea.
This is the comment I was looking for. I was watching the video wondering if he would pick banks of 6, or 8 to push it, never 24. That's actually endangering the data as you said. I wouldn't call it a 100% failure guarantee but it's moved the risk needle way to the top. This means 3 disks failing will fail all 24, etc.
RAID6 should definitely use pools of 5 to maybe a top of 10 or so drives, especially in a large array like this. Also I spoke to Wendell about it a bit, and he suggested leaving some drives offline until I get it more full, to cut down on power use until I really need all the space. Of course, for SloMo footage... I guess they're already using all that space lol. But having a ton of drives like this is one area where ZFS can really shine. Unfortunately Synology and other NAS vendors don't necessarily support more advanced drive layouts as easily as something like TrueNAS.
In some ways, totally wild how much memory you can fill up in such a short amount of time. In other ways, super cool to see how other folks who need storage/memory/bandwidth solutions can get so much more common/non-specialized access to high-bandwidth tech like this in such an accessible way :) always excited to see whether I can learn something from this to apply to my day-to-day large-scale-data-related job. Thanks Gavin!
Consider using jumbo frames. The default MTU of 1500 bytes per frame was designed for 1 to 10 megabits/sec. You are running 10 to 25 gigabits/sec. Most NIC's support higher MTU settings. The key is that every NIC and switch in your entire network must be set to negotiate a higher MTU. Once setup you will see a massive increase in network speed.
I've deployed one of these units in production for a university a few years ago (using 8TB drives). They are every bit as good as much more expensive NASs. Congratulations.
LTT themselves are in a very similar situation to you with file bloat, they're running like 8 RUclips channels. They also actually had a fire in their server rack.
Honestly, I'd say Linus is one of the few big youtubers that could go toe to toe with Gavin's storage usage. Like you said, he has multiple channels churning out videos on the reg. They shoot on high end, top of the industry video cameras at 4k resolution and they keep all RAW files of which I can assure they produce at least a couple hundred a day minimum.
@@jjpark98 They actually film at 8k at LTT, and have some stuff they did at 12k. Which makes the storage needs all that more necessary. They said they do that for zoom effects while still delivering true 4k.
@@mastershooter64 When they upload to RUclips, yeah, they use H.264. But having the raw files available allows them to do a lot more with things like color grading.
The only problem with raid 5/6/7 is the write hole when you lose power during a write operation but it sounds like you have that covered. With a good UPS and a redundant PSU it's unlikely you'll lose power in the middle of a write operation without safely shutting down (provided automatic shutdown is configured) Also don't forget to scrub your data at least once a month! You can still get bit rot with any drive configuration!
In the beginning, you highlight the reason why you keep the entire 2 seconds, and honestly, I think a lot of people watch the videos partially because of that, the number of times I've only seen the "important" part and no context of slow-mo stuff on the internet and I was sitting there frustrated to not see more and you guys make it available for us. As an ltt fan, I've seen Linus build its own petabyte server and it's a huge amount of storage and that's definitely something that content creators will need more over time. For Linus to post 17 videos a week (the last time I heard them talk about it it was 17), you need a lot of storage. And you highlighted perfectly why you need It too. I think everybody hears 288Go per file and doesn't really realize that most smartphones couldn't even store a single file like that on it.
I feel like the importance of Smee’s contribution to this video cannot be overstated. A true master thriving-nay, REVELING!-in a highly specialized field.
I know you do your own thing and you guys are probably busy or whatnot but your 130TB Collab with Linus was absolutely hilarious. Maybe when you need a 100PB or 1 EXB server you can do another one!!!
Congrats on all your success and great content over the years! I always enjoy the content. Cool getting a peek behind the curtain here, and I will say that the speedup portions are enjoyable and put the slo-mos into context very well. Didn't think about storage, but yeah I can see why network storage would help! Synology has a good reputation too, the Twit TV folks are always recommending them.
Not that lossless general purpose compression or data deduplication should have any real significant space saving benefits for your use case necessarily, but I think it would still be interesting for just exploring if you can at a larger scale of the slow mo guys storage usage. Windows Server since version 2012r2 has had a way to get a file system to run deduplication at the level of the filesystem and if you prefer Mac OS or Linux, there is a way to get a similar if not better solution (ZFS). I guess my point is that at your scale of sheer quantity of storage in terabytes used, even a few percent of savings at this scale could start having significant cost savings and thus more room for more video or just giving you more money since you might have less need for a few of the drives. Also it's kinda industry standard at least in advertising storage to use base 10 notation to describe the capacity instead of binary. So yeah it seems like there's a small amount of tradition in rounding up storage space. Also parity and redundancy and on occasion formatting of the filesystem might loose a little bit of space.
On 5:12 I notice you still on 1500 MTU, if you up the number to 9014 MTU, you will have much better performance, but you also need to change the MTU on the network switch and on the computers that you wish to file transfer. This will help a lot since you dealing with very large files.
Curious if this will make the Blu-ray idea mentioned a while back easier? I don't own any Blu-ray disks, but would definitely make an exception for this 🙂
You've probably already thought about it since it's mentioned in the video, but consider having an off site backup. The biggest hazard to the server is something like a fire or a storm or something, which would also threaten your tapes if they're stored in the same location.
I'm awestruck by whatever cataloging system you must use to keep your data storage straight and reasonably available for your use. A petabyte I can imagine. Keeping it usefully, I can't imagine.
You could make a detailed tour of your editing workflow? From ingesting, over editing in Resolve to delivery. Would be awesome! What Programs do you use apart from Resolve?
You don't have to worry about storage, for now 😅 In 5-10 years this 4 Petabyte storage will seem like the 250 TB storage you just replaced! Regarding the power draw, long time ago I learned that you should calculate approx. 10 Watts per disk (and of course add the rest of the hardware as well), so this system is probably drawing around 750-800 Watts I guess?
I've got a Synology DS420j, it's getting on a bit now but it's not let me down yet. I'm only rocking about 32TB of storage in my 4 bay NAS, but it works for our needs!
Currently studying digital preservation and whilst I only understood about half of the words you said this is a fascinating solution study for large files and example of readability needs and variations! Ta.
do you realise how easy it would be to lift 1000 1TB microSD cards? at ~250mg each, lifting up 1PB worth of data as microSD cards would only weigh 250 grams. seems an awful lot lighter than the Synology setup. oh yeah, it's Synology by the way, not 'synospstic'. not sure how you arrived at that spelling there, Einstein.
I'm a data hoarder myself but I only got Synology DS920+ with 16TB of space, approaching 13TB. Great unit. I just got the new Synology router which is a beast and so far I'm very pleased with it
You might want to look into colo (Colocation) server racks. Basically it is a datacenter that sells rack space. I have used it for my homelab and the energy and ISP charges are typically reasonable and would take down the power/heat/noise in your house. Also room for growth with those expansion bays.
For those of you interested. the Synology HAS5300-16T drives cost about 600 gbp / 750 usd a piece racking up something like 36K gbp / 45K usd just for the drives alone. Im sure cheaper as part of a bundle though. Free for the slo mo guys though!
I have almost no idea what any of what Gavin talked about meant, but I could listen to Gavin talk about his tech hardware for hours. It's weirdly therapeutic or cool.
God I love these behind the scenes videos, especially when it's about tech or specifically cameras/storage. I started out with a WD myCloud with 2x2TB (Raid 1), after that got filled I put in 2x4TB (also Raid 1). After that was filled I mangled my system storage with 2TB without any backup. Now, just before I went to Iceland, where I shot a lot with my Ninja V in ProResRAW ("only" around 500-ish MB/s and still no native support in DaVinci), I bought a QNAP extension unit with 2 bays. It's basically an external hard drive enclosure with a hardware raid controller, which could be used to, as the name said, a QNAP NAS to extend it without needing to buy new storage or a bigger NAS. The extension unit got loaded with 2x18TB Seagate Drives (could've gone for 20TB, but the 18TB drives were a way better bang for the buck) also in Raid 1, and is now filled with 7TB just after the Iceland Trip. Hopefully it will last me a long time until I'll eventually go for a 12 bay unit with 18 or 20 TB drives. It's not the best solution, but a surprisingly cheap and easy option for people who need big storage on the go, to terminally safe it on a NAS in the long run.
Back in 2012 a friend of mine built a PC with 1TB of storage and that was astronomical at the time. PB is one thousand times bigger than that. So much space!!!
based on your level of storage consumption, a petabyte is going to be filled really fast. like i said, the lhc (large hadron collider) uses this type of storage system with a few other levels. first it hits ram, then a temp storage buffer (ssd array) then it offloads to the primary buffer (a giant array, think they went ssd not long ago, but it is several petabytes by itself) then they offload that to tapes in duplicate or more, then even that gets offloaded to more tapes. they also have another step of a robotic tape machine that will allow researchers from around the world to get a look at data without being connected to the primary storage server. the lhc does at least 15 petabytes a year though, you should be fine for a while.
What the slow mo guys need is actually a ceph cluster! This allows one to organically add (and remove) storage. Also, had synology had not required use of their proprietary drives, this server could have used 22TB drives and stored considerably more (almost a third more.)
First time ever you don’t have to worry about storage. I don’t believe that. The speed at which you’ve filled everything else I have no doubt this new server will be bursting at the seams very soon.
I'm a music producer and thankfully audio is SIGNIFICANTLY smaller than video. My total storage I have for my music is about 10 TB across various places. Cloud, harddrive, computer etc. Basically as backups. But I probably use about half of that .. for now. Quite a lot of the space is taken up by sound libraries I've collected across the 5 years I've been making music. I keep everything I do and try my best to backup as much as possible. Even for me storage is one of the chores of my job, can't even begin to imagine what its like for you :)
you should see about using TureNAS one of these times for your next storage server. 100Gbe support natively (hardware dependent) and also able to have more control over what is going on.
Really appreciate all the advice in the comments on this one.
Some changes I made:
Switched to 5x12 drive vdevs.
Link agg off.
Jumbo frames on for the whole network.
Data Scrubbing scheduled.
I haven't enabled snapshots because I think the max size for that is 200TB.
Let me know if there's anything else you recommend!
Is it worth having a bit of additional SDD for the few videos you are currently editing at any given time? Would that speed up the editing?
@@stevengreidinger8295 or caching
You guys need proper storage devices that also considers cold storage, read time, malware, ups etc. You need a professional storage solution and relevant network to make use of it. Maybe I can hook you up from worlds largest ICT vendor.
How much would this have cost if you paid for it?
@@SilentRacer911 the case without the drives is listing at $17,000 - $19,000 depending on the seller.
Still can't get over Linus' face when Gavin told him his server would last him "until the end of summer", when Linus thought he'd be good for at least a year or two lmao
Hehe, I just commented the same thing
.
I was thinking the same thing. Good to see that storage server still working.
Linus underestimated the slo mo lol
Did I miss something? What part of this was Linus in??
@@kaedenmurphy9937 ohhhh… thanks.
Thank you for your speed ramping Gav! It's part of the authentic SMG video feel. We appreciate you and your ongoing quest for storage.
but for speed ramping, the cine files could drop the frames not needed any more. It would solve the storage issue for another couple years.
The format definitely supports that (I go back and forth on "cine is a wonderfully flexible format well-suited to scientific video" and "it's just a stack of BMP files and some timing information"), but I'm not sure there's good tooling for it and I don't think Gavin wants to add a couple of hours of swearing at Matlab or Python to his editing workflow.
If there is good tooling for it lmk, my data is about to get at least 50% better-looking.
@@nicholastaylor3128 no way, the frames are not in a compressed format?
Nope, just brightness recorded for each pixel, like the raw output of a camera. The main customer for the high-end cameras is research, and the whole field has spent too much time going "Compression artefact or interesting physics?" to trust most video compression.
@@nicholastaylor3128 I mean, there are lossless compression options, and slow motion video has many not changing pixels. I guess just a compression and deleting frames from speedramped sections, could mean saving 90% of the storage space.
I'm almost sad that it's not a 3 hour video of Gavin pulling harddrives out boxes in slow motion.
The fact that you guys need a petabyte of storage is absolutely mind blowing. It's inconceivable how much that really is.
Pretty conceivable if you do slowmo, apparently
For a sense of scale, a 4k blu ray movie is typically on a 50GB disc. 1 PB of storage is roughly equivalent to 20,000 4k blu ray discs
@@NPzed jesus christ.
Now...understand that the Apple //e I used in high school was 64K. Total.
Mindblowing.....beyond mindblowing.
@@RICDirector 64k of ram, no hard drive/on board storage at all.
Personally the speed-ramping, iconic background music and added sfx is what makes Slow Mo Guys. Well worth the added storage requirements and associated cost. Great stuff Gav, keep it up!
i only ever understand about half of what gavin says in these types of videos, but it's always fun to hear people much smarter than me talk about stuff they're interested in
Second that...!
He has a great ability to explain things in the simplest of terms
About 6 months in IT and you'll understand everything, it's not really advanced stuff
I understand all of it and that's what mortifies me. This dude probably has more HDD space in his living room than I ever encountered across my whole life.
I think I speak for all of us when I say: Gav and Dan don’t ever quit
Indeed.
And with that much data storage available, hopefully they'll have enough for the next few years, or *at the very least* until the end of the coming summer (and that assumes they don't get an even better ultra-high-speed camera that generates even more data per second).
but they do slow down.
As a sysadmin it's so satisfying to hear you talking about LACP and SMB and expandable storage and having a really practical use for it all. Most people myself included either have the skills but not the use case or vice versa.
Your situation is super unique in that you both have the need and the expertise 😀
As a sysadmin I can agree!
"Most people myself included either have the skills but not the use case"
Everyone has the use case but not the money to spend. Like just think about saving Porn-h-u-b to local storage, that takes a few TB.
@@Varangian_af_Scaniae Me while reading through your comment: "no thats not true, i only have a 8TB and it fits all my da.... yea i agree."
@@GamingXperienceYeah one have to be prepared for the day the Internet goes down. Haha😄
@@Varangian_af_Scaniae Yea, lol. I play a lot of msfs2020 and I dream of downloading a local copy of their world map. But that about 50 times what Gavin has here and costs as much as a house....
I’m fascinated to know how much total storage you guys have
Last time I counted, I think I have around 30TB, but I've gotten rid of a few computers here and there. I use it because I archive a lot of online stuff. So if a RUclipsr deletes a video, I can still watch it / reupload it for archive reasons.
2TB
Onsite? Offsite? Or total?
ive got 40TB stuffed into my PC case lol
I've got 5TB in my main PC for OS and games and 6TB Raid 1 in my NAS
Gav in 10 years: Welcome to our newly purchased Server Farm, with one Zetabyte of storage. Hopefully, this will keep us going until the summer.
Get two UPS, one for each power supply. All major data centers include to completely separate power rails.
I always love seeing Gavin talk technical, it's just so obvious that he loves talking about what he talks about
This is an almost impossible amount of storage to wrap my mind around.
Ltt has 2petabytes i think
Now imagine how much storage google/youtube has to support all those videos.
Well there is a camera named after Vera Rubin. Which captures 3.2 GIGApixel photos and it's meant to use 24/7. The write speed of that is 60 petabytes in a day. That is roughly 4x more than the Slow Mo Guys best camera can capture.
@@CPT_Nelson At least multiple Exabytes- thousands of times what Slow Mo Guys has
@@CPT_Nelson Probably close to the amount of shekles embezzled by Israeli politicians.
Its so satisfying to see everything in one folder.. Congrats dude
It's something we strive to achieve in the storage world. One folder and done.
@@jjpark98 its so majestic
I get the feeling that the technical explanations you're giving are 95% explaining to the audience, and 5% re-convincing yourself that, yes, you do need this capability. :)
I love this channel. The behind the scenes stuff for all of this content is so interesting. Especially, since there's another video on this channel of you just shoving hard drives into a massive PC case and calling it a day. Nice to see that you finally have a professional storage system that can grow with you and Dan and your antics :)
It had never occurred to me that speed ramping was so incredibly demanding. Cheers for the lesson, Gav.
I LOVE these 2nd channel videos. There's just something so soothing about listening to someone talk about something they're passionate about. I could watch all day and never get bored.
These videos are some of my favorites that you do, no joke. The techy insanity of storage and possibilities.
I was worried when you did the tape backup video you wouldn't also supplement that with an online storage system, so I'm glad you're adding this petabyte.
Now I'm counting down the days until we see you become the first RUclipsr with an exabyte!
We’ve actually had this synology longer than the tape. Tapes are currently the backup of this server in two locations. And all the data was put on by our older NAS units that combined are another complete backup. Sounds like I should use vdevs with less drives though.
@@TheSlowMoGuys2 As you can see how many tech guys comment, should make follow up video about using less drives per vdev.
Linus will probably beat him to an exabyte with all the servers they keep making
@@WayStedYou which Whonnock server is he up to now? 4? 5?
I lost count after new NEW Whonnock.
I’ve noticed the slow down for a long time and it’s always been one of my favourite parts of your videos, truly seeing the whole thing
I love the ramp ups. It's certainly one of the greatest things about what you guys do.
Honestly, I really appreciate the technical level that you were explaining this video in. I've been working my first IT job and am going into the Networking field, and being able to keep up with the protocols you were talking about just made realize how much more talented you and Dan are outside of your perceived field.
I just wanted to say, since you did that collaboration with Linus, I have fallen in love with his content and RUclips channels and have been watching essentially every upload of his since. It's so funny how one collaboration can greatly impact new viewers, using myself as a prime example.
Of course, I still love The Slow Mo Guys' stuff, too!
Nice to see how helpful Smee is when you unpack something!
Nerdy hard drive videos with Gav are my absolut favorites. Guess my delievery deadline can be pushed another 10 minutes 🤣😅
Out of the entire video, the one thing I appreciated most was right at the start, you straight up saying that you got the system for free. No trying to hide it or be discrete, just straight up "Hi, here's all this stuff. It came from this company and they gave it to us for free."
I have been binge rewatching @howridiculous for the last day or so, I wonder what their data storage looks like considering they do slow mo too! Would love to see a collab between you lot! The banter would be off the charts!
And that they release a 15-20 minute video every week, and usually looks like they're running 5-10 cameras a shoot
@@CK-ceekay 10 cameras with 3 hours 4k footage would generate 500GB or so. If they don't discard any footage (and they probably do), they still wouldn't be close to a Petabyte.
The howridiculous guys dont use a phantom that can take 90k frames a second right so there files are probably big but not as big as a single 2.5TB video thats arround 5 to 8 minutes of slowmo footage
Yeah I never thought about how much data slow mo footage was taking up. It’s awesome and now I also want to know what those aussies are packing in their editing dept.
"won't have to worry about storage space" - fifteen minutes later "we've filled it up"
Hey Gav, You have two PSU, you really should really also go with two UPS. And if possible two different breakers.
One failure mode of psu is to fail short. This mean that the UPS connected to it will fail, and may trip the breaker.
If the breaker trip, then you lose power, and possibly lose data.
Also, make sure that both UPS are sending signal to your server so if the power fail the server knows about and will shutdown safelly.
Also, you really should get a spare hard drive!
To add to this, rack-mounted UPS would be the way to go. They should be on your network for remote monitoring. The rack probably needs at least 3x 20A circuits (each with a UPS): 2x for the server and a third for the network equipment.
+1 to the spares. with that many drives you don't want to try searching for same model replacements years later. Buy 4-5 spares now and be set for years to come.
And don't forget your thermal management. A sound-dampened cabinet is great, until it becomes an oven.
@@BenjaminKramerI left a comment telling him to get a large window
A/C to help keep the room specified for the server rack to cool it down to 60*F. I’m a mechanical engineer for 30 years. I’ve designed my fair share of them and that’s my set point on a separate roof top unit. If he really wants to get fancy he can duct part of the discharge from the A/C directly into bottom portion of the server cabinet. Open the top for the heat to escape. There’s a lot of ways to get it done. It just needs to be done.
Or 5
We use Synology as our backup at my job. Extremely useful, robust, and can make backing up data super easy to store with their snapshots.
Sound cabinet still needs to have some form of proper external ventilation connection, otherwise the system will cook itself off or experience heat related failures under load. If the fans are just recycling the same hot air in a confined space it will rapidly create a thermal feedback loop which risks significant damage.
I grateful for the content you and the guys create, surely one of the top prime content on the tube! I always look forward to yet another slomo video, with great commenting and humour! Cheers!
Love it. I've got a DS918+ so I'm a wee bit behind Gav's level of usage, but I am certainly a Synology fan!
I'm thinking about getting one but I can't decide between the DS418 / DS420j / DS420+ ... What justifies the cost of going for the + ones over just the j?
@@gargaj oh, I just found out that only the + models (or more technically correct: the Intel CPU models) can run docker! So there's a big differentiator right there, if you care about that.
And outside that? I'm not particularly interested in Docker.
@@gargaj Plex? Can you think of anything you might use a NAS for over what the stock apps give you? If not then a J will do you just fine.
Honestly not much, it'd be more of a backup storage thing. Then again I have no idea what I'd be missing out on.
In all the years I’ve watched your content I’m always impressed with it all. Cheers.
I'm a techno-geek. This was fascinating. Thank you!
See, i think i know a real good bit about building PCs and other tech, then hearing Gavin talking about cameras and servers makes me realize I can definitely learn more.
Also some of Gavins files are larger than both my SSDs, Jesus.
Mechanical Engineer here. If I were you, the room that you store that server and cabinet in your house, I would get a window air conditioner big enough to cool that room to 60*F for the simple fact that server DO NOT like heat and tend to melt hard drives. I know this is late to the party but this is the cheapest way to cool the server itself while not killing your house A/C or your wallet. I’ve designed several casino’s around me and other server room systems. I always allocate a separate unit or multiple units for those rooms depending on how big they are and what the heat load calculations show me. Like I said, I know I’m late but I hope you get it figured out. Love what you do and I sub to both channels.
You put 24 drives into a raid 6. You're going to lose everything, guaranteed. The likelihood of two additional drives failing during the very first rebuild you have to do is effectively 100%.
The way you've set this up is insane. If you don't want to take my word for it, then please, for the sake of all your video, send Linus a message and ask him if a 24 drive raid 6 array is bad idea.
This is the comment I was looking for. I was watching the video wondering if he would pick banks of 6, or 8 to push it, never 24. That's actually endangering the data as you said. I wouldn't call it a 100% failure guarantee but it's moved the risk needle way to the top. This means 3 disks failing will fail all 24, etc.
@Level1Techs
I can't speak to whether you're right, but he explains this isn't backup. This is access. The backup to this was covered in an earlier video.
RAID6 should definitely use pools of 5 to maybe a top of 10 or so drives, especially in a large array like this. Also I spoke to Wendell about it a bit, and he suggested leaving some drives offline until I get it more full, to cut down on power use until I really need all the space.
Of course, for SloMo footage... I guess they're already using all that space lol.
But having a ton of drives like this is one area where ZFS can really shine. Unfortunately Synology and other NAS vendors don't necessarily support more advanced drive layouts as easily as something like TrueNAS.
@@JeffGeerling Agreed I have two hotspares on my array.
In some ways, totally wild how much memory you can fill up in such a short amount of time. In other ways, super cool to see how other folks who need storage/memory/bandwidth solutions can get so much more common/non-specialized access to high-bandwidth tech like this in such an accessible way :) always excited to see whether I can learn something from this to apply to my day-to-day large-scale-data-related job. Thanks Gavin!
Consider using jumbo frames. The default MTU of 1500 bytes per frame was designed for 1 to 10 megabits/sec. You are running 10 to 25 gigabits/sec. Most NIC's support higher MTU settings. The key is that every NIC and switch in your entire network must be set to negotiate a higher MTU. Once setup you will see a massive increase in network speed.
Synology, this is money well spent. Best ad ever.
Yes definitely keep the speed ramp. Makes the video so much better
I've deployed one of these units in production for a university a few years ago (using 8TB drives). They are every bit as good as much more expensive NASs. Congratulations.
LTT themselves are in a very similar situation to you with file bloat, they're running like 8 RUclips channels. They also actually had a fire in their server rack.
Honestly, I'd say Linus is one of the few big youtubers that could go toe to toe with Gavin's storage usage. Like you said, he has multiple channels churning out videos on the reg. They shoot on high end, top of the industry video cameras at 4k resolution and they keep all RAW files of which I can assure they produce at least a couple hundred a day minimum.
@@jjpark98 They actually film at 8k at LTT, and have some stuff they did at 12k. Which makes the storage needs all that more necessary. They said they do that for zoom effects while still delivering true 4k.
@@ajs787 cant they all use like super good lossless compression algorithms?
@@mastershooter64 When they upload to RUclips, yeah, they use H.264. But having the raw files available allows them to do a lot more with things like color grading.
Wasn’t that vid a prank though?
Thank you for your past frustrations and struggles with data storage. Your channel and videos are amazing!
The only problem with raid 5/6/7 is the write hole when you lose power during a write operation but it sounds like you have that covered. With a good UPS and a redundant PSU it's unlikely you'll lose power in the middle of a write operation without safely shutting down (provided automatic shutdown is configured)
Also don't forget to scrub your data at least once a month! You can still get bit rot with any drive configuration!
I was thinking of upgrading from the RS1619xs+ to the HD6500. This video helped move that process along. Thanks
At this rate, Gav will soon need an entire server room to store all his footage lmao
In the beginning, you highlight the reason why you keep the entire 2 seconds, and honestly, I think a lot of people watch the videos partially because of that, the number of times I've only seen the "important" part and no context of slow-mo stuff on the internet and I was sitting there frustrated to not see more and you guys make it available for us.
As an ltt fan, I've seen Linus build its own petabyte server and it's a huge amount of storage and that's definitely something that content creators will need more over time. For Linus to post 17 videos a week (the last time I heard them talk about it it was 17), you need a lot of storage. And you highlighted perfectly why you need It too.
I think everybody hears 288Go per file and doesn't really realize that most smartphones couldn't even store a single file like that on it.
I feel like the importance of Smee’s contribution to this video cannot be overstated. A true master thriving-nay, REVELING!-in a highly specialized field.
I watched it to the end. Love all your content. Can’t look away from the little beard bit sticking out.
I went from no hard drive (5.25 floppies) to 20mb and I was such a happy camper! I had so much space, I really thought I'd never fill it. 🤣
Super awsome to see you guys got a large solution for storage. Also cool that synoligy pitched in and made this happen.
I see that your package passed the obligatory cat scan. 🐈👍
Big fan of the Synology products; glad they are supporting your channel!
I know you do your own thing and you guys are probably busy or whatnot but your 130TB Collab with Linus was absolutely hilarious. Maybe when you need a 100PB or 1 EXB server you can do another one!!!
Congrats on all your success and great content over the years! I always enjoy the content.
Cool getting a peek behind the curtain here, and I will say that the speedup portions are enjoyable and put the slo-mos into context very well.
Didn't think about storage, but yeah I can see why network storage would help! Synology has a good reputation too, the Twit TV folks are always recommending them.
Not that lossless general purpose compression or data deduplication should have any real significant space saving benefits for your use case necessarily, but I think it would still be interesting for just exploring if you can at a larger scale of the slow mo guys storage usage. Windows Server since version 2012r2 has had a way to get a file system to run deduplication at the level of the filesystem and if you prefer Mac OS or Linux, there is a way to get a similar if not better solution (ZFS). I guess my point is that at your scale of sheer quantity of storage in terabytes used, even a few percent of savings at this scale could start having significant cost savings and thus more room for more video or just giving you more money since you might have less need for a few of the drives. Also it's kinda industry standard at least in advertising storage to use base 10 notation to describe the capacity instead of binary. So yeah it seems like there's a small amount of tradition in rounding up storage space. Also parity and redundancy and on occasion formatting of the filesystem might loose a little bit of space.
ZFS......ZeFrankSpace? Hope Jerry stays out of it.....
🤣🤣🤣🤣🤣🥰
I remember years and years ago I bought a pendrive of 2Gb at EU$200 and I thought it was the best thing ever XD How fast time goes by, it's amazing.
On 5:12 I notice you still on 1500 MTU, if you up the number to 9014 MTU, you will have much better performance, but you also need to change the MTU on the network switch and on the computers that you wish to file transfer. This will help a lot since you dealing with very large files.
Spinning up you can estimate around 20W per drive and between 8-14 Watts idle and file accessing.
So 1200W alone for the disks when starting the rig
Curious if this will make the Blu-ray idea mentioned a while back easier? I don't own any Blu-ray disks, but would definitely make an exception for this 🙂
Loved the irony of the drive install timelapse from the FastMo Guys.
You've probably already thought about it since it's mentioned in the video, but consider having an off site backup. The biggest hazard to the server is something like a fire or a storm or something, which would also threaten your tapes if they're stored in the same location.
I'm awestruck by whatever cataloging system you must use to keep your data storage straight and reasonably available for your use. A petabyte I can imagine. Keeping it usefully, I can't imagine.
You could make a detailed tour of your editing workflow? From ingesting, over editing in Resolve to delivery. Would be awesome! What Programs do you use apart from Resolve?
He has made a video a while back about editing, I don't know if you have watched that.
@@kempo_95 This one "Slow Mo Sound is FAKE"? ruclips.net/video/EHD5PRrS4Ns/видео.html (no rickroll)
@@oso3557 yes that one
I really appreciate all the effort you put into your videos they always turn out really good
You don't have to worry about storage, for now 😅
In 5-10 years this 4 Petabyte storage will seem like the 250 TB storage you just replaced!
Regarding the power draw, long time ago I learned that you should calculate approx. 10 Watts per disk (and of course add the rest of the hardware as well), so this system is probably drawing around 750-800 Watts I guess?
I've got a Synology DS420j, it's getting on a bit now but it's not let me down yet. I'm only rocking about 32TB of storage in my 4 bay NAS, but it works for our needs!
Cut to next year when Gav teaches us about his new storage, because this one is full xD
Currently studying digital preservation and whilst I only understood about half of the words you said this is a fascinating solution study for large files and example of readability needs and variations! Ta.
The ability to be able to lift a petabyte is actually crazy. Well done synospstic.
do you realise how easy it would be to lift 1000 1TB microSD cards? at ~250mg each, lifting up 1PB worth of data as microSD cards would only weigh 250 grams. seems an awful lot lighter than the Synology setup. oh yeah, it's Synology by the way, not 'synospstic'. not sure how you arrived at that spelling there, Einstein.
I'm a data hoarder myself but I only got Synology DS920+ with 16TB of space, approaching 13TB. Great unit. I just got the new Synology router which is a beast and so far I'm very pleased with it
8:14 It would be interesting to see why you chose RAID-6 instead of SHR-2 (similar to RAID-6) or RAID-10
You might want to look into colo (Colocation) server racks. Basically it is a datacenter that sells rack space. I have used it for my homelab and the energy and ISP charges are typically reasonable and would take down the power/heat/noise in your house. Also room for growth with those expansion bays.
In fairness 1 Petabyte would equal out to roughly 909 Terabytes after formatting so you technically are not lying with the name of the video.
Never store your back up in the same building !!!! Love your staff
Did you hear the Vatican is making a 150,000 terabyte could server? They needed a place to keep their peta-files.
For those of you interested. the Synology HAS5300-16T drives cost about 600 gbp / 750 usd a piece racking up something like 36K gbp / 45K usd just for the drives alone.
Im sure cheaper as part of a bundle though. Free for the slo mo guys though!
0:01 GENERAL KENOBI
This should've got so many more likes
I have almost no idea what any of what Gavin talked about meant, but I could listen to Gavin talk about his tech hardware for hours. It's weirdly therapeutic or cool.
Thanks for the insight and thank you for all you do to keep us entertained. Merry Christmas.
God I love these behind the scenes videos, especially when it's about tech or specifically cameras/storage.
I started out with a WD myCloud with 2x2TB (Raid 1), after that got filled I put in 2x4TB (also Raid 1). After that was filled I mangled my system storage with 2TB without any backup. Now, just before I went to Iceland, where I shot a lot with my Ninja V in ProResRAW ("only" around 500-ish MB/s and still no native support in DaVinci), I bought a QNAP extension unit with 2 bays. It's basically an external hard drive enclosure with a hardware raid controller, which could be used to, as the name said, a QNAP NAS to extend it without needing to buy new storage or a bigger NAS. The extension unit got loaded with 2x18TB Seagate Drives (could've gone for 20TB, but the 18TB drives were a way better bang for the buck) also in Raid 1, and is now filled with 7TB just after the Iceland Trip. Hopefully it will last me a long time until I'll eventually go for a 12 bay unit with 18 or 20 TB drives. It's not the best solution, but a surprisingly cheap and easy option for people who need big storage on the go, to terminally safe it on a NAS in the long run.
Back in 2012 a friend of mine built a PC with 1TB of storage and that was astronomical at the time. PB is one thousand times bigger than that. So much space!!!
I love those more technical videos. Please consider expanding the second channel with more content like this
based on your level of storage consumption, a petabyte is going to be filled really fast. like i said, the lhc (large hadron collider) uses this type of storage system with a few other levels. first it hits ram, then a temp storage buffer (ssd array) then it offloads to the primary buffer (a giant array, think they went ssd not long ago, but it is several petabytes by itself) then they offload that to tapes in duplicate or more, then even that gets offloaded to more tapes. they also have another step of a robotic tape machine that will allow researchers from around the world to get a look at data without being connected to the primary storage server. the lhc does at least 15 petabytes a year though, you should be fine for a while.
This is what everyone’s main concern was when you got the LTO! I think it’s safe to say you’ll never lose your backups
Never ever say tha,t it attracts the attention of the data eating gremlins...
@@EdBruceWRX Absolutely true. There's always something!
What the slow mo guys need is actually a ceph cluster! This allows one to organically add (and remove) storage. Also, had synology had not required use of their proprietary drives, this server could have used 22TB drives and stored considerably more (almost a third more.)
First time ever you don’t have to worry about storage.
I don’t believe that. The speed at which you’ve filled everything else I have no doubt this new server will be bursting at the seams very soon.
Love these hardware/upgrade videos
It's always fun to get a look behind the scenes
I'm a music producer and thankfully audio is SIGNIFICANTLY smaller than video. My total storage I have for my music is about 10 TB across various places. Cloud, harddrive, computer etc. Basically as backups. But I probably use about half of that .. for now. Quite a lot of the space is taken up by sound libraries I've collected across the 5 years I've been making music. I keep everything I do and try my best to backup as much as possible. Even for me storage is one of the chores of my job, can't even begin to imagine what its like for you :)
You've done this to yourself, but it's worth it. It looks so, so, so good in the end.
Loving the video about storage and server stuff. Keep them coming!
I worked on machines 20 years ago that powered up their drives in groups to spread out the initial current draw as they ramp up.
For the amount of this video I understood it may as well have been in another language XD Happy for you, Gav.
you should see about using TureNAS one of these times for your next storage server. 100Gbe support natively (hardware dependent) and also able to have more control over what is going on.
"What a fool i am" the most relatable statement ever. Except you don't fit the bill Gav. Lol.
Gav has just jinxed himself and the next vid will be a all of our data is gone vid