Having worked with many versions of LTO drives/tapes over the last 20 years I know all too well that they can/do fail, don’t fall into the trap of thinking that the data/tapes won’t degrade over time or get chewed up by a bad tape drive leader. I highly recommend you create two tapes for each backup and keep a good backup of your backup library master data base, so you don’t have to rescan all the tapes to rebuilt it. It’s also important that you follow the correct storage guidelines (temperature, humidity etc)
I was thinking the same thing, ... I also worked with tapes and they are not always reliable... storage is an important part; I would keep the tapes in an airtight box keeping it away from heat.
I too had worked with tapes in the past with a large 50+ tape machines and was thinking the same thing. At least once a week a tape would fail requiring backups to be ran almost constantly. Plenty of multi tape systems out there which could allow easy striping as just a backup... Of the backup.
As an IT guy all I have to say is "MAKE SURE YOUR BACK UP YOUR CATALOG DATABASE" this is the database that tells you what is on each tape and test its restore every so often on a test machine. This is so important and no one thinks about backing this data up. Make sure the backup is automated and is taken off site and ensure you have multi revisions of this database. If you don't have this catalog all your tapes are useless or you will have to re catalog all you tapes. If you only have 4 not an issue but once you get to a 100... you could spend a month cataloging your tapes!
@@jirehla-ab1671 I'm sorry you want to reuse your SSD or HDD after it's repaired? If you have any issue with any media you 100% replace it and recover the data to a new medium if it's important
@@steve_main cause I already backed up the data to the cloud i am also planning to backup it to my other google drive, do you think it is also better to seek replacement through a warranty or just buy 2 new hdd / ssd instead?
@@jirehla-ab1671 If it's under warranty then have a replaced under warranty. why would you buy new hardware if it's under warranty? 100% replace it if you've had any issues with it. if not, any data you put on it expect it to be gone
@@steve_main cause for me I accidentally fall it to the floor and I don't know if there's already physical damage to the external hdd and western digital said that the warranty will be VOIDED if there is a physical damage and will NOT be replaced
If you really care to preserve everything then 3-2-1 backups are critical. Tape can be a key part of this, but it can't be the only thing. 3 - Total copies 2 - Different types of media 1 - Copy at a different physical location
plus 1 - Copy offline. If you delete a file unintentionally and the deletion is immediately mirrored to all your backups you are screwed. Better have one backup run from time to time so you can go back to your backup from last week or so, where the file has not been deleted yet.
for example: a file saved on your computer and mirrored to the cloud plus backup every x days to an external hard drive fullfills all of those criteria.
As has been said below already, make sure you have two copies. One on-site and one off-site. The fact that you haven't lost data is more of a miracle than anything else. Also tapes do suffer from bit rot as well (they are magnetic as well) Insure you store them in a stable environment, while they are great for archive storage the archive part only works if they are treated as per their specs.
Yes, now he has 45 tapes, time to get a second set of 45, a second drive and make a copy of each one as it is done. That will at least give single redundancy. Then a nice data safe that is rated for at least 3 hours fire resistance to store them in, preferably in a room above flood level, and with non flammable walls, and temperature control. That 30 year life assumes storage at constant 20C and 50%RH, which is only achievable in a massive data centre, and also assumes that you use no compression, and maximum error correction on the tapes.
While this is a big improvement vs random hard drives, as someone who archives data for a living I'd caution you against just putting everything on these tapes once and assuming they'll be safe for 30 years. That guarantee is more like a warranty than a specification, and I've had tapes fail much earlier than their rated lifespan. An immediate solution would be to use some form of parity or mirroring to ensure every file can't be lost with a single tape failing, but longer term I'd highly recommend backing up your archives using multiple forms of storage, in multiple locations, especially since this is your livelihood and if something like a flood hit wherever you store your tapes, you'd lose everything. A cheap option would be to make a JBOD server using your existing HDDs you just moved your data off of. JBOD is short for just a bunch of disks, and is exactly what it sounds like. It would also let you use a filesystem that automatically checks for and fixes bitrot, like ZFS, and mean you would have access to your entire archive without having to plug in a certain tape, or even multiple as LTO tapes can store a single file across multiple drives to ensure you're using all the storage it offers. This is why they have the barcodes as standard, as many commercial archives using LTO tapes will have a tape library, which are basically jukeboxes with tape instead of vinyl, and the software can automatically switch tapes to retrieve data spread across multiple drives.
Not to forget that that spec only applies if you store the tapes and drives dust-free and climate controlled (temperature and humidity). That's a lot to ask even for a small business.
To repeat what others already said... But since it's important and could avoid data loss. Do NOT expect 30 years out from those tapes. LTO is by no means safe from data/bit rot and for important data you should rotate every 5-10 years depending on storage conditions. The 30 years they market them with is during very specific conditions when it comes to temp and humidity, treat it as marketing buzz. Also, do NOT throw them around in luggage and avoid X-rays, strong magnetic fields and so on. And for the love of God, do NOT drop them. They are fragile and you just have to disassemble one to realize exactly how fragile they are. Have had more then one eaten by tape drives in the past (LTO4/5). You should have two copies of all critical data. If possible, use LTFS to avoid being locked into one vendors proprietary db's and system and keep backups of any eventual databases. But yes! When dealt with properly and when knowing about the quirks and limitations and gotchas LTO is very cost effective and nice. But like everything, LTO have its limitations and should not be treated as a indestructible bullet proof Jesus-medium that will save us all. 😎👍
I've worked with the LTO tape drives for a long time, although I worked primarily with Tape Library machines, that is, robots that you load dozens of tape into and has more than one drive. The backup software has a master database of all the files and tapes, and does backups, automatically changing the tape when the one in the drive becomes full. As other people have said you need *two* backups of everything, and store one copy in another site, preferably that is climate controlled. You should also periodically verify your backups. In the long term, you should also consider budgeting to change storage mediums every 5-10 years. Just like you are going from Hard Drives to LTO Tapes, in 5 - 10 years, you should be looking at the next great thing...maybe SSDs, or maybe holographic storage...
I've used LTO-4 for 12 years, and suddenly tape drives fail and HPE has forgotten how to repair the old drives, apparently throwing away all the repair manuals. That plus the nightmare of Indian support outsourcing plus only about 2 companies left actually making LTO products, despite originally introduced as a monopoly-free category as DLT had become a Quantum monopoly. So, an old suggestion that I sadly ignored, is to purchase a spare drive (hugely expensive), test it and store it with the off site backups. Once you need the spare, immediately transfer all backups to a new physical format that can be secured with the same measures. Oh, and in case you missed the small print: New LTO versions cannot read tapes from previous versions, except for one overhyped situation. Now I'd need to purchase an otherwise useless old LTO-5 drive to read the old tapes, while also investing in a new format to store new backups and copies of the old ones.
@@johndododoe1411 I totally agree with everything you said, especially the backup drive unit! But I would rotate it in every once in a while since it is bad for equipment to sit unused for long periods of time.
Magnetic tape is so cool, i'm still shocked after i found out that a vhs can hold gigabytes of digital data depending on the length. 18tb in such a small package is unbelievable data density
Theoretically yes. vhs is helical scan, the data is on a constant spiral, not linear. it doing so because vhs is optimised for analogue video, each scan line is physically present on the width of the tape. data however, is linear in nature, thats why we have data tapes like lto are always store in linear fashion. makes reading and writing much faster and searching easier.
@@gernhartreinholzen3992 Back? Tape never left at least in industrial and professional enviroments. Lto is a backup standard for decades now with 9 generations of tape and more are planned so its not obsolete by any means.
@@gernhartreinholzen3992 We never stopped using tape. A lot of supercomputers have physical tape storage libraries where an arm chooses a tape and inserts it into the drive for the computer
For someone who only genuinely enjoys the content and knows nothing about the logistics and workings of your results, its really cool to see this. As much as I wish you both were super close to have more time in this channel, life moves on and project priorities change i am glad you keep slowmoguys still going!
Just don't fall into the trap of routinely cleaning an LTO drive, clean it when the drive requests a cleaning by itself or in case performance takes a sudden dive.
Specifically, the Single Character Display (technical term for that amber 7-segment display) will display "C" when it wants a cleaning tape. Unless there is a serious problem, it will wait for you to eject the tape that's in there first before displaying it. As well, the drive has a brush built into it. Every once in a while, you will notice it will stall for a few minutes while it unloads the tape, brushes the head, reloads the tape, and then goes back to the point it left off. Another thing: If you are writing a lot of small files with LTFS, consider using the sync_type=unmount option, so that it's not spending a bunch of time shoeshining between the index and data partitions.
Of all the RUclipsrs, your channel has been the one that has been in the back of my mind constantly. as someone who's a tech enthusiast, knowing how much raw data you generate every video and trying to come up with a solution had to come in occasional mental exercise for myself. I knew tape would probably be your archival solution but I didn't know the technology had progressed to the point where the write speeds would actually be faster than most hard drives. Your probably still needing a Linus grade storage server for, effectively, a scratch disk during editing but now you can of-load one your done and can stop having to build servers. Sweet
I'm just going to echo what other people are saying and strongly consider duplicating your backups and storing them off site somewhere. Redundancy is just as important as backing up the data itself. It doesn't do you much good to back up all your data if it's all sitting in one location waiting to get destroyed in a fire or a tornado. You have a lot of very valuable data and you're now stepping into the world of enterprise level solutions. I would find an off-site backup location in another city, or state if you can mange it. Redundancy, redundancy, redundancy!!!
Love that you now have more cohesive way to store your vids, wondering what happened with the NAS LTT helped you out with. Love all the vids! Cannot stress enough that you should always have AT LEAST 2 copies of all your data (or anybody else reading this comment)
just wish the tape drives weren't so expensive. Or it was easier to rent them or something like that. since archived storage that doesnt need to be accessed you dont need the drive all the time.
The NAS would be great for more short term storage when he needs access to data but his local computer doesn't have enough storage/he might want to reference later. The tapes are great for the long term storage and archiving of his footage.
Gav has always been so good at making the tech side as interesting as the demonstration side. Cheers Gav! I am from Bristol! :D I live in Louisiana now, I miss the UK :(
That's bonkers! I didn't know such efficient storage even existed! What also surprised me was that 8mm film converter on the shelf there; I didn't expect you of all people to have one of those, lol!
I remember fixing a pretty big tape library machine a while ago (a tiny gear had cracked), it was super cool to see how it worked with a moving, mechanical arm on a pulley system to move the tapes around and stuff inside
I find it crazy that these machines that cost sometimes as much as a house use any plastic gears in them. I suppose they are expected to be decommissioned and superseded long before the plastic gears degrade, but that's clearly not always the case.
@@jamescollins6085 Hehe, the one I worked on wasn't quite a house in terms of price, but it sure saved the client probably a good few thousand euros. Bought a pack of gears from ebay, used a soldering iron to melt it onto the shaft as it was smooth.. and a few attempts later it worked just fine
Thanks for the tech update. It's always interesting seeing storage problems solved, that us mere mortals (in a data size sense of the word) will never face!
I've always found these types of videos interesting, but since I started properly working as a video editor and working with VR video a year ago these are absolutely amazing. That much storage is a dream
The barcodes were used for larger units which had mechanical arms which auto loaded the tapes into the reader/writer all enclosed into a cabinet which would live in the rack room. I remember those days...
Very helpful! We’re going to need a long term backup solution for our observatory data, and this is exactly what we’ve been looking for. Definitely going to back up with tape! It makes it so easy to back up off site
@@Bruh-zx2mc yup! We’re going to have our NAS store the main sets of data, and then tape on site and off site for long term storage. Our current NAS is 36 tb but I’m sure we will get it closer to 120 tb. We will save up for tape when we can afford it
0:46 Some fun facts: 720p at 82,000 frames per second is equivalent to 8K video at 2,278 frames per second, and an hour-long 720p@82,000 video that is actually 1 hour long and not a 90 second slo-mo stretched to 1 hour would be 324TB, and he would only be able to store 2.5 1hr long "720p" videos in his 810TB drive. Even if you somehow found the capacity to store a video that big, good luck 1. finding storage that is fast enough to show 82,000fps 1280x720 video (90GB/s is 12x faster than the fastest consumer-grade NVMe storage that is available today, and that $850,000 1 petabyte 100GB/s storage server that Linus showcased in his video a couple weeks ago (normally $1,000,000 but ignoring the $250,000 GPU cluster with 8x Nvidia A100 GPUs that is irrelevant to the storage portion of it that it includes) would only *theoretically* be able to handle it), 2. and *then*, good luck finding a CPU fast enough to decode (play back) such a video in real time. This is not a perfect comparison because of different codecs of varying quality, but knowing that my M1-powered 13" MacBook Pro can't quite play back 8K@30 VP9 video on RUclips without buffering, you'd need a CPU 76x faster than an M1 to be able to decode an 8K@2,278 VP9 video uploaded to RUclips (assuming RUclips even allows that video to be uploaded). Even if you bought 2 of the just announced $12,000 flagship AMD 96-core Zen 4 Epyc server processor, theoretically it would only take you 40% of the way there (30x faster than M1).
Just to chime in here: definitely keep a second storage backup if it's world-ending if you lose the raw. Tapes are pretty hardy but the chance of a file corrupting or at least degrading in that time span is pretty high (though better than a HDD). Would reccomend a second tape backup for any files you definitely need, and a third backup in a different storage medium (HDD, cloud, etc.) for anything absolutely critical. 2 tapes will do you for 30 years, but not if there's a fire where they're stored. Off-site backup in a different medium is airtight for anything critical and gets the best results :) But if it's just nice to have, one tape will keep the majority of your stuff for the 30 years bar any catastrophe
Former IT gal here. I can't stress hard enough how right all the other IT professionals and data archivists commenting on here are. You need *_at least_* two copies, one stored safely offsite.
Another benefit of tape is how natural is to have offsite and offline backups (the first protects you from disasters like fire or crime and the second one protects you from ransomware)
This was basically just an advert, but very interesting and nice to see Gavin is getting more help in the inevitable data hoarding that this job requires. Linus tried, but the Slow Mo Guys are on another level of storage needs.
@@akyhne They gifted him the drive and tapes in exchange for him creating this video in which their brand is mentioned to hundreds of thousands of people. That's literally what an advert is.
I have 1.04PB of storage which is backed up to large archive drives. I thought about using tapes, but they're likely to degrade at the same speed as a hard drive. The fundamental difference is that with a hard drive you can repair the fading magnetic fluxes. So every few years, just copy off the data, reformat the drive and then rewrite it. This is something that would be more difficult to do with tape. So as someone has pointed out... Don't think of tapes as a very long term backup solution. Not unless you're willing to maintain them too. Also bear in mind that unlike a hard drive, your tape machine is going to wear over time and you also run the risk of tape getting chewed up. These issues are less of a problem to a hard drive as it is a self-contained unit. Moreover, tapes will be much more sensitive to things like moisture and temperature than a HD because the platters in a HD are sealed. You do run the risk of motors failing in spinning discs, but this is more likely through use, rather than sitting idle and certainly less than the degrading of a LTO machine. So always have two drives and make sure they're refreshed at a reasonable interval.
You can repair the magnetic flux as you say, if the rest of your drive still works. Thats the thing right there, these tapes dont have any subsystems that will fail and entomb the data in a dead box. If the tape drive dies, you get a new one. As this is LTO 9 you could argue it may be cheaper to try and move a dead HDD's platters to another compatible drive, if you have someone who has a working drive to transplant them into. Plus the tapes are very immune to fading like old reel to reel audio tape can be because they are made of very different stuff. You are more likley to have problems with tape damage over anything to do with the magnetic fields.
@@dlarge6502 I've used tapes, lots of them and the magnetic fluxes of tape degrade at about the same rate as a HD. It's slightly higher for tapes because tape layers are wound upon each other. HDs die from use, not usually from sitting idle. They die either from motor failure or the controller dies which is why they have a MTBF indication. Each HD is self contained so the MTBF is spread over the number of drives for the archive, but a tape machine is used for all the media, so it's failure rate is significantly higher as is the chance of it chewing or having a r/w failure. The storage requirements in terms of moisture and temperature is also significantly higher for tape. Although admittedly, moving platters between HDs is extremely tricky, it can be done as a last resort. But get a chewed tape and it's game over. The only real benefit a tape gives you over a HD is from shock resistance, if it's a spinning disc and size. This can be eliminated with SSD storage though, obviously at a higher cost. Add to that the random access of a HD coupled with the higher transfer speed, especially from SSD, the HD always ends up on top in terms of long term storage.
@@thewelder3538 Well, at least your second comment made the argument sound more factual and not just a HDD fanboy making stuff up (your first comment gave that impression).
@@johndododoe1411 Agreed, it was a little wishy-washy. Although I do enterprise backups, it's hard to find the level between trying to sound like the ultimate authority on the subject and someone who's just a fan-boy.
Our biggest fear in our computer workshop and data storage room (at a university) was that one day, someone would come in with a degaussing wand and destroy all our backup tapes. We were so paranoid that we had an off-site secure storage area that only 3 people knew of.
That's a bit much don't you think? Wouldn't that storage room only be able to be accessed by verified people anyway? Or was it not locked down. Sorry, not trying to come off as rude just genuinely want to know why.
@@johndenver6769 The uni didn't quite have the budget or sense to provide a secure server room. So we took our own measures. Losing the data of thousands of off-campus distance ed students wouldn't have been a good thing. Not to mention study materials etc. We looked after data for the publishing section as well.
@@johndenver6769On a campus, IT and Facilities are often forced to share spaces. An off site colocation facility goes a long way to having dedicated space for IT to have sole access to.
Make sure you do regular catalog backups. You should also consider using a tape vault service. That way its offsite for DR redundancy, and they will keep it in a humidity/temp controlled storage to keep them in the best condition so they last for a long time
"All in the same format, and I only need one device." I would suggest buying three units, one of which is tested and repacked safely, and that you save the data on at least three tapes. And then make Dan keep that third copy of the tapes at his house, in case there's a fire. Also, keep the original hard drives as a last ditch solution. If you could copy the data to clean fresh disks, that'd probably be good too.
It is better to copy the data to tapes and then after checking that the data is fine format the drives and give it away / use it for something different. Hard drives are not the best long term storage solution and can be used. It is a huge opportunity cost to keep them around.
Part of the reason the BBC wipe over recordings was cost (magnetic tapes cost much much more than they do now), another key reason was contractually in some cases they were not allowed to reuse recordings, there was no reuse infrastructure or licensing agreements back then. Many historical recordings were destroyed just because they were not allowed to reuse it, even on film as opposed to on tape.
Until I retired, I was heavily involved in the design of truly large scale mission-critical IT systems. I was also the first in the company to exploit LTO (generation 1). It proved to be a good bet on the technology, and we eventually had massive robotic libraries which went through generations of LTO. However, one single copy is not enough. If you are going to use if for archive, then do keep multiple copies. That's a lot of data to lose in one go when a tape fails. Also, one copy should be kept off-premises. Also, be prepared to do a media conversion with new generation of LTO, and it's very good practice to periodically refresh anyway, to prove you can read a tape and then you can deal with a failed tape using one of your alternative copies. This also means a single drive isn't going to do the job either. Note that an LTO can write data a lot faster than it can be read off a single HDD. What you do not want to get into is a situation where you LTO drive is not streaming and you are in write, run out of data, rewind a bit and start writing again. It massively slows throughput and is very bad for the tape mechanism and tapes if it's extreme. LTO drives will throttle down write speed to some extent. It it's a problem, then stage the data to an SSD first before backing up. Of course writing large amounts of data to an SSD will affect it's life too, but at least it would be done once. In our case, we had massive enterprise array storage systems which striped data over RAID sets and could keep up (whilst still keeping systems going), but even then it could still be a problem to keep the LTO drives streaming efficiently. Also, if LTO is used for archival purposes, the tapes must be kept in a temperature and humidity controlled environment free of dust. Nothing too drastic, but something to be careful of. Extreme heat would be bad for them. There is quite a lot more to backup and archive on tape then there might appear to be, and it really requires experience. Of course, if you are keeping tapes and the original disk drives that does limit some of the risks, but this all needs a lot of care.
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Storing on harddrives, would be incredibly expensive.
@@HydroHUN I'd say hard drives experience mechanical failure ("click of death") long before bit rot. The HDD in my IBM Model 30 from 87 still works and that machine was rescued from a damp container at the recycling center
@@sylvester4207 It is semi manual, he inserts the tape drive and it just goes in the background. As far as I understand the file movement is automatic as long as the drive is not full
This, this and 100% this! Please make sure you have an off-site backup of your entire library; a theft, fire, flood can wipe out everything in minutes...
A few notes on LTO as someone who's used it extensively for video backup: 1) Generational compatibility -- while they tapes may be guaranteed to last upwards of 30 years, the same definitely isn't true of the drives. A new generation is released roughly every 3-4 years. So LTO-10 is due in about 2 years. And LTO-9 drives will likely still be available and in warranty for a couple of years after that. And LTO-10 will read LTO-9 tapes. But once LTO-11 is on the scene your days are numbered when it comes to being readily able to read LTO-9 cartridges. 2) Tape failure -- they are not perfect. Probably more reliable than 3.5" drives in cold storage, but not 100% reliable. Our backup plans always called for duplicate copies, stored in physically separate locations. On restores we did see failures. Also, while they're a great low-cost high-reliability storage, tapes are slow and annoying in many ways. I've typically preferred to keep media backed up on 3.5" drives also, as a first-instance archive that I can generally access quickly and with little effort.
DO NOT take the tapes on a plane, the scanners may wipe or corrupt them. When you are copying, you need to use file verification. Nothing worse than thinking you have a file, only to find it was corrupted right off the start. Take the original drives and store them at another location. Also, you may want to create some parity files (.par2) just in case it gets corrupted later or as a later verification of integrity.
Scanners won't do anything to modern tapes. Maybe 30 years ago the motors that drive the conveyor belt might have caused problems for low-density tapes. High density tapes like LTO need a huge amount of flux to cause issues. Scanners won't do it.
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes. Storing on harddrives, would be incredibly expensive. So no, an LTT server isn't necessarily the best choice.
The benefit of tape is the cost per GB at scale - the system Gavin has, with 810TB, comes to 0.024 per GB, physical HDDs would be about 0.035 per GB, and the cheapest AWS storage tier (Glacier Deep Archive) would be 0.046 per GB over 5 years (assuming no data is ever actually retrieved).
It's awesome to see how things evolve. It would be cool to see what your production path looks like from recording to ingestion to process to archive. Do you still have the 45 Drives server as well from Linus? Great video!
I had a job at a datacenter once where I was managing daily backups and they used these tapes in this bigass IBM tape ROBOT. It was basicly a couple of serverracks made into an enclosed tape station where it had like 8 tape stations that all writing simulatiously, an "arm" like a printer or something where it would move between the tape writers and this huge array/cluster of available drives, just sitting in slots all over the place. I think they're called "IBM Storage robots" or something like that. One of the daily tasks was to have it eject the tapes that needed to be put in a remote location, and swap them out with the old drives that needed to be erased and put back into the array.
I've somehow managed to cram my whole computer life into 1tb up till now. Finally had to order a 2nd TB last week though, because my old 1TB drive is completely full and I don't have anything disposable to free up space. Managing almost a petabyte of data sounds hard.
If I have learned anything about specialty hardware from my time working in IT, it's that you should go ahead and buy another one of the symply drives. No matter how good the company seems to be doing now, that unit could fail when you really need it and there will be none available
This is what Linus should have supplied during your collab, tape is the obvious solution to your problems. When I interned at IBM in a datacenter position, they had a machine with hundreds and hundreds of these tapes. Unit had a robot arm to read the barcodes in the library of tapes and load the appropriate tape on demand. Had to be petabytes of data on them. I remember the 4 tape heads on the high speed recorders were named John, Paul, Ringo and George lol. Someone there had a sense of humor. Then a guy would manually load them from the high speed output to the reading library with all the others, and add the barcodes to their spines.
@@jshklsn Linus is good for general education and entertainment. But his approach o industry standard equipment is usually quite lacklustre and backwards. His use of storage systems is quite painful to watch.
To be clear, I'm on the side of using things for what they are designed. Need to store massive data over quite a few hours? Tape. Need to store data real fast, multi-user? SAN or something local. I just think tape better fits this use case.
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes. Storing on harddrives, would be incredibly expensive. I'm pretty sure, they wouldn't use tapes, if they sucked!
Didn't you get a 130TB storage server with LinusTechTips? Or was that never used? Edit: Thanks guys for the update. I didn't know it filled up that fast !
Tapes can definitely have bit rot. And these tapes are guaranteed to last 30 years, just like LED lights are guaranteed to last ten years -- I.e., not. Tapes are a fine backup medium on top of other mechanisms, but I'd never use tape as my only backup. Not even in a 3-2-1 pattern. Disclaimer: I was a technical reviewer of the O'Reilly book "Backup & Recovery" by my friend and former co-worker, Curtis Preston.
If you're curious about how much it'd cost to store 810TB in the cloud while keeping access to it, you're looking at a monthly fee of about $3,000. As a backup solution (accessible once or twice per year) it's a tenth of that.
Every time I look into tape I am shocked at the prices of drives. Been sticking to HDD and a HDD dock and basically using the HDDs like tapes. I need to completely revamp my backups though so I can have more automated versioning. Right now it's kinda manual and is just me choosing what drive to put in to update to latest backup.
I'm shocked not only by the prices of the drives, but of the tapes as well. In many cases, you aren't saving much over the cost of a comparable hard drive, and you have the added inconvenience of packing and unpacking all the data every time you want to access any part of it from the tape. I also don't understand the obsession with keeping source material that so many RUclips creators seem to have when the bitrate that it will be compressed to when streaming is hundreds of times lower than source anyway. I'm sure he could re-encode the footage using the new AV1 encoding format and save mountains of space with no decrease in visual fidelity. Perhaps like with photography, it is necessary to keep the files in their raw form to aid in post-processing. Either way, it is his footage to treat in any way he sees fit.
@@jamescollins6085 The tapes are brand new, this is literally the first time I've seen one outside of a product sheet, thus they will be pricey for a few years. I'm totally shocked that anyone would pay $1000 for a phone too.
I agree it's tragic that so many shows and pieces of footage are lost forever just because some corporation was too lazy to store a copy. I love to watch old entertainment from back in the pre-internet ages. I often wonder if the old lesser known cartoons I used to watch still exist in a vault somewhere, or if they only exist on random people's VHS tapes.
As many people have noted, tape drives are a "quantity over quality" upgrade. You sure are getting a lot of storage, but that storage may or may not be less reliable than a mechanical drive or SSD. Mechanical drives at least have the advantage of warning you when they start to die, unlike tape and SSDs.
No, we are not going "back". Tape has always been essential for large capacity storage. 20 years ago, I worked for a company, that made the physical negatives for color printing. These were cut by lasers. Each project took around a few gigabytes, which was an enormous amount of data, back then, when you had hundreds of costumer projects. CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes. Storing on harddrives, would be incredibly expensive.
We are not going "backwards". Tape has always been essential for large capacity storage. 20 years ago, I worked for a company, that made the physical negatives for color printing. These were cut by lasers. Each project took around a few gigabytes, which was an enormous amount of data, back then, when you had hundreds of costumer projects. CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes. Storing on harddrives, would be incredibly expensive.
I remember being interested in digital tape storage as a kid, back when I was backing up my Windows 98 files on 250mb ZIP Discs. I had no idea they were still used, and had grown to that capacity. From what I (barely) remember back then, the largest tapes you could buy in stores were like 40/80gb. This probably would have been around 2001 or 02.
0:05 It technically is a paid video. Paid with a free product that allows content and gives them free advertising, so check that box. If you could send the tape drive over here when you're done that would be great ;o)
It will be interesting, As a former main frame computer engineer form the days of huge tape files, in general a tape reel would be at least a mile long and often twice that. We used to run archive tapes from end to end periodically in fast forward to make sure they wouldn't crosstalk between tape layers with time. Tape can and does deteriorate over time, but good luck. The bigger the tape the more you loose when it goes bad! Add to that access is serial (because it's a tape,) getting to a file at the other end can take an appreciable time where as solid state or HD is much faster to access.
So, we're back to where we were forty years ago. Tape on cartridges/cassettes for data storage. Gives the micro-tapes from the original Star Trek new credibility.
No, we are not going "back". Tape has always been essential for large capacity storage. 20 years ago, I worked for a company, that made the physical negatives for color printing. These were cut by lasers. Each project took around a few gigabytes, which was an enormous amount of data, back then, when you had hundreds of costumer projects. CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes. Storing on harddrives, would be incredibly expensive.
I just saw a short video on RUclips, from CERN. They explained, that they are connected to 170 data centers around the globe, to process the data from the LHC. They also have their own computer on site, with over a million cores.
I hadn't heard about this tapes, but it sounds a lot like a miracle product. I think it would be best to use them as one more backup besides the HDDs and SSDs.
a tape backup is usually a 3rd level backup, not a primary, normally you would have a live source, a hot backup, normally on instant access HDD (NAS/iSCSI), and then cold storage, aka tape. then if you are wanting to achieve it, you would normally have two independent copies, plus a tape backup.
![Six Hundred Strike [EPIC: The Musical] Full Animatic](http://i.ytimg.com/vi/zov6NXIAuow/mqdefault.jpg)
Having worked with many versions of LTO drives/tapes over the last 20 years I know all too well that they can/do fail, don’t fall into the trap of thinking that the data/tapes won’t degrade over time or get chewed up by a bad tape drive leader. I highly recommend you create two tapes for each backup and keep a good backup of your backup library master data base, so you don’t have to rescan all the tapes to rebuilt it. It’s also important that you follow the correct storage guidelines (temperature, humidity etc)
I was thinking the same thing, ... I also worked with tapes and they are not always reliable... storage is an important part; I would keep the tapes in an airtight box keeping it away from heat.
Exactly. At least 2 copies of each file is necessary.
I keep 9 copies of each tape incase the first 8 fail on me. Thinking of going to 10 soon.
I too had worked with tapes in the past with a large 50+ tape machines and was thinking the same thing. At least once a week a tape would fail requiring backups to be ran almost constantly.
Plenty of multi tape systems out there which could allow easy striping as just a backup... Of the backup.
Never forget about 3-2-1, even with tapes.
As an IT guy all I have to say is "MAKE SURE YOUR BACK UP YOUR CATALOG DATABASE" this is the database that tells you what is on each tape and test its restore every so often on a test machine. This is so important and no one thinks about backing this data up. Make sure the backup is automated and is taken off site and ensure you have multi revisions of this database. If you don't have this catalog all your tapes are useless or you will have to re catalog all you tapes. If you only have 4 not an issue but once you get to a 100... you could spend a month cataloging your tapes!
If my ssd or hdd failes, is it better to replace it than repairing without considering data recovery?
@@jirehla-ab1671 I'm sorry you want to reuse your SSD or HDD after it's repaired? If you have any issue with any media you 100% replace it and recover the data to a new medium if it's important
@@steve_main cause I already backed up the data to the cloud i am also planning to backup it to my other google drive, do you think it is also better to seek replacement through a warranty or just buy 2 new hdd / ssd instead?
@@jirehla-ab1671 If it's under warranty then have a replaced under warranty. why would you buy new hardware if it's under warranty? 100% replace it if you've had any issues with it. if not, any data you put on it expect it to be gone
@@steve_main cause for me I accidentally fall it to the floor and I don't know if there's already physical damage to the external hdd and western digital said that the warranty will be VOIDED if there is a physical damage and will NOT be replaced
If you really care to preserve everything then 3-2-1 backups are critical. Tape can be a key part of this, but it can't be the only thing.
3 - Total copies
2 - Different types of media
1 - Copy at a different physical location
plus 1 - Copy offline. If you delete a file unintentionally and the deletion is immediately mirrored to all your backups you are screwed. Better have one backup run from time to time so you can go back to your backup from last week or so, where the file has not been deleted yet.
for example: a file saved on your computer and mirrored to the cloud plus backup every x days to an external hard drive fullfills all of those criteria.
oh and TEST YOUR BACKUPS!!!!
@@cobaltno51 Mutahar (Some Ordinary Gamers) recently said his system makes a backup every other day so at most he loses one day of data.
@@cobaltno51 A plan you don't test is no plan at all.
Big props to those tapes for ensuring that Dan stays the most photographed person on earth for the next years.
Sounds like a Guinness record
He probably is!
Thats just one afternoon for an instagram influencer.
If you count each frame as a photo, he’s almost certainly the most photographed person by a couple of orders of magnitude 😁
As has been said below already, make sure you have two copies. One on-site and one off-site. The fact that you haven't lost data is more of a miracle than anything else.
Also tapes do suffer from bit rot as well (they are magnetic as well)
Insure you store them in a stable environment, while they are great for archive storage the archive part only works if they are treated as per their specs.
Yes, now he has 45 tapes, time to get a second set of 45, a second drive and make a copy of each one as it is done. That will at least give single redundancy. Then a nice data safe that is rated for at least 3 hours fire resistance to store them in, preferably in a room above flood level, and with non flammable walls, and temperature control. That 30 year life assumes storage at constant 20C and 50%RH, which is only achievable in a massive data centre, and also assumes that you use no compression, and maximum error correction on the tapes.
Knowing how tape worked, I always thought they would demagnetize. I guess the only surefire cold storage is archival m-disc
(And also Sony's proprietary optical disc cartridge)
The rule now is three backups, two of which can be on-site, but at least one of which must be off-site.
That's the 3-2-1 rule.
@@shubinternetwhat about concerns about earthquakes? Tsunami?
While this is a big improvement vs random hard drives, as someone who archives data for a living I'd caution you against just putting everything on these tapes once and assuming they'll be safe for 30 years. That guarantee is more like a warranty than a specification, and I've had tapes fail much earlier than their rated lifespan. An immediate solution would be to use some form of parity or mirroring to ensure every file can't be lost with a single tape failing, but longer term I'd highly recommend backing up your archives using multiple forms of storage, in multiple locations, especially since this is your livelihood and if something like a flood hit wherever you store your tapes, you'd lose everything. A cheap option would be to make a JBOD server using your existing HDDs you just moved your data off of. JBOD is short for just a bunch of disks, and is exactly what it sounds like. It would also let you use a filesystem that automatically checks for and fixes bitrot, like ZFS, and mean you would have access to your entire archive without having to plug in a certain tape, or even multiple as LTO tapes can store a single file across multiple drives to ensure you're using all the storage it offers. This is why they have the barcodes as standard, as many commercial archives using LTO tapes will have a tape library, which are basically jukeboxes with tape instead of vinyl, and the software can automatically switch tapes to retrieve data spread across multiple drives.
Interesting, which companies/brands would you recommend?
Tape guaranteed for 30 years, and you get a replacement if it fails. But not the data that was on the tape.
It sounds like he plans on ditching HDD storage and simply copying SSD to tape. Safety of the data will not have improved much.
This Might be one of the Most informative comment ever posted
Not to forget that that spec only applies if you store the tapes and drives dust-free and climate controlled (temperature and humidity). That's a lot to ask even for a small business.
To repeat what others already said... But since it's important and could avoid data loss. Do NOT expect 30 years out from those tapes. LTO is by no means safe from data/bit rot and for important data you should rotate every 5-10 years depending on storage conditions. The 30 years they market them with is during very specific conditions when it comes to temp and humidity, treat it as marketing buzz. Also, do NOT throw them around in luggage and avoid X-rays, strong magnetic fields and so on. And for the love of God, do NOT drop them. They are fragile and you just have to disassemble one to realize exactly how fragile they are. Have had more then one eaten by tape drives in the past (LTO4/5). You should have two copies of all critical data. If possible, use LTFS to avoid being locked into one vendors proprietary db's and system and keep backups of any eventual databases. But yes! When dealt with properly and when knowing about the quirks and limitations and gotchas LTO is very cost effective and nice. But like everything, LTO have its limitations and should not be treated as a indestructible bullet proof Jesus-medium that will save us all. 😎👍
To be fair the raw footage is not critical at all, it's just "nice to have", even the edited versions are, because the last backup is on YT itself.
Any competitors making better ones? (than LTO)
I've worked with the LTO tape drives for a long time, although I worked primarily with Tape Library machines, that is, robots that you load dozens of tape into and has more than one drive. The backup software has a master database of all the files and tapes, and does backups, automatically changing the tape when the one in the drive becomes full. As other people have said you need *two* backups of everything, and store one copy in another site, preferably that is climate controlled. You should also periodically verify your backups. In the long term, you should also consider budgeting to change storage mediums every 5-10 years. Just like you are going from Hard Drives to LTO Tapes, in 5 - 10 years, you should be looking at the next great thing...maybe SSDs, or maybe holographic storage...
I've used LTO-4 for 12 years, and suddenly tape drives fail and HPE has forgotten how to repair the old drives, apparently throwing away all the repair manuals. That plus the nightmare of Indian support outsourcing plus only about 2 companies left actually making LTO products, despite originally introduced as a monopoly-free category as DLT had become a Quantum monopoly.
So, an old suggestion that I sadly ignored, is to purchase a spare drive (hugely expensive), test it and store it with the off site backups. Once you need the spare, immediately transfer all backups to a new physical format that can be secured with the same measures.
Oh, and in case you missed the small print: New LTO versions cannot read tapes from previous versions, except for one overhyped situation. Now I'd need to purchase an otherwise useless old LTO-5 drive to read the old tapes, while also investing in a new format to store new backups and copies of the old ones.
@@johndododoe1411 I totally agree with everything you said, especially the backup drive unit! But I would rotate it in every once in a while since it is bad for equipment to sit unused for long periods of time.
Magnetic tape is so cool, i'm still shocked after i found out that a vhs can hold gigabytes of digital data depending on the length. 18tb in such a small package is unbelievable data density
Theoretically yes. vhs is helical scan, the data is on a constant spiral, not linear. it doing so because vhs is optimised for analogue video, each scan line is physically present on the width of the tape. data however, is linear in nature, thats why we have data tapes like lto are always store in linear fashion. makes reading and writing much faster and searching easier.
I'd never thought we'd go back to tape someday.
@@gernhartreinholzen3992 Back? Tape never left at least in industrial and professional enviroments. Lto is a backup standard for decades now with 9 generations of tape and more are planned so its not obsolete by any means.
Makes me think everyone's been distracted by "Big Pharma" when the _real_ villain of our age has been "Big Hard Drive" all this time!
@@gernhartreinholzen3992 We never stopped using tape. A lot of supercomputers have physical tape storage libraries where an arm chooses a tape and inserts it into the drive for the computer
Very interesting. Thanks for taking the time to go through the process of migrating large amounts of data from hdd's to LTO tape.
How did u send this 1 hour ago I cl I cked on it when it was posted
@@AbsentNinjawaffles4 your notification was 1 hour late?
I have RUclips notify me of videos just posted, but a few days after I watched them!
@@dlarge6502 same there were were ekss than a hundred comments all were seconds and minutes just thought the glitch was funny that I saw it
For someone who only genuinely enjoys the content and knows nothing about the logistics and workings of your results, its really cool to see this. As much as I wish you both were super close to have more time in this channel, life moves on and project priorities change i am glad you keep slowmoguys still going!
Don't forget a cleaning tape or two for the drive! They need to be cleaned every so often (40+ hours of writing, for example) for best operation.
As I'm using older equipment I tend to avoid cleaning tapes as all they do is sandpaper the tape path. I like to open the driven and manually clean.
Just don't fall into the trap of routinely cleaning an LTO drive, clean it when the drive requests a cleaning by itself or in case performance takes a sudden dive.
Specifically, the Single Character Display (technical term for that amber 7-segment display) will display "C" when it wants a cleaning tape. Unless there is a serious problem, it will wait for you to eject the tape that's in there first before displaying it.
As well, the drive has a brush built into it. Every once in a while, you will notice it will stall for a few minutes while it unloads the tape, brushes the head, reloads the tape, and then goes back to the point it left off.
Another thing: If you are writing a lot of small files with LTFS, consider using the sync_type=unmount option, so that it's not spending a bunch of time shoeshining between the index and data partitions.
Love the breakdown, it’s obviously not just a hobby for you guys! Keep bringing us that sweet delicious slo-mo greatness ♥️
Oh and give Dan some more explosives
Of all the RUclipsrs, your channel has been the one that has been in the back of my mind constantly. as someone who's a tech enthusiast, knowing how much raw data you generate every video and trying to come up with a solution had to come in occasional mental exercise for myself. I knew tape would probably be your archival solution but I didn't know the technology had progressed to the point where the write speeds would actually be faster than most hard drives. Your probably still needing a Linus grade storage server for, effectively, a scratch disk during editing but now you can of-load one your done and can stop having to build servers. Sweet
There is something about tape that I'm happy to see back.
I'm just going to echo what other people are saying and strongly consider duplicating your backups and storing them off site somewhere. Redundancy is just as important as backing up the data itself. It doesn't do you much good to back up all your data if it's all sitting in one location waiting to get destroyed in a fire or a tornado. You have a lot of very valuable data and you're now stepping into the world of enterprise level solutions. I would find an off-site backup location in another city, or state if you can mange it. Redundancy, redundancy, redundancy!!!
Flashbacks to my old job. So much time spent with LTO tapes.
IKR, tape is so cool
Love that you now have more cohesive way to store your vids, wondering what happened with the NAS LTT helped you out with. Love all the vids!
Cannot stress enough that you should always have AT LEAST 2 copies of all your data (or anybody else reading this comment)
just wish the tape drives weren't so expensive. Or it was easier to rent them or something like that. since archived storage that doesnt need to be accessed you dont need the drive all the time.
Preferably 3 copies tbh, for stuff that's irreplaceable
The NAS would be great for more short term storage when he needs access to data but his local computer doesn't have enough storage/he might want to reference later. The tapes are great for the long term storage and archiving of his footage.
@@ge2719 er if thats expensive to you, what are the cheaper options? 18tb for 170$ is way cheaper than hard drives.
@@hailgod1 He's not talking about the tapes, which are cheap. He's talking about the drives, the reader/writer device, which is expensive.
I’m a data scientist in the banking industry and I train models with tape archives all the time. Super reliable form of cold storage.
Gav has always been so good at making the tech side as interesting as the demonstration side.
Cheers Gav!
I am from Bristol! :D
I live in Louisiana now, I miss the UK :(
I note you didn't say you miss Briz tho lol
If I'd grown up in the UK I, too, would miss the NHS if I ever moved to the US.
>I miss the UK :(
LMAO why would you
@@xamanto he downgraded to the US
@@upsilondiesbackwards7360 *upgraded
That's bonkers! I didn't know such efficient storage even existed!
What also surprised me was that 8mm film converter on the shelf there; I didn't expect you of all people to have one of those, lol!
new LTO drives cost between 5K-12K😭 and they are limited to own and 2 prior generations of the tapes that it can read
me thinking tape storage was dead, and you come along and show me this thing, Heck
Love a slo mo guys 2nd channel video! So glad this channel is still active
Super nice video to see genuine appreciation for each other. Every company in this video is well intended and represented.
I remember fixing a pretty big tape library machine a while ago (a tiny gear had cracked), it was super cool to see how it worked with a moving, mechanical arm on a pulley system to move the tapes around and stuff inside
I find it crazy that these machines that cost sometimes as much as a house use any plastic gears in them. I suppose they are expected to be decommissioned and superseded long before the plastic gears degrade, but that's clearly not always the case.
@@jamescollins6085 Hehe, the one I worked on wasn't quite a house in terms of price, but it sure saved the client probably a good few thousand euros.
Bought a pack of gears from ebay, used a soldering iron to melt it onto the shaft as it was smooth.. and a few attempts later it worked just fine
Nice to see tape is still alive and well
Thanks for the tech update. It's always interesting seeing storage problems solved, that us mere mortals (in a data size sense of the word) will never face!
LOVE these videos on the technical background behind Slow Mo Guys! Thanks for sharing!
I hope this is the next step in your Blu-ray journey! 🤞
I've always found these types of videos interesting, but since I started properly working as a video editor and working with VR video a year ago these are absolutely amazing. That much storage is a dream
The barcodes were used for larger units which had mechanical arms which auto loaded the tapes into the reader/writer all enclosed into a cabinet which would live in the rack room. I remember those days...
Very helpful! We’re going to need a long term backup solution for our observatory data, and this is exactly what we’ve been looking for. Definitely going to back up with tape! It makes it so easy to back up off site
I'll just forward what others have been saying - keep three total copies of everything, two different types of media and one copy offlocation.
@@Bruh-zx2mc yup! We’re going to have our NAS store the main sets of data, and then tape on site and off site for long term storage. Our current NAS is 36 tb but I’m sure we will get it closer to 120 tb. We will save up for tape when we can afford it
Interesting how I was just researching LTO tape this morning, and then this video came up! Nice job Gav haha
"Phantom Operator" is the best job title I've ever heard of 🤘🏿
Hey Cedders! Make sure you get one of those fireproof media storage cabinets. Would be a shame if who ever stole your flamethrower returned.
the channel is so old were starting to see the salt and pepper in gavs hair, cant wait for more wise and wisdom from this channel.
The real slo mo storage.
Wrong.
@@_GhostMiner Why? Linear storage is much slower than HDDs.
I was going to give this post a like, but it was at a nice number so I didn't :)
@@Scoots1994 77?
@@hitsmike2046 because he said the speed is ≈300MB/s
0:46
Some fun facts: 720p at 82,000 frames per second is equivalent to 8K video at 2,278 frames per second, and an hour-long 720p@82,000 video that is actually 1 hour long and not a 90 second slo-mo stretched to 1 hour would be 324TB, and he would only be able to store 2.5 1hr long "720p" videos in his 810TB drive.
Even if you somehow found the capacity to store a video that big, good luck
1. finding storage that is fast enough to show 82,000fps 1280x720 video (90GB/s is 12x faster than the fastest consumer-grade NVMe storage that is available today, and that $850,000 1 petabyte 100GB/s storage server that Linus showcased in his video a couple weeks ago (normally $1,000,000 but ignoring the $250,000 GPU cluster with 8x Nvidia A100 GPUs that is irrelevant to the storage portion of it that it includes) would only *theoretically* be able to handle it),
2. and *then*, good luck finding a CPU fast enough to decode (play back) such a video in real time.
This is not a perfect comparison because of different codecs of varying quality, but knowing that my M1-powered 13" MacBook Pro can't quite play back 8K@30 VP9 video on RUclips without buffering, you'd need a CPU 76x faster than an M1 to be able to decode an 8K@2,278 VP9 video uploaded to RUclips (assuming RUclips even allows that video to be uploaded). Even if you bought 2 of the just announced $12,000 flagship AMD 96-core Zen 4 Epyc server processor, theoretically it would only take you 40% of the way there (30x faster than M1).
I love learning about this side of the whole process. Things we as viewers may take for granted
Just to chime in here: definitely keep a second storage backup if it's world-ending if you lose the raw. Tapes are pretty hardy but the chance of a file corrupting or at least degrading in that time span is pretty high (though better than a HDD). Would reccomend a second tape backup for any files you definitely need, and a third backup in a different storage medium (HDD, cloud, etc.) for anything absolutely critical.
2 tapes will do you for 30 years, but not if there's a fire where they're stored. Off-site backup in a different medium is airtight for anything critical and gets the best results :)
But if it's just nice to have, one tape will keep the majority of your stuff for the 30 years bar any catastrophe
Former IT gal here. I can't stress hard enough how right all the other IT professionals and data archivists commenting on here are. You need *_at least_* two copies, one stored safely offsite.
Another benefit of tape is how natural is to have offsite and offline backups (the first protects you from disasters like fire or crime and the second one protects you from ransomware)
This was basically just an advert, but very interesting and nice to see Gavin is getting more help in the inevitable data hoarding that this job requires. Linus tried, but the Slow Mo Guys are on another level of storage needs.
Just an ad?
Why is it, that every time someone mentions a brand, it's definitely an ad in someone's eyes?!
@@akyhne They gifted him the drive and tapes in exchange for him creating this video in which their brand is mentioned to hundreds of thousands of people.
That's literally what an advert is.
I have 1.04PB of storage which is backed up to large archive drives. I thought about using tapes, but they're likely to degrade at the same speed as a hard drive. The fundamental difference is that with a hard drive you can repair the fading magnetic fluxes. So every few years, just copy off the data, reformat the drive and then rewrite it. This is something that would be more difficult to do with tape. So as someone has pointed out... Don't think of tapes as a very long term backup solution. Not unless you're willing to maintain them too. Also bear in mind that unlike a hard drive, your tape machine is going to wear over time and you also run the risk of tape getting chewed up. These issues are less of a problem to a hard drive as it is a self-contained unit. Moreover, tapes will be much more sensitive to things like moisture and temperature than a HD because the platters in a HD are sealed. You do run the risk of motors failing in spinning discs, but this is more likely through use, rather than sitting idle and certainly less than the degrading of a LTO machine. So always have two drives and make sure they're refreshed at a reasonable interval.
You can repair the magnetic flux as you say, if the rest of your drive still works. Thats the thing right there, these tapes dont have any subsystems that will fail and entomb the data in a dead box.
If the tape drive dies, you get a new one. As this is LTO 9 you could argue it may be cheaper to try and move a dead HDD's platters to another compatible drive, if you have someone who has a working drive to transplant them into.
Plus the tapes are very immune to fading like old reel to reel audio tape can be because they are made of very different stuff. You are more likley to have problems with tape damage over anything to do with the magnetic fields.
@@dlarge6502 I've used tapes, lots of them and the magnetic fluxes of tape degrade at about the same rate as a HD. It's slightly higher for tapes because tape layers are wound upon each other. HDs die from use, not usually from sitting idle. They die either from motor failure or the controller dies which is why they have a MTBF indication. Each HD is self contained so the MTBF is spread over the number of drives for the archive, but a tape machine is used for all the media, so it's failure rate is significantly higher as is the chance of it chewing or having a r/w failure. The storage requirements in terms of moisture and temperature is also significantly higher for tape. Although admittedly, moving platters between HDs is extremely tricky, it can be done as a last resort. But get a chewed tape and it's game over.
The only real benefit a tape gives you over a HD is from shock resistance, if it's a spinning disc and size. This can be eliminated with SSD storage though, obviously at a higher cost.
Add to that the random access of a HD coupled with the higher transfer speed, especially from SSD, the HD always ends up on top in terms of long term storage.
@@thewelder3538 Well, at least your second comment made the argument sound more factual and not just a HDD fanboy making stuff up (your first comment gave that impression).
@@johndododoe1411 Agreed, it was a little wishy-washy. Although I do enterprise backups, it's hard to find the level between trying to sound like the ultimate authority on the subject and someone who's just a fan-boy.
Always enjoy learning about data management and your process Gav.
Our biggest fear in our computer workshop and data storage room (at a university) was that one day, someone would come in with a degaussing wand and destroy all our backup tapes. We were so paranoid that we had an off-site secure storage area that only 3 people knew of.
That's a bit much don't you think? Wouldn't that storage room only be able to be accessed by verified people anyway? Or was it not locked down.
Sorry, not trying to come off as rude just genuinely want to know why.
@@johndenver6769 The uni didn't quite have the budget or sense to provide a secure server room. So we took our own measures. Losing the data of thousands of off-campus distance ed students wouldn't have been a good thing. Not to mention study materials etc. We looked after data for the publishing section as well.
@@johndenver6769On a campus, IT and Facilities are often forced to share spaces. An off site colocation facility goes a long way to having dedicated space for IT to have sole access to.
Make sure you do regular catalog backups. You should also consider using a tape vault service. That way its offsite for DR redundancy, and they will keep it in a humidity/temp controlled storage to keep them in the best condition so they last for a long time
"All in the same format, and I only need one device."
I would suggest buying three units, one of which is tested and repacked safely, and that you save the data on at least three tapes. And then make Dan keep that third copy of the tapes at his house, in case there's a fire. Also, keep the original hard drives as a last ditch solution. If you could copy the data to clean fresh disks, that'd probably be good too.
It is better to copy the data to tapes and then after checking that the data is fine format the drives and give it away / use it for something different. Hard drives are not the best long term storage solution and can be used. It is a huge opportunity cost to keep them around.
Seeing all that data without any backups makes my palms sweaty. This has been long overdue!
Part of the reason the BBC wipe over recordings was cost (magnetic tapes cost much much more than they do now), another key reason was contractually in some cases they were not allowed to reuse recordings, there was no reuse infrastructure or licensing agreements back then. Many historical recordings were destroyed just because they were not allowed to reuse it, even on film as opposed to on tape.
Yesh, all European broadcasters were doing the same thing.
Until I retired, I was heavily involved in the design of truly large scale mission-critical IT systems. I was also the first in the company to exploit LTO (generation 1). It proved to be a good bet on the technology, and we eventually had massive robotic libraries which went through generations of LTO.
However, one single copy is not enough. If you are going to use if for archive, then do keep multiple copies. That's a lot of data to lose in one go when a tape fails. Also, one copy should be kept off-premises. Also, be prepared to do a media conversion with new generation of LTO, and it's very good practice to periodically refresh anyway, to prove you can read a tape and then you can deal with a failed tape using one of your alternative copies. This also means a single drive isn't going to do the job either.
Note that an LTO can write data a lot faster than it can be read off a single HDD. What you do not want to get into is a situation where you LTO drive is not streaming and you are in write, run out of data, rewind a bit and start writing again. It massively slows throughput and is very bad for the tape mechanism and tapes if it's extreme. LTO drives will throttle down write speed to some extent.
It it's a problem, then stage the data to an SSD first before backing up. Of course writing large amounts of data to an SSD will affect it's life too, but at least it would be done once. In our case, we had massive enterprise array storage systems which striped data over RAID sets and could keep up (whilst still keeping systems going), but even then it could still be a problem to keep the LTO drives streaming efficiently.
Also, if LTO is used for archival purposes, the tapes must be kept in a temperature and humidity controlled environment free of dust. Nothing too drastic, but something to be careful of. Extreme heat would be bad for them.
There is quite a lot more to backup and archive on tape then there might appear to be, and it really requires experience. Of course, if you are keeping tapes and the original disk drives that does limit some of the risks, but this all needs a lot of care.
I didn’t even realize tape format was still being used for storage, that’s awesome
Yep, this is the latest generation LTO9 and its the first time I've seen it out in the wild
yep reliable and secure :)
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out.
They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive.
Storing on harddrives, would be incredibly expensive.
The missed opportunity not naming this channel “The Slow Mo(re) Guys”
7:03 Gav: Correction. Tape is also subject to bitrot. Just like a HDD.
No storage type is flawless, eventually the data will start to rot. Though HDDs will experience bit rot way sooner than tape drives
@@HydroHUN but if you have a HDD server it can be done automatically, he has to do it manually
@@HydroHUN I'd say hard drives experience mechanical failure ("click of death") long before bit rot.
The HDD in my IBM Model 30 from 87 still works and that machine was rescued from a damp container at the recycling center
@@DecibelAlex damp containers won't inherently hurt the HDD. It's not a physical rot
@@sylvester4207 It is semi manual, he inserts the tape drive and it just goes in the background. As far as I understand the file movement is automatic as long as the drive is not full
That write speed is phenomenal!
You might want to make a second run of tapes for of-site backup. Just in case the house burns down or something like that.
This, this and 100% this! Please make sure you have an off-site backup of your entire library; a theft, fire, flood can wipe out everything in minutes...
$170 per tape times 45 equal $7650. Real pricey but I guess worth it depending on how much he values his files.
A few notes on LTO as someone who's used it extensively for video backup:
1) Generational compatibility -- while they tapes may be guaranteed to last upwards of 30 years, the same definitely isn't true of the drives. A new generation is released roughly every 3-4 years. So LTO-10 is due in about 2 years. And LTO-9 drives will likely still be available and in warranty for a couple of years after that. And LTO-10 will read LTO-9 tapes. But once LTO-11 is on the scene your days are numbered when it comes to being readily able to read LTO-9 cartridges.
2) Tape failure -- they are not perfect. Probably more reliable than 3.5" drives in cold storage, but not 100% reliable. Our backup plans always called for duplicate copies, stored in physically separate locations. On restores we did see failures.
Also, while they're a great low-cost high-reliability storage, tapes are slow and annoying in many ways. I've typically preferred to keep media backed up on 3.5" drives also, as a first-instance archive that I can generally access quickly and with little effort.
DO NOT take the tapes on a plane, the scanners may wipe or corrupt them.
When you are copying, you need to use file verification. Nothing worse than thinking you have a file, only to find it was corrupted right off the start. Take the original drives and store them at another location.
Also, you may want to create some parity files (.par2) just in case it gets corrupted later or as a later verification of integrity.
Scanners won't do anything to modern tapes. Maybe 30 years ago the motors that drive the conveyor belt might have caused problems for low-density tapes. High density tapes like LTO need a huge amount of flux to cause issues. Scanners won't do it.
Maybe if you’re traveling with them all the time and are exposed to the X-rays multiple times…otherwise, no this isn’t entirely correct
Glad someone else is finally aware of these things outside industry
Linus tech tips be like: "but can you make a 2 Petabyte server?"
1500 exabyte sever time :D
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out.
They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes.
Storing on harddrives, would be incredibly expensive.
So no, an LTT server isn't necessarily the best choice.
Glad they provided you the things you need. $5200 for just the drive is expensive!
The benefit of tape is the cost per GB at scale - the system Gavin has, with 810TB, comes to 0.024 per GB, physical HDDs would be about 0.035 per GB, and the cheapest AWS storage tier (Glacier Deep Archive) would be 0.046 per GB over 5 years (assuming no data is ever actually retrieved).
I just imagine Gavin being like 80 years old just watching back all the footage
You're the Linus tech tip we should have gotten. Because you actually understand WTF your talking about. Well explained. 💯
It's awesome to see how things evolve. It would be cool to see what your production path looks like from recording to ingestion to process to archive. Do you still have the 45 Drives server as well from Linus? Great video!
I had a job at a datacenter once where I was managing daily backups and they used these tapes in this bigass IBM tape ROBOT. It was basicly a couple of serverracks made into an enclosed tape station where it had like 8 tape stations that all writing simulatiously, an "arm" like a printer or something where it would move between the tape writers and this huge array/cluster of available drives, just sitting in slots all over the place. I think they're called "IBM Storage robots" or something like that.
One of the daily tasks was to have it eject the tapes that needed to be put in a remote location, and swap them out with the old drives that needed to be erased and put back into the array.
I've somehow managed to cram my whole computer life into 1tb up till now. Finally had to order a 2nd TB last week though, because my old 1TB drive is completely full and I don't have anything disposable to free up space.
Managing almost a petabyte of data sounds hard.
If I have learned anything about specialty hardware from my time working in IT, it's that you should go ahead and buy another one of the symply drives. No matter how good the company seems to be doing now, that unit could fail when you really need it and there will be none available
Truth, keep another unit tucked away that isn't open, maybe even 2 or 3 of them if you have the money.
This is what Linus should have supplied during your collab, tape is the obvious solution to your problems. When I interned at IBM in a datacenter position, they had a machine with hundreds and hundreds of these tapes. Unit had a robot arm to read the barcodes in the library of tapes and load the appropriate tape on demand. Had to be petabytes of data on them. I remember the 4 tape heads on the high speed recorders were named John, Paul, Ringo and George lol. Someone there had a sense of humor. Then a guy would manually load them from the high speed output to the reading library with all the others, and add the barcodes to their spines.
Please dont tell me they have Linus "advising" them...?
@@jshklsn Linus is good for general education and entertainment. But his approach o industry standard equipment is usually quite lacklustre and backwards. His use of storage systems is quite painful to watch.
To be clear, I'm on the side of using things for what they are designed. Need to store massive data over quite a few hours? Tape. Need to store data real fast, multi-user? SAN or something local. I just think tape better fits this use case.
LTO backup is still one of the best ways to archive footage. It's what we teach in all our courses!
I've worked with tapes in the past and never trusted them to be reliable. I just hope this setup works out for Gav.
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out.
They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes.
Storing on harddrives, would be incredibly expensive.
I'm pretty sure, they wouldn't use tapes, if they sucked!
Love this kind of background information, thank you for making a video about it!
Yea LTO what an interesting medium. Funny how the drives cost so much.
Now you can just store them all in a single fireproof foam case, like a Pelican case! This is a really awesome bts vid.
Didn't you get a 130TB storage server with LinusTechTips? Or was that never used?
Edit: Thanks guys for the update. I didn't know it filled up that fast !
He used it. 130tb Storage didn't last long however
he filled that up before the video was put out.. you see he did a little update at the end showing it full.
You use such a server more for live projects. So dump tapes to the server, or hold a catalog of lower quality videos for searching.
I remember him tweeting that it got filled up in about 5 months
Thanks guys for the update. I didn't know it filled up that fast !
Tapes can definitely have bit rot. And these tapes are guaranteed to last 30 years, just like LED lights are guaranteed to last ten years -- I.e., not.
Tapes are a fine backup medium on top of other mechanisms, but I'd never use tape as my only backup. Not even in a 3-2-1 pattern.
Disclaimer: I was a technical reviewer of the O'Reilly book "Backup & Recovery" by my friend and former co-worker, Curtis Preston.
What is the oldest recording you own, gav? Like, the oldest thing pre-slo mo guys?
You got my heart racing when you hovered your mouse over the "format" button at 4:15, lol.
If you're curious about how much it'd cost to store 810TB in the cloud while keeping access to it, you're looking at a monthly fee of about $3,000. As a backup solution (accessible once or twice per year) it's a tenth of that.
S3 Glacier Deep Archive cost (monthly): 822.89 USD
That is a supremely impressive tape drive. I certainly hope all your hard-drives and physical copies are insured in some way.
Every time I look into tape I am shocked at the prices of drives. Been sticking to HDD and a HDD dock and basically using the HDDs like tapes. I need to completely revamp my backups though so I can have more automated versioning. Right now it's kinda manual and is just me choosing what drive to put in to update to latest backup.
I'm shocked not only by the prices of the drives, but of the tapes as well. In many cases, you aren't saving much over the cost of a comparable hard drive, and you have the added inconvenience of packing and unpacking all the data every time you want to access any part of it from the tape. I also don't understand the obsession with keeping source material that so many RUclips creators seem to have when the bitrate that it will be compressed to when streaming is hundreds of times lower than source anyway. I'm sure he could re-encode the footage using the new AV1 encoding format and save mountains of space with no decrease in visual fidelity. Perhaps like with photography, it is necessary to keep the files in their raw form to aid in post-processing. Either way, it is his footage to treat in any way he sees fit.
Its because the users of tape have IT budgets and we throw money about to companies who know we have the money to play with.
@@jamescollins6085 The tapes are brand new, this is literally the first time I've seen one outside of a product sheet, thus they will be pricey for a few years. I'm totally shocked that anyone would pay $1000 for a phone too.
@@dlarge6502 Agreed. After a year, most phones will cost roughly half of what they were initially sold for at launch anyway.
I agree it's tragic that so many shows and pieces of footage are lost forever just because some corporation was too lazy to store a copy. I love to watch old entertainment from back in the pre-internet ages.
I often wonder if the old lesser known cartoons I used to watch still exist in a vault somewhere, or if they only exist on random people's VHS tapes.
Whatever happened to Linus’s backup system?
I remember gavin tweeting 5 months later that it got full
Interesting and informative. I'm amazed tape is used till this day in this way.
Can these go through x-ray at the airport?
As many people have noted, tape drives are a "quantity over quality" upgrade. You sure are getting a lot of storage, but that storage may or may not be less reliable than a mechanical drive or SSD. Mechanical drives at least have the advantage of warning you when they start to die, unlike tape and SSDs.
I love it. We've brought back the flip phone, we've brought back disposable cameras, and now we've brought back the granddaddy of them all... the VHS!
No, we are not going "back". Tape has always been essential for large capacity storage.
20 years ago, I worked for a company, that made the physical negatives for color printing. These were cut by lasers.
Each project took around a few gigabytes, which was an enormous amount of data, back then, when you had hundreds of costumer projects.
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out.
They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes.
Storing on harddrives, would be incredibly expensive.
tape never left dawg
Sweet, no more stressing out over stuff like rotational velocidensity!
It's so cool to me that going "backwards" is such a great leap forward
We are not going "backwards". Tape has always been essential for large capacity storage.
20 years ago, I worked for a company, that made the physical negatives for color printing. These were cut by lasers.
Each project took around a few gigabytes, which was an enormous amount of data, back then, when you had hundreds of costumer projects.
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out.
They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes.
Storing on harddrives, would be incredibly expensive.
I remember being interested in digital tape storage as a kid, back when I was backing up my Windows 98 files on 250mb ZIP Discs. I had no idea they were still used, and had grown to that capacity.
From what I (barely) remember back then, the largest tapes you could buy in stores were like 40/80gb. This probably would have been around 2001 or 02.
As someone who is not knowledgeable at all on products like these and those in the space, i believe this is an awesome and useful product
3-2-1 Backup.
3 copies, 2 media, 1 copy offsite.
I dare say we're not seeing the entire back-up strategy here and LTO is just one media
We us LT06 in our job. Very safe way to store data for a long time.
0:05 It technically is a paid video. Paid with a free product that allows content and gives them free advertising, so check that box. If you could send the tape drive over here when you're done that would be great ;o)
It will be interesting, As a former main frame computer engineer form the days of huge tape files, in general a tape reel would be at least a mile long and often twice that. We used to run archive tapes from end to end periodically in fast forward to make sure they wouldn't crosstalk between tape layers with time. Tape can and does deteriorate over time, but good luck.
The bigger the tape the more you loose when it goes bad!
Add to that access is serial (because it's a tape,) getting to a file at the other end can take an appreciable time where as solid state or HD is much faster to access.
So, we're back to where we were forty years ago.
Tape on cartridges/cassettes for data storage.
Gives the micro-tapes from the original Star Trek new credibility.
No, we are not going "back". Tape has always been essential for large capacity storage.
20 years ago, I worked for a company, that made the physical negatives for color printing. These were cut by lasers.
Each project took around a few gigabytes, which was an enormous amount of data, back then, when you had hundreds of costumer projects.
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out.
They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes.
Storing on harddrives, would be incredibly expensive.
@@akyhne I did not know that.
Are those still magnet tapes? How are they able to have that high storage capacity?
I just saw a short video on RUclips, from CERN.
They explained, that they are connected to 170 data centers around the globe, to process the data from the LHC.
They also have their own computer on site, with over a million cores.
Crazy we are back to tapes.
We never stopped using tape. It backs up the majority of the data in the world. Your data is on it right now, somewhere.
I hadn't heard about this tapes, but it sounds a lot like a miracle product. I think it would be best to use them as one more backup besides the HDDs and SSDs.
a tape backup is usually a 3rd level backup, not a primary, normally you would have a live source, a hot backup, normally on instant access HDD (NAS/iSCSI), and then cold storage, aka tape. then if you are wanting to achieve it, you would normally have two independent copies, plus a tape backup.
what happened to linus' server
Probably was only used for new footage, here he's mostly talking about old footage, pre ltt server
Remember to store tapes upright and in the proper humidity/temperature.