Enterprise Hard Drive VS Consumer Hard Drive VS Enterprise SSD - Which Will Die First? (SSD Update)
HTML-код
- Опубликовано: 15 июл 2024
- Every second Tuesday, we will be releasing a tech tip video that will give users information on various topics relating to our Storinator storage servers.
This week, Doug and Brett are back to give an update on our "Which Drives Will Die First" testing. Guess what? The SSDs died first!
Revisiting the project - back in July 2023 we started looking at the lifetime of consumer hard drives versus enterprise hard drives versus enterprise SSDs, when writing to them constantly over a long period of time. Which drives will die first? That's what we aim to answer in this ongoing experiment. After a few months of testing, we finally have an update.
Chapters:
00:00 - Introduction
00:31 - Revisiting the Testing Method for Hard Drives vs SSDs
01:28 - Common Misconception of SSD Durability
02:17 - Timeline of Storage Media Endurance Testing
04:28 - What Data Measurement Software is Being Used? Grafana and Prometheus
05:30 - What Kind of SSD was used? Micron 5400 SATA
06:35 - SSD Report - 9500 Drive Writes Sequentially
07:14 - SSD Report - 4,750TB Used / Rated for 1,300TB
08:30 - SSD Report - What Were The Write Speeds of the Micron 5400
09:05 - How Long with the Enterprise and Consumer Hard Drives Last?
13:04 - SATA vs NVMe - Difference in Lifetime
15:35 - Outro
Visit our website: www.45drives.com/
Check out our GitHub: github.com/45drives
Read our Knowledgebase for technical articles: knowledgebase.45drives.com/
Check out our blog: www.45drives.com/blog
Single Server Buying Guide: www.45drives.com/products/net...
Ceph Clustering Buying Guide: www.45drives.com/solutions/di...
To enroll in our 2-day Clustered Storage Bootcamp: www.45drives.com/support/clus...
#45drives #storinator #stornado #storageserver #serverstorage #storagenas #nasstorage #networkattachedstorage #proxmox #virtualization #cephstorage #storageclustering #virtualmachines #cephcluster #storagecluster #proxmox #ansible #nasstorage #prometheus #samba #cephfs #allflash #ssdstorage #ssdserver #allflashserver #allflashstorage #zfs #ransomwareprotection #linux #linuxtraining #nasstorage #selfencryptingdrives #jbod #justabunchofdisks #justabunchofdrives #serverexpansion #esxi #migratevirtualmachines #ssd #ssdtrim #sed #dataencryption 
Наука
I really like the dynamic you guys have going and I truely mean this in a positive way - It's like "storage dad" and one of his "kids" - Brett is eager to get things going and Doug keeping the big overview. Generations see things in different ways and combined it leads to great tech. Big respect to a CEO who let's his employees break things to test the limits for the products and help innovate. Plus you taking your audience/customers along the way makes it feel like we're part of it.
You mention the spinning rust drives and how long it will take to get as many drive writes as the SSDs. But really, Seagate rates mechanical drives as a static data transfer rate per year (reads AND writes) no matter the size of the disk! This annual workload for enterprise drives is set to 550 TB per year for 5 years. For a 20 TB drive this means 137,5 full drive reads/writes.
Spinning disks aren't likely to fail quickly in a sequential workload, it's very easy on the drives, and as long as you keep them at the correct temperature, around 42c, they'll last far longer than you need them to. It's likely that you'll need more capacity before they actually fail. Realistic workloads tend to cause failures, because as far as the disk is concerned, that's random, and it experiences rapid changes in power usage, vibration, motor load, actuator load, etc.
The math for the real world is pretty simple, and you work it backwards from workload, which determines the performance needed, and then you can evaluate options as to how to meet that workload demand. Outside of DVR's, there are not pure sequential workloads in the real world because as soon as you start queuing writes from multiple clients, or queuing reads and writes, everything is fairly random. The degree of random is important, but, with enough users, you can assume fully random and go from there.
What you arrive at is, with a 5 year drive lifetime, flash storage is actually very cost competitive because you can use relatively few flash devices to get the same performance of many, many spinning disks, and use less power while doing so. Remember in the datacenter that less power is less heat as well, and heat is really just additional power usage to manage. Napkin math is you add about 30% to your power usage for cooling, so that really adds up quickly.
We can definitely see a difference in failure rate for drives we pull/put -1 PB/year on them and disks that mange around 50TB/year. But in general most of the drives work well inside the 5 year warranty
Another thing is GRC Spinrite 6.1 (out soon) is great at recovering from spinning drive issues. I have found it to be the best tool to stress test a new spinning drive. This is also would like to see more OS's use RAM DISK solutions for cache, and temporary storage solution.
I used GRC Spinrite many years ago to fix some very old 500GB hard drives or 1TB HD's. Excellent product. Found him on YT again a few months ago talking about he's coming out with the new 6.1 with a lot of features.
I would do 2tb in each segment that way it's slightly balanced.
Helpful co text and observations as always!
Hello,
Please Show a Chart with Detailed information of all the SSDs and HDDs you are Testing ,plus your lates test results!
Thanks.
@45drives I get what you were trying to convey in the video which was great. However, it was flawed as a test. What you should have done is to get a workload such as 4K uncompressed camera footage and have it write to all drives simultaneously. This will hopefully "fix" the rate of the data being ingested and written and then you could report on who dies first, the amount of data written when it failed and what percentage of the rated endurance it reached. This would align with your sequential testing and then you do an updated video. This would be more realistic to how the drives would be used for sequential workloads. And to make it read/ write test then justplay back the footage from each drive.
I agree but that would require a much lager scale of test (>10,000) drives to be significant and not many companies have the budget to perform such a test.
Have quite a few concerns with your testing and comments. Which I am glad you kind of start addressing around the 11 minute mark.
The Micron 5400s are more like entry level to mid range enterprise flash drives, yes these Micron's and the Seagate Exos they are both SATA but that's really not a great relational measure for comparison. Given the size of the drives, the Microns would likely be OS boot disks, not a use case for the workload you're putting on them.
The biggest issue with the testing is if you want to compare lifespan is to throttle the benchmarks to a speed that is equal on HDD/SSD or NVMe like 200MB/s (something they can all do) on drives of equal size (which happy was brought up). If you're just all out full speed writing to flash then it's skewed is it not? For example of course the 7450 NVMe would fail faster than the 5400 SSD it's doing what, 8 or 9x the workload? Same idea for the SSD > HDD, might be closer on sequential but the SSD will be able to accomplish more work in the same time frame that they are both still running and alive. Flash is also going to generate a ton of heat at 100% workload so can't imagine that's really that helpful.
None of this isn't to say that the spinning disks wouldn't last longer, I've had great longevity out of spinning disks, but hardware decisions should be workload based, especially with storage and I'm not seeing that being emphasized in the video so I find the content a bit deceiving. The benchmarks that matter are really the ones based against manufacturer warranty and spec in combination with use case are they not?
Micron makes "max" variants of these disks that allow for significantly more drive writes, and they're not really all that more expensive.
The real problem with the test is that sequential workloads don't exist in real life. You always have a mix of reads/writes/random reads/random writes, which is hell on a spinning disk, SSD's don't care much about such things because to them IOPS are IOPS, sure you can get into write amplification sometimes, but, modern controllers are very good at avoiding that.
I would like to see a 6 disk raidz10 of all these disks in a mixed workload. I'd bet that they'd still kill the flash storage first, but, the spinning disks likely wouldn't transfer more data before failure than the flash storage did. From there you could test again, but find where the predicted equivalency point is where, your performance and capacity are similar, and look at what's actually cheaper over a 5 year period. You'll find that, when you have actual work to do, flash is very competitive, even if you're feeding it fresh drives fairly often.
@@wojtek-33 That's not really an accurate statement. The 2.5" case acts as a heatsink to some degree. Unless the m.2 option had a heatsink the 2.5" will always run much cooler. I have fans blowing across m.2 NVMe disks with no impeded airflow and they get a lot toastier than the 2.5" ones in the same chassis under load.
@@wojtek-33 the point is they are both flash. Yes I see your point but higher temps are harder on flash.
@@hamburgler1 Nah, there's a sweet spot. NAND doesn't like being too cold or too hot. But usually the NAND isn't the part that gets hot anyway, the controller does.
I recently had an SSD (nvme gen3) fail on me. And it was just a few months old.
I never ever expected that to happen.
so I am now building a NAS based in Truenas scale, because I lost data.
I had been disapointed by Segate Skyhawk 2TB in NVR writing 30Mbps 24/7 and it died in 2 years. It started clicking the head, no data available. Surprisingly it is 324GB a day and 118TB a year. So it writen 236TB and died.
Statistics is one thing, individual experience is another. Any drive can fail, even the best of them. If you need guidance when buying them, though, the only proper way to do that is to refer to real-world statistics based on large samples such as published by backblaze. There's a good analysis of hdd afr data by sometechguy.
You should do a test of *un-powered* cold storage "data retention". ..how long before data loss due to magnetic fade on the hdd and how long before data loss due to electron leakage on the ssd
Seagate had a really bad time with drive quality around the 500Gb driver era, that's when I moved over to using higher end Western digitals.
I had the same experience with early big drives 250GB-1TB from Western Digital and DeathStar.
May also be cool to do single disk ZFS just to keep a real time watch on data integrity for SSDs, do a % write, read it, delete it, repeat.
Thanks for posting. What brand/model of Storage Controller was used in your system?
LSI 9305-16i - hope this helps!
I don't have a lot of knowledge about hard drives although I'm aware of the different kinds there are. This may be a hard question to answer or not, but has there been any tests done to see which hard drive type has the longest "read" life time? I'm going to assume it is an SSD, but I'm generally more curious how long Enterprise HD and Consumer HD will read for? Looking at filling up some higher capacity Enterprise HD for media library storage and playback using something like Plex or Jellyfin. So lets say you fill up a 18TB Enterprise drive with media, how long could this hypothetically play content for continuously?
thanks "Bob and Doug" for that awesome update! - I have been informally doing something similar for many years with hard drives going back to the late eighties/early nineties but also specifically with >2TB (post RAID) setups primarily Seagate drives (SATA and SAS) in ZFS pools - I have multiple ZFS pools of 3-8TB drives that get regular usage and periodic scrubs key findings: consumer last almost as long as enterprise 24/7 duty cycle but all seem to have issues/fail with Power_On_Hours >25,000 regardless of number of read/write cycles.
edit 1) I have a few ZFS pools of 1TB and 4TB SSD based drives that are getting the same regular usage and periodic scrubs but they are too young to provide any useful statistics.
edit 2) believe it or don't but power conditions have a huge impact on the lifespan of electronics thus I have had at least one layer of power conditioning (UPS) on all my computers since the late eighties.
Would also want to see a 50/50 or a 60/40 workload.
How long will the Micron 5400 Enterprise SSD last, if the data is written to it just once, and then later on used just for read purposes. Running TrueNAS Scale, all SSD, 24x7. Use case: Data Archival, monthly ZFS scrubs.
Would want to see something like a Samsung PM1643 SSD or a Koxia PM6/7
Do they still even sell 500GB spinners anymore for the enterprise? I cant remember the last time I saw one for sell under 1TB.
not new thats for sure, I was shopping for a 500GB drive to replace a failed 500GB Seagate I had and the only places were eBay and Amazon referb sellers.
Only really from OEM's nowadays, and those are all 10k+ RPM SAS drives, which are loud, power hungry, and slow, and really only offered as replacement parts for servers that are still in use.
Enterprise moved away from that model to flash storage because the performance/density/power consumption per rack unit is much better. Spinners for nearline storage is very much still a thing, mainly because disk capacity has gone up significantly so it's still worth it. It's really just an eventuality that everything will eventually be flash based though, we really just need a way to make high density, cheap, SLC flash and it's game-over for spinning disks, and we're closer to that reality than most people think in the enterprise world.
Would love to see a visual graphic
Thks, I've been wanting to see a real test like this one.
Like the other comments ; the old-guy & young-guy scheme definitely has synergy.
In 50years the young-guy will be the old-guy doing a similar video with another young-guy (or gal of-course ;).
I'm a little confused since you continuously mention "lifetime" is at zero (for the smart data on the SSDs). Do you mean that pre-programmed value by the manufacturer, Life Remaining? If so, it's meaningless and fits more to the manufactures warranty than actual issues and is far lower than actual expected life of the NAND. Next time you do this test, you need to be looking at unused spare blocks and any data integrity errors. NOT remaining life.
This is constant write to fail. In a typical environment, you're not going to kill the SSD, in fact, you'll probably get upwards of 5-10 years or longer of writing to it and eons longer reading from it. You have to realize this test was with a single drive. Imagine multiple drives in a raid array or even ZFS, the SSD's will probably outlive you.
I like the spirit, but this is pretty mickey mouse science. Sample size way too low, no hypothesis before hand, small selection of models/brands, no blind testing.
If you, as a storage business, want to know how to improve resilience of your systems, I think the better approach is to come up with a partnership with a small handful of universities, get some actual scientists on these questions, and throw some money at it. As it stands, once your experiment ends, my hypothesis is you're going to have a whole lot more questions than answers. Better to stand on the shoulders of giants.
I called it, SSDs just are not there yet. I have 20+ year old HDDs that are still going but in 5 years I've had to RMA 3 SSDs 1 Samsung, 1 WD, and 1 PNY.
That's like saying screws are just not there yet because hammers exist. SSDs are very much "there" and in use at a massive scale because of what they're capable of. If you're looking for cold archive storage, spinning disks are still king, sure, but if you're talking about something doing work in a rack, spinning disks are more or less exclusively a capacity/cost discussion, and you see the workloads they're well suited for shrinking substantially over time. It's an eventuality that flash will destroy the spinning disk market completely, and it's really only dependent on cost at this point.
Also, even in this limited example, those SSD's wrote ALOT of data before failure, and it's going to take a very long time for the spinning disks to catch up to that. The odds of failure increase significantly over time for a spinning disk, so, they may never actually get there before they, themselves, fail. This is, of course, an ideal test for spinning disks, sequential writes are very easy on a mechanical drive, they don't last nearly as long in mixed use, and because they're so much less performant in mixed use, they'll certainly fail before they reach the same data transfer numbers as the flash devices did.