If also need a step by step guide to building a NAS with a Raspberry Pi, then I have a video about that as well: ruclips.net/video/8fIeiZx4voU/видео.html
Tittle is miss leading. Title implies a tutorial of building a rpi nas. This gives speed test results of different HD configurations. Useful information but not why I started this video. Rename this so it describes Video appropriately.
Same here... With a title "Build a Raspberry Pi etc, etc" in a YT channel called... "Gary Explains"... I was kind of expecting a step by step tutorial of some sort... but ok... I already found some option and some older info elsewhere...
@@le8a9p easy. Create a RAID 5 storage of 3 or more disks. Make a LVM2 on top of that RAID device. Install Samba or NFS or iSCSI to distribute disks to clients. From here you can also install other services for your LAN. You are welcome. :-) And the most important. Boot your RPi from USB instead of µSD. USB memory sticks are more stable and easy to manage then µSD. You could even use a USB disk for your OS, instead of those flimsy µSD.
Anders Jackson Software RAID 5 using LVM or maybe ZFSoL? Kind of tempted to try something like this it would certainly use much less power than my DL380 freenas box lol.
Pleeze, the nerd solution to NAS, which I have nicknamed the NASty is to get an old desktop computer, whatever you have laying around. Then figure out how many hard drives you can connect inside the case. Then salvage every hard drive you can find that is "decent enough" and shove them all inside. Load Linux and do some network wizardry and you have yourself a NASty for free. You can get a little fancy with this if you want, or you could go full-on nasty and just have each drive shared on the network.
that's what I'm thinking on doing, buying a pack of old/failing disks if I can find for cheap (no seensitive data, just messing with VMs and nerdy stuff)
@@FeelingShred You can find some cheap TB drives on ebay. It may be hit or miss though. I got one for about $20 several months ago for my desktop. It was high-hours but it was still healthy. Another option is to roll around the nicer neighborhoods in your area on trash day and look for any computers being thrown out. Could also put an ad on Facebook marketplace seeking "old computers" and maybe offer like $5 for a complete computer. Might be surprised at who will respond. I miss the days of finding free computers everywhere.
@@NotSoCrazyNinja Thanks :) Around the place I live people don't toss computers like that, a few days ago I saw some people still charging something like 80 Dollars for a functioning (allegedly functioning) 486, outrageous... Some cheap p4/dual core CPUs though... But I won't drag you down with how boring life feels around here xDDD
Hi Gary, Great video. I just wondered what would happen if you would take 2 powered usb hubs attached to 2 different ports on the pi with a raid 10 setup. Would this increase the R/W speed or is the bottleneck dependent on the raspi internals? Thanks for your answer :)
Great video! I also wonder how would 3 or 4 disks configured with RAID 5 perform. By the way, what if instead of connecting 4 disks to a hub that would use one USB 3 port, I would instead use two hubs with two disks each, using up both USB 3 ports of the Pi? Would it make a difference in terms of speed?
I tried various combinations including 2 x hubs with 2x HDD in each. I didn't see any performance difference. My guess is that the 2x USB ports are just a built-in mini-hub connected to the same controller. I can't find any information on the Pi website or on Broadcom's website to confirm or deny this. All I have is, "much faster input/output, due to the incorporation of a PCIe link that connects the USB 2 and USB 3 ports, and a natively attached Ethernet controller."
I was thinking the same thing with respect to using a hub from a single port. After looking around I believe that the usb 3 slots share a controller so it probably wouldn't help... however, if you set it up and then administer everything remotely I think you could use one usb 3 hub for 2 drives and both usb 2 ports and perhaps that would let you bump up the net smb raid write speeds. You would probably be throttled thanks to using USB 2 but in the aggregate gain a few Megs/sec (maybe up to 40 or 45)... but that requires the usb 2 ports having a separate controller.
I have a Frankenserver that does something similar to this but uses LVM to tie the drives together. In this way I can create some large areas for storage that aren’t mirrored but then I can also create areas that are double or triple mirrored across the drives. It’s seemingly a good way to use up all those disparate old drives that collect about the place. No idea what the performance is like though as I basically just use it for warm archiving.
The problem with JBOD (Just a bunch of disks, like RAID 0 or LVM with more then one disk) is that you lose most/all your data if you lose one disk. And the risk of loosing all data will actually double with each disk you add. Better to have a RAID 5 under LVM2 (In principle data on all disks except one that do have check sums so you can recreate one, but only one, lost disk).
LVM mirroring duplicates data across disks in the volume group so that data has some level or redundancy I believe. The data that goes on the spanned logical volumes is disposable things like internet downloads that take up a lot of room and don’t matter if they are lost.
@@Alan.livingston as long as you know and remember that data on a single disk is not actually safe, it is ok JBOD. Where one disk failure will make you loose all data there. And yes, LVM mirroring are a RAID light, but not a proper RAID. But again, your mileage might vary. :-)
If you're going to do this, be aware that your USB-SATA device maybe doesn't support SMART or have incomplete support. And when the time comes to rebuild, you'll probably want to have a PC with some SATA ports available.
What's the best way to power 3 SSD's to use as a NAS on the Pi 4? Will the USB hub be necessary? I've been told SSD's don't need need as much power as conventional HDD's.
Very helpful indeed, but for me, I didn't think it would be worth the effort to create a RAID 1. Can you please make a video about this "night job"? Do you use anything else or just Task Scheduler?
I'm looking to create a RAID 1 setup using two M.2 SSDs on my Raspberry Pi 5 (4GB). I already have the Geekworm X1004 adapter. What software would you recommend for setting up the RAID 1 and creating a private cloud using Nextcloud? Could you provide a step-by-step guide or even a video tutorial on how to do this, including how to access the Nextcloud with RAID protection? My goal is to move away from Google Drive and Google Photos. Additionally, I'm interested in having multiple Raspberry Pi 5 devices in different locations for data redundancy. For example, one at my house, one at my parents', and another at my grandmother's. If one fails, I'll still have my data available on another device. Is it possible to create a RAID 1 configuration across multiple Raspberry Pi 5 devices? Any help would be greatly appreciated!
Raid 1 works great for file serving. I use an externally powered, Sabrient dual SATA III usb 3 system. I have two 7200 RPM drives in there now just because I had them on hand. I can connect from Windows, iPhone, iPad, Raspbian or whatever. I tried putting own cloud over top of that but just found it largely unnecessary for basic file serving. I may try it again with K3 and Docker. I boot to the other USB 3 port on 120 SSD from western digital. Pi4 is powering the USB boot drive. But if I place a microSD in the pi it automatically use it instead.
There's one thing I didn't get while watching... Did you use the two RPi USB ports with the two USB drives to avoid USB 3 bandwidth cap? The same goes for four drives, and RAID 10... Are the two "RAID 0" drives connected to one USB port and the other two over another one? I think that may change performance results quite a bit according to USB ports distribution
The way you described making the manual raid with four drives I thought would be called 0+1 and not 1+0. 1+0 is making sets (usually pairs) of mirrored drives, and then striping all of the mirrored sets. 0+1 is making striped sets and then mirroring the striped sets. 0+1 is a lot less safe than 1+0. Or am I getting things backwards?
Ah, yes, I am always confusing them myself. So, I went back and check my notes. What I actually tested was RAID 1+0, but I mistakenly described RAID 0+1 in the video! 😕
Thanks for clearing that up. It is good to know what the Pi 4 can do as a NAS. I'm thinking of getting a NAS and I would definitely be more inclined to make a DIY solution like this rather than using something more conventional.
If you tried NFS you would probably got much better performance when reading and writing over network. Reading and writing from /dev/zero and /dev/null wold be even faster.. ;-) What about RAID 5 and RAID 6? Or even BTRFS or ZFS, which have RAID and LVM2 in them. RAID and Backup, as you mentioned in the end is clearly not the same. Backup is saving data from accidental removing/overwriting. RAID "only" save you from loosing a disk.
@@AndersJackson It takes less than 3 minutes to set them both up and test it. NFS is still the best but it isn't that much better, anymore. 10 years ago it was twice as fast, then it was 30% then it was 15%.. etc
I wonder ... concerning your RAID 1+0 test ... if you used both of the Pi's onboard USB 3.0 ports and dedicated each port to a RAID-0 set then created the RAID-1 from those, if the performance would change significantly or not ... if each USB port has a dedicated USB controller, then the performance should dramatically improve ... in theory ... it should go even faster if each USB port had a RAID 1 so that your final setup was RAID 1+1 ... although that would cut total storage capacity in half.
Regarding 4 drives Vs 2 drives - was the throughput limitation due connecting 4 drives over a single USB 3 port via the powered USB hub? If you used two USB hubs with two drives per hub and each hub connected to its own USB 3 port on the Raspberry pi, would you expect to see any improvement in overall throughput for 4 drives? Or is the throughput limitation observed actually a limitation of the raspberry pi's USB 3 controller and how it connects to the SOC's IO?
I used two hubs, one on each port. I also did try other combinations and I didn't see any performance difference. My guess is that the 2x USB ports are just a built-in mini-hub connected to the same controller. I can't find any information on the Pi website or on Broadcom's website to confirm or deny this. All I have is, "much faster input/output, due to the incorporation of a PCIe link that connects the USB 2 and USB 3 ports, and a natively attached Ethernet controller."
@@GaryExplains Thank you for an excellent video, and clarification of the test process. Regardless, it is still an excellent way to make a low-cost RAID 0 NAS.
Nice presentaion but it stands more in the performance perspective ..and while thinking of something what about if you have set a healthy raid (any level) with 2, 3 or 4 hdds plugged in the usb hub and the cleaning lady accidentally unplug them all while pi is off and replug them in any (wrong) order in the hub? Is actually the pi able to assort each disk as the correct raid member or will set the raid and the raid volume down.. and will ask for rebuilding array or re-create it?
Thanks for the great videos. I didn't think it was a good idea to run RAID over USB but have I got that wrong? If it's ok then I'm thinking of doing this but with a dumb 3.5" 4 bay USB 3 enclosure and connecting that to the Pi. The Pi will just see 4 drives and then I'll use OMV to RAID them. Is it ok to have the RAID hardware connected to the RAID software via USB in this manner?
I feel you should have spent a little time explaining the speed constraint that will arise if using drives formatted other than in Linux-friendly ext2/3/4 etc. The PI now has great i/o but the CPU will still constrain throughput if your USB drive(s) are formatted NTFS as many may be when first bought. My own pi NAS showed almost as much CPU load from accessing an NTFS drive as from running SMB so that's almost halving potential performance.
@@GaryExplains Well, there are valid reasons to share an NTFS drive. I like the Pi solution for a NAS. Should I have a Pi die, a new one can be had quite quickly. Backup the SD card often! Anyway, using Filezilla to read my NTFS drive on the PI, I can pump about 80% of the gigabit ethernet connection. There are gaps between files. Writing on the other hand is much slower. I get about 25-30 mB per second. Mount.ntfs does eat a bunch of CPU. Advantage of the NTFS drive: No problem using it directly on my Win7 machine, which I used to populate at USB 3 speeds (which still took days to fill the 16T drive from backups). Fast I/O at this point isn't a goal.
Hypothetically, 1)if we were to set a raid mirror, then somehow a USB cable got disconnected while we still are operating with the system, after realizing the cable is loose, we connect it back. I assume the raid will start to resync the disconnected drive. If so, what performance hit it will encounter, assuming we already have lots of data in the system. 2)What would be the performance hit of a Raid 5 in a similar resync? 3) Considering the high possibility of getting one of the USB cable/PS disconnects, how about using a dock station that handles 4 SATA with one single USB 3? Will that allow me to run Raid 5 or Raid 1 with the 4 drives?
I was thinking the same thing. RAID 5 is, I find, the best tradeoff between size and reliability, and, realistically, much more practical for anything that might use a Raspberry Pi (that is, non-industrial). Although it probably suffers from the same performance ceiling that this demo suffered from on RAID 10.
Raid 5 is highly vulnerable to read errors on rebuild. Raid 6 or RAID 10 is much more suitable to today's large hard drive sizes. Raid 5 is not really recommended any more. It's OK for home use if you also have a good backup.
I just bought a Pi 4, but the drive I'm using is a 4TB external drive that's externally powered. I had bad experience with drives powered by the USB port alone.
Great video! But I don't understand why the writing on the raid 0 and raid 1 configuration is limited to 57 MB/s. Is the ethernet port or the samba protocol limiting the speed?
If it wasn't for shear mass storage needs vs the cost of SSD's spinning hard drives would have been killed off years ago. Right now we are quickly approaching the point where fitting a traditional HDD just does not make sense for a consumer PC vs the performance lost in relation to Sata SSD's and NVMe based SSD's. The last bastion for this tech is literally things like NAS storage where outright speed is limited anyway by the networking standards and shear capacity is the major concern. Outside of greater storage capacities, I don't see any companies with large enough pockets putting up the money to further develop the old spinning disk tech. The speeds have been pretty similar on hard drives for the last since the mid 1990' (with the exception of SSHD's which just bolt on a small SSD to a HDD) so that part is not likely to change. I believe that we are pretty close to the end of life for this tech and thus it is as mature as it is ever likely to get.
Do you not look into it? There's assloads of new innovation this year. We have dual actuators, HAMR, etc all coming out. Hard drives in the next few years will saturate SATA3 speeds as long as all the actuators are being used for throughput. Same as SSDs do. Just seek times will be better (not including NVME/m.2 drives) SATA needs to die or get upgraded.
Well speed race on HDD was over many years ago when ssd showed up en mass. Only thing that keeps mechanical hdd alive is its capacity/$ advantage, which has flattened out over the years as well. That means hdds days are numbered. I give it 7 years plus minus 3 years.
This sounds good in theory. But I have tried it myself with a Pi3, and it was extremely crashy and unstable. I tried RAID5 myself. A Raspberry Pi is great if you want to connect one drive. Maybe even drive mirroring for backup purposes. But RAID? No. The lack of SATA connectivity basically defeats the purpose of RAID. Does the addition of USB 3.0 ports on the Pi4 change this significantly?
The other thing I struggled with was the form factor. It would be great if someone could design an enclosure that would hold the Pi and the hard drives elegantly.
the quality of your audio while shooting in front of camera is better, than that recording while screen capturing/recording. Also the audio seems unnatural while screen recording. Please continue using same mic/audio setup for desktop recording session as well. Thanks for the informative video and your time and effort being put up to compare these , much appreciated. :)
Yup. I bought Sabren DS-4SSD and mod it to fit Seagate External 3.5" 5TB HDD. Results? 4 HDD instant access offline/online powered by Raspberry Pi4.. what else could you ask for? :D
I’m perplexed as to why you didn’t use SSDs instead. They typically have faster write speeds and that would have been a big boost in performance of the write speed of the various RAID configurations. Also, it would have been helpful to see the results of a RAID 5 configuration where three drives are all that is required, thus not wasting the capacity and I/O of a fourth.
You are joking about the SSDs, right? I want storage primarily, terabytes of redundant storage and I am using a $35 computer to build it. If anyone could afford terabytes of SSD storage they wouldn't be using a Pi. SSD is faster, sure, but price per MB you need HDD.
@@GaryExplains - For the tests of speed it seems that SSDs would have deserved a place in the RAID comparison. If budget is the driving factor, then definitely HDDs are the obvious choice. But budget, even using a Pi, is not always the driver. Perhaps a mobile NAS is desired.
Hi Gary! Please, help me. I have an RPI4 running a Plex server, Pihole, etc etc. Power kept dropping while running certain tasks involving the attached HDD, so I started using a powered USB3 hub - problem solved. But now the Pi won't boot with the powered USB hub attached. I have to unplug it, wait for boot, then plug it back in. What’s happening? Why does it fail with a powered hub 3.0? Thanks!
As for read times, I found those alarmingly high. Did you clear the linux OS cache and buffers before attempting the read test? # sync; echo 3 > /proc/sys/vm/drop_caches
Fantastic video. I have been struggling to figure out LVM on my 2 banks of 2 HDD's (yeah they are the slower ones because I am cheap). This does make so much more sense, and I already have Open Media on the pi, I just thought I had to get everything set up before I could use it to start moving stuff around. Thank for the smartness.
Aww I wanted to see the RAID 5 speeds! Contrary to some others I have more interest on the hard drive performance than "how to build a NAS" (come on people, there are great tutorials for that everywhere and it's not even a Raspberry Pi specific thing).
nas with hdmi mini can do 5GB output each for a 10GB NAS We can ssh in but need work for hardward and software done. internet will not sell faster connected device. watch for dos attacks
Gary Explains thank you for your reply, suggestions, and information, I am agreeing with you, it will safe to buy 4gb since who’s know I need to repurpose it in the future.
Could you please name the model that RPi4 can handle with more than one plate ? None of my active USB hubs or 2.5" enclosures will work when more than one 2.5" HDD or SSD is active on the RPi4 @4GB. Thanks
Let's be real here. The only option you have for storage on RPi is to use the USB 3 interface. RAID over USB is a disaster awaiting to happen, and you'll most likely lose all your data. Unless you get an enclosure that has built in hardware RAID. If you're not getting an enclosure with built in NAS, My suggestion is to run UnionFS or MergerFS. It has no redundancy, but at least, it's growable/expandable. If one hard drive dies, you don't lose all data, you only lose data that's stored on that particular HDD. If you want redundancy, what you can do is setup R Sync for one particular folder/path, which is where you put all your super important datas. Then R Sync it across all the HDD arrays, or a separate disk. At least if any disk dies, you still have your important data available. Thank me later.
Yeah, I also would love to know what brand/type of SSD drives were used here. I have bought external USB drives in the past, and they were ruined with really odd access software etc. I want to get an external USDB3 -> SSD of say 500gb, and have it JUST WORK out of the box with the Pi.
I was using hard drives, not SSD. I used 1 SSD as a baseline. I have never had any problem with any external HDDs or SSDs with Linux or the Pi. I would be interested to hear what brand/model of external drives you have bought and what problems it had, I have never heard of that kind of problem before.
These were actually both pretty old drives from about 6 years back. One was a Western Digital, the other a Sumsung device. Both USB and based on latptop hard disks rather than SSD as I recall. One of these two had some kind of hardware/software combo that made it dificult to use cross device. As I recall, I used I think it was the WD drive on a PC and a Mac, using s format both could read, but one of those two OS had issues recognising the drive. Anyway, no worries, if thats not really an issue with current hard disks then thats fine. I think Ill try the Samsung drive with a new Kodi build I am testing on a Pi 4. Its connector appears to be USB 3, so it could actually be of use on something Im trying to keep costs down on.
No, I don't think so. I think the limit is for all the ports together. My guess is that the two ports are really just a hub in themselves and not two separate USB subsystems.
YMMV. The usb3 ports are shared, and software raid is limited by the USB + HDD controllers and their latency. The practical benefit would be mitigated by the expense of good usb3 hubs. For the ~$200 in similar hardware outlay, you could probably get a NUC or mini ITX motherboard/refurbished system for 2-4 drives. Using 2 usb3 hubs would add more latency bottlenecks, but improve linear performance. The rpi hardware isn't built for NAS, and testing would likely require a USB 3.0 or 3.1 SSD to assess the PCIe bandwidth and throughput, vs a 3.0/3.1 desktop system, just to make sure the usb3 chipset on the rpi 4 isn't a bottleneck. The typical spinning HDD doesn't chew up the bandwidth of usb3, as long as you don't try to do things like RAID, because it is a serial, sequential bandwidth. What slows raid is waiting for hardware to catch up, and not having buffer/cache for frequent / hot data. Which is going to be the typical OMV or software raid experience. Stripe and mirror LVM isn't super helpful, but it is practical in the short term. Long term, HDD and SSD data loss is inevitable and drives are expendable, not reliable. Unfortunately, the results aren't going to be helpful without knowing what the peak throughput is, ie using SSD hardware as a comparison for the rpi 4 SoC and the via VL805 USB chipset.
The Raspberry Pi doesn't have a SATA interface. Here I use a USB to SATA adapter. Most HATs for the Pi are just USB to SATA convertors. The Compute Module 4 does have a PCIe interface, so there is the option to add SATA or NVMe.
Are you saying that a NAS isn't worth it? If you are then I strongly disagree, I have 3 and use them all every day. My network isn't special just gigabit.
You fail to mention the capabilities of the SMB device at the other end. Do you know for a fact that it can fully saturate the gigabit network on read and write (eg. it uses a ramdisk or SSD)? If not, that's the limiting factor and the SMB tests are invalid.
I installed openmediavault on Raspberry Pi 3B+ and noticed RAID was not supported over USB devices. Not sure how to do this using Pi with openmediavault 5.0
Hello Gary, many thanks for the review, it is very useful for me. I'd like to ask about stability of having 4 drives connected via USB3.0 hub. Lately I bought ROCK64 SBC and wanted to have the same setup as in your video (SBC -> USB3.0 HUB -> HDDs) but it actually came up that due to some driver-related issues it is not possible on ROCK64 using its USB3 port and USB3 HUB. I was still getting dmesg errors like "usb 5-1.3: reset SuperSpeed Gen 1 USB device number 4 using xhci-hcd" and HDDs were getting connected and disconnected over and over. On USB2.0 port it was OK. In the end I've found out that this is common issue for the ROCK64. How about RPI4? Did you observe any stability issues? Thank you in advance for your answer.
I did see some stability errors however I think they were all related to the drives not getting enough current (i.e. when I didn't use a powered hub). Also, I had a drive fail and when it was acting bad it did tend to upset the whole USB system. Once it was removed stability returned.
You can easily get to 116MiB/s from gigabit ethernet regardless of it is SMB, NFS, iSCSI, FCoE or even RoCE. You have not even tried to get theoretical maximum speed by even simplest method, which is calculation I have been moving data over gigabit ethernet over many different protocols. I know real performance and 116MB is absolutely normal
Actually all the numbers i presented where derived from calculation. Windows Explorer reported faster rates than my calculations. And what do you mean by "you have not even tried". How do you know what I tried and what I didn't try?
Actually it depends on how you are measuring it. Linux and Windows differ in what is megabyte, it is 1000K or 1024K? Like a megawatt is 1000 watts. I have a whole video about it. You should really understand these things before writing comments.
@@GaryExplains Difference between 1000K and 1024K is about 2%, rising up to 4-5% if you are talking millions of bytes per second vs MB/s, while the difference between 95 and 110 is about 15%, depending which direction you look. Definitely, GbE is not maxed out at 95 whatever units you can think of, including protocol overhead. You should really not underestimate your viewers' knowledge.
And since the theoretical maximum of Gb Ethernet is 116MB/s not including CIFS... I don't know how you can get 115MB/s easily over SMB. That is very clever of you.
Having said that. I just re-ran the test using a different server and it was slightly higher than 95MB/s. It looks like because the Pi's speed was close to the maximum of Gb Ethernet I assumed it was the maximum. I guess some further investigation could be done to find out exactly where the bottleneck is, but it won't change the results of the video. Using a Pi with 1 drive is enough to saturate the system and/or the network interface.
@@GaryExplains Saturate the system - yes. Saturate RGMII interface between SoC and network chip - unlikely, but maybe. But definitely not enough to saturate network interface. I'm insisting on this because the the biggest selling point of new Pi is that NIC and USB are not internally limited by USB 2.0 bus and, therefore, this is the feature that I think should be tested the most thoroughly. And yes, I can get 113-114 MB/s (measured by Win Explorer) and 115800 KB/s (measured by Total Commander) by copying from Qnap NAS to my desktop over GbE, which falls into previously stated 110-115 MB/s category. i.imgsafe.org/85/85a493fbe6.png
I tried RAID 1 with USB drives a few years past. One of the was slightly faster than the other, got mounted first, and flagged the array as degraded on every boot. The drives would then resync (copy every single sector) before returning to a safe state. It's this still an issue?
I'm wondering if you could power more hard drives from the Pi itself if you staggered their spin-up. From what I know of hard drives they draw more current at spin-up than during continuous use, and don't big commercial NAS systems also do this? BTW great channel
Hi, am having 1Tb usb 2.0 and 2tb usb 3.0 drives. Am planning to set up a nas server in my home. Rpi4 4gb or TP-Link archer c2300 router which will be good for NAS performance? Can you plz suggest me...
Awesome video. I'm wondering if you could achieve higher network speeds by attaching a USB 3.0 to Gigabit Ethernet adapter and do link aggregation. Idk. You would probably saturate the USB 3.0 controller if you attached 4 HDDs AND an Ethernet adapter though... Just some random thoughts.
Thank you for interesting video, made me think about "upgrading" of my old NAS. What about the total power consumption (Pi4 + 4 drives @ full HDD load)? Can you measure it for us?
Hi i would like to know is it possible to use a old laptop as a raspberry pi. I want to use 2 hdd of 2 tb on a usb hub that is connected via usb on to the laptop. Can i then use raid an nas on these 2 devices?
Attempting to build a cloud server with 3 drives, I am using a powered usb hub to connect the drives and am having issues. A drive/drives will disconnect randomly while I am setting up the raid and the whole pi freezes I can't move my mouse or use the keyboard. Any help would be nice
Openmediavault does not support Raid over USB, suggesting that users do this would require knowledge of using mdadm from the command line. With 2 x USB3 and 2 x USB2 your bottleneck is always going to be the USB2 ports, the most sensible option from this video is at the end -> use a single hard drive with a second hard drive running rsync as a cron job overnight. For the a home user a Raid configuration offers no advantage over using single hard drives.
@@GaryExplains I never said you did but you referenced configuring a Raid 10, I'm not trying to do down your video I'm simply trying to point out that using a Raid config for home users offers no advantage. A better configuration would be to use the 2 x USB3 as data and the 2 x USB2 as rsync running cron jobs overnight. The Pi4 has finally caught up with other SBC's and as time goes on will offer the home user something worth using, I have 2 Pi's but they both do specific jobs on my network.
Storage servers also have fans in them so that seems like a good idea
5 лет назад
Is OpenMediaVault really running on Raspberry Pi 4? How did you do that? It didn't run for me (I tried OMV4 image for RPi3 which didn't boot at all and also package version on newest Raspbian OS) and now I'm desperately waiting for OMV5 or some other solution…
Current OMV devs are ignoring the most popular sbc because they feel other boards are better suited for OMV NAS. Don't expect Pi4 support to be any better in OMV 5.
What is the point of RAID 0 if there's no fault tolerance whatsoever? RAID 1 or RAID 5 seem to be the best ones... en.wikipedia.org/wiki/Standard_RAID_levels#Comparison
Hi! I am in the process of building and testing a setup with ZFS, and I would love to compare my results to yours. Do you remember how you went about testing? How big was the file sent over the network? And the one from and to the RAM disk? Did you just `time` the commands in the cli and manually calculate the throughput? Or is there a good tool that runs the tests and returns the numbers? How can we make sure the cache on the HDDs does not affect the test results?
If also need a step by step guide to building a NAS with a Raspberry Pi, then I have a video about that as well: ruclips.net/video/8fIeiZx4voU/видео.html
Tittle is miss leading. Title implies a tutorial of building a rpi nas. This gives speed test results of different HD configurations.
Useful information but not why I started this video.
Rename this so it describes Video appropriately.
I don't see the words "how to" or "tutorial" in the title or the thumbnail. Plus the description describes exactly what is in the video.
@@GaryExplains I can understand the confusion. Building, usually imply s that the steps are covered in a project.
Same here... With a title "Build a Raspberry Pi etc, etc" in a YT channel called... "Gary Explains"... I was kind of expecting a step by step tutorial of some sort... but ok... I already found some option and some older info elsewhere...
Agreed. I was also expecting build / software info and none was provided. The title should definitely be revised.
yeah i also thought it was a tutorial but no.
let me fix your title "Gary explains data of Networking with Raspberry Pi" there... you're welcome...
True man...the title is severely misleading...*presses dislike button
I just watched a 14min video waiting for him to actually build a nas...
Thanks for saving my time dude.
where is the Build a Raspberry Pi 4 NAS tutorial, Sir?
also wanting to see this :)
@@le8a9p easy.
Create a RAID 5 storage of 3 or more disks.
Make a LVM2 on top of that RAID device.
Install Samba or NFS or iSCSI to distribute disks to clients.
From here you can also install other services for your LAN. You are welcome. :-)
And the most important. Boot your RPi from USB instead of µSD. USB memory sticks are more stable and easy to manage then µSD. You could even use a USB disk for your OS, instead of those flimsy µSD.
Anders Jackson Software RAID 5 using LVM or maybe ZFSoL? Kind of tempted to try something like this it would certainly use much less power than my DL380 freenas box lol.
@@AndersJackson I'm not too sure about USB sticks reliability over micro sd cards. ;)
@@goku445 well. My experiance is different. 😜
Agree with misleading title, build equals guide or tutorial and this isn’t it
Pleeze, the nerd solution to NAS, which I have nicknamed the NASty is to get an old desktop computer, whatever you have laying around. Then figure out how many hard drives you can connect inside the case. Then salvage every hard drive you can find that is "decent enough" and shove them all inside. Load Linux and do some network wizardry and you have yourself a NASty for free. You can get a little fancy with this if you want, or you could go full-on nasty and just have each drive shared on the network.
that's what I'm thinking on doing, buying a pack of old/failing disks if I can find for cheap (no seensitive data, just messing with VMs and nerdy stuff)
@@FeelingShred You can find some cheap TB drives on ebay. It may be hit or miss though. I got one for about $20 several months ago for my desktop. It was high-hours but it was still healthy. Another option is to roll around the nicer neighborhoods in your area on trash day and look for any computers being thrown out. Could also put an ad on Facebook marketplace seeking "old computers" and maybe offer like $5 for a complete computer. Might be surprised at who will respond. I miss the days of finding free computers everywhere.
@@NotSoCrazyNinja Thanks :) Around the place I live people don't toss computers like that, a few days ago I saw some people still charging something like 80 Dollars for a functioning (allegedly functioning) 486, outrageous... Some cheap p4/dual core CPUs though... But I won't drag you down with how boring life feels around here xDDD
@@FeelingShred Life is boring practically everywhere, sadly. Just have to do what you can do and try to have fun.
@@NotSoCrazyNinja Yeah, nerdy things put a smile on my face =D
Hi Gary,
Great video.
I just wondered what would happen if you would take 2 powered usb hubs attached to 2 different ports on the pi with a raid 10 setup.
Would this increase the R/W speed or is the bottleneck dependent on the raspi internals?
Thanks for your answer :)
He didn't actually build anything...
Great video! I also wonder how would 3 or 4 disks configured with RAID 5 perform.
By the way, what if instead of connecting 4 disks to a hub that would use one USB 3 port, I would instead use two hubs with two disks each, using up both USB 3 ports of the Pi? Would it make a difference in terms of speed?
I tried various combinations including 2 x hubs with 2x HDD in each. I didn't see any performance difference. My guess is that the 2x USB ports are just a built-in mini-hub connected to the same controller. I can't find any information on the Pi website or on Broadcom's website to confirm or deny this. All I have is, "much faster input/output, due to the incorporation of a PCIe link that connects the USB 2 and USB 3 ports, and a natively attached Ethernet controller."
The protocol overhead for SATA over USB is the killer however Lycom make Pi hats
I was thinking the same thing with respect to using a hub from a single port. After looking around I believe that the usb 3 slots share a controller so it probably wouldn't help... however, if you set it up and then administer everything remotely I think you could use one usb 3 hub for 2 drives and both usb 2 ports and perhaps that would let you bump up the net smb raid write speeds. You would probably be throttled thanks to using USB 2 but in the aggregate gain a few Megs/sec (maybe up to 40 or 45)... but that requires the usb 2 ports having a separate controller.
I have a Frankenserver that does something similar to this but uses LVM to tie the drives together. In this way I can create some large areas for storage that aren’t mirrored but then I can also create areas that are double or triple mirrored across the drives. It’s seemingly a good way to use up all those disparate old drives that collect about the place. No idea what the performance is like though as I basically just use it for warm archiving.
The problem with JBOD (Just a bunch of disks, like RAID 0 or LVM with more then one disk) is that you lose most/all your data if you lose one disk. And the risk of loosing all data will actually double with each disk you add.
Better to have a RAID 5 under LVM2 (In principle data on all disks except one that do have check sums so you can recreate one, but only one, lost disk).
LVM mirroring duplicates data across disks in the volume group so that data has some level or redundancy I believe. The data that goes on the spanned logical volumes is disposable things like internet downloads that take up a lot of room and don’t matter if they are lost.
@@Alan.livingston as long as you know and remember that data on a single disk is not actually safe, it is ok JBOD. Where one disk failure will make you loose all data there.
And yes, LVM mirroring are a RAID light, but not a proper RAID.
But again, your mileage might vary. :-)
If you're going to do this, be aware that your USB-SATA device maybe doesn't support SMART or have incomplete support. And when the time comes to rebuild, you'll probably want to have a PC with some SATA ports available.
What's the best way to power 3 SSD's to use as a NAS on the Pi 4? Will the USB hub be necessary? I've been told SSD's don't need need as much power as conventional HDD's.
Very helpful indeed, but for me, I didn't think it would be worth the effort to create a RAID 1.
Can you please make a video about this "night job"? Do you use anything else or just Task Scheduler?
Finally a Pi4 NAS test
I'm looking to create a RAID 1 setup using two M.2 SSDs on my Raspberry Pi 5 (4GB). I already have the Geekworm X1004 adapter. What software would you recommend for setting up the RAID 1 and creating a private cloud using Nextcloud? Could you provide a step-by-step guide or even a video tutorial on how to do this, including how to access the Nextcloud with RAID protection?
My goal is to move away from Google Drive and Google Photos.
Additionally, I'm interested in having multiple Raspberry Pi 5 devices in different locations for data redundancy. For example, one at my house, one at my parents', and another at my grandmother's. If one fails, I'll still have my data available on another device. Is it possible to create a RAID 1 configuration across multiple Raspberry Pi 5 devices?
Any help would be greatly appreciated!
This came at the perfect time man. I was struggling to setup a pi nes with plex compatibility
Raid 1 works great for file serving. I use an externally powered, Sabrient dual SATA III usb 3 system. I have two 7200 RPM drives in there now just because I had them on hand.
I can connect from Windows, iPhone, iPad, Raspbian or whatever.
I tried putting own cloud over top of that but just found it largely unnecessary for basic file serving. I may try it again with K3 and Docker.
I boot to the other USB 3 port on 120 SSD from western digital. Pi4 is powering the USB boot drive. But if I place a microSD in the pi it automatically use it instead.
There's one thing I didn't get while watching... Did you use the two RPi USB ports with the two USB drives to avoid USB 3 bandwidth cap? The same goes for four drives, and RAID 10... Are the two "RAID 0" drives connected to one USB port and the other two over another one? I think that may change performance results quite a bit according to USB ports distribution
The bandwidth cap would be shared across the USB 3 ports so you wouldn't be able to avoid it by using both ports unfortunately.
If complete redundancy isn't necessary, what do you think about having 3 independent drives with a parity drive for backup? Basically unRaid
Has anyone ever told you that you look like varg vikernes.
Also, this channel deserves more subscribers.
Since Varg is younger than me, I guess it is only correct to say that he looks like me, not the other way around!
@@GaryExplains It makes me happy to know that you know who Varg is and for replying.
Also, I actually thought you were the younger one.
@@_syedmx86 The Professor is a Time Lord, that might explain the age confusion.
The way you described making the manual raid with four drives I thought would be called 0+1 and not 1+0. 1+0 is making sets (usually pairs) of mirrored drives, and then striping all of the mirrored sets. 0+1 is making striped sets and then mirroring the striped sets. 0+1 is a lot less safe than 1+0.
Or am I getting things backwards?
Ah, yes, I am always confusing them myself. So, I went back and check my notes. What I actually tested was RAID 1+0, but I mistakenly described RAID 0+1 in the video! 😕
Thanks for clearing that up. It is good to know what the Pi 4 can do as a NAS. I'm thinking of getting a NAS and I would definitely be more inclined to make a DIY solution like this rather than using something more conventional.
If you tried NFS you would probably got much better performance when reading and writing over network.
Reading and writing from /dev/zero and /dev/null wold be even faster.. ;-)
What about RAID 5 and RAID 6? Or even BTRFS or ZFS, which have RAID and LVM2 in them.
RAID and Backup, as you mentioned in the end is clearly not the same. Backup is saving data from accidental removing/overwriting. RAID "only" save you from loosing a disk.
NFS isn't that much better than samba speeds anymore.
@@9001greg that I have to have some data on to belive. Because that was not the case last time I saw some objectively data on this.
@@AndersJackson It takes less than 3 minutes to set them both up and test it. NFS is still the best but it isn't that much better, anymore. 10 years ago it was twice as fast, then it was 30% then it was 15%.. etc
I wonder ... concerning your RAID 1+0 test ... if you used both of the Pi's onboard USB 3.0 ports and dedicated each port to a RAID-0 set then created the RAID-1 from those, if the performance would change significantly or not ... if each USB port has a dedicated USB controller, then the performance should dramatically improve ... in theory ... it should go even faster if each USB port had a RAID 1 so that your final setup was RAID 1+1 ... although that would cut total storage capacity in half.
How did you attach the USB Hub to your Pi? Do you attach the whole hub to one USB 3 port? To both USB ports? Or to all 4 USB ports?
I used two USB hubs, one in each of the USB 3.0 ports.
Wow !
Gary Sir,
*You are really masterclass*
One of the finest teacher on youtube.
Keep it up sir.
Truth!
Great experiment! Is the atolla USB hub UASP compatible?
Thanks Gary, very interesting. One small question what was the make and capacity of the drives you tested with?
and how were they fomatted? NTFS? XFS? EXT4? other?
Regarding 4 drives Vs 2 drives - was the throughput limitation due connecting 4 drives over a single USB 3 port via the powered USB hub? If you used two USB hubs with two drives per hub and each hub connected to its own USB 3 port on the Raspberry pi, would you expect to see any improvement in overall throughput for 4 drives? Or is the throughput limitation observed actually a limitation of the raspberry pi's USB 3 controller and how it connects to the SOC's IO?
I used two hubs, one on each port. I also did try other combinations and I didn't see any performance difference. My guess is that the 2x USB ports are just a built-in mini-hub connected to the same controller. I can't find any information on the Pi website or on Broadcom's website to confirm or deny this. All I have is, "much faster input/output, due to the incorporation of a PCIe link that connects the USB 2 and USB 3 ports, and a natively attached Ethernet controller."
@@GaryExplains Thank you for an excellent video, and clarification of the test process. Regardless, it is still an excellent way to make a low-cost RAID 0 NAS.
Nice presentaion but it stands more in the performance perspective ..and while thinking of something what about if you have set a healthy raid (any level) with 2, 3 or 4 hdds plugged in the usb hub and the cleaning lady accidentally unplug them all while pi is off and replug them in any (wrong) order in the hub? Is actually the pi able to assort each disk as the correct raid member or will set the raid and the raid volume down.. and will ask for rebuilding array or re-create it?
Thanks for the great videos. I didn't think it was a good idea to run RAID over USB but have I got that wrong? If it's ok then I'm thinking of doing this but with a dumb 3.5" 4 bay USB 3 enclosure and connecting that to the Pi. The Pi will just see 4 drives and then I'll use OMV to RAID them. Is it ok to have the RAID hardware connected to the RAID software via USB in this manner?
I feel you should have spent a little time explaining the speed constraint that will arise if using drives formatted other than in Linux-friendly ext2/3/4 etc. The PI now has great i/o but the CPU will still constrain throughput if your USB drive(s) are formatted NTFS as many may be when first bought. My own pi NAS showed almost as much CPU load from accessing an NTFS drive as from running SMB so that's almost halving potential performance.
Why would you use anything other than "Linux-friendly ext2/3/4 etc." I didn't cover it because that isn't something you should be doing.
@@GaryExplains Well, there are valid reasons to share an NTFS drive. I like the Pi solution for a NAS. Should I have a Pi die, a new one can be had quite quickly. Backup the SD card often! Anyway, using Filezilla to read my NTFS drive on the PI, I can pump about 80% of the gigabit ethernet connection. There are gaps between files. Writing on the other hand is much slower. I get about 25-30 mB per second. Mount.ntfs does eat a bunch of CPU. Advantage of the NTFS drive: No problem using it directly on my Win7 machine, which I used to populate at USB 3 speeds (which still took days to fill the 16T drive from backups). Fast I/O at this point isn't a goal.
Hypothetically, 1)if we were to set a raid mirror, then somehow a USB cable got disconnected while we still are operating with the system, after realizing the cable is loose, we connect it back. I assume the raid will start to resync the disconnected drive. If so, what performance hit it will encounter, assuming we already have lots of data in the system. 2)What would be the performance hit of a Raid 5 in a similar resync? 3) Considering the high possibility of getting one of the USB cable/PS disconnects, how about using a dock station that handles 4 SATA with one single USB 3? Will that allow me to run Raid 5 or Raid 1 with the 4 drives?
Nicely explained video. But what about raid 5 across 4 disks..?
I was thinking the same thing. RAID 5 is, I find, the best tradeoff between size and reliability, and, realistically, much more practical for anything that might use a Raspberry Pi (that is, non-industrial). Although it probably suffers from the same performance ceiling that this demo suffered from on RAID 10.
Raid 5 is highly vulnerable to read errors on rebuild. Raid 6 or RAID 10 is much more suitable to today's large hard drive sizes. Raid 5 is not really recommended any more. It's OK for home use if you also have a good backup.
Wish you could have also set the drives up in RAID-5 or RAID-6, to compare them with the other raid setups like 10 and 1+0.
I just bought a Pi 4, but the drive I'm using is a 4TB external drive that's externally powered. I had bad experience with drives powered by the USB port alone.
Great video! But I don't understand why the writing on the raid 0 and raid 1 configuration is limited to 57 MB/s. Is the ethernet port or the samba protocol limiting the speed?
Why did you skip RAID 5 & 6? That's what I was interested in.
Do you think that HDD technology has matured as much as it can? Or do you think there are more innovations waiting, for spinning magnetic disk drives?
If it wasn't for shear mass storage needs vs the cost of SSD's spinning hard drives would have been killed off years ago. Right now we are quickly approaching the point where fitting a traditional HDD just does not make sense for a consumer PC vs the performance lost in relation to Sata SSD's and NVMe based SSD's. The last bastion for this tech is literally things like NAS storage where outright speed is limited anyway by the networking standards and shear capacity is the major concern.
Outside of greater storage capacities, I don't see any companies with large enough pockets putting up the money to further develop the old spinning disk tech. The speeds have been pretty similar on hard drives for the last since the mid 1990' (with the exception of SSHD's which just bolt on a small SSD to a HDD) so that part is not likely to change. I believe that we are pretty close to the end of life for this tech and thus it is as mature as it is ever likely to get.
Do you not look into it? There's assloads of new innovation this year. We have dual actuators, HAMR, etc all coming out. Hard drives in the next few years will saturate SATA3 speeds as long as all the actuators are being used for throughput. Same as SSDs do. Just seek times will be better (not including NVME/m.2 drives) SATA needs to die or get upgraded.
@@jameslewis2635 10gbE is rather cheap so network speed isn't much of a bottleneck when you're wanting 'more speed' it is far from the big cost.
Well speed race on HDD was over many years ago when ssd showed up en mass. Only thing that keeps mechanical hdd alive is its capacity/$ advantage, which has flattened out over the years as well. That means hdds days are numbered. I give it 7 years plus minus 3 years.
95 ms/s is not the maximum for gigabit ethernet, its more like 120. What you are seeing is a limitation of the Pi CPU not just the network itself.
This sounds good in theory. But I have tried it myself with a Pi3, and it was extremely crashy and unstable. I tried RAID5 myself. A Raspberry Pi is great if you want to connect one drive. Maybe even drive mirroring for backup purposes.
But RAID? No. The lack of SATA connectivity basically defeats the purpose of RAID.
Does the addition of USB 3.0 ports on the Pi4 change this significantly?
Where you using drives with external power supplies?
@@GaryExplains I used a powered USB hub.
The other thing I struggled with was the form factor. It would be great if someone could design an enclosure that would hold the Pi and the hard drives elegantly.
I have no experience with the Raspberry Pi and was wondering... Doesn't it have any controller for SATA, etc.?
the quality of your audio while shooting in front of camera is better, than that recording while screen capturing/recording. Also the audio seems unnatural while screen recording. Please continue using same mic/audio setup for desktop recording session as well. Thanks for the informative video and your time and effort being put up to compare these , much appreciated. :)
Yup. I bought Sabren DS-4SSD and mod it to fit Seagate External 3.5" 5TB HDD. Results? 4 HDD instant access offline/online powered by Raspberry Pi4.. what else could you ask for? :D
pictures/links ? please please please =D just for "educational purposes"
I'm thinking myself of buying a lot of used/failing hard disks and building a small storage hub out of them (no sensitive data)
I’m perplexed as to why you didn’t use SSDs instead. They typically have faster write speeds and that would have been a big boost in performance of the write speed of the various RAID configurations. Also, it would have been helpful to see the results of a RAID 5 configuration where three drives are all that is required, thus not wasting the capacity and I/O of a fourth.
You are joking about the SSDs, right? I want storage primarily, terabytes of redundant storage and I am using a $35 computer to build it. If anyone could afford terabytes of SSD storage they wouldn't be using a Pi. SSD is faster, sure, but price per MB you need HDD.
@@GaryExplains - For the tests of speed it seems that SSDs would have deserved a place in the RAID comparison. If budget is the driving factor, then definitely HDDs are the obvious choice. But budget, even using a Pi, is not always the driver. Perhaps a mobile NAS is desired.
Would having 2 usb hubs and both USB 3 ports have made a difference to speeds?
great way to create a own nas with the little raspberry.
I think you just described RAID 0+1 instead of RAID 1+0.
I did. Sorry about that. 😔 I checked my notes and the numbers are definitely for 1+0.
Hi Gary! Please, help me.
I have an RPI4 running a Plex server, Pihole, etc etc.
Power kept dropping while running certain tasks involving the attached HDD, so I started using a powered USB3 hub - problem solved.
But now the Pi won't boot with the powered USB hub attached. I have to unplug it, wait for boot, then plug it back in.
What’s happening? Why does it fail with a powered hub 3.0?
Thanks!
As for read times, I found those alarmingly high. Did you clear the linux OS cache and buffers before attempting the read test? # sync; echo 3 > /proc/sys/vm/drop_caches
Great overview. Would have been really nice to see a comparison with 3 (or more) drives in a RAID 5 though.
Fantastic video. I have been struggling to figure out LVM on my 2 banks of 2 HDD's (yeah they are the slower ones because I am cheap). This does make so much more sense, and I already have Open Media on the pi, I just thought I had to get everything set up before I could use it to start moving stuff around. Thank for the smartness.
Glad it helped!
Aww I wanted to see the RAID 5 speeds!
Contrary to some others I have more interest on the hard drive performance than "how to build a NAS" (come on people, there are great tutorials for that everywhere and it's not even a Raspberry Pi specific thing).
nas with hdmi mini can do 5GB output each for a 10GB NAS We can ssh in but need work for hardward and software done. internet will not sell faster connected device. watch for dos attacks
Did you ever figure out a way to get around the read over network limitation? Is there a way to add another Ethernet port to the PI?
Which raspberry pie 4 1, 2 or 4 Gb ram that you used for the test? I tried to find it in video but don’t get the answer
Anyway thank you for your test
It will work with all of them. But I would recommend the 2GB one and the 4GB is always nice to have.
Gary Explains thank you for your reply, suggestions, and information, I am agreeing with you, it will safe to buy 4gb since who’s know I need to repurpose it in the future.
Could you please name the model that RPi4 can handle with more than one plate ? None of my active USB hubs or 2.5" enclosures will work when more than one 2.5" HDD or SSD is active on the RPi4 @4GB. Thanks
Let's be real here. The only option you have for storage on RPi is to use the USB 3 interface.
RAID over USB is a disaster awaiting to happen, and you'll most likely lose all your data.
Unless you get an enclosure that has built in hardware RAID.
If you're not getting an enclosure with built in NAS, My suggestion is to run UnionFS or MergerFS. It has no redundancy, but at least, it's growable/expandable. If one hard drive dies, you don't lose all data, you only lose data that's stored on that particular HDD.
If you want redundancy, what you can do is setup R Sync for one particular folder/path, which is where you put all your super important datas. Then R Sync it across all the HDD arrays, or a separate disk. At least if any disk dies, you still have your important data available.
Thank me later.
*GARY!!!*
*Good Morning Professor!*
*Good Morning Fellow Classmates!*
Man, am I really late for this one!
Check that notification bell folks!
Yeah, I also would love to know what brand/type of SSD drives were used here. I have bought external USB drives in the past, and they were ruined with really odd access software etc. I want to get an external USDB3 -> SSD of say 500gb, and have it JUST WORK out of the box with the Pi.
I was using hard drives, not SSD. I used 1 SSD as a baseline. I have never had any problem with any external HDDs or SSDs with Linux or the Pi. I would be interested to hear what brand/model of external drives you have bought and what problems it had, I have never heard of that kind of problem before.
These were actually both pretty old drives from about 6 years back. One was a Western Digital, the other a Sumsung device. Both USB and based on latptop hard disks rather than SSD as I recall. One of these two had some kind of hardware/software combo that made it dificult to use cross device. As I recall, I used I think it was the WD drive on a PC and a Mac, using s format both could read, but one of those two OS had issues recognising the drive. Anyway, no worries, if thats not really an issue with current hard disks then thats fine.
I think Ill try the Samsung drive with a new Kodi build I am testing on a Pi 4. Its connector appears to be USB 3, so it could actually be of use on something Im trying to keep costs down on.
Is there any difference if you use 2 USB hubs with 2 hdd each one? (Using 2 USB 3 ports)
No, I don't think so. I think the limit is for all the ports together. My guess is that the two ports are really just a hub in themselves and not two separate USB subsystems.
YMMV. The usb3 ports are shared, and software raid is limited by the USB + HDD controllers and their latency.
The practical benefit would be mitigated by the expense of good usb3 hubs. For the ~$200 in similar hardware outlay, you could probably get a NUC or mini ITX motherboard/refurbished system for 2-4 drives.
Using 2 usb3 hubs would add more latency bottlenecks, but improve linear performance.
The rpi hardware isn't built for NAS, and testing would likely require a USB 3.0 or 3.1 SSD to assess the PCIe bandwidth and throughput, vs a 3.0/3.1 desktop system, just to make sure the usb3 chipset on the rpi 4 isn't a bottleneck.
The typical spinning HDD doesn't chew up the bandwidth of usb3, as long as you don't try to do things like RAID, because it is a serial, sequential bandwidth. What slows raid is waiting for hardware to catch up, and not having buffer/cache for frequent / hot data. Which is going to be the typical OMV or software raid experience.
Stripe and mirror LVM isn't super helpful, but it is practical in the short term. Long term, HDD and SSD data loss is inevitable and drives are expendable, not reliable.
Unfortunately, the results aren't going to be helpful without knowing what the peak throughput is, ie using SSD hardware as a comparison for the rpi 4 SoC and the via VL805 USB chipset.
Do you hace a video with 4 sata hard drives?
What is the difference in performance usb vs sata
The Raspberry Pi doesn't have a SATA interface. Here I use a USB to SATA adapter. Most HATs for the Pi are just USB to SATA convertors. The Compute Module 4 does have a PCIe interface, so there is the option to add SATA or NVMe.
Great video, thanks! NB: re NAS not worth it, depends on the type of network/IFs you use.
Are you saying that a NAS isn't worth it? If you are then I strongly disagree, I have 3 and use them all every day. My network isn't special just gigabit.
Well Done, hmmmm hopefully this is not a foolish question but what about RAID5 ?
This is a very late comment but is there any advantage in using RAiD 0 with Timeshift on a Linux machine? Timeshift working on its own HD/SSD.
You fail to mention the capabilities of the SMB device at the other end. Do you know for a fact that it can fully saturate the gigabit network on read and write (eg. it uses a ramdisk or SSD)? If not, that's the limiting factor and the SMB tests are invalid.
Yes, I know for a fact that it saturated the Gigabit network.
I installed openmediavault on Raspberry Pi 3B+ and noticed RAID was not supported over USB devices. Not sure how to do this using Pi with openmediavault 5.0
Is it possible using 4 hdd 3.5?
great video
Hello Gary,
many thanks for the review, it is very useful for me. I'd like to ask about stability of having 4 drives connected via USB3.0 hub. Lately I bought ROCK64 SBC and wanted to have the same setup as in your video (SBC -> USB3.0 HUB -> HDDs) but it actually came up that due to some driver-related issues it is not possible on ROCK64 using its USB3 port and USB3 HUB. I was still getting dmesg errors like "usb 5-1.3: reset SuperSpeed Gen 1 USB device number 4 using xhci-hcd" and HDDs were getting connected and disconnected over and over. On USB2.0 port it was OK. In the end I've found out that this is common issue for the ROCK64. How about RPI4? Did you observe any stability issues? Thank you in advance for your answer.
I did see some stability errors however I think they were all related to the drives not getting enough current (i.e. when I didn't use a powered hub). Also, I had a drive fail and when it was acting bad it did tend to upset the whole USB system. Once it was removed stability returned.
You can easily get to 116MiB/s from gigabit ethernet regardless of it is SMB, NFS, iSCSI, FCoE or even RoCE.
You have not even tried to get theoretical maximum speed by even simplest method, which is calculation
I have been moving data over gigabit ethernet over many different protocols. I know real performance and 116MB is absolutely normal
Actually all the numbers i presented where derived from calculation. Windows Explorer reported faster rates than my calculations. And what do you mean by "you have not even tried". How do you know what I tried and what I didn't try?
95MB/s is not GbE maxed out. You can get 110-115 MB/s easily over SMB using PCs or proper NAS. You should've found where that limit is.
Actually it depends on how you are measuring it. Linux and Windows differ in what is megabyte, it is 1000K or 1024K? Like a megawatt is 1000 watts. I have a whole video about it. You should really understand these things before writing comments.
@@GaryExplains Difference between 1000K and 1024K is about 2%, rising up to 4-5% if you are talking millions of bytes per second vs MB/s, while the difference between 95 and 110 is about 15%, depending which direction you look. Definitely, GbE is not maxed out at 95 whatever units you can think of, including protocol overhead. You should really not underestimate your viewers' knowledge.
And since the theoretical maximum of Gb Ethernet is 116MB/s not including CIFS... I don't know how you can get 115MB/s easily over SMB. That is very clever of you.
Having said that. I just re-ran the test using a different server and it was slightly higher than 95MB/s. It looks like because the Pi's speed was close to the maximum of Gb Ethernet I assumed it was the maximum. I guess some further investigation could be done to find out exactly where the bottleneck is, but it won't change the results of the video. Using a Pi with 1 drive is enough to saturate the system and/or the network interface.
@@GaryExplains Saturate the system - yes. Saturate RGMII interface between SoC and network chip - unlikely, but maybe. But definitely not enough to saturate network interface. I'm insisting on this because the the biggest selling point of new Pi is that NIC and USB are not internally limited by USB 2.0 bus and, therefore, this is the feature that I think should be tested the most thoroughly. And yes, I can get 113-114 MB/s (measured by Win Explorer) and 115800 KB/s (measured by Total Commander) by copying from Qnap NAS to my desktop over GbE, which falls into previously stated 110-115 MB/s category. i.imgsafe.org/85/85a493fbe6.png
Did you try splitting the raid 0 pairs over two usb 3.0 ports and test the results? The video implies not.
Yes, I did. The ports share a common path on the controller, so it doesn't make much difference.
I suspected that but hoped otherwise.
I tried RAID 1 with USB drives a few years past. One of the was slightly faster than the other, got mounted first, and flagged the array as degraded on every boot. The drives would then resync (copy every single sector) before returning to a safe state. It's this still an issue?
I'm wondering if you could power more hard drives from the Pi itself if you staggered their spin-up. From what I know of hard drives they draw more current at spin-up than during continuous use, and don't big commercial NAS systems also do this?
BTW great channel
Hi, am having 1Tb usb 2.0 and 2tb usb 3.0 drives. Am planning to set up a nas server in my home. Rpi4 4gb or TP-Link archer c2300 router which will be good for NAS performance?
Can you plz suggest me...
Could I install android prime os on that raspberry super computer? and (could I) play the server for a long time?
Awesome video. I'm wondering if you could achieve higher network speeds by attaching a USB 3.0 to Gigabit Ethernet adapter and do link aggregation. Idk. You would probably saturate the USB 3.0 controller if you attached 4 HDDs AND an Ethernet adapter though... Just some random thoughts.
I wonder how well will perform a RAID 4 or RAID 5 setup over the 4 disks.
I couldn't add disks through usb to RAID system. How can i do it?
im hoping 2xusb3.0 flash drives can be striped and booted from on pi4.. if it works itll be a fast system :)
Dont let Gary fool you, Open Media Vault can be a pain to get up and running! It wasnt all that hard though
M R - OMV? Thank you, I thought I'd never find out through the hyperbole!
Thank you for interesting video, made me think about "upgrading" of my old NAS. What about the total power consumption (Pi4 + 4 drives @ full HDD load)? Can you measure it for us?
Hi i would like to know is it possible to use a old laptop as a raspberry pi. I want to use 2 hdd of 2 tb on a usb hub that is connected via usb on to the laptop. Can i then use raid an nas on these 2 devices?
Yes you can do the same thing using a laptop.
Did you enable jumbo frames on the Pi NIC and your switch?
Good benchmarking but bad video title. Also missing that all important Raid 5 benchmark. Were you using mdadm or ZFS?
Gary, what is the cpu during raid1/5 write and read. I am concern lack of cpu for media transcoding when full load.
You should take 4 or 5 Raspberry Pi 4s and make a cluster and add one or two SSD to each...Like to see how that performs!
Attempting to build a cloud server with 3 drives, I am using a powered usb hub to connect the drives and am having issues. A drive/drives will disconnect randomly while I am setting up the raid and the whole pi freezes I can't move my mouse or use the keyboard. Any help would be nice
Also using the exact usb hub you showed in the video
Openmediavault does not support Raid over USB, suggesting that users do this would require knowledge of using mdadm from the command line. With 2 x USB3 and 2 x USB2 your bottleneck is always going to be the USB2 ports, the most sensible option from this video is at the end -> use a single hard drive with a second hard drive running rsync as a cron job overnight.
For the a home user a Raid configuration offers no advantage over using single hard drives.
I didn't use the USB 2 ports.
@@GaryExplains I never said you did but you referenced configuring a Raid 10, I'm not trying to do down your video I'm simply trying to point out that using a Raid config for home users offers no advantage. A better configuration would be to use the 2 x USB3 as data and the 2 x USB2 as rsync running cron jobs overnight.
The Pi4 has finally caught up with other SBC's and as time goes on will offer the home user something worth using, I have 2 Pi's but they both do specific jobs on my network.
I will be streaming 24/7 for media. Should I get hdd inclosher's with fan's for my 3.5" nas quality hdd's ?
Storage servers also have fans in them so that seems like a good idea
Is OpenMediaVault really running on Raspberry Pi 4? How did you do that? It didn't run for me (I tried OMV4 image for RPi3 which didn't boot at all and also package version on newest Raspbian OS) and now I'm desperately waiting for OMV5 or some other solution…
Current OMV devs are ignoring the most popular sbc because they feel other boards are better suited for OMV NAS. Don't expect Pi4 support to be any better in OMV 5.
on a gigabit Ethernet a fresh SSD via usb 3 should give atleast 200MB/s sequential write, shouldn't it?? does this mean pi 4 interfaces are slower?
200 MB/s is greater than the speed of Gigabit Ethernet.
@@GaryExplains oops...I missed that...125MB/s, forgive my stupidity
@Gary what was the filesystem on the hdds? It would also be an interesting comparison to do a ntfs vs ext4.
I was using ext4.
What is the point of RAID 0 if there's no fault tolerance whatsoever? RAID 1 or RAID 5 seem to be the best ones... en.wikipedia.org/wiki/Standard_RAID_levels#Comparison
Improved read and write speed, but as you say without fault tolerance. But it can also be combined with RAID 1 in a RAID 10 setup.
Hi! I am in the process of building and testing a setup with ZFS, and I would love to compare my results to yours.
Do you remember how you went about testing? How big was the file sent over the network? And the one from and to the RAM disk? Did you just `time` the commands in the cli and manually calculate the throughput? Or is there a good tool that runs the tests and returns the numbers? How can we make sure the cache on the HDDs does not affect the test results?
But what about Raid 5/6? If speed isnt a concern, how well does such a hackjob USB raid array fare?
How about spliting 2 drives in raid 0 into 2 separate usb channel?
So it's "I've Build a Raspberry Pi NAS"
I can write 80 MB/s over SMB from a 9 year old laptop to an old Intel SSD
doesn't the laptop consume more power?