With no staggered spinup enabled on your storage server, the pod boots up all 45 drives at once usually creating a beginning surge of 1000 - 1200 watts in current. We show you the steps on how to enable staggered spinup on the Storinator, allowing all 45 drives to start up individually, evening the power surge and maxing out at only 300-350 watts.
Andrey, Thanks for your comments - we love hearing from the community. You have a lot of great points here, so we’ll handle them as per your paragraphs: (1) In our case, our single non-redundant PSU is 850W and the triple redundant PSU is 950W. Both of these power supplies can handle surges up to 150% of rated load. (2) Any number of drives can be present to enable staggered spinup - provided that the drives support it. You can even boot with 15 drives (or any number) and then manually add the rest of them in at runtime if you wish. Simply refresh the GUI as new drives are added and you will be able to enable staggered spinup. (3) Great point. As long as the surge time is shorter than the maximum surge time provided by the PSU specs, we will be fine. The calculated 945W for 45 Drives upon start up is a fair assumption but it should be noted that the triple redundant PSU is rated 950W, so including front-end system power requirements we pull no more than 125% rated load at startup. From the power analysis of our pods, we saw that it pulled 125% rated power for no more than four seconds upon startup - well within the manufacturer’s specs. With that being said, if you’re using the single PSU (850W), while it will work on start up as it will pull ~140% rated power, it would recommend to enable staggered spinup on some drives to be less demanding of the PSU. (5) Your calculation is correct assuming a 650W PSU, but that is not the case for our pods now. In our opinion, a few startups at 125% or even 140% rated power will not be an issue for these PSUs, especially since in most cases these pods are booted once and left running for long periods of time. If you are really worried about the start-up power draw, you can always enable staggered spinup. If using the Rocket 750 cards, the only way to enable this is through their dedicated WebGUI. (6) You are correct. If a drive fails and you replace it, you would have to enable staggered spinup for that drive in the WebGUI. Once again, thank you for the feedback. If you have any more questions, please ask us.
The purpose of staggered spinup is two fold, reduce instantaneous load on the power supply and the load on the circuit breaker of the data center/rack/cabinet. If let's say we calculate that during a full read/write each disk consumes at most ~6.5W which is typical of most (7200/5400 RPM) drives, with 45 drives, we dedicate 300W to HDD power, the rest to the system and PSU inefficiency. We come up with a 650W PSU. The problem with in the video is the way the staggered spinup is setup through the web control interface of the Rocket 750 is that it looks like it requires all the intended drives to be populated before you could enable staggered spin up on each disk. This in part defeats the purpose and increases risk of PSU failure since it has to at least once power a large load due to an instantaneous spin up of all 45 drives before staggered spinup configuration could be set. I should note that although PSUs are capable of handling surges, it's not good for them and the risk of failure increases depending on the number of surges, the quality of PSU, and the rated continuous load relative to the surge load. A WD disk spec sheet indicated a peak of 1.75A @ 12V, or 21W, for an instantaneous spin up of 45 disks that's 945W. Unless there is an option at BIOS level before the OS to configure staggered spinup in Rocket 750 before the disks are spin up, the only way to ensure the load does not exceed the PSU continuous load specification is to add HDDs in groups of no more than 15. (650W*0.8 - 200W = 320W remaining power for HDDs) 200W for system boot including misc. 0.8 efficiency factor (80%) Without having a pod unit to play with, it remains unclear what happens to the staggered spinup configuration in Rocket 750 configuration when HDDs are replaced by new disks, such as after a HDD failure. One would assume that the configuration is per HDD ID, so that means staggered spinup would need to be reapplied to that drive in the config, but at least you won't be trying to power up more than 15 drives at the same time with a 650W PSU.
You have to use some thinking. The Rocket 750, like most RAID cards supports hot plug. So turn it on with no drives, plug in a few, configure, and plug in the next few. If for whatever reason your setup doesn't allow hot plug (a bad idea), install a few, configure, shut down, install the next couple, and so forth.
Damn there is a lot of noise in that room! I hope you don't have to work in that noise without hearing protection. Take care! You will have hearing issues in the future if you don't think about this stuff.
As you can imagine, the #storinators draw a whack of power on startup. Our R&D guys have done extensive testing on staggered spinup - different methods, which drives are compatible and more. Our testing's been documented on our wiki (bit.ly/1Cgh7m4) and Steve explains more in our video. But we want to hear what you have to say. Do the ***** fans have opinions on staggered spinup?
Can that controller/system work as JBOD, with every physical disk used by the system, so that ZFS can be used up to a full potential. Can one do that also in JBOD mode? (I suppose it's firmware setting to spin up gradually, but just to ask).
Hi mrjonnoma, the Rocket 750 are JBOD cards. Every physical disk can be independently accessed by the OS,. Unless a custom build,all our servers are JBOD. We do not recommend HW RAID under ZFS. So yes staggered spin up works in JBOD mode.
WOULD SOMEONE PLEASE EDIT OUT THAT ANNOYING WHINE--! Informative video, otherwise. Staggered spinup causes what looks like a several-minute delay in case of a cold boot; would like to see if spinning up 2x drives at once (one per controller?) would cut that time in half...
well, at least you aren't using seagate. Huge power draw or not, those will fail prematurely, leaving you with no data. Toshiba drives are just renamed Hitachi deskstars, which are owned by WD now. So basically WD deskstars.
With no staggered spinup enabled on your storage server, the pod boots up all 45 drives at once usually creating a beginning surge of 1000 - 1200 watts in current.
We show you the steps on how to enable staggered spinup on the Storinator, allowing all 45 drives to start up individually, evening the power surge and maxing out at only 300-350 watts.
Andrey,
Thanks for your comments - we love hearing from the community. You have a lot of great points here, so we’ll handle them as per your paragraphs:
(1) In our case, our single non-redundant PSU is 850W and the triple redundant PSU is 950W. Both of these power supplies can handle surges up to 150% of rated load.
(2) Any number of drives can be present to enable staggered spinup - provided that the drives support it. You can even boot with 15 drives (or any number) and then manually add the rest of them in at runtime if you wish. Simply refresh the GUI as new drives are added and you will be able to enable staggered spinup.
(3) Great point. As long as the surge time is shorter than the maximum surge time provided by the PSU specs, we will be fine. The calculated 945W for 45 Drives upon start up is a fair assumption but it should be noted that the triple redundant PSU is rated 950W, so including front-end system power requirements we pull no more than 125% rated load at startup. From the power analysis of our pods, we saw that it pulled 125% rated power for no more than four seconds upon startup - well within the manufacturer’s specs. With that being said, if you’re using the single PSU (850W), while it will work on start up as it will pull ~140% rated power, it would recommend to enable staggered spinup on some drives to be less demanding of the PSU.
(5) Your calculation is correct assuming a 650W PSU, but that is not the case for our pods now. In our opinion, a few startups at 125% or even 140% rated power will not be an issue for these PSUs, especially since in most cases these pods are booted once and left running for long periods of time. If you are really worried about the start-up power draw, you can always enable staggered spinup. If using the Rocket 750 cards, the only way to enable this is through their dedicated WebGUI.
(6) You are correct. If a drive fails and you replace it, you would have to enable staggered spinup for that drive in the WebGUI.
Once again, thank you for the feedback. If you have any more questions, please ask us.
Using the Kill a watt as an example was awesome! thank you
The purpose of staggered spinup is two fold, reduce instantaneous load on the power supply and the load on the circuit breaker of the data center/rack/cabinet.
If let's say we calculate that during a full read/write each disk consumes at most ~6.5W which is typical of most (7200/5400 RPM) drives, with 45 drives, we dedicate 300W to HDD power, the rest to the system and PSU inefficiency. We come up with a 650W PSU.
The problem with in the video is the way the staggered spinup is setup through the web control interface of the Rocket 750 is that it looks like it requires all the intended drives to be populated before you could enable staggered spin up on each disk. This in part defeats the purpose and increases risk of PSU failure since it has to at least once power a large load due to an instantaneous spin up of all 45 drives before staggered spinup configuration could be set.
I should note that although PSUs are capable of handling surges, it's not good for them and the risk of failure increases depending on the number of surges, the quality of PSU, and the rated continuous load relative to the surge load.
A WD disk spec sheet indicated a peak of 1.75A @ 12V, or 21W, for an instantaneous spin up of 45 disks that's 945W.
Unless there is an option at BIOS level before the OS to configure staggered spinup in Rocket 750 before the disks are spin up, the only way to ensure the load does not exceed the PSU continuous load specification is to add HDDs in groups of no more than 15.
(650W*0.8 - 200W = 320W remaining power for HDDs)
200W for system boot including misc.
0.8 efficiency factor (80%)
Without having a pod unit to play with, it remains unclear what happens to the staggered spinup configuration in Rocket 750 configuration when HDDs are replaced by new disks, such as after a HDD failure. One would assume that the configuration is per HDD ID, so that means staggered spinup would need to be reapplied to that drive in the config, but at least you won't be trying to power up more than 15 drives at the same time with a 650W PSU.
You have to use some thinking. The Rocket 750, like most RAID cards supports hot plug. So turn it on with no drives, plug in a few, configure, and plug in the next few. If for whatever reason your setup doesn't allow hot plug (a bad idea), install a few, configure, shut down, install the next couple, and so forth.
Damn there is a lot of noise in that room! I hope you don't have to work in that noise without hearing protection. Take care! You will have hearing issues in the future if you don't think about this stuff.
As you can imagine, the #storinators draw a whack of power on startup. Our R&D guys have done extensive testing on staggered spinup - different methods, which drives are compatible and more. Our testing's been documented on our wiki (bit.ly/1Cgh7m4) and Steve explains more in our video. But we want to hear what you have to say. Do the ***** fans have opinions on staggered spinup?
that is a very loud workspace!
Can that controller/system work as JBOD, with every physical disk used by the system, so that ZFS can be used up to a full potential.
Can one do that also in JBOD mode? (I suppose it's firmware setting to spin up gradually, but just to ask).
Hi mrjonnoma, the Rocket 750 are JBOD cards. Every physical disk can be independently accessed by the OS,. Unless a custom build,all our servers are JBOD. We do not recommend HW RAID under ZFS.
So yes staggered spin up works in JBOD mode.
Interestingly low power draw - What does that draw when booted up and all drives are doing writes as fast as they can?
Please,what is the site of your computer case ?
+vladakov4ik Please check us out at www.45drives.com - if you have any questions, feel free to contact us. Thanks!
Thanks for site :)
WOULD SOMEONE PLEASE EDIT OUT THAT ANNOYING WHINE--!
Informative video, otherwise. Staggered spinup causes what looks like a several-minute delay in case of a cold boot; would like to see if spinning up 2x drives at once (one per controller?) would cut that time in half...
This is very interesting.
well, at least you aren't using seagate. Huge power draw or not, those will fail prematurely, leaving you with no data. Toshiba drives are just renamed Hitachi deskstars, which are owned by WD now. So basically WD deskstars.