Absolutely brilliant video. Thanks for highlighting the delay function. I never even thought about how it would help my system from being slammed but the different phases or the final destination copy process.
if u have 3 HDD u can try add 3 Qeue with differents names ( def 1, def 2, def 3) and dont use same HDD in the final destination for evade the bottlenecking in copy, i run 4 plots in paralell without delay using this method and works for me. Nice video for sure
This has been helpful for me! Thanks for the info. I have just been parallel plotting multiple at a time because i have a 12/24 cpu. downside is that if you are busy at the time that they are done plotting then you are not plotting more until you que up more plots. I did not know about the thread requirements of the different phases, so that was good info. On a side note, the 7.51 hours total time should have been 7 hours and 30 minutes not 51 minutes, cuz decimals. But your point still stands.
I am glad you find useful info in this video! Yeah I think I made that mistake a couple times, saying the decimal in minutes rather than converting it, I knew it was gonna happen too!
Thanks for putting these out. I find them very helpful. Looking forward to your next post about plotting from the CLI. I have been running my plots from CLI for about 5 days now, works well but I don’t get plotting logs anymore. I hope you cover logging on your next video.
Don't forget you can also create Multiple Plot Queues. Option 2 in Plotting click "show Advanced Options" then if you want to create a second queue of plots that will run in parallel to a running plot just change the "Queue Name" to any other name and add some more plots to the queue. :)
While you are in the plotting GUI: Mouse over to >ACTION >the three dots>Log. It will open a CL window and show you a real-time log. Optimize there first, then deploy.. Pretty sure the Phase 1 plotting for all will be with a few percentage points. My Phase 1 are 164 minutes and I've been running them 8 mins (!) apart. No wonder I have crazy long transfers to the farming drive :-( I found all this out in large part by watching your videos. TY
Great Video. This is my first time touching powershell. Being able to loop concurrent parallel plot queues is what I'm looking for. I want the staggered start delay and be able to set up days/weeks/endless plotting. What i don't get is that adding another plotting queue causes them to both go at the same time, which isn't how a queue should work in my mind. The plotter needs some serious work. I'm not ready to dive into linux for the luxury of figuring out how to use Plotman which seems to have all the tools we really need for proper plotting.
He made a great plotting tool in powershell, look for his video "Chia Parallel Plotting Using Powershell (With Delay & Logs) BASICS - Chia Plotting Part 4"
I think I am missing something: You said that 3 delayed parallel plots takes 11 hours to finish, where as 3 no delay plots take 7 each, but as they all start together (cpu in overdrive) ) its 11 vs 7, so no delay looks better. Than you made a comparison table for the plots made in 24 hours, and showed that eventually, after 12th plot, delayed parallel will start to come ahead of plotting without delay. So far i'm on track right? Ok, here is my question, on your comparison table: You start at 5.47am and after just less than 6 hours (11:26) you already have 3 delayed plots? But 3 delayed plots takes 11 hours all to finish isn't it? I think you mixed the total time with time for one plot So in 11:26 column, there should be 1 plot delayed and at 4PM 3 finished delayed plots.. Am I right? Know that I am a bit thick skulled, so please go easy on me if I am missing something obvious.
I will have to go over the chart again but you could be right that I messed up somewhere when extending the time out, however the chart was just for a visual representation of what would eventually happen which is that the stagger plots will eventually out perform the ones without stagger in the long run.
I am building a new rig with 16cores, i am sure if i put all 14/15 plots without delay it will take significantly longer than 7 hours and probably, as your calculations showed, even longer than 30-60 delayed.. at least thats what i hope, otherwise whats the point 🤷🏼, Cheers
Congratulations great video, I was working into the same to make posible to create parallel in a 512GiB ssd and I had success but since I was looking to start after the disk space usage start to decrease, that happened after phase 2 adding 10 minutes more. Thanks again
Great timing, no pun intended. my attempt to create 2 plots at once on a 500gb nvme, by using the secondary temp drive option failed btw. no Bueno. Update : the same thing on my 4 plots to a 1tb nvme. I would love to learn what the purpose of the secondary temp is for and how it works.
I've found some interesting info looking in Chia CLI guide, (github.com/Chia-Network/chia-blockchain/wiki/CLI-Commands-Reference) says: Define a secondary temporary directory for plot creation. This is where Plotting Phase 3 (Compression) and Phase 4 (Checkpoints) occur. Depending on your OS, -2 might default to either -t or -d. Therefore, if either -t or -d are running low on space, it's recommended to set -2 manually. The -2 dir requires an equal amount of working space as the final size of the plot.
dude you are amazing! you got a new sub! I am just starting plotting my 8 TB system. I only have 1 SSD of 500 GB. Seems like it will take a while, maybe a month or two to plot everything. But I will look forward to your videos to learn with them better ways to pl be more efficient while plotting. Of course I can't plot in parallel with my SSD, but I have 4 HDD of 2 TB. Do you recommend that I start plotting in parallel with them? Using one or two as temp? My system is this: I7 5820k 16 GB of ram, Windows and 4 HDDs of 2 TB and 1 SSD of 500 GB. So far my first plot is at 15% in 2 hours, quite disappointing to be honest :( Thanks so much for making these videos I will look forward to more
Hello! I like the way you improve your little written software. You already can gather all information with one line of code. I hope you will make Patreon page were i can donate you for automatic tool that seeds your hdd with most efficent way with one click.
Your right on the final copy, normally it takes 165 seconds to do the final copy for me 4 drive raid0 pretty quick. but 2 of my plots are finishing at the same time and its been over 15 minutes and they are still copying... thanks for the tip. I will delay for now on its worth it. I was watching task manager on my 4x raid0 final storage location and the write speed was only hitting 198mb/s. when one plot finished copying it shot up to almost 900mb a asecond to finish the 2nd one.
Great Video! I'm IT for many years, and this was great info. Just starting to plot myself so I'm a novice in this. Delay for copy phase during the end....?, would it make sense to delay plots by 15 to 20 minutes. your copy phase was 13 minutes so copy would be complete or almost complete per plot allowing for finishing time to improve? I'm I thinking right? Anyway, thanks for great info.
OMG, im pretty ok with powershell, but just by watching this, i learn a lot more on some powershell usage which is not shared in powershell training such as "measure", "select -first #"
Thumbs Up, thank you for all tutorials , i wanted to ask you if i am doing 15 plots in Parallels (as example) can i set the Final Directory on a network share (doing this in a 10gb network) ?
Thanks for the video! Very imformative. Can you please tell me the break down of phases. What resource is each phase using? Following this video I am running 3 in parallel with a further 3 starting just after phase 2 starts however the first 3 sit at 31% and dont progress whilst the next 3 carry on up. Ubuntu Ryzen 9 3950x DDR4 3600 (mobo overclocked as only supports 3200) 2TB NVME doing 8 threads and 4650mb per plot. Your video says once phase 2 hits you are only using one thread for that plot, so 32 threads, I should now be using 27 threads with the 6 plots running?
Brilliant ! I just built my Chia rig, has 2TB nvme,32gb ram and 5.2Ghz i9-10850k , been trying to get some use out of the 2TB as it finishes plots so fast I only use 600GB with plots queued with 95 mins, should I start another parallel plot ?
Exactly my toughts. U just want to let the next plot in after your cpu is no more at 100%. I have a server with 32 threads and I think I will average those times
Hi! On your video, you mentioned discussing the following tip/tweak: When plotting in parallel, a command or way to auto start new plots once one is complete, so you always have the total number of plots that can be plotting. For example, if you can plot 3 in parallel, as your system is, you star Plot 1, then delay Plot 2 & Plot 3. Well, once Plot 1 completes, Plot 2 & Plot 3 are still running, but you could have a new plot starting to replace the completed Plot 1. Did you figure out how to achieve this, or do you just need to wait for all 3 to complete, and then start a new plot procedure for 3 new plots?
Yes I have another video on who to do this, but I feel like I explained it poorly. Basically you setup 3 different queue with different names and manually start them with the delay you want by waiting. I will also release a video tomorrow on how to do this powershell.
@@SysDeskDev Great, I'll look for the video. Just to through it out there and get your thoughts. So, I've just built the following, Ryzen 7 5800 (8c,16th), 64gb DDRAM, x2 Sabrent SDD 2TB 3600. So, in RAID 0, I should be able to parallel 14 plots, right? With say, 1hr delay. So may around 28 plots/day?
Thanks for the videos! I'm thinking your module will not work if you've only plotted from CLI because it doesn't seem to appear to create any log files that I can find when you plot by CLI and running the module throws a "Get-ChildItem : Cannot find path 'C:\Users\username\.chia\mainnet\plotter\' because it does not exist." I'm still poking around, but I'm wondering if you have to run a bunch of plots in the GUI for the module to work correctly.
Great video. But, you're talking CPU optimization in this video. I did the same optimization like your with accounting the whole plotting including the transfer time (plotting drive to farming drive). Based on my calculations I could do 3 plotting in a 1TB NVME SSD with around 120 - 130 mins delay.
I also talk about the transfer time in this video and I ended up concluding that was the real bottleneck and not the CPU in phase 2. Did you watch the whole video or just the first part?
Hiya, in this video you have demonstarted difference in completion of 0 delay and next start after phase 1 is complete. What if you create another parallel plot of start the next plot when the first plot is 70% in progress. Would this be faster than the original start next plot when plot 1 is done? THank you for the awesome video :D next to avoid is getting them to finish at the same time!
Thanks for the statistics script. I determined I was wasting too much time in the copy from the Temp drive to the Final drive (HDD), ~2.5 hrs. I added another SSD as an intermediary Final drive now my copy times are down to ~1.6 minutes. When I finish copying to the intermediary drive I turn around and copy to the final HDD. Then I can immediately start plotting again because I won't be using the Final (Intermediary) drive until the end of the plot. So, I can copy the previous plots off and farm new ones at the same time.
hey, amazing video, let me allow one question, why do you not allocate all threads you have on your cpu to phase one of all plots? it would reduce the plot time i guess and then your cpu would not be overloaded. or am i wrong
seems to me like you want a 15 minute stager so that each copy isn't happening at the same time. i've been running then as ques but I make the que at start at different times (i made 11 today so far but i'm going to set up a raid 0 once ssd's 4 and 5 arrive soon so i'll have to redo the maths) That way you can could have 57 total in the que with your system (I'm hoping for 20 in parallel next week on a 16 core)
Hi there short question...you said you should make a few plods to get an average on the time per Plot - BUT: when we make more plods at the same time the average (for 1 Plot) goes up by alot (of course the average time per plods goes still down). So how to meassure the average time per plods by max amout of plods in queue?
Wow.Thank you for this content its 100%++ helpful! I have a question. What is your Farm HDD? will this is the main reason why transferring file its take so much time?
Well generally your plots will be stored on HDDs (Hard disk drive) which have slower write speeds than SSDs so that is why it is taking up to 15 minutes. My farm HHD that I was transferring to was an external HDD by Seagate.
You will have to use Format-Table -Property prop1,prop2,prop3, etc to create your own format table view instead of using the default one I created. so Get-ChiaPlottingStatistic | Format-Table -Property Time_Started,KSize,etc or Get-ChiaPlottingStatistic | Format-Table -Property * to list all properties. You can use the -path parameter to specify a different path to the log file. ex. Get-ChiaPlottingStatistic -Path C:\ChiaLogs\Chialog1.log
@@SysDeskDev Wow i was not expecting such a quick reply. I already started adding the date to the switch -Wildcard ($line){ and then i was going to modify the table... wonder if i get lucky and can pull it off. I see that Get-ChiaPlottingStatistic | Format-Table -Property * has the creation time at the end but the time part is cut off early.
Ahh i downloaded the zip and was trying to edit the scrips but i cant seem to run it from there. So that doesn't seem to work for me. I have to move the logs into the Dir that it wants to look at. Its too bad i cant get the format simplified to hours like you set it but it will work. thanks!
Thank you for your value Video. I was wondering, if i have 3 SSD's for plotting installed whoch paramters should i choose? Or have i start start the gui 3 times?
Saludos desde Ecuador, es de los mejores aportes que he visto. Porque no realizaste las pruebas también con Linux? Sabes si es cierto que 20% más eficiente? Excuseme, do you know if linux is 20% more efficient? Great job
Hey men. Just wanted to give you a big THANK YOU for your work.... secondly I have a question: When im trying to import pschiaplotter to my ps with this command: "Import-Module PSChiaPlotter" it says that it could not be importet because importing is deactivated on my win system.... can you maybe help me with that? Kind Regards..
I had the same issue. Run Get-ExecutionPolicy. If it says Restricted then you have to run this command: Set-ExecutionPolicy -ExecutionPolicy RemoteSigned and select Yes if you feel it's a safe script. After that, it should run.
Great job. One question, why not to modify a script for delay based on 1st phase finish time. In this case there is no need to calculate on aproximate time of 1st fase. Script will see that it's over and start next session. Is it possible?
Say I'm limited by a 1 TB SSD so I can only run 3 plots in parallel as in your example. Then say that I what to create 15 plots. Should I set the delay to the time the phase 1 takes or to a third of the total time for a plot? I see a scenario where the phase 1 time is less then a third of the total time and it will try to start the forth plot and by that fill up the SSD. Or maybe it checks so that the free space is enough before starting a plot and by that way will avoid the problem?
I'm wondering how many threads should use, I have 11900k 32GB and 3TB NVme drive. I do 5 in parallel plots use 4 thread, should I use 2 thread and do 8 in parallel plots? Thanks
Is that an 8 core 16 thread processor? If so you have enough RAM and CPU power, and Plotting space on your NVMe's to do 8 parallel plots. I would opt for doing 8 in parallel over 5 for sure. I would start off with 2 threads for each plot and then try out 4 if you want to squeeze more out of your processor. The only way to find out for each computer is to test it out.
In the minute 4:14 you show a k32 plot created with 6780 of RAM and 4 Threads. In the next line you are using 12000 of RAM and 2 threads and it uses more or same time in each phase. Then, why both k32 plots finish after 5,34 hrs?
So at that time the command had a bug in that I didn't clear the variables at the end of each loop, so if the next plot didn't finish it would insert the value from the last plot log. It is now fixed. I had a lot of plot logs that didn't finish since I had RAM issues in the beginning.
I do want to make content on Ubuntu, and all my harvesters are Ubuntu. Hopefully I can get there eventually, I do this mostly in my spare time and wasn't expecting to make so many videos. People seem to find them helpful so I kept making them.
@@SysDeskDev for someone like me that is not a computer expert, you are a great help. I'm farming on windows and I hope you can teach us how to do it with ubuntu. Many thanks for your time in making these videos 🤗
@@SysDeskDev Thanks for the amazing content! I'm looking forward to the Ubuntu version! I just got myself a plotting rig but I'm not very skilled with coding. Thumbs up!
I'm #69 TU :) After Viewing all your vids in this series, I have one lingering question. How can I fine the optimal # of plots in parallel (or timed sequence parallel) for my given SSD vs NVME drive. I heard you mutter a number in one of your past vids, but how do we find this number for the specific drive we have??
If you are just looking at your SSD then you will have to divide the capacity of the SSD by 256.6 (which the temp space in GB which is the unit SSD's capacity is measured in) and round down. Technically you might be able squeeze another parallel plot in if you time the delay just right, but it is important to remember that for many SSD their performance is worsen the more filled up they are. Make sure to check your RAM and CPU to find your bottleneck as your SSD (depending on the size) might not be your limiting factor. I hope I explained it okay.
@@SysDeskDev I found some info in PS online proccess at the end of the plotting phase: 269.319 GiB was used total (excl. full file) in the plotting - this statistics shows 1 plotting process eats 288,8365507 Gbytes of space, not 256. It means that 1TB nvme = 3 Plotting and 2TB = 6 maxed To be sure. Am i right?
my ryzen 5950x is doing 2 plots for 2 phases in parallel. used 16 threads per plot and 3800mb ram. (should i put next time max threads to 32?) Here are the results> phase 1 avg. - 6425s phase 2 avg. - 3346s So i tried to use Patro approach from video u suggested which means > phase 1 +phase 2/60=163 min so how do i calculate now the real delay time to set? 6 plots for 2tb nvme> 163/6? or did i miss smth important? That means 28 min delay for each plot? considering that i have now 2tb nvme and planning to go to 4x2tb in asus hyper m.2 gen4 ... How can i manage then this maths?:D
What happens if you use your max threads for the plots? so the first plot will use all threads *lets say 12* for phase 1, then when it's complete it moves down to 1 thread in phase 2. Then your second plot starts up, will this second plot try and use all 12 threads or will it only use 11 seeing how your first plot is now using 1 thread? Thanks. My idea here is to power throught phase 1 as fast as possible. Great video by the way, I think i learned more watching this then any other video ive seen, im probably gonna watch it a 2nd time too! Also could you do a video or share some insight on your nvme/ fast drive storage use? is there different phases that use up more space then others? and is there a way to add that into the equation to get even more plots. Thanks.
you dont get any gains by giving it more than 4 threads and 6750GB per plot. At least thats what i understood. It then depends by the clock speed of you cpu and the model.
Friend firstly thanks for video. I got 2x raid 0 corsair Mp510 960GB---> Total 1920GB , AMD 4650G (12 threads)Cpu and 32GB ram. Which one must i use? One by one with delay or parallel plotting how many? TY.
Download and install Teracopy....set that as your default copy/paste/move application. Then use threadsin a python program to do the plot and then use the join method to then fire off another thread to do the move command to the plot's final home. KEEP your -2 = -d directories the same.
If you make a loop mechanic to keep the plotter at 100% efficiency That Would Be amazing. Also can you make a short video of how you start making plots your way (:
Get-ChiaPlottingStatistic : The 'Get-ChiaPlottingStatistic' command was found in the module 'PSChiaPlotter', but the mo dule could not be loaded. For more information, run 'Import-Module PSChiaPlotter'. At line:1 char:1
Try changing the execution policy by running Set-ExecutionPolicy -ExecutionPolicy RemoteSigned or if that doesn't work Set-ExecutionPolicy -ExecutionPolicy ByPass
I just tested file transfers between HDD's ... it looks like transfer times are unaffected as long different drives are used for both read and write as well.
Help with the amount of threads to put on per plot... I have a 8-core 16 thread. Some videos say utilize all of them others say 2 .I think there's some real confusion.
Experiment and see which gives the best result, For me 4 threads and 3408 memory help completed phase 1 in about 2 hours, but i will be staggering them so will try 2 threads also.
Great question! I did want to focus only on the delay feature in this video so that people understand each part of plotting and not change more than one variable to give reliable results. I do plan to cover changing the thread count as well and eventually tie everything to together in recap video. To answer your questions, I think using more than 2 thread (at this time since it is only used in phase although this could change) will decrease over all plot time if you staggered them correctly but just how many thread really varies on a per system basis.
Can we have multiple computer to farm chia within different network (not in the same local network) but having the same chia wallet or harvesting address?
The best chia explanatory video... not for amateurs..
This content is literally gold. Thanks for taking the time to try different variables. I'm not even mining but have been considering using a MBP 2015.
Your videos are amazing. The technical depth on Chia is unparalleled. Thank you.
Wow, thank you!
Absolutely brilliant video. Thanks for highlighting the delay function. I never even thought about how it would help my system from being slammed but the different phases or the final destination copy process.
if u have 3 HDD u can try add 3 Qeue with differents names ( def 1, def 2, def 3) and dont use same HDD in the final destination for evade the bottlenecking in copy, i run 4 plots in paralell without delay using this method and works for me. Nice video for sure
Always a great value to have details tutorials!
Thank you! Keep up the great work on your channel as well!
great video !
Thank you!
This has been helpful for me! Thanks for the info. I have just been parallel plotting multiple at a time because i have a 12/24 cpu. downside is that if you are busy at the time that they are done plotting then you are not plotting more until you que up more plots. I did not know about the thread requirements of the different phases, so that was good info. On a side note, the 7.51 hours total time should have been 7 hours and 30 minutes not 51 minutes, cuz decimals. But your point still stands.
I am glad you find useful info in this video! Yeah I think I made that mistake a couple times, saying the decimal in minutes rather than converting it, I knew it was gonna happen too!
Thanks for putting these out. I find them very helpful. Looking forward to your next post about plotting from the CLI. I have been running my plots from CLI for about 5 days now, works well but I don’t get plotting logs anymore. I hope you cover logging on your next video.
Don't forget you can also create Multiple Plot Queues. Option 2 in Plotting click "show Advanced Options" then if you want to create a second queue of plots that will run in parallel to a running plot just change the "Queue Name" to any other name and add some more plots to the queue. :)
While you are in the plotting GUI: Mouse over to >ACTION >the three dots>Log. It will open a CL window and show you a real-time log. Optimize there first, then deploy.. Pretty sure the Phase 1 plotting for all will be with a few percentage points. My Phase 1 are 164 minutes and I've been running them 8 mins (!) apart. No wonder I have crazy long transfers to the farming drive :-( I found all this out in large part by watching your videos. TY
Great Video. This is my first time touching powershell. Being able to loop concurrent parallel plot queues is what I'm looking for. I want the staggered start delay and be able to set up days/weeks/endless plotting. What i don't get is that adding another plotting queue causes them to both go at the same time, which isn't how a queue should work in my mind. The plotter needs some serious work. I'm not ready to dive into linux for the luxury of figuring out how to use Plotman which seems to have all the tools we really need for proper plotting.
He made a great plotting tool in powershell, look for his video "Chia Parallel Plotting Using Powershell (With Delay & Logs) BASICS - Chia Plotting Part 4"
best video ever about the theme
This was excellent. Thanks so much for taking the time to make this!
Glad it was helpful!
This is the content i am looking for! Thanks!
I think I am missing something:
You said that 3 delayed parallel plots takes 11 hours to finish, where as 3 no delay plots take 7 each, but as they all start together (cpu in overdrive) ) its 11 vs 7, so no delay looks better.
Than you made a comparison table for the plots made in 24 hours, and showed that eventually, after 12th plot, delayed parallel will start to come ahead of plotting without delay.
So far i'm on track right?
Ok, here is my question, on your comparison table: You start at 5.47am and after just less than 6 hours (11:26) you already have 3 delayed plots? But 3 delayed plots takes 11 hours all to finish isn't it?
I think you mixed the total time with time for one plot So in 11:26 column, there should be 1 plot delayed and at 4PM 3 finished delayed plots..
Am I right?
Know that I am a bit thick skulled, so please go easy on me if I am missing something obvious.
I will have to go over the chart again but you could be right that I messed up somewhere when extending the time out, however the chart was just for a visual representation of what would eventually happen which is that the stagger plots will eventually out perform the ones without stagger in the long run.
I am building a new rig with 16cores, i am sure if i put all 14/15 plots without delay it will take significantly longer than 7 hours and probably, as your calculations showed, even longer than 30-60 delayed.. at least thats what i hope, otherwise whats the point 🤷🏼,
Cheers
Dude, you just solved all my chia problems :) this video series is just top top work. Cheers from Portugal
Great to hear!
cannot wait for the next one you mentioned showing parallel PowerShell. fantastic series!
Thank you! I plan on releasing it tomorrow!
Congratulations great video, I was working into the same to make posible to create parallel in a 512GiB ssd and I had success but since I was looking to start after the disk space usage start to decrease, that happened after phase 2 adding 10 minutes more. Thanks again
Thank you kind human for your videos.
Great timing, no pun intended. my attempt to create 2 plots at once on a 500gb nvme, by using the secondary temp drive option failed btw. no Bueno.
Update : the same thing on my 4 plots to a 1tb nvme. I would love to learn what the purpose of the secondary temp is for and how it works.
I've found some interesting info looking in Chia CLI guide, (github.com/Chia-Network/chia-blockchain/wiki/CLI-Commands-Reference) says:
Define a secondary temporary directory for plot creation. This is where Plotting Phase 3 (Compression) and Phase 4 (Checkpoints) occur. Depending on your OS, -2 might default to either -t or -d. Therefore, if either -t or -d are running low on space, it's recommended to set -2 manually. The -2 dir requires an equal amount of working space as the final size of the plot.
dude you are amazing! you got a new sub! I am just starting plotting my 8 TB system. I only have 1 SSD of 500 GB. Seems like it will take a while, maybe a month or two to plot everything. But I will look forward to your videos to learn with them better ways to pl be more efficient while plotting.
Of course I can't plot in parallel with my SSD, but I have 4 HDD of 2 TB. Do you recommend that I start plotting in parallel with them? Using one or two as temp?
My system is this: I7 5820k 16 GB of ram, Windows and 4 HDDs of 2 TB and 1 SSD of 500 GB. So far my first plot is at 15% in 2 hours, quite disappointing to be honest :(
Thanks so much for making these videos I will look forward to more
Great job, you put this module together pretty quickly
this is pretty awesome utility! thank you! makes optimizing so much easier.
waiting for a next video about it, thanks!!!
this video was created really to teach , not for more views ! Appreciate !
Hello! I like the way you improve your little written software. You already can gather all information with one line of code. I hope you will make Patreon page were i can donate you for automatic tool that seeds your hdd with most efficent way with one click.
Your right on the final copy, normally it takes 165 seconds to do the final copy for me 4 drive raid0 pretty quick. but 2 of my plots are finishing at the same time and its been over 15 minutes and they are still copying... thanks for the tip. I will delay for now on its worth it. I was watching task manager on my 4x raid0 final storage location and the write speed was only hitting 198mb/s. when one plot finished copying it shot up to almost 900mb a asecond to finish the 2nd one.
Great Video! I'm IT for many years, and this was great info. Just starting to plot myself so I'm a novice in this. Delay for copy phase during the end....?, would it make sense to delay plots by 15 to 20 minutes. your copy phase was 13 minutes so copy would be complete or almost complete per plot allowing for finishing time to improve? I'm I thinking right? Anyway, thanks for great info.
Thank for sharing this 🤝 You’re amazing bro. Keep it up!!! 👍🏻👍🏻👍🏻👍🏻
Thanks for the visit
Great video!! Very useful!
Glad it was helpful!
Just finished the video and can't thank you enough for making these tutorials and creating the module for us to use. Where's the donate link🙂
Just being able to help other's is enough for me :)
Thanks man! Really appreciate this. Keep doing it!
Thank you for the insight! .... my phase one completes in 77 minutes after some tweaking.
OMG, im pretty ok with powershell, but just by watching this, i learn a lot more on some powershell usage which is not shared in powershell training such as "measure", "select -first #"
Teaching people powershell indirectly using Chia has been the biggest reward making these videos, so I am very happy to hear this :)
Very nice Tool you wrote there. But i needed to exec "Set-PSRepository -Name PSGallery -InstallationPolicy Trusted" in PS to utilize it.
great video, thank you very much, great knowledge
Thanks a lot for making that module, It helped a lot!
Great content, thanks for sharing this!
Very useful information, thank you!
this is a great video! Keep it up!
Thank you so much!
Exelent Explanation, really appriciate that!
Thank you!
nice analysis! any test on the RAM usage? using the staggered trick to plotting more in parallel?
Thumbs Up, thank you for all tutorials , i wanted to ask you if i am doing 15 plots in Parallels (as example) can i set the Final Directory on a network share (doing this in a 10gb network) ?
Thanks for the video! Very imformative. Can you please tell me the break down of phases. What resource is each phase using? Following this video I am running 3 in parallel with a further 3 starting just after phase 2 starts however the first 3 sit at 31% and dont progress whilst the next 3 carry on up. Ubuntu Ryzen 9 3950x DDR4 3600 (mobo overclocked as only supports 3200) 2TB NVME doing 8 threads and 4650mb per plot. Your video says once phase 2 hits you are only using one thread for that plot, so 32 threads, I should now be using 27 threads with the 6 plots running?
Hey, great stuff obviously. Any idea if I can use this to read Swar logs??
Brilliant ! I just built my Chia rig, has 2TB nvme,32gb ram and 5.2Ghz i9-10850k , been trying to get some use out of the 2TB as it finishes plots so fast I only use 600GB with plots queued with 95 mins, should I start another parallel plot ?
Would you actually want to delay by just the average. Wouldn't the highest value make more sense to ensure the phase 1s don't overlap?
Exactly my toughts. U just want to let the next plot in after your cpu is no more at 100%. I have a server with 32 threads and I think I will average those times
Hi! On your video, you mentioned discussing the following tip/tweak: When plotting in parallel, a command or way to auto start new plots once one is complete, so you always have the total number of plots that can be plotting. For example, if you can plot 3 in parallel, as your system is, you star Plot 1, then delay Plot 2 & Plot 3. Well, once Plot 1 completes, Plot 2 & Plot 3 are still running, but you could have a new plot starting to replace the completed Plot 1. Did you figure out how to achieve this, or do you just need to wait for all 3 to complete, and then start a new plot procedure for 3 new plots?
Yes I have another video on who to do this, but I feel like I explained it poorly. Basically you setup 3 different queue with different names and manually start them with the delay you want by waiting. I will also release a video tomorrow on how to do this powershell.
@@SysDeskDev Great, I'll look for the video. Just to through it out there and get your thoughts. So, I've just built the following, Ryzen 7 5800 (8c,16th), 64gb DDRAM, x2 Sabrent SDD 2TB 3600. So, in RAID 0, I should be able to parallel 14 plots, right? With say, 1hr delay. So may around 28 plots/day?
Thanks for the videos! I'm thinking your module will not work if you've only plotted from CLI because it doesn't seem to appear to create any log files that I can find when you plot by CLI and running the module throws a "Get-ChildItem : Cannot find path 'C:\Users\username\.chia\mainnet\plotter\' because it does not exist." I'm still poking around, but I'm wondering if you have to run a bunch of plots in the GUI for the module to work correctly.
Great video.
But, you're talking CPU optimization in this video.
I did the same optimization like your with accounting the whole plotting including the transfer time (plotting drive to farming drive).
Based on my calculations I could do 3 plotting in a 1TB NVME SSD with around 120 - 130 mins delay.
I also talk about the transfer time in this video and I ended up concluding that was the real bottleneck and not the CPU in phase 2. Did you watch the whole video or just the first part?
Thanks for the great video, mate! 👍🏼👍🏼👍🏼
Hiya, in this video you have demonstarted difference in completion of 0 delay and next start after phase 1 is complete. What if you create another parallel plot of start the next plot when the first plot is 70% in progress. Would this be faster than the original start next plot when plot 1 is done? THank you for the awesome video :D next to avoid is getting them to finish at the same time!
Thanks for the statistics script. I determined I was wasting too much time in the copy from the Temp drive to the Final drive (HDD), ~2.5 hrs. I added another SSD as an intermediary Final drive now my copy times are down to ~1.6 minutes. When I finish copying to the intermediary drive I turn around and copy to the final HDD. Then I can immediately start plotting again because I won't be using the Final (Intermediary) drive until the end of the plot. So, I can copy the previous plots off and farm new ones at the same time.
Hello, which command should I use to get a average value of total plot time ?
p.s. Great video !!!
hey, amazing video, let me allow one question, why do you not allocate all threads you have on your cpu to phase one of all plots? it would reduce the plot time i guess and then your cpu would not be overloaded. or am i wrong
seems to me like you want a 15 minute stager so that each copy isn't happening at the same time.
i've been running then as ques but I make the que at start at different times (i made 11 today so far but i'm going to set up a raid 0 once ssd's 4 and 5 arrive soon so i'll have to redo the maths)
That way you can could have 57 total in the que with your system (I'm hoping for 20 in parallel next week on a 16 core)
Thank you very much for the knowledge that is invaluable to me. :D
Hi there short question...you said you should make a few plods to get an average on the time per Plot - BUT: when we make more plods at the same time the average (for 1 Plot) goes up by alot (of course the average time per plods goes still down). So how to meassure the average time per plods by max amout of plods in queue?
great video! thanks for sharing
Thanks for watching!
GREAT VIDEO! How can we run that shell command in Ubuntu? Does it work with Ubuntu or only Windows?
Wow.Thank you for this content its 100%++ helpful! I have a question. What is your Farm HDD? will this is the main reason why transferring file its take so much time?
Well generally your plots will be stored on HDDs (Hard disk drive) which have slower write speeds than SSDs so that is why it is taking up to 15 minutes. My farm HHD that I was transferring to was an external HDD by Seagate.
@@SysDeskDev Thank you very much :)
Great stuff, im learning a lot but how do you add a date and time started to the columns? Also can i choose the log directory?
You will have to use Format-Table -Property prop1,prop2,prop3, etc to create your own format table view instead of using the default one I created.
so Get-ChiaPlottingStatistic | Format-Table -Property Time_Started,KSize,etc
or Get-ChiaPlottingStatistic | Format-Table -Property *
to list all properties.
You can use the -path parameter to specify a different path to the log file.
ex. Get-ChiaPlottingStatistic -Path C:\ChiaLogs\Chialog1.log
@@SysDeskDev Wow i was not expecting such a quick reply. I already started adding the date to the switch -Wildcard ($line){ and then i was going to modify the table... wonder if i get lucky and can pull it off.
I see that Get-ChiaPlottingStatistic | Format-Table -Property * has the creation time at the end but the time part is cut off early.
Ahh i downloaded the zip and was trying to edit the scrips but i cant seem to run it from there. So that doesn't seem to work for me. I have to move the logs into the Dir that it wants to look at. Its too bad i cant get the format simplified to hours like you set it but it will work. thanks!
Thank you for your value Video. I was wondering, if i have 3 SSD's for plotting installed whoch paramters should i choose? Or have i start start the gui 3 times?
Saludos desde Ecuador, es de los mejores aportes que he visto. Porque no realizaste las pruebas también con Linux? Sabes si es cierto que 20% más eficiente? Excuseme, do you know if linux is 20% more efficient? Great job
Brilliant app, thank you so much👍🏾
So that means its not necessary 1TB SSD , 500GB ssd enough for that(I mean unless we run it at the same time)? We can add plot in parallel with delay
Hey men. Just wanted to give you a big THANK YOU for your work....
secondly I have a question: When im trying to import pschiaplotter to my ps with this command: "Import-Module PSChiaPlotter" it says that it could not be importet because importing is deactivated on my win system.... can you maybe help me with that?
Kind Regards..
I had the same issue. Run Get-ExecutionPolicy. If it says Restricted then you have to run this command: Set-ExecutionPolicy -ExecutionPolicy RemoteSigned and select Yes if you feel it's a safe script. After that, it should run.
@@raicard26 Thanks a lot. that helped me :)
Very informative video
Yo This is GOLD
Great job. One question, why not to modify a script for delay based on 1st phase finish time. In this case there is no need to calculate on aproximate time of 1st fase. Script will see that it's over and start next session. Is it possible?
Definitely possible, I do want to add that to the script but just haven't found the time.
@@SysDeskDev it would be great.
Say I'm limited by a 1 TB SSD so I can only run 3 plots in parallel as in your example. Then say that I what to create 15 plots. Should I set the delay to the time the phase 1 takes or to a third of the total time for a plot? I see a scenario where the phase 1 time is less then a third of the total time and it will try to start the forth plot and by that fill up the SSD. Or maybe it checks so that the free space is enough before starting a plot and by that way will avoid the problem?
thanks my dude champion
Genius ! Thank you so much
I'm wondering how many threads should use, I have 11900k 32GB and 3TB NVme drive. I do 5 in parallel plots use 4 thread, should I use 2 thread and do 8 in parallel plots? Thanks
Is that an 8 core 16 thread processor? If so you have enough RAM and CPU power, and Plotting space on your NVMe's to do 8 parallel plots. I would opt for doing 8 in parallel over 5 for sure. I would start off with 2 threads for each plot and then try out 4 if you want to squeeze more out of your processor. The only way to find out for each computer is to test it out.
thanks a lot for your videos, it really helps...
excellent! thank you very much
Thank you for your videos
In the minute 4:14 you show a k32 plot created with 6780 of RAM and 4 Threads. In the next line you are using 12000 of RAM and 2 threads and it uses more or same time in each phase. Then, why both k32 plots finish after 5,34 hrs?
So at that time the command had a bug in that I didn't clear the variables at the end of each loop, so if the next plot didn't finish it would insert the value from the last plot log. It is now fixed. I had a lot of plot logs that didn't finish since I had RAM issues in the beginning.
Is there SSD capacity to consider when plotting more than 1 plot as shown in your video?
your plotting drive should be twice the size of the maximum amount of plots you plan to run in parallel
good analysis👍
Thumbs up. I wish you'd make some content for Ubuntu as well 🤞
I do want to make content on Ubuntu, and all my harvesters are Ubuntu. Hopefully I can get there eventually, I do this mostly in my spare time and wasn't expecting to make so many videos. People seem to find them helpful so I kept making them.
@@SysDeskDev for someone like me that is not a computer expert, you are a great help. I'm farming on windows and I hope you can teach us how to do it with ubuntu. Many thanks for your time in making these videos 🤗
@@SysDeskDev Thanks for the amazing content! I'm looking forward to the Ubuntu version! I just got myself a plotting rig but I'm not very skilled with coding. Thumbs up!
@@SysDeskDev Please make them for UBUNTU.
I'm #69 TU :) After Viewing all your vids in this series, I have one lingering question. How can I fine the optimal # of plots in parallel (or timed sequence parallel) for my given SSD vs NVME drive. I heard you mutter a number in one of your past vids, but how do we find this number for the specific drive we have??
If you are just looking at your SSD then you will have to divide the capacity of the SSD by 256.6 (which the temp space in GB which is the unit SSD's capacity is measured in) and round down. Technically you might be able squeeze another parallel plot in if you time the delay just right, but it is important to remember that for many SSD their performance is worsen the more filled up they are. Make sure to check your RAM and CPU to find your bottleneck as your SSD (depending on the size) might not be your limiting factor. I hope I explained it okay.
@@SysDeskDev I found some info in PS online proccess at the end of the plotting phase:
269.319 GiB was used total (excl. full file) in the plotting - this statistics shows 1 plotting process eats 288,8365507 Gbytes of space, not 256.
It means that 1TB nvme = 3 Plotting
and 2TB = 6 maxed
To be sure.
Am i right?
my ryzen 5950x is doing 2 plots for 2 phases in parallel. used 16 threads per plot and 3800mb ram. (should i put next time max threads to 32?)
Here are the results>
phase 1 avg. - 6425s
phase 2 avg. - 3346s
So i tried to use Patro approach from video u suggested which means >
phase 1 +phase 2/60=163 min
so how do i calculate now the real delay time to set?
6 plots for 2tb nvme> 163/6? or did i miss smth important?
That means 28 min delay for each plot?
considering that i have now 2tb nvme and planning to go to 4x2tb in asus hyper m.2 gen4 ... How can i manage then this maths?:D
Which pc setup u plotting?
Nvme or ssd? Could u give me that info?:) thank you, thx for the videos.
Great video!!
great content! with a 7700k 32gb ram 4tb m.2, what do u suggest me?
What happens if you use your max threads for the plots? so the first plot will use all threads *lets say 12* for phase 1, then when it's complete it moves down to 1 thread in phase 2. Then your second plot starts up, will this second plot try and use all 12 threads or will it only use 11 seeing how your first plot is now using 1 thread? Thanks. My idea here is to power throught phase 1 as fast as possible. Great video by the way, I think i learned more watching this then any other video ive seen, im probably gonna watch it a 2nd time too! Also could you do a video or share some insight on your nvme/ fast drive storage use? is there different phases that use up more space then others? and is there a way to add that into the equation to get even more plots. Thanks.
you dont get any gains by giving it more than 4 threads and 6750GB per plot. At least thats what i understood. It then depends by the clock speed of you cpu and the model.
Very good program!!
How come when I start my powershell it defaults to PC>C:\Users\[username]? Why does yours only shows PS>?
is the same. Just do it what he said.
Friend firstly thanks for video. I got 2x raid 0 corsair Mp510 960GB---> Total 1920GB , AMD 4650G (12 threads)Cpu and 32GB ram. Which one must i use? One by one with delay or parallel plotting how many? TY.
Download and install Teracopy....set that as your default copy/paste/move application. Then use threadsin a python program to do the plot and then use the join method to then fire off another thread to do the move command to the plot's final home. KEEP your -2 = -d directories the same.
If you make a loop mechanic to keep the plotter at 100% efficiency That Would Be amazing. Also can you make a short video of how you start making plots your way (:
You can already do that with a plot manager like swar
I'm currently plotting with 0 delay to temporary sata SSD then coping the plots to big HDD. My copy time is 5-15mins for a plot.
Helpful as F DUDE, thanks!
Get-ChiaPlottingStatistic : The 'Get-ChiaPlottingStatistic' command was found in the module 'PSChiaPlotter', but the mo
dule could not be loaded. For more information, run 'Import-Module PSChiaPlotter'.
At line:1 char:1
Gotta run Set-ExecutionPolicy -ExecutionPolicy RemoteSigned
@@SysDeskDev ohh it worked, thank you sir
Hello, I'm unable to run Get-ChiaPlottingStatistic | select -First 1, it says cannot load module, what step I am wrong?
Try changing the execution policy by running Set-ExecutionPolicy -ExecutionPolicy RemoteSigned
or if that doesn't work
Set-ExecutionPolicy -ExecutionPolicy ByPass
@@siriuscleveland4836 Thank you sir! Very helpful
How do you think it would affect simultaneous copy times if multiple destination drives are used?
I just tested file transfers between HDD's ... it looks like transfer times are unaffected as long different drives are used for both read and write as well.
Help with the amount of threads to put on per plot... I have a 8-core 16 thread. Some videos say utilize all of them others say 2 .I think there's some real confusion.
I have the same question too
Experiment and see which gives the best result,
For me 4 threads and 3408 memory help completed phase 1 in about 2 hours, but i will be staggering them so will try 2 threads also.
Great question! I did want to focus only on the delay feature in this video so that people understand each part of plotting and not change more than one variable to give reliable results. I do plan to cover changing the thread count as well and eventually tie everything to together in recap video. To answer your questions, I think using more than 2 thread (at this time since it is only used in phase although this could change) will decrease over all plot time if you staggered them correctly but just how many thread really varies on a per system basis.
@@SysDeskDev thanks 1 set my system 2tb ssd up to do parallel plots every 1.5 hours with 16 threads ,
Let's see how it goes!
Can we have multiple computer to farm chia within different network (not in the same local network) but having the same chia wallet or harvesting address?