I was using exact command for copying 17+ million files/folders from one external SSD to another. Dell Precision with Intel Xeon, 32 GB. No logs, no validation - took 2.5 hours. Pretty amazing result. thank you!
I remembered watching this a while back and returned to try it out consolidating a bunch of old files on from PETA external USB 2 drives to a 4tb USB 3 external. Holy cow did this make a huge difference! Thanks. Side note, I am testing the addition of /R:3 /W:10 to stop endless retries for files that error when copying (permissions, errors on the drive, etc.).
Just a note for average viewers like me: not everyone's processor can multithread 32 streams of information. You need to search to find how many threads your CPU can handle.
TIPS, excessive threads (MT) in Robocopy can SLOW DOWN process substantially (depending on other parameters), ALSO, IF you do NOT use LOG, (display on screen)the process will be slower.
I have an issue with copying files to do with retro gaming. I know the windows prevents the creation of the files with certain names including con and aux et cetera. Do you know how I can get around this? Have you already got a video about this subject, or can point me in the right direction please?
This is great thank you so much. On a side note. I couldn't make it work when copying i.e. a cellphone camera folder to an external drive folder. Because the mobile storage is not presented as a drive but rather shared internal storage. So there's no drive letter to point to :(
Dear christ i dont ever comment and probably you wont see this but my god how insanely interesting this was. I do cisco networking for the most part and this has intrigued me that if this is even a glimpse at the power of working with powershell that i wanna look into learning PS more. Incredibly informative vid, gave a like and subscribe instantly and will be sure to check out your other content.
Tested with a 5.21 GB Crossfire installer files and below are the result Using: File explorer - 3min 09s Powershell Copy-Item - 3min 12s Powershell robocopy (again thing) - 4min 13s any other copy utility software recommendations? I need to transfer a 200GB file. Thanks
I know this is an old video by now, but I was wondering if you know of any reasons why robocopy wouldn't work? I've typed in exactly what you did into powershell, both regular and admin mode, moving almost the same amount of files as in your video, and it just sits rather still and not doing anything. The files are all .png and small too btw. I've left it going for hours before and it hasn't done anything yet, not even displaying things by default like I read up on it and it was said to do. Task Manager says powershell is using almost 9% of my cpu, so it must be doing something, but I can't tell what. It isn't logging because I turned all of that off since it's just moving things from one file to the next on a pc rather than a work computer. My computer has a i7-10700KF and 32gb of ram, so there's no hardware reason for this to be the case. I hope that you see this soon, even if its on an old video
Hi Jon. Looks like you're into SQL and programming and often moving loads of small files... ;-) Great to stumble upon this video of yours. Good you worked and showed it all methodically. I am into audio-files. Merging large collections. Ao having trouble in getting a NAS on speed. DIY NAS on FreeNAS. Via SMB/Samba going to workstation on Windows. Slow r/w, slow dir.listing in several sw packages, slow file-deletes. Didnt know but tried your Robocopy command and reads go up 50%. Nice and good to see it has an effect (many tunings did nothing). But probably the command-settings can be tuned/improved to my file-situation?
Am having audio-files (from small MP3 to large WAV), file-size-range 4MB-250MB with heavy emphasis on ~10MB and ~35MB. Often moving say 300.000 files from workstation to NAS (via 10Gb-connection). Then waiting-game starts... You gave a command-line that you probably tested and ended up with as best for your (4k-thumbnail-file) situation. Can this be improved upon for the file-situation I have (also: files moving from NAS/Linuxlike to workstation/Windows)? Any input much appreciated. Cheers. Gerard
there is no point to use 32 threads if your computer has only 4 or 8. Learn your basics. Your pc will slow down due to thread management (stoping and starting threads)
Your test are somehow wrong, only first one really copying files everything else is just pasting your ram content to the target disk again and again while you are selecting the same files and robocopy failed 1st time because you were replacing files so on the target were placing more i/o operations instead of just writing on the target disk. Remake your tests by rebooting the machine before each test, find a way to measure time without using an app on the pc, use your phone or a smart watch and delete the target disk (even the partition if its probably a test disk) and recreate.
i really appreciate these how to tutorials, but if the fastest way to transfer files , is to open cmd, etc defeats the whole point of this..i just wanted a faster way to copy & paste, or send to by right clicking on 1 drive to send the data to another, so i am bamboozled by all this..it would take another tutorial to do all this, so this fastest way would take us PC novices longer to do this stuff to improve transfer speed, than it would the old fashioned way, right click & send.......thank you stay safe but i will stick to the Pony Express method, hi ho Silver
I appreciate you sharing this but what I really want to know is why on earth my 10 year old Window 7 system is so much faster at standard copy/pasting/moving huge numbers of files than my shiny new Windows 10 machine, where on paper the hardware blows the old pc out of the water? The Win 10 box has a new 1tb Samsung SSD plus a 4tb standard HDD and it was slow to handle files from the beginning. I'm almost at the point of exploring ways of running Win 7 safely and switching back.
@@telejonjensen they both have windows defender and nothing else, same chrome extensions, onedrive, Creative Cloud and Office 365. Thank you for your reply! There is mostly the same software installed but I tend to only run a few programs anyway.
One has the heebie-jeebies and the other doesn't. They don't bother giving the "hot-foot" to older systems because they get rarer ever day. Unless of course you are a high-profile pain in their ass!
dude you are epic unfortunately last one is still slow for me . Doing 3tb data not 1 to 3 min to 7 sec but same slow laborious using various methods like the ones you described. 1hr about 100 gb which is poopy slow.
How about large files? Have been trying exactly the command recommended in this vid but for large video files instead and after 12 mins it's still stuck at 0%! I just can't seem to find any more specific instructions for such files.
@13:24 is where the fastest way is demonstrated. The robocopy command didn't show up in my powershell but it does show up in the regular CMD prompt. Any idea why?
@@telejonjensen I have no idea why it wasn't showing up in the path on that day I tested it but it is now showing up just fine. The command was always there, just didn't show up in the path so I had to go into the directory where it was located and run from there. I'm guessing it got fixed in one of the updates that happened recently. My current version of PS is 18362 at present and as I said it's working now. My OS version is 10.0.18263.
Microsoft is still stuck on these problem comparing with Mac its fast transferring files but windows still stuck there i think they never think about it . That's the reason I hit windows I work with big capacity fils and time to move from c to d you need to go take a shower go to gym eat some pasta then you kome back you see the f.... 99.9999999
I was using exact command for copying 17+ million files/folders from one external SSD to another. Dell Precision with Intel Xeon, 32 GB. No logs, no validation - took 2.5 hours. Pretty amazing result. thank you!
This was fantastic thank you! Shouldn't only /E or /S be used, but not both?
robocopy 'folder location' 'destination folder location' /S /E /MT:32 /NFL /NDL /NJH /NJS
hope this helps☝
maybe you could upload these CMD paths to the description below video it would save a hellova time
I remembered watching this a while back and returned to try it out consolidating a bunch of old files on from PETA external USB 2 drives to a 4tb USB 3 external. Holy cow did this make a huge difference! Thanks. Side note, I am testing the addition of /R:3 /W:10 to stop endless retries for files that error when copying (permissions, errors on the drive, etc.).
Just a note for average viewers like me: not everyone's processor can multithread 32 streams of information. You need to search to find how many threads your CPU can handle.
Thanks dude, you saved my day! I was moving 65GB storage to another disk
So it's cool in this example, but I'm pretty sure it doesn't change much if you have for example much larger files, and there are less of them.
Well done, thanks! But does running RC from Powershell instead of the command line have any impact on the speed?
TIPS, excessive threads (MT) in Robocopy can SLOW DOWN process substantially (depending on other parameters), ALSO, IF you do NOT use LOG, (display on screen)the process will be slower.
I have an issue with copying files to do with retro gaming. I know the windows prevents the creation of the files with certain names including con and aux et cetera. Do you know how I can get around this? Have you already got a video about this subject, or can point me in the right direction please?
Wonderful! Thank you! Saves incredible time! I don't understand why Microsoft doesn't do this as the default!
I was wondering the same thing; what a head-scratcher .....
This is great thank you so much. On a side note. I couldn't make it work when copying i.e. a cellphone camera folder to an external drive folder. Because the mobile storage is not presented as a drive but rather shared internal storage. So there's no drive letter to point to :(
I am working on around Huge amount of files and I've just this method.
wow, it really worked fast and save a lot of time.
Quality life improving content. Thanks a million!
Happy to help!
Wow. What a MASSIVE time saver --- thanks Jon!
Dear christ i dont ever comment and probably you wont see this but my god how insanely interesting this was. I do cisco networking for the most part and this has intrigued me that if this is even a glimpse at the power of working with powershell that i wanna look into learning PS more.
Incredibly informative vid, gave a like and subscribe instantly and will be sure to check out your other content.
Tested with a 5.21 GB Crossfire installer files and below are the result
Using:
File explorer - 3min 09s
Powershell Copy-Item - 3min 12s
Powershell robocopy (again thing) - 4min 13s
any other copy utility software recommendations? I need to transfer a 200GB file. Thanks
I know this is an old video by now, but I was wondering if you know of any reasons why robocopy wouldn't work? I've typed in exactly what you did into powershell, both regular and admin mode, moving almost the same amount of files as in your video, and it just sits rather still and not doing anything. The files are all .png and small too btw. I've left it going for hours before and it hasn't done anything yet, not even displaying things by default like I read up on it and it was said to do. Task Manager says powershell is using almost 9% of my cpu, so it must be doing something, but I can't tell what. It isn't logging because I turned all of that off since it's just moving things from one file to the next on a pc rather than a work computer. My computer has a i7-10700KF and 32gb of ram, so there's no hardware reason for this to be the case. I hope that you see this soon, even if its on an old video
Hi, Robocopy is very sensitive to proper syntax, also copying to mapped drives can be problematic. it takes time and study in some cases to figure it.
/s /e are mutually exclusive now according to Microsoft docs...
Won't MT:32 cause mass fragmentation? even SSD aren't immune to that
Hi Jon. Looks like you're into SQL and programming and often moving loads of small files... ;-) Great to stumble upon this video of yours. Good you worked and showed it all methodically. I am into audio-files. Merging large collections. Ao having trouble in getting a NAS on speed. DIY NAS on FreeNAS. Via SMB/Samba going to workstation on Windows. Slow r/w, slow dir.listing in several sw packages, slow file-deletes. Didnt know but tried your Robocopy command and reads go up 50%. Nice and good to see it has an effect (many tunings did nothing). But probably the command-settings can be tuned/improved to my file-situation?
Am having audio-files (from small MP3 to large WAV), file-size-range 4MB-250MB with heavy emphasis on ~10MB and ~35MB. Often moving say 300.000 files from workstation to NAS (via 10Gb-connection). Then waiting-game starts... You gave a command-line that you probably tested and ended up with as best for your (4k-thumbnail-file) situation. Can this be improved upon for the file-situation I have (also: files moving from NAS/Linuxlike to workstation/Windows)? Any input much appreciated. Cheers. Gerard
Hi,
Going across the network or to a different operating system I usually just copy to a USB3 drive and do it that way.
Jon
there is no point to use 32 threads if your computer has only 4 or 8. Learn your basics. Your pc will slow down due to thread management (stoping and starting threads)
Your test are somehow wrong, only first one really copying files everything else is just pasting your ram content to the target disk again and again while you are selecting the same files and robocopy failed 1st time because you were replacing files so on the target were placing more i/o operations instead of just writing on the target disk. Remake your tests by rebooting the machine before each test, find a way to measure time without using an app on the pc, use your phone or a smart watch and delete the target disk (even the partition if its probably a test disk) and recreate.
Like the last one. You saved my day!
Does the last command copy all types of file?
i really appreciate these how to tutorials, but if the fastest way to transfer files , is to open cmd, etc defeats the whole point of this..i just wanted a faster way to copy & paste, or send to by right clicking on 1 drive to send the data to another, so i am bamboozled by all this..it would take another tutorial to do all this, so this fastest way would take us PC novices longer to do this stuff to improve transfer speed, than it would the old fashioned way, right click & send.......thank you stay safe but i will stick to the Pony Express method, hi ho Silver
I agree with you, it not friendly with user. we need simple and easy
Anyone know what is the purpose the three 000's in the command line?
Great Job !!! Thank You !!!!
Hey I want to know how to copy zip files using robocopy because it say invalid directory
RoboCop approves of robocopy
Many thanks. This is very, very useful.
Greetings from Germany :)
Really good. Thanks a lot
I appreciate you sharing this but what I really want to know is why on earth my 10 year old Window 7 system is so much faster at standard copy/pasting/moving huge numbers of files than my shiny new Windows 10 machine, where on paper the hardware blows the old pc out of the water? The Win 10 box has a new 1tb Samsung SSD plus a 4tb standard HDD and it was slow to handle files from the beginning. I'm almost at the point of exploring ways of running Win 7 safely and switching back.
Great question - have you turned virus scanning off or is the virus scanning configuration different between the two systems? Thanks for watching!
Jon
@@telejonjensen they both have windows defender and nothing else, same chrome extensions, onedrive, Creative Cloud and Office 365. Thank you for your reply! There is mostly the same software installed but I tend to only run a few programs anyway.
One has the heebie-jeebies and the other doesn't. They don't bother giving the "hot-foot" to older systems because they get rarer ever day. Unless of course you are a high-profile pain in their ass!
Try disabling Drive Indexing on W10, you may need to restart. you can re-enable indexing after you do copying.
@@awesomearizona-dino thank you I'll try that, I've got a lot of renders and opening the folders in explorer just chokes it
Another solution, primo cache and a ssd or another caching program
Not useful mate, Your data will be cached untill it's fully copied.
Mateeee, you’re a fucking legend!
Great video bro
dude you are epic unfortunately last one is still slow for me . Doing 3tb data not 1 to 3 min to 7 sec but same slow laborious using various methods like the ones you described. 1hr about 100 gb which is poopy slow.
Thank you kindly
How about large files? Have been trying exactly the command recommended in this vid but for large video files instead and after 12 mins it's still stuck at 0%!
I just can't seem to find any more specific instructions for such files.
/j - would that help?
Your awesome man🔥 thankyou
@13:24 is where the fastest way is demonstrated. The robocopy command didn't show up in my powershell but it does show up in the regular CMD prompt. Any idea why?
Hi Michael,
What version of Powershell and Windows are you using?
Thanks for the question.
Jon
@@telejonjensen I have no idea why it wasn't showing up in the path on that day I tested it but it is now showing up just fine. The command was always there, just didn't show up in the path so I had to go into the directory where it was located and run from there. I'm guessing it got fixed in one of the updates that happened recently. My current version of PS is 18362 at present and as I said it's working now. My OS version is 10.0.18263.
Appreciate the video.
Thanks for the video. It help me for copy pasting my old data speedly with the help of ppowershell method
Thanks a lot!
Phenominal!
Microsoft is still stuck on these problem comparing with Mac its fast transferring files but windows still stuck there i think they never think about it . That's the reason I hit windows I work with big capacity fils and time to move from c to d you need to go take a shower go to gym eat some pasta then you kome back you see the f.... 99.9999999
Fantastic copy rate
awesome, 10000000 thanks.
👍
From my testing >>> robocopy 'C:\Users\San\Desktop\TEST' 'X:\TEST' /S /E /MT:32 /NFL /NDL /NJH /NJS