Thanks for the video. I think many people don't understand the problems with man pages is the sheer depth of data in them. My one feedback to anyone teaching command line stuff to the masses is do what you did here, show the command with actual use cases and explain them. In so many cases, the man pages will tell a user how to do 10,000 things, but the average usage cases are really a smaller subset which many people would find very useful if they could get the narrowed down + examples. Once they understand the examples, its then much easier to absorb the additional options in the man pages.
If I hadn't watched this video, I would never have learned about rsync as a tool to use for my projects. Thank you so much for the content you create Luke!
rsync ?? That's proper old-skool, I used that in my early days as a sysadmin in the mid-late '90's. Good to know some folks are still appreciating the simplicity and goodness of stuff that predates the horror of the modern Linux world.
before there was easy p2p progs, we used to scan for open ports, creating a folder deep in, turn off read, allow anon ftp. and share the ip and path/to/the/warez in alt. bin. ftp groups (aka the zoo found by local.newbie.school) rsync and FXP ftw on 14.4k
Holly cow, I was running a git server on my lan for backup when I could've just used this.
4 года назад+1
The thing that's cool about your stuff is that this is a way to create your own entire systems. These days we all use simple things like gmail, office, onedrive/dropbox, externally managed calendars. But your videos inspire me to build my own appliances. All the privacy and control, and thanks to your video's easy to pick up. Thank you! 😊 (Except, I will need to spend more for my own 2x1TB raid 1 cloud storage application.)
A really useful feature is that rsync does not mirror by default, i.e. unless you set the --delete switch, it does not delete files at the destination that have been deleted at source. This is useful for a safety backup, so if someone accidentally deletes a file, the file is always guaranteed to still be in the backup. Of course this does not apply to files that have been changed. This feature can have other potential uses too. Another really useful feature is being able to skip directories such as the many application specific directories.
Hi. After hearing you comment about the size of your video file, I was wondering if you could talk a bit about how you choose and store your video encodings for your channel and maybe for your family home videos.
At my second job we developed a whole deployment system using rsync, it worked very well actually and had a nice GUI people could use to deploy their stuff, was a huge upgrade from the SFTP processes they had when i got there lol
wish you uploaded this sooner, while I was following your website tutorial and wanted to update my site, took me two days before I discovered the scp command
Rsync by default uploads only the files which have been changed. Your explanation about "-u" option needs to be corrected. "-u" option allows you to preserve files in destination which have newer modification time.
Around minute 8, you talked about not wanting to ssh into your server to make small changes. Presumably because your internet connection is bad and you'd have to wait for every keystroke to do something. You see, if you were using Emacs, you wouldn't have that problem. You wouldn't need to ssh into your sever. Instead, you'd run Emacs on your local machine, it'd remote mount the server's file system automatically, and you could edit the file just as if you were editing locally.
can you do the same thingswith git? because i recall hearing funtoo uses git for their package repos as opposed to gentoo which uses rsync supposedly it's faster or something, maybe the use cases are different and not comparable?
Program "rsync" je alat koji svaki linux server administrator mora u malom prstu da drži. Namerno pišem na maternjem jeziku, da vidim da li će neko van mog govornog područja upotrebiti google translate. Mada, linux jezik je isti širom univerzuma. Živeo Luke Smith!
I tend to use rsync for my backups, as a cron job. But when I want to manually copy something via ssh, I tend to use scp. Are there any benefits to use rsync in this case instead?
Yes. rsync -e 'ssh -ax' will funnel your files over ssh. rsync is better than scp because it supports resuming if the transport fails, and only transfers the necessary bits.
@@AvindraGoolcharan Just keep in mind it might not work the same way everywhere. I've seen some systems interpret the syntax differently and use some other port for the connection, which caused problems.
10:52: That's verbose, not visual. Also, did you know about openrsync? If not, it's a reimplementation under a nicer license, so rsync could become part of OpenBSD's core utilities. When they reimplement the same thing elsewhere in Unix-land, that's when you know the program is becoming part of the standard Unix toolkit.
I had a client that wanted an automated backup solution for his linux server. I wrote an rsync script in Perl. Needless to say he wasn't impressed but it did exactly what he wanted. I think my client imagined that I'd create his own version of Filezilla in an afternoon. Hard to please normies.
@@epajarjestys9981 Today people don't want to think for themselfs, they want someone answer all their questions NOW and in FullHD video or they will be unhappy and frustrated.
rsync wrapped inside of rsnapshot is awesome. rsnapshot (when driven by cron) creates generations of differential backups that are based on symbolic links.
I use and appreciate rsync. However rsync (and unrelatedly, hardware raid controllers) are made obsolete by zfs and btrfs with snapshots, replication, with striped mirrors or parity. You should be using one of the two for important data that don't have a significant enough performance requirement that necessitates a non cow filesystem like ext4 or xfs. RSync is certainly a great tool for file transfer vs a simple cp over ssh or whatever. However thanks to the nature of cow filesystems snapshots can be made nearly instantaneously providing built in versioning, and because of this all the work rsync needs to go through is unnessesary and in many cases zfs can be massively faster (in terms of cpu usage and local io), and use less bandwidth as well.
What DE is that? i3? I'm thinking of getting back into Linux and this looks VERY nice! I'd love to try a tile based DE and your setup looks like it has mouse support.
I don't understand what is so special about rsync. The showed use-cases are covered by cp and scp. The only use I find for the rsync is `--partial` and `--progress`
rsnapshot has saved my ass a few times: it uses rsync and cp together to create backups. It's FOSS (of coss!) and I've used it to create a live backup server for a company that made backups but never used them because they were too much hassle to get at (!). The backups were available (read-only) to everyone who needed them, and they didn't have to ask me to find them for them. When they messed up, as everyone does, they could fix it themselves.
"ssh: Could not resolve hostname dom: Temporary failure in name resolution rsync: connection unexpectedly closed (0 bytes received so far) [sender] rsync error: unexplained error (code 255) at io.c(228) [sender=v3.2.3]" Yes, very useful
Let's say you're updating a directory with files that have been deleted. --delete-after removes deleted files after adding the new files, while --delete-before deletes them before transfering new files (in case you are close to running out of space).
@@LukeSmithxyz Depends on the usecase, if I want to recursively search through a couple gigabytes, ripgrep beats grep by a couple orders of magnitude. Example on a 6.4G folder on my harddrive: ripgrep took 0.88s, grep took 27.5s For just filtering a stdout stream, I'd use grep.
![BLACK BAG - Official Trailer [HD] - Only in Theaters March 14](http://i.ytimg.com/vi/Du0Xp8WX_7I/mqdefault.jpg)
My brain: "You've been using rsync for years, you already know this"
Also my brain: "Yes, but I want to hear Luke tell me about it"
My brain:
Also my brain:
Consoom Luke Smith content
This new Pajeet sound effect is really cool !
I use rsync a lot, because it's when the transfer happens, it's important to inherit the permissions and attributes.
cp -a does that as well
How long until we have a node.js version of the coreutils? Will it be named soyutils?
At least we already have busybox.
I really want coreutils as a webapp made with 40 different frameworks and technologies.
what a nightmare
www.npmjs.com/package/cash
@@letsgooo7713 Ok, I'm out
rsync to move stuff around my network
1990s toyota pickup to move stuff around my property
walking to move myself around wood trails
I think you mean an old subaru forester with the seatbelts ripped out
toyota pickups are pretty based
Too bad they don't make the 70 series LC for the USA.
Luke just blew my eardrums at 10:29
I am on my way to the GNU/eardoctor
jesus
Do we need to rsync one new pairs for u?
Thanks for the video. I think many people don't understand the problems with man pages is the sheer depth of data in them. My one feedback to anyone teaching command line stuff to the masses is do what you did here, show the command with actual use cases and explain them. In so many cases, the man pages will tell a user how to do 10,000 things, but the average usage cases are really a smaller subset which many people would find very useful if they could get the narrowed down + examples. Once they understand the examples, its then much easier to absorb the additional options in the man pages.
If I hadn't watched this video, I would never have learned about rsync as a tool to use for my projects. Thank you so much for the content you create Luke!
rsync ?? That's proper old-skool, I used that in my early days as a sysadmin in the mid-late '90's. Good to know some folks are still appreciating the simplicity and goodness of stuff that predates the horror of the modern Linux world.
before there was easy p2p progs, we used to scan for open ports, creating a folder deep in, turn off read, allow anon ftp.
and
share the ip and path/to/the/warez
in alt. bin. ftp groups
(aka the zoo found by local.newbie.school)
rsync and FXP ftw on 14.4k
Rsync is old skool and new skool too.
rsync is not "proper old-skool" it's a core tool widely used today.
@@entelin exactly, it's like saying sudo is old school. What's up with these people lol
Old school = best school, not even a boomer either lol
Holly cow, I was running a git server on my lan for backup when I could've just used this.
The thing that's cool about your stuff is that this is a way to create your own entire systems. These days we all use simple things like gmail, office, onedrive/dropbox, externally managed calendars. But your videos inspire me to build my own appliances.
All the privacy and control, and thanks to your video's easy to pick up. Thank you! 😊
(Except, I will need to spend more for my own 2x1TB raid 1 cloud storage application.)
A really useful feature is that rsync does not mirror by default, i.e. unless you set the --delete switch, it does not delete files at the destination that have been deleted at source. This is useful for a safety backup, so if someone accidentally deletes a file, the file is always guaranteed to still be in the backup. Of course this does not apply to files that have been changed. This feature can have other potential uses too. Another really useful feature is being able to skip directories such as the many application specific directories.
I use rsync as a replacement of robocopy since I switched to Linux. Thanks for the video.
Great to see you finally covering rsync. Its an awesome tool that everyone should use
rsync has been a lifesaver for me many times.
I use rsync at work hundreds of times a day. A great little program.
Boomer switches to pajeet mic after 11 minutes of ranting.
This comment is gold :)
10:28
Hello! Welcome to tutorial. How to website, now turn on server and listen to me talk out presentation points!
rooooooooofl 😂👏🏻
@Warren Tristian comments that both posted 3 days ago?! How coincidental!
I need a java implementation of soygrep so bad
Hi. After hearing you comment about the size of your video file, I was wondering if you could talk a bit about how you choose and store your video encodings for your channel and maybe for your family home videos.
At my second job we developed a whole deployment system using rsync, it worked very well actually and had a nice GUI people could use to deploy their stuff, was a huge upgrade from the SFTP processes they had when i got there lol
Oh damn, I didn't even know this existed. Thanks a lot Luke
Rsync is awesome. And its creator is also really awesome. He works on open source drone technology now. His name is Andrew Tridgell. Really cool guy.
wish you uploaded this sooner, while I was following your website tutorial and wanted to update my site, took me two days before I discovered the scp command
I enjoyed the bonus track at the end
That ending was loud good god lol
It's my #1 Linux utility. And it's insanely fast.
Rsync by default uploads only the files which have been changed. Your explanation about "-u" option needs to be corrected. "-u" option allows you to preserve files in destination which have newer modification time.
rsync is very powerful and a tool I used daily back in the day.
8:25 What this "se" alias stands for? Some dmenu magic?
Wehre can I get your background-Image. It Looks nice.
Around minute 8, you talked about not wanting to ssh into your server to make small changes. Presumably because your internet connection is bad and you'd have to wait for every keystroke to do something. You see, if you were using Emacs, you wouldn't have that problem. You wouldn't need to ssh into your sever. Instead, you'd run Emacs on your local machine, it'd remote mount the server's file system automatically, and you could edit the file just as if you were editing locally.
Yeah, but then you'd be using Emacs
People keep telling me to use rsync, might have to check that out
Syncthing is really good, for continuous sync.
I use Syncthing between PC and Android
can you do the same thingswith git? because i recall hearing funtoo uses git for their package repos as opposed to gentoo which uses rsync
supposedly it's faster or something, maybe the use cases are different and not comparable?
Thank you so much for this. I wish I found this out sooner.
Program "rsync" je alat koji svaki linux server administrator mora u malom prstu da drži. Namerno pišem na maternjem jeziku, da vidim da li će neko van mog govornog područja upotrebiti google translate. Mada, linux jezik je isti širom univerzuma. Živeo Luke Smith!
Thanks Luke! I was using scp for uploading files. Rsync seems better.
i could not pay attention because i kept wondering what thermalpaste_foryt.mp4 contains and why it is in home directory
Well, now I know
I tend to use rsync for my backups, as a cron job. But when I want to manually copy something via ssh, I tend to use scp. Are there any benefits to use rsync in this case instead?
Yes. rsync -e 'ssh -ax' will funnel your files over ssh. rsync is better than scp because it supports resuming if the transport fails, and only transfers the necessary bits.
@@AvindraGoolcharan That's an anachronism. In current version of rsync it is not necessary to declare the "-e ssh".
@@laszu7137 thx, good to know... been using rsync to deploy for a while, will be glad to drop this bit.
@@AvindraGoolcharan Just keep in mind it might not work the same way everywhere. I've seen some systems interpret the syntax differently and use some other port for the connection, which caused problems.
Ok, waiting for Luke to tell us all youngsters about ls, cp and mv
Rsync vs Syncthing? Looking for a solution for Logseq to Ubuntu desktop and iPhone X.
10:52: That's verbose, not visual.
Also, did you know about openrsync? If not, it's a reimplementation under a nicer license, so rsync could become part of OpenBSD's core utilities. When they reimplement the same thing elsewhere in Unix-land, that's when you know the program is becoming part of the standard Unix toolkit.
I was just wondering what would be a good filesyncing program. Thank you, very cool!
rsync indeed works using that protocol
I've always used scp for transferring files between machines. What's the difference?
Rsync is really great if you have a local file server
I've been using scp all the time and now I'm confused what are the differences
so rsync=cp&scp, where the -u option also applies to scp, and a new -P option for backups and some other fancy stuff, you probably won't need
what's that script you got there named "se"?
Thank you Senpai
What is your suggestions for sync directories automatically or to mount through network?
I had a client that wanted an automated backup solution for his linux server. I wrote an rsync script in Perl. Needless to say he wasn't impressed but it did exactly what he wanted.
I think my client imagined that I'd create his own version of Filezilla in an afternoon. Hard to please normies.
soygrep is a powerful program. I use it all the time!
I use scp to copy my files over to remote computers. What are the advantages of scp and rsync?
Maybe watch the video or read the manpage.
@@epajarjestys9981 Today people don't want to think for themselfs, they want someone answer all their questions NOW and in FullHD video or they will be unhappy and frustrated.
Now I can edit html in a local git repo, git push to a bare repo on my VPS, then rsync to /var/www/ on the same VPS. It's big brain time.
What is this 'se' command/script?
Hi Luke Smith, what happend if we dont use "--update", I mean just "rsync -rv" ??
Please make a video about gopher sites and the gopher protocol.
I can't live without cronjob'd rsyncs.
rsync wrapped inside of rsnapshot is awesome. rsnapshot (when driven by cron) creates generations of differential backups that are based on symbolic links.
would like to see borg backup here
Afaik rsync also supports checksumming do verify the files are 1:1 cp does not neither does sftp
I use and appreciate rsync. However rsync (and unrelatedly, hardware raid controllers) are made obsolete by zfs and btrfs with snapshots, replication, with striped mirrors or parity. You should be using one of the two for important data that don't have a significant enough performance requirement that necessitates a non cow filesystem like ext4 or xfs. RSync is certainly a great tool for file transfer vs a simple cp over ssh or whatever. However thanks to the nature of cow filesystems snapshots can be made nearly instantaneously providing built in versioning, and because of this all the work rsync needs to go through is unnessesary and in many cases zfs can be massively faster (in terms of cpu usage and local io), and use less bandwidth as well.
What do you think about chunk-based backups like Borg?
Hi Luke. Thoughts on Matrix protocol ?
Dude. What is your background image?
a mention of `scp` would've been cool. I'd say that's the unix "equivalent"
Do you mean that im wrong by cat'ing my 1gb file through ssh is not a good idea?
wonder what that camera is for?
Yes, you can just scare me with that audio at the end.
Great video Luke.
You have mentioned using SSH instead of user and password. I wonder if you have a video explaining SSH?
Do you mean ssh keys because ssh requires a user name and password otherwise unless you leave it without one which is asking to get hacked
you update your website with rsync? too cool for version control?
Comparison with rsync service would have been nice
What DE is that? i3? I'm thinking of getting back into Linux and this looks VERY nice! I'd love to try a tile based DE and your setup looks like it has mouse support.
dwm
How do you log on to your desktop with ssh without having to enter any credentials?
Is this what happens if your private key is not secured by a password?
Fo next video i recommend unison ;)
Does this work with Wayland?
I don't understand what is so special about rsync. The showed use-cases are covered by cp and scp. The only use I find for the rsync is `--partial` and `--progress`
15MB/s in the local network? that seems oddly... slow
I prefer git for anything website related, because it has dat audit trail. But rsync is pretty amazing.
Do rsync works with some cloud storage?
Luke can I use it to sync files to my Music player?
rsnapshot has saved my ass a few times: it uses rsync and cp together to create backups. It's FOSS (of coss!) and I've used it to create a live backup server for a company that made backups but never used them because they were too much hassle to get at (!). The backups were available (read-only) to everyone who needed them, and they didn't have to ask me to find them for them. When they messed up, as everyone does, they could fix it themselves.
What is 4C7DE2A5267EC851 ?
Use it for uploading ArchBang to sourceforge....
but can it be integrated with DIRED?
Boomer here, 6:34 why don't you just push your changes to git and pull them in your site ?
Can you make a vidéo on NixOS ? Plz !
"ssh: Could not resolve hostname dom: Temporary failure in name resolution
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(228) [sender=v3.2.3]" Yes, very useful
If you can't figure this out... Mannn...
@@randykitchleburger2780 Yeah my mistake forgot to turn ssh on
I used scp all the time. When I found out about rsync my mind was blown :P
your affiliate links don't work, do they ?
Ive been using scp but i really should try rsync as its more powerful
What is that *se* command at 8:26
looks like fzf (fuzzy finder)
Reject (soydev) modernity, embrace tradition(al UNIX tools)
what exactly does "delete-after" do?
Let's say you're updating a directory with files that have been deleted. --delete-after removes deleted files after adding the new files, while --delete-before deletes them before transfering new files (in case you are close to running out of space).
@@LukeSmithxyz so basically like a "git rm" between two commits?
advantages compared to git?
So basically just scp?
>literally has 3 folders in his ~
Teach me your ways
Boomer: You have to write things to work fast, every millisecond matters
Also boomer: No need to replace grep, grep is gud
No one is going to install a dumb fan-fiction knock-off version of grep just to run a basic script.
@@LukeSmithxyz Depends on the usecase, if I want to recursively search through a couple gigabytes, ripgrep beats grep by a couple orders of magnitude. Example on a 6.4G folder on my harddrive: ripgrep took 0.88s, grep took 27.5s
For just filtering a stdout stream, I'd use grep.
Luke: *too lazy to increase fontsize*
360p gang: :(