Backup and Restore Your Linux System with rsync
HTML-код
- Опубликовано: 11 сен 2024
- From this video you will learn how to backup your whole Linux system and how to restore it back if something bad happens to it.
The command you need is this one:
sudo rsync -aAXv --delete --exclude=/dev/* --exclude=/proc/* --exclude=/sys/* --exclude=/tmp/* --exclude=/run/* --exclude=/mnt/* --exclude=/media/* --exclude="swapfile" --exclude="lost+found" --exclude=".cache" --exclude="Downloads" --exclude=".VirtualBoxVMs" --exclude=".ecryptfs" /source /destination
You can also add the option --dry-run to simulate the backup process.
The command to restore your system is shorter:
sudo rsync -aAXv --delete --exclude="lost+found" /backup /system
To see more options run:
rsync --help
How to make a bootable USB drive • Make a bootable USB dr...
################################################
Support on Patreon: / averagelinuxuser
Donate through PayPal: www.paypal.me/...
################################################
More cool Linux stuff at other social media:
Google+ - Best Linux Stuff: plus.google.co...
Facebook - Top Linux Tips: / averagelinuxuser
Twitter - Daily Linux News: / avglinuxuser
Website: averagelinuxuse...
You can also unite all exclude options with --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/home/*,/lost+found}. But make sure you cd to root (cd /) before you run rsync, otherwise the joint exclude option may not work.
PS. Thanks to MoreChannelNoise for this tip. The comment is below.
Dude, bootable dose not equal to LIVE. Cmon! LIVE Usb - is from u can launch the whole system! Bootable usb - is from u can install system on PC. Launch != install. It`s quite confusing rlly.
Hi there! Can u tell me please, wich OS + Panel settings are you using? Do you have a video hoe to install exactly the same? Thank you.
Also /mnt is short for ‘mount’! Was funny hearing you say m-n-t instead of saying “mount”. Your English is really good, and thanks for the great content! Spasiba
@@user-cb9gk3oc5w thanks for letting us know.
According to the man page, "--delete" actually deletes extraneous files and directories from the destination. I think this means that it deletes all files that aren't in the source. Just to clarify this, because the explanation given in the video sounds (to me), like that option only transfers the files that are missing in the destination, which is the default behavior for rsync.
Was going to reply with this comment. The deleted files would have been restored w/out this option. Other wise nice succinct vid, thanks!
Great video. Just to say you can combine all your excludes in one line like this:
sudo rsync -aAXvh --progress --delete --exclude={".cache","/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"}
I have seen people using --exclude it this way but it didn't work for me. I don't know why. Only separate --exclude worked.
Another marvellous video, ALU! I am learning more from you in a couple of months than I have ever learned before! Keep up the wonderful work ;) !
I've learn a lot about Linux since I started this channel too. Thanks for your support.
That dry run option was very smart ! I did not know such a command existed.... Thanks for sharing !!
My pleasure
@@AverageLinuxUser Yes, Sudip, I think the ALU really does a good job on these videos. And, I love his accent too. New to his channel, but he makes me realize I can do Linux too.
It took me three weeks to figure this out lol. I tried to backup with rsync to my external SSD that is formatted as a fat file system. I came here and I simply copy pasted your commands. I searched the resulting error codes on the internet for days until I realized I just have the wrong file system.
Linux can be hard if you're not paying attention :D
Thank you for this video!
Really great video as always, I've watched your Arch Installation video like 15 times and finally put Arch to metal first try without issue.
Hello Devon, Thank you for you comment. I am glad my Arch video was helpful for you. Enjoy your Arch!
Anonymous RUclips comment challenge: install Artix linux(arch without systemD)
great
Wow just yesterday I backedup my home folder to USB harddisk using Grsync. I will suggest Grsync, its just a frontend to rsync and people who are not comfortable with command line will find it useful. Thank you for sharing.
Thanks for the suggestion. I will have a look at it. I used to backup with lukcyBakcup. It is also rsycn GUI front end.
Чув, твій сайт -- це просто бімба, дякую!
Thank you, again!
Now, when I have finished watching your guide I can say: people could really benefit from using this lifesaving guide!
Personally, I would like to see a guide showing how to transition my existing Ubuntu-based system to a ZFS volume.
So not only the data HDDs but my OS which is written on nvme SSD are being bulletproofed by using the numerous benefits of the ZFS on Linux.
Go ahead and do it yourself for us then
@@InspiredFortunes - I had commented two years ago. And you have a few hours ago. Nowadays, there is an option to do Ubuntu on ZDF with its native installer.
@@trubadyr99 oh pardon me, So sorry I wasnt in your time schedule. BUMMER
Thank you for this.
These days UEFI is part of a modern PC.
People need to know how to build and restore systems taking that under consideration.
The functional EFI partitions and GRUBs in their machines need work well together after they are created/restored.
Thanks for a great & informative video. Also Thanks for being a total asset to the Linux community!!!
I really liked waht you showed us. I learned a lot. I am slowly getting into Linux, and people (like you) make it easier for the Noobs (like me) to start feeling confident and make switch from Windows.
well this video covers all that needs to be mention. it helps me a lot! thank you!!!
I RARELY sign in when watching YT vids. I know likes and subs help but..lazy.
Sorry (to all content makers) but I know a lot of people are like me (seen a video where a person noticed he had a lot of views [>50/100K] but very few likes/subs).
This was so comprehensive yet simple that I couldn't not sign in to like, subscribe and comment (not sure if you get notifications for older vids).
Although I think you just sent me on a warpath of videos I know I liked/loved and now my whole day/week will be gone...
Thank you VERY much!
Dear ALU, I have only one word for your videos. AWESOME!
Thank you for your work, it helps a lot.
Thank you so much. I am so happy to be helpful. All your encouraging commends give me a lot of energy to keep doing these videos. Thank you for such nice comments. They help a lot.
Thank you for the great video, it's still relevant in 2022 for ext4 users
I found this very helpful. Adding from experience using this method on Raspberry Pi, rysnc is the best for backing up data. However, installed software may not work correctly. I had problems with something as simple as web browser (chromium) after system restore by this method. I have ended up using rsync to backup data (files incl. code, images, etc.) and the SD card copier on the Pi for cloning the entire system including installed software. The installed software works correctly by this method. Thanks.
Great video.
I maintain a file for all my exclusions and then call it from my rsync script with --exclude-from=$HOME/bin/excludefile
I find it easier to maintain that way
I just store my the whole rsync in as a shell script file (backup.sh) and simply run it when I need.
@@AverageLinuxUser Thank you both for these ideas. New to Linux, but I am realizing how it is SO MUCH BETTER than Windows. Boy, was I blind before.
Hi ALU,
Thanks for your Video. I wasn't using -A -X options; from now on I will.
Tack så mycket, detta var mycket lärorikt! :)
Thank you for watching. Now, I have another excuse to do youtube videos. I can say I improve my Swedish here by reading comments like yours 😆
Ah, sorry I thought that you can speak Swedish as you've said in one of your videos, that you live in Sweden so I assumed that you can speak Swedish, but I can continue speaking Swedish tough, if that helps. :P
I must say that your videos are VERY informative and I have learned a lot about Arch. After I watched your video I started using Arch full time both at work and home, though some stuff still doesn't work but that is not Arch's fault. Ex. Pulse Secure VPN.
Keep up the good work!
No problem :-) I use Cisco VPN with openconnect and it works fine. Doesn't Pulse Secure VPN work with openconnect?
@@AverageLinuxUser Please, make a video about your VPN setup, where all of us can learn how to do it too.
thank you for all your videos. they are all very clear and extremely informative. cheers!
Thank you for the feedback. It's encouraging.
Thanks! It was very clear and easy to understand.
thank you for sharing this. using it to backup both systems and documents.
Thank you for providing subtitles, the automatic ones are really bad sometimes
This is so great, none of the other tutorials prove the system restoration... great workflow!
"--delete
This tells rsync to delete extraneous files from the receiving side (ones that aren't on the sending side), but only for the directories that are being synchronized. You must have asked rsync to send the whole directory (e.g. lqdirrq or lqdir/rq) without using a wildcard for the directory's contents (e.g. lqdir/*rq) since the wildcard is expanded by the shell and rsync thus gets a request to transfer individual files, not the files' parent directory. Files that are excluded from the transfer are also excluded from being deleted unless you use the --delete-excluded option or mark the rules as only matching on the sending side (see the include/exclude modifiers in the FILTER RULES section)."
This doesn't "restore only the files which have been deleted"
Thank you very much.
I have a question though: why do you exclude lost+found from restored files when it was already excluded from backup?
Thanks for the good content. Like it and I will follow this method. I use rsync only for my data backup with great satisfaction.
Continue your great linux education, you do it well.
Ha det bra ....
Thanks a lot. I see I am getting some Swedish audience here 🙂
1. When you specified --exclude=".VirtualBoxVMs", this path is relative to the source directory being backed up. Since you were backing up the root directory, this means you are excluding /.VirtualBoxVMs. But VirtualBox stores your VMs under your home folder (unless, of course, you explicitly tell it to do it somewhere else).
2. Just a tip: when you have so many folders you want to exclude, you can put them all in a text file and point rsync to that file with the --exclude-from= option.
3. One other rsync option I like is --stats. It displays a summary of total number of files/folders created, updated, deleted, etc.
4. That's a nice Arch wallpaper you have there! Could you please put a link to it? Thanks.
Thank you for such a detailed feedback.
1. I don't think so. ".VirtualBoxVMs" means to exclude any pattern that has .VirtualBoxVMs. I used it many times and it works as expected.
2. You are right. I actually have this command in shell script file. So I just execute that script.
3. I will try it next time I backup my system. Thanks!
4. I post all wallpapers I use in my videos in my G+ collection plus.google.com/b/105235169774065518394/collection/YlkQNE
exclude accept pattern, it will work
I believe the exclude pattern should be --exclude="VirtualBox VMs"
On my Linux Mint system, there is a space in that directory name.
Hi Nick, You are right. The default name is with a space. But I don't like spaces in names, so I renamed it to VirtualBoxVMs.
"rsync warning: some files vanished before they could be transferred (code 24) at main.c(1189) [sender=3.1.3]"
How do I not get this message? ):
This is a nice video tutorial keep up the good work!
Thank you, Rodney!
Halo ALU great as usual consider weekly Linux news show/live streams.
I am thinking about this.. Thanks for the suggestion.
Your video is very helpful and thank you for this share with us. I have some questions and few things I don't like. First of all, this backup seems to not care about partitions, if I have 4 partitions and I destroy 2 partitions after restore do I have back my destroyed partitions? Also having on the USB just copied files but not compressed packet in one or two files like iso or gho (from ghost) it's not very friendly having files opened to check by anyone anytime. After a backup like this, is it possible to restore to a new HDD not partitioned? What other backup-restore alternative we have for Linux system?
Great video! Definitely a useful thing to know.
Thank you
Much appreciated. Keep up the nice work!
Thank you. I will do my best.
Nice video!!! Is there an option to run multiple threads of rsync? I usually have to backup a lot of big video file and it would be great if I could tell the software to transfer 4 files at a time.
You can open multiple terminal sessions and run an rsync in each of them, but there's not likely to be much benefit (especially if you're backing up to a remote system) bc rsync will use as much bandwidth as possible. Video files are also sequential reads and writes, so running multiple threads may actually slow your backup down.
The method is good when the partition structure on the disk is intact and only some files or directories have been destroyed, which are then easily recreated from the backup. The problem arises when the partition structure on the disk is destroyed and even GRUB is not loading. And what to do then?
reinstall the OS and perform the rsync restore
Buy Linux compatible Hard Drives www.amazon.com/shop/averagelinuxuser
Useful and educational. Thank you.
Excellent tutorial, thanx.
Excellent tutorial I have subscribed
Very useful. Thank you.
Very! Instructive Thanks!!!
You are very welcome
Great one, good way to back up and restore :)
Thanks. back up is essential!
why you put quotations around the excluded files?? I cannot get it work if I keep these quotations
This was perfect :) Pure magic! Thanks.
Thank you and there is no magic :-)
Yes, very useful and professional video, thank you :)
Thank you for commenting so often.
I know I am a bit late in asking this question but here I go. I am trying to create linux (ubuntu) root / directory (ext4 file system) backup on a zfs file system (TrueNAS) via rsync and ssh. I use the options -aAXHvz but the preserve access control list option "A" fails by throwing "protocol incompatibility" error. I can't seem to find a way around this. Without using the A option the file/directories are saved with invalid ACLs and are messed up on restore. Is there anyway to solve this problem?
Very good guide, thank you!
ALU not sure what i did wrong. i used your command in terminal and the destination for my ext hdd. Terminal shows files are being copied but to where? They are not on my ext hdd and i don't know where they are going and ext hdd is blinking but where are my backup files????
Hi, thank you for the concise explanation. What's the difference between rsync and dd, considering you can use either of those two to backup the system?
du is disk usage
if most commonly we use du -csh
if you are new the consider rsync as scp without overwriting
@@PravinJanjal92 sorry, seems like I wasn't sober enough when typing the comment. I meant to write dd instead of du. Earlier comment is edited to prevent further confusion.
And yes, you're right about du as a tool to see disk usage.
This helps lot i hope you do the one without boot loader....Thank you any way i have some questions ..I have already googles and tried but still confused that why i m asking below questions..
1-which software are you using to record these videos, im using Manjaro..?
2-should i use gnome or KDE (i saw you re using KDE..)?
1. SimpleScreenRecorder + guvcview for the webcam.
2. I vote for KDE because I like it, but there are many folks who loves GNOME.
Thanks Bro
Thank you! I imagine that this will come in handy some time in the future, definitely. I look forward to the video on using the GUI version, too. Maybe that's a little bit easier, I don't know. Anyway, thanks for the video! :) Дякую!
Robert Andersson the first time I used a GUI to restore a backup I accidentally copied my damaged system to my backup. I lost everything because I didn't understand what the GUI program was doing with the underlying commands. That's when I learned how to use rsync
I started learning rsync with GUI too. The program was called luckyBackup. Funnny name is it is. It was just a front tend of rsync. Then I decided to get rid of that mediator and started using rsync directly. I need to have a look at GUI options yet. I's been a while since I used them.
P.S. Tack-tack :) Trevligt att se ukrainska ord
Is there a way to test if my backup has worked without destroying my system?
I'm fairly sure it worked well because there were no errors in the log, but I want to test it without destroying my computer.
I did backup and restore linux system as per your instruction, but restore is not working and my Linux system is down !!!!
Hi.
And what about the /home?
Can i exclude, when i make backup with option --exclude="/home*", and when i will do restore, i will use the same option?
This is what I use:
sudo rsync --recursive --times --perms --owner --group --executability --acls
--xattrs --delete --delete-excluded --checksum --links --hard-links
--specials --devices --human-readable --progress --verbose --stats
--protect-args --log-file="rsync.log" [excluded] [src] [dest]
will appreciated if I can know your opinion
Btw: on the restoring part --delete will specifies: "delete extraneous files from dest dirs", if there is any file(s)/dir not exist in the source but exist in the dest will be deleted from the dest , the reason is to make it sync
Hi Mani, Thanks for sharing your command. Looks good. --log-file="rsync.log" is handy.
You should've described a safe way to test a backup, otherwise someone's gonna intentionally destroy their system just to realize their backups don't work, and their server clients just messaged them for panel password two minutes ago...
this is great but instead of "wasting" a device just to backups i would rather put my backup on the device im currently backing up, then putting it all into a tarball or iso (dd comes to mind if that works) and then place that somewhere else (/safe) and decompress it on the target machine as needed
hi Alu, very nice video and easy to follow! Can you show a video of how to back up via SSH? on my external hard drive. is that possible?
Excellent guide, though rsync hangs again and again when I try to back up my Raspberry Pi. I thought dry-run would expose the issue but it does fine in dry-run and then hangs up during back-up attempt. The backup drive has enough space, I’ve made sure of that. Disappointing, really, because rsync seems so apt.
I face issue while restoring the Backup to a new machine.
I exclude Boot direcrotry and /etc/fstab file, while restoring, then the apt-get update does not function. I am able to install new programs but not able to update. using linux 16.0 with kernell 4.4.0-21-generic
kernel of backup is 4.4.0-161-generic. Do you have any idea how to rectify this problem?
Excellent. Cheers!
Would this be possible?
use a online cloud storage as the backup target location? I don't want to backup on storage in my home and then have to move that to some for example dropbox folder.
It is possible but tricky. The easier option will be to use Deja dup as in this example ruclips.net/video/BLVtxpm5c2A/видео.htmlm49s
Thanks! this helps a lot!
I am getting chown "name of current file" operation not permitted a lot of times, with --dry-run it runs smoothly, I am using sudo in the command. does somebody knows what to do? I want to update my arch linux install without the fear of breaking the installation
Great tutorial!!
could you suggest pls how in your opinion could i do that command for doing os centos 6 backup, full, including all ? and using this backup file produced, for restoring my rootvg system ?
Would this process work to, say, burn an image of your old system on a new machine? If you do this, how to account for the different partions (/efi, /boot / /home) from the old image? Thank you!
Great video, very nice!!!
Thank you, Mauricio
Thank you very much for your help, you really helped me !!!
My pleasure.
I am new to linux so I want to know if this method, will restore everything. What I am interested in . Let's assume I have Samba file sharing, with DC, DNS, a bunch of users and groups with privileges, the server with it's users and groups, all that mess working just fine. Can this method that is basically copy-paste restore the server over a new install? (Sure we assume the exact distribution and release is installed).
Do you have a video on how to restore the system to a new drive ? Thx
How do I back up an external hard drive that's failing to another external hard drive?
Ummm... You might want to read what the --delete flag does before you destroy your backups... It doesn't do what the video says and will delete files in your backup if used incorrectly.
you can use only one --exclude with many directories like this: --exclude={dir1,dir2,file1.txt,andSoOn}
I was confused when I checked other tutorials which stated that -a archive mode preserves permissions. From searching on stack overflow it turns out you don't need the X and A options as archive mode encompasses those. Anyone know any different?
Can I use an iscsi target storage to backup my linux system with rsync?
Can you permalink or paste your notes for the lazy? Thanks, Great video! I also added --progress option since I am transferring large amount of data.
Hi Joe, Thank you watching the video. The commands are in the description, aren't they?
Shouldn't you be using §--checksum§ always ?
Great video!
Thank you for watching the video
I suppose this will also work using software raid, correct?
Two questions: 1. How do you rsync to Dropbox, Google Drive or S3?
2. Can rsync --delete check file contents and overwrite broken config files? e.g. if I mess up a vital config, can I restore the original this way?
Your first question: you may want to consider not backing up to a file hosting service because it will be a very large file and take a long time to do the initial backup and restore. However if you are simply backing up some configuration files you can do that by changing the destination to your Dropbox folder on your system. Your second question: a simple answer yes. It is much more efficient however to use the move command before you edit your configuration file. For example, you can do mv ~/.config/file.conf ~/.config/file.conf.bak and then change file.conf. if you need to revert back, just mv ~/.config/file.conf.bak ~/.config/file.conf
I agree with Derrick Mulder. 1. You cannot use rsync with these services. You need to use their apps to sync. 2. You will need to replace configs you messed up manually with mv. Or you can restore the whole system from a backup, by overwriting all files. But this way you will loss all the information generated after the backup time. So, moving is still better
Hi, thanx, but i would like to do a centos6 os backup, all .... not excluding nothing.
How do I do rsync with a seperate root and home partition
Can you restore the backup on a different pc, using this technique? I would like to change my laptop without reconfigure everything
Yes you can. You only will need to install GRUB from a live USB.
I ran the rsync command as presented with the destination to my USB stick. The dryrun showed NO error. But the real run did. See below. rsync: chown "/media/ng/KINGSTON32/opt/brackets/www/extensions/default/brackets-eslint/node_modules/es5-ext/string/#/.uncapitalize.js.BTCxGr" failed: Operation not permitted (1)
rsync: chown "/media/ng/KINGSTON32/opt/brackets/www/extensions/default/brackets-eslint/node_modules/es5-ext/string/#/@@iterator/.implement.js.EkPBd0" failed: Operation not permitted (1)
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at io.c(521) [generator=3.1.0]
Can you provide some insight into what went wrong please?
Hi. So a random question: my laptop died yesterday and I've just ordered another one. I did manage to recover my SSD from the dead laptop, so I'll just throw it into the new machine. So I'm wondering, will my Ubuntu installation work straight away or I will have to configure certain stuff?
If SSD wasn't damaged itself then, most likely, it will work straight away. If you switched from amd to nvidia or vice verse, then you'll need to reinstall video drivers. Also, if you used UEFI to boot - you'll need to create a new record for your new UEFI firmware, unless you had your bootloader .efi file in /EFI/BOOT/BOOTX64.EFI, which all UEFI firmwares read by default.
You can copy all files to a new system, but you will definitely need to install a boot-loader. Steps:
1. Boot from a Live Ubuntu.
2. Rsync the backup to a new system.
3. Install bootloader help.ubuntu.com/community/Grub2/Installing
Sir can you tell me how to increase the size of centos
От души делаешь)
how to exclude * .tar.gz files from various folders
If I restore to a new disk ? How Thank you!
Hei! kan you please make a video on start clonezilla from harddrive, so can boot eiter from normal linux or boot from clonezilla life standalone.
Thank you.
You are welcome
Hello. Can you make video about installing Arch with already installed Windows (bios)?
Maybe...
Ty
Can you please create an example "How to install samba and configure it with CLI and WEBUI" it may have something new in 2019
thank you
Hi, do you have a link to that Arch wallpaper? :-)
All wallpapers used in videos are copied to this G+ collection plus.google.com/b/105235169774065518394/collection/YlkQNE
Nice video
Thanks
please make a video for crontab and its best use
That was cool
Thanks
Great video! Can you do a video using "grsync"?
Thanks, I missed it...
grsync was not my favorite GUI for rsync, but I will have a look at it and present you what works the best in my view. I juts need one-two weeks.