Perfect timing for this video -- going to bail on the OMV box and build a second FreeNas box ... thank you for all your great videos -- you obviously enjoy your work !
Love your videos... been a FreeNAS fan for a decade or more now. They just work when you build them right, proper hardware you just forget they exist they run forever reliably.
Great explanation! For those interested, I just tried last night and the replication through the GUI works equally well to a generic linux ZFS dataset.
@@Felix-ve9hs but why? At that point buy a mini atx board or 8GB RPI and at least have another set of HDDs. You can even buy a quad HDD USB Enclosure and make the pi a secondary both local and remotely.
I got a good deal on 24 sas drives, it worked out around £5 per drive, it quickly turned into one of the most expensive set of drives when I had to get a system to put them all in. Ended up with a 720xd
It seems like everytime I'm learning something new in my homelab, you JUST posted a video about it. I just started looking into replications to an older end optiplex last night and you just answered some of my questions with a video today! Quit reading my mind! Or don't :D
Warmest wishes Tom. All the best for CNWT combo! 1:40 -- TN Replication requires the source and the destination to be on ZFS; what I needed to confirm clearly from an expert like you. For some use cases like cold storage, I prefer ext4 backup, for its universality and closer to simple PNP ability. Kindest regards, friends and neighbours.
You should show the new replication feature in 11.3 that lets you define different retention of snapshots on the back location. You just passed over it. Bit if it works as they describe that's a huge deal. It allows actual retention of backups without keeping those snapshots on the main box. Something that has been missing forever.
Greetings. I have a question and can't find a user friendly answer in RUclips. I have build my first Freenas from old PC with one 8TB disk, as time go on and I start loving my Freenas and migrating to Better hardware I bought same model HDD and now I want to Mirror my Pool. So my question is, how easy it is? What are the best Practices? Thanks for the great show.
Tom, With the rampant spread of cybercriminals using ransomware, maybe in one of your future videos you can address what solutions are available or that you have seen in creating air gaps(ie tape) in backup solutions for Freenas, zfs, etc.
This is an awesome video but I have some newbie questions. I want to try to do this between my FreeNAS server to a friend's FreeNAS folder (for offsite safety). 1) how to I secure the data so if someone were to steal my friends FreeNAS they can't access the data from the drives ( I assume this is just encrypting the pool) 2) can it be setup so the Admin on my Friend's FreeNAS can't read my data... I will have financial files and customer files in the pool... not that he would but this just eliminated the possibility Thanks in advance.
Great video, would be nice to see two more things: using the backup nas as a primary for a short time and then seeing the pull back to the primary. Second thing would be cleaning up months of snapshots, because that's a hell of a lot of snapshots. Can they all be merged back into the first snapshot after say a month? If I delete snapshots on the primary, is that reflected on the secondary machine?
Perfect! Could i feasibly replicate to another pool, eject the drive(s) and take them offsite? I don’t currently have a second freenas box and would like an offsite backup. I would use backblaze but my upload speed is far too slow.
What are the best practices? Using freenas as a Virtual Machine on Windows? using a VM over Linux? using Proxmox and virtualize a Ubuntu and install Freenas, using direct Freenas on a raid 1 with 2 ssd? If you have 2 servers to redundancy, use both as a mirror, share the data?
Would someone be so kind as to write out the general 'zfs' commands needed to achieve this in the command line? I'm guessing something like 'zfs send -R / | ssh user@ zfs recv /' with the 'ssh' part being (optional) encryption.
Thank you very much for this video! I have one question regarding the replication and its workings: If i were to set up a backup server with single disk vdevs because the machine would be only up and running once a week and for cost efficiency. If one of those drives would fail, would the replication need to restore the whole snapshot or only the lost data on said drive? (talking private use case here, nothing business related) thank you very much!
If you try a local replication (pool1 --> pool2 on the same machine) it doesn't work... (using the GUI); it works perfectly using CLI ZFS send/receive commands. So, iX scripts are not tested before putting them in production.
I have 2 pools in the same box one older smaller drives 3TB another new set which is 6TB. I deleted all the snapshots I had created a new one ran the replication over but only part of the files copy over.
Let's assume your main server explodes, is it possible to use the backup to restore not only the data, but also Jails, VMs, general FreeNAS settings and configurations ?
You can backup with a cronjob the configuration .db file of FreeNAS and then restore it when needed on the second machine or go the HA cluster way with TrueNAS, there's a video on this channel about it!
So, I have a zfs pool (striped) of one 4 tb drive on a XigmaNAS box. I am getting another 4 tb drive soon that I want to use for backup purposes. Can I set up that 2nd 4 tb drive as another zfs pool and replicate from one to the other? Or is there a more preferred approach>
one of the configuration items was the retention period (default of 2 weeks - yes it can be changed). So unless you are short of space, the answer is no.
Does using this method and taking snapshots backup everything? I just built my first TrueNAS server and I have a few jails. Nextcloud I use for family photos etc. Does everything get backed up? Or do I have to backup Nextcloud separately?
On my destination server, the replicated datasets arent accessable to shares or jails/plugins, they appear to be there, but the dataset doesnt seem accessable, even with 777 Ok, so, RTFM: i tested with a much smaller(not 20TiB) dataset, the dataset has to completely finish copying before any of the data is accessable on the destination server
Hi video is really good i setup my freenas server and create iscsi target now i want to setup replication of that iscsi to another freenas So that my iscsi targer will not go down if main server is down could you please help me in this .
Hi, thank you for the video on 11.3 replication. There is not a lot out there for 11.3 by now. earlier on 11.2 I used non root users for replicating my snapshots. This option is missing from the 11.3 Manual. Do you know if it is still possible to use non root users for the replication?
In over 4yrs, first sectors to be reported as unreadable on my Seagate NAS drives. Time to setup the replication server. Not that I'm too concerned about my data as it is stored on a z2 pool. ok, ok, I just want to buy some tech ! :)
Your video in past allowed me to set up replication. Question now. In original setup, the retention policy is set to none and machine is full with thousands of snapshots and sending replication fail notices. How can I edit the replication snapshot retention policy? When I change to same as source, it throws a dialogue box up with the following --- [EINVAL] replication_update.target_dataset: Target dataset should end with source dataset name without the first element (without pool name) for Legacy transport --- and will not allow. Thanks much for all you do
So if we use something like xcp-ng + xen orchestra with replication there, this is sort of useless, right? I mean it has its use case but it would be quicker to recover via xen orchestra rather than shutdown the problematic machine, go to the passive machine, and make an iscsi/nfs share, and then have to update things in your cluster.
The point of ZFS replication is to backup the data on your FreeNAS to another FreeNAS (or ZFS pool) But if you had your backups going from server A to server B and server A suffered a catastrophic failure, you could quickly be backup up and running by creating the shares on Server B and the data would be only as old as the last snapshot/sync task.
Can I have a main set up with two sections. Then a second back up that is off 99% of time that only backs up one section of the main? E.g. one section is stuff that want zero lose eg photos of kids and files. Then a second that is PLEX like media that can always get most again and no big loss or need for back up.
What about the replication of ONLY all the user accounts (including passwords) on the main nas? I have about 400 user accounts on the main nas, and if the main NAS goes up in a smoke cloud i want just to rip out the LAN cable out of it, change the IP address of the backup Nas to the IP address of the former main NAS and make it read/write, so that the user's experience nothing apart from a short LAN-Drive unavailability, and all of the SMB credentials wil work just fine.
I am confused. If a snapshot is only the changes made to the file system and a replication is only a copy of these snapshots, all that will be copied is changes to the original data and never the original data will be copied. So, suppose the source pool is destroyed, how will this make me able to reconstruct it?
If your true nas machine has DVD burner attach to it can you burn to M disk going term storage eg photos? Anyone know can only find about coping to else where not to a DVD drive?
I am currently running TrueNas would I be able to setup a Freenas box as the 2nd or slave box to do this? I ask as I know they do the Freenas updates before they release them to TrueNas. I am in the latest TrueNas Stable.
after trying this, I was getting SUCCESSFUL instantly on the srouce server. More research showed my snapshot tasks on the soruce server were PENDING. I delete the tasks, recreated... still PENDING. Anybody have thoughtss?
True, a celeron could do the job just fine I mean look at Synology NAS boxes that offer 4K streaming with a modern celeron! RAM on Synology machines however is pitiful and their Linux distros are very very closed in my opinion.
FreeNAS has such a beautiful UI, just WOW!
Perfect timing for this video -- going to bail on the OMV box and build a second FreeNas box ... thank you for all your great videos -- you obviously enjoy your work !
Every time I think of a new thing to implement and look for how to do it. You already have a wonderful step by step guide. THANKYOU!!!!
Love your videos... been a FreeNAS fan for a decade or more now. They just work when you build them right, proper hardware you just forget they exist they run forever reliably.
Hi Lawrence. Excellent Video, running FreeNAS since many years, but just did my first FreeNAS Replication setup for backups today . Well done, thanks
An excellent explanation of this replication. I’m ordering my backup freenas now.
We're here if you need some help with that!
For more information on FreeNAS (soon to be TrueNAS CORE), check out our site here: www.freenas.org
Thank you for the guide! Just setup my backup system according to the guide in 15 minutes. Now waiting for 9 TB of data to be copied over :-)
A big thank you. I did my first replication task ever from a 11.3 box to another 11.3 box. Both using zfs. It worked, it was easy, it was fast.
Great explanation! For those interested, I just tried last night and the replication through the GUI works equally well to a generic linux ZFS dataset.
Do you think it would be possible to replicate to a ZFS Pool on a external USB Drive on Linux?
@@Felix-ve9hs but why? At that point buy a mini atx board or 8GB RPI and at least have another set of HDDs. You can even buy a quad HDD USB Enclosure and make the pi a secondary both local and remotely.
I got a good deal on 24 sas drives, it worked out around £5 per drive, it quickly turned into one of the most expensive set of drives when I had to get a system to put them all in. Ended up with a 720xd
Thanks Tom for this. I got this working this morning replicating a TrueNas server onto a FreeNas server.
It seems like everytime I'm learning something new in my homelab, you JUST posted a video about it. I just started looking into replications to an older end optiplex last night and you just answered some of my questions with a video today!
Quit reading my mind! Or don't :D
Thanks Tom, great video, super easy to follow!!
Warmest wishes Tom. All the best for CNWT combo!
1:40 -- TN Replication requires the source and the destination to be on ZFS; what I needed to confirm clearly from an expert like you.
For some use cases like cold storage, I prefer ext4 backup, for its universality and closer to simple PNP ability.
Kindest regards, friends and neighbours.
CNWR sorry lol
Yes, replication requires ZFS at both ends.
@@LAWRENCESYSTEMS Thank you!
Thank you for the step-by-step instructions this is very helpful!!!
You should show the new replication feature in 11.3 that lets you define different retention of snapshots on the back location. You just passed over it. Bit if it works as they describe that's a huge deal. It allows actual retention of backups without keeping those snapshots on the main box. Something that has been missing forever.
for home use nothing beats this replication to the same system IMHO, no need for another box in the house :)
Greetings.
I have a question and can't find a user friendly answer in RUclips.
I have build my first Freenas from old PC with one 8TB disk, as time go on and I start loving my Freenas and migrating to Better hardware I bought same model HDD and now I want to Mirror my Pool. So my question is, how easy it is? What are the best Practices?
Thanks for the great show.
Tom, With the rampant spread of cybercriminals using ransomware, maybe in one of your future videos you can address what solutions are available or that you have seen in creating air gaps(ie tape) in backup solutions for Freenas, zfs, etc.
I always like your videos Lawrence. You do all the same things I do. Thanks for sharing.
Love your videos. This one helps me sleep.
This is an awesome video but I have some newbie questions. I want to try to do this between my FreeNAS server to a friend's FreeNAS folder (for offsite safety).
1) how to I secure the data so if someone were to steal my friends FreeNAS they can't access the data from the drives ( I assume this is just encrypting the pool)
2) can it be setup so the Admin on my Friend's FreeNAS can't read my data... I will have financial files and customer files in the pool... not that he would but this just eliminated the possibility
Thanks in advance.
Thanks for your wonderful videos. I’m learning so much.
Great video, would be nice to see two more things: using the backup nas as a primary for a short time and then seeing the pull back to the primary.
Second thing would be cleaning up months of snapshots, because that's a hell of a lot of snapshots. Can they all be merged back into the first snapshot after say a month? If I delete snapshots on the primary, is that reflected on the secondary machine?
Perfect! Could i feasibly replicate to another pool, eject the drive(s) and take them offsite? I don’t currently have a second freenas box and would like an offsite backup. I would use backblaze but my upload speed is far too slow.
What are the best practices? Using freenas as a Virtual Machine on Windows? using a VM over Linux? using Proxmox and virtualize a Ubuntu and install Freenas, using direct Freenas on a raid 1 with 2 ssd?
If you have 2 servers to redundancy, use both as a mirror, share the data?
Would someone be so kind as to write out the general 'zfs' commands needed to achieve this in the command line? I'm guessing something like 'zfs send -R / | ssh user@ zfs recv /' with the 'ssh' part being (optional) encryption.
Thank you very much for this video! I have one question regarding the replication and its workings:
If i were to set up a backup server with single disk vdevs because the machine would be only up and running once a week and for cost efficiency. If one of those drives would fail, would the replication need to restore the whole snapshot or only the lost data on said drive? (talking private use case here, nothing business related) thank you very much!
If you try a local replication (pool1 --> pool2 on the same machine) it doesn't work... (using the GUI); it works perfectly using CLI ZFS send/receive commands. So, iX scripts are not tested before putting them in production.
I have 2 pools in the same box one older smaller drives 3TB another new set which is 6TB. I deleted all the snapshots I had created a new one ran the replication over but only part of the files copy over.
An excellent explanation! Thanks for share!
Another #iXciting video, Tom!
how about Pull Replication between FreeNas as a primary server and TrueNas as a replication backup
Another great informative video. Thank you!
Pushing or pulling data - is one faster than the other?
3:26 OMG MY EYES!!!!!
Let's assume your main server explodes, is it possible to use the backup to restore not only the data, but also Jails, VMs, general FreeNAS settings and configurations ?
You can backup with a cronjob the configuration .db file of FreeNAS and then restore it when needed on the second machine or go the HA cluster way with TrueNAS, there's a video on this channel about it!
Please make video on *"How to restore if primary NAS destroyed"* because of any reason.
So, I have a zfs pool (striped) of one 4 tb drive on a XigmaNAS box.
I am getting another 4 tb drive soon that I want to use for backup purposes.
Can I set up that 2nd 4 tb drive as another zfs pool and replicate from one to the other? Or is there a more preferred approach>
good video, does anything need to be done to clean up those snapshots?
one of the configuration items was the retention period (default of 2 weeks - yes it can be changed). So unless you are short of space, the answer is no.
I made in FreeNAS 11.3 Virtual Machine (Debian 10) How is possible make snapshot (or backup) for it? And how restore it if I need it?
Does using this method and taking snapshots backup everything? I just built my first TrueNAS server and I have a few jails. Nextcloud I use for family photos etc. Does everything get backed up? Or do I have to backup Nextcloud separately?
is this meant that the files are also accessed and changed on BOX2? or is this more meant as a do not touch one way backup?
What if your destination pool is slightly smaller than your source? (21.1 tb to 20.9 tb?)
On my destination server, the replicated datasets arent accessable to shares or jails/plugins, they appear to be there, but the dataset doesnt seem accessable, even with 777
Ok, so, RTFM: i tested with a much smaller(not 20TiB) dataset, the dataset has to completely finish copying before any of the data is accessable on the destination server
I hope it works better on truenas because it's failed on me three times on freenas
Hi video is really good i setup my freenas server and create iscsi target now i want to setup replication of that iscsi to another freenas
So that my iscsi targer will not go down if main server is down could you please help me in this .
Hi, thank you for the video on 11.3 replication. There is not a lot out there for 11.3 by now. earlier on 11.2 I used non root users for replicating my snapshots. This option is missing from the 11.3 Manual. Do you know if it is still possible to use non root users for the replication?
Nice video, free nas is the best storage
In over 4yrs, first sectors to be reported as unreadable on my Seagate NAS drives. Time to setup the replication server. Not that I'm too concerned about my data as it is stored on a z2 pool. ok, ok, I just want to buy some tech ! :)
Your video in past allowed me to set up replication. Question now. In original setup, the retention policy is set to none and machine is full with thousands of snapshots and sending replication fail notices. How can I edit the replication snapshot retention policy? When I change to same as source, it throws a dialogue box up with the following --- [EINVAL] replication_update.target_dataset: Target dataset should end with source dataset name without the first element (without pool name) for Legacy transport --- and will not allow. Thanks much for all you do
Not sure, try their forums www.ixsystems.com/community/
So if we use something like xcp-ng + xen orchestra with replication there, this is sort of useless, right? I mean it has its use case but it would be quicker to recover via xen orchestra rather than shutdown the problematic machine, go to the passive machine, and make an iscsi/nfs share, and then have to update things in your cluster.
The point of ZFS replication is to backup the data on your FreeNAS to another FreeNAS (or ZFS pool) But if you had your backups going from server A to server B and server A suffered a catastrophic failure, you could quickly be backup up and running by creating the shares on Server B and the data would be only as old as the last snapshot/sync task.
Can I have a main set up with two sections. Then a second back up that is off 99% of time that only backs up one section of the main?
E.g. one section is stuff that want zero lose eg photos of kids and files. Then a second that is PLEX like media that can always get most again and no big loss or need for back up.
you can setup how ever many variations you want with replication per dataset
@@LAWRENCESYSTEMS is there a way to auto back up a mobile phone by any chance?
@@ausguy4385 not that I know of
@@LAWRENCESYSTEMS thanks, be great if was a way, since we all now manual back ups never happen till to late 🤦♂️
What about the replication of ONLY all the user accounts (including passwords) on the main nas? I have about 400 user accounts on the main nas, and if the main NAS goes up in a smoke cloud i want just to rip out the LAN cable out of it, change the IP address of the backup Nas to the IP address of the former main NAS and make it read/write, so that the user's experience nothing apart from a short LAN-Drive unavailability, and all of the SMB credentials wil work just fine.
400 local users? You should probably setup AD or equivalent.
can't you make one with backup to not freenas but an external usb disk or other type of nas that runs smb
There is not anything built into TruenNAS for that, it would take some third party tool or plugin.
I am confused. If a snapshot is only the changes made to the file system and a replication is only a copy of these snapshots, all that will be copied is changes to the original data and never the original data will be copied. So, suppose the source pool is destroyed, how will this make me able to reconstruct it?
It sends the snapshot and all the data leading up to the snapshot.
If your true nas machine has DVD burner attach to it can you burn to M disk going term storage eg photos?
Anyone know can only find about coping to else where not to a DVD drive?
My god... so many commercials... there were 4 commercial pop ups for me
RUclips is getting aggressive
I am currently running TrueNas would I be able to setup a Freenas box as the 2nd or slave box to do this? I ask as I know they do the Freenas updates before they release them to TrueNas. I am in the latest TrueNas Stable.
Yes, this works with TrueNAS too
@@LAWRENCESYSTEMS Thank you
after trying this, I was getting SUCCESSFUL instantly on the srouce server. More research showed my snapshot tasks on the soruce server were PENDING. I delete the tasks, recreated... still PENDING. Anybody have thoughtss?
Do you need the ssh to login as a root account or can you have it log in as a non-root account?
By default it only works with root
@@LAWRENCESYSTEMS
Isn't that a security concern? I was looking for an off site backup solution between my 2 truenas systems.
"This lower end server", ah yes the very low end i7 4770K desktop CPU from Intel. :D
:D well compared to a Server with a 24 Core / 48 Threads CPU and 64 GB of RAM
But IMHO this System would at least be midrange
True, a celeron could do the job just fine I mean look at Synology NAS boxes that offer 4K streaming with a modern celeron! RAM on Synology machines however is pitiful and their Linux distros are very very closed in my opinion.
definitely liked it better before the pluig in the beginning wasnt pre-recorded
I got tired of repeating myself, also if you have seen it before you know exactly how far to jump ahead :)