Thanks as always, your videos help a lot of us get a better understanding of how this very cool technology works. Two reminders: A container is restarted when it migrates whereas a VM does not stop. A VM is migrated without any interruption. Use Ceph storage for your VMs so you do not need to replicate across the nodes. With Ceph, all three nodes will be updated concurrently, in real time, they will always be the same. Think of Ceph as RAID across nodes. In this way, your VM moves instantly without any data loss due to the replication period. Ceph works very well "on a budget". It does not need to be super performant to really earn is keep for HA VMs. Give Ceph a try. You'll never go back. :)
Cloud you do a video on how to setup cloudflare to connect to tailscale into Nginx trying to figure out how people do this so you can keep everything local without port forwarding stuff. I’m trying to do this with my nas tbh cloudlfare/tailscale/into nginx/to truenas.
HA + Very Low Budget - not sure those mix very well. The problem with the replication path is that the replica is only as good as the last update. That can be fine in the scenario you used (putting node into maintenance), where it can update the snapshot at that time before migrating. But a hardware failure pretty much guarantees that the snapshot won't be available to update, so a bit stale on the new node. Also it duplicates the storage requirement, so actually that means you need to be able to provision the storage on multiple nodes. The next step up from that is ZFS replication, but I think that's out of scope at this level of build. A better option under a budget is shared storage on a NAS - you can put your drive images on the NAS. This has downsides too - one is the network itself, and the overhead of that over a protocol like NFS or SMB. I would recommend at least 2.5GbE for a small cluster. The next step up is Ceph - and that's a whole different topic, but a ton of fun in a homelab. But the cost goes up a lot.
Ceph, the only way to fly! Ceph does not cost "a lot". High performance Ceph might, but the basic functionality of a replicated x3 pool for VM storage costs no more than the three locals he's using in this demo, and the Proxmox replication is not needed. Build the OSDs on LVM partitions, one on each node, big enough to hold your VMs. This will work just fine over a 1Gig network. They are concurrent in real time, so there is no replication lag. I've been doing this for months and can bounce the nodes up, down, and sideways all day long. The VM just won't quit. The migration happens in seconds with only milliseconds gap or pause when moving from one active node to another. The fact that VMs don't shut down when migrating convinced me to get away from containers for my core services. I run a network server VM with Pi-hole running DHCP and Unbound, a DDNS updater, Nginx reverse proxy, and Wireguard on 1GB of ram and 5GB of disk. The VM migrates in 4-5 seconds because there's only the state file to move, the Ceph storage is already there. :)
![Noob To Max With DRAGON REWORK In Blox Fruits [FULL MOVIE]](http://i.ytimg.com/vi/LnBMOinoOvA/mqdefault.jpg)
![Felix "Unfair" | [Stray Kids : SKZ-PLAYER]](http://i.ytimg.com/vi/Oswujxm2Ag0/mqdefault.jpg)
Thanks as always, your videos help a lot of us get a better understanding of how this very cool technology works.
Two reminders:
A container is restarted when it migrates whereas a VM does not stop. A VM is migrated without any interruption.
Use Ceph storage for your VMs so you do not need to replicate across the nodes. With Ceph, all three nodes will be updated concurrently, in real time, they will always be the same. Think of Ceph as RAID across nodes. In this way, your VM moves instantly without any data loss due to the replication period.
Ceph works very well "on a budget". It does not need to be super performant to really earn is keep for HA VMs.
Give Ceph a try. You'll never go back. :)
Thank You keep up the good work........
Thanks for the video. Could you pl add a link to Mac Mini model used in cluster. Thanks
Cloud you do a video on how to setup cloudflare to connect to tailscale into Nginx trying to figure out how people do this so you can keep everything local without port forwarding stuff. I’m trying to do this with my nas tbh cloudlfare/tailscale/into nginx/to truenas.
HA + Very Low Budget - not sure those mix very well. The problem with the replication path is that the replica is only as good as the last update. That can be fine in the scenario you used (putting node into maintenance), where it can update the snapshot at that time before migrating. But a hardware failure pretty much guarantees that the snapshot won't be available to update, so a bit stale on the new node. Also it duplicates the storage requirement, so actually that means you need to be able to provision the storage on multiple nodes. The next step up from that is ZFS replication, but I think that's out of scope at this level of build. A better option under a budget is shared storage on a NAS - you can put your drive images on the NAS. This has downsides too - one is the network itself, and the overhead of that over a protocol like NFS or SMB. I would recommend at least 2.5GbE for a small cluster. The next step up is Ceph - and that's a whole different topic, but a ton of fun in a homelab. But the cost goes up a lot.
Ceph, the only way to fly! Ceph does not cost "a lot". High performance Ceph might, but the basic functionality of a replicated x3 pool for VM storage costs no more than the three locals he's using in this demo, and the Proxmox replication is not needed.
Build the OSDs on LVM partitions, one on each node, big enough to hold your VMs. This will work just fine over a 1Gig network. They are concurrent in real time, so there is no replication lag.
I've been doing this for months and can bounce the nodes up, down, and sideways all day long. The VM just won't quit. The migration happens in seconds with only milliseconds gap or pause when moving from one active node to another.
The fact that VMs don't shut down when migrating convinced me to get away from containers for my core services. I run a network server VM with Pi-hole running DHCP and Unbound, a DDNS updater, Nginx reverse proxy, and Wireguard on 1GB of ram and 5GB of disk.
The VM migrates in 4-5 seconds because there's only the state file to move, the Ceph storage is already there. :)