Thanks to You I now understand Traefik and almost hole my lab is on docker with traefik. Next step docker swarm and kubernetes :) good job like always!
@@Jims-Garage Which ubuntu 23.04 image did you specifically use? Just wondering? Is it this one (lunar-server-cloudimg-amd64.img) instead of (lunar-server-cloudimg-amd64-disk-kvm.img) Thanks
I am glad you and a few others (I found videos posted earlier this year) are talking about Docker Swarm. This technology should not be left behind because while Kubernetes has its place, it is overkill and unnecessarily complex for some applications. Docker Swarm and Gluster running on a clustered hypervisor is an awesome highly available solution. I hope your video brings a renewed interest in the continued development and support for Docker Swarm.
I completely agree. I've been using Docker standalone for a few years and recently upgraded to Docker Swarm with Gluster. I absolutely love it. I tried Kubernetes as well (K3S). I was able to setup Rancher with Longhorn for storage, but I wasn't loving it. It was overkill and extremely complex for what I wanted to do. While I won't be giving up on Kubernetes, I'm glad I went back and tried Docker Swarm after because I appreciated even more with how simple it is. I'm hoping that within a few years Kubernetes will become easier as innovations are introduced. Surely there will be improvements.
Thank you so much for this video. I was able to get the 5 node swarm cluster up and running, with Portainer with your script. Took a quite a bit to figure out the SSH files and stuff, but finally got it. Really appreciate the video, instructions, and script.
Hey Jim, thanks for posting this. I haven't used docker swarm in a long time because I thought it wasn't support any longer but I will just back in and give it go again. Thanks for your hard work sir. I made try to redeploy this into a LXD service as well.
@@Jims-Garagejust set it up recently. The only thing is you have to enable ip forward for ingress routing....google docker swarm lxc....running nicely.
Great video, been waiting for it for a while. A few questions/comments: - Why not use tail scale so all nodes would see each other and avoid opening ports across different geographic locations? - Why not use a proxmox container instead of the VMs? Any performance issues? - Why not create a container template and easily add a clone to the swarm? - What performance issues to expect with GlusterFS?
Thanks :) 1) This cluster is local, VPNs are not required. 2) LXCs are notorious for causing problems with the overlay network. Plus, VMs are more secure than LXCs due to using a different kernel. 3) I used cloud-init for creating the nodes, I mentioned this and referenced the video I have on it. 4) GlusterFS isn't the most performant, but as the data is replicated on each node it should be sufficient. If you want performance you're better off with Ceph or Kuberenetes.
Hey Jim. I had used docker swarm long time ago and was really impressed. Too bad it faded out of fashion. As you said, it would be a very good fit for homelabs.
@Jims-Garage so just replace with literally any other file syncing tool is my proposed solution. The files don't necessarily need to be clustered so long as they're synced. I could see that getting complicated with unbound files inside the containers. Mostly... I'm a fan of docker compose and I think you should do a video covering how to leverage compose with swarm.
@@Jims-Garage For me, I had to ask myself whether I actually required Swarm or needed high availability in the homelab setting, and it turned out I did not. What I cared about was the overlay network, but I now use 'traefik-kop' to connect containers on other machines to a single Traefik instance. And using Ansible roles to set containers up, it's all fine.
Really like all of your videos, your home lab like how I want mine to be, with the exception of the Sophos firewall!! (turned my XG into a PFSense box) Except I'm using XCP-ng at the moment as it seems to be more in use in the commercial environment than Proxmox.
Hi there! Just wanted to say a huge thank you @Jim's Garage for the awesome video on Docker Swarm and GlusterFS. I took a slightly different route and have now installed Docker Swarm, GlusterFS and Keepalived. I've got a little question about GlusterFS and Docker Bind-Mount and Volumes. I found some really helpful examples online, and they used bind mounts. But now I'm trying to use a volume and I've got a question about whether this is possible. I'd have to move /var/lib/docker to /mnt, which I'm a bit worried about as this directory has all my volumes and Docker configurations in it. Is this a good idea or is there a better way?I hope I've explained my situation clearly. Thank you so much for your help, I really appreciate it! Best wishes
@Jims-Garage I for the life of me can't get this to work. It's a great script, but I can't get the SSH auth to work properly. I get a permission denied error whether I use the public key from the Proxmox host or the "admin" VM. I do have a SSH public key for my laptop as part of the cloud-init process. That could be the problem but I'm pretty green with Linux. I don't want to manually run the install process from the script but that's where I'm at now. Help please...
I know this is an old video and I already got a K3s cluster with 3 masters and 3 workers on 3 nodes, but I wanted to try this docker swarm script. I needed to tweak it a little to work on my homelab env but I did make it work. Now, one thing I see in my case is that I do have all my agents installed in all nodes and I can see them ok in portainer but looks like theres multiple agents running on some nodes. For example in nodes 2,4,5 is running 2 agents nodes.
This sounds like a silly question, being that I'm coming to this about a year later: What about a local Syncthing container on each node that just syncs the bind mounts to each device in the cluster?
I tried following your kubernetes videos, ended with only nginx running, too complicated to go immediately after docker (sooner or latter I'll understand it better, but now right now). I will be trying this docker swarm very soon.
Thanks for the excellent explanation. I have a question, after all the role of Docker Swarm is to continue work on another server. According to your video everything is done from one server. How to connect another server and install Docker Swarm
HI, can you help me please? I can't understand how I should go about creating the 2 files id_rsa and id_rsa.pub If I go to see the proxmox certificates what should I do? Do I copy and paste the certificates into a file and save it naming it id_rsa? If I copy and paste the proxmox certificate (which of the 2?) onto the SSH public key of cloud-init it gives me an error, what should I do? Please help me! Thank you!
@@Jims-Garage Yes, but it is the same certificate that I find in the proxmox certificates menu but it cannot be used in the cloud-init settings, I need id_rsa and id_rsa.pub ( minute 16.10 of the video ) but I can't find them or...... generate them?
thank you Jim i have a question concerning the certificates i have a 5 server cluster and want to spread the load over the several servers ergo than i need all 5 different certificates to be specified or can i use only one from my proxmox cluster master?
The best way is perhaps to generate your own key and use that. Otherwise, create on a single host and then migrate the VMs. Should honour the original key.
@@Jims-Garage thank you again Jim and will do the second option you suggested but first need to create 5 cloud-init servers...thank you for that tutorial too
Great stuff man thanks for sharing! Quick question though what if we have 3 proxmox nodes with ceph enabled and setup 3 docker managers and 3 workers on each of the proxmox nodes; from storage perspective do I need GlisterFS if I mount ceph using ceph-fuse package as a second storage for the VMs?
Great video! The script executes fine and the logs look great but cant access portainer webui on any ports that are presented. Any ideas? Portainer agent are running on all and main portainer on one of the nodes from what i can see. Backed a snapshot and tried again with the same result...
Thanks for a very nice walkthrough of Docker Swarms, I will for sure try it. How come you have 3 managers, and 2 workers? Wouldn't 2 managers be enough? To do failover?
Thanks so much. This is very interesting and I think swarm is so much easier then K18n. However it seems that GlusterFS is not actively maintained, or at a very low volume of commits. I read that RedHat decided to stop it's active development. So would you still recommend it ?
Hello, if i create vms with virtulbox, i don't understand what i should do with the ssh key, does all the machines needs to have the same ssh key then ?
Hey Jim. I get it, Docker Swarm benefits with a number of worker nodes and all that jazz, sure. But probably, I'm missing some distinctive detail, the most important thing - what are the benefits of a virtual cluster, whose nodes resides on the same physical server? How this HA setup will benefit, for example, when file system starts to fail or some RAM issues occurs on the physical server itself? Or are these kind of setup designed to solve other problems, not related with some hardware issues?
Hey, you raise a valid point. There's less point on the same machine, but if you've watched my Kubernetes videos you will know that I split between physical Proxmox nodes. This gives all of the benefits you are asking about. On the same machine, the best use is that you can reboot the VM for updates without breaking your services.
I've set this up in a test lab with a 3-node manager cluster and use NFS as the central storage for volumes. Works great in my testing, but now I'm wondering how easy it would be to migrate an existing standalone environment into the cluster? Any comments on this aspect?
@@Jims-Garage OK, but is that just for stacks or does that apply to containers as well? I'm testing "SnapDrop" right now which was created as a container; it does not seem to replicate like stacks do. Would I need to deplay everything using a stack? Actually, I just discovered that you need to add these as a "Service" so that you can then indicate whether they are Global or Replicated. My SnapDrop test is working great now. Thanks for your awesome video - docker swarm is still useful for the homelab, that's for sure!!!
Hi Jim, Thanks for the video. I have a question about portainer; since the free version supports 3 nodes only, did you buy a license to support 5 nodes? Or is it possible to support 5 nodes without purchasing a license?
Finally tried to take this on today (2 Node Proxmox/CEPH). I tried with cloud image VM and they just hang on the -serial0 message when loading up the console. Tried with LXC containers, they will spin up and I can manually install docker and such, but the script from github appears to work, hangs a bit prompts me for sudo but the password appears in plain text and you have to ctrl+c to terminate. So I am going to go to bed and try again in the morning.
@@Jims-Garage after I said I was going to bed, I kept at is. No luck with VMs but in LXC containers they cannot be “unprivileged” or you cannot create the volume. I seem to have everything working except the mount itself. I was able to write to fstab, but the actual “mount.glusterfs … /mnt” returns a mount failed error. So that’s the next rabbit hole to dive into. And then of course back tracking to figuring out the VM side of things I wonder if it’s the issue you faced in the stream with the kvm image (I haven’t watched that yet) but my next order of trouble shooting it to you the regular cloud image.
@@Jims-Garage came back to it. I have it working on Proxmox VM's now. 3 managers 3 workers. might add 3 more workers (1 manager 2 workers per Proxmox nodes? maybe? )
As always.....Good job man! I have a question: When I have configured everything as in this video, can I then install a Docker application directly using Portainer and I will have all the benefits such as HA etc? Second thing: have you ever seen SWARMPIT? Maybe you've already seen it but it looks like a very nice GUI to "handle" all the Docker Swarm! What do you think?
@@Jims-Garage OK, great, but can I use a docker compose yml file as if I installed the application on "single" docker or do I have to make some changes in the yml file? I see that Portainer when I create a new stack asks me where I want to install it for example on Worker-1... is that enough? Thanks again for your patience!
@@Jims-Garage Hi, I'm trying to implement a yml file to install pihole via portainer but I can't get it to work. Do you have any references that I can follow to be able to install it correctly on the SWARM + GlusterFS cluster? Please help me, thanks!
Gluster is basically end-of-life (in a year) so I wouldn’t base a new setup on it, I think there are better options. It also has terrible performance in my experience, but that might be related to the amount of data.
@@Jims-Garage it isn’t really a matter of performance. Just like you wouldn’t install Windows 7 on a new computer, you shouldn’t build a new setup using Gluster. It’s fine to keep using it if you already are, but don’t invest energy into building on it.
@@Jims-Garage First, thanks for a great video! In the process of setting up something new, do you have suggestion(s) for an alternative to Glusterfs for a hybrid CPU (arm, amd, etc) set up? Maybe BeeGFS?
It should work with databases with replicas set to 1. As with all databases it's recommended to backup using database tooling, and if you can't you should then stop the service first.
Hey Jim. I was wondering if it's possible to run something like gluetun and qbittorrent in swarm. I can't seem to get it working. I tried using the shadowsocks proxy that comes with gluetun but cannot seem to get it working. Either way thanks fora great video :) Subscribed
If you have the time and resources then k3s (or preferably RKE2 - I have a video) is better. However, docker swarm is a viable option for a homelab that is far simpler and does most of what you'd want.
I have checked out your RKE2 video yesterday. Very nicely explained how to set it up, thank you. But it seems too resource intensive for a small homelab. I like docker and the simplicity of it, if it can be set up for HA and LB I think I’d give it a go.
With the news of RH dropping support for Gluster, do you still think this is a viable path to start on if one is about to deploy a network storage solution from scratch?
@@Jims-Garage My goal is to set up a HA 5-node cluster for file storage and backup. Looking like Proxmox VE with CEPH may check the boxes, I'll just have a lot of learning ahead.
GlusterFS performance is pathetic even after allocating a separate physical network for its brick. Secondly, its quite heavy on processor. I would certainly look into any other fs giving similar features.
Thanks to You I now understand Traefik and almost hole my lab is on docker with traefik. Next step docker swarm and kubernetes :) good job like always!
That's awesome, thanks.
@@Jims-Garage Which ubuntu 23.04 image did you specifically use? Just wondering? Is it this one (lunar-server-cloudimg-amd64.img) instead of (lunar-server-cloudimg-amd64-disk-kvm.img) Thanks
@@chrisumali9841 cloud-images.ubuntu.com/lunar/current/lunar-server-cloudimg-amd64.img
I am glad you and a few others (I found videos posted earlier this year) are talking about Docker Swarm. This technology should not be left behind because while Kubernetes has its place, it is overkill and unnecessarily complex for some applications. Docker Swarm and Gluster running on a clustered hypervisor is an awesome highly available solution. I hope your video brings a renewed interest in the continued development and support for Docker Swarm.
Thanks, appreciate the kind words. I completely agree, Swarm is probably the sweet spot for homelab use.
I completely agree. I've been using Docker standalone for a few years and recently upgraded to Docker Swarm with Gluster. I absolutely love it.
I tried Kubernetes as well (K3S). I was able to setup Rancher with Longhorn for storage, but I wasn't loving it. It was overkill and extremely complex for what I wanted to do. While I won't be giving up on Kubernetes, I'm glad I went back and tried Docker Swarm after because I appreciated even more with how simple it is.
I'm hoping that within a few years Kubernetes will become easier as innovations are introduced. Surely there will be improvements.
Thanks for the demo and info, Happy Holidays, and have a great day.
Happy holidays!
Thank you so much for this video. I was able to get the 5 node swarm cluster up and running, with Portainer with your script. Took a quite a bit to figure out the SSH files and stuff, but finally got it. Really appreciate the video, instructions, and script.
Glad it helped!
Thank you, one of best in homelab, As always, excellent deployment process with detailed explanation.
Very kind, thanks!
Hey Jim, thanks for posting this. I haven't used docker swarm in a long time because I thought it wasn't support any longer but I will just back in and give it go again. Thanks for your hard work sir. I made try to redeploy this into a LXD service as well.
Thanks. From everything I've heard LXC doesn't play nicely with the overlay network.
@@Jims-Garagejust set it up recently. The only thing is you have to enable ip forward for ingress routing....google docker swarm lxc....running nicely.
Great video, been waiting for it for a while. A few questions/comments:
- Why not use tail scale so all nodes would see each other and avoid opening ports across different geographic locations?
- Why not use a proxmox container instead of the VMs? Any performance issues?
- Why not create a container template and easily add a clone to the swarm?
- What performance issues to expect with GlusterFS?
Thanks :)
1) This cluster is local, VPNs are not required.
2) LXCs are notorious for causing problems with the overlay network. Plus, VMs are more secure than LXCs due to using a different kernel.
3) I used cloud-init for creating the nodes, I mentioned this and referenced the video I have on it.
4) GlusterFS isn't the most performant, but as the data is replicated on each node it should be sufficient. If you want performance you're better off with Ceph or Kuberenetes.
Awesome video. Thank you for being thorough.
Glad it was helpful!
Glad it was helpful!
Thanks Jim, nice job 🥷
Thanks 👍
Hey Jim. I had used docker swarm long time ago and was really impressed. Too bad it faded out of fashion. As you said, it would be a very good fit for homelabs.
Totally agree. It's super lightweight in comarison.
Excellent, thank you!
You are welcome!
This project is end of life. Do you have an updated video @Jim's Garage ?
Unfortunately I do not and I'm not sure if there is a common replacement. If you know of something then I am happy to update it.
@@Jims-Garage What exactly became EoL?
@joediliberto6244 glusterFS
@Jims-Garage so just replace with literally any other file syncing tool is my proposed solution. The files don't necessarily need to be clustered so long as they're synced. I could see that getting complicated with unbound files inside the containers.
Mostly... I'm a fan of docker compose and I think you should do a video covering how to leverage compose with swarm.
@@Jims-Garage For me, I had to ask myself whether I actually required Swarm or needed high availability in the homelab setting, and it turned out I did not. What I cared about was the overlay network, but I now use 'traefik-kop' to connect containers on other machines to a single Traefik instance. And using Ansible roles to set containers up, it's all fine.
Really like all of your videos, your home lab like how I want mine to be, with the exception of the Sophos firewall!! (turned my XG into a PFSense box) Except I'm using XCP-ng at the moment as it seems to be more in use in the commercial environment than Proxmox.
Thanks. Both of those are solid alternatives, good work
Excellent walk through! 🎉
Thanks
You can have a 3 node cluster, with all 3 being managers/worker combo’s.. and you always need an uneven number of managers for quorum consistency.
Thanks, Neil. That's good to know for people who want a 3-node setup.
Yes that's exactly what I am looking for! Further I need a script similar to what was in video for 5 nodes, where can I find it?
Hi there!
Just wanted to say a huge thank you @Jim's Garage for the awesome video on Docker Swarm and GlusterFS. I took a slightly different route and have now installed Docker Swarm, GlusterFS and Keepalived. I've got a little question about GlusterFS and Docker Bind-Mount and Volumes.
I found some really helpful examples online, and they used bind mounts. But now I'm trying to use a volume and I've got a question about whether this is possible. I'd have to move /var/lib/docker to /mnt, which I'm a bit worried about as this directory has all my volumes and Docker configurations in it. Is this a good idea or is there a better way?I hope I've explained my situation clearly.
Thank you so much for your help, I really appreciate it!
Best wishes
Great video! I want to do this in my VMware home lab.
You should!
@Jims-Garage I for the life of me can't get this to work. It's a great script, but I can't get the SSH auth to work properly. I get a permission denied error whether I use the public key from the Proxmox host or the "admin" VM. I do have a SSH public key for my laptop as part of the cloud-init process. That could be the problem but I'm pretty green with Linux. I don't want to manually run the install process from the script but that's where I'm at now. Help please...
I know this is an old video and I already got a K3s cluster with 3 masters and 3 workers on 3 nodes, but I wanted to try this docker swarm script. I needed to tweak it a little to work on my homelab env but I did make it work. Now, one thing I see in my case is that I do have all my agents installed in all nodes and I can see them ok in portainer but looks like theres multiple agents running on some nodes. For example in nodes 2,4,5 is running 2 agents nodes.
amazing
all my project now running on docker swarm even wordpress
@@VividAstral great! Enjoy the redundancy 👍
This sounds like a silly question, being that I'm coming to this about a year later: What about a local Syncthing container on each node that just syncs the bind mounts to each device in the cluster?
It might work. You could likely use NFS or iSCSI as well.
I tried following your kubernetes videos, ended with only nginx running, too complicated to go immediately after docker (sooner or latter I'll understand it better, but now right now).
I will be trying this docker swarm very soon.
Great, it's a good introduction into the world of container orchestration. Hop on Discord if you're stuck.
Thanks for the excellent explanation.
I have a question, after all the role of Docker Swarm is to continue work on another server. According to your video everything is done from one server. How to connect another server and install Docker Swarm
As long as networks are routable and the right ports are open, nodes can be on different machines and networks.
HI, can you help me please? I can't understand how I should go about creating the 2 files id_rsa and id_rsa.pub
If I go to see the proxmox certificates what should I do? Do I copy and paste the certificates into a file and save it naming it id_rsa?
If I copy and paste the proxmox certificate (which of the 2?) onto the SSH public key of cloud-init it gives me an error, what should I do?
Please help me! Thank you!
Either create your own and add them, or go to the root home folder and find the certs.
@@Jims-Garage I tried to see inside the proxmox's root folder but it's empty.....i don't know where can i find these files....
@@zippi777 I think it's here /etc/pve/nodes/NODENAME/pve-ssl.pem
@@Jims-Garage Yes, but it is the same certificate that I find in the proxmox certificates menu but it cannot be used in the cloud-init settings, I need id_rsa and id_rsa.pub ( minute 16.10 of the video ) but I can't find them or...... generate them?
thank you Jim
i have a question concerning the certificates
i have a 5 server cluster and want to spread the load over the several servers ergo than i need all 5 different certificates to be specified or can i use only one from my proxmox cluster master?
The best way is perhaps to generate your own key and use that. Otherwise, create on a single host and then migrate the VMs. Should honour the original key.
@@Jims-Garage thank you again Jim and will do the second option you suggested but first need to create 5 cloud-init servers...thank you for that tutorial too
Great stuff man thanks for sharing! Quick question though what if we have 3 proxmox nodes with ceph enabled and setup 3 docker managers and 3 workers on each of the proxmox nodes; from storage perspective do I need GlisterFS if I mount ceph using ceph-fuse package as a second storage for the VMs?
Ceph is likely a better option, and especially now that glusterfs is being retired. I'm planning to move onto it later.
Great video! The script executes fine and the logs look great but cant access portainer webui on any ports that are presented. Any ideas? Portainer agent are running on all and main portainer on one of the nodes from what i can see. Backed a snapshot and tried again with the same result...
@@linusgrip6400 did you use https and the correct port?
@@Jims-Garage yes i did. Tried every node.
@@Jims-Garage Spun up a vms from cloud init noble ubuntu 24.04 instead of jammy 22.04 and now it works
Thanks for a very nice walkthrough of Docker Swarms, I will for sure try it.
How come you have 3 managers, and 2 workers?
Wouldn't 2 managers be enough? To do failover?
@@ReinkeDK you need quorum (read up on that term). It requires a minimum of 3 as there always needs to be consensus
@@Jims-Garage Thanks for the reply,
I will definitely read up on it :)
Thanks so much. This is very interesting and I think swarm is so much easier then K18n. However it seems that GlusterFS is not actively maintained, or at a very low volume of commits. I read that RedHat decided to stop it's active development. So would you still recommend it ?
@@hithere626 no, you probably want to move to a different storage provider. Something like ceph for example
Hello, if i create vms with virtulbox, i don't understand what i should do with the ssh key, does all the machines needs to have the same ssh key then ?
Yes, use the same SSH key for all of them. You can create your own if you want to.
Hey Jim.
I get it, Docker Swarm benefits with a number of worker nodes and all that jazz, sure.
But probably, I'm missing some distinctive detail, the most important thing - what are the benefits of a virtual cluster, whose nodes resides on the same physical server?
How this HA setup will benefit, for example, when file system starts to fail or some RAM issues occurs on the physical server itself? Or are these kind of setup designed to solve other problems, not related with some hardware issues?
Hey, you raise a valid point. There's less point on the same machine, but if you've watched my Kubernetes videos you will know that I split between physical Proxmox nodes. This gives all of the benefits you are asking about. On the same machine, the best use is that you can reboot the VM for updates without breaking your services.
@@Jims-Garage Turns out, I learned something, huh. Thank you for your videos and this clarification.
I've set this up in a test lab with a 3-node manager cluster and use NFS as the central storage for volumes. Works great in my testing, but now I'm wondering how easy it would be to migrate an existing standalone environment into the cluster? Any comments on this aspect?
Should be a case of copying the data over and amending the deployment yaml. Should be easy to do.
@@Jims-Garage OK, but is that just for stacks or does that apply to containers as well? I'm testing "SnapDrop" right now which was created as a container; it does not seem to replicate like stacks do. Would I need to deplay everything using a stack?
Actually, I just discovered that you need to add these as a "Service" so that you can then indicate whether they are Global or Replicated. My SnapDrop test is working great now.
Thanks for your awesome video - docker swarm is still useful for the homelab, that's for sure!!!
@@scubeedu2 you need to deploy using the swarm yaml template. It looks a little different to a standalone deployment.
Hi Jim, Thanks for the video. I have a question about portainer; since the free version supports 3 nodes only, did you buy a license to support 5 nodes? Or is it possible to support 5 nodes without purchasing a license?
I use the community edition, there's no limit for nodes.
Finally tried to take this on today (2 Node Proxmox/CEPH). I tried with cloud image VM and they just hang on the -serial0 message when loading up the console. Tried with LXC containers, they will spin up and I can manually install docker and such, but the script from github appears to work, hangs a bit prompts me for sudo but the password appears in plain text and you have to ctrl+c to terminate.
So I am going to go to bed and try again in the morning.
Let me know how you get on
@@Jims-Garage after I said I was going to bed, I kept at is. No luck with VMs but in LXC containers they cannot be “unprivileged” or you cannot create the volume.
I seem to have everything working except the mount itself. I was able to write to fstab, but the actual “mount.glusterfs … /mnt” returns a mount failed error.
So that’s the next rabbit hole to dive into. And then of course back tracking to figuring out the VM side of things I wonder if it’s the issue you faced in the stream with the kvm image (I haven’t watched that yet) but my next order of trouble shooting it to you the regular cloud image.
@@Jims-Garage came back to it. I have it working on Proxmox VM's now. 3 managers 3 workers. might add 3 more workers (1 manager 2 workers per Proxmox nodes? maybe? )
As always.....Good job man! I have a question: When I have configured everything as in this video, can I then install a Docker application directly using Portainer and I will have all the benefits such as HA etc?
Second thing: have you ever seen SWARMPIT? Maybe you've already seen it but it looks like a very nice GUI to "handle" all the Docker Swarm!
What do you think?
Yes, installing via Portainer will give you the benefits of swarm. I haven't used swarmpit as I find Portainer to be perfect.
@@Jims-Garage OK, great, but can I use a docker compose yml file as if I installed the application on "single" docker or do I have to make some changes in the yml file?
I see that Portainer when I create a new stack asks me where I want to install it for example on Worker-1... is that enough?
Thanks again for your patience!
@@Jims-Garage Hi, I'm trying to implement a yml file to install pihole via portainer but I can't get it to work. Do you have any references that I can follow to be able to install it correctly on the SWARM + GlusterFS cluster?
Please help me, thanks!
Gluster is basically end-of-life (in a year) so I wouldn’t base a new setup on it, I think there are better options. It also has terrible performance in my experience, but that might be related to the amount of data.
I agree, there are more performant options, but it's a good option to get started. I'll look to replace glusterfs in the future.
@@Jims-Garage it isn’t really a matter of performance. Just like you wouldn’t install Windows 7 on a new computer, you shouldn’t build a new setup using Gluster. It’s fine to keep using it if you already are, but don’t invest energy into building on it.
Oh really, I was about to setup my own Now! Is there a suggestion for me? Which one should I go for in docker swarm if not glusterfs?
@@Jims-Garage First, thanks for a great video! In the process of setting up something new, do you have suggestion(s) for an alternative to Glusterfs for a hybrid CPU (arm, amd, etc) set up? Maybe BeeGFS?
@@raymerp9 thanks, sadly not anything I have experience with as I'm x86
Can i use the same script even if i run the nodes n esxi instead of proxmox?
Yes, should work.
Cool, does GlusterFS work also with databases? Do you need to turn off the cluster to do system updates?
It should work with databases with replicas set to 1. As with all databases it's recommended to backup using database tooling, and if you can't you should then stop the service first.
Hey Jim.
I was wondering if it's possible to run something like gluetun and qbittorrent in swarm. I can't seem to get it working. I tried using the shadowsocks proxy that comes with gluetun but cannot seem to get it working. Either way thanks fora great video :) Subscribed
Thanks, it should certainly be possible but I haven't done it I'm afraid. I use Kubernetes
Could you explain how to fit traefik into this docker swarm setup?
I'll likely come back to swarm in the near future.
I’m adding this to the watchlist to check out later. Is this viable for homelab deployment? Everyone seems to suggest k3s lately.
If you have the time and resources then k3s (or preferably RKE2 - I have a video) is better. However, docker swarm is a viable option for a homelab that is far simpler and does most of what you'd want.
I have checked out your RKE2 video yesterday. Very nicely explained how to set it up, thank you. But it seems too resource intensive for a small homelab. I like docker and the simplicity of it, if it can be set up for HA and LB I think I’d give it a go.
@@NikolaNovoselec swarm will give you that
With the news of RH dropping support for Gluster, do you still think this is a viable path to start on if one is about to deploy a network storage solution from scratch?
No. I'd look into ceph for now (albeit Kubernetes is perhaps where you might want to be)
@@Jims-Garage Ugh, those both sound way above my skill level. Thanks for the gut check though!
@@RyanMeray if you simply want to play with swarm and learn fundamentals this will be fine. It's just not something I recommend building around.
@@Jims-Garage My goal is to set up a HA 5-node cluster for file storage and backup. Looking like Proxmox VE with CEPH may check the boxes, I'll just have a lot of learning ahead.
Why not using three managers who also will do workloads?
It's good practice to split them out, but for a homelab it's probably okay to amalgamate.
@@Jims-Garage yes, you're right about best practice, my question was focused on homelab usage.
how to setup 5 nodes with proxmox ?
Check my Kubernetes video on cloud-init. That will show you how to create multiple VMs quickly.
GlusterFS performance is pathetic even after allocating a separate physical network for its brick. Secondly, its quite heavy on processor. I would certainly look into any other fs giving similar features.
Local performance seems acceptable though?
I am curious, do you have any benchmark available? Would NFS be a better solution?
first xD
My hero 😻