So rude of me to say - 2 of my favorite homelabbers exchanging niceties when there are many other niceties,,I mean, homelab enthusiasts and RUclipsrs... but I am of the opinion there should be many more of you (unfortunately I had some Uni professors who were top researchers but lousy communicators - that is the reason your channels are jumping out). ... anyways, I didn't change my mind regarding shitification of the internet space and although we are witnessing a lot of newcomers (example - great security and VPN companies.. with quite decent free Tiers and even Open Source tech which is very important to us. Knowledge and hobby projects unfortunately are becoming (imho) somewhat of a existential threat. Imagine Google tomorrow saying they need to charge for email? So many user take it for granted and have their entire lives on their servers that 99.9% would pay. Hence the future of self-hosting is very important (or I'm just watching too much Louis Rossmann) for free and normal internet experience. I hope I'm just pissing against the wind... but what if I'm not? When I tell my friends I haven't seen any ad on my laptop last 4 years they laugh at me... (ok, I have so many blockers I cannot unblock even the creators I do want to support with running ads on their site - hmm.. this is then my apology). But life without ads is possible, especially if we gang together. Ok, main message conveyed and in utterly disorganized way because I'm typing as ideas emerge - so sorry for no structure. All the best gentleman. Cheers
Currently, I am deploying Kubernetes on 20 VMs as a homelab, spread across 10 servers (10 controllers and 10 workers). I have NGINX Ingress, MetalLB, and Cert Manager set up, along with some test deployments. At the same time, I am creating an Ansible playbook to automate the installation because, in the future, I would like to at least double the number of servers. I am currently working on optimizing the playbook to reduce network usage by downloading all packages on one node and copying them to the other nodes. What I have learned during this process is invaluable and worth the effort.
@@6713G Hi, to answer your questions. 1. Why 20VMs across 10 servers? It is beacouse I am working on ansible playbook and I want to test it with diffrent ammounts of nodes and in future I will have it on baremetal, so I want to find and repair all bugs and I have server at home as backup and at work as main, so I need some way to learn how to manage them and that means, I sometimes break something, so I have redundancy. 2. What am I wunning on homelab? Currently I have? - DB (mariaDB) - immich - fileshare - authentik and more. In future (max end of this year) I want to write a blog, so I will update it here. I have domain and everything ready, so I need to write some page to display blog posts, as I dont want to use wordpress, but for now it is possible, that I will use wordpress for some time.
@@christianlempa forgot to mention: I can't wait, because I've got two Pi-s some time ago, and they're still waiting for some workload. Gonna do something with it this summer, so you have perfect timing. But no pressure. ;-P PS. New subscriber, nice content! :-)
Love to see the shift to Kubernetes. 💖 Even if initially a bit harder to reason about than docker, after discovering kubernetes operators for DB management, I can't see myself going back to managing DBs in Docker anymore. And since most self hosted apps will require a DB - extra complexity pays of in in the long run.
Kubernetes operators for DB management? Which ones, that sounds interesting! So much stuff I want to run in my homelab is SQLite-only, I find myself dealing with the PV/PVC constructs in my homelab k8s cluster a lot of the time
SQLite is actually pretty good and easy to run, no shame in using that. For some more advanced use-cases I use PostgreSQL. An example of an operator I use is Crunchy Data's PGO (Postgres Operator). With just a simple YAML you can just spawn and scale PostgreSQL instances as you see fit. It also takes care of automating backups, users, databases and credentials etc. It does all the PV/PVCs the secrets, services etc. for you!. I actually have a video on that on my channel. There are a bunch of alternatives to PGO however I found this one works best for my use-case.
@@christianlempa I am using PostgreSQL if I have the chance and for that I use Crunchy Data PGO. For other databases, there is a company called Percona which makes all sorts of open source goodies for most databases out there! They also have Kubernetes operators for MySQL, so not exactly MariaDB but close. I think specifically for MariaDB there is an open source operator called mariadb-operator which seems to have all the features you'd want. But I couldn't tell you if it's any good or not.
Honestly after you get over the hurdle of installing them and configuring them, operators just seem like magic :) I also used operators for RabbitMQ, MongoDB and Redis and even entire apps like ArgoCD, Grafana, Keycloak etc.
Great videos as always! Maybe a topic could be logs, which of them are useful for cluster,which for apps and which we can discard. Maybe a third party solution to ingest them and check for security problems. Thank you!
Because there is a lot to learn about k8s, it is better to get started and actually operate it on the virtual machine. Actions speak louder than words. 🤣 Thanks for sharing. 👍 Can you make a video about K8S grayscale deployment and canary deployment?
Hey, someone have successfully deploy a k8s cluster using kubeadm on a cloud provider like azure or OpenStack ? I am on it since 2 weeks and I always have a problem, the pod to pod communication doesn’t works when they are in différents workers nodes. If someone can help me
Why MetalLB? Why not KeepAlived? This would allow you to move the IP for your control plane as well as your data plane. Also it keeps "OS" services out of your workload management tool, Kubernetes.
Nice! A couple of other areas.. multiple architecture worker nodes (building/deploying needed versions for each), simulate/have multiple sites (use labels for example for different sites) ha deployments across sites etc. if using secrets/configmaps... things like stakater/reloader (or even thinking about how to update/rotate certs/creds etc :) )
Good Video, I also started my own learning experience with k3s and kubeadm, and I defenetly learned a lot, but I had issues and complications when deploying k3s on diffrent distributions with diffrent networking stacks, as some of the tools would interact with iptables, some of them with nftables and some of them with ebpf, and in combination I often got broken network configurations. I do have to say it was an awesome experience, but might I point you at Sideolabs and Talos, I think it is a much superior but yet easier way to run kuberentes and get somewhat easy going. Have you already heard of it? And if yes, why do you not recommend it for beginners?
Hey so I know you read these comments so to keep this short... Can you cover Talos Linux and Sidero Omni? K3s is nice and all, but Talos is a completely new kind of beast, very impressive.
I had a look at it but so far I’m not sold. Docs and customization options for Omni are still terrible and understanding Talos is a different story. Maybe when I get more time I will look at it again
@@christianlempa Yeah so Omni is a bit new, but Talos itself is a fantastic platform. It takes a little bit to get your head around so that might be a challenge, what with just not having a shell at all, but I've been around since Talos .3 and the growth has been astounding. Thanks for reading btw. :)
i know kubernetes but in home lab i much more default to just using docker well most specifically docker-compose most commonly due to simplicity... since high availability is not important for my home use, and even when i used kubernetes for in home I've still used it as single node server so didn't see any point of using extra resources for kuberenetes as well as using more setup time to prepare additional yaml configs / or helm files to run some simple docker containers
YEah, I deployed k3s ad spent an inordinate aount of time switching all my stack from docker to Kubernetes... that was interesting. Totally overkill and from then I switched some stuf back to simple docker compose, but kept the Kubernetes cluster. It came with new problem I didn't know (or understand) I would have.. to solve. I had planned to do that on vacation... from a few days, took me three weeks. But... it's fun and overkill for homelab. Now I have to understand all that talk about self-healing capabilities because it's far from "self-healing".
The thing about HA in Kubernetes is you can't take any old application and make it HA. An application has to be designed in such a way that its amenable to it. For example, if an application only supports SQLite as its persistence mechanism, good luck!
@@apolloeosphoros4345 Yeah, I did get to that part... and was a bit disappointed on that front, actually. HA in a proxmox cluster setup with enough nodes was more helpful than doing what I did ;-) But... overall, it was definitely a good and interesting learning experience. I should do like Christian advised and start reading the documentation to familiarize with the environment.
@@floriantthebault521I've done a hybrid approach.. in that ha k3s, clustered proxmox with ceph storage), along with an external haproxy/keepalived/load balanced setup of pizeros ( usb serial ports for lighting system access) 😂
@@apolloeosphoros4345 some apps are "sort-off" HA just by being able to auto-restart in case of failure.... but in kubernetes it is slightly more than on docker-compose since it can restart it self on different host, unless you tainted it from doing so, and that is assuming that you have configured HA storage solution perhaps something with rook operator, lets say ceph, longhorn, gluster where each of them come with a lot of technical knowledge to use properly... i'm in this world for quiet long time but thrust me its a rabbit hole.
I've tried K3S on raspberry-pi some time ago, and encountered a sever problem: k3s doing su much i/o on the poor sd card that ssh could not function any more.
im very hopeful longhorn can run my workloads- im gonna find out soon- u got me curious now- was there a particular use case u had problems with christian or was it just buggy? i heard u *do not* wanna run out of space and that if the underlying disk needs fsck - u need to b careful - longhorn wont fix that for u- this is just my research- maybe beefs is another option? ill also be deploying minio- that should be fun 😁- grateful for ur🫵work and smile friend!🙏🥰🥳✌️
Just in the last few days I've been learning Kubernates, and I've just managed to get to stage four by deploying Nginx on Kubernetes, after seeing this video I feel like I'm on the right path.🥳
Awesome to see more love for Kubernetes. Great video, Christian.
Thank you so much! 😊 your videos are super cool 😎
So rude of me to say - 2 of my favorite homelabbers exchanging niceties when there are many other niceties,,I mean, homelab enthusiasts and RUclipsrs... but I am of the opinion there should be many more of you (unfortunately I had some Uni professors who were top researchers but lousy communicators - that is the reason your channels are jumping out).
... anyways, I didn't change my mind regarding shitification of the internet space and although we are witnessing a lot of newcomers (example - great security and VPN companies.. with quite decent free Tiers and even Open Source tech which is very important to us. Knowledge and hobby projects unfortunately are becoming (imho) somewhat of a existential threat.
Imagine Google tomorrow saying they need to charge for email? So many user take it for granted and have their entire lives on their servers that 99.9% would pay.
Hence the future of self-hosting is very important (or I'm just watching too much Louis Rossmann) for free and normal internet experience. I hope I'm just pissing against the wind... but what if I'm not? When I tell my friends I haven't seen any ad on my laptop last 4 years they laugh at me... (ok, I have so many blockers I cannot unblock even the creators I do want to support with running ads on their site - hmm.. this is then my apology). But life without ads is possible, especially if we gang together. Ok, main message conveyed and in utterly disorganized way because I'm typing as ideas emerge - so sorry for no structure.
All the best gentleman. Cheers
@@christianlempawhat is meaning lempa😊
Currently, I am deploying Kubernetes on 20 VMs as a homelab, spread across 10 servers (10 controllers and 10 workers). I have NGINX Ingress, MetalLB, and Cert Manager set up, along with some test deployments. At the same time, I am creating an Ansible playbook to automate the installation because, in the future, I would like to at least double the number of servers. I am currently working on optimizing the playbook to reduce network usage by downloading all packages on one node and copying them to the other nodes. What I have learned during this process is invaluable and worth the effort.
May i ask you something, but what is the use of 20 vm are you efficiently using all the resources and what are the selfhosted apps running
@@6713G Hi, to answer your questions.
1. Why 20VMs across 10 servers?
It is beacouse I am working on ansible playbook and I want to test it with diffrent ammounts of nodes and in future I will have it on baremetal, so I want to find and repair all bugs and I have server at home as backup and at work as main, so I need some way to learn how to manage them and that means, I sometimes break something, so I have redundancy.
2. What am I wunning on homelab?
Currently I have?
- DB (mariaDB)
- immich
- fileshare
- authentik
and more. In future (max end of this year) I want to write a blog, so I will update it here.
I have domain and everything ready, so I need to write some page to display blog posts, as I dont want to use wordpress, but for now it is possible, that I will use wordpress for some time.
Like your videos very much!
The "F" in Kubernetes stands for User-FRIENDLYNESS
😂
Friendlyless. ;)
Now I can't wait for Raspberry K8s cluster! ❤
Haha nice! coming soon :)
@@christianlempa forgot to mention: I can't wait, because I've got two Pi-s some time ago, and they're still waiting for some workload. Gonna do something with it this summer, so you have perfect timing. But no pressure. ;-P
PS. New subscriber, nice content! :-)
Love to see the shift to Kubernetes. 💖 Even if initially a bit harder to reason about than docker, after discovering kubernetes operators for DB management, I can't see myself going back to managing DBs in Docker anymore. And since most self hosted apps will require a DB - extra complexity pays of in in the long run.
Kubernetes operators for DB management? Which ones, that sounds interesting!
So much stuff I want to run in my homelab is SQLite-only, I find myself dealing with the PV/PVC constructs in my homelab k8s cluster a lot of the time
SQLite is actually pretty good and easy to run, no shame in using that.
For some more advanced use-cases I use PostgreSQL.
An example of an operator I use is Crunchy Data's PGO (Postgres Operator).
With just a simple YAML you can just spawn and scale PostgreSQL instances as you see fit. It also takes care of automating backups, users, databases and credentials etc. It does all the PV/PVCs the secrets, services etc. for you!.
I actually have a video on that on my channel.
There are a bunch of alternatives to PGO however I found this one works best for my use-case.
Awesome! That's actually a topic that I'd like to get into as well, which operators are you using for mariadb/mysql? (if you are)
@@christianlempa I am using PostgreSQL if I have the chance and for that I use Crunchy Data PGO.
For other databases, there is a company called Percona which makes all sorts of open source goodies for most databases out there! They also have Kubernetes operators for MySQL, so not exactly MariaDB but close.
I think specifically for MariaDB there is an open source operator called mariadb-operator which seems to have all the features you'd want. But I couldn't tell you if it's any good or not.
Honestly after you get over the hurdle of installing them and configuring them, operators just seem like magic :)
I also used operators for RabbitMQ, MongoDB and Redis and even entire apps like ArgoCD, Grafana, Keycloak etc.
Great videos as always!
Maybe a topic could be logs, which of them are useful for cluster,which for apps and which we can discard.
Maybe a third party solution to ingest them and check for security problems.
Thank you!
good topic!
k9s is my preferred management tool
nice :D
For me Longhorn is one of the best storages. With one single command you can have storage solution for your Kubernetes cluster
Because there is a lot to learn about k8s,
it is better to get started and actually operate it on the virtual machine.
Actions speak louder than words. 🤣
Thanks for sharing. 👍
Can you make a video about K8S grayscale deployment and canary deployment?
Hey, someone have successfully deploy a k8s cluster using kubeadm on a cloud provider like azure or OpenStack ? I am on it since 2 weeks and I always have a problem, the pod to pod communication doesn’t works when they are in différents workers nodes. If someone can help me
Why MetalLB? Why not KeepAlived? This would allow you to move the IP for your control plane as well as your data plane. Also it keeps "OS" services out of your workload management tool, Kubernetes.
Nice! A couple of other areas.. multiple architecture worker nodes (building/deploying needed versions for each), simulate/have multiple sites (use labels for example for different sites) ha deployments across sites etc. if using secrets/configmaps... things like stakater/reloader (or even thinking about how to update/rotate certs/creds etc :) )
Good Video,
I also started my own learning experience with k3s and kubeadm, and I defenetly learned a lot, but I had issues and complications when deploying k3s on diffrent distributions with diffrent networking stacks, as some of the tools would interact with iptables, some of them with nftables and some of them with ebpf, and in combination I often got broken network configurations. I do have to say it was an awesome experience, but might I point you at Sideolabs and Talos, I think it is a much superior but yet easier way to run kuberentes and get somewhat easy going.
Have you already heard of it? And if yes, why do you not recommend it for beginners?
Hi Christian, love the channel. It would be great if you could cover gatekeeper opa to allow one to set controls on a k3s cluster. Thank you
4:54 What are the 3 devices in the rack?
There are NVMe 'hats' on top but what's underneath them?
Hey so I know you read these comments so to keep this short...
Can you cover Talos Linux and Sidero Omni? K3s is nice and all, but Talos is a completely new kind of beast, very impressive.
I had a look at it but so far I’m not sold. Docs and customization options for Omni are still terrible and understanding Talos is a different story. Maybe when I get more time I will look at it again
@@christianlempa Yeah so Omni is a bit new, but Talos itself is a fantastic platform.
It takes a little bit to get your head around so that might be a challenge, what with just not having a shell at all, but I've been around since Talos .3 and the growth has been astounding.
Thanks for reading btw. :)
Sir like your content it's amazing ❤
Thank you so much 😀
i know kubernetes but in home lab i much more default to just using docker well most specifically docker-compose most commonly due to simplicity... since high availability is not important for my home use, and even when i used kubernetes for in home I've still used it as single node server so didn't see any point of using extra resources for kuberenetes as well as using more setup time to prepare additional yaml configs / or helm files to run some simple docker containers
YEah, I deployed k3s ad spent an inordinate aount of time switching all my stack from docker to Kubernetes... that was interesting. Totally overkill and from then I switched some stuf back to simple docker compose, but kept the Kubernetes cluster. It came with new problem I didn't know (or understand) I would have.. to solve. I had planned to do that on vacation... from a few days, took me three weeks. But... it's fun and overkill for homelab. Now I have to understand all that talk about self-healing capabilities because it's far from "self-healing".
The thing about HA in Kubernetes is you can't take any old application and make it HA. An application has to be designed in such a way that its amenable to it. For example, if an application only supports SQLite as its persistence mechanism, good luck!
@@apolloeosphoros4345 Yeah, I did get to that part... and was a bit disappointed on that front, actually. HA in a proxmox cluster setup with enough nodes was more helpful than doing what I did ;-)
But... overall, it was definitely a good and interesting learning experience. I should do like Christian advised and start reading the documentation to familiarize with the environment.
@@floriantthebault521I've done a hybrid approach.. in that ha k3s, clustered proxmox with ceph storage), along with an external haproxy/keepalived/load balanced setup of pizeros ( usb serial ports for lighting system access) 😂
@@apolloeosphoros4345 some apps are "sort-off" HA just by being able to auto-restart in case of failure.... but in kubernetes it is slightly more than on docker-compose since it can restart it self on different host, unless you tainted it from doing so, and that is assuming that you have configured HA storage solution perhaps something with rook operator, lets say ceph, longhorn, gluster where each of them come with a lot of technical knowledge to use properly... i'm in this world for quiet long time but thrust me its a rabbit hole.
Thanks Christian.
Please tell us about the Linux distribution for Kubernetes - Talos :)
Probably a project I will look at somewhere next year
Kube-doo!
do a updated video on argo cd !
I've tried K3S on raspberry-pi some time ago, and encountered a sever problem: k3s doing su much i/o on the poor sd card that ssh could not function any more.
That's why I used NVMEs for my RPi cluster, this is exactly the problem I wanted to avoid. Thanks for sharing :)
After almost trying all storage options, longhorn seems to be the best
Maybe I need to give it another try ;)
im very hopeful longhorn can run my workloads- im gonna find out soon- u got me curious now- was there a particular use case u had problems with christian or was it just buggy? i heard u *do not* wanna run out of space and that if the underlying disk needs fsck - u need to b careful - longhorn wont fix that for u- this is just my research- maybe beefs is another option? ill also be deploying minio- that should be fun 😁- grateful for ur🫵work and smile friend!🙏🥰🥳✌️
@christianlempa -- I don't think that Hostinger link works to click on. It's easy enough to type manually, of course.
Oops thank you for the heads up! Link repaired 😜
Any suggestion regards talos os
Not yet, it's something I probably will take a closer look at next year
@@christianlempa I am experimenti on it but I am in gettig-started
Just in the last few days I've been learning Kubernates, and I've just managed to get to stage four by deploying Nginx on Kubernetes, after seeing this video I feel like I'm on the right path.🥳
I use HA K3S, mysql database, METAL LB load balancer, Ingress traffic, and Rancher management tool
For ssl i use cert-manager with cloudflare dns challenge
Awesome! Good job :)
Installing k8s on-premise + configurating the ingress-controller => 💀 Easy my butt