Building a 10-Node Kubernetes Cluster on Raspberry Pi & Ubuntu Server
HTML-код
- Опубликовано: 15 июл 2024
- Have you thought about setting up your very own Kubernetes cluster consisting of multiple Raspberry Pi's? It's not as hard as it sounds, and in this video, I'll show you how to set it up. Although this video will show the process of creating a ten node cluster, you don't have to have 10 nodes - as long as you have at least two, you'll be all set. By the end of the video, you'll have your very own Kubernetes cluster that will be ready to go for my upcoming Kubernetes tutorial series.
Time codes
0:00:00 - Intro
0:02:32 - Getting Started
0:07:30 - Writing the SD Card
0:14:12 - Building the rack
0:28:22 - Initial Pi setup
0:40:03 - Installing Docker
0:59:59 - Launching containers
Support LearnLinuxTV (commission earned)
📖 Check out Jay's latest book, Mastering Ubuntu Server 4th Edition. Covers Ubuntu 22.04!
➡️ ubuntuserverbook.com
🙌 Support me on Patreon and get early access to new content!
➡️ learnlinux.link/patron
☁️ Support LearnLinuxTV and Set up your own cloud server with Akamai Connected Cloud:
➡️ learnlinux.link/akamai
🛒 Affiliate store for Linux compatible hardware/accessories (commission earned):
➡️ learnlinux.link/amazon
💻 Check out the Tiny Pilot KVM for your Homelab (commission earned):
➡️ learnlinux.link/tinypilot
Links specific to this video
🐧 Wiki article (contains all the commands used in this video):
➡️ www.learnlinux.tv/?p=961
🐧 Components used in this build (commission earned):
➡️ kit.co/jay_lltv/raspberry-pi-...
🐧 Download Ubuntu Server for Raspberry Pi:
➡️ ubuntu.com/download/raspberry-pi
🐧 Download usbimager:
➡️ bztsrc.gitlab.io/usbimager/
🐧 Also, check out my series on Docker:
➡️ learnlinux.link/learn-docker
LearnLinuxTV Links
🐧 Main site:
➡️ www.learnlinux.tv
🐧 LearnLinuxTV Community:
➡️ community.learnlinux.tv
About Me
🐦 Follow me on Twitter!
➡️ learnlinux.link/twitter
👨 More about me:
➡️ www.jaylacroix.com
➡️ www.learnlinux.tv/about-me
Recommended evergreen videos:
💽 How to create a bootable flash drive for installing Linux
➡️ linux.video/flash-usb
🐧 Understanding Linux permissions
➡️ linux.video/perms
🐧 OpenSSH Guide
➡️ linux.video/ssh
📖 LVM Deep-dive:
➡️ linux.video/lvm
🔐 How to better secure OpenSSH:
➡️ linux.video/secure-ssh
☁️ How to create a cloud Linux server with Linode:
➡️ learnlinux.link/create-linode
FAQ
🐧 Which distro do I use?
➡️ learnlinux.link/mydistro
💽 My recording gear (commissions earned):
➡️ learnlinux.link/recording-stuff
#Kubernetes #K8s #RaspberryPi 
Наука
![XO (Only If You Say Yes) - ENHYPEN エンハイプン 엔하이픈 [Music Bank] | KBS WORLD TV 240712](http://i.ytimg.com/vi/wBT1bKKwPTo/mqdefault.jpg)
I can't still believe that your high-quality tutorials are free. Would love to learn more about Kubernetes from your channel
Nice vid man, this whole 10 nodes setup/config would be perfect candidate for Ansible to orchestrate - You´d run all the setup from one shell... and share the playbooks on your wiki.
You're a great teacher.
You have a sense for making advanced topics beginner friendly.
I think I learnt more in this one and half hour video than all the searches that I did in last few days! Thank you for the great content and presentation.
Thanks for this video.
Very interesting! I am looking forward to you kubernetes tutorial.
Thanks Jay for taking your time to teach us. I really value how much patience you've and your in depth material never assuming things. Moreover thanks for making this content available for us free of charge.
from Cuba ..thanks a lot Jay and please keep on your teachings
Nice. (Especially the size of the cluster. Yes, there are bigger clusters, but there is a "too big" when you could simply go for 2+ normal computers. Yes, there are clusters with 4k cores. But no normal person needs that.)
Some ideas for improvement below.
This is quite easy:
There are tools to run commands on all servers in a list.
- You might want to have a test machine to run the commands just to be sure that you do not crash all nodes. (So e.g. run the command on node 1 and use the tool to run the update for all nodes and maybe the controller.)
You can add a second network over the same hardware interface - did this for a different purpose while using VLAN. (Ignore if you won't move the cluster to other people's networks.)
- While using the normal network to connect to the internet you can use the second network (with static ip addresses) to connect to the PIs easily,
- Biggest problem would be when you connect to a network that collides with the internal cluster network. (e.g. both use 192.168.123.x - but there are 255 other ip ranges that might be used for the network you connect to. So if you use a random number it should be unlikely to happen?)
These are harder, but quite nice in the long run:
You can add a proxy with cache for updates (when using apt / apt-get).
- You might want to add a disk to one of your machines. (That machine will download everything once and the proxy will distribute the data to all machines that use the proxy.)
- No problem with downloading everything 11 times and getting blocked. And you have more bandwidth for other things.
(You can add your own registry for docker.
- In this case the disk should be "huge". (Some updates should not be a problem for a while. But docker images... well...)
- Same as above. More bandwidth for other machines. And of course you could use that service for other computers as well. (Of course you will most likely need to download different images.)
You can install tools like Rancher in a VM on your desktop to manage the k8s cluster.
- The PIs are not powerful enough to host Rancher. And you do not need Rancher to be running all the time. But it is quite nice to have a UI for some things admins need/want to do?
...)
This is sooo cool! Thanks for making this video!! 😎 Also love the case!
Nicely prepared and well paced. Many thanks.
Great video! I’m impatiently waiting on Amazon to deliver all the items and I will be building mine next week. Thank you for the detail in your video. I’m learning Linux, as well, and really appreciate your channel.
Let us know how it goes! I’m rebuilding mine (4-node) upgrading to 22.04 LTS
Thank you. I've been trying to learn Kubernetes for a few months, and I feel like I'm getting the hang of it, but one thing I've never been able to find is step-by-step how to install my own Kubernetes cluster on my own servers. Everything I've found either recommends a hosted service, which is not at all helpful for trying to learn how Kubernetes works, or is too high level, "you need to install several packages." So thank you!
You're the MVD (Most Valuable Dude) when it comes to Linux . Thank you !
Great work, Jay!
Just FYI, if you add a '00:00:00 - Intro' line to the top of the time codes list in the description (minus the ' ' of course) then RUclips will add chapters to the video, which might make it easier for people to navigate.
Wow, thank you so much!
It would be great for you to give a lot of recommendations on things to run that would take advantage of clusters, and maybe even stuff like load balancing so you could put in one address and it just pops up no matter which it lands on
You are really great!!!
Thank you!
At last after many attempts, trying to follow a different tutorial series, I have five Pi-4s running in a cluster. What’s more I understand exactly what I’ve been doing and when the dreaded typo crept in, where it was and how to fix it.
A big thank you to you Jay👏
I’m starting to learn. What can you do with the cluster now? Like what do you do with it?
I’m starting to learn. What can you do with the cluster now? Like what do you do with it?
I’m starting to learn. What can you do with the cluster now? Like what do you do with it?
Great content
very cool video (Y) it is well explained
Best tutorial on the youtube for the subject.
Totally Professional, Well-Explained, Couldn't be done better than this.
Which Zsh (terminal) theme is this you're using ?
Great lesson. I think now I understand the process. However I’m left feeling that you break the D.R.Y principal by provisioning each card separately. Might it not be more efficient to fully provision one card, without errors(!) and then use dd to clone the rest. Yes you have to change host name and hosts as well as the special case of the controller? What no Ansible! 😂
Wow 1 hour gold video.
Hi Jay and greetings from Ireland. I followed your proxmox tutorial series and successfully set up a 3 machine cluster from old pc’s that I was told to scrap . My budget is very limited and I would need to cannibalise my proxmox setup. I Know all learning is good and I’m not asking for anything other than your opinion.
My question is this, “As you know both Proxmox and Kubernetes, which of the two would you say is the most flexible and worth devoting time to learn?”
I hope you get a moment to answer. Your videos are excellent. I can honestly say that everything I have been inspired to try from your channel the complete resources and information has been available. Keep up the great work.!
You could probably Etcher instead of USB imager, that automatically erases your SD cards before writing your image. So you don't have to manually delete the partition table every time. Works on windows, linux and macOS..
First video i ll watch when i ll buy my turing pi 2
When that comes out, i should have a dedicated video on that if all goes according to my master plan.
Love the tutorials. Thanks much. A linux vlan tutorial would be appreciated. Isn't much quality content on the subject on youtube. Thanks again.
Hi, I would like to thank you for all the work you put in this. Truly amazing! Is there a way that you can build on this video? meaning, instead of using Flannel maybe using Metallb? thank you for the help.
Great video. I have it up and running, but now what can I use it for. My original thought was to use it as a NAS (FreeNAS) server or a Plex server. But I haven't been able to find any information on exactly how to do that. Again, great video and thanks for providing these instructions.
Thanks a lot! I just finished to build the same supplier case for 4 pi. After installing the k3s cluster I don't know where to go and how to access my containers from outside.
Thank you for the great content.
I would loke your help about two questions. 1- is it possible to run runcher or portainer with kubernetes on only two machines (controler and node)? In case of a three machine setup (controler and two nodes). If one of the nodes is down, is it possible to start this node with the cluster again without reinstalling everything? Thank you for all your knowledge.
YEEESSSS
Hi Jay, thank you very much for this very detailed and useful video. I sincerely it. I have a question if there's any specific reason for not using microk8s by Canonical as opposed to k8s? Thank you very much.
Great video. Can you share your bash prompt $PS1 config? Really dig that!
It should be possible to power the Pi via GPIO and boot the PI via network so maybe it would be possible have one PI as a boot server. A way of powering a bunch of PI with an left over PSU from a computer would do.
so if smokeping starts on another node does it still have access to the data? Or is it only persistent on that particular node?
I came 11 minutes after the upload :)
It surely is an interesting project. How would such a cluster compete performancewise against a conventional server build using more conventional hardware of the same budget?
There is ~2000€ of gear involved. You can build a quite serious server power for that kind of money! Two if you wish to get some increased reliability..
try using a piece of tape overhanging a corner to pull on next time you need to peal :)
Oh man.... I really wish this was done across his 2 node Proxmox cluster. I've been trying to setup this with 2 controller machines with a load balancer but it has been difficult.
Brief suggestions: Docker is not required. K3S or microk8s are better choices of Kubernetes distributions. Use Ansible.
On Windows 10 you can use PowerShell to connect to the server with SSH
It's built in to Windows 10. Same commands and all.
Use powershell kid.
Great tutorial as usual! I'm trying this on 3 computers. I managed to follow all the steps except "edit /boot/firmware/cmdline.txt", dont have that file on pc. I managed to get nginx up and running. But the problem is i can only reach nginx via ip on the node it is on, not on the other 2. Any idea why anyone
Wow I definitely want to watch this video soon... I've just purchased my first Pi 4 with cluster hat and 4 Pi Zeros 1.3... I want to setup my first cluster... The hat came with instructions how to setup cluster with rasbian os but I wonder if I shouldn't try to do your setup with cubernites... Do you think I could do that on a Pi 4 8gb controller and 4 Pi Zero 1.3's?
Any updates on the Kubernetes course this summer? highly interested!
It will be nice to see how can we setup a reverse proxy (nginx) with some apache server behind. I have tried with LXD/LXC but I am out of hair on my head. So I lost my resources :)
the ansible dev in me cringed so hard, but awesome video walking beginners through!
Hi Jay, couldn’t find your Kubernetes videos you mentioned in this video. Have you done the series on this?
Is there a reason why you didn't use the fabric package to update all the pis?
head's up if your trying to to boot via USB 3 with a flash drive, SSD or m.2 you'll need to use the 21.04 version of Ubuntu Server. The "20.04 LTS" doesn't support EEPROM update to RPi4. It is disappointing that the "long term support" ha snot been fixed yet
Is there a simple way to configure booting to USB on all nodes including the master VIA YAML instead of using the SD card?
Can replace the sd cards by small ssd drives?
58:16 "I keep re-running this command periodically..." - or you could just do "watch 'kubectl get nodes'" and let it refresh automatically every few seconds
So. Can i build a less than 10 nodes cluster?
What should I do with node modules. Just delete the directory or should I no-detect a no-defined line with missing semicolon on my python dataframe with a .jar go file or a .rb scala file? I just don't get it.
A couple of things I have a question on - could you set up one of the nodes as an authentication server (and one or two as backup)? That way you just need to do user management in one place, and have that authenticate through the cluster? On your entire network for that matter. Or right a bash (or zsh, sh, or whatever shell) script to propagate to each node?
Also, write a script to execute the set up commands for you instead of having to switch to each node.
Lastly, NAS seems like a wonderful solution here, allowing each node.access to common files and a central place to put logs things like logs. Certain things like hostname and resollv would probably need to be on each node.
Have you considered using k3s and k3sup for Kubernetes cluster and setup? Makes things a lot simpler for home clusters
May I know how do I install the PiHole and Home assistant into my Pi Cluster? I am totally new to this
Is PoE Hat a good idea on all the raspberry pis on cluster mode?
Can you explain more about that specific POE hat? I've been using the official one but like the low profile of the one you have.
Yes! I would love if you can add a link to where we can get one.
@@nizmister @Aaron Weiss Read the description "Components used in this build"
Good morning do you have video on installing multiple node. I planning to run multiple node like ethereum, and other cryptocurrency node. Thanks and more power hope you will reply
Hi, when you are editing the hosts file do you have to add a similar line for all the nodes i.e. "127.0.1.2 node02" many thanks
did you find out if that was needed ???
I don't know but I think it'd be easier to follow if u turn the camera 180degrees so that your hands are at the bottom instead of at the top..or you turn it upside down at post. nice content though.
I'm just curious: is there any scalable and efficient way to install and configure all raspberry PI nodes in one go? Most of the operations are repetitive and tedious.
Is the rack mount compatible with the pi5?
How do you run the akash network on this
I love the look of the case but the problem I see is that if you have a single node fail, you have to pull the whole thing apart to replace it. Doing that without powering down the cluster looks nigh on impossible which makes this case practically useless. The whole idea of a cluster is that it is resilient.
Dear, the step at 38:35 is not possible for ubuntu users, since the firmware directory doesn't exist in /boot.
Is it a fundamental thing to do?
Thank you
how about storage? how do we connect them?
Can you pls link the case I cant find it on their Website
Hey I have a question! Could I build a Kubernetes Cluster to run a Minecraft Server. And have a headless linux server to manage the Kubernetes Cluster?
Does powershell not support SSH?
Have you ever ran benchmarks on a raspberry pi Kubernetes setup?
Does this work with ubuntu 22.04.1 LTS?
when the 2nd video is uploded ?
Unable to update or re -prepare the video? It's been two years and I'm sure there have been changes since.
I have a 5 Pi cluster case, is liquid cooling going too far?
That sounds interesting, do you have a picture? I've never heard of anyone doing that with Raspberry Pi before.
Possibly, but I doubt anyone here will judge.
@@LearnLinuxTV no pic but there's plenty of youtubes: ruclips.net/video/x3RexBAKB8g/видео.html
Can we host a website on that cluster?
Isn't 3-5 pies enough to learn some tricks in cluster computing? That 10 pies funny project won't beat one Ryzen 7 (with a MB, RAM and PSU) for the same price. About the same.
Important note on POE HATs: If you buy the official Raspberry PI POE+ HAT to use in this setup, you will not be able to connect the fans, as the POE+ HAT uses all of the power pins, leaving none left over for the fans! A costly mistake I made!
Why use Ubuntu server over piOs lite? Also is it possible to migrate current docker containers over without recreating from scratch? I have a pi 4 with pi OS lite running OMV 5 and docker with several containers. I want to create a cluster with my 2 extra pi 3 b but grudging over rebuilding my entire setup if I create a cluster with Ubuntu or piOs lite. Thanks
It doesn't matter till you keep your system lightweight.
K8 uses the same image for containers however it differs in YAML configuration compared to the docker YAML file.
You can use K8 even with one node (one pi) others are just workers (but can also be controllers) however configuration on nodes can be totally different in OS, hardware, accessibility, or location. K8 is just distributed and helps scale containers across all your machinces.
He showed a little overkill configuration, there is k3s (Kubernetes light) prepared for less powerful full devices.
Deploying cluster on the Raspberry PI is more Lab than practice solution as performance is not really satisfy (mainly due to big ram consumption)
@@16pxdesign Thanks
Why make a 10 node cluster when you don't use more than one node? Otherwise a great video, but it would be nice to see some practical application for a multi-node cluster.
This is great ...... one thing I learn't - DON'T put a Tab in a YML file!
I still dont understand why you are showing somethng that runs only background processs. Kubernetes is not for front end application. It is not interesting either !!
I'm sorry bro, but I cannot consume linux how-to learning videos with Indian accent:/
please make shorter videos.
i watched this with 3x in newpipe
Trying to get this running on 4 Pi 3 B+ units that I had lying around and I'm running into in impasse with memory. I get it's significantly lower than a model 4, but this was just a fun little project, nothing super serious. Without pasting the entire thing and only including the error and beyond. Adding the flag to ignore errors and specifying Mem doesn't resolve. Thoughts?
...
CGROUPS_MEMORY: missing
CGROUPS_PIDS: enabled
CGROUPS_HUGETLB: enabled
CGROUPS_BLKIO: enabled
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR Mem]: the system RAM (906 MB) is less than the minimum 1700 MB
[ERROR CRI]: container runtime is not running: output: E0108 03:07:34.733930 4263 remote_runtime.go:948] "Status from runtime service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
time="2023-01-08T03:07:34-08:00" level=fatal msg="getting status of runtime: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1
[ERROR SystemVerification]: missing required cgroups: memory
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
Well "go big or go home", you're already home so you don't need to go big. It would be a terrible video but don't take that saying as a contractual obligation
you should go watch someone else. Jay is a legend. Thanks Jay!