How to build your own Raspberry Pi Kubernetes Cluster
HTML-код
- Опубликовано: 22 авг 2024
- By popular demand, I'll show you how to set up your very own Kubernetes cluster. But not just any boring old cluster, I'll show off the process of doing this on the Raspberry Pi platform, which is a cheaper and more energy-efficient foundation for your clustering needs.
Note: Be sure to check out the wiki article for this video, it has a few very important changes you'll want to follow. Otherwise your nodes won't be able to access the Internet. One of my viewers alerted me to this, but I cannot find his comment to give him credit. Thanks to him for bringing this to my attention. The wiki article contains the fixed instructions.
🙌 Support me on Patreon and get early access to new content!
➡️ learnlinux.lin...
☁️ Get $100 in credit toward deploying your very own Linux servers on Linode!
➡️ linode.com/lear...
🛒 Affiliate store for Linux compatible hardware/accessories (commission earned):
➡️ learnlinux.lin...
💻 Check out the Tiny Pilot KVM for your Homelab (commission earned):
➡️ learnlinux.lin...
🐦 Follow me on Twitter!
➡️ learnlinux.lin...
💬 Official LearnLinux.tv Community:
➡️ community.lear...
📖 Check out jay's latest book, Mastering Ubuntu Server 3rd Edition. Available now!
➡️ ubuntuserverbo...
Wiki article for this video:
wiki.learnlinu...
Raspbian download link:
www.raspberryp...
👨 More about me:
➡️ www.jaylacroix...
➡️ www.learnlinux...
💽 How to create a bootable flash drive for installing Linux:
➡️ linux.video/fl...
🐧 Which distro do I use?
➡️ learnlinux.lin...
🔐 How to better secure OpenSSH:
➡️ linux.video/se...
☁️ How to create a cloud Linux server with Linode:
➡️ learnlinux.lin...
That is great, but what do you do with it?
This video is just for creating the cluster. As far as what to do with it, you can run any containers on it that you may want to. I'll create some additional tutorials that will show how to use it at a later date.
@@LearnLinuxTV It'd be great to see you create a deployment, a load balanced service, and how you access that service!
Lol I could relate looks really cool but what is it for
I'd say there's great value in knowing how to roll out a cluster completely from scratch. The pi lets you do precisely that for an incredibly low cost. Let the software team worry about what to put on it; the DevOps team has to know how to set it up and get it working.
I think it's always valid to answer the question 'why would I want one?'. I happen to know, but that's just me.
Jay, Yes please continue the Kubernetes series. I am totally enjoying the videos and of course learning.
I'll be glad to watch more content about containers and Kubernetes
I second that
Third
@@coletraintechgames2932 Fourth
After setting up the hosts and hostname files you could have installed ClusterSSH or Ansible on the Pi's so you could run the same commands on the pi's at once, saving tonnes of doubling and tripling up commands ;)
Very true, but Ansible setup would've made the video much longer. I'll be creating Ansible content as soon as I can, it's a great solution.
Or fabric! One thing more to learn, but it would probably make the video shorter. Anyway, good and inspiring video!
You could open everything with "terminator" instead of the default terminal. There's a feature to send your keystrokes to all three servers at once. Worth checking out if you have to do a lot of maintenance on this cluster, and you're looking for a quick and dirty solution.
LearnLinuxTV +1 for Ansible. Perhaps, a follow up video that shows how you could setup Ansible on your master node (or a management node) and then manage all kubernetes nodes and then create a playbook (or role) to install all Kubernetes components on all nodes as required.
Could you not just set everything up on one 'node' and then just copy the image to a bunch of additional node sd cards, changing only the hostname?
When I looked further into this, my first thought was, Google? Ruh roh. They tend to drop the mic on a lot of their projects. Then I found that Kubernetes is an open source container cluster manager, which was contributed to the Linux Foundation by Google as a seed technology. I guess this makes me feel better about diving into this. But hey, it's not like I couldn't find multiple other uses for my 3 new Raspberry Pi kits. Meanwhile, thanks for the work you put into your videos.
K8s is here to stay for a while. Not only because Google basically signed it over, but their record is actually pretty good on the backend, so Google will keep contributing
I love this video, you're super reasonable in your approach and you tell people EXACTLY (!!) what they need to know and everything.
Nice, nice!
This is the beginners content I am starving for!
I am literally looking to become a programmer. I want to make a business using Docker and Kubernetes.
But I am a beginner!
I was a beginner once as well, but we all have to start from somewhere. Keep it going!
LearnLinuxTV oh! :o
yes, please. show how to use the cluster to perform some workload. great work. definitely one of the best in youtube
Wow, that was great. Thank you for sharing.
I noticed that many people don't seem to know what to do with a Raspberry Pi cluster.
To start with a Pi cluster is a supercomputer at very low price.
I plan to build the cluster and installing LAMP and Drupal so I can create my own website and run my own .com !
This is great. I would just leave one suggestion here, for people interested in getting this setup. Specially if you are starting to learn kubernetes, from time to time, you will mess your cluster, and I find it's best to have some sort of script to set it. There are lots of commands, and some of them take a very long time. Ansible is a tool to help you there. But in general, get this process scripted is a time saver.
Again, this is great. Hoping to see more of Kubernetes in here.
Hello Jay,
Thank you for this great content.
I’m new to docker and kubernetes. Can you make a content that show the use of kubernetes alongside with docker.
Keep your great work.
And yes, please. I'd like more Kubernetes content here. Thanks.
Hi! Just to say : Thank you ! Just followed this tutorial and my first k8s cluster is up and running...will backup my SD cards now before begining to play (1 master, 2 workers on Raspberry 4 - 4giga RAM). Some of you are asking what can you do with k8s : will use it to try some stuff on PostgreSQL, Cassandra, InfluxDB, NodeRED, mosquitto (mqtt), etc...
Excellent video Jay! It would be cool for you to do a walkthrough on your hardware setup-the switch, cluster case, fans, cabling, etc.
Thank you for all the hard work that you are doing . Please continue making more videos on Kubernetes Cluster. Thank you again.
Would absolutely love to see more cluster videos. Especially with raspberry pi. Practical home and other applications would be really cool
A great tutorial, thank you. Managed to the the cluster up and running with no hitches.
I'm french your video is perfect your english is very clear and of course I want more video about Kubernetes.
Ansible can be used to configure the Pi's before installing k8s. It obviously has a learning curve, but in the long run it is so much faster. A quick tip :)
You might want to save a ton of time with clusterssh which lets you run the same command on all your pi machines at once. Rather than keep copy/pasting everywhere.
Cool! Thank you. I'll look into it!
Good suggestion Alan. I use tmux-cluster myself. Easy to configure, but it also requires an understanding of tmux, which is my primary motivation for using it. If you are interested you can find it here: github.com/davidscholberg/tmux-cluster
pssh is one I've used in the past
I need to determine what my actually needs turn out to be. I'd love to use Pssh or something similar to keep things simple. I may need to go to Ansible however if much variation is needed in maintaining sets of nodes. I'll determine first if simply using MPICH will suffice.
"Only '60ies kids will remember en.wikipedia.org/wiki/Expect"
You should deploy an app to your cluster.
Another great video Jay. Please continue with more on kubernetes.
This is a superb video and the explanations / wiki are great. I run across so many videos or explanations that start out good, but leave out explanations, or get you 80% there, then break. You were very careful on this and really got to a good level without overkill. I really appreciate this one & now have a great little Kuberetes cluster sitting on my desktop. I can't wait to put it to use!
Jay I hit the like button as I like the stuff you make.
absolute saviour! been trying to understand kubernetes for my new job and this just helped a ton
This article and video saved my sanity! Thank you so much for this video. Big thumbs up for this amazing effort!
For all those wondering, what it does, here's a breakdown. If you are a developer or devops engineer or thinking of becoming one, you would be pretty pissed all the time to configure your working environment, store different version of your builds, and provide high-availability, fault-tolerance, disaster-recovery and load-balancing, than this wonderful rig can help you get started. For those who cannot go with this route, you can simply use 12 months cloud services either in Azure or Aws and you would do the same. Except now you have a physical access to your small SOHO infrastructure. But don't get me wrong, the knowledge you'll learn apply directly to any datacenter level infra management.
You know, this would be a terrific example of how Ansible saves you time.
Perfect, I was just planning on doing this over the weekend!
Yes please want see more docker/k8s goodies
Hey Jay! I think more Kubernetes would be awesome. Also now that you have a cluster at your disposal you can also do a video on using parallel computing using MPI.
Thumb up and Sub for NOT having an intro that makes my ears bleed like 90% of RUclipsrs do!
Great great video!! I would be more than interest on other videos on how to deploy app on raspberry base k8s cluster!!
Running ur own cluster is cool, but with todays wie variety of hardware like Intel NUCs and used hardware that can be bought on eBay for little money but provides lots computational power.
How do the PIs scale when compared to desktop CPUs?
A comparison video would be interesting to see the power consumption vs. Computing power between those 2 factions.
Building a cluster of Raspberry Pis has never been about the performance. It is about the learning experience: learning how to setup and manage a cluster on the cheap. If you then really need the performance, you buy bigger hardware or, more likely, set up a k8s cluster on some cloud server provider.
I have a bunch of pis and have been doing some similar fun things. The issues I have run into is where volumes are needed or some sort of data preservation is required. If that isnt the case these are pretty damn good at spinning up and down based on the workloads. I went about it differently of course because i didnt use Kube.
Walked through this step-by-step. It was great, thank you!
This is so awesome, thank you. I am going to setup even more pis on the cluster tomorrow!
Really helpful and explains in details. Appreciate indeed.
Amazing tutorial. Was step by step. No issues. Great. Thank you.
Fantastic video.
And please make a video with an example of a great way to actually use this cluster with some common applications.
Thanks.
Definitely - that's the next phase. :)
(try) Shift+Insert [to paste; its like Ctrl+v] Good practical introduction to Kubernetes. Thanks.
Thanks! Where has this been all these years!!
Thanks so much for your clarity on this topic better than anybody else on the net. Here’s a simple question once I’ve got the set up what can I do with it sorry might be a dumb question
I think I'll name my child coobernuttys.
Nice video. I’d love to see a follow-up video where you add services to it. A lot of the videos out there I’ve seen use k3s and it doesn’t seem to translate to k8s.
Fantastic .... I want more of it. What can you do with it?
highly recommend tmux to ssh into multiple clinets and send same terminal command to all of them simultaniously using "setw synchronize-panes on"
I just read all about it aj and love it! I intend to give it a try soon. Thanks so much! Now, if tmux (or something similar) can somehow perform the following sequence, I will be ALL SET:
Transmit a slightly different (incremental) command to each node (or open window or ssh connection) on my cluster.
Example:
...cp node1.msg newmsg
...cp node2.msg newmsg
... etc.
given a suitable syntax.
Many smart guys here. Sure hope somebody can provide a solution! Thanks!
@@donporter8432 if the number will be consistent for each machine across the same session, like if the 1 in node{1} will always be the same for that box, then you could set up a unique SHELL variable on each box beforehand, aliased to the same name.
www.digitalocean.com/community/tutorials/how-to-read-and-set-environmental-and-shell-variables-on-a-linux-vps#creating-shell-variables
How totally COOL is that aj! 👍♥️🤗
Hi there I wanted to review the "Wiki for this video" but I keep getting "The site cannot be reached...."
Hey! Please continue this
Nice! Thank you for this video. Willing to watch more on this... did you deployed kubernetes dashboard?
Awesome video, would love to see more Kubernetes
Why not have a PoE switch, PoE HAT, and use an other, perhaps Raspberry, or any other computer, to net-boot the cluster, negating the need for micro-SD's? (though you would need one)
Very interesting video, thank you for sharing
I wanted to conduct the experiment - now I know where to get the best into
Beautiful! Thank you very much.
Great tutorial. Thanks.
Excellent! Definitely going to build one of these very soon.
More kubernetes please! Forgot to mention - liked and subscribed.
More is definitely coming. I'm targeting April for more Kubernetes content, but if I can get to it sooner, I definitely will.
@@LearnLinuxTV Thanks. Looking forward to them whenever you can.
The new opening is dope
Awesome vid 💖 , more about k8s pls
1:59 pc fan in the background starts spinning
thumbs up!
I would like to see a follow-up video about storage and maybe utilize your freenas as an nfs share storage for the containers /config and /data
Karlo Abaga make that a rpi 4b as a nas /nfs.
Very fun stuff, Jay!
Looking forward to seeing a Container utilizing this technology.
I’ve been using my old s8+ charger for the last two months on my pi 4. I have never had a problem with it.
Newer pi 4s have a fix for the issue, and it's actually the cable, not the charger - cables which are "e-marked" will not work, whereas cheaper cables will work just fine, even with chargers that reportedly don't work. Pretty much only Apple cables are e-marked, so the vast vast majority of android chargers will work out of the box with no issues.
wow, you made it look super easy.
Great video Jay
Would be cool for a Kubernetes proxmox VM cluster.
Great video Can you put together a video on how to build the Cluster and where to get the parts
Great tutorial. Is this kubernetes within docker? If yes, why not only kubernetes install to containerize?
Great job with your presentation along with a style / approach is very learnable. Ansible is a different topic but you can see how others are adopting automation. If the feedback warrants, I'd like your thoughts on "things" that can now be deployed due to the increased capability. OpenCV facial recognition etc. Especially if there's a method to measure the increased performance and throughput.
Thanks, i'm interested in how to manage letsencrypt and selfsigned https certificates in kubernetes
Awesome turorial. Would love to see how to get a webserver or sql server up and running in this Kubernetes environment.
This is excellent , thanks for doing this. Do you any more plans to do a follow up please?
Nice work... suggest ssh key gen to master and copy rsa public key to each machines Authorized_keys file in .ssh so no password is required from master and SCP of any files will run without password ... .. great work and presentation ..
Minute 2:36 yesterday I seen a vid where someone is showing an cluster of 80 RPI.
Hello Jay, you dont have to reboot a machine when the docker config changed. Just use sysytemctl to restart docker.
I would love to see more about containers and k8s please 🥺
What can this be used for?
Calculating digits of pi
@@silenthill4 are you sure? I would think that calculating pi would be very single threaded because you need to finish calculating the prior digits before starting the next division.
@@mannyr9225 you can break it down into sections
@@silenthill4 how? You wouldn't be able to start the next section until the end of the previous reached it. You wouldn't know what to divide by at the beginning of the sections.
Emmanuel Rovirosa there are these math equations that are called faucet equations and they let you pick a place to calculate
daemon.json content in video has a typo in it. Systemctl in current raspbian lite reports:
unable to configure the Docker daemon with file /etc/docker/daemon.json: invalid character '"' after object key:value
JSON code should be:
{
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m" },
"storage-driver": "overlay2"
}
This will allow docker to start.
Very good i liked it go ahead with the next video :)
-y for Apr commands!
What's a good learning materials, unfortunately I cannot find the wiki article for this video, I want to copy all the commands. Many thanks
Great tutorial, please make a tutorial on how to run multiple websites on kubernetes cluster with docker containers
Thank you for the video. It was very interesting. Are you planning to do some videos regarding operators?
Great video. Might be worth mentioning that this will fail for Raspberry Pi Zeros as ARMv6 no longer supported by k8s.
pretty cool, like to see a video on what you can do with it?
👍 for going with kubeadm!
Maybe if you put a link to the case, you would be helping us and getting a little money as well..... lol's Haven't found the case yet but I'm still looking
It is on Amazon. I have 3. "Cloudlet case".
www.amazon.com/dp/B07D5NM9ZG/?ref=exp_learnlinux.tv_dp_vv_d&th=1
@@lowtek42 Thank you..
Joshua Burton Thank you!
If you blur out stuff, check that's not anywhere else mentioned, like in the title bar ;)
Great video thanks for sharing
Great video!!!
I wonder what is better? I have one old Rbp3b and I'm planning to buy a model 4. Is it better to have one 8GB or two 4GB for the cluster? And once I have the cluster which pi would be better to have as the master? the 3b or the 4?
Thanks for the video.
Your intro should have discussed what a Kubernetes cluster is and why one would want to build one with Raspberry Pi computers. Also, mention one or more use cases for this.
Was the video incorporating Ansible to do this ever made? I'm thinking about parts that *sometimes* erred out, and other statements that might or might not be safe to repeat in subsequent runs. The word I've been taught is "idempotent".
Thank you very very much
That would be cool water cooled ;)
Thank you! more, more more!
More content on this please!
I'm definitely hoping to do that. I already have March's videos planned out, so possibly in April.
This is epic thank you!
Can you use external storage devices with these clusters? I'm thinking about starting a cluster project to get experience and skills for network administration and I'm not sure if I should get larger microSD cards or if I can use a spare external hdd that I have laying around.
Super neat! I'd love to see more videos on kubernetes. Also, I know you use Proxmox form your videos. Are dockers supported on ProxMox along with containers? Thanks a bunch!
Nice vídeo, but I have this small thing to point out:
Anyone who doesn't have some understanding of Linux systems should not be using this tutorial. So why are you explaining what "apt update" does and not those boot options? What do they do to the raspberry runtime and how do they improve K8s performance?
I like that you explain everything. But you can't explain to the level of updating repository source list and not explain (in a high level understanding) of what those kernel options do.
(Obviously I can search them online, but still..)
Edit: also, explain why you've set those options for docker daemon. I know what they are for (logs), but why? You should've explained that otherwise, the output log of the containers will completely destroyed that free space
I don't really think it's nice. There is no info in this video that can't be written on half a page of getting started tutorial, he's not explaining anything interesting anyway.