How to install OpenShift 4 on Bare Metal - User Provisioned Infrastructure (UPI)
HTML-код
- Опубликовано: 11 сен 2024
- Quickly spin up an OpenShift 4 cluster to start building and running your container workloads.
This tutorial follows the Bare Metal UPI method of installation and uses ESXi to virtualise the 'bare metal' machines. More information and details instructions are available in the GitHub Repo.
GitHub Repo: github.com/rya...
---
Official OpenShift documentation: docs.openshift...
---
Music: Chill, Sakura Hz - • Chill - sakura Hz (No ...
Excellent video! So clear and well laid out. I like also how you give an overview of the whole process in the beginning, since this is almost impossible to get from the RedHat documentation when you start with it for the first time.
Thank you so much Ryan for doing this! Great How-To. Calm and very clear and structured explanations! Perfekt!
Just followed these instructions for a 4.6 install and it worked like a charm... only thing I did different is that I simply allowed the coreos image to boot completely and then I cut and pasted the following:
sudo coreos-installer install /dev/sda \
--insecure \
--image-url 192.168.22.1:8080/ocp4/rhcos \
--insecure-ignition \
--ignition-url 192.168.22.1:8080/ocp4/bootstrap.ign
(replacing bootstrap.ign with master.ign and worker.ign respectively)
I like this approach because it was much easier to cut and paste into the running vm than it is to type out the kernel options when booting each VM.
Had done this with 4.2. But i used Ansible. Doing this by hand is pretty time consuming. Ocp-svc server usually is called Bastion server. :) Will do this in couple of days again but this time for provisioning will use Terraform and Vault for secrets. Hope there are not a lot of changes since 4.2. Great video! Also... if interested in S3 compatible alternative, there are great project called MinIO.
Hi ! Please, do you have a gitrepo or something like that where we could see a template of the provisioning with Terraform or Ansible ? Thanks
Hi Ryan, an awesome video. One addition, you need to add a firewall rule to ocp-svc - firewall-cmd --zone=internal --add-port=53/tcp --permanent. I found that random Pods would fail DNS lookup. Further investigation showed that happened when the DNS response exceeded 512 bytes on UDP.
This is amazing and supremely helpful! Especially the architecture definition and deployment flow in the beginning. Thank you so much.
Thanks man your video helped alot understanding pre install setup.
warning: utilize your .ign files with in 24 hours of creation else it will give certificate error.
Hey Ryan, the HAProxy stats page is totally cool - but I just learned a valuable lesson. I have followed these instructions a couple of times now, with great success. Today, I started working with a slightly larger cluster - one with 3 workers instead of 2. When I looked at the haproxy stats page - one of my worker nodes was red. I freaked out and spent a bunch of time trying to figure it out. Turns out, Openshift "ingress" spins up 2 pods listening on http (80) and https (443) - so you are only going to get "green" on two of your worker nodes at any time.
TLDR; if you have more than 2 worker nodes, it's ok if HAProxy thinks some of them are down. Trust `oc get nodes` to tell you if the nodes are up and running!
Hi Eric, good point, I should have mentioned what the HAProxy check was doing. I recall someone else in the comments ran into a similar issue when they made the master nodes schedulable. The console pods were spun up on the master nodes instead of the workers and the HAProxy config wasn't pointing to the masters for http/https. I guess I could have left the HAProxy stats page out but I just found it convenient to visualise installation progress on a long running install.
While the documentation of openshift is a complete mess, you explain the things like a walk in the park. Bravo!
Just wanted to say, many months on....this vid is still mighty useful! :)
Great to hear!
Thank you very much Ryan, It helped me a lot in setting up my first OCP4.6 .👍👍
quickly, LOL, uh no.....but so far that best instructions I have found!
Wow, this is very good. Thanks a lot Ryan! You won the internet with this video :)
Glad you liked it!
Thank you! I followed your steps and got my cluster up and running. Sweet!
Worth to mention for windows users (I'm coming from .NET world), I used mobaxterm to edit all configuration files graphically as I'm not as good as Ryan with vim. When configuring certain files Ryan used vim shortcuts to search, find/replace text which I'd have missed easily.
I also first deployed all VMs inside a virtual machine of esxi just to test. Although it succeeded, the ocp web console wasn't stable and signs out randomly few minutes after login (the session should last for 24 hours). But when deployed to an esxi on bare metal (exactly like the video) it's perfect and totally stable. I actually used my workstation for that, made myself a vm for development, and passed the GPU/USBs to it using esxi passthrough.
Make sure you watch Ryan's other videos to setup nfs storage and load balancer.
Thanks a lot for the video and git hub repo. it was extremely helpful to install and setup OCP in UPI method.
What OS you used?
this video is so helpful to me! thanks
Glad it helped!
Actually this is brilliant. Thank you, good guy!
Neat, Clear and Complete. Thanks, mate. I could never tell how much time you saved me
please keep this up2date!
Hi Ray, this is an excellent video. We were using this guide video for a long time since 4.6.8 version. But with current 4.7.0 version, though all the machines turn into green but, the installation never completes. "authentication, machine-api, openshift-apiserver" cluster operators never gets up. Also we have installed, assisted version of this 4.7.0. It also stucks at that point. Do you have any experience with 4.7.0?
great work. Thanks a LOT. Should have watched this a week before, may have saved time.
Ryan video is super explained, really appreciate your efforts
Glad it was helpful!
I was in need of this. Thanks @Ryan Hay :)
Many Thanks Bryan, this clean and perfect! I could never express how much time and efforts you save me
Great to hear!
What an amazing video Ryan. You ROCK !!. Thank you so much Sir. Really helpful and beneficial.
Hi Bro ,I followed you tutorial to install openshift, but while fetching ignition file from server it encounters some connection exception .
I tried ping the host after not interrupting live boot then it shows like unable to reach host.Any idea on what is the issue?
wow this video is excellent resource , thanks for your hard work. keep working like this.
This was terrific. Very well thought out and executed.
How i can access to dashboard from my laptop?
It is impossible to get it done by reading the official documentation, by watching this video, I believe that I can do it ! Thank you ! On question, It is the same way to installing the OKD?
Thanks Paul, yes the same method can be used.
Very Well done! I'm not an experet in this field. Certainly you need to know the basics of Linux, Routing and Protocolas like DHCP or TCP. But man...I by watgin the vide a have a pretty good sense how things need to executed in which order and why. thanks!
Amazing...Ryan. Easy to understand.
Great to hear!
Hi Ryan, I encountered an issue when installing the openshift cluster. After bootstrap+cp+workers are installed, all the vms are responding to "crictl ps" with "failed to connect, make sure you are running as root and the runtime has been started: context deadline exceeded". I am trying to install the 4.6.6 openshift cluster. Do you have any ideea regarding this issue?
i'm facing same issue on 4.6.8. How did you fix it?
@@purushdas1577 I didn't :)). I followed Ryan github to install openshift 4.5.6
@@nicu9735 - Also seeing this error. @Ryan Hay - Any thoughts?
: DEBUG OpenShift Installer 4.6.8
DEBUG Built from commit f5ba6239853f0904704c04d8b1c04c78172f1141
INFO Waiting up to 20m0s for the Kubernetes API at api.lab.ocp.lan:6443...
INFO API v1.19.0+7070803 up
INFO Waiting up to 30m0s for bootstrapping to complete...
W0427 15:10:21.649653 32330 reflector.go:326] k8s.io/client-go/tools/watch/informerwatcher.go:146: watch of *v1.ConfigMap ended with: very short watch: k8s.io/client-go/tools/watch/informerwatcher.go:146: Unexpected watch close - watch lasted less than a second and no items received
E0427 15:10:22.651115 32330 reflector.go:153] k8s.io/client-go/tools/watch/informerwatcher.go:146: Failed to list *v1.ConfigMap: Get "api.lab.ocp.lan:6443/api/v1/namespaces/kube-system/configmaps?fieldSelector=metadata.name%3Dbootstrap&limit=500&resourceVersion=0": EOF
E0427 15:10:23.652676 32330 reflector.go:153] k8s.io/client-go/tools/watch/informerwatcher.go:146: Failed to list *v1.ConfigMap: Get "api.lab.ocp.lan:6443/api/v1/namespaces/kube-system/configmaps?fieldSelector=metadata.name%3Dbootstrap&limit=500&resourceVersion=0": EOF
E0427 15:10:24.654194 32330 reflector.go:153] k8s.io/client-go/tools/watch/informerwatcher.go:146: Failed to list *v1.ConfigMap: Get "api.lab.ocp.lan:6443/api/v1/namespaces/kube-system/configmaps?fieldSelector=metadata.name%3Dbootstrap&limit=500&resourceVersion=0": EOF
E0427 15:10:25.655750 32330 reflector.go:153] k8s.io/client-go/tools/watch/informerwatcher.go:146: Failed to list *v1.ConfigMap: Get "api.lab.ocp.lan:6443/api/v1/namespaces/kube-system/configmaps?fieldSelector=metadata.name%3Dbootstrap&limit=500&resourceVersion=0": EOF
Really appreciate your work....thank you so much for your guidance
Thank you so so so much Ryan for sharing this!! Helped me a lot!
Glad it helped!
Hi Ryan, I am unable to boot both worker node after reboot. Getting Get Result :internal server error but
bootstrap and all CP nodes boot properly after reboot. What is the issue??
Thanks alot Ryan :) very informative and helped to connect the dots. cheers!
Hi Ryan Hay,
I want to install the latest version of OpenShift on VMWare but there is a condition that I would not have internet access instead of that I have to create a mirror Repository server where I will put all the images and software. So what required changes do I have to make on the Bootstrap virtual machine (ocp-bootstrap) machine and Services virtual machine (ocp-svc). If I follow your video for installation.
Great tutorial! I have a Question: I would like to change some network settings (IP, Domain). What files I need to change from your git repository?
Great question! I did the same and the dns wasn't right due to the db files having specific names. I changed them and tried redoing the steps but still couldn't resolve dns correctly.
Useful information. Tks Ryan.
Hi Ryan,
This video is very helpful and easy understanding.
One question: what content we put in /share/registry (nfs share)
Great Explanation. Could you please create more videos for OpenShift related? Not much videos for openShift in the RUclips found... Ty
Hi Ryan thanks for the comprehensive tutorial. I have a question what could be the architectural changes in case DNS being part of a separate machine ? does that machine also requires ocp network ? and do we need to update api, api-int, ocp-svc and * records to be pointing to external IP of machine having loadbalancer ?
Very well structured video.
Thank you very much! Great howto!
I get internal server errors from the http server on the bootstrap node when a worker node tries to bootstrap.
Great Video Ryan
Thanks!
Hi Ryan Hay. Thank you very much for making the video available. I have one question. Your process is also supposed to work with the latest version of Red Hat CoreOS, at 4.72, right? I ask this because I am having some problems raising the initial setup
Forget. I put more power(CPU and memory) in my cluster and everything works fine
Ryan ,video very excellent..I am planning to install 4.11version , i believe installation instructions got changed. I request you to add new video specific to 4.11 version
Hi Ryan, i tried your mentioned method but two operators (Console and Authentication) shows not available, can you guide me on that i am stuck here.
Hello Ryan I was not able to find the CentOS 8 ISO image for linux online, instead I installed CentOS 8 Stream. Is there any difference? I am stuck at the Deploy Openshift step, the bootstrap command on git is not working. Is it because of the different vesion I installed. Any suggestion how to fix it?
Hi Ryan,
many thanks for this excellent tutorial, i have done on 4.12 successfully.
Now I am able to login with kube:admin user only, not from other admin users. Can you or anyone can help on this?
Thanks a million. I learned a lot.
Glad to hear it!
I installed openshift 4.15, i made a few tries until i configured openshift with dedicated cp nodes and 2 worker nodes (like in the movie). If there were 3 cp+worker nodes (default setting), authentication operator was in degraded mode , console didn't start , also there was something on ingress.
And a big thanks for the howto :).
Were you able to get it working im having a similar issue with 4.15. I had my console up, but it eventually failed, not sure if software dev's did something or not, but the whole cluster went to a not ready state
@@fraatzwDid you update the install yaml file to use OVNKubernetes or OpenShiftSDN? My botstrap fails to pull the image during install "unable to pull OpenShift Release Image "
Hello, i am not sure what is wrong, but the farest I have gone is when bootstrap is ready but the controll plane nodes are going DOWN and UP.
I have restarted the procedure at least 6 times from scratch.
At one point I have realized the ocp-svc node has different timezone than the bootstrap and controll plane nodes hence the certificates generated by the installer on the svc node are different and bootstrap and controllers failing.
After fixing this, API is still not stable. Don't have a clue what is wrong.
Just while I have been typing the API bootstrap and one controller came UP, but after copule of minutes they usually go DOWN again. Never seen full green by all 3 cp nodes both API and config backend.
Hello Ryan, I have a question , Is it necessary to use vmware vsphere? is it possible with workstation?
Or virtual box on windows?
Hi Ryan, followed your video, but i am stuck waiting for all cluster operator to be available. Especially authentication, console, monitoring and openshift-apiserver. Any advice on this?
Hi Ryan
I have separate servers for DNS,AD and DHCP so which part I have to skip for creating it. As there are 18 section for building UPI.
Thank you. Very clear.
Hey Ryan, Great Video.... what spec NUC would you recommend to get this working?
was looking at a NUC10i7FNH, NUC 10 Core i7-10710U, 64GB DDR4 Mem, 128GB
SATA M.2 SSD, No OS
with NAS 4tb synology.
Thanks
Hi, I need your help. I have doubt in the HAproxy section, I can not start the Haproxy it showing. please give me advice
Hi Ryan,
OKD port group is the interface that facing to internet right ?
Hi Ryan. When I power on bootstrap after writing the url in boot time after loading, I’m getting error “7 unrandom warning’s missed due to ratelimiting “ do you have any idea about this.
Thanks In advance.
Thank you so much Ryan, for doing this quality content very clear explanation and step by step. I'm planning my own lab with ESXi 6.7 and all VMS using Centos core OS, any suggestion on this?, I think using Centos on all VMware's is not a restriction forOpenShit :), am I right?.
Kind regards from Lima, Peru.
Thank you! Im not familiar with CentOS Core OS and can't find anything on it. The OpenShift documentation does however specify that the bootstrap and control plane machines must use Red Hat Enterprise Linux CoreOS (RHCOS) as the OS, the compute can use either RHCOS or RHEL.
Hi Ryan, I am getting error while boostrap image from coreos os.. it says compressed file format is not support.
Hello Ryan, I'm trying to assign my worker/control plane/bootstrap nodes with ip addresses for the open-shift network "bottom portion of the diagram" however, when I edit the network configurations using nmtui command it does not retain the configs. I also tried restarting the network service using 'systemctl restart network' and 'systemctl restart networkmanager' after making these changes but I'm prompted for a password that I never created for these vms. this is preventing us from deploying open-shift on my worker/control plane/bootstrap nodes. any clue or insight would help!
1) What is the process for booting the master/workstations from the ignition files?
Bootstrap,master,worker all should be powered on at the same time.
Bootstrap first then the master machine about an hour later, then the workers after another hour
What does the wait for bootstrap to complete mean
1) The bootstrap server has all the files from the bastion server independent of the status of the master/nodes
2) Or masters and workers are all configured and working
What indication can we look at to see what is happening on the bootstrap server. The are logs but what should we look for in the logs.
Thanks
Loved the video, appreciate the effort. But just a warning to viewers. Don't spent all your time trying to set this up as a test environment and if you need to actually test something anytime soon. I tried this twice from front to back using the latest versions of RedHat with CentOS 8.3, triple checked every step, and I used all the time I had from Friday night to 2AM Monday. Its really close but non functional. Same errors on the GET calls to the api host. So I burned 3 days I didn't have to lose. Don't make my mistake! It looks so close to working but not quite there.
agreed! already spent 3+ days and still errors on GET calls will try a little bit more before I wrap up for a while I guess...
Thanks for sharing this fantastic tutorial! I hope to try it myself. I do not have vmware esxi so would virtualbox or Hyper-V be an acceptable substitute?
I have a question. Is this tutorial works for OKD ? Because,, i dig other videos and tutorial.. i try my best to install this OKD on my Hyper V,, but it wont works.
Hey Ryan , can l use just 1 network , for example if l dedicate complete 1 esxi to ocp ??
Hi Ryan, this is very helpful! Thank you! Are there plans for you to step through the creation of the Openshift UI for your CRs deployed by your custom Operators, using Spec Descriptors? If not, I'd love to see it! Subscribed! (See: Declarative Dynamic UI for your Operator)
Hi Ryan, great video! very informative. Stuck on deploying the control-plane nodes however wondering if you could advise.... ignition[805] GET error: Get api-int.ocp.lan:22623/config/master I have validated DNS is working, however that file doesn't exist on the services node at all if I attempt to wget it etc...
Resolution was update from RHEL 8.2 --> 8.3. It seems there was a bug which was fixed with an update. bugzilla.redhat.com/show_bug.cgi?id=1708663
Hi @@matthewklos7184 , I have the same error...
ignition[925] GET error: Get api-int.lab.ocp.lan:22623/config/master: EOF
I'm using CentOS-8.2.2004-x86_64-dvd1.iso for ocp-svc
Shall I use rhel 8.3 for ocp-svc instead of CoreOs 8.2?.. will this solve the issue?. Thanks!
Im not sure about this issue. I would SSH into the bootstrap node and take a look around for any errors/containers not starting because it seems like the machine config server is not coming up for some reason.
@Matthew Klos
was the issue resolved, I hit the same error.
@@juanchicchon5017 Seeing this same issue now, i tried to send a CURL request to that endpoint and i keep getting this : OpenSSL SSL_connect: SSL_ERROR_SYSCALL. How did you manage to resolve this problem?
thanks for the demo Ryan!! question for you: why the svc machine has to be in the same network than your local network ?
sometimes it works sometimes it doesnt, is this anything to do with the masking or DNS setup?
Great video, I have a question, does bootstrap have to have internet? I did the lab but bootstrap doesn't have internet, how did you get it to?
Hello Ryan, I am trying to setup the lab by following this video. I am using VMware Vsphere to setup the openshift container cluster. I have a VLAn created for internal network but for external network, I don't know how to create an external VM network which you added as an additional network for the helper machine. I got struck here for past 1 week. I don't know how to take it further. Since I new to VMware and linux , learning it. Please guide me how to setup the external network and connect?
Hi! Is it possible to add the pull secret after install? I used {"auths":{"fake":{"auth":"aWQ6cGFzcwo="}}} but now I need redhat operators, specifically Cluster Logging.
What is the best option to choose if I want to create private docker images registry, e.g. on the University server for the students? Portus/Harbor?
Great Vid!
Thanks Sebastiaan!
hi @ryan i have some issue here regarding fetching rhcos image from http but when i test the web page i can find it and install it on desktop and your response appreciated
Whats the difference between this and OKD?
Hey Ryan,
If I correctly understand you have installed VMesxi on your MAC and then deploy OCP cluster? What is the specs your MAC have? I have a host with 64 ram and 1tb ssd and win 10 on board, will it be enough to deploy such environment using your guide?
Hi Vadim, I have ESXi booting off a USB on a Dell t140. I just connect to that host from my Mac. It has 64GB RAM, 2TB HDD & 6 core CPU @3.5GHz.
very helpful, thanks !
i tried to install 3 master, 7 worker cluster. Installed only 3 masters. Help pls.
Many thanks Rayan👍👍👍
Hi Ryan,
I want to set static ip to workers node. How can I achieve that?
Hi @ryan great tutorial! One question why did you use baremetal insteand of esx/vsphere provision?
Hey @czerogr, thanks! 2 reasons; firstly, I just wanted to explore a lot of the things the Installer Provisioned Infra (IPI) method was doing behind the scenes. Secondly, because of system resources... The host I was using only has 64GB of RAM and the vCenter Server required for the vSphere install I think at minimum takes up around 12GB of RAM which starts to take away from my already under spec'd master and worker nodes.
where you get the bat script to show with line numbers and filename?
github.com/sharkdp/bat
Thank You so much Ryan
No worries!
What’s ssh client you use in MacOS ? Thanks and excellent video
Simply Awesome!
Thank you! Cheers!
Can it be like 3 control planes as VM and 2 worker nodes as baremetal ?
Hi Ryan,
I’ve done this following your instructions a couple of times now. Thank you!
One bottleneck I’ve found is that after the initial reboot the masters and bootstrap nodes do a lot of downloading from the internet while provisioning. So much so, it consumes most of our bandwidth!
We have a need to be able to setup and tear down these clusters on an adhoc basis.
It’s obvious that RHCOS has a lot of things it needs to download from the internet after grabbing the image locally. Is there a way I can download those dependencies to location on my local network, saving RHCOS having to always go out to the internet to download?
Hi man, Ineed your help
Excellent video Ryan!, It helped me a lot to understand the whole installation process. I would like to know how to add advanced network configuration, for example to add bonding, I found related information on the RedHat documentation:
bond=bond0:em1,em2:mode=active-backup
ip=bond0:dhcp
Where em1 and em2 are the physical network devices. Those two lines should be added during PXE booting on the RHCOS machines? I found also that we could modify a live install ISO image, with customized network settings profile (using coreos-installer binary). Could you please help me to clarify?
Hello Ryan, do you provide any online training on openshift ? thanks
Can you post some trouble shooting info on your github site
can we install windows platform and linux platform on rhel openshift at same time?
Hi
Is it workable on my laptop (8 Core , 64GB RAM).
Regards
Zaheer
Thank you so much by the video. Can you help me? Is possible install OKD4 on a VPS Ubuntu 20.04 with 24gb for a homelab cluster all in one?
Hi Anderson, It depends if the VPS supports nested virtualisation for you to provision your cluster nodes. You would probably require at least double the RAM though.