🔴 - To support my channel, I’d like to offer Mentorship/On-the-Job Support/Consulting - me@antonputra.com 👉 [UPDATED] AWS EKS Kubernetes Tutorial [NEW]: ruclips.net/p/PLiMWaCMwGJXnKY6XmeifEpjIfkWRo9v2l&si=wc6LIC5V2tD-Tzwl
This is a great tutorial, you are helping me alot for acquiring my first devops position, i have an home assignment to provision a vpc and eks cluster that integrates with ecr and this is great and very relevant. Altough im having issues with "allow_assume_eks_admins_iam_policy" module which says i have some syntax error in the hidden directory .terraform within the project, weird but i'll troubleshoot it. Thanks alot!!
awsome tutorial i created the EKS Cluster but having issue with load balancer it througing an error Kuberntess cluster unreachable :the server has asked for the client to provide credentials . can you assist please thank you
I found your channel and I am absolutely in love with your content dude! Huge thanks. By any chance - on 12:15min do you know how we can specify a private repo image from ECR in the deployment? For example deploy wordpress.
Do you plan on creating a wordpress guide anytime soon that's deployed through terraform to EKS and uses RDS ? I have difficulties creating my yaml deployments and services and not quite sure if you are able to help on that.
Great tutorial, I replicated all this and I moved a step ahead with setting up rbac but I got stuck with error authorizing namespace setup and other resources related for the rbac
Thanks, generally only user or role that created EKS cluster would have admin access (for example update config map in kube-system ns). Check with kubectl auth can-i "*" "*"
Thanks a lot for the lesson, much better than the one with just AWS resources as here the cluster autoscaler is also part of terraform; One question though: when scaling up, e.g. from 1 to 2 pods, clearly you get more nodes because of the auto-scaler (also 2), but then when decreasing pods to 1 in the deployment, num. of pods gets reduced to 1 but not amount of nodes (so still 2). What happens is that when I create an auto-scaler, I get 2 nodes and my desired num. of nodes is changed to 2 instead of initial 1. Is this intended? and if yes, how do I also destroy the nodes to scale down (keep desired at 1)? Cheers!
I encountered an issue when deploying microservices with this solution. When my own API gateway was hosted in the SPOT instance and the other node API servers and databases were in the general instance, then the communication between the pods did not work. I fixed the issue by making the general instance larger so all the pods were hosted there from the start. Is cross-node communication blocked by default, do I have to change security group rules or is there something else I am missing?
No, it's not blocked. Maybe it was in the termination phase, it's hard to say. But it's totally common to create spot and on-demand instances in the same cluster, and networking should work. I have updated video with update source, pls take a look - ruclips.net/video/5XpPiORNy1o/видео.html
Its spot on. gr8 quick question why can we use aws_auth_roles = [ { rolearn = "arn:aws:iam::66666666666:role/role1" username = "role1" groups = ["system:masters"] }, input of terraform-aws-eks module for adding role to EKS?
Hi , awesome video, I have been following you since long, really like all the stuff you share. I am just stuck on creating service account through terraform for eks? as I have to manually do kubectl apply -f serviceaccount.yaml for load balancer (EKS), how can I automate this part, will appreciate it, if you can provide some knowledge
Really helpful !! Can you make an video for how to do authorization and authentication in ingress controller instead at pod level for microservices running under kubernetes EKS
Hey. Thanks for the awesome tutorial. You are the first one who was capable to make terraform tutorial which actually works). If you don’t mind I have a small question. I followed you tutorial and all works find but if I try to add something (another helm chart for example) terraform recreates aws_config map every time I do terraform apply. Any ideas why it’s happening?
Yes, but If you want to further harden it, you would change one parameter to make it a private cluster. This means the control plane endpoint becomes private, and you would need a VPN or a bastion host.
@@AntonPutra also what type of VPN is normally used by companies? I know of OpenVpn from AWS marketplace but are there any cheaper alternatives? Or methods to setup VPN?
Great tutorial man thank you so much! One question. When I run tf destroy the load balancer is not deleted, which leads to issues with deleting other resources like VPC. How would you fix this?
Hi, thanks for the video. I implemented the code. It is working except disk_size . Even though this parameter is set to 128 default disk size of 20 is applied to eks nodes.Any suggestions on how to implement it.with launch templates into the current code is not working
@@preethi-devops Here is an example - github.com/antonputra/tutorials/blob/482596538abd03dd823cd0fe42e5ae2ea77a0765/lessons/102/terraform/7-nodes.tf#L76-L89
@@AntonPutra can’t we use it with eks module rather than creating from resource types. I need to use launch template with existing code with eks module just to change disk size
@@preethi-devops Yes you can "Support for custom AMI, custom launch template, and custom user data including custom user data template" Try to use that launch template in `self_managed_node_groups` block in the module
Thanks for the great video. I follow the steps and had a problem in 2 cases. 1. during a failure at the first deployment I tried to rerun TF and got resources that already exist on the cloud watch IAM group and role. Then I tried to destroy the entire stack and got another error. The only way to redeploy is to delete all zombies manually because they are not fully deleted. how do you suggest dealing with it?
I had similar issue with netscope client, disable it before running terraform (or similar monitoring tool on your latop). After terraform fails in the middle, you can either import resources (manually) or delete them (manually)...
@@AntonPutra Thanks for your answer. Import actually doesn't work. I deleted them manually. Wanted also to ask you regarding "AWS Load Balancer Controller" is there a way to run 2 different controllers (one for internal and second for ext). I'm asking because they should run with a different annotations. also what is the exact different between was load balancer controller to ingress-nginx (in ingress nginx I managed to run 2 different ingresses)Thank you !
Whats this lesson? I am appling this code with terragrunt and not work, this error: permissions_boundary must be 20 characters or higher (aws_iam_role_invalid_permissions_boundary)
Hi, I followed all these steps but there's one more thing is when I delete the nginx deployment then all the nodes is not going to scale down. Is there any thing that we can improve this ?
At around the 09:17 mark, my "data" resources do not work. I have data "aws_eks_cluster" "default" { name = module.eks.cluster_id } I get: │ Error: Missing required argument │ │ with data.aws_eks_cluster.default, │ on eks.tf line 47, in data "aws_eks_cluster" "default": │ 47: name = module.eks.cluster_id │ │ The argument "name" is required, but no definition was found. I tried created an output based on module.eks.cluster_id, and I don't get one (although I can get one for module.eks.cluster_name.
Actually, just noticed in the module documentation: Description: The ID of the EKS cluster. Note: currently a value is returned only for local EKS clusters created on Outposts If you change the definition of those data fields from cluster_id to cluster_name, it works!
One more question if you don't mind :) When I run locally everything is provisioned, when I use GitLab CI (with oidc), everything is provisioned until this step: "module.eks.kubernetes_config_map_v1_data.aws_auth[0]: Creating... Error: kube-system/cluster-autoscaler failed to create kubernetes rest client for update of resource: Unauthorized What resources should be created in addition so that the pipeline is successful, might you have hints? :) Cheers!
Only user/iam role that initially created EKS cluster has access to aws auth config map. First check if you use the same entity to modify aws config map also check how kubernetes provider is setup in your live env. github.com/antonputra/tutorials/blob/main/lessons/160/git-infrastructure-live/dev/kubernetes-addons/terragrunt.hcl#L38-L52
@@AntonPutra only got around to fixing it now; the problem was that the gitlab runner on the TF apply stage didn't have aws cli installed; really hope you'll make longer tutorials for deploying EKS via GitLab CI or Github actions with a sample app, e.g. a DB and a public facing app, so together with ArgoCD.. maybe encorporating hashicorp vault..? Thanks so much, your channel + That DevOps Guy are godsends.
@@rohitnarula7038 EKS has HA control plane setup and even if the control plane go down for a little bit, it only means that Kubernetes API server is down. You won't be able to deploy new applications to Kubernetes and some other features such as scheduler won't be available, but it will not affect your application (no downtime for the app). When upgrading the nodes, K8s first drain the node than reschedule the pod, so I would highly recommend to define Pod Disruption Budget (PDB) to minimize the damage to your app. With PDB you can say for example that only 1 pod at a time can be down if you have 5 or 10 of them.
in eks module version 19.10.0 i think alrd exist "ingress_allow_access_from_control_plane" ports build in, so if we use "node_security_group_additional_rules" we take arror that we have duplicate security group. I am right?
I'm having with the section involving IAM. This block: manage_aws_auth_configmap = true aws_auth_roles = [ { rolearn = module.eks_admins_iam_role.iam_role_arn username = module.eks_admins_iam_role.iam_role_name groups = ["system:masters"] }, ] I have it in my eks module after eks_managed_node_groups. The terraform syntax profiler doesn't like it and when I do a terraform plan, it errors out: │ An argument named "manage_aws_auth_configmap" is not expected here. │ An argument named "aws_auth_roles" is not expected here.
Try to use exactly the same Terraform and Terraform module version. After you successfully provision your infrastructure, you can start upgrading one by one.
🔴 - To support my channel, I’d like to offer Mentorship/On-the-Job Support/Consulting - me@antonputra.com
👉 [UPDATED] AWS EKS Kubernetes Tutorial [NEW]: ruclips.net/p/PLiMWaCMwGJXnKY6XmeifEpjIfkWRo9v2l&si=wc6LIC5V2tD-Tzwl
this is the best k8s setup tutorial in the whole youtube i swear
Thanks Alex =)
I'm impressed by how clearly you expressed your knowledge in a short but saturated video.
Thanks!
Another Master Class Anton! My friend and I worked thru this yesterday and had two EKS clusters up and running. Thank you for all that you do!
Incredible tutoria, you rock Anton!! Greetings from Argentina!
This is a great tutorial, you are helping me alot for acquiring my first devops position, i have an home assignment to provision a vpc and eks cluster that integrates with ecr and this is great and very relevant.
Altough im having issues with "allow_assume_eks_admins_iam_policy" module which says i have some syntax error in the hidden directory .terraform within the project, weird but i'll troubleshoot it. Thanks alot!!
Thanks Ariel!
Just what I was looking for. Perfect timing. Thanks for uploading.
welcome!
Indeed, quite handy and Insight. Any chance on Azure AKS
thanks! i have one for azure - ruclips.net/video/8HmReos6dlY/видео.html
Thank you for your work and your great videos.
could you please show me how to add karpenter for scaling to eks module.
Thanks, I have a lesson - studio.ruclips.net/user/videoC_YZXpXwtbg/edit
awsome tutorial i created the EKS Cluster but having issue with load balancer it througing an error Kuberntess cluster unreachable :the server has asked for the client to provide credentials . can you assist please thank you
I found your channel and I am absolutely in love with your content dude! Huge thanks.
By any chance - on 12:15min do you know how we can specify a private repo image from ECR in the deployment? For example deploy wordpress.
Thanks! It's going to be in the following format (just copy from the ecr)
.dkr.ecr..amazonaws.com/:
thank you Anton, you always bring interessant subjects
thanks Habib
Do you plan on creating a wordpress guide anytime soon that's deployed through terraform to EKS and uses RDS ? I have difficulties creating my yaml deployments and services and not quite sure if you are able to help on that.
I think I can do it in the future. You need to use EFS with readwritemany to mount the same volume to database and wordpress itself.
nice article, can we pass kubernetes/helm provider to child modules from root module, if both are in saperate folders?
Thanks, you can use terraform_remote_state
developer.hashicorp.com/terraform/language/state/remote-state-data
@@AntonPutra i thought provider "kubernetes { }..how can i apss to child modules?
Really helpful!! Thanks a bunch.
You're welcome!
Great tutorial, I replicated all this and I moved a step ahead with setting up rbac but I got stuck with error authorizing namespace setup and other resources related for the rbac
Thanks, generally only user or role that created EKS cluster would have admin access (for example update config map in kube-system ns). Check with kubectl auth can-i "*" "*"
Thanks a lot for the lesson, much better than the one with just AWS resources as here the cluster autoscaler is also part of terraform;
One question though: when scaling up, e.g. from 1 to 2 pods, clearly you get more nodes because of the auto-scaler (also 2), but then when decreasing pods to 1 in the deployment, num. of pods gets reduced to 1 but not amount of nodes (so still 2).
What happens is that when I create an auto-scaler, I get 2 nodes and my desired num. of nodes is changed to 2 instead of initial 1.
Is this intended? and if yes, how do I also destroy the nodes to scale down (keep desired at 1)?
Cheers!
Thanks! It should scale down to 1 in about 5-10 minutes. You can also configure desired profile on the autoscaler using cli flags or helm variables.
I encountered an issue when deploying microservices with this solution. When my own API gateway was hosted in the SPOT instance and the other node API servers and databases were in the general instance, then the communication between the pods did not work. I fixed the issue by making the general instance larger so all the pods were hosted there from the start. Is cross-node communication blocked by default, do I have to change security group rules or is there something else I am missing?
No, it's not blocked. Maybe it was in the termination phase, it's hard to say. But it's totally common to create spot and on-demand instances in the same cluster, and networking should work.
I have updated video with update source, pls take a look - ruclips.net/video/5XpPiORNy1o/видео.html
can u a do video on eks fargate core dns patching using terraform and python lambda function, as described in aws documentation
I'll take a look
Thanks!
Kudos for the Tutorial,
Also, could you show how to use karpenter module to the same setup without clusterautoscaler.
Thank you Ashwaq!
Well i have a tutorial (Karpenter) - github.com/antonputra/tutorials/tree/main/lessons/114
Its spot on. gr8 quick question why can we use aws_auth_roles = [ { rolearn = "arn:aws:iam::66666666666:role/role1" username = "role1" groups = ["system:masters"] }, input of terraform-aws-eks module for adding role to EKS?
Hi , awesome video, I have been following you since long, really like all the stuff you share. I am just stuck on creating service account through terraform for eks? as I have to manually do kubectl apply -f serviceaccount.yaml for load balancer (EKS), how can I automate this part, will appreciate it, if you can provide some knowledge
Thanks AMMAR, you can use kubectl terraform provider or kubernetes provider.
registry.terraform.io/providers/gavinbunney/kubectl/latest/docs
@@AntonPutra
Really helpful !! Can you make an video for how to do authorization and authentication in ingress controller instead at pod level for microservices running under kubernetes EKS
Welcome, do you mean something like network policies, istio or something else? mTLS?
@@AntonPutra Yes istio and can you also make video for how to use stickiness session for AWS ALB and NGINX load balancer ?
Hey. Thanks for the awesome tutorial. You are the first one who was capable to make terraform tutorial which actually works). If you don’t mind I have a small question. I followed you tutorial and all works find but if I try to add something (another helm chart for example) terraform recreates aws_config map every time I do terraform apply. Any ideas why it’s happening?
recreates or updates? you could add ignore block on the terraform resource but i don't think that's the issue
@@AntonPutra it do in-place upgrade every time even if terraform code has been unchanged. And every time it replace group arn.
hi sir, is this a production grade setup or do we need some changes?
Yes, but If you want to further harden it, you would change one parameter to make it a private cluster. This means the control plane endpoint becomes private, and you would need a VPN or a bastion host.
@@AntonPutra I see, thank you :)
@@AntonPutra also what type of VPN is normally used by companies? I know of OpenVpn from AWS marketplace but are there any cheaper alternatives? Or methods to setup VPN?
Great tutorial man thank you so much! One question.
When I run tf destroy the load balancer is not deleted, which leads to issues with deleting other resources like VPC. How would you fix this?
Thanks! You need to first delete all resources from k8s like ingresses (and services of type LoadBalancer) that creates load balancers
@@AntonPutra you mean manually?
@@tomasferrari92 kubectl delete -f file.yaml
@@AntonPutra got it. Thanks!
Hi, thanks for the video. I implemented the code. It is working except disk_size . Even though this parameter is set to 128 default disk size of 20 is applied to eks nodes.Any suggestions on how to implement it.with launch templates into the current code is not working
try to create custom launch template
@@AntonPutra can we use custom launch template with the current eks module used in the demo. Appreciate any references. Thanks much
@@preethi-devops Here is an example - github.com/antonputra/tutorials/blob/482596538abd03dd823cd0fe42e5ae2ea77a0765/lessons/102/terraform/7-nodes.tf#L76-L89
@@AntonPutra can’t we use it with eks module rather than creating from resource types. I need to use launch template with existing code with eks module just to change disk size
@@preethi-devops Yes you can "Support for custom AMI, custom launch template, and custom user data including custom user data template"
Try to use that launch template in `self_managed_node_groups` block in the module
Thanks for the great video. I follow the steps and had a problem in 2 cases. 1. during a failure at the first deployment I tried to rerun TF and got resources that already exist on the cloud watch IAM group and role. Then I tried to destroy the entire stack and got another error. The only way to redeploy is to delete all zombies manually because they are not fully deleted. how do you suggest dealing with it?
I had similar issue with netscope client, disable it before running terraform (or similar monitoring tool on your latop). After terraform fails in the middle, you can either import resources (manually) or delete them (manually)...
@@AntonPutra Thanks for your answer. Import actually doesn't work. I deleted them manually. Wanted also to ask you regarding "AWS Load Balancer Controller" is there a way to run 2 different controllers (one for internal and second for ext). I'm asking because they should run with a different annotations. also what is the exact different between was load balancer controller to ingress-nginx (in ingress nginx I managed to run 2 different ingresses)Thank you !
Whats this lesson? I am appling this code with terragrunt and not work, this error: permissions_boundary must be 20 characters or higher (aws_iam_role_invalid_permissions_boundary)
did you misconfigur iam role? based on the error this one seems similar to yours - github.com/hashicorp/terraform-provider-aws/issues/5546
Thank you
my pleasure
Hi, I followed all these steps but there's one more thing is when I delete the nginx deployment then all the nodes is not going to scale down. Is there any thing that we can improve this ?
Check the logs, make sure that autoscaler actually schedules the node for termination
At around the 09:17 mark, my "data" resources do not work.
I have
data "aws_eks_cluster" "default" {
name = module.eks.cluster_id
}
I get:
│ Error: Missing required argument
│
│ with data.aws_eks_cluster.default,
│ on eks.tf line 47, in data "aws_eks_cluster" "default":
│ 47: name = module.eks.cluster_id
│
│ The argument "name" is required, but no definition was found. I tried created an output based on module.eks.cluster_id, and I don't get one (although I can get one for module.eks.cluster_name.
Actually, just noticed in the module documentation:
Description: The ID of the EKS cluster. Note: currently a value is returned only for local EKS clusters created on Outposts
If you change the definition of those data fields from cluster_id to cluster_name, it works!
Is the LoadBalancer Controller really necessary my friend?
soon, original cloud controller that responsible for create lbs in aws will be removed, same happened with ebs storage class
How to know if the Cluster Autoscaler actually works?
One more question if you don't mind :)
When I run locally everything is provisioned, when I use GitLab CI (with oidc), everything is provisioned until this step:
"module.eks.kubernetes_config_map_v1_data.aws_auth[0]: Creating...
Error: kube-system/cluster-autoscaler failed to create kubernetes rest client for update of resource: Unauthorized
What resources should be created in addition so that the pipeline is successful, might you have hints? :)
Cheers!
Only user/iam role that initially created EKS cluster has access to aws auth config map. First check if you use the same entity to modify aws config map also check how kubernetes provider is setup in your live env. github.com/antonputra/tutorials/blob/main/lessons/160/git-infrastructure-live/dev/kubernetes-addons/terragrunt.hcl#L38-L52
@@AntonPutra only got around to fixing it now; the problem was that the gitlab runner on the TF apply stage didn't have aws cli installed; really hope you'll make longer tutorials for deploying EKS via GitLab CI or Github actions with a sample app, e.g. a DB and a public facing app, so together with ArgoCD.. maybe encorporating hashicorp vault..?
Thanks so much, your channel + That DevOps Guy are godsends.
@@mantikoragio6555 got it thanks. Yes i think to have a sample app definitely can help
Hi guys, I am looking for eks managed groups , is this the same?
Yes =)
github.com/antonputra/tutorials/blob/main/lessons/125/terraform/2-eks.tf#L20
Love it 🔥
I'm glad
Can you showcase how to upgrade the cluster, not via AWS UI . Is there any other way to do it
It's very easy, update the control plane version and run terraform apply, then node pools one by one
@@AntonPutra but that would result in down time . As mater plane will go down . Nodes will be fine as they will go down one by one
@@rohitnarula7038 EKS has HA control plane setup and even if the control plane go down for a little bit, it only means that Kubernetes API server is down. You won't be able to deploy new applications to Kubernetes and some other features such as scheduler won't be available, but it will not affect your application (no downtime for the app). When upgrading the nodes, K8s first drain the node than reschedule the pod, so I would highly recommend to define Pod Disruption Budget (PDB) to minimize the damage to your app. With PDB you can say for example that only 1 pod at a time can be down if you have 5 or 10 of them.
@@AntonPutra Thanks alot will try that
@@rohitnarula7038 no problem
Monitor EKS & EC2 instances with MANAGED Prometheus - ruclips.net/video/-nUQNFAX5TI/видео.html
Get Full-Length High-Quality DevOps Tutorials for Free - Subscribe Now! - ruclips.net/user/AntonPutra
in eks module version 19.10.0 i think alrd exist "ingress_allow_access_from_control_plane" ports build in, so if we use "node_security_group_additional_rules" we take arror that we have duplicate security group. I am right?
Perhaps, I haven't tried it
@@AntonPutra can you help, how we can add custom headers in this setup?
@@ArtemKovalchukl I don't think that you can add custom headers at the ALB level. You can do it on nginx ingress controller with configmap.
@@AntonPutra do you have some examples how to use nginx ingress instead alb with setup like in this tutorial?
@@ArtemKovalchukl github.com/antonputra/tutorials/tree/main/lessons/082
🟢 [New] Terragrunt Tutorial: Create VPC, EKS from Scratch! (Step-by-Step) - ruclips.net/video/yduHaOj3XMg/видео.html
Hi, thanks for the video. Can you also show for GKE
Yes, sure
❤Performance Benchmarks Playlist - ruclips.net/p/PLiMWaCMwGJXl-h2RgOSpdO-pQaSRwlVjd
How to add efs in this eks cluster which you explained
here is an example from my another video - github.com/antonputra/tutorials/blob/main/lessons/113/terraform/13-efs.tf
👉 How to Manage Secrets in Terraform - ruclips.net/video/3N0tGKwvBdA/видео.html
👉 Terraform Tips & Tricks - ruclips.net/video/7S94oUTy2z4/видео.html
👉 ArgoCD Tutorial - ruclips.net/video/zGndgdGa1Tc/видео.html
I'm having with the section involving IAM. This block:
manage_aws_auth_configmap = true
aws_auth_roles = [
{
rolearn = module.eks_admins_iam_role.iam_role_arn
username = module.eks_admins_iam_role.iam_role_name
groups = ["system:masters"]
},
]
I have it in my eks module after eks_managed_node_groups. The terraform syntax profiler doesn't like it and when I do a terraform plan, it errors out:
│ An argument named "manage_aws_auth_configmap" is not expected here.
│ An argument named "aws_auth_roles" is not expected here.
Try to use exactly the same Terraform and Terraform module version. After you successfully provision your infrastructure, you can start upgrading one by one.
The latest versions of the module are using access_entries to setup access instead of configmap.
Example:
access_entries = {
# One access entry with a policy associated
admin-role-entry = {
kubernetes_groups = []
principal_arn = "arn:aws:iam::example_account_id:role/example_role"
policy_associations = {
admin-policy = {
policy_arn = "arn:aws:eks::aws:cluster-access-policy/AmazonEKSAdminPolicy"
access_scope = {
type = "cluster"
}
}
}
}
}
@@ZooMweBG Thanks.
I followed the tutorial exactly step by step however I got stuck at 9:29 The configmap "aws-auth" does not exist
add namespace "kubectl get cm aws-auth -n kube-system -o yaml"
@@AntonPutra error: You must be logged in to the server (Unauthorized)
supper
thanks
it would be awesome to share the source of code to us.😀
It's in the video description - github.com/antonputra/tutorials/tree/main/lessons/125