Kubernetes Persistent Volumes NFS (AWS | EKS | EFS Storage Class | EFS Provisioner | ReadWriteMany)
HTML-код
- Опубликовано: 7 авг 2024
- 🔴 - To support my channel, I’d like to offer Mentorship/On-the-Job Support/Consulting - me@antonputra.com
👉 [UPDATED] AWS EKS Kubernetes Tutorial [NEW]: • AWS EKS Kubernetes Tut...
▬▬▬▬▬ Experience & Location 💼 ▬▬▬▬▬
► I’m a Senior Software Engineer at Juniper Networks (12+ years of experience)
► Located in San Francisco Bay Area, CA (US citizen)
▬▬▬▬▬▬ Connect with me 👋 ▬▬▬▬▬▬
► LinkedIn: / anton-putra
► Twitter/X: / antonvputra
► GitHub: github.com/antonputra
► Email: me@antonputra.com
▬▬▬▬▬▬ Related videos 👨🏫 ▬▬▬▬▬▬
👉 [Playlist] Kubernetes Tutorials: • Kubernetes Tutorials
👉 [Playlist] Terraform Tutorials: • Terraform Tutorials fo...
👉 [Playlist] Network Tutorials: • Network Tutorials
👉 [Playlist] Apache Kafka Tutorials: • Apache Kafka Tutorials
👉 [Playlist] Performance Benchmarks: • Performance Benchmarks
👉 [Playlist] Database Tutorials: • Database Tutorials
=========
⏱️TIMESTAMPS⏱️
0:00 Intro
2:15 Create EFS File System
6:04 Create Kubernetes NFS Provisioner
12:35 Create EFS Storage Class EKS
18:44 Create NFS Persistent Volume Claim
22:25 Share ReadWriteMany Volume with 2 Apps
=========
Source Code
📚 - Instructions: antonputra.com
🖥️ - GitHub: github.com/antonputra/tutoria...
#Kubernetes #EFS #NFS #AWS
🔴 - To support my channel, I’d like to offer Mentorship/On-the-Job Support/Consulting - me@antonputra.com
👉 [UPDATED] AWS EKS Kubernetes Tutorial [NEW]: ruclips.net/p/PLiMWaCMwGJXnKY6XmeifEpjIfkWRo9v2l&si=wc6LIC5V2tD-Tzwl
Really appreciate the time you put into making this. Your video was extremely helpful. I was having trouble setting up EFS using the "kubernetes-sigs/aws-efs-csi-driver" and never quite got the controller to start. Following your video using "kubernetes-sigs/nfs-subdir-external-provisioner" worked perfectly! I just subscribed. Looking forward to other videos you put out. Thanks again!
Glad it helped!
Great live saver content always. Love your calm and thorough step-by-step approach.
Thank you Anton..you saved me.. my PVC was always showing in pending... i had almost tried all solutions from internet but didn't found proper sol-n and explation.. but your video did it.. i just changed my security group and its working now.. Thank you so much .. sending lots of love to you .... :)
appreciate your effort and way of explaination. must say the best tutorial on efs + eks. expecting more such tutorials
bro you just solved the many issues i was having with mounting volumes
welcome :)
🟢 [New] Terragrunt Tutorial: Create VPC, EKS from Scratch! (Step-by-Step) - ruclips.net/video/yduHaOj3XMg/видео.html
This is very helpful! Thank you.. Keep it up! You deserve more views and subs :)
Thank you Abda)
👉 How to Manage Secrets in Terraform - ruclips.net/video/3N0tGKwvBdA/видео.html
👉 Terraform Tips & Tricks - ruclips.net/video/7S94oUTy2z4/видео.html
👉 ArgoCD Tutorial - ruclips.net/video/zGndgdGa1Tc/видео.html
Hi Anton, thanks for such a clear descriptive tutorial on EFS addon for EKS.
One suggestion, the format of live demo with real-time commentary from you is more appealing than other video format where you pre-record with screenshots. Hope you consider current video format for your next releases.
... and thanks a Million for creating these contents. I have greatly benefitted from your videos. Cheers
👍🏆💯
Thanks for the feedback!
Very helpful, appreciated! Thanks for the explanation
You're welcome!
You are rocking dude. keep going..!
Thanks, will do!❤
Proud of you man ...anton you are great....I wish you will get 1M subscribers 1 day :)
Thank you Piyush:)
@@AntonPutra Brother keep rocking🥰🥰
Get Full-Length High-Quality DevOps Tutorials for Free - Subscribe Now! - ruclips.net/user/AntonPutra
thanks, this helped to migrate from cinder to efs storage
Glad it helped!
Very good explanation for beginners
Glad you liked it
Hi Anton,
Thank you for this great tutorial.
Are there any plans to create a tutorial for aws-efs-csi-driver? This is the official way to use EFS together with the EKS now. A lot of people, including myself, are struggling to get the controller up and running. I think a lot of people might find this helpful.
Thank you!
Thank you, I will create one in the future, at this point a have a similar one with csi driver tutorial to access secret manager. Here is a link - ruclips.net/video/Rmgo6vCytsg/видео.html
Really good video!! I have a question I noticed that pvc creates a new directory in the efs and when you destroy it destroys also. Is there any way to just use the folders that's already in the efs?
Amazing information
Thanks
Hi Anton, thanks for the great video!
With your tutorial I'm able to create EFS and mount to my pods, and write files in the efs directory. but as I change my PVC metadata name and kubectl appy, then the files in my efs are gone. Is there a way to keep all the files in my efs directory?
FYI, as I delete pods and check the files in efs directory, it stays well.
Awesome....request you to do a video on configuring proxy IPs on worker nodes for internet access.
Thanks!
This is very helpful! Thank you, buddy :)
One suggestion: voice is very low. Please increase voice for your upcoming videos :)
Noted
This was very helpful. Thank you for all your videos. can i apply this process to eks fargate clusters too?
Thank you, I haven't tried it myself, but I will defiantly give it a shot sometime in the future.
Thank you!
You're welcome!
Great Stuff, Can you provide some light on how can I share the EFS/ PVC across different namespace. this tutorial deal with sharing storage within the same namespace. Is it possible in first place ?
Thanks, a pvc is a namespaced resource, pods on different namespaces can't share the same pvc. In a few days I'll release a new updated video on how to deal with efs and eks blueprints.
¡Gracias!
Thanks Eneldo!
Hi Sir,thanks for ur time on sharing knowledge. while u created efs in aws console it was showing 6kb,but u allocated 10 mi in pvc.please crt my understanding.
It's elastic, meaning when you create PVC, you can use any number; it wouldn't matter, as PV will grow automatically. It's just that PVC has a mandatory field, and you need to put something there.
hello. this was a great tutorial and helped me a great deal several months ago. I upgraded from kuberentes 1.19 to 1.20 to 1.21 as per AWS. I currently don't have the new elb aws csi driver installed and I am trying to create another pvc using that storageclass in same namespace. However, I am stuck in ExternalProvisioning, waiting for a volume to be created, either by external provisioner "efs-storage" or manually created by system administrator. My first PVC worked great and bound immediately. Googling the error everyone is noting that PCV is one to one with PV but I don't see a PV anywhere in the tutorial. am I doing something wrong or is it not possible to create multiple PVCs against the same efs storageClass in the same namespace?
I've been using this approach with older k8s versions only, I hope someone else can help to solve the issue
great video. can same EFS be shared between multiple EKS clusters? if so how. thanks
I think you can create Kuberentes storage classes based on the same EFS instance. What is your use case? Do you want to share data or just EFS?
@@AntonPutra Thanks for the reply really appreciated. my use case is that I have apps in different eks clusters that need to share persistent data. thanks for any help
at 4:20 SG of mount points should be same as of nodegroup, that solved my issue
cool
Hi you have explained each step brilliantly. I have followed each steps and not get any error but getting this error
Normal ExternalProvisioning 4m7s (x142 over 39m) persistentvolume-controlle r waiting for a volume to be created, either by external provisioner "efs-stora ge" or manually created by system administrator
Hey, check logs on controller and verify that you entered correct security group
Hi Anton,
After creating the deployment and PVC, in deployment logs, I am getting the following error
`provision "default/test-claim" class "efs": unexpected error getting claim reference: selfLink was empty, can't make reference`.
Using AWS-EKS with Kubernetes version 1.21
I may need to update that video if it's related to the verion of K8s
Thanks for the knowledge, but i didn't where the PV was created and if its dynamically, i couldn't fine any PV... I followed it step by step but my pvc is stuck in "Pending" state ... because "waiting for a volume to be created, either by external provisioner "efs-storage" or manually created by system administrator"
Yes you can create "dynamic" storage class, in a few days I'll release a new updated video (eks blueprints + efs and many more)
Hi ... when is the pv created ... I did this step by step but my pvc is stuck in "Pending" state ... because "waiting for a volume to be created, either by external provisioner "efs-storage" or manually created by system administrator"
Seems i need to create updated video.. new api
Hello. I have eks cluster running on kubernetes version 1.21 , I followed all the steps you mentioned in a video, but I am getting below message and pvc is still in pending state
Normal ExternalProvisioning 3m13s (x142 over 38m) persistentvolume-controller waiting for a volume to be created, either by external provisioner "efs-storage" or manually created by system administrator
check the logs of the provisioner and security groups
I have my 100Tb hardware in my room. Could i share it to cloud or make it run as cluster with gks or eks. Thanks
Yes you can, it depends on your cloud provider. There are many different options, for example you can use Site-To-Site Cloud VPN or AWS direct connect. Maybe you want to run only CI/CD on that hardware, in that case you can integrate it with github actions easily to pull jobs and run your builds.
@@AntonPutra thanks, but did you see 100Tb i mentioned. Just for storage. I do know delay or slow latency but it’s just for backup all my media files instead of cloud provider storage. What should i do now. Thanks again
My PVC always showing a message "persistentvolume-controller waiting for a volume to be created, either by external provisioner "efs-storage" or manually created by system administrator". I confirm SecurityGroup for EFS and Nodes are on same. Please help.
take a look on this answer - stackoverflow.com/questions/65376314/kubernetes-nfs-provider-selflink-was-empty
where did u install nfs server on master or node ?? A bit confused Anton!
You just need to deploy EFS provisioner in your cluster. You not going to be able to deploy any pods on masters in EKS since they isolated from you. It's just regular deployment - github.com/antonputra/tutorials/blob/main/lessons/041/efs/7-deployment.yaml#L1-L37
You explain things very well, but, as of K8s v1.20, this code doesn't work, the EFS-provisioner deployment fails with a "SelfLink" problem. If you have the solution to this perhaps you can make an update to this video.
Thanks for letting me know. Will verify and create a new one if it's true.
@@AntonPutra I also got the "Self Link" issue.
@@AntonPutra I also have the Self link issue. Yet to figure out how to resolve it
Hi Buddy,
thanks for the video.
Normal ExternalProvisioning 8s (x2 over 21s) persistentvolume-controller waiting for a volume to be created, either by external provisioner "efs-storage" or manually created by system administrator
I followed you as per your session but after the deployment i could see above message when i'm doing describe pvc
could you please help me.
could you please confirm mountPath: "/data" this directory we have created manually before executing deployment?
Please check the security group for EFS is a external service and k8s controller has to have access to it.
@@AntonPutra issue has been fixed now we have replace the image name with below (deploymeny.yaml
)
k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
@@palanisamy-dl9qe Got it
@@palanisamy-dl9qe thank you very mutch for this tip, save here !!!!
hi Anto
I replicated the steps as yours but my PVC status keep "pending" status, Any hints from you please?
try to describe k8s object to get mode info (kubectl describe )
@@AntonPutra
used command kubectl describe pvs test-claim i got this info, seem it does not tell me where the error. The storageclass was created successfully. Do we need to create a PV before the creating the PVC? In your tutorial i dont see the PV created
Name: test-claim
Namespace: staging
StorageClass: efs
Status: Pending
Volume:
Labels:
Annotations: kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"annotations":{},"name":"test-claim","namespace":"staging"},"spec":{"accessModes":["ReadW...
volume.beta.kubernetes.io/storage-provisioner=efs-storage
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode: Filesystem
@@ThanhNguyenVan-ss7yy I've got the same issue: test-claim has a Pending status... It would be great to resolve the issue... Thanks for any help
Awesome tutorial! Just bare IP address is not nice for viewers. DNS would be probably ideal. Thanks +1 from me.
Thanks for the suggestion
I think deploying nfs provisioner using helm is much easier
Agree, it's much easier to deploy but not to maintain over multiple years. Typically, in large companies, we maintain large Kubernetes clusters for a year or two and then perform migrations due to API changes, etc
what would be the case if need update two IP in regional wise? Just correct me if anything i'm doing wrong here.
env:
- name: PROVISIONER_NAME
value: efs-storage
- name: NFS_SERVER
value: 10.25.35.45
- name: NFS_SERVER
value: 10.25.35.50
- name: NFS_PATH
value: /
volumes:
- name: nfs-client-root
nfs:
server: 10.25.35.45
server: 10.25.35.50
path: /
Do you get an error? Do you have a log?
@@AntonPutra No i don't see any error. but basically we need point right our two NFS server path right because we deployed regional.
The EFS has DNS name that can be used for the distributed storage. otherwise, you can use EBS. Instead of the IP, you should use the DNS name. This tutorial is misleading to the potencial of the EFS.
Why would it be misleading? just because I use ip instead of dns? :)
Checked the NFS client provisioner logs, I see an error message "E0404 03:29:28.919349 1 controller.go:1004] provision "staging/test-claim" class "efs": unexpected error getting claim reference: selfLink was empty, can't make reference" Any help please ?.
Looks like it has something to do with k8s version, check this answer - github.com/kubernetes-sigs/nfs-subdir-external-provisioner/issues/25#issuecomment-742616668