- Видео 31
- Просмотров 29 243
Samos123
Добавлен 19 сен 2013
Видео
Tutorial: Weaviate with Local Embedding and Generation models using KubeAI
Просмотров 1,3 тыс.2 месяца назад
KubeAI, private Open AI on K8s: github.com/substratusai/kubeai Tutorial with copy pasteable steps: github.com/substratusai/kubeai/blob/main/docs/tutorials/weaviate.md Learn how to deploy Weaviate, Embedding and Generate models all within the same K8s cluster. It works on both CPU or GPU, but in the tutorial CPU is used.
Demo: Private RAG with local Mistral 7b LLM and Weaviate on K8s
Просмотров 1,7 тыс.5 месяцев назад
Live demo that shows how to deploy an end-to-end Retrieval Augmented Generation stack including the LLM and Embedding Server on top of any K8s cluster. Lingo as the model proxy and autoscaler: github.com/substratusai/lingo Verba as the RAG application: github.com/weaviate/Verba Weaviate as the Vector DB: github.com/weaviate/weaviate Mistral-7B-Instruct-v2 as the LLM STAPI with MiniLM-L6-v2 as t...
Deploying Mixtral on GKE with just 2 x 24 GB L4 GPUs
Просмотров 1,9 тыс.9 месяцев назад
Lingo, open source ML Proxy and autoscaler for K8s: github.com/substratusai/lingo Blog post with copy pasteable instructions: www.substratus.ai/blog/deploying-mixtral-gptq-on-gke-l4-gpus Learn how to deploy Mixtral on GKE using just 2 x L4 24 GB GPUs. We do this by using GPTQ which loads Mixtral on 4 bit mode. 0:00 - Introduction 0:12 - Calculating GPU memory required for Mixtral with GPTQ 1:40...
Kind + GPU: A Local K8s cluster that can run GPU pods
Просмотров 568Год назад
Blog post: www.substratus.ai/blog/kind-with-gpus 0:00 Intro 0:18 Fun background on the PR that wasn't needed 2:28 Step by step tutorial using the blog post to configure kind with support 8:00 Verifying we can create a GPU pod
Substratus.ai - Deploying and finetuning LLMs intro and demo
Просмотров 190Год назад
Github repo: github.com/substratusai/substratus Try it yourself: www.substratus.ai/docs/quickstart Substratus.ai lets you deploy and finetune Open Source LLMs without needing to write. Substratus itself is free and open source. Currently only supports GCP, want it on another cloud provider? Let us know! Book a session with me to get it up and running in your environment: topmate.io/samos123
K8s Preventing Privileged Pods using Pod Security Admission / Standards
Просмотров 2,5 тыс.Год назад
Prevent privileged pods from being created to improve security of your K8s cluster. This video shows how to do this using Pod Security Admission/Standards through a fun hands-on screenshare with explanations. Blog post: samos-it.com/posts/Preventing-Privileged-pods-using-Pod-Security-Admission-Standards.html Content 0:00 Introduction 1:42 Background on Pod Security Standards and Admission 2:43 ...
ChatGPT Python Unofficial Library tutorial with screenshares
Просмотров 1 тыс.Год назад
Blog post with copy paste able instructions: samos-it.com/posts/chatgpt-python-using-an-unofficial-library.html Library used: github.com/acheong08/ChatGPT This video shows how to use one of the unofficial Python libraries for ChatGPT. You can use this to incorporate ChatGPT in your personal project until ChatGPT releases an official library. Content: 00:00 Introduction 00:16 Setting up virtuale...
Playing tag in a rainy day at Japanese Garden
Просмотров 26Год назад
Can't sit inside all day when it rains. Umbrellas and a game of tag while taking shelter in the Hayward Japanese Garden.
K8s Validating Admission Policy AKA CEL Admission Hands-on Tutorial New in 1.26
Просмотров 419Год назад
Live demo trying out the new Validating Admission Policy fom K8s 1.26 which is also known as CEL admission policy. Commands can be copy pasted from blog: samos-it.com/posts/trying-cel-admission-control-validation-admission-policy.html Outline: 0:00 Introduction and agenda 0:22 Creating a K8s 1.26 cluster using minikube with ValidatingAdmissionPolicy enabled 1:27 Creating a ValidatingAdmissionPo...
Deploying K8s on your laptop with Minikube - Hands-on lab / screenshare
Просмотров 242Год назад
Deploying K8s on your laptop with Minikube - Hands-on lab / screenshare
K8s Migrating from PSP to PSA with pspmigrator and mutating PSPs
Просмотров 4472 года назад
K8s Migrating from PSP to PSA with pspmigrator and mutating PSPs
K8s Migrating from PSP to PSA with mutating PSP
Просмотров 5272 года назад
K8s Migrating from PSP to PSA with mutating PSP
K8s Migrating from PSP to PSA - easy and fast path
Просмотров 1,2 тыс.2 года назад
K8s Migrating from PSP to PSA - easy and fast path
Live demo to learn What is Network Address Translation (NAT)?
Просмотров 1022 года назад
Live demo to learn What is Network Address Translation (NAT)?
K8s GKE Pod Conatiner Networking and ip-masq-agent deep-dive
Просмотров 1,4 тыс.2 года назад
K8s GKE Pod Conatiner Networking and ip-masq-agent deep-dive
PSPMigrator Demo - Migrating from PSP to PSA CLI tool
Просмотров 1,2 тыс.2 года назад
PSPMigrator Demo - Migrating from PSP to PSA CLI tool
Arthurx at Fremont Slap Face Coffee Open Mic
Просмотров 656 лет назад
Arthurx at Fremont Slap Face Coffee Open Mic
Openstack node install with Fuel stuck on partitioner
Просмотров 16210 лет назад
Openstack node install with Fuel stuck on partitioner
wow looks super cool
You can give it a try on your laptop even: github.com/substratusai/kubeai
can you pelase share git link to this project. its interesting, many thanks
Lingo: github.com/substratusai/lingo Chat UI: github.com/substratusai/aichat-ui vLLM: github.com/vllm-project/vllm
@@Samos123x thanks
Hello! Great video! Thanks for share. Obrigado! :)
Please can i get this for k8s am using containerd as my container runtime
k8s is still using containerd I think. You just need docker, podman or nerdctl on your laptop / device that runs kind.
I really appreicate this - i wanted a dev envrionment and found 'kind' and it was amazing to see it had GPU support ! Saved me hours of fighting with kubernetes and nvidia plugins. Fantastic work.
will be linking to your video in my phd blog :) (alanknipmeyer.phd)
Hey I am new to gke How to become master in gke
Thanks! Great information.
Hey Samos, firstly big thank you for this awesome tutorial! very detailed and to the point! :) I was wondering if you have been able to get this working on WSL2.. I have spent countless days back and forth with no luck! In a nutshell all works up until installing the gpu-operator helm chart, all pods are created and running EXCEPT for the nvidia-* pods (daemonset, exportor, etc..) which do not even appear in the list of pods (kubectl get pods -A) they just never seem to get created. i am able to run nvidia-smi successfully as an FYI and am able to run standalone docker containers with GPU access but having issues when using KinD..
When I process the command: $ helm install --wait --generate-name -n gpu-operator --create-namespace nvidia/gpu-operator --set driver.enabled=false I met an error: FailedMount pod/gpu-feature-discovery-9bnmp MountVolume.SetUp failed for volume "run-nvidia" : hostPath type check failed: /run/nvidia is not a directory But when I created the cluster, I used containerPath: /var/run/nvidia-container-devices/all. Where is this /run/nvidia needed?
Interesting. I haven't seen that before. What host are you running this on?
Thank you Sam! I use Ubuntu22.04 local host. I waited for quite a long time and this error has been somehow automatically solved. And I want to add one more point that we should make sure the system uses cgroup v2, which can be check by 'docker info | grep -i cgroup'. Otherwise it may cause an error 'adding pid to cgroups: failed to write' during executing gpu-operator-XXX-node-feature-discovery-master pod. Thanks again! Your tutorial is very clear and useful! @@Samos123x
super super helpful 👍
Amazing work!
very good video thank you man
Glad you liked it!
Promo sm
dude the only reason i am watching this is because everyone talks about PSS being in the latest security policy but it is now PSA so good work! can you tell me what other security futures can be enabled.
That really depends on what you need from a security perspective. The most secure pod security standard would be applying Restricted but that would most likely require modification to your pod specs.. This section has a good list of other things you can do to secure workloads: kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#controlling-the-capabilities-of-a-workload-or-user-at-runtime
hahaha, see you again! sam
like 911 inform?
informative!
*deer
TIL. Thanks!
🥺🥰🤩
que hermosura de bebe
❤..Awee So adorable..Jiji
Clean 🔥
Nice 👍
Good job! Keep it up!
Solid kickflip 👊