- Видео 84
- Просмотров 44 093
ZenML
Германия
Добавлен 26 июн 2020
ZenML is an extensible, open-source MLOps framework for creating portable, production-ready MLOps pipelines. Built to enable collaboration among data scientists, ML Engineers, and MLOps Developers, it has a simple, flexible syntax, is cloud- and tool-agnostic, and has interfaces/abstractions that are thoughtfully designed for ML workflows.
ZenML is not here to replace the great tools that solve these individual problems. Rather, it offers an extensible framework and a standard abstraction to write and build your workflows.
🚀 Website - zenml.io
☁️ Sign up - cloud.zenml.io
📕 Documentation - docs.zenml.io
⭐ GitHub Repository - github.com/zenml-io/zenml
🧵 End-To-End Examples - github.com/zenml-io/zenml-projects
📞 Join our Slack channel - zenml.io/slack
ZenML is not here to replace the great tools that solve these individual problems. Rather, it offers an extensible framework and a standard abstraction to write and build your workflows.
🚀 Website - zenml.io
☁️ Sign up - cloud.zenml.io
📕 Documentation - docs.zenml.io
⭐ GitHub Repository - github.com/zenml-io/zenml
🧵 End-To-End Examples - github.com/zenml-io/zenml-projects
📞 Join our Slack channel - zenml.io/slack
ZenML Fundamentals: Your First Step into Production MLOps
🔥 Master the Fundamentals of ZenML: Your Gateway to Production MLOps
✔️ Get started with ZenML: github.com/zenml-io/zenml
☁️ Try ZenML Pro: cloud.zenml.io
📞 Join our community: zenml.io/slack
In this guide, we break down the essential concepts of ZenML and show you how to build production-ready ML/AI pipelines. Learn how ZenML simplifies your MLOps and GenAI workflows and helps you manage your entire ML lifecycle - from classical ML models to agentic workflows using LLMs!
Chapters:
0:00 Introduction
01:04 Pipelines and runs
02:39 Understanding Stacks
05:42 Artifacts and Materializers
06:59 Models
09:05 Secrets & Service Connectors
11:45 ZenML Pro Concepts
13:19 Organizations
13:27 Tenants
13:55 Users a...
✔️ Get started with ZenML: github.com/zenml-io/zenml
☁️ Try ZenML Pro: cloud.zenml.io
📞 Join our community: zenml.io/slack
In this guide, we break down the essential concepts of ZenML and show you how to build production-ready ML/AI pipelines. Learn how ZenML simplifies your MLOps and GenAI workflows and helps you manage your entire ML lifecycle - from classical ML models to agentic workflows using LLMs!
Chapters:
0:00 Introduction
01:04 Pipelines and runs
02:39 Understanding Stacks
05:42 Artifacts and Materializers
06:59 Models
09:05 Secrets & Service Connectors
11:45 ZenML Pro Concepts
13:19 Organizations
13:27 Tenants
13:55 Users a...
Просмотров: 194
Видео
VS Code Extension Updates: DAG Visualizer and Stack/Component Registration
Просмотров 2796 месяцев назад
The ZenML VS Code Extension has just been updated to include DAG visualization and stack and component registration/updates/deletions. Thanks to Christopher Perkins and Erik Wiens for this open-source contribution! Download / install the extension here: marketplace.visualstudio.com/items?itemName=ZenML.zenml-vscode
Easier MLOps Stacks 3: Infrastructure-as-Code with ZenML Terraform Modules for AWS, GCP, and Azure
Просмотров 3616 месяцев назад
Docs: docs.zenml.io/how-to/stack-deployment/ Website: zenml.io GitHub: github.com/zenml-io/zenml Discover how ZenML's new Terraform modules are changing the game for MLOps teams! In this video, we demonstrate: ✅ Seamless infrastructure provisioning and ZenML stack registration ✅ Streamlined deployment across AWS, GCP, and Azure ✅ Automatic configuration of tools and resources for ML pipelines S...
Easier MLOps Stacks 2: Easy MLOps Stack Registration Wizard
Просмотров 1116 месяцев назад
Docs: docs.zenml.io/how-to/stack-deployment/register-a-cloud-stack Website: zenml.io Hey everyone, Baris back again, introducing the new easy stack registration wizard starting from version 0.62.0. This feature simplifies stack creation, crucial for running pipelines remotely, by easily inputting cloud credentials and creating a stack from your choices. Users can access it via the dashboard or ...
Easier MLOps Stacks I: Introducing 1-click cloud stack deployments
Просмотров 3126 месяцев назад
Docs: docs.zenml.io/how-to/stack-deployment/deploy-a-cloud-stack Website: zenml.io Hey everyone, I'm Baris from the ZenML team, introducing our new one-click deployment tool starting from version 0.61.0. This feature simplifies stack creation, crucial for running pipelines remotely, by deploying all stack components with a single click. Users can access it via the dashboard or CLI, currently su...
GenAI Model Cards (ZenML Hackathon 2024 Winner)
Просмотров 1027 месяцев назад
"Team A Team" won the internal ZenML hackathon in 2024. The hackathon was themed around GenAI and the winning entry showcased the autogeneration of custom model cards in the dashboard and on the CLI. The model cards take pipeline and model metadata and generate various reports using LLMs. Interested in seeing this feature make its way into the ZenML Dashboard? Send us a message in our Slack com...
Building and Optimizing RAG Pipelines: Data Preprocessing, Embeddings, and Evaluation with ZenML
Просмотров 1,1 тыс.7 месяцев назад
www.zenml.io/newsletter-signup In the latest ZenML webinar, we dove deep into the world of Retrieval-Augmented Generation (RAG) pipelines and how ZenML can streamline your RAG workflows. In this hands-on workshop, Alex guides us through the essential components of building and optimizing RAG pipelines. We cover: * The fundamentals of RAG, discussing why it exists and the problems it solves in...
Automating the Workflow: End-to-end MLOps for Computer Vision
Просмотров 7798 месяцев назад
☁️ Get a managed MLOps control plane for your cloud: cloud.zenml.io 📞 Questions? Ask us on Slack: zenml.io/slack In this hands-on session, we'll focus on building an efficient workflow using tools like Voxel51, ZenML, and Streamlit. Hamza will guide you through creating an automated pipeline for computer vision tasks, demonstrating how you can transition smoothly from data labeling to model tra...
Run your first MLOps pipeline in 11 minutes | Tutorial
Просмотров 3,4 тыс.9 месяцев назад
✔️ Try the code: github.com/zenml-io/zenml-projects/tree/main/train_and_deploy ☁️ Get a managed MLOps control plane for your cloud: cloud.zenml.io 📞 Questions? Ask us on Slack: zenml.io/slack 0:00 Intro 00:42 Clone your MLOps project 03:12 Run your first machine learning pipeline 04:53 Train on the cloud with Google Cloud Platform Vertex AI 07:38 Register models with the ZenML Model Control Pla...
ZenML Cloud - Getting Started
Просмотров 17610 месяцев назад
This video is meant for new users of ZenML Cloud to get started quickly using the starter setup. 💻 GitHub Repository github.com/zenml-io/zenml 📞 Questions? Ask us on Slack zenml.io/slack ☁️ Start now with ZenML Cloud Cloud - cloud.zenml.io 🔥 About ZenML - 🤹 ZenML is an extensible, open-source MLOps framework for creating portable, production-ready machine learning pipelines. By decoupling infra...
[5/5] Huggingface to Sagemaker with ZenML Pipelines - Deploying to AWS Sagemaker Endpoints
Просмотров 325Год назад
While almost every Huggingface model can be easily deployed to AWS Sagemaker endpoints with a few lines of code, it is often desirous to automate this flow, and have this process track the entire lineage of the model as it goes from training to production. Deployment is the ultimate step we use a ZenML pipeline to automate the deployment of the slated production model to a Sagemaker endpoint. T...
[3/5] Huggingface to Sagemaker with ZenML Pipelines - Training a BERT based Sentiment Classifer
Просмотров 213Год назад
While almost every Huggingface model can be easily deployed to AWS Sagemaker endpoints with a few lines of code, it is often desirous to automate this flow, and have this process track the entire lineage of the model as it goes from training to production. The training pipeline orchestrates the interactions with Huggingface to train and store model files and ensure that each component, includin...
[4/5] Huggingface to Sagemaker with ZenML Pipelines - Promoting the model to Production
Просмотров 131Год назад
While almost every Huggingface model can be easily deployed to AWS Sagemaker endpoints with a few lines of code, it is often desirous to automate this flow, and have this process track the entire lineage of the model as it goes from training to production. Following training, the automated promotion pipeline evaluates models against predefined metrics, identifying and marking the most performan...
[2/5] Huggingface to Sagemaker with ZenML Pipelines - Feature engineering with a Huggingface Dataset
Просмотров 265Год назад
While almost every Huggingface model can be easily deployed to AWS Sagemaker endpoints with a few lines of code, it is often desirous to automate this flow, and have this process track the entire lineage of the model as it goes from training to production. Automated feature engineering forms the foundation of this MLOps workflow. Data transformation steps are encoded into pipelines, ensuring re...
[1/5] Huggingface to AWS Sagemaker with ZenML Pipelines - Introduction to the MLOps process
Просмотров 575Год назад
While almost every Huggingface model can be easily deployed to AWS Sagemaker endpoints with a few lines of code, it is often desirous to automate this flow, and have this process track the entire lineage of the model as it goes from training to production. This project showcases one way of using ZenML pipelines to achieve this: - Create and version a dataset in a feature_engineering_pipeline. -...
MLOps Platform Sandbox: Deploy ZenML + Kubeflow + MLflow + Minio in minutes
Просмотров 1,7 тыс.Год назад
MLOps Platform Sandbox: Deploy ZenML Kubeflow MLflow Minio in minutes
ChatGPT Failure Hooks with ZenML Pipelines
Просмотров 177Год назад
ChatGPT Failure Hooks with ZenML Pipelines
Using Failure and Success Hooks with ZenML
Просмотров 147Год назад
Using Failure and Success Hooks with ZenML
Creating stacks and stack components via the ZenML Dashboard 🌌
Просмотров 288Год назад
Creating stacks and stack components via the ZenML Dashboard 🌌
Outsourcing image builds for your MLOps pipeline to Kaniko, Google Cloud Build, and more!
Просмотров 205Год назад
Outsourcing image builds for your MLOps pipeline to Kaniko, Google Cloud Build, and more!
Log ML Metadata with Neptune - ZenML Integration
Просмотров 1202 года назад
Log ML Metadata with Neptune - ZenML Integration
Tool Agnostic MLOps with ZenML - By Hamza Tahir at MLOps Community Munich
Просмотров 1,2 тыс.2 года назад
Tool Agnostic MLOps with ZenML - By Hamza Tahir at MLOps Community Munich
Deploy and Scale Models with BentoML
Просмотров 1,3 тыс.2 года назад
Deploy and Scale Models with BentoML
Team Fuzzy Labs - 1st Place in ZenML Month of MLOps 2022
Просмотров 1032 года назад
Team Fuzzy Labs - 1st Place in ZenML Month of MLOps 2022
Shivalika Singh - 3rd Place in ZenML Month of MLOps 2022
Просмотров 772 года назад
Shivalika Singh - 3rd Place in ZenML Month of MLOps 2022
Andrew Hinh & Aleks Hiidenhovi - Most Promising Entry in ZenML Month of MLOps 2022
Просмотров 452 года назад
Andrew Hinh & Aleks Hiidenhovi - Most Promising Entry in ZenML Month of MLOps 2022
Team Two - 2nd Place in ZenML Month of MLOps 2022
Просмотров 392 года назад
Team Two - 2nd Place in ZenML Month of MLOps 2022
Marina Wyss - Most Promising Entry in ZenML Month of MLOps 2022
Просмотров 452 года назад
Marina Wyss - Most Promising Entry in ZenML Month of MLOps 2022
Thanks! Good work very nice
Wow nice! LLM Engineer Handbook sent me
Hi! Amazing tool! Is it posible to integrate it with a already deployed OIDC like keycloak in a kubernetes cluster?
It is! For now though, thats only possible with ZenML Pro: zenml.io/pro
This doesn't look like production-ready at all. Missing CI/CD - git workflow, Mono repo -project structure, auto execution of pipeline (periodic), incremental data load, etc
Can only do so much in one video! But this was a 2 year old demo. Try out the latest stuff with all that and more here: docs.zenml.io/user-guide/production-guide
Your work is great
i couldn't find where the /predict api was created, or how it was created. could you point me at the right bit of code?
Sure! The relevant code snippets are here: github.com/zenml-io/zenml-projects/tree/main/train_and_deploy/steps/deployment github.com/zenml-io/zenml-projects/blob/main/train_and_deploy/pipelines/deployment.py
Hi @Hamza can we have github code repository for this exercise.
Yes sure, the code is here: github.com/zenml-io/zenml-projects/tree/main/llm-complete-guide
Congrats Hamza. wow can't believe it's your first video, you crushed it. Looking forward to many more 💯
Thank you Saboor, we'll pass along the compliments!
Loved it sir
Thank you so much!
Great series 🎉
basically need help with zenml + docker, is their a way to use a cache from previous image when running the docker compose run ?
Join our slack for support: zenml.io/slack
hello i try zenml in my ml project but i'm using docker and every time i run docker container it run's new pipeline and i'm not able to use cache any help !!!
why not you uploaded the high resolution videos? Its hardly see the video quality is worst.
How did you end up creating a video at 720p in today's world? I guess I'm spoiled with 4K UHD monitors and content.
Thank you sir, please how can i go about viewing the dashboard in windows PC?
ZenML is like coding in MOJO, great 👏👏👏
It's just amazing
This is incredible. Having a small knowledge of MLOps, you can start having fun and start experimenting/giving your first steps with ZenML, and then dive deeper as you wish. Thanks!
Excellent
🎉 wow this is great!
That pipeline syntax… it looks like it destroys autocompletion in your editor :(
Hi I am getting error on the step zenml stack recipe deploy k3d-modular --no-server Error- invalid character 'U' in string escape code
This video is slightly outdated and we need to shoot a new one soon! Maybe for now try just following the example: github.com/zenml-io/zenml/tree/main/examples/kubeflow_pipelines_orchestration here and join the slack community to ask for help: zenml.io/slack
This does not work. It says "zenml.integrations.sklearn.helpers" does not exist. also -f is not a valid option on the integration install
Yes indeed this video is a year old and ZenML has progressed a lot since then! Try out the new README in the example to see the latest code: github.com/zenml-io/zenml/tree/main/examples/kubeflow_pipelines_orchestration
How to deploy models using BentoML on Azure Databricks?
LOVED IT! 😍
Thank you!!
Impressive
Glad you liked it!
Great
Glad you like it!
Can we add a testing module as a step?
Yes for sure! You can add normal tests and data tests with the data validators (docs.zenml.io/v/docs/mlops-stacks/data-validators)
prime content
Thank you Sulaiman!
If someone is getting error by following command:- zenml stack register local_kubeflow_stack \ -m local_metadata_store \ -a local_artifact_store \ -o kubeflow_orchestrator \ -c local_registry then you should see the name of your default components, assuming it's default, you have to do:- zenml stack register local_kubeflow_stack -m default -a default -o default -c local_registry and If you want to confirm this is the exact name, then you can also do zenml metadata-store list and similarly for other components.
OMG This is awesome!!!! I was very curious how would you run this on vertex had a workaround but this is super helpful!
Glad you liked it!
Congrats on your seed round!! Many more to come!
Thanks a bunch!