MLflow Pipelines: Accelerating MLOps from Development to Production

Despite being an emerging topic, MLOps is hard and there are no widely established approaches for MLOps. What makes it even harder is that in many companies the ownership of MLOps usually falls through the cracks between data science teams and production engineering teams. Data scientists are mostly focused on modeling the business problems and reasoning about data, features, and metrics, while the production engineers/ops are mostly focused on traditional DevOps for software development, ignoring ML-specific Ops like ML development cycles, experiment tracking, data/model validation, etc.
In this talk, we will introduce MLflow Pipelines, an opinionated approach for MLOps. It provides predefined ML pipeline templates for common ML problems and opinionated development workflows to help data scientists bootstrap ML projects, accelerate model development, and ship production-grade code with little help from production engineers.
Connect with us:
Website: databricks.com
Facebook: / databricksinc
Twitter: / databricks
LinkedIn: / data. .
Instagram: / databricksinc

[@swatikarot8272]

Love this. Thanks for the great session. 👍

[@ousmanetraore597]

Why every one using yaml everywhere? with no code completion, difficult to test/validate, every thing needs to be in a single huge file because we can't use function abstraction ? This is fine for simple "transform"-> "train" -> "test" pipeline, but become very hard for complexe ones. I prefer the Airflow way of defining pipelines with Python code.

[@risebyliftingothers]

managing airflow infra in house is a task in itself. flexibility comes at a cost. and btw yaml is what kubernetes thrives on and most of infra-as-code tools :)

[@bharathjc4700]

How do we move the artifacts to prodiution

[@jerryyang7270]

This is great!

[@LavaKafleNepal]

wow awesome

[@rohitchatterjee2327]

this was a very good session

[@sitrakaforler8696]

Cool

[@geleshgomathil3274]

Notebook & Slides Link