Databricks Asset Bundles: A Standard, Unified Approach to Deploying Data Products on Databricks

In this session, we will introduce Databricks Asset Bundles, provide a demonstration of how they work for a variety of data products, and how to fit them into an overall CICD strategy for the well-architected Lakehouse.
Data teams produce a variety of assets; datasets, reports and dashboards, ML models, and business applications. These assets depend upon code (notebooks, repos, queries, pipelines), infrastructure (clusters, SQL warehouses, serverless endpoints), and supporting services/resources like Unity Catalog, Databricks Workflows, and DBSQL dashboards. Today, each organization must figure out a deployment strategy for the variety of data products they build on Databricks as there is no consistent way to describe the infrastructure and services associated with project code.
Databricks Asset Bundles is a new capability on Databricks that standardizes and unifies the deployment strategy for all data products developed on the platform. It allows developers to describe the infrastructure and resources of their project through a YAML configuration file, regardless of whether they are producing a report, dashboard, online ML model, or Delta Live Tables pipeline. Behind the scenes, these configuration files use Terraform to manage resources in a Databricks workspace, but knowledge of Terraform is not required to use Databricks Asset Bundles.
Talk by: Rafi Kurlansik and Pieter Noordhuis
Connect with us: Website: databricks.com
Twitter: / databricks
LinkedIn: / databricks
Instagram: / databricksinc
Facebook: / databricksinc

[@aviadshimoni175]

Can we get a link to the repo that was used for the demo? TIA

[@tingxie2292]

14:00 demo start

[@user-nu3vc1be5p]

an we get a link to the repo that was used for the demo?

[@PrebenOlsen90]

Is there a way to extract variables set in the bundle file, in the notebooks? So that we can set 'environment' = 'prod' and do a spark.conf.get("bundle.variables.environment") type?

[@lostfrequency89]

For notebooks should we use even integrate github or we can use dabs for that matter ? I’m kinda confused

[@JulPinkie-ns5rs]

cheers

[@namanbhayani1016]

So now what will happen to DBX?
This looks like a replacement for it.

[@villetakoo]

DBX is not officially supported by Databricks. It is a Labs project. If Asset Bundles will cover all use cases DBX has been used for, I predict developers will migrate to use AB and the contributions to DBX will fade in time and it might become depracated.