The new dagster-dbt integration: an overview of the new capabilities

[@dickduran]

it is possible to integrate spark/pyspark into this?

[@CuongNguyen-gu9fl]

Yes, you can integrate pretty much anything by defining a resource manager.

[@dickduran]

@@CuongNguyen-gu9fl Thanks Im already did, I made my own Pyspark IO manager to use in postgres

[@CuongNguyen-gu9fl]

@@dickduran Nice, so you transform the data on top of Postgres?
Do you have the code public so I can take a look at it?

[@dickduran]

@guyen-gu9fl not in a public repo, but I can invite you to, its a dagster-pyspark-airbyte-dbt-postgres-duckdb stack integration.

[@CuongNguyen-gu9fl]

@@dickduran Thank you.
My github is ｄｅｄｙｏｃ
(special font cuz my comment keeps getting deleted)