Hey.. Thanks for putting this. Lot of action. is this committed on github? This will be a learning project for people who are starters in airflow. Additionally I have one question : Why have we hardcoded customer in get_dag_conf, should that not be dynamic basis your list of 3 tables? Also in the tutorial, I think you didnt show customer.sql which you are using in extract task, and modifying to add where clause. Can you show how it looks like?
It sure is! github.com/manmeetkaur/etl-airflow/tree/main Can see the customer.sql file there, and in practice you wouldn't want to hardcode, I just did that for ease of use for this example
Why snowflake for OLAP? Postgres, airflow, dbt are all open source (and cosmos). But snowflake isn't. I don't get why everyone uses it. Why not an open source alternative? What about Clickhouse?
![[DRAGON BALL LEGENDS REVEALS ＆ STUFF] LEGENDS FESTIVAL 2024 Special Edition Part 1](http://i.ytimg.com/vi/mV5bGIswTtQ/mqdefault.jpg)
Thanks George and Manmeet for this. Very well explained 🙌
Thank you, will pass the message along to Manmeet!
Niccce! BIG thumbs up for some EndToEnd-projects!
Thanks Rasmus, this one was for you!
@@thedataguygeorge really appreciate it!!
Hey.. Thanks for putting this. Lot of action. is this committed on github? This will be a learning project for people who are starters in airflow.
Additionally I have one question : Why have we hardcoded customer in get_dag_conf, should that not be dynamic basis your list of 3 tables?
Also in the tutorial, I think you didnt show customer.sql which you are using in extract task, and modifying to add where clause. Can you show how it looks like?
It sure is! github.com/manmeetkaur/etl-airflow/tree/main
Can see the customer.sql file there, and in practice you wouldn't want to hardcode, I just did that for ease of use for this example
@@thedataguygeorge Thanks for replying. Very kind of you
Why snowflake for OLAP? Postgres, airflow, dbt are all open source (and cosmos). But snowflake isn't. I don't get why everyone uses it. Why not an open source alternative? What about Clickhouse?
Snowflake requires almost 0 set up or management expertise which is a big value add for a lot of people