Thanks for the video! Is it straightforward to implement a Star Schema with SCD Type 2 in Snowflake using dynamic tables? I'm having difficulty understanding how the keys and IDs are linked between tables (dim and fact), given that dynamic tables are created from queries on the fly. Thanks in advance!
I could not find anywhere in documentation, but is there any optimisation in create-replace of a dynamic table. Does it have any way to know that may be some partial data is already there. For example when we introduce a new column, it is highly likely that no other data was changed but does dynamic table consider this when it refreshes on create-replace.
Thanks for your question! Create or replace for Dynamic Tables works just like a create or replace for any ordinary table, so the dynamic table would indeed get rebuilt.
Hi, How can we write sql unit tests for a whole dynamic table pipeline? That is, given a base table data, run it through the whole pipeline thru the test and verify if the results are as expected
Hi, i create a dynamic table and have task that validates data on tables (standard tables) that are used in the dynamic table. The thing is when i finish the validations i need, i put an ALTER DYNAMIC TABLE IF EXISTS xxx REFRESH; inside the task, and when the task is executed the refresh is not starting. If i execute manually the command it works but is like it doesn't when is inside the task, is this a limitation right now??
dynamic tables can be monitor through snowsight->activity->dynamic table I need to know which snowflake account usage table it is using to display dynamic tables details in snowsight? I need to get all the refresh history of all the tables, so how to get this details?
Check this page out on monitoring dynamic table refresh history: docs.snowflake.com/en/user-guide/dynamic-tables-tasks-monitor#label-dynamic-tables-monitoring-sql
DT are very useful to simplify pipelines and object management, support, monitoring, documentation and support.
Totally agree!
Thanks for the video! Is it straightforward to implement a Star Schema with SCD Type 2 in Snowflake using dynamic tables? I'm having difficulty understanding how the keys and IDs are linked between tables (dim and fact), given that dynamic tables are created from queries on the fly. Thanks in advance!
When would I use a materialized view vs a dynamic table?
Can I create the DT on top of external table ! looks it won't allow, is there any work around if in case I need to use external table ?
I could not find anywhere in documentation, but is there any optimisation in create-replace of a dynamic table. Does it have any way to know that may be some partial data is already there.
For example when we introduce a new column, it is highly likely that no other data was changed but does dynamic table consider this when it refreshes on create-replace.
Thanks for your question! Create or replace for Dynamic Tables works just like a create or replace for any ordinary table, so the dynamic table would indeed get rebuilt.
Hi,
How can we write sql unit tests for a whole dynamic table pipeline?
That is, given a base table data, run it through the whole pipeline thru the test and verify if the results are as expected
we can use non dynamic tables in the dynamic sql?
Yes
Hi, can we write dynamic table definition using python instead of SQL?
Hi, i create a dynamic table and have task that validates data on tables (standard tables) that are used in the dynamic table. The thing is when i finish the validations i need, i put an ALTER DYNAMIC TABLE IF EXISTS xxx REFRESH; inside the task, and when the task is executed the refresh is not starting. If i execute manually the command it works but is like it doesn't when is inside the task, is this a limitation right now??
Pretty awesome
Glad you enjoyed!
where you define call start_data_gen(); and call send_alert_email(); function ?
THE SAME ISSUE!!!
dynamic tables can be monitor through snowsight->activity->dynamic table
I need to know which snowflake account usage table it is using to display dynamic tables details in snowsight?
I need to get all the refresh history of all the tables, so how to get this details?
Check this page out on monitoring dynamic table refresh history: docs.snowflake.com/en/user-guide/dynamic-tables-tasks-monitor#label-dynamic-tables-monitoring-sql
Also, table_type = base table, is_dynamic= true is how a Dynamic Table shows up in account usage
The demo example is pretty complex and the presentor should have chosen a simple example to explain things.
Thank you for your feedback!
where you define call start_data_gen(); and call send_alert_email(); function ?