Nice explanation. I do have a doubts what's the need for broadcast if the filter condition is already being applied to dimensions table and if it is required than tables which are greater than broadcast threshold in those case will this technique not work or just join type will be different.
from what I understand, filter condition, or predicate pushdown as Databricks calls it, works only when querying single table. When you join two tables, you need to 'broadcast' the filter to the other table being joined.
Nicely explained !....but how does the filter is transferred to order table..since where condition is on year and month , query on fact table would still have to figure out what is full_date (of dimension table)column values on for 2021 Feb...could be multiple full_dates for month,year .?
Genious explanation. Not only on that certain topic, but all of the author's videos and articles are also totally clear. Thanks a lot. Subscribed!
Best Explanation I have found till date about Dynamic Partition Pruning and infact about Predicate Pushdown and Partition Pruning as well
super explanation! simple to understand, thanks for showing the execution plans!
Amazing explanation!!!
Very useful and expiation is pretty clear, thank you!
Very informative thanks for deep diving.
What if I just add year and month to the 'ON' statement ? I believe partition pruning will work, but AQE should be enabled.
Nice explanation.
I do have a doubts what's the need for broadcast if the filter condition is already being applied to dimensions table and if it is required than tables which are greater than broadcast threshold in those case will this technique not work or just join type will be different.
Same doubt brother. If u get the answer can u please share it.
from what I understand, filter condition, or predicate pushdown as Databricks calls it, works only when querying single table. When you join two tables, you need to 'broadcast' the filter to the other table being joined.
Hi sir., can we use apache kafka for sending emails? Please sir, I need it help
Neat feature
Him: Make Sense?
Me: Nope
Him: Great
hahah, feel exactly the same
Nicely explained !....but how does the filter is transferred to order table..since where condition is on year and month , query on fact table would still have to figure out what is full_date (of dimension table)column values on for 2021 Feb...could be multiple full_dates for month,year .?
probably it would do date like '2021-02%'
Hi Prashant, you know your concepts. but stop saying : makes sense.. or simple etc.. that is too uncle like.. b cool