Awesome series! Thanks for all your hard work in making these! One question I have is that in your examples you tend to use multidimensional data structures (i.e. dim and fact tables). However power BI datasets (that I assume is going to be one of the main uses of the data) is built on the Tabular engine (vertipaq?). Do you see any limitations in using more tabular structures in the data lake, and then querying those in a tabular way within power BI data sets, rather than structuring the data in the data lake as multidimensional, and using Power Bi datasets to basically query like a multidimensional model? Personally since switching to tabular, I dread the thought of having to go back to multidimensional modelling.
Thank you for watching and commenting. Well in the Data Lake in the curated/serving layer, it's common to structure the data to allow for easy importing/querying in BI tools (Power BI etc) and that includes structuring as a dimensional model. Formats like Parquet and Delta work well with dimensionally modelled data (especially Delta).
Hi. Very nice video! I was wondering if you know of any possibility to update a lake db without publishing. Since I am mostly not working on main branch I can not publish all, that is why newly created tables don't appear in my sql querries.
Hi, thank you. That's a great question and I honestly don't know the answer, but I would say that if you're unable to publish then the tables won't appear.
Not directly no. However it’s possible to use COPY INTO in Dedicated SQL Pool to load data in from the same location that the Serverless SQL Pool external table is pointing to
This tute is amazing - thank you!
Awesome series! Thanks for all your hard work in making these!
One question I have is that in your examples you tend to use multidimensional data structures (i.e. dim and fact tables). However power BI datasets (that I assume is going to be one of the main uses of the data) is built on the Tabular engine (vertipaq?). Do you see any limitations in using more tabular structures in the data lake, and then querying those in a tabular way within power BI data sets, rather than structuring the data in the data lake as multidimensional, and using Power Bi datasets to basically query like a multidimensional model? Personally since switching to tabular, I dread the thought of having to go back to multidimensional modelling.
Thank you for watching and commenting. Well in the Data Lake in the curated/serving layer, it's common to structure the data to allow for easy importing/querying in BI tools (Power BI etc) and that includes structuring as a dimensional model. Formats like Parquet and Delta work well with dimensionally modelled data (especially Delta).
Hi. Very nice video!
I was wondering if you know of any possibility to update a lake db without publishing. Since I am mostly not working on main branch I can not publish all, that is why newly created tables don't appear in my sql querries.
Hi, thank you. That's a great question and I honestly don't know the answer, but I would say that if you're unable to publish then the tables won't appear.
is there anyway we can load serverless external table to dedicated pool internal table?
Not directly no. However it’s possible to use COPY INTO in Dedicated SQL Pool to load data in from the same location that the Serverless SQL Pool external table is pointing to