This was incredibly helpful! I am prepping to teach the DP-500 and one of the things I am trying to figure out is how to compare/contrast/talk about "traditional" Synapse vs. Fabric. Your insights really help!
Finally, I understood the whole buzz around Fabric. Main differentiators from Synapse look like SaaS | OneLake | Dropping MPP aka Dedicated Pool | Compute/storage decoupling (official) |...plue few more. I am not sure how well "Auto discovery and registration of table" feature will work - specially if metastore already exists in Databricks. Will Databricks share its metastore with Fabric or we will recreate metastore here? Also, will metastore be at workspace level or tenant level?
If you were to create a shortcut to a CSV file , right now it seems that you have to manually create the table, and it does not update if the source file changes. It's quite common that we have folders in the datalake containing versions of a csv. Sales_01 , Sales_02 etc. In serverless sql you could just target them with openrowset and an asterisk. Sales_* would just load all the files in the folder. And since it was a view you would always gaurantee the data was not stale. Are there plans to improve the way data is loaded as tables in the lakehouse to support this?
Does MS Fabric supports things like Private connections to on-prem, key vaulted credentials/configs like Azure does? Is it something we can expect to see in future if not there already?
Hi James, thanks for the good introduction! Good insights into the architecture of Fabric/OneLake. Will it be possible to use data virtualization as a layer between on-premise SQL Server 2022 and OneLake (maybe, in conjunction with the new feature Shortcuts). I remind myself, that SQL Server 2022 is able to use the Polybase v3 feature in conjunction with ADLSv2 access (virtually), but since OneLake is somehow extra/segregated from ADLSv2, I doubt that it will be possible, at the moment. A workaround for me would be to data virtualize between on-premise and ADLSv2, and then shortcut/"bridge over logically" to OneLake (hosted in Fabric capacity). Thanks in advance!
Hi Thomas, I expect you will eventually see Synapse link integrated into Fabric. Synapse Link supports data virtualization to SQL Server 2022. Your workaround will do in the meantime 🙂
Hi James, first of all, great video! Most of our current dashboards are built on Tableau which connects to our SQL database. If we moved to Fabric and used OneLake, would we still be able to connect to Tableau?
ruclips.net/video/a6A3jtvB62U/видео.html "Do not [...] use Pipelines within Synapse" I hope we are fine as long as we don't use (Synapse) Data Flow activities.
Hi Martin, I would also avoid using Synapse pipelines and use ADF pipelines instead. This is because there will be a migration tool for ADF pipelines to Fabric much sooner than a migration tool for Synapse pipelines to Fabric
I want to say a big thank you for incredibly useful and easy explanations. Well done.
Thanks for the kind comment!
Great Presentation , thank you
Nicely explained! Thanks
Nice talk to understand the high-level how all connected together and compare with existing offering.
This was incredibly helpful! I am prepping to teach the DP-500 and one of the things I am trying to figure out is how to compare/contrast/talk about "traditional" Synapse vs. Fabric. Your insights really help!
Finally, I understood the whole buzz around Fabric. Main differentiators from Synapse look like SaaS | OneLake | Dropping MPP aka Dedicated Pool | Compute/storage decoupling (official) |...plue few more. I am not sure how well "Auto discovery and registration of table" feature will work - specially if metastore already exists in Databricks. Will Databricks share its metastore with Fabric or we will recreate metastore here? Also, will metastore be at workspace level or tenant level?
Once again, Great and useful video!
Thank you James!
If you were to create a shortcut to a CSV file , right now it seems that you have to manually create the table, and it does not update if the source file changes. It's quite common that we have folders in the datalake containing versions of a csv. Sales_01 , Sales_02 etc. In serverless sql you could just target them with openrowset and an asterisk. Sales_* would just load all the files in the folder. And since it was a view you would always gaurantee the data was not stale. Are there plans to improve the way data is loaded as tables in the lakehouse to support this?
Does MS Fabric supports things like Private connections to on-prem, key vaulted credentials/configs like Azure does? Is it something we can expect to see in future if not there already?
How can I get access to the links on the presentation?
Hi James, thanks for the good introduction! Good insights into the architecture of Fabric/OneLake.
Will it be possible to use data virtualization as a layer between on-premise SQL Server 2022 and OneLake (maybe, in conjunction with the new feature Shortcuts). I remind myself, that SQL Server 2022 is able to use the Polybase v3 feature in conjunction with ADLSv2 access (virtually), but since OneLake is somehow extra/segregated from ADLSv2, I doubt that it will be possible, at the moment.
A workaround for me would be to data virtualize between on-premise and ADLSv2, and then shortcut/"bridge over logically" to OneLake (hosted in Fabric capacity).
Thanks in advance!
Hi Thomas, I expect you will eventually see Synapse link integrated into Fabric. Synapse Link supports data virtualization to SQL Server 2022. Your workaround will do in the meantime 🙂
Hi James, first of all, great video!
Most of our current dashboards are built on Tableau which connects to our SQL database. If we moved to Fabric and used OneLake, would we still be able to connect to Tableau?
Yes, Tableau can pull in data from OneLake, since the data in OneLake is stored in delta format, which Tableau can read
“things i like behind me” for some reason i thought to myself he likes printers 😂 … jk this is a great video Thank you!
ruclips.net/video/a6A3jtvB62U/видео.html "Do not [...] use Pipelines within Synapse"
I hope we are fine as long as we don't use (Synapse) Data Flow activities.
Hi Martin, I would also avoid using Synapse pipelines and use ADF pipelines instead. This is because there will be a migration tool for ADF pipelines to Fabric much sooner than a migration tool for Synapse pipelines to Fabric