This framework looks fantastic! How are you handling the unknown source metadata (Consider if this were the FIRST run you're doing) mapping tab/component in the parameterized Copy activities that you have at the end of your loop component?
Can we build metadata driven dataflows in Fabric.? for example: In the synapse Dataflows we have a Schema Drift/Map Drift in Derived activity and Auto column map on the Sink, with these features we can make the dataflow as Metadata driven, do we have the same in Fabric.?
Any ETA to have parameterise connections in Fabric ? (like you could with old ADF linked services) Without that ability, a metadata driven pipeline in Fabric becomes useless in real world …. Otherwise you'd need a separate connection for EVERY database (even if they're all SQL server for example) which is totally daft, thus defeating the point of using a meta data table driven pipeline like shown in this video 😢
We are implementing this in our Company at the moment. Seems to be working ok.
Will Azure SQL ever be part of Fabric? 🤔 And is there any explicit reason was a Fabric DWH was not used as the control DB in the demo?
This framework looks fantastic! How are you handling the unknown source metadata (Consider if this were the FIRST run you're doing) mapping tab/component in the parameterized Copy activities that you have at the end of your loop component?
We are using Dataverse, so we can use a Model Driven App Application to maintain the Metadata
Can we build metadata driven dataflows in Fabric.?
for example: In the synapse Dataflows we have a Schema Drift/Map Drift in Derived activity and Auto column map on the Sink, with these features we can make the dataflow as Metadata driven, do we have the same in Fabric.?
Any ETA to have parameterise connections in Fabric ? (like you could with old ADF linked services) Without that ability, a metadata driven pipeline in Fabric becomes useless in real world …. Otherwise you'd need a separate connection for EVERY database (even if they're all SQL server for example) which is totally daft, thus defeating the point of using a meta data table driven pipeline like shown in this video 😢
Yes, parameterized connection support is in the works!
And to think..... all this time I thought you were just revelling in 2022 and 2023 Phillies baseball for the last few years.....
Phillies, Sixers, Eagles ... It's a weird feeling when they're all winning!