Hello I tried this solution but I am facing some issue when I export the schema it worked fine and schema got exported but when I try to export the data I am facing the below error. Could not create container, This request is not authorized to perform this operation.
Unfortunately this is what Synapse has amounted to as a solution for the dynamics products. What was designed to be a better performing , easy to use architecture has significantly increased the complexity. Ultimately an ADF pipeline connecting to BC dataverse and pulling the data into an Azure serverless V-core SQL database (or DTU model) with stored procedures to carry out append ( for transactions) and merge ( for master tables) is a far easier to implement solution. Ultimately it will also result in a cheaper solution. A dedicated synapse instance represents a considerable increase on an Azure SQL V core model and a synapse serverless solution performs very poorly when querying CSV files. I would have expected an export to datalake solution to provide an incremental export based on change tracking ( as is the case for CE and finance) as opposed to schedule and export. I am also surprised that there is a need to create pipelines to carry out the merge process for the delta files ( I would export the export process to manage this for me). In short - this is a far more convoluted solution that brings significantly more complexity without any benefits in terms of reduced latency or costs. You now have additional resources ( azure data lake, synapse (including synapse studio) and mapping dataflows) combined with either a dedicated instance or a serverless database. All of these resources comes with their own cost, security , access and network considerations. In additional if this is being implemented for a customer - the customers will need the skills with all of these products in order to manage the solution going forward.
Can you tell me more on how to connect ADF to BC dataverse? I've got a customer who's BC partner is totally unwilling to install any form of Extensions. So we are only left with the BC Odata webservice or Rest API. We are experienced in ADF, but not in pulling in data from BC without an Extension providing a table list.
Hello
I tried this solution but I am facing some issue when I export the schema it worked fine and schema got exported but when I try to export the data I am facing the below error.
Could not create container, This request is not authorized to perform this operation.
Unfortunately this is what Synapse has amounted to as a solution for the dynamics products. What was designed to be a better performing , easy to use architecture has significantly increased the complexity. Ultimately an ADF pipeline connecting to BC dataverse and pulling the data into an Azure serverless V-core SQL database (or DTU model) with stored procedures to carry out append ( for transactions) and merge ( for master tables) is a far easier to implement solution. Ultimately it will also result in a cheaper solution. A dedicated synapse instance represents a considerable increase on an Azure SQL V core model and a synapse serverless solution performs very poorly when querying CSV files. I would have expected an export to datalake solution to provide an incremental export based on change tracking ( as is the case for CE and finance) as opposed to schedule and export. I am also surprised that there is a need to create pipelines to carry out the merge process for the delta files ( I would export the export process to manage this for me).
In short - this is a far more convoluted solution that brings significantly more complexity without any benefits in terms of reduced latency or costs. You now have additional resources ( azure data lake, synapse (including synapse studio) and mapping dataflows) combined with either a dedicated instance or a serverless database. All of these resources comes with their own cost, security , access and network considerations. In additional if this is being implemented for a customer - the customers will need the skills with all of these products in order to manage the solution going forward.
Can you tell me more on how to connect ADF to BC dataverse? I've got a customer who's BC partner is totally unwilling to install any form of Extensions. So we are only left with the BC Odata webservice or Rest API. We are experienced in ADF, but not in pulling in data from BC without an Extension providing a table list.