Real-World Data Movement and Orchestration Patterns using Azure Data Factory V2

[@diegoctn]

Hi, where can I find the samples templates?

[@HybridVirtualGroup]

Hope this is what you are looking for Diego docs.microsoft.com/en-us/azure/data-factory/solution-templates-introduction

[@srikantha7290]

Sir , i need training for Azure Data Engineer please suggest 🙏

[@MrMailKamath]

Merge and Incremental load explanation is bit confusing. Would you please bit elaborate on it

[@BigBossInd7236]

Merge approach is used if you don't have a delta identifier in the source table e.g. created and updated dates. In that situation, form a hash-key using all non-key columns in a source table. Then using the primary key to match source and target table row and comparing hash-key to decide whether source record is changed or not. If primary key matches and hash-key doesn't then either insert a new updated record or update existing target record. If the primary key doesn't match then insert in target. There is a situation when the primary key doesn't match, either source record doesn't exist in target or it exists in the target but got deleted from the source. You need to handle this accordingly.
Incremental is simple, it means you have a delta identifier in the source. Initially you will load all the records from the source into the target table and save maximum of either the created or updated date in the control table for each source table. So, next time you load data, pick the records after that maximum date you have in the control table and keep on repeating it.
Hope you have understood it 🙂

[@shrutichikode8059]

@@BigBossInd7236 great explanation, thank you :)