SSIS Series: Incremental Load using Lookup Task (Logic + ETL)

In this video we will see how to incremental/delta data load using SSIS.
This is going to be a detailed walk through providing all the necessary steps for you to DIY.
Also note that this will be a continuation from the previous video:
• Introduction to SSIS D...
Install SSIS: Visual Studio 2017/2019
docs.microsoft...

[@quinnhogan545]

hahaha the windows sound at 5:52 scared me!! well explained video though thank you

[@rahulgautam511]

Great. Nice & simple explanation..!

[@dataexplained7305]

Thank you...

[@rahulgautam511]

Please upload video on CDC.

[@dataexplained7305]

Cmg soon!!

[@apbreports]

It doesn't seem too efficient to keep updating all rows... In source table you should have a column: RevisionDate and set a trigger on this to auto update. Then filter on this column.

and if you dont have any column modified date, you can compare all columns with except command of sql ? do you have any better idea for me :) ?

[@user-mu1tg5wv4v]

Please put a video on : How do we add parameter for the ADO source, like using variables to filter the records. select * from table1 where date > ? I am connecting to a DB2 source and retrieving data, it already got 20million records on it, i just what to upload it incrementally on last modified date. need to query the DB2 using the date modified.

[@reddyabhik]

super well explained

[@dataexplained7305]

Thank you..

[@vg7181]

Love it!

[@dataexplained7305]

Thanks a lot..

[@malihehgordoghli3186]

Thank you for this informative video. Can you also explain how we could create Fact Table by extracting data from DB after generating this dim packages?

[@dataexplained7305]

Thanx..Fact table generation is the time consuming one.. will do one soon along with different deployment ways..

[@sharavananp5570]

Error: OLE DB error occurred while populating internal cache. Check SQLCommand and SqlCommandParam properties.". I get this when i execute, should i execute DROPCLEANBUFFERS and if so will this create any data loss.

[@dataexplained7305]

Hope you have figured it out by now. I would need to chat in detail on this or you can do delete insert rather than update.. let me.know if you need a video on this.. I can upload one..
Also consider cleaning cache like yours aid if it's your local host.. if its upper environments avoid that, as those may be from other parallel processes

[@remyarajan2103]

but the update will keep updating all the records rite whether there is any modification or not ?

[@alexrubin5955]

What program did you use to draw the flow chart?

[@thinhtruong6683]

How to if i use ODBC Destination pls?

[@dataexplained7305]

Create odbc connection manager use that in destination. Make sure the ETL deployment server has the ODBC for this destination as well

[@roshnib9360]

@@dataexplained7305 but u used ole db command for updating, how to use that in case odbc destination?