Databricks Delta Lake Change Data feed - Real World Use case

This video is a real world demonstration of how we can implement databricks change data feed to optimize our ELT pipeline in Delta Lake house.
You can use this for practicing your data engineering skills.
I go over full load, incremental load, Merge Upsert, Change Data capture, ETL logic.
I hope this really helps someone gain experience as a Data engineer.
Find the notebook and source files here
github.com/nobieyi00/Cdcproject
Connect with me on Linkedin: / nobieyisi
connect on medium: / afroinfotech
#data #chatgpt #spark #dataengineering #dataengineer #datascience #sql
#data #dataengineering #etl #elt #databricks

[@Bijuthtt]

Good job Obi 👍

[@juanzambrano688]

Great video brother

[@user-lr3sm3xj8f]

Great tutorial, I would like to see autoloader as the source -> raw and then go through the medallion layers

[@Zafary465]

Hello Did you found how to implement it? Any ideas?
Thanks

[@neek6327]

Thanks for the vid. So you have to read the entire refined table in order to get the last commit version? Is there any more efficient way to get around this?