Hi Marc, I love the way you make Airflow interesting to jump at for a beginner like myself. Am completely new to Airflow world, as a matter of fact, your video on online retail is my first bite. But unfortunately, i got choked 😔 along the line 😭 tried to figure things out but couldn't. The task the error was coming from was in the task that upload gcs to bigqery, this task failed with error decodedencoded utf-8. But when i check BigQuery the raw_invoices table was created but no dataset 😢 wish i could fix this or find way around this. Could you as well be kindly make a video on dynamic task maping as mentioned in this video. Maybe from API to data warehouse. Thanks for the great and concise videos. 🎉
Hi Marc, great video yet again! You mention around 3:35 that you can have multiple replicas of the metadata DB, I thought this was not possible by design. How did you set that up?
Airflow needs a database to work and there is nothing that prevents you from making your database highly available. If you use postgres, you can look at the following doc: www.postgresql.org/docs/current/high-availability.html To be clear, it's not an Airflow thing, it's db thing and it depends on the database you use :)
Hi Marc, I took your course last year, and I don't remember any discussion around dynamic workflow and data lineage. Are these new concepts? Do you have any videos about these concepts? Thanks in advance :)
If you want start learning Airflow today: www.udemy.com/course/the-complete-hands-on-course-to-master-apache-airflow/
Hi Marc,
Need your help, am new for airflow how to schedule the job monthly 5th, if the 5th is holiday the job will run on 6th of every month
Hi Marc, I love the way you make Airflow interesting to jump at for a beginner like myself. Am completely new to Airflow world, as a matter of fact, your video on online retail is my first bite. But unfortunately, i got choked 😔 along the line 😭 tried to figure things out but couldn't. The task the error was coming from was in the task that upload gcs to bigqery, this task failed with error decodedencoded utf-8. But when i check BigQuery the raw_invoices table was created but no dataset 😢 wish i could fix this or find way around this. Could you as well be kindly make a video on dynamic task maping as mentioned in this video. Maybe from API to data warehouse. Thanks for the great and concise videos. 🎉
Hi Marc, great video yet again! You mention around 3:35 that you can have multiple replicas of the metadata DB, I thought this was not possible by design. How did you set that up?
Airflow needs a database to work and there is nothing that prevents you from making your database highly available. If you use postgres, you can look at the following doc: www.postgresql.org/docs/current/high-availability.html
To be clear, it's not an Airflow thing, it's db thing and it depends on the database you use :)
Hi Marc, I took your course last year, and I don't remember any discussion around dynamic workflow and data lineage. Are these new concepts? Do you have any videos about these concepts? Thanks in advance :)
Great question. I’m planning to update the courses this year. Stay tuned 🥹
@@MarcLamberti looking forward to it :)
to learn airflwo which all tools we haveto learn like python, sql?
Python essentially
what you think about prefect ?
Didn’t try yet