Focus On: The Apache Airflow PythonOperator, all you need in 20 mins!

Focus On: The Apache Airflow PythonOperator, all you need in 20 mins!
In this video you are going to discover everything about the PythonOperator in Airflow 2.0
At the end of the video you will be able to:
⏺ Execute a Python function (Amazing isn't it?)
⏺ Pass positional arguments to your Python function
⏺ Pass keyword arguments to your Python function
⏺ Evaluate your arguments at runtime and pass variables
⏺ Reduce the number of calls to your database
⏺ Make your DAG cleaner
⏺ Get the current execution date in Airflow 2.0
⏺ Leverage the TaskFlow API to transform your PythonOperator in a mind-blowing way!
Materials:
www.notion.so/The-PythonOpera...
The NEW WAY OF CREATING DAGS:
• TaskFlow API in Airflo...
Enjoy ❤️

[@goodmanshawnhuang]

Great job, thanks for sharing it.

[@chris0628]

To the point, no fluff 👍🏽

[@steliosioannides7128]

Thanks Marc for this very useful video! High appreciated ! Keep it up !!

[@Leonardo-jv1ls]

Thank you. You've helped me a lot.

[@andrikramer686]

Awesome. Thanks a lot, Marc.

[@amitjain000]

Awesome.. thanks for such a beautiful knowledge share.

[@averychen4633]

you are the best

[@JosePla]

This is exactly what I was looking for

[@AbhishekAmeria]

That's was very useful. Thanks :)

[@vladimirobellini6128]

this is great. thanks!

[@FranVarVar]

Amazing. Thanks Marc!

[@Astronomer]

My pleasure!

[@shivangitomar5557]

Very good vidoe!!!

[@manjunathmani3678]

nice man

[@SpiritOfIndiaaa]

Excellent thanks a lot...I have to check oracle table if a record exists for current date ,if exists then only call downdown stream task ,how to achieve this ?

[@sridhar43299]

How to send object from gcs to email bu airflow. If possible send me code

[@bullandrooster]

Can I run 2 dags with different python version and different pandas version? For now I see airflow offers only one python version at the time, and only predefined required dependencies.
And all my dags must match that specs.

[@GeandersonLenz]

How to turn ON this module includes?

[@leomax87]

did you resolve it?

[@brendoaraujo9110]

instead of calling the script function, is there an operator that I can call the script completely to be executed?

[@eunheechoi3745]

whenever I first run the airflow test command, it throws an error saying 'sqlalchemy.exc.NoReferenceTableError: Foreign key associated with column 'dag_run_note.user_id' could not find table 'ab_user' with which to generate a foreign key to target column 'id'
do you what it is about? and how to fix it?
Second run it doesn't occur....

[@Astronomer]

Hmmmmm interesting, if it works on the second run maybe it needs to generate the table first?

[@BalvanshHeerekar]

Hey, while using includes I get the error - ModuleNotFoundError: No module named 'includes'

[@leomax87]

I'm facing the same error. How Can I resolve it?

[@eldrigeampong8573]

@@leomax87 you can import the sys module in python after which you will navigate a number of levels either up or down into the directory your file is located.
import sys
sys.path.append(".") # that's move a level up into the dags folder
from includes.my_dags.functions import process

[@valetta6789]

I am trying to pull the value from another task in a different folder, however, I see only the current task instance. Is this new behaviour, or am I unable to pass between different directories?

[@Astronomer]

So your DAG's are in different folders?

[@valetta6789]

@@Astronomer I have no idea what I was talking about😄

[@archanam4224]

import airflow cannot be resolved