- Видео 126
- Просмотров 171 866
Rajesh Jakhotia
Индия
Добавлен 10 фев 2012
Hello, Welcome to your own Channel on Data Science, Machine Learning, and Artificial Intelligence.
I am Rajesh Jakhotia and I have 22+ years of Business Intelligence and Data Analytics experience. In this channel, I have uploaded some of my best-curated videos on Python Programming, R Programming, Data Engineering, ETL, Statistics, Data Science, Predictive Analytics, Machine Learning, and Artificial Intelligence.
My objective is to make quality content available free to all.
How can you help me in my endeavor?
If you like my training content, then, I request you to share my video link with others who may benefit from them.
Secondly, please subscribe, like, and comment only if you found my videos helpful to you.
I am reachable at ar.jakhotia@k2analytics.co.in and +91 8939694874
I am Rajesh Jakhotia and I have 22+ years of Business Intelligence and Data Analytics experience. In this channel, I have uploaded some of my best-curated videos on Python Programming, R Programming, Data Engineering, ETL, Statistics, Data Science, Predictive Analytics, Machine Learning, and Artificial Intelligence.
My objective is to make quality content available free to all.
How can you help me in my endeavor?
If you like my training content, then, I request you to share my video link with others who may benefit from them.
Secondly, please subscribe, like, and comment only if you found my videos helpful to you.
I am reachable at ar.jakhotia@k2analytics.co.in and +91 8939694874
Airflow Sensors | FileSensor | How to use Sensors, Poke, Reschedule and more
In this video on Airflow FileSensor we will learn to use FileSensor for sensing file having some naming convention like DATA_YYYYYMMDD.csv. We will use File Connection, Task Instance, XCom Push Pull concepts.
Apache Airflow Sensors are a special type of Operator that are designed to do exactly one thing - wait for something to occur. It can be time-based, or waiting for a file, or an external event, but all they do is wait until something happens, and then succeed so their downstream tasks can run.
Airflow is a platform to programmatically author, schedule, and monitor workflows.
Use Airflow to author workflows as Directed Acyclic Graphs (DAGs) of tasks. The Airflow scheduler executes your t...
Apache Airflow Sensors are a special type of Operator that are designed to do exactly one thing - wait for something to occur. It can be time-based, or waiting for a file, or an external event, but all they do is wait until something happens, and then succeed so their downstream tasks can run.
Airflow is a platform to programmatically author, schedule, and monitor workflows.
Use Airflow to author workflows as Directed Acyclic Graphs (DAGs) of tasks. The Airflow scheduler executes your t...
Просмотров: 66
Видео
Airflow Sensors | FileSensor | How to use Sensors, Poke, Reschedule and more
Просмотров 78Месяц назад
Apache Airflow Sensors are a special type of Operator that are designed to do exactly one thing - wait for something to occur. It can be time-based, or waiting for a file, or an external event, but all they do is wait until something happens, and then succeed so their downstream tasks can run. Airflow is a platform to programmatically author, schedule, and monitor workflows. Use Airflow to auth...
schedule=@continuous to run the DAG, Data Pipeline Continuously |max_active_runs=1 k2analytics.co.in
Просмотров 75Месяц назад
Scheduling a DAG to Run Continuously in AIRFLOW Application can be to process new incoming data like transaction data on a continuous basis. Airflow is a platform to programmatically author, schedule, and monitor workflows. Use Airflow to author workflows as Directed Acyclic Graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified de...
Getting Started With Our First DAG in Airflow
Просмотров 387Месяц назад
For Airflow 2.0 and above version, replace DummyOperator by EmptyOperator Connect with us on Whatsapp : 91 8939694874 Website Blog: k2analytics.co.in/blog Write to me at : ar.jakhotia@k2analytics.co.in Airflow is a platform to programmatically author, schedule, and monitor workflows. Use Airflow to author workflows as Directed Acyclic Graphs (DAGs) of tasks. The Airflow scheduler executes your ...
Bias Variance trade-off concept in Supervised Machine Learning Model | Overfitting & Underfitting
Просмотров 685 месяцев назад
This video explains the concept of Bias and Variance Error Trade-Off. The difference in the Actual Value and the Predicted Value is called Error. The error has three components - Bias, Variance, and Noise. Noise cannot be predicted or eliminated as such, we cannot do anything about it. However, we can reduce the Variance and Bias error. A model is susceptible to small changes in the training da...
Random Forest using R
Просмотров 418 месяцев назад
In this PlayList on Random Forest, we will learn the concepts step-by-step. The Table of Contents of the entire Random Forest PlayList is given below: What is Ensemble Modeling? What is Bagging? Random Forest Algorithm Out of Bag Error Rate Tuning the Random Forest Model Finding Optimal Number of Trees Finding Optimal Number of Variables to Select Variable Importance Subscribe to our RUclips Ch...
Python Programming for Data Science | Modules and Packages | import modules - math, os, sys
Просмотров 199Год назад
This video covers the concepts related to the Python Module and Package. It explains the various syntaxes to import the packages in Python: import module_name import module_name as alias_name from module import object_name from module import object_name as alias_name Python Programming is the most in-demand programming language and Data Scientist is the most sought-after resource in the job mar...
Python Programming for Data Science | Data Structures - List, Tuple, Dictionary, Set, Mutable Object
Просмотров 127Год назад
This video covers the 4 Basic Data Structures available in the builtins module of Python. 1) List 2) Tuple 3) Dictionary 4) Set We will also understand the concept of mutable and immutable Objects and why List, Dictionary, and Set are Mutable Objects whereas a Tuple is an Immutable Object in Python In the previous video, we learned about the basic Data Types 1) Numeric 2) String 3) Date 4) Bool...
Python Programming for Data Science | Basic Data Types - Numbers, String, Date, and Boolean
Просмотров 137Год назад
This is an introductory video on the 4 Basic Data Types available in the builtins module of Python. 1) Numeric 2) String 3) Date 4) Boolean. This Video will help you to get started with Jupyter Noteboom and learn how to effectively use Anaconda Jupyter Notebook for Python Programming. Python Programming is the most in-demand programming language and Data Scientist is the most sought-after resou...
Retail Banking Analytics | Basics of Banking | Types of Banks - Retail, Commercial, Cooperative Bank
Просмотров 271Год назад
Retail Banking Analytics | Basics of Banking | Types of Banks - Retail, Commercial, Cooperative Bank
Retail Banking Analytics | Finance Concepts - Future Value, Net Present Value, IRR, Discount Rate
Просмотров 262Год назад
Retail Banking Analytics | Finance Concepts - Future Value, Net Present Value, IRR, Discount Rate
Airflow TriggerDagRunOperator | Configure DAG dependencies at ease | ETL Pipelines | Master DAG
Просмотров 2,3 тыс.Год назад
Airflow is a platform to programmatically author, schedule, and monitor workflows. The Airflow TriggerDagRunOperator is used to configure the DAG Dependencies. What does it mean? Assume you have DAGs for Branch Master, Customer Master, Account Master, and Transaction data. Each DAG has a dependency on the previous dag, meaning that the Customer Master DAG can't run until the data has been extra...
Object Oriented Programming Concepts in Python - Class, Inheritance, Polymorphism, Encapsulation
Просмотров 193Год назад
Object Oriented Programming concepts in Python. This Python Programming video series will give provide a detailed explanation of the following concepts - Object Orient Programming, Class, Object i.e. Instance of Class, Inheritance, Polymorphism - Method Overload and Method Overriding, Encapsulation - Public, Private, Protect access modifiers Other Must-Watch Videos for Python Beginners: Python ...
Getting Started with Pytest | Understanding the usage of Assert Keyword | Practical pytest example
Просмотров 127Год назад
Getting Started with Pytest | Understanding the usage of Assert Keyword | Practical pytest example
Introduction to Automated Testing using Pytest in Python | Unit Testing | Applications of pytest
Просмотров 163Год назад
Introduction to Automated Testing using Pytest in Python | Unit Testing | Applications of pytest
Test your Math Operators & Logic Development Skills using Python | Floor Division, Modulus
Просмотров 48Год назад
Test your Math Operators & Logic Development Skills using Python | Floor Division, Modulus
Learn Python Q&A Style | Test your Conditional Statement Knowledge | if-elif-else & common mistakes
Просмотров 54Год назад
Learn Python Q&A Style | Test your Conditional Statement Knowledge | if-elif-else & common mistakes
Learn Python Q&A Style | For Loop | input, int, loops
Просмотров 37Год назад
Learn Python Q&A Style | For Loop | input, int, loops
Learn Python Q&A Style | FAQs List Manipulation | append, sort, index, reverse, insert, pop, len
Просмотров 33Год назад
Learn Python Q&A Style | FAQs List Manipulation | append, sort, index, reverse, insert, pop, len
Learn Python Q&A Style | FAQs on String Manipulation | find, partition, index, split, f string
Просмотров 74Год назад
Learn Python Q&A Style | FAQs on String Manipulation | find, partition, index, split, f string
Missing Value Treatment in Python | K Nearest Neighbours - KNN Technique | k2analytics.co.in
Просмотров 93Год назад
Missing Value Treatment in Python | K Nearest Neighbours - KNN Technique | k2analytics.co.in
Apache Airflow | Trigger DAG with Config Parameters | get_current_context() | **kwargs | k2analytics
Просмотров 16 тыс.2 года назад
Apache Airflow | Trigger DAG with Config Parameters | get_current_context() | kwargs | k2analytics
SQL Window (Analytical) Functions Part 2 | lag, lead, preceding, following | k2analytics.co.in
Просмотров 1382 года назад
SQL Window (Analytical) Functions Part 2 | lag, lead, preceding, following | k2analytics.co.in
SQL Window Functions P1 | Analytical Func | Rank, Dense Rank, Row Number, Partition, Over, Order By
Просмотров 1922 года назад
SQL Window Functions P1 | Analytical Func | Rank, Dense Rank, Row Number, Partition, Over, Order By
SQL Functions | String, Data, Numeric Functions | Coalesce, Convert_TZ, Soundex, Year, Month, Cast
Просмотров 2272 года назад
SQL Functions | String, Data, Numeric Functions | Coalesce, Convert_TZ, Soundex, Year, Month, Cast
SQL Joins - Inner, Left, Right, Full Outer, Semi, Anti, Cartesian | Cardinalities: One-to-One, etc
Просмотров 2472 года назад
SQL Joins - Inner, Left, Right, Full Outer, Semi, Anti, Cartesian | Cardinalities: One-to-One, etc
very useful. Thank you so much. Looking forward to watch other videos too.
Very good explanation....thank you
Should the time period of data be at the time of application or time period of data can be before loan default (specific case for approval of loan) because for new application we would not be having variables disbursed Amy and ltv etc
I have clearly got the idea of sensors from your short video but its different from what i expected.You are triggering dag manually and the dag listens for new file to appear and whenever its available it continues to downstream tasks and then stops but what i want is to continue this process that is whenver a new file generated on a file folder i want the dag to be triggered automatically can i do that in airflow
@@harigovindi8217 You will get the answer to your query in my Part 2 video on FileSensor. It will be uploaded soon. Thanks
@@RajeshJakhotiaAIML thank you i will be waiting for that
Check out this video - ruclips.net/video/MXVrbMA80do/видео.html
Very informative. Please upload the next video asap.
Other videos are uploaded in the Play List
This is pure gold content
could you share the code ?
Great detail! Very nicely done
Hi,thank you for the video Let's say the extract function is failed , then it would cause the entire etl task to fail right, is this the better approach?
Yes
Character capacity condition capital collateral
This is the first time I have seen such a detailed video on hypothesis testing and how to understand data to perform variable transformations meaningfully.
this is not complete playlist
Thank you for the simple explanation on PDO. I additionally explained the intuition about the factor here - ruclips.net/video/fYuhigdRK90/видео.htmlsi=XNZZnAzeaFa8elPU in case any of the viewers are interested
Very helpful video. Thank you.
Thank you, sir. Keep the great work up!
doing god's work
Thank you sir!
can u please arrange the videos in playlist , it’s not in sequence, it’s bery hard for us otherwise to follow , it would be helpful if u can make the video in sequence
Kindly share the code please.
Thank you 🎉❤
boring , you dont have teaching skills teaching is skill not everyone can have, you may be good at etl but your teaching skills are fusssss in minus ,boring anyone can sleep while watching this video
Sir, I have a doubt. ( At 31:00 ) [ Correct me if I am wrong ] alpha = 0.001 and p-value = 0.000 The null hypothesis mentioned was higher the LTV higher the risk of default. and as the P value is less than alpha we should reject the null hypothesis This means LTV and Risk of default are inversely related. This should also be the case practically since the loan value is less and the value is high => lower LTV => lesser the risk. but you have accepted that and made a positively correlated graph and Idk why the coefficient is also positive. Could you please consider my doubts? Regards
@@mohitkhushlani6733 The p-value is less than alpha means we may reject the Null Hypothesis and Accept the Alternate Hypothesis. It does not mean the relation is inverse or direct.
Sir, u explain very well. Sir, way in which you have explained, no one can explain, u are awesome. Fantastic
Clicking that button just start running, it does not give me the option to trigger dag with config, perhaps it changed from when you did this but it seems like this tutorial is no longer useful. :(
Hi sir , what is the technological infrastructure used to host this ml model in a company . for ex - in HDFC , axis banks etc
@@abh1shek26 at that we used to work on SAS for Model Development, SQL Server for data storage and SSIS for ETL
Great explanation
how to launch airflow webserver -p 8080 in terminal when i type these it is showing in terminal as airflow is not recognized in cmdlet
would you please share the code and csv files?
You can download the CSV file from elearning.k2analytics.co.in/SpApp/resources
Thanks for such a great content.
How to determine the number of partitions to use
Based on you cluster configuration & data size
Amazing series, thank you for putting this on YT
Very detailed explanation with examples.. Thanks!!
Hi! Thank you for this and I really learned a lot... I will just update you that the DummyOperator is already deprecated as per checking.
Yes...
Hello sir. You were going to start a batch. So has it been planned!
thank you sir
Why probability is belongs to 1( responder class) only? Why not 0?
Probability is always mentioned for the Event of Interest (Favourable Event). If there are two teams A & B playing a match. You are interested in Team A winning the match, then, you will naturally express the probability in terms of Team A winning the match (and not Team A losing the match)
@@RajeshJakhotiaAIML but how model know our class of interest?
nice explaination but , kindly provide code also.
Pause the video and type. It will be useful practice
Do we have complete online training available?
Yes. I will soon be starting a training batch
@@RajeshJakhotiaAIML ok thnks sir. How can we get the updates on the same! I also texted you on WhatsApp
@@SumitKumar-uq3dg I will soon share the link on RUclips community
Note that the DummyOperator is deprecated and beginning with the version 2.4.0 is not supported any more. You should use: from airflow.operators.empty import EmptyOperator
Thanks for this video. Really a great video on KS Statistics.
Hi! Could you please further explain what is the meaning of 1212 in 'random_state' ?
random_state = 1212 is just a seed. You can set the value to any number.
Hi! Could you please explain further what is the meaning of 1212 in 'random_state' ?
Random State 1212 is just a random number to ensure you get exactly the same output each time you execute the code. You can change 1212 by any random number say 1. Now if you run the code repeatedly using the value 1 you will get exactly the same output.
Hello Sir, where can I down load the file to work with this video?
elearning.k2analytics.co.in/SpApp/resources
Hi, Can you please provide the code which is used in the videos?
@Rajesh Jakhotia, is there a way we can pass this custom value in the code and not by UI
🌟🌟🌟Magnificent🌟🌟🌟 Very well explained. I don't think I am getting any problem in this again. Thanks Sir.
Great Explaination Rajesh, Really helped. Please let me know if we can connect.
Yes. Feel free to connec
I want to know why you used exec method to interpret the scripts file instead of using import functions in the DAG file.
When manually executing the master DAG, it calls the secondary DAG but does not execute the secondary DAG immediately as in the video. any idea how to solve it?
Encountered same problem. Apparently for mine, issue looks like its because it's using default sqlite, which doesn't allow parallelism, so DAG can only run sequentially. It will not execute secondary DAG since master DAG is still running.
make more videos like this sir , your video is more conceptual clear ,salute you sir