NOTE: You can support StatQuest by purchasing the Jupyter Notebook and Python code seen in this video here: statquest.gumroad.com/l/uroxo Support StatQuest by buying my books The StatQuest Illustrated Guide to Machine Learning, The StatQuest Illustrated Guide to Neural Networks and AI, or a Study Guide or Merch!!! statquest.org/statquest-store/
Dear Josh ... I have a request for new videos or livechats ... could you explain us these tests maybe ? ... Tukey, Bonferroni and Scheffé , it's hard for me to understand , you explain everything so well ... could be very helpful for a lot of people out there ... have a nice day , greetings from Europe
Please keep doing these long-form Python tutorials on the various ideas we've covered in earlier 'Quests. They're great for those of us working in Python, and they give me another way to support the channel. It has been a more-than-pleasant surprise that as I've grown from learning the basics of stats to machine learning and eventually deep learning, StatQuest has grown along with me into those very same fields. Thanks Josh.
You are amazing. Can't imagine how much work you put into those step-by-step tutorials. Just bought the Jupyter Notebook code and it's beyond worth it! Thank you :)
Hey Josh, I just purchased all of your 3 Jupyter Notebook! I transferred from Econ major to Data Science, it was a nightmare before I find your channel. Your channel shed the light upon my academic career! Look forward to more of the 'Python from Start to Finish' series, and I will definitely support it!
I am purchasing the Jupiter notebook to contribute to your work! Thanks a lot for this video! You are awesome! Will be very very happy to have more ML tutorials and thank you Josh!
Your explanation of XGBoost was absolutely outstanding-clear, concise, and engaging. From my perspective, this video felt like a true Master Class! Your excellent pronunciation and delivery made it easy to follow and understand. You've definitely earned a new subscriber-thank you for sharing such high-quality content! Thank you so much.
Wow. Finally I see a face for the name. Your previous videos have had immensely helpul. I assumed you are a very senior person. I am not measuring your age. I mean, your way of explaining seemed like a professor with half a century of experience. But in reality, you are quite young. Thank you for all your simple-yet-detailed videos. No words to quantify how much I appreciate them. 🙏
Josh, this video is epic and really helped me understand the actual process of tuning hyperparameters, something that had been a bit of a black box until I saw this video. Your channel is awesome too - great jingles as well :D
Outstanding! Complete with the thinking path - how to analyze variables in a logical way - and with common errors. Just purchased the Notebook. Thank you for all of your work on this channel.
@@statquest one small request..can you provide some valuable information through a video like which model to chose for different datasets..how do we decide what model we should chose...thanks in advance
Absolutely loved this video Josh. It breaks down everything into understandable chunks. Thank you and God bless. BAM! The only thing I missed (and its very minor) was taking in a new data row and making an actual prediction by using the model.
You're a very kind human being Josh!! Thank you so much for making these videos. Your content is gold!!! I am new to data science and this is exactly what I needed!! :) Much love from India!
@@statquest Hey Josh! I am learning about Bayesian Optimizer and I don't seem to get it even after watching tons of tutorials, can you suggest where I should learn it from please? I couldn't find a video on your channel on this.
Hey Mate, amazing tutorial. Very complex problem explained in really simple and effective way. I am using XGBOOST for one of the classification model and after watching your video it made me realise I can further improve my model. So thank you again and keep making those videos. Kudos to you and long live data science 🙏🙏
It was extremely helpful. Please continue making these videos. I suggest making a video to explain the clustering with unlabeled data, and predicting the future trend in time-series data.
41:21 In my version of xgboost were parameters early_stopping_rounds=10 and eval_metric='aucpr' moved to XGBClassifier, so if it is not working I suggest trying this. Although I really appreciate value of StatQuest videos, I have to admit I really hate all that singing and Bams. Makes me feel like attending school for slower children
Thank you for this great tutorial Josh! Your videos have immensely helped in understanding some of the complex topics. One thing I noticed while watching this tutorial is the handling of categorical features. I think the explanation you gave for "Why not to use LabelEncoding?" is applicable for models like Linear Regression, SVM, NN but not for Trees because they only focus on the order of the feature values. For example, in a set of [1,2,3,4], threshold < 1.5 would be equivalent to threshold == 1. Please let me know if my thought process in wrong.
To be honest, I'm not really sure I understand your question or your example. If we have a categorical feature with 4 colors, red, blue, green, and black, and we give them numbers, like 1, 2, 3 and 4, then a threshold < 2.5, would not make much sense and, based on how trees are implemented, there would be no options for threshold == 2 or threshold == 3. So we wouldn't be able to separate colors very well.
@@nikhilshaganti5585 Sure, you can continue to separate things in later branches - but the greedy nature of the algorithm doesn't ensure that you'll get to those later branches. So you start by making a guess that it makes sense to group red and blue together.
First you’ve saved me this is super clear! I love all your videos so much 😊 I do have two questions… 1. How would you handle a classification problem with time series data? 2. Is there any other evaluation test you should or could do to evaluate the effectiveness of your model?
1. I've never used XGBoost with time series (or done much of any time series stuff before), so I can't answer this question. 2. There are lots of ways to evaluate a model. I only present a few, but there are many more, and they really depend on what you want your model to do. Just google it.
Again another quality video, I was following along with your every word, which did bring up come questions: 1. When XGBoost deals with missing data, does it ever consider splitting the missing data in half? --Using your example, would it ever do 1 blue and 1 green? What would happen if XGBoost encountered a data set with alot of missing values? 2. When you ran your Cross-Validation, was there a reason you only used 3 values for each hyperparameter? Could you have done more if you wanted to? 3. When I ran my Cross-Validation, my scale_pos_weight didn't change even though I used the same parameters you did. What do you think the problem could be?
1. Not that I know of. If there was a lot of missing values, it would still proceed just as described. 2. I wanted the cross validation to run in a short period of time, so I picked 3 values for each hyperparameter. If I had more time on my hand, or a cluster of computers, I might have considered trying more. 3. I'm not sure.
@@statquest Gotcha! Thank you for actually answering my questions haha I really appreciate the help as someone getting into more Machine Learning. Not sure what your upcoming videos will be but I think some great videos would be: -How you did the Cross Validation for the hyper parameters in this video? ( I have watched your Cross-Validation video, but I want to learn more about actually doing it) [I also still can get my first run to output the same values you had lol] -XGBoost for Regression in Python -How to select the best algorithm based on the scenario for Regression or Classification Just some thoughts :)
This is a great piece of work, thanks for sharing it! Maybe the only additional piece I'd add which I've found useful on the documentation of XGBoost is that one can take advantage of parallel computing (more cores or using a graphic card your machine or you could have on the cloud) by simply passing the parameter (n_jobs = -1) while doing both, the RandomizedSearchCV stage and the setting the XGB regressor type (XGBRegressor for example).
Wonderful video josh.....pleasee pleasee pleasee make more videos on start to finish on python for different models.....i havr actually submitted my assignments using your techniques and got better results than what i have learned in my class Waiting for more to come especially on python :)
Greatly appreciated this videoLike you said, telcos should gives more effort to tie the current customers. In real practice you want to know what the probability is that a current customer will no longer renew the subscription. You should then try to bind the customer with a high risk with incentives.
Thanks Josh for another GREAT video! Just some sharing and minor questions. 1. try pandas_profiling when doing EDA. I personally love it. :) 2. some features are highly correlated (eg: city name and zip code). Do we need to handle that before running XGB? 3. Why choose 10 for early_stopping_rounds 4. What’s the difference between - df.loc[df['Total_Charges']==' '] - df[df['Total_Charges']==' '] 5. What’s the difference between - y=df['Churn_Value'].copy - y=df['Churn_Value'] Many thanks in advance! H
1) Thanks for the tip on pandas_profiling. 2) No. 3) It's a commonly used number 4) I don't know. 5) I believe the former is copy by value and the latter is copy by reference.
Appreciate the Python related videos...helps to manoeuvre the code when I try to replicate the method later on...easy to follow the whole thing, also for beginners... 🙂
josh, i just absolutely love this video, after watching your videos about the theory part, then implement with real coding, is the best comb for learning. just have a few questions 1. when you use get_dummy, do we have to specify drop_first = True? normally, when should we drop first? 2. after converting using onehot, too many features were created and making X very sparse, is there a conventional way to deal with this problem? (say if instead of using xgb, using random forest, too many categorical converted features will dominate in feature selection for building the tree, how can we handle this problem?) 3. for labels, do we have to oversample the 1's since it's only 26%? i think this is a very interesting topic. as always, love your video and i want to show you my support and my appreciation to your amazing videos
just came up with a very silly question, i saw people are recommending using onehot to convert categorical variable, and not labelencoder (even doc says so), but tree based model like xgb and random forest, dont actually care about the ordinality, they care about gain and impurity reduction, so why we cannot use labelencoder?
1) In this case we don't need to drop the first - we're not worried about inverting a matrix or something like that to solve for parameters (i.e. like we would with linear regression). 2) If you use RandomForest in R, then you don't have to convert the categorical variables to one-hot, and you can avoid the sparse matrix problem. In general, the R implementation of RandomForests is infinitely better than the Python implementation. 3) It really depends.
Quick note: At 43:30 instead of using the plot_confusion_matrix() which is now depreciated, you need to use ConfusionMatrixDisplay.from_estimator(). This can be done as follows: Include: from sklearn.metrics import ConfusionMatrixDisplay at the start with the other imports. Then when printing the confusion matrix you need to use the line: ConfusionMatrixDisplay.from_estimator(clf_xgb, X_test, y_test, display_labels=["Did not leave", "Left"])
I am getting this error following what you say: XGBoostError: [17:07:52] c:\buildkite-agent\builds\buildkite-windows-cpu-autoscaling-group-i-08de971ced8a8cdc6-1\xgboost\xgboost-ci-windows\src\c_api\c_api_utils.h:167: Invalid missing value: null Do you know why it might be? Thanks.
I solved the above error by setting missing=0 on the above code as follows: clf_xgb = xgb.XGBClassifier(objective='binary:logistic', missing=0, seed=42, ## the next three arguments set up early stopping. eval_metric='aucpr', early_stopping_rounds=10)
Yikes, if I ever understand something enough to explain it as succinctly as you do then I'd be very happy. I've been smashing through a lot of your videos the last few days after spending countless months on python, sklearn and all the usual plug and play solutions and it's not been until I've started watching these that I've started to feel things click into place
38:05 stratify=y is not for yes is for dependent variable y. if you have a different variable, you gotta pass your response variable's name to stratify
Amazing. Wonderful videos. I started only 3 months ago and with your videos I am very confident to do nalysis with Python. Manny thanks. Is it possible to create a video for Nomogram for competic#ing risks for Time-Event (survival analysis) based on CPH outputs?
This is gold, thank you! I am a rookie of this stuff, still I am unsure one-hot encoding is the best to do, especially to encode the city; being a category with high cardinality, all those variables for 1-hot encoding will require many splits (I guess). Perhaps using a different encoding, like mean encoding or frequency encoding, would be better, may allow to have a good fit with fewer splits.
Awesome Video. Thanks Josh. I had 2 questions: 1. I think we can remove one 1-Hot encoded columns for each categorical variable. For example if we had a variable cat_or_dog with only 2 categorical values: Cat and Dog. Knowing that it is a cat means that is not a dog and vice versa. So we need only one column to encode this variable with 2 values. I think if we use all the one-hot encoded columns of a variable for linear/ logistic regression it will make the Hat Matrix non-invertible but dropping one column for every variable should work? 2. Also, I have seen people retain a validation set to evaluate the model when it is fitting and test data for final testing the model just so that they don't overfit to the validation set?
Hey! Thank you so much for all the work you have been doing! I have a question regarding the leaves values if we use xgboost in a regression problem. What do they mean then? Is it probability that the average of the observations that were segregated to that particular leaf has this and this probability of contributing to the loss function? Thank you so much for your help!
@@statquest Sir your double and triple bam has really taught lots of things for me to be honest. Me as a student doing Post-Graduate in AI, lots of love form Nepal.
Thank you for the excellent content and walk through. While One Hot Encoding the categorical variables should not we use k-1 variables for k categories? For example, for the Payment_Method column if Mailed_Check=0, Electronic_Check=0, and Bank_Transfer=0 doesn't that imply Credit_Card=1 and make Credit_Card column redundant?
hey first of all thank you so much for all your videos! I understood everything and i am hyped about trying out what i have learned, but i have one question. So after we built our model and we are happy with how it performs, how do we feed it with actual new unseen data? If i have for example just the data for one specific customer and i want to check if he/she leaves or stays, what would the code look like.
Prof Starmer, I’ve a question regarding what you said at 31:30 on OHE not working for regression models. Would you be able to kindly explain how to encode categorical data before training these models then?
Hi Josh. one request can you do more of these start to finish series. Also please create a separate playlist for these start to finish videos. Thank you so much for your efforts.
NOTE: You can support StatQuest by purchasing the Jupyter Notebook and Python code seen in this video here: statquest.gumroad.com/l/uroxo
Support StatQuest by buying my books The StatQuest Illustrated Guide to Machine Learning, The StatQuest Illustrated Guide to Neural Networks and AI, or a Study Guide or Merch!!! statquest.org/statquest-store/
Dear Josh ... I have a request for new videos or livechats ... could you explain us these tests maybe ? ... Tukey, Bonferroni and Scheffé , it's hard for me to understand , you explain everything so well ... could be very helpful for a lot of people out there ... have a nice day , greetings from Europe
Please keep doing these long-form Python tutorials on the various ideas we've covered in earlier 'Quests. They're great for those of us working in Python, and they give me another way to support the channel. It has been a more-than-pleasant surprise that as I've grown from learning the basics of stats to machine learning and eventually deep learning, StatQuest has grown along with me into those very same fields.
Thanks Josh.
That's the plan!
You are amazing. Can't imagine how much work you put into those step-by-step tutorials.
Just bought the Jupyter Notebook code and it's beyond worth it! Thank you :)
Thank you very much for your support! :)
Hey Josh, I just purchased all of your 3 Jupyter Notebook! I transferred from Econ major to Data Science, it was a nightmare before I find your channel. Your channel shed the light upon my academic career! Look forward to more of the 'Python from Start to Finish' series, and I will definitely support it!
Awesome! Thank you!
Your pronunciation is the most authentic and clearest that I have ever heard
Wow! Thank you!
I am purchasing the Jupiter notebook to contribute to your work! Thanks a lot for this video! You are awesome! Will be very very happy to have more ML tutorials and thank you Josh!
Thank you very much! :)
This is hands down the best Python tutorial on RUclips.. not just for XGBoost, but overall Python logic and syntax. Nice work, subscribed!!
Wow! Thank you!
Your explanation of XGBoost was absolutely outstanding-clear, concise, and engaging.
From my perspective, this video felt like a true Master Class! Your excellent pronunciation and delivery made it easy to follow and understand.
You've definitely earned a new subscriber-thank you for sharing such high-quality content! Thank you so much.
Thank you very much!
Extremeley helpful - would love to see more from the "start to finish" series
I'm working on it.
Josh, you’re well and truly phenomenal ! Love from Madras !
Chennai
BAM! Thank you very much!!!
Hi Rahul I taught atIIT-madras 19192-1993 lived on campus across from post office josh visited us there
Frank Starmer Hello Frank, wow! That’s great to know ! :) I’m sure you must have had a good time here. Cheers :)
@@starmerf wow the world is a small place ☺
Wow. Finally I see a face for the name. Your previous videos have had immensely helpul. I assumed you are a very senior person. I am not measuring your age. I mean, your way of explaining seemed like a professor with half a century of experience. But in reality, you are quite young. Thank you for all your simple-yet-detailed videos. No words to quantify how much I appreciate them. 🙏
Wow, thanks!
Josh, this video is epic and really helped me understand the actual process of tuning hyperparameters, something that had been a bit of a black box until I saw this video. Your channel is awesome too - great jingles as well :D
Thank you!
Outstanding! Complete with the thinking path - how to analyze variables in a logical way - and with common errors. Just purchased the Notebook. Thank you for all of your work on this channel.
Triple bam!!! Thank you very much for supporting StatQuest! :)
Thank you for the great work!
Wow! Thank you so much for supporting StatQuest!!! BAM! :)
What did we do to deserve a great guy like Josh ? Thank you Josh!
Thanks! :)
Thanks for the great tutorial! You covered a lot of details (mostly data cleaning) that are often overlooked or skipped as 'trivial' steps.
Thank you! Yes, "data cleaning" is 95% of the job.
This kind of content is SUPER HARD to produce. I really understand and appreciate your effort here. Thanks and congratulations.
Thank you very much!
Can't thank you enough for the clearest and best explanation on RUclips
Thank you!
I love your teaching style. Extremely helpful for a beginner like me. Really helped me a lot in my exams. No words. You are the best!!!!
Thank you!
"25:36" that's what i was waiting for from the beginning...Truly amazing.. You are providing precious information..CHEERS
Glad it was helpful!
@@statquest one small request..can you provide some valuable information through a video like which model to chose for different datasets..how do we decide what model we should chose...thanks in advance
@@RahulVarshney_ I'll keep that in mind. In the mean time, check out: scikit-learn.org/stable/tutorial/machine_learning_map/index.html
@@statquest that is amazing ...i will complete it today itself thanks again for your prompt reply
Can i get your email
I watched your all video for XGBoost. It helps me a lot. very appreciated!
Glad it helped!
I was wondering how to find stuff regarding dealing with actual churn data and sampling issues. The tutorial addressed a lot of them. Thanks!
Thanks!
Absolutely loved this video Josh. It breaks down everything into understandable chunks. Thank you and God bless. BAM! The only thing I missed (and its very minor) was taking in a new data row and making an actual prediction by using the model.
Thanks! For new data, you just call clf_xgb.predict() with the row of new data.
Hi Josh, great job really helpful material as I'm discovering XGBoost just now.
Thank you and keep you great work!
Thank you very much! :)
Awesome video! The cleanest xgboost explanation a have ever seen.
Wow, thanks!
Such an amazing job Josh.. Couldn't find any better explanation than this!
Mesmerizing!
Wow, thanks!
I understand you better than if it would be explained in my native language) Thank you for your work!
Thank you!
A true, real Master Class - You got my support!
Thank you! :)
You're a very kind human being Josh!! Thank you so much for making these videos. Your content is gold!!! I am new to data science and this is exactly what I needed!! :)
Much love from India!
Glad you like my videos!! BAM! :)
@@statquest Hey Josh! I am learning about Bayesian Optimizer and I don't seem to get it even after watching tons of tutorials, can you suggest where I should learn it from please? I couldn't find a video on your channel on this.
@@muskanroxx22 Unfortunately I don't know of a good source for that.
Hey Mate, amazing tutorial. Very complex problem explained in really simple and effective way. I am using XGBOOST for one of the classification model and after watching your video it made me realise I can further improve my model. So thank you again and keep making those videos. Kudos to you and long live data science 🙏🙏
Glad it helped!
I'm very grateful to have you as my teacher.
Thanks!
I really appreciate your content Josh. Thanks for your time
Thank you!
Thanks!
WOW! Thank you so much for supporting StatQuest!!! BAM! :)
I haven't watched it yet but I know this will be great!!!!!!!! Thank you Josh.
BAM! :)
That's the Best video I've ever seen. Period.
TRIPLE BAM! :)
Wow, thanks!
It was extremely helpful. Please continue making these videos. I suggest making a video to explain the clustering with unlabeled data, and predicting the future trend in time-series data.
I'll keep that in mind. :)
BAMMMMM !!!
This is awesome 👍 Josh !! Thank you for your contribution, really helpful for new learners.😊😊😊
Glad you liked it!
@@salilgupta9427 Thanks!
Thank you Josh. Needed this tutorial to better solve a ML Problem as part of my internship :)
Glad it helped!
I'm so glad you are a bad-ass stats guru and a teacher waaaaaaaaay before a singer and a guitarist ...Thank you! ;)
joshuastarmer.bandcamp.com/
StatQuest with Josh Starmer ...not bad. A poor man’s Jack Johnson 🤔
Just pulling your leg. Thanks for all the content on stats
41:21 In my version of xgboost were parameters early_stopping_rounds=10 and eval_metric='aucpr' moved to XGBClassifier, so if it is not working I suggest trying this.
Although I really appreciate value of StatQuest videos, I have to admit I really hate all that singing and Bams. Makes me feel like attending school for slower children
Sorry to hear that.
@@statquest i like it. its cool.
Yes pls more videos with python❤thank u for the webinar
Thanks! :)
hurray, i picture you totally different!
Thanks a lot for all the videos!
Glad you like them!
Thank you for this great tutorial Josh! Your videos have immensely helped in understanding some of the complex topics. One thing I noticed while watching this tutorial is the handling of categorical features. I think the explanation you gave for "Why not to use LabelEncoding?" is applicable for models like Linear Regression, SVM, NN but not for Trees because they only focus on the order of the feature values. For example, in a set of [1,2,3,4], threshold < 1.5 would be equivalent to threshold == 1. Please let me know if my thought process in wrong.
To be honest, I'm not really sure I understand your question or your example. If we have a categorical feature with 4 colors, red, blue, green, and black, and we give them numbers, like 1, 2, 3 and 4, then a threshold < 2.5, would not make much sense and, based on how trees are implemented, there would be no options for threshold == 2 or threshold == 3. So we wouldn't be able to separate colors very well.
right, in your example, the threshold of threshold
@@nikhilshaganti5585 Sure, you can continue to separate things in later branches - but the greedy nature of the algorithm doesn't ensure that you'll get to those later branches. So you start by making a guess that it makes sense to group red and blue together.
just here to say thank you! will come back in a month when I have time to watch it. :)
BAM! :)
Great video! Love it!
request that you do a comparison of XGBoost, CatBoost, and LightGBM, and a quest on ensemble learning.
I'll keep those topics in mind.
Thank you for your job, the explanation of the topic is very clear and transparent.
Thank you very much! :)
Liked, favorited, recommended, shared, and sacrificed my first-born to this video.
TRIPLE BAM! :)
😂😂😂
First you’ve saved me this is super clear! I love all your videos so much 😊
I do have two questions…
1. How would you handle a classification problem with time series data?
2. Is there any other evaluation test you should or could do to evaluate the effectiveness of your model?
1. I've never used XGBoost with time series (or done much of any time series stuff before), so I can't answer this question.
2. There are lots of ways to evaluate a model. I only present a few, but there are many more, and they really depend on what you want your model to do. Just google it.
Again another quality video, I was following along with your every word, which did bring up come questions:
1. When XGBoost deals with missing data, does it ever consider splitting the missing data in half?
--Using your example, would it ever do 1 blue and 1 green? What would happen if XGBoost encountered a data set with alot of missing values?
2. When you ran your Cross-Validation, was there a reason you only used 3 values for each hyperparameter? Could you have done more if you wanted to?
3. When I ran my Cross-Validation, my scale_pos_weight didn't change even though I used the same parameters you did. What do you think the problem could be?
1. Not that I know of. If there was a lot of missing values, it would still proceed just as described.
2. I wanted the cross validation to run in a short period of time, so I picked 3 values for each hyperparameter. If I had more time on my hand, or a cluster of computers, I might have considered trying more.
3. I'm not sure.
@@statquest Gotcha! Thank you for actually answering my questions haha I really appreciate the help as someone getting into more Machine Learning. Not sure what your upcoming videos will be but I think some great videos would be:
-How you did the Cross Validation for the hyper parameters in this video? ( I have watched your Cross-Validation video, but I want to learn more about actually doing it) [I also still can get my first run to output the same values you had lol]
-XGBoost for Regression in Python
-How to select the best algorithm based on the scenario for Regression or Classification
Just some thoughts :)
@@andrewnguyen5881 I'll keep those topics in mind.
Man, this video is awesome! Congratulations!
Thank you! :)
It is really lovely to be able to put a face to the "Hooray!", "BAM !!!" and "Note:"s 😄❤
bam!
Good to also see you sing rather than just hear :).. i had to comment this even before starting the training
😊 thanks
could not help u with money right now , but i watched all the adds in video , hope that helps u financially . love u videos . keep up!!
I appreciate that
This is a great piece of work, thanks for sharing it!
Maybe the only additional piece I'd add which I've found useful on the documentation of XGBoost is that one can take advantage of parallel computing (more cores or using a graphic card your machine or you could have on the cloud) by simply passing the parameter (n_jobs = -1) while doing both, the RandomizedSearchCV stage and the setting the XGB regressor type (XGBRegressor for example).
Great tip! BAM!
Lovely and priceless video Josh...BAM BAM BAM as usual !! :) God bless. .
Thank you very much! :)
Wonderful video josh.....pleasee pleasee pleasee make more videos on start to finish on python for different models.....i havr actually submitted my assignments using your techniques and got better results than what i have learned in my class
Waiting for more to come especially on python :)
Thanks! There should be more python coming out soon.
you are simply an amazing human being, also the notebooks are great! :D
Thanks!
Amazingly organized and well explained!
Thank you!
Thank you so much for the work that you used in step by step tutorial. it was amazing.
You're very welcome!
Hard work here, I'ts funny how the responsabile scientist and the funny guy coexist, very useful lesson, thanks!
Thanks! 😃!
many thanks to your great and so understandable video. It literaly helps me a lot in Python and XGBoost package
Glad it helped!
Greatly appreciated this videoLike you said, telcos should gives more effort to tie the current customers. In real practice you want to know what the probability is that a current customer will no longer renew the subscription. You should then try to bind the customer with a high risk with incentives.
True!
I love the channel! Eu aprendo + aqui do que a Graduação! You great josh!
Muito obrigado! :)
Great video! You did a wonderful job of explaining the process. Thanks!
Thanks!
Freaking amazing! You explain everything so well. Thank you!
Thank you!
Josh, you're the didactic in person form.
Thanks!
I appreciate that!
Jesus, i just learned more over 10 minutes of this than i did throughtout an entire semester of a similar subject on CS. ++ tutorial
Thank you!
Thank you Josh!!
As a suggestion, you could do a StatQuest explaining the measures in market basket analysis?
I'll keep that in mind.
all week searched for this thank u very much
Enjoy!
Thanks Josh for another GREAT video! Just some sharing and minor questions.
1. try pandas_profiling when doing EDA. I personally love it. :)
2. some features are highly correlated (eg: city name and zip code). Do we need to handle that before running XGB?
3. Why choose 10 for early_stopping_rounds
4. What’s the difference between
- df.loc[df['Total_Charges']==' ']
- df[df['Total_Charges']==' ']
5. What’s the difference between
- y=df['Churn_Value'].copy
- y=df['Churn_Value']
Many thanks in advance!
H
1) Thanks for the tip on pandas_profiling.
2) No.
3) It's a commonly used number
4) I don't know.
5) I believe the former is copy by value and the latter is copy by reference.
This guy is amazing.
DOUBLE BAM 💥 💥
Thank you! :)
Appreciate the Python related videos...helps to manoeuvre the code when I try to replicate the method later on...easy to follow the whole thing, also for beginners... 🙂
Thanks! There will be a lot more python stuff soon.
Very enjoyable webinar Josh. Thanks for posting. I'm not 100% sure how to interpret the leaves. The largest leaf value is 0.188 where Dependents_No
The leaves are how much to increase or decrease the log(odds) for one category. For more details, see: ruclips.net/video/8b1JEDvenQU/видео.html
josh, i just absolutely love this video, after watching your videos about the theory part, then implement with real coding, is the best comb for learning. just have a few questions
1. when you use get_dummy, do we have to specify drop_first = True? normally, when should we drop first?
2. after converting using onehot, too many features were created and making X very sparse, is there a conventional way to deal with this problem? (say if instead of using xgb, using random forest, too many categorical converted features will dominate in feature selection for building the tree, how can we handle this problem?)
3. for labels, do we have to oversample the 1's since it's only 26%? i think this is a very interesting topic.
as always, love your video and i want to show you my support and my appreciation to your amazing videos
just came up with a very silly question, i saw people are recommending using onehot to convert categorical variable, and not labelencoder (even doc says so), but tree based model like xgb and random forest, dont actually care about the ordinality, they care about gain and impurity reduction, so why we cannot use labelencoder?
1) In this case we don't need to drop the first - we're not worried about inverting a matrix or something like that to solve for parameters (i.e. like we would with linear regression).
2) If you use RandomForest in R, then you don't have to convert the categorical variables to one-hot, and you can avoid the sparse matrix problem. In general, the R implementation of RandomForests is infinitely better than the Python implementation.
3) It really depends.
To be honest, I'm not sure.
@@statquest thank you for your clarification josh, i will definitely dig deeper into how to solve the sparsity problem once utilize onehot
@@statquest this is a quite interesting problem, lol, never mind, let me find out what others do on kaggle
Quick note: At 43:30 instead of using the plot_confusion_matrix() which is now depreciated, you need to use ConfusionMatrixDisplay.from_estimator(). This can be done as follows:
Include: from sklearn.metrics import ConfusionMatrixDisplay at the start with the other imports. Then when printing the confusion matrix you need to use the line: ConfusionMatrixDisplay.from_estimator(clf_xgb, X_test, y_test, display_labels=["Did not leave", "Left"])
Thank you very much! I really appreciate it.
@@statquest no problem!
I am getting this error following what you say:
XGBoostError: [17:07:52] c:\buildkite-agent\builds\buildkite-windows-cpu-autoscaling-group-i-08de971ced8a8cdc6-1\xgboost\xgboost-ci-windows\src\c_api\c_api_utils.h:167: Invalid missing value: null
Do you know why it might be? Thanks.
I solved the above error by setting missing=0 on the above code as follows:
clf_xgb = xgb.XGBClassifier(objective='binary:logistic',
missing=0,
seed=42,
## the next three arguments set up early stopping.
eval_metric='aucpr',
early_stopping_rounds=10)
@@emilioluissaenzguillen5719 thanks it works
Love u Josh.... you are a TRIPLE BAM!!! Greetings from Bogotá, Colombia.
Muchas gracias!!! :)
Yikes, if I ever understand something enough to explain it as succinctly as you do then I'd be very happy. I've been smashing through a lot of your videos the last few days after spending countless months on python, sklearn and all the usual plug and play solutions and it's not been until I've started watching these that I've started to feel things click into place
Awesome! I'm glad my videos are helpful! :)
Great explanation and walk-through, big thanks!
Glad you enjoyed it!
Excellent Video @StatQuest ! Can we please have more Start to Finish python videos? Like Lightgbm maybe?
I'll keep that in mind! :)
38:05 stratify=y is not for yes is for dependent variable y. if you have a different variable, you gotta pass your response variable's name to stratify
Oops! thanks for catching that.
Amazing! Thanks so much for the detailed video
Thanks!
Amazing. Wonderful videos. I started only 3 months ago and with your videos I am very confident to do nalysis with Python. Manny thanks.
Is it possible to create a video for Nomogram for competic#ing risks for Time-Event (survival analysis) based on CPH outputs?
I'll keep that topic in mind.
Love these Python tutorials after watching theory videos:D
Glad you like them!
it would be a great idea to have an entire series of "start to finish"
Thanks!
Hello josh, you are doing amazing work keep doing
Thanks!
amazing tutorial Josh! Shared with my friends =D
Could you do one of these about pygam? It would be amazing :)
I'll keep that in mind.
This is gold, thank you! I am a rookie of this stuff, still I am unsure one-hot encoding is the best to do, especially to encode the city; being a category with high cardinality, all those variables for 1-hot encoding will require many splits (I guess). Perhaps using a different encoding, like mean encoding or frequency encoding, would be better, may allow to have a good fit with fewer splits.
Maybe. Try it out and let me know if you get something that works better.
awesome!! I hope you can do a video about XGBoost with regression too.
I'll keep that in mind.
Awesome Video. Thanks Josh. I had 2 questions:
1. I think we can remove one 1-Hot encoded columns for each categorical variable. For example if we had a variable cat_or_dog with only 2 categorical values: Cat and Dog. Knowing that it is a cat means that is not a dog and vice versa. So we need only one column to encode this variable with 2 values. I think if we use all the one-hot encoded columns of a variable for linear/ logistic regression it will make the Hat Matrix non-invertible but dropping one column for every variable should work?
2. Also, I have seen people retain a validation set to evaluate the model when it is fitting and test data for final testing the model just so that they don't overfit to the validation set?
1) Yes, you can do it that way. For XGBoost and trees in general, we don't have to, but we can.
2) Yes, you can use a validation set if you want.
Hey! Thank you so much for all the work you have been doing! I have a question regarding the leaves values if we use xgboost in a regression problem. What do they mean then? Is it probability that the average of the observations that were segregated to that particular leaf has this and this probability of contributing to the loss function? Thank you so much for your help!
see: ruclips.net/video/OtD8wVaFm6E/видео.html
I am not a math geek. I am here strictly for the intro song!
same here
DOUBLE BAM! :)
@@statquest Sir your double and triple bam has really taught lots of things for me to be honest. Me as a student doing Post-Graduate in AI,
lots of love form Nepal.
Great video! Very informative and clearly explained! Could you please also present BART?
I'll keep that in mind.
Thank you for the excellent content and walk through. While One Hot Encoding the categorical variables should not we use k-1 variables for k categories? For example, for the Payment_Method column if Mailed_Check=0, Electronic_Check=0, and Bank_Transfer=0 doesn't that imply Credit_Card=1 and make Credit_Card column redundant?
In this setting it doesn't matter - unlike linear models, xgboost doesn't have to worry about inverting a matrix to solve for parameters.
hey first of all thank you so much for all your videos! I understood everything and i am hyped about trying out what i have learned, but i have one question. So after we built our model and we are happy with how it performs, how do we feed it with actual new unseen data? If i have for example just the data for one specific customer and i want to check if he/she leaves or stays, what would the code look like.
Just use "clf_xgb.predict(YOUR DATA)", where "YOUR DATA" is formatted in the same order as the data used for training.
@@statquest oh god of course thank you! I had a blackout for a moment haha
wow you are great instructor, very well explained! Are you also familiar with big data such as apache spark?
Not yet.
This man deserves Nobel Prize for peace of mind ❤❤
bam! :)
Prof Starmer, I’ve a question regarding what you said at 31:30 on OHE not working for regression models. Would you be able to kindly explain how to encode categorical data before training these models then?
Sure, see: ruclips.net/video/CqLGvwi-5Pc/видео.html
Hi Josh. one request can you do more of these start to finish series. Also please create a separate playlist for these start to finish videos.
Thank you so much for your efforts.
I want to do more. And until I get that playlist made, you can find all of my webinars listed here: statquest.org/video-index/#webinars
Thank you so much Josh Starmer! BAM!
bam!
If "Stay Cool" had a face, it'd be you :)
Bam!