Boosting Explained-AdaBoost|Bagging vs Boosting|How Boosting and AdaBoost works

Boosting Explained-AdaBoost|Bagging vs Boosting|How Boosting and AdaBoost works
#AdaBoosting #BaggingVsBoosting #UnfoldDataScience
Hi,
My name is Aman and I am a data scientist
About this video:
This video tutorial will help you understand all about Boosting Machine Learning
and boosting algorithms and how they can be implemented
to increase the efficiency of Machine Learning models.
The following topics are covered in this session:
Why Is Boosting Used?
What Is Boosting?
What are differences between bagging and Boosting?
How Boosting Algorithm Works?
Types Of Boosting
How Adaboost works?
Understanding Adaboost with an example
About Unfold Data science: This channel is to help people understand basics of data science through simple examples in easy way. Anybody without having prior knowledge of computer programming or statistics or machine learning and artificial intelligence can get an understanding of data science at high level through this channel. The videos uploaded will not be very technical in nature and hence it can be easily grasped by viewers from different background as well.
Join Facebook group :
www.facebook.c...
Follow on medium : / amanrai77
Follow on quora: www.quora.com/...
Follow on twitter : @unfoldds
Get connected on LinkedIn : / aman-kumar-b4881440
Follow on Instagram : unfolddatascience
Watch Introduction to Data Science full playlist here : • Data Science In 15 Min...
Watch python for data science playlist here:
• Python Basics For Data...
Watch statistics and mathematics playlist here :
• Measures of Central Te...
Watch End to End Implementation of a simple machine learning model in Python here:
• How Does Machine Learn...
Have question for me? Ask me here : docs.google.co...

[@preranatiwary7690]

Adaboost was very confusing for me always, thanks for making it simple.

[@UnfoldDataScience]

Most Welcome :)

[@maheshreddy3488]

Hello sir..!
Sir I have a question. I'm have recently completed my graduation I'm interested in data science should I directly go for a data science or python?
I don't even know basics of python.

[@afeezlawal5167]

@@UnfoldDataScience thanks prof, But since Adaboost is generating weight different weight for each example(row) how will it know the weight to be given to an unseen data? Also, Since it keep updating the weight, this algorithm is prone to overfit the dataset right ?

[@denzelomondi6421]

@@maheshreddy3488 Start with learning python then you can learn Machine Learning

[@Birdsneverfly]

If one listens carefully, half of what you have explained are easily part of in-depth interview questions. Kudos!

[@UnfoldDataScience]

Thank you 🙂

[@kotireddykandula6761]

@@UnfoldDataScience you are videos are really awesome, could you provide the materials like handwritten notes or PDfs plz

[@ganeshgunjal4220]

these videos gives clear understanding than my course videos.
watching these instaed of my course. i take topic from course and watch video here.

[@mayurjadhav5332]

You indeed simplified this concept. Lot of videos confused me but yours is really simplest

[@وذكرفإنالذكرىتنفعالمؤمنين-ق7ز]

Thanks for your amazing explanation but I have 2 questions:
first: If I have 2000 features, the n_estimator will be 2000 also? or how to determine the number of them?
Second: If I have a big dataset as 100000 samples, is adaboost works well with it?
Thanks again 😀

[@HudsonDKent]

Thank you so much! I just finished listening to the whole playlist it is fantastic and your explanations are incredible

[@UnfoldDataScience]

Glad you like them! Happy Learning! Tc

[@Izumichan-nw1zo]

Sir u r literally life saver thank u so much

[@samrs007]

With a very practical & realistic example, enjoyed the way you explained such a complex topic using the building blocks as fundamentals. Your lucid & crisp explanation with a relevant example/illustration makes learning interesting. It indeed helps!!!

[@UnfoldDataScience]

Thanks a lot for motivating me through your comments. Keep watching and sharing :)

[@samrs007]

@@UnfoldDataScience Dear Aman, I would be greatly pleased if you can share the list of Books & Videos (& other reference materials) for grasping the Logical Flow of the Concepts & Theoretical Aspects better. If required, you can reply to my email id as shared via Google Forms.

[@PramodKumar-su8xv]

simple yet powerful explanation

[@nikhilgupta4859]

Thanks Aman for such informative video. Can you tell if the row selection with replacement exists here as well. Also if same applies for columns or not.

[@sandipansarkar9211]

great explanation. No where there is given such easy explanation especially from the point of beginners. Will jot down each and every point in my email and thoroughly go through it and subsequently add these to my notes. Your explanation is always from the point of first principles and not solely depend on libraries to perform a given task.

[@UnfoldDataScience]

So nice of you Sandipan.

[@pavanim6258]

Thanks Aman for ur detailed clear explanation in a very easy way to remember.Ive one question.How will we test propensity models and recommendation models in production on live data?Is there any way to test them?Could u pls let me know.

[@anirbansarkar6306]

You explain the data science concepts is such a simple manner that concepts which I used to presume as a mountain, turns into water after watching your videos. Please keep making such contents. This not only motivates us towards the stream(data science) but also boosts confidence and self belief. 🙏Thank you so much

[@UnfoldDataScience]

Cheers Anirban.

[@shreyasb.s3819]

Thanks a lot for easy and simple explaination

[@UnfoldDataScience]

Welcome Shreyas, thanks for watching

[@sudhanshusoni1524]

Beautifully explained.

[@UnfoldDataScience]

Thanks Sudhanshu.

[@Kumarsashi-qy8xh]

It's very much helpful

[@UnfoldDataScience]

Thank you for your support.

[@maheshreddy3488]

Sir I have a question. I'm have recently completed my graduation I'm interested in data science should I directly go for a data science or python?
I don't even know basics of python.

[@dataguy7013]

Excellent explanation , very intuitive!

[@mansibisht557]

Thankyou!! Simple and crisp explaination..

[@UnfoldDataScience]

You are welcome Mansi.

[@riniantony2325]

Thank you for the tutorial :)
Could you please let me know whether you have made a video explaining Adaboost more in detail? I mean with the formulas?

[@UnfoldDataScience]

Thank you. ada boost video is available.

[@SreepradhaaS]

Excellent explanation.. thanks

[@kariza87]

Very helpful video with simple explanation, plz make a video for adjustment of weights

[@UnfoldDataScience]

Thanks Sunil.

[@gatleekaw1825]

Thanks for the explanation! Really helped and I liked that you compared and contrasted with Bagging and also explained where Ada-Boost fit into the bigger picture of classifiers. Keep it up!

[@UnfoldDataScience]

Glad it was helpful. Thank you. Stay Safe. Tc.

[@TheOraware]

well explained simple is beatiful

[@UnfoldDataScience]

Thanks alot

[@Rajesh-nj7lw]

Very good explain technique you have. Great work of sharing your knowledge.If possible you can publish the book for ML.

[@rameshmamilla5392]

Thanks Aman! Nicely explained with example.

[@UnfoldDataScience]

Welcome Ramesh :)

[@RameshYadav-fx5vn]

Very nicely presented.

[@UnfoldDataScience]

Thank you :)

[@nivednambiar6845]

Nice tutorial aman. I was looking for this, Thanks for your nice explanation. I had one query, in this video where you are explaining the difference between the random forest and adaboost, you are saying that random forest have fully grown decision trees and adaboost as stumps
From the fully grown decision trees, did you meant whether the max_depth=None or was it just a way of conveying when comparing with adaboost algorithm stumps

[@samruddhideshmukh5928]

Good explanation as always! Keep up the good work.

[@UnfoldDataScience]

Thanks Samruddhi.

[@kamran_desu]

Very clear explanations, a happy new subscriber :)

[@UnfoldDataScience]

Thanks and welcome :).

[@bahram-848durani2]

Sir plz give a numerical example on Adaboost.

[@NeeRaja_Sweet_Home]

Nice Intuition…!!! Thanks for sharing looking for more ML videos.
Suppose we have 1 data set then how we will decide, which one we have to apply bagging or boosting or we have apply all algorithms and take the best accuracy one.

[@UnfoldDataScience]

Hi Rajesh, this intelligence will come as you work with multiple data sets on various use cases.Sometimes there are other factors as well. Join my live this Sunday 4PM ISTwe can discuss more. happy learning .tc

[@akifadas4202]

Thank you very much

[@UnfoldDataScience]

Welcome Akif :)

[@sandipansarkar9211]

finished watching

[@mohammedmunavarbsa573]

super sir plz continue series

[@UnfoldDataScience]

Thanks a lot Munavar.

[@sadhnarai8757]

Very nice Aman

[@UnfoldDataScience]

Thanks a lot :)

[@MrCEO-jw1vm]

thanks sir!!

[@sadhnarai8757]

Very nice

[@UnfoldDataScience]

Thank you.

[@61_shivangbhardwaj46]

Thnx sir😊

[@UnfoldDataScience]

Welcome Shivang.

[@abhishekgautam231]

Great job!

[@UnfoldDataScience]

Thanks Abhishek :)

[@dees900]

so the final prediction is based on the final adjusted model created sequentially and not on the average of the previous models?

[@pratyushdey5566]

Hi, your videos are really helpfull.
How are the weights accounted while determining the stumps?

[@UnfoldDataScience]

Hi Pratyush, If i am getting your question right, weights are adjusted after each iteration and then used in next calculations. Thanks

[@libinkoshy632]

Your videos are miraculous 😃

[@UnfoldDataScience]

Thanks Libin. Happy Learning. keep watching and take care!

[@AmitYadav-ig8yt]

May you explain with an example what do you mean by All the models have less say in the final result or weightage in the model?

[@UnfoldDataScience]

Hi Amit, thats a very nice question
So to start with, In Bagging models, lets say you have 500 individual models, so all 500 models will have equal weight to the prediction which means the final prediction will be average of 500 predictions if it is a regression use case.
On the other hand if its a boosting model like ada boost, all the individual 500 models are not same for the prediction. Every individual model has different amount of weight/amount of say in final prediction. So this is how it works:
Significance of evry model = 1/2log(1-total error)/total error
so the error from each stump is calculated and then in final prediction these significance numbers are attached to those models.
If you see above formula carefully, a single model which makes "less error" will have "more say" in final prediction and vice versa.
Let me know if more doubts.
Thanks
Aman

[@AmitYadav-ig8yt]

@@UnfoldDataScience Thank you very much , Aman for the explanation

[@viveksingh-rt4py]

Hi Aman, excellent explanation. Understood the concept completely. Thanks again for spreading the knowledge.
May I know when model will stop to learn. Will it stop when Gini index is 0 or at some other point ?
Thanks,
Vivek

[@UnfoldDataScience]

There are parameters we can set to stop further iteration.
Please see this once:
scikit-learn.org/stable/modules/generated/sklearn.ensemble.AdaBoostClassifier.html

[@ravim49]

Thank you so much for a nice explanation of bagging and adabooost .can these all be fit into technical data like process industry data.

[@UnfoldDataScience]

Yes, may be we need to do some data pre processing then train

[@ravim49]

Thank you so much for a quick response .Can you please share your personal email.I need to discuss my thesis work related to this .

[@maheshreddy3488]

Hello sir I have seen your profile in quora app and subscribed your channel.
Sir I have a question. I'm have recently completed my graduation I'm interested in data science should I directly go for a data science or python?
I don't even know basics of python.

[@UnfoldDataScience]

Hi mahesh, pls share your profile with me at [e-mail]. Let me have a look and i can comment

[@nishidutta3484]

How is the classification done for the final model? Like in bagging we use voting to predict the final class, how does adaboost predict the final class?

[@UnfoldDataScience]

Hi Nishi, that's a good question. It will not be a general voting rather as I mentioned all stumps have "amount of say" In final model hence based on weightage of each of the stump in final iteration, class will be predicted. Whicever is more. For example total amount of say for class 0 is 0.8 where total amount of say for class 1 is 0.2 then prediction will. Be class 0.

[@nishidutta3484]

@@UnfoldDataScience got it!! Thanks for clarifying

[@Sagar_Tachtode_777]

Thanks for the nice explanation.
Do we chose the models for training or the algorithm choses it in ensemble Learning Algorithm?

[@UnfoldDataScience]

Hi Sager, we need to choose which algorithm we want to run.

[@nitianbro7029]

can in fresher interview mathematical implementation also asked? or just an overview is good to go for fresher????

[@anbesivam7686]

You are a great teacher. You should teach to graduate students.

[@UnfoldDataScience]

Thanks Siva for your feedback :)

[@vinushan24]

Love you!

[@kritikamaheshwari3455]

can we apply diffrent algorithms in single boosting like some weak learners are using randon forest some are logistic and some knn in analyzing single data

[@datascienceworld7041]

In the Initial model the entire data will be taken or RowSampling and Column Sampling will be done like Random Forest .

[@UnfoldDataScience]

Yes.

[@raghusrivatsamp751]

How is the second model chosen?

[@moccadrea2932]

How if ada boost combine with other classifier algorithm to ensemble?

[@rajinanisar]

what is the meaning of can't have equal say in adaboost?

[@subodhagrawal4087]

You didn't explain that how many model it will create and on what basis which model will get used as final model?

[@UnfoldDataScience]

Ok Feedback taken. Thank you.

[@sameerpandey5561]

Hi Aman,
Thanks for the video...
Could you please explain what are the various ways through which we can increase the weight of mis-classified points.
Also How initial weights are initialized .

[@UnfoldDataScience]

It is taken care internally as per my knowledge . any other ways? possible not aware as of now.

[@ajaybandlamudi2932]

I have a question could you please solve it e what is the difference and similarities of Generalised Linear Models (GLMs) and Gradient Boosted
Machines (GBMs)

[@ajaybandlamudi2932]

Could please clarify my question?

[@gopalgopesh4680]

In boosting algo do we use all training data and iterate

[@UnfoldDataScience]

Depends on which boosting and what parameters we use. Not always but sometimes yes.

[@shakyaz]

Hi Aman, how does the algorithm "try to classify" the previously incorrect and given higher weightage, better in the second sequence or iteration?

[@UnfoldDataScience]

Hi Yogendra, if there are 10 records, equal weight will be 1/10 for all records, but in second iteration, let's say, two records are given 2/10 weight, then these record take preference while training model.

[@shakyaz]

@@UnfoldDataScience thanks for replying,, i wanted to understand how what portion of the algorithm helps it "take preference" of the high weightage records..

[@rnsahu2002]

When will the modelling stop if keep increase and decrease weight? If suppose after 100 models few prediction is wrong so should we keep creating models ?

[@UnfoldDataScience]

Hi Ram, it depends on what is the input we have given for "n" while calling the model. Usually 1000 is a good starting value for n. Thank you

[@HimanshuNarula]

Correct me if I am wrong.
In Adaboost, first, a weak learner is created then trained and then tested using the data by which the model got trained?

[@UnfoldDataScience]

yes, and the accuracy that u get is called train accuracy.

[@HimanshuNarula]

@@UnfoldDataScience Thanks for the reply sir.
Keep up the good work.

[@ravanshyam7653]

when we apply adabost and when we apply random forest how can we choose ?

[@UnfoldDataScience]

This depends on which data we are dealing with, there cannot be "one for all" fit.

[@Sagar_Tachtode_777]

Which one is preferred, bagging or boosting method?

[@UnfoldDataScience]

Depends on the use case.It can be subjective.

[@gowtamkumar5505]

Hi sir is adaboost only works on classification problem??

[@UnfoldDataScience]

No. For regression as well. Happy Learning. Tc

[@sidharthpanda7827]

Hii please share the formula how weight are given

[@UnfoldDataScience]

Hi Sidharth,
Initially weights are 1/n where n is number of records
Next iteration weight = initial weight * ( e to the power significance)
Where significance for every stump is
1/2log(1-total error)/total error

[@sidharthpanda7827]

@@UnfoldDataScience thanks for clearing the doubt.

[@rafsunahmad4855]

do we use stump in every weak learner?

[@UnfoldDataScience]

Yes.