There is an error in your formula for Shapley values (compared to christophm's book and wikipedia). Most write the weight as (M choose 1, |S|, M-|S|-1) where S is some subset WITHOUT feature i. However, you are summing over subsets z' WITH feature i. Thus, the weight should be (M choose 1, |z'| - 1, M-|z'|).
When you use summary force_plot to get an individual points contribution is the prediction shown the log of odds? If so how do I show the actual probability
Hi! Sorry didn't see that comment somehow. Did you have a look at this post: github.com/slundberg/shap/issues/963 ? For Tree Explainer you have an option to get probabilities, according to that.
Does the classification threshold have an effect on shapley values? The default threshold in binary classification is 0.5, what if one changes it to 0.7, how to incorporate this in the shap library?
Hi! Do you mean the bars wrapped around z? That comes from set theory in math and the notation stands for cardinality = the number of elements in the set. It doesn't mean the "abs" function. Is that what you were referring to? :)
@@DeepFindr yes, thank you for clarifying. One more doubt, if I put z' =M then the (M -|z'| -1)! becomes (-1) ! which is not defined. What am I missing here?
@@yashnarendra Hi, good catch! In the original formula of Shapley values this can never happen, because the sum is over subsets without the feature i. Therefore F will always be greater than S (with the notation of the original shapley formula in the paper). But you are right, this is not really reflected in the SHAP formula. However in the paper they state that they excluded |z'| = 0 and |z'| = M as both are not defined.
Great Video. I have one question.... Can I use this SHAP model for Graph classification techniques, to see which nodes contribution is how much and kind of show a grad cam like effect? If you got any sort of resoyrces regarding this, please share. Thank you.
This is a tour de force. A book I read with like-minded themes was also magnificent. "Game Theory and the Pursuit of Algorithmic Fairness" by Jack Frostwell
LIME is independent from the size of the Dataset. The only question is if the (Blackbox) model works on the Dataset. Can you maybe share some more details what makes you raise this question? :)
I have a very small dataset that surprising do well on prediction of 15 different class of identical species. In my black box model, I use three layers of feature extraction methods and finish off with one random forest model. I am not sure whether I can implement LIME for this situation because my black box were mostly feature extractions rather than ensemble of models.
Which feature extraction layers are you using? Is it trained end-to-end with the RF? It doesn't really matter what is happening inside your model. LIME is able to explain the input-output relation in a local area of a single prediction :)
Amazing video again bro. It helped a lot! One question: Do you know the reference where I can find the proof of the 2nd theorem in the SHAP paper? I can't find it :(
Hey :) thanks! The supplement can be downloaded here: papers.nips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html Plus there is another link in a discussion on Github, which might be helpful as well: github.com/slundberg/shap/issues/1054 Hope this helps :)
There is an error in your formula for Shapley values (compared to christophm's book and wikipedia). Most write the weight as (M choose 1, |S|, M-|S|-1) where S is some subset WITHOUT feature i. However, you are summing over subsets z' WITH feature i. Thus, the weight should be (M choose 1, |z'| - 1, M-|z'|).
Thanks for pointing out, you are absolutely right! I'll pin this comment so that everyone can see it.
Best explanation over internet..you should do more videos!
Really insightful. Thank you for the video. It was a great explanation and demonstration to begin with.
Great example with the competition!
straight to the point and detailed explanation
Great explanation, would love more videos from you!
Hi, great video, thanks! Is there a way to use SHAP for ARIMA/SARIMA?
Thank you for your excellent explanation.
Great explanation. Thanks!
all our videos are great. thanks a lot!
Informative!
Excellent video, thanks for sharing !
Thank you very much for explaining the concept so clearly.
Amazing explaination and to the point. Thank you!
thanks for the video!
Great video, thanks!
Hi, May I know the dataset you used for the demo?
When you use summary force_plot to get an individual points contribution is the prediction shown the log of odds? If so how do I show the actual probability
i have the same issue ... did you find something out?
Hi! Sorry didn't see that comment somehow.
Did you have a look at this post: github.com/slundberg/shap/issues/963
?
For Tree Explainer you have an option to get probabilities, according to that.
Great Presentation Thanks!
Thanks, an Exceptional Explanation! Looking Forward to More Videos!.
Can you send me any links where we we can use Explainable on audio data.
Does the classification threshold have an effect on shapley values? The default threshold in binary classification is 0.5, what if one changes it to 0.7, how to incorporate this in the shap library?
Hi! This discussion might be what you are looking for :)
@@DeepFindr did you mean to add a link to the discussion?
Yep, sorry here: github.com/slundberg/shap/issues/257
Might be a very stupid question but why is z' under modulus, if it represents number of features in the subset, it should be positive always, right.
Hi! Do you mean the bars wrapped around z? That comes from set theory in math and the notation stands for cardinality = the number of elements in the set. It doesn't mean the "abs" function.
Is that what you were referring to? :)
@@DeepFindr yes, thank you for clarifying. One more doubt, if I put z' =M then the (M -|z'| -1)! becomes (-1) ! which is not defined. What am I missing here?
@@yashnarendra Hi, good catch!
In the original formula of Shapley values this can never happen, because the sum is over subsets without the feature i. Therefore F will always be greater than S (with the notation of the original shapley formula in the paper).
But you are right, this is not really reflected in the SHAP formula. However in the paper they state that they excluded |z'| = 0 and |z'| = M as both are not defined.
@@DeepFindr Thanks a lot, really appreciate your efforts in replying to my queries.
Great Video. I have one question.... Can I use this SHAP model for Graph classification techniques, to see which nodes contribution is how much and kind of show a grad cam like effect? If you got any sort of resoyrces regarding this, please share. Thank you.
Hi! I have a video called "how to explain graph neural networks" that exactly addresses this question :)
Amazing tutorial. Thanks a lot!
Can we use shap for multiclass classification, is there any resources which you can suggest.
Hi! Have a look at this discussion: github.com/slundberg/shap/issues/367
This is a tour de force. A book I read with like-minded themes was also magnificent. "Game Theory and the Pursuit of Algorithmic Fairness" by Jack Frostwell
Does shap works on small dataset ?
LIME is independent from the size of the Dataset. The only question is if the (Blackbox) model works on the Dataset. Can you maybe share some more details what makes you raise this question? :)
I have a very small dataset that surprising do well on prediction of 15 different class of identical species. In my black box model, I use three layers of feature extraction methods and finish off with one random forest model. I am not sure whether I can implement LIME for this situation because my black box were mostly feature extractions rather than ensemble of models.
Which feature extraction layers are you using?
Is it trained end-to-end with the RF?
It doesn't really matter what is happening inside your model. LIME is able to explain the input-output relation in a local area of a single prediction :)
@@DeepFindr I see, thanks! I will try this in my project soon.
OK good luck! If you have any problems let me know :)
Amazing video again bro. It helped a lot! One question: Do you know the reference where I can find the proof of the 2nd theorem in the SHAP paper? I can't find it :(
Hey :) thanks!
The supplement can be downloaded here: papers.nips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html
Plus there is another link in a discussion on Github, which might be helpful as well: github.com/slundberg/shap/issues/1054
Hope this helps :)
@@DeepFindr There's nothing else I can say because you're the boss! ❤