There is an error in your formula for Shapley values (compared to christophm's book and wikipedia). Most write the weight as (M choose 1, |S|, M-|S|-1) where S is some subset WITHOUT feature i. However, you are summing over subsets z' WITH feature i. Thus, the weight should be (M choose 1, |z'| - 1, M-|z'|).
in case you face an error in this cell, i have been debugging for a while: shap.force_plot(explainer.expected_value[1], shap_values[..., 1], X_test[start_index:end_index])
This is a tour de force. A book I read with like-minded themes was also magnificent. "Game Theory and the Pursuit of Algorithmic Fairness" by Jack Frostwell
When you use summary force_plot to get an individual points contribution is the prediction shown the log of odds? If so how do I show the actual probability
Hi! Sorry didn't see that comment somehow. Did you have a look at this post: github.com/slundberg/shap/issues/963 ? For Tree Explainer you have an option to get probabilities, according to that.
Great Video. I have one question.... Can I use this SHAP model for Graph classification techniques, to see which nodes contribution is how much and kind of show a grad cam like effect? If you got any sort of resoyrces regarding this, please share. Thank you.
Hi! Do you mean the bars wrapped around z? That comes from set theory in math and the notation stands for cardinality = the number of elements in the set. It doesn't mean the "abs" function. Is that what you were referring to? :)
@@DeepFindr yes, thank you for clarifying. One more doubt, if I put z' =M then the (M -|z'| -1)! becomes (-1) ! which is not defined. What am I missing here?
@@yashnarendra Hi, good catch! In the original formula of Shapley values this can never happen, because the sum is over subsets without the feature i. Therefore F will always be greater than S (with the notation of the original shapley formula in the paper). But you are right, this is not really reflected in the SHAP formula. However in the paper they state that they excluded |z'| = 0 and |z'| = M as both are not defined.
Does the classification threshold have an effect on shapley values? The default threshold in binary classification is 0.5, what if one changes it to 0.7, how to incorporate this in the shap library?
Amazing video again bro. It helped a lot! One question: Do you know the reference where I can find the proof of the 2nd theorem in the SHAP paper? I can't find it :(
Hey :) thanks! The supplement can be downloaded here: papers.nips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html Plus there is another link in a discussion on Github, which might be helpful as well: github.com/slundberg/shap/issues/1054 Hope this helps :)
LIME is independent from the size of the Dataset. The only question is if the (Blackbox) model works on the Dataset. Can you maybe share some more details what makes you raise this question? :)
I have a very small dataset that surprising do well on prediction of 15 different class of identical species. In my black box model, I use three layers of feature extraction methods and finish off with one random forest model. I am not sure whether I can implement LIME for this situation because my black box were mostly feature extractions rather than ensemble of models.
Which feature extraction layers are you using? Is it trained end-to-end with the RF? It doesn't really matter what is happening inside your model. LIME is able to explain the input-output relation in a local area of a single prediction :)
There is an error in your formula for Shapley values (compared to christophm's book and wikipedia). Most write the weight as (M choose 1, |S|, M-|S|-1) where S is some subset WITHOUT feature i. However, you are summing over subsets z' WITH feature i. Thus, the weight should be (M choose 1, |z'| - 1, M-|z'|).
Thanks for pointing out, you are absolutely right! I'll pin this comment so that everyone can see it.
in case you face an error in this cell, i have been debugging for a while: shap.force_plot(explainer.expected_value[1],
shap_values[..., 1],
X_test[start_index:end_index])
Really insightful. Thank you for the video. It was a great explanation and demonstration to begin with.
Best explanation over internet..you should do more videos!
Great example with the competition!
Great explanation, would love more videos from you!
straight to the point and detailed explanation
all our videos are great. thanks a lot!
Amazing explaination and to the point. Thank you!
Thank you very much for explaining the concept so clearly.
Thanks, an Exceptional Explanation! Looking Forward to More Videos!.
Can you send me any links where we we can use Explainable on audio data.
Hi, great video, thanks! Is there a way to use SHAP for ARIMA/SARIMA?
Excellent video, thanks for sharing !
Great explanation. Thanks!
Thank you for your excellent explanation.
This is a tour de force. A book I read with like-minded themes was also magnificent. "Game Theory and the Pursuit of Algorithmic Fairness" by Jack Frostwell
Hi, May I know the dataset you used for the demo?
Informative!
When you use summary force_plot to get an individual points contribution is the prediction shown the log of odds? If so how do I show the actual probability
i have the same issue ... did you find something out?
Hi! Sorry didn't see that comment somehow.
Did you have a look at this post: github.com/slundberg/shap/issues/963
?
For Tree Explainer you have an option to get probabilities, according to that.
Great Presentation Thanks!
Amazing tutorial. Thanks a lot!
Great Video. I have one question.... Can I use this SHAP model for Graph classification techniques, to see which nodes contribution is how much and kind of show a grad cam like effect? If you got any sort of resoyrces regarding this, please share. Thank you.
Hi! I have a video called "how to explain graph neural networks" that exactly addresses this question :)
Might be a very stupid question but why is z' under modulus, if it represents number of features in the subset, it should be positive always, right.
Hi! Do you mean the bars wrapped around z? That comes from set theory in math and the notation stands for cardinality = the number of elements in the set. It doesn't mean the "abs" function.
Is that what you were referring to? :)
@@DeepFindr yes, thank you for clarifying. One more doubt, if I put z' =M then the (M -|z'| -1)! becomes (-1) ! which is not defined. What am I missing here?
@@yashnarendra Hi, good catch!
In the original formula of Shapley values this can never happen, because the sum is over subsets without the feature i. Therefore F will always be greater than S (with the notation of the original shapley formula in the paper).
But you are right, this is not really reflected in the SHAP formula. However in the paper they state that they excluded |z'| = 0 and |z'| = M as both are not defined.
@@DeepFindr Thanks a lot, really appreciate your efforts in replying to my queries.
thanks for the video!
Great video, thanks!
Can we use shap for multiclass classification, is there any resources which you can suggest.
Hi! Have a look at this discussion: github.com/slundberg/shap/issues/367
Does the classification threshold have an effect on shapley values? The default threshold in binary classification is 0.5, what if one changes it to 0.7, how to incorporate this in the shap library?
Hi! This discussion might be what you are looking for :)
@@DeepFindr did you mean to add a link to the discussion?
Yep, sorry here: github.com/slundberg/shap/issues/257
Amazing video again bro. It helped a lot! One question: Do you know the reference where I can find the proof of the 2nd theorem in the SHAP paper? I can't find it :(
Hey :) thanks!
The supplement can be downloaded here: papers.nips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html
Plus there is another link in a discussion on Github, which might be helpful as well: github.com/slundberg/shap/issues/1054
Hope this helps :)
@@DeepFindr There's nothing else I can say because you're the boss! ❤
Does shap works on small dataset ?
LIME is independent from the size of the Dataset. The only question is if the (Blackbox) model works on the Dataset. Can you maybe share some more details what makes you raise this question? :)
I have a very small dataset that surprising do well on prediction of 15 different class of identical species. In my black box model, I use three layers of feature extraction methods and finish off with one random forest model. I am not sure whether I can implement LIME for this situation because my black box were mostly feature extractions rather than ensemble of models.
Which feature extraction layers are you using?
Is it trained end-to-end with the RF?
It doesn't really matter what is happening inside your model. LIME is able to explain the input-output relation in a local area of a single prediction :)
@@DeepFindr I see, thanks! I will try this in my project soon.
OK good luck! If you have any problems let me know :)