Hello Sir, I think there is mistake in this video for backpropagation. Basically to find out (del L)/(del (w11^2)), we don't need the PLUS part. Since here O22 doesn't depend on w11^2. Please look into that. The PLUS part will be needed while calculating (del L)/(del (w11^1)), there O21 & O22 both depend on O11 and O11 depends on w11^1.
You are correct concerning that, but I think he wanted to take derivative w.r.t O11 since it is present in both nodes of f21 and f22, so if we replace w11^2 in the equation by O11 the equation would be correct
I don't want to calulate Loss function to your videos and no need to propagate the video back and forward i.e you explained in such a easiest way I have ever seen in others. Keep doing more and looking forward to learn more from you. Thanks a ton.
It has been years since I had solved any mathematics question paper or looked at mathematics book. But the way you explained was damn good than Ph.D. holder professors at the University. I did not feel my away from mathematics at all. LoL- I do not understand my professors but understand you perfectly
Thank you for the perfect DL Playlist to learn, wanted to highlight a change to make it 100% useful (Already at 99.99%), 13:04 - For Every Epoch, the Loss Decreases adjusting according to the Global Minima.
This is simply yet Superbly explained. When I learnt earlier, it stopped at Back Propagation. Now, learnt what is in Backpropagation that makes the Weights updation in an appropriate way, i.e., Chain rule. Thanks much for giving clarity that is easy to understand. Superb.
Nice informative video. It helped me in understanding the concept. But i think at end there is a mistake. You should not add the other path to calculate the derivative for W11^2. Addition should be done if we are calculating the derivative for O11. w11^2(new) = (dl/dO31 * dO31/dO21 * dO21/dW11^2)
Of the two connections from f11 to the second hidden layer, w11^2 is affecting only f21 and not f22(as it affected by w21^2). So, dL/dw11^2 will only have one term instead of two. Anyone, pls correct me if i am wrong.
Well Explained sir ! Before starting the deep learning, I have decided to start the learning from your videos. You explain in very simple way ...Anyone can understand from your video. Keep it up Sir :)
the last partial derivative of Loss we have calculated w.r.t. (w11^2) is that correct how we have shown there that it is dependent upon two paths one w11^2 and other w12^2 ......... Please make it clear i am confused about it ??????
assume he is explaining about W11^1 and youll understand everything. From the diagram itself, you can see the connections and can clearly imagine which weights are dependent on each other . Hope this helps
Awesome Mate. however, I think you got carried away for the second part to be added. read the comments below and correct, please. W12 may not need to be added. But it all makes sense. A very good explanation.
I think this part dL/dw11^2 should be (dL/dO31 *dO31/O21 *dO21/dO11^2). If we are taking derivative of dL w.r.t w11^2 then,w12^2 doesn't come into play. So,in that case, dL/dO12^2= (dL/dO31 *dO31/O22 *dO22/dw12^2)
Not required, its not correct as w11^2 is not affected by lower weights. The 1st part is correct and summation is required , when we are thinking about w11^1.
Loved it man... Great effort in explaining the maths behind it and chain rule. Pls make a video on its implementation soon. as usual great work.. Looking forward for the videos. Cheers
Hello Sunny, I myself have stitched an absolutely brilliant repository explaining all the implementation details behind an ANN. See this: github.com/jalotra/Neural_Network_From_Scratch
His sir i think there is something wrong wrong because the w11 to the suffix 2 is not impacted with the w12 to the suffix 2..! But this playlist is really helpfull to me thankyou sir...:)
Im able to understand the concepts you are explaining, but I dont know that from where do we get values for weights in forward propgation.Could you brief about that once if possible.
Hey Krish, your way of explanation is good. I think there is one correction. In the end, you explained for w11^2, what I feel is, it is for w11^1. It would be really helpful if you correct it because many are getting confused with it.
sir i think one thing you are doing is worng. as w^(3)11 impacts O(31) , here is one activation part. so the dL/dw^(3)11 = dL/dO(31) . d0(31)/df1 . df1/dw^(3)11 I might be wrong, can you please clear my query ?
Thank you so much for all your videos. I have a question respect of the value to assign to bias. This value is a random value? I will appreciate your answer.
thank you for great explanation, i have a question, with this formula which generates for ( diff(L) / diff (W11)) is completely same for ( diff(L) / diff (W12)) i am i right? does both value gets same difference in weights while back propagation ( though W old value will be different
Same question. What I think, as we are finding out the new weights, the W11 and W12 for HL2, both should be different and should not be added, or I am missing something.
We are solving supervised learning problem that's why we have loss as actual-predicted , what in case of unsupervised where we don't have y actual how the loss is calculated and how the updation happen
Isnt the dL/dw2-11 independent of dL/dw2-12? At 12:21 why is dL/dw2-11 those two terms added up? dL/dw2-11 is the first line of additions, and dL/dw2-12 is the second line of additions.
yeah I did understand chain rule but being a fresher please provide some easy to study articles on chain rule so that i can increase my understanding before proceeding further.
Excellent video, I'm new in the field, could someone explain me how the O's are obtained. Are that O's the result of each neuron computation? are the O's numbers equations?
Great Video and a Great initiative sir from 12:07 if we use same method to calculate dL/dW12^2 it will be the same as dL/dW11^2. is this the correct way or am I getting it wrong thank you!
"Why" back propagation works in learning weights of the neural networks? What is the intuition behind using back propagation to update the weights? I know that we are trying to make corrections w.r.t the predicted value if the predicted value has some errors when compared to the actual value.
so depending on output predicted and output expected, we derive our loss function or cost function. By any mean if we can minimize overall loss of network our predicted output and expected output will reach closer which we want. Now think what we have in our hands to tweak so that loss can be minimized: 1. model hyper parameters (learning rate, no of layers, units in layer), 2. weight and bias for units across all layers. For option 2, we use back propagation where we take partial derivative of loss function w.r.t to unit weight and adjust that in unit weight. When we say partial derivative on loss function that actually mean drawing gradient(literal meaning slope in multidimensional plane), that gradient could be in upward direction or downward direction, so if gradient is negative then we are walking in downward direction which mean we are minimizing the total loss per loss function.
@@shashishankar1352 Thanks for your reply. BGD/SGD is used to solve the optimization problem at hand and back propagation is a technique that is used in sync with Gradient descent for tuning weights and bias. Whatever you explained are all facts that have been researched, documented and the same are being used in implementing solutions across various fields. I'm looking for a mathematical and geometrical explanation as well as proof on why back propagation works.
Hi Krish, can you pls let me know, if we are calculating the derivative of W2 11 weight then why we are adding derivative of W2 12 weight in that. ? pls clear
When we update the weights, book says to travel in opposite direction of gradient. But gradient on loss function curve could be negative or positive, so when gradient is negative why we should travel in opposite direction?
Sir , If to every single neuron in hidden layer we are giving same weights and features with bias then what is the use of multiple neurons in single layer?
Hello Sir, I think there is mistake in this video for backpropagation. Basically to find out (del L)/(del (w11^2)), we don't need the PLUS part. Since here O22 doesn't depend on w11^2. Please look into that. The PLUS part will be needed while calculating (del L)/(del (w11^1)), there O21 & O22 both depend on O11 and O11 depends on w11^1.
Yes brother there is mistake what is said is correct
Yes, This is correct. Thank you for pointing this out.
true that
You are correct concerning that, but I think he wanted to take derivative w.r.t O11 since it is present in both nodes of f21 and f22, so if we replace w11^2 in the equation by O11 the equation would be correct
it took me time to understand it but now I got the point thanks man but I can assure you that @krish naik is the first professor I have
I don't want to calulate Loss function to your videos and no need to propagate the video back and forward i.e you explained in such a easiest way I have ever seen in others. Keep doing more and looking forward to learn more from you. Thanks a ton.
Jabardast sir, i am watching ur videos after watching Andrew Ng's lecture of deep learning. I will say you simply explained even more easily. Superb.
Yes man, he's very good.
This is the most clear mathematical explanation I have ever seen till now.
ruclips.net/video/Ixl3nykKG9M/видео.html
It has been years since I had solved any mathematics question paper or looked at mathematics book. But the way you explained was damn good than Ph.D. holder professors at the University. I did not feel my away from mathematics at all. LoL- I do not understand my professors but understand you perfectly
you are no one but the perfect teacher,keep on adding playlist
Thank you for the perfect DL Playlist to learn, wanted to highlight a change to make it 100% useful (Already at 99.99%),
13:04 - For Every Epoch, the Loss Decreases adjusting according to the Global Minima.
But for negative slopes loss has to increase know to reach global maxima
@@vishnukce For negative slopes weights need to be increased instead of a loss
No one has ever explained like you did.hatts off!!
one of the best videos, I have seen in my life!!
great explanation of the chain rule in backpropagation.. all my doubts are cleared!!
thankss
This is simply yet Superbly explained. When I learnt earlier, it stopped at Back Propagation. Now, learnt what is in Backpropagation that makes the Weights updation in an appropriate way, i.e., Chain rule. Thanks much for giving clarity that is easy to understand. Superb.
You have saved my life, i owe you everything
Deep Learning Playlist concepts are very clear and anyone can understand easily. Really have to appreciate your efforts 👏🙏
Great explanation. I was looking for this clarity since long...
Your videos are really helping me to learn Machine learning as an actuarial student who is from a pure commerce/ finance background
first time i undestand very well by your explanation.
hats off to you sir,Your explanation is top level, THnak you so much for guiding us...
Nice informative video. It helped me in understanding the concept. But i think at end there is a mistake. You should not add the other path to calculate the derivative for W11^2. Addition should be done if we are calculating the derivative for O11.
w11^2(new) = (dl/dO31 * dO31/dO21 * dO21/dW11^2)
Yes deepak, I noticed the same thing. There's a mistake around 12:21. no addition is needed.
yes deepak you are correct. I also think the same.
Is that because we are calculating based on o3 and 03 depends on both output from second layer
simply one word "Great"
Of the two connections from f11 to the second hidden layer, w11^2 is affecting only f21 and not f22(as it affected by w21^2). So, dL/dw11^2 will only have one term instead of two.
Anyone, pls correct me if i am wrong.
I agree. i dont know why others didn't realized this same mistake!!!
i agree, i was looking for someone has the same remark :)
That's the point I am actually looking
Exactly, cause if I solve the derivative of two terms it results d/dw11^2 *L = d/dw11^2 *L + d/dw12^2 *L , which is wrong
Absolutely.
Thank you so much for all your efforts to give such an easy explanation🙏
Well Explained sir ! Before starting the deep learning, I have decided to start the learning from your videos. You explain in very simple way ...Anyone can understand from your video. Keep it up Sir :)
Bro, there is a correction needed in this video... watch out for last 3 mins and correct the mistake. Thanks for your efforts
your right
OP... Nice Teaching... Why don't we get teachers like u in every institute and college??
This is really cool. First time samjh aaya. Hats off Man.
This video explained everything I needed to know about backpropagation. Great video sir.
the last partial derivative of Loss we have calculated w.r.t. (w11^2) is that correct how we have shown there that it is dependent upon two paths one w11^2 and other w12^2 ......... Please make it clear i am confused about it ??????
I think this is wrong! Maybe he wanted to discuss about the w11^1? However, a forth term should be add in the sum. Idk
@@wakeupps yes, i think he got confused and it was w11^1
assume he is explaining about W11^1 and youll understand everything. From the diagram itself, you can see the connections and can clearly imagine which weights are dependent on each other .
Hope this helps
Yes, he should not have added the second term in the summation.
@@akrsrivastava Correct no second term needed for W11^2
Krish your awesome finally I understood the chain rule from you thanks Krish again
Awesome Mate. however, I think you got carried away for the second part to be added. read the comments below and correct, please. W12 may not need to be added. But it all makes sense. A very good explanation.
Same remark concerning W12, good job Krish Naik and thank you for your efforts
Hi Both, I also have same query
You have explained it very well. Thanks a lot!
Really appreciable the way you taught Chain rule...awesome..
great video especially you are giving the concept behind it, love it.. thank you for sharing with us.
Thank you so much for this! You are a good teacher
Amazing Videos...Only one word to say "Fan"
❤. God bless you, Sir.
i guess differentiation done at 11:26 is bit wrong, r u sure about? i mean why do we have to addan extra term of delta loss by delta w12
yes correct. It seems a mistake. addition part will come when we will calculate derivative of w11 for layer 1, not for derivative of w11 for layer 2.
Yes you are correct !!
@@debasispatra8368 but why we dont have to add for layer 2 and add to layer 1
@@bhavyaparikh6933 same question here....if you got it, can you explain.. I have just started deep learning.
@@debasispatra8368 Hi, can you just tell how initially weights are assign and how many hidden layers and no. of neurons on each layer should be there?
Excellent presentation Krish Sir .. You are great
I am going through tour videos. You are Rocking Bro.
Your*
Thank you Sir 🙏🙏🙏🙏♥️☺️♥️
I think this part dL/dw11^2 should be (dL/dO31 *dO31/O21 *dO21/dO11^2). If we are taking derivative of dL w.r.t w11^2 then,w12^2 doesn't come into play. So,in that case, dL/dO12^2= (dL/dO31 *dO31/O22 *dO22/dw12^2)
agree...dw11^2 should be (dL/dO31 *dO31/O21 *dO21/dO11^2). not extra afte addition
@ 10:28 - 11:22 krish do we need both the paths to get added . since w11 suffix 2 is not affected by lower path ie w12 suffix 2? please tell
The second part of the summation should not come in the picture as it will come only when we will be calculating (dL/dw12) with suffix as 2.
@@amit_sinha i think that is correct.
@@amit_sinha
Yes I have the same doubt!
Not required, its not correct as w11^2 is not affected by lower weights. The 1st part is correct and summation is required , when we are thinking about w11^1.
@@vishaldas6346 Yes!
Hi Krish,
Please upload videos on regular basis. I'm eagerly waiting for your videos.
Thanks in Advance
Uploaded please check the tutorial 7
@@krishnaik06 thank you..please keep posting more videos..I'm really waiting to watch your videos..really liked your way of explanation
Very very good explanation..very much understandable. Can I know how many days ur planning to complete this entire playlist?
love you sir, love ur effort. love from Bangladesh.
Loved it man... Great effort in explaining the maths behind it and chain rule. Pls make a video on its implementation soon. as usual great work.. Looking forward for the videos. Cheers
Hello Sunny, I myself have stitched an absolutely brilliant repository explaining all the implementation details behind an ANN. See this: github.com/jalotra/Neural_Network_From_Scratch
@@shivamjalotra7919 Great effort. Starred it. ⭐👍🏼
@@kshitijzutshi try to implement it yourself from scratch. See george hotz twitch stream for this.
@@shivamjalotra7919 Any recommendation for understanding image segmentation problem using CNN? resources?
Thanks a lot for the videos it helped me a lot
Hey Krish, god explanation
I think there is one correction. In the end, you explained for w11^2, what I feel is, it is for w11^1.
Thanks ! That was really awesome.
for the dL/w11^3 it should be dL/w11^3 = (dL/dO31 * dO31/dO31(before activation) * dO31(before activation)/dW11^3) right?
His sir i think there is something wrong wrong because the w11 to the suffix 2 is not impacted with the w12 to the suffix 2..! But this playlist is really helpfull to me thankyou sir...:)
Hi Kasim, I am also having same query
Just awsome explanation of gradient descent.
Im able to understand the concepts you are explaining, but I dont know that from where do we get values for weights in forward propgation.Could you brief about that once if possible.
You are too Good Krish , nice Data science content
Your teaching is great sir. But can we get some video also about how we will apply these practically in python?
Hey Krish, your way of explanation is good.
I think there is one correction. In the end, you explained for w11^2, what I feel is, it is for w11^1. It would be really helpful if you correct it because many are getting confused with it.
I think the same.. But great method of teaching.. there is no doubting that
sir i think one thing you are doing is worng.
as w^(3)11 impacts O(31) , here is one activation part.
so the dL/dw^(3)11 = dL/dO(31) . d0(31)/df1 . df1/dw^(3)11
I might be wrong, can you please clear my query ?
thank you sir, you explain very good keep it up.
Great stuff for free. Kudos to you and your channel
clearly understood very much appreciated for your effort :)
Thank you so much for all your videos. I have a question respect of the value to assign to bias. This value is a random value? I will appreciate your answer.
thank you for great explanation,
i have a question, with this formula which generates for ( diff(L) / diff (W11)) is completely same for ( diff(L) / diff (W12))
i am i right? does both value gets same difference in weights while back propagation ( though W old value will be different
Same question.
What I think, as we are finding out the new weights, the W11 and W12 for HL2, both should be different and should not be added, or I am missing something.
@@SunnyKumar-tj2cy Yeah, Both should not be added as they are diff...
Yes i have same question too!
@@abhinaspadhi8351 its wrong
Great job! Does the last derivative need the second part? I do not get it.
d(O22) will also be differentiated but with respect to w11, thus it will come out to be zero. Hence take it or not, result will be the same
here we used optimizer to update the weight slope is dl/dw so w here is w_old or something else.
Can you please do a Live Q&A session !? Great video... Thank you
Let me upload some more videos, then I will do a Live Q&A session.
Nice and requested to please add some videos on optimizer...
Great way to explain man.... keep on going
We are solving supervised learning problem that's why we have loss as actual-predicted , what in case of unsupervised where we don't have y actual how the loss is calculated and how the updation happen
I don't think there will be back propogation in unsupervised learning!
Could you please recheck the video at around 11:00, W11 weight updation should be independent of W12.
Isnt the dL/dw2-11 independent of dL/dw2-12? At 12:21 why is dL/dw2-11 those two terms added up? dL/dw2-11 is the first line of additions, and dL/dw2-12 is the second line of additions.
yeah I did understand chain rule but being a fresher please provide some easy to study articles on chain rule so that i can increase my understanding before proceeding further.
Thank you sir.
so insightful @krish
Excellent video, I'm new in the field, could someone explain me how the O's are obtained. Are that O's the result of each neuron computation? are the O's numbers equations?
krish sir, is it w12^2 is depends on w11^2 then only we can do differentiation. w12^2 is going one way and w11^2 is going another way.
Nice one thnks a lot!
Sir, O31 is also impacted by weight W11(3) ryt? why we are not taking that derivative in chain rule?
In the step where dL/dw[2]11 was shown as addition of two separate chain rule outputs, should it not be dL/dw[2]1 ?
Brilliant explanation!
Sir, I have a doubt , how will we calculate del(o31)/del(o21) , both are functions
Great Video and a Great initiative sir
from 12:07 if we use same method to calculate dL/dW12^2 it will be the same as dL/dW11^2.
is this the correct way or am I getting it wrong
thank you!
Skip to 3:50 If you've watched the previous videos
how do you take the derivative of d(O31)/dO21? what kind of equations are those?
Thanks Krish...
"Why" back propagation works in learning weights of the neural networks? What is the intuition behind using back propagation to update the weights? I know that we are trying to make corrections w.r.t the predicted value if the predicted value has some errors when compared to the actual value.
so depending on output predicted and output expected, we derive our loss function or cost function. By any mean if we can minimize overall loss of network our predicted output and expected output will reach closer which we want. Now think what we have in our hands to tweak so that loss can be minimized: 1. model hyper parameters (learning rate, no of layers, units in layer), 2. weight and bias for units across all layers. For option 2, we use back propagation where we take partial derivative of loss function w.r.t to unit weight and adjust that in unit weight. When we say partial derivative on loss function that actually mean drawing gradient(literal meaning slope in multidimensional plane), that gradient could be in upward direction or downward direction, so if gradient is negative then we are walking in downward direction which mean we are minimizing the total loss per loss function.
@@shashishankar1352 Thanks for your reply. BGD/SGD is used to solve the optimization problem at hand and back propagation is a technique that is used in sync with Gradient descent for tuning weights and bias. Whatever you explained are all facts that have been researched, documented and the same are being used in implementing solutions across various fields. I'm looking for a mathematical and geometrical explanation as well as proof on why back propagation works.
Best video on back proportional on internet
for calculating the loss function wrt W112 why do you also consider the other branch leading to the output ?? Kindly reply
it's mentioned clearly that it's wrt only W112 - the reason I'm asking this question
thank you ser
How do we know that we reached to a global mimima and we don’t need to update weights?
11:17 are you sure we have to sum them? It doesn't seems like the the two sides are equal when we "cancel" the chain
Nice 👍👏🥰
so helpful video :)
thanks
Really helpful for me.
We have to store the output and weights of each neuron and then use them to update.
Hi sir, Sorry to say you that which degree you have completed,you are awesome!
finally i understand it
Hi Krish, can you pls let me know, if we are calculating the derivative of W2 11 weight then why we are adding derivative of W2 12 weight in that. ? pls clear
When we update the weights, book says to travel in opposite direction of gradient. But gradient on loss function curve could be negative or positive, so when gradient is negative why we should travel in opposite direction?
Hi Shankar ! Can I contact you through email to learn about neural networks.If you give me the email,I mail you...
Sir , If to every single neuron in hidden layer we are giving same weights and features with bias then what is the use of multiple neurons in single layer?
great effort...