This is the best video on YT that I know of, that explains back propagation and gradient descent clearly, I've tried so many but this one is by far the best. Thanks for putting this together.
I have seen so many yt videos these days about backprop & gradient descent, you r the clearest among them, including a video with millions of watch. This video deserves more exposure. Thank you!
I am doing my bachelor in insurance mathematics, and one of my tasks were to model a forward neural network. Had no clue what it was. Watched 50 minutes of this guy and now I understand everything. Really great videos!
Very clear and easiest explanation. Not only did you explain backward propagation in easy way but you also clarified a lot of other concepts as well. Thanks and a lot of love ❤❤❤
There are a lot of nice looking videos on bp, but this one finally makes it clear that there is not so much dependency on all previous neurons rather than on last layer only. It was intimidating and overwhelming to think that I have to keep track of all neurons and their derivatives, but now it is clear that i can do everything one step at a time. I might sound chaotic and incoherent, but I'm just so excited to finally find a video not too simple and not too heavy on math notation, yet still it makes things clear
This is the very best video on explaining Back Propagation! It is very clear and well-designed for anyone needing to learn more about AI. I look forward to seeing other videos from Bevan.
I am completely amazed!! The clarity of explanation is at the highest level possible! Thank you very much, Sir, for this video and for all the efforts you put to make it so clear! Such a great talent to explain complex ideas in a clear and concise manner is indeed very rarely seen!
Just subbed to your channel because of the extremely clear explanations. Backprop has always been a sticking point for me due mostly due to the fact no one else is willing to get down from their 'math jargon' throne and actually explain the variables and functions in human language, You, Sir, are a gem and deserve all kudos you can get. Years ago I wrote a couple ANN programs with BP but I didn't understand it, I just wrote the calculations. Now, I can't wait to try it out again with a new understanding of the subject. Thank you again.
The best video I've seen so far. Such a clear and concise explanation. I finally understood in a smoothy way the different concepts related to the Back and forward propagation. I'm grateful
I had to pause the video at the point you mentioned the chain rule and go back to learn calculus, I took Krista Kings's calc class on Udemy, I am finally back to understand these concepts! 1 month later :)
You are the RUclipsr I have met who can explain all the specific calculation processes clearly and patiently. I appreciate you creating this video. It helps a lot. I wonder if you can make a video about Collaborative filtering?
amazing video. well explained and easy to understand. by far the best thing I have found for now that explains everything with good, supportive visuals
Man, this is at least the 15th video on the topic I watch, including several books, related to the back propagation, and this is the best one. All previous videos just skip a lot of explanation, focusing on how this backpropagation is important and crucial and what it allows to do, instead of doing the step-by-step overview. This video contains zero bs, and only has clear explanations, thank you
I really appreciate this video!! Believe me, I have been looking in books and in other videos, but this is the only one that tell the entire story on a very clear way (besides Stack Quest channel)! thanks a lot!! god bless you!
Your calculations(values) for W4 are incorrect so the updated value of W4 is incorrect. The correct updated value should be approximately 37.16. Your value is 49.5. Apart from that, thanks so much for your explanations. First time in years that I have fully understood the step by step logic of this operation. Much appreciated.
I was reading a book about neural networks, so in one of the last steps i have a great question about how a number was calculated, so i got frustrated, the book had a error, and i could find it thanks to this video, thanks a lot, very good explanation
best explanation over the internet..........could u please make videos on the different activation functions that u mentioned(tanh and the ReLU)..........it would be really nice of you !!
Every learning channels should follow you. Your learning method is amazing sir, also the last questions are actually in my mind but you cleared it.. Thank alot 🌼
I had gotten confused on the notion of "with respect to x" which after some studies means, when you take the derivative of a multi-variable function, you only differentiate the x and keep y exactly the same. God this learning it taking forever! :/
Hello Bevan Thank you for your excellent videos on neural networks. I have a question pertaining to this video covering Back Propagation. At about 14:30 you present the equation for determining the updated weight, W7. You are subtracting the product of η and the partial derivative of the Cost (Error) Function with respect to W7. However, this product does not yield a delta W7, i.e., a change in W7. It would seem that the result of this product is more like a delta of the Cost Function, not W7, and it is not mathematically consistent to adjust W7 by a change in the Cost Function. Rather we should adjust W7 by a small change in W7. Put another way, if these quantities had physical units, the equation would not be consistent in units. From this perspective, It would be more consistent to use the reciprocal of the partial derivative shown. I’m unsure if this would yield the same results. Can you explain how using the derivative as shown to get the change in W7 (or indeed in any of the weights) is mathematically consistent?
So what is the clever part of back prop? Why does it have a special name and it isn't just called "gradient estimation"? How does it save time? It looks like it just calculates all derivatives one by one
it is the main reason why we can train neural nets. The idea in training neural nets is to obtain the weights and biases throughout the network that will give us good predictions. The gradients you speak of get propagated back through the network in order to update the weights to be more accurate each time we add in more training data
Believe you used an learning rate of 0.001 not 0.01. Or you did use an learing rate of 0.01 and are having an error in the updated w7 as this is 12.42 and not 12.04.
Hi, thanks for the video! This question might be a little stupid, but aren´t the weight updates in 24:27 wrongly computated? shouldnt the first value be 0,29303 rather than 193? thanks for an answer!
Ugh, I am slowly getting it. First you take the derivative of the cost function with respect to Ypred, which is 2 times the difference between Ypred and Yact and then you multiply that by the partial derivative of Yppred with respect to w7. which is just g1. Then you multiply them together to obtain the partial derivative of the cost function with respect to w7. Then you take that partial derivative multiply that by a learnign rate and then subtract that from the original value of w7, this then becomes the new value of w7. I am still confused on the next part, calculating the partial derivatives of the hidden layers, but hey that is some progress so far, right? :/
i just wondering in the last part when i try to calculate partial derivative of w4 the result i got is -3711 but in the video it is -4947. then i make sure so i changed the last equation part to x1 (60) and it gives me the same result like in the video which -2783, so im not sure if i miss something since he didnt write the calculation from w4
In gradient descent we want to tweak the weights/biases until we obtain a minimum error in our cost function. So for that we need to compute the negative of the gradient of the cost function, multiply it by a learning rate and add it to the previous value. This negative means we are moving downhill in the cost function (so to speak)
Hi Bevan, I plugged your original weights and biases and got 24.95, which is correct, using inputs 60, 80, 5. I then entered all your modified values at frame 24:39 and got 42.19. I was hoping to get very close to ~82. Are you sure that you applied the local minimum value during gradient decent?
It could be a mistake on my part. I unfortunately don't have time now to go back and check. However, the important thing is that you understand how it all works. Cheers
@@bevansmithdatascience9580 I think I need to re-forward pass the features back into the NN with the adjusted values and recalculate the cost function until it reaches an acceptable local minimum. I’ll give that a try.
@@bevansmithdatascience9580 I made a bad question what I meant is what happened to dellZ1 , why does it disapper? why dellg1/dellz1 equals to only the derivative of the sigmoid function
This is the best video on YT that I know of, that explains back propagation and gradient descent clearly, I've tried so many but this one is by far the best. Thanks for putting this together.
I have seen so many yt videos these days about backprop & gradient descent, you r the clearest among them, including a video with millions of watch. This video deserves more exposure. Thank you!
After 2 years since publishing, your video is still a gem 💥
I am doing my bachelor in insurance mathematics, and one of my tasks were to model a forward neural network. Had no clue what it was. Watched 50 minutes of this guy and now I understand everything. Really great videos!
Way more better than all these videos with fancy graphics. Keep it simple, that's the way to understand complex things
Genius! Better than any professor in my school. Most helpful lecture I have ever found. Thanks a lot!!!!!!!
Very clear and easiest explanation. Not only did you explain backward propagation in easy way but you also clarified a lot of other concepts as well. Thanks and a lot of love ❤❤❤
Glad it was helpful Muhammed! Please do comment on what other concepts you were helped on. Thanks
You really have a talent for explaining this difficult subject cleary so that it makes sense and links up to the intuitive notions.
There are a lot of nice looking videos on bp, but this one finally makes it clear that there is not so much dependency on all previous neurons rather than on last layer only. It was intimidating and overwhelming to think that I have to keep track of all neurons and their derivatives, but now it is clear that i can do everything one step at a time. I might sound chaotic and incoherent, but I'm just so excited to finally find a video not too simple and not too heavy on math notation, yet still it makes things clear
One of the best videos covering the whole scenario with an example not just part of the process
This is one the clearest explanation on back propagation I’ve come across. Thanks!
Finally a very well done job about back propagation
This is the very best video on explaining Back Propagation! It is very clear and well-designed for anyone needing to learn more about AI. I look forward to seeing other videos from Bevan.
I am completely amazed!! The clarity of explanation is at the highest level possible! Thank you very much, Sir, for this video and for all the efforts you put to make it so clear! Such a great talent to explain complex ideas in a clear and concise manner is indeed very rarely seen!
Just subbed to your channel because of the extremely clear explanations. Backprop has always been a sticking point for me due mostly due to the fact no one else is willing to get down from their 'math jargon' throne and actually explain the variables and functions in human language, You, Sir, are a gem and deserve all kudos you can get. Years ago I wrote a couple ANN programs with BP but I didn't understand it, I just wrote the calculations. Now, I can't wait to try it out again with a new understanding of the subject. Thank you again.
_What a great teaching! The way you have explained all the nuts and volts, It's just amazing._
By far this is the best explanation. Clear, precise, detailed instructions. Well good and thank you so much 🙏
This is the best explanation of back propagation I ever came across.
You have got the skill to explain ML to even an 8-year-old.
Glad it helps Faisal :)
THE BEST VIDEO FOR UNDERSTANDING BACK PROPAGATION!!!! Thank you sir
Best explanation of gradient descent available. Thank you!
The best explanation on back propagation. Many thanks!
The best video I've seen so far. Such a clear and concise explanation. I finally understood in a smoothy way the different concepts related to the Back and forward propagation. I'm grateful
Best video I have found for BP! Thanks for all your efforts.
Amazing video. I was really confused by other videos but yours really explained every bit of it simply. thanks!
Thank you so much , Sir.
Best explanation I have seen on this platform .
I had to pause the video at the point you mentioned the chain rule and go back to learn calculus, I took Krista Kings's calc class on Udemy, I am finally back to understand these concepts! 1 month later :)
EXCELLENT!!!!!!!!!!!!!!!!JOB WELL DONE!!!!!!!!!!!!!!!!!!!!!!!!!!!!Wish you would make a video on batch gradient descent.
Just Beautiful !
Your math notation combined with your skill of teaching just made it so simple! Forever indebted. Thank you so much
Please teach us more. It was a great explanation, and you made this too easy for us. Thanks lot.
You are the RUclipsr I have met who can explain all the specific calculation processes clearly and patiently. I appreciate you creating this video. It helps a lot. I wonder if you can make a video about Collaborative filtering?
amazing video. well explained and easy to understand. by far the best thing I have found for now that explains everything with good, supportive visuals
Man, this is at least the 15th video on the topic I watch, including several books, related to the back propagation, and this is the best one. All previous videos just skip a lot of explanation, focusing on how this backpropagation is important and crucial and what it allows to do, instead of doing the step-by-step overview. This video contains zero bs, and only has clear explanations, thank you
thanks so much for the clarity. helps tremendously! lvoe this.
The best and simplest explanation ever
you are an amazing teacher, thank you for taking the time to create and to share your knowledge with us. I am grateful,
I really appreciate this video!! Believe me, I have been looking in books and in other videos, but this is the only one that tell the entire story on a very clear way (besides Stack Quest channel)! thanks a lot!! god bless you!
well clarifying, not too long not too short, just enough! really thanks!
Your calculations(values) for W4 are incorrect so the updated value of W4 is incorrect. The correct updated value should be approximately 37.16. Your value is 49.5.
Apart from that, thanks so much for your explanations. First time in years that I have fully understood the step by step logic of this operation. Much appreciated.
That's such a good video. I had these in a lecture and don't understand anything. That was really a relief for me. Thank you for that!
Great presentation. Good job 👍
This channel is so underrated...
My gosh... The clarity is amazing... Thanks Bevan
Thanks a lot for such a video. Simplest, easy, and thorough explanation for both beginners as well as advanced learners.
I'm very glad to hear that it helps Shobha. Good luck!
The best i have seen till date .. superb
The best and simplest explanation ever. Thanks man :)
Good to hear that. Good luck!
this is the best explanation i found, thanks a lot
Very simple and clear explanation ever
I was reading a book about neural networks, so in one of the last steps i have a great question about how a number was calculated, so i got frustrated, the book had a error, and i could find it thanks to this video, thanks a lot, very good explanation
Thank you so much. it was so simple
just outstanding - re watched it and it made it so clear!
This is amazing, so clear and easy to understand!
Awesome, clearly explained in simple way👍
best way of teaching.Thanks a lot
Excellent explanation Thank you ❤
Finally i understood something on this topic
The best explanation ever
Thank you so much for such an amazing explanation, the best I ever saw before.
Thanks very much, God bless you.
Amazing. I finally learned it. Thank you so much.
Thanks a lot! Really helpful!
Sr awesome job explaining this amazing topic
best explanation over the internet..........could u please make videos on the different activation functions that u mentioned(tanh and the ReLU)..........it would be really nice of you !!
Great idea. I hope to do that in the future
God bless you thank you ,you saved my life !!!!!
Every learning channels should follow you. Your learning method is amazing sir, also the last questions are actually in my mind but you cleared it.. Thank alot 🌼
I'm very glad you find it helpful :)
Absolutely enjoyed your explanation. Good job sir.
Mister, you have saved my life lol, thank you!!!
Excellent SIr
you are great .. thanks for such amazing videos
Glad to help Prashant :)
Your explanation is fantastic!! Thanks
You are welcome!
Thanks a lot .
well done!
YOU ARE REALLY THE BEST ONE 🤗
Finally! Got It! :) Thank You Sir very much
Best of the best. Thank you 🙏
Thank you very much for the video!
great!! thank you
I had gotten confused on the notion of "with respect to x" which after some studies means, when you take the derivative of a multi-variable function, you only differentiate the x and keep y exactly the same. God this learning it taking forever! :/
Awesome
illegally underrated
Hello Bevan
Thank you for your excellent videos on neural networks.
I have a question pertaining to this video covering Back Propagation. At about 14:30 you present the equation for determining the updated weight, W7. You are subtracting the product of η and the partial derivative of the Cost (Error) Function with respect to W7. However, this product does not yield a delta W7, i.e., a change in W7. It would seem that the result of this product is more like a delta of the Cost Function, not W7, and it is not mathematically consistent to adjust W7 by a change in the Cost Function. Rather we should adjust W7 by a small change in W7. Put another way, if these quantities had physical units, the equation would not be consistent in units. From this perspective, It would be more consistent to use the reciprocal of the partial derivative shown. I’m unsure if this would yield the same results. Can you explain how using the derivative as shown to get the change in W7 (or indeed in any of the weights) is mathematically consistent?
Great content! Thank you!
So what is the clever part of back prop? Why does it have a special name and it isn't just called "gradient estimation"? How does it save time? It looks like it just calculates all derivatives one by one
it is the main reason why we can train neural nets. The idea in training neural nets is to obtain the weights and biases throughout the network that will give us good predictions. The gradients you speak of get propagated back through the network in order to update the weights to be more accurate each time we add in more training data
Best
Atlast i understood 😌
Believe you used an learning rate of 0.001 not 0.01. Or you did use an learing rate of 0.01 and are having an error in the updated w7 as this is 12.42 and not 12.04.
22:53 Why Z1 is equal to -0,5?
Did I miss updating the biases? I think you only showed the partial derivative chain for weights. Not biases.
Hi, thanks for the video! This question might be a little stupid, but aren´t the weight updates in 24:27 wrongly computated? shouldnt the first value be 0,29303 rather than 193?
thanks for an answer!
Thanks for the message. It could be that I have made a mistake. But I hope that you can understand the principle of how to do it. Thanks
It is correctly mentioned. You may confused 19,303 with 19.303.Best
Ugh, I am slowly getting it. First you take the derivative of the cost function with respect to Ypred, which is 2 times the difference between Ypred and Yact and then you multiply that by the partial derivative of Yppred with respect to w7. which is just g1. Then you multiply them together to obtain the partial derivative of the cost function with respect to w7. Then you take that partial derivative multiply that by a learnign rate and then subtract that from the original value of w7, this then becomes the new value of w7.
I am still confused on the next part, calculating the partial derivatives of the hidden layers, but hey that is some progress so far, right? :/
And how to calculate the first partial derivative?
Great explanation thanks. But i think 12 - 0.01 * 42 .2 is 11.578 and not 12.04. @15:37 by the way amazing job well concept explanations 🙏
watch out when multiplying two minus numbers.
haha South African accent. baie dankie Bevan!
lekker bru
i just wondering in the last part when i try to calculate partial derivative of w4 the result i got is -3711 but in the video it is -4947. then i make sure so i changed the last equation part to x1 (60) and it gives me the same result like in the video which -2783, so im not sure if i miss something since he didnt write the calculation from w4
You are right. He has an error somewhere in his calculations.
So you do a new forward pass after each X back propagation? X1 forward , back...X2 forward, back...X3 forward, back.
Please check out my video on What is an epoch? That should clear things up. If not, ask again.
Why does weight updating use a minus sign, instead of a plus sign? 24:34
In gradient descent we want to tweak the weights/biases until we obtain a minimum error in our cost function. So for that we need to compute the negative of the gradient of the cost function, multiply it by a learning rate and add it to the previous value. This negative means we are moving downhill in the cost function (so to speak)
How to update b1 , I don’t know how to update it
Hi Bevan, I plugged your original weights and biases and got 24.95, which is correct, using inputs 60, 80, 5. I then entered all your modified values at frame 24:39 and got 42.19. I was hoping to get very close to ~82. Are you sure that you applied the local minimum value during gradient decent?
It could be a mistake on my part. I unfortunately don't have time now to go back and check. However, the important thing is that you understand how it all works. Cheers
@@bevansmithdatascience9580 I think I need to re-forward pass the features back into the NN with the adjusted values and recalculate the cost function until it reaches an acceptable local minimum. I’ll give that a try.
how does b1 and b2 calculate ?
g1/z1 okey dz1 is the derivative of the sigmoid function but what happens with de dg1 in the numerator
Please check 20:50 Let me know if that helps?
@@bevansmithdatascience9580 I made a bad question what I meant is what happened to dellZ1 , why does it disapper? why dellg1/dellz1 equals to only the derivative of the sigmoid function
besttt
😍😍