This guy is literally the most knowledgeable and knows the art and magic of how to teach. He is just like MIT Guy Gilbert Strang. Even better, because he knows where students trap points. The positive atmosphere , crystal clear explanation, and a great voice. The best course NPTEL
This teacher is best of all, I spent 6 months after all these minute explainations but couldnt find it in any single place. Please make his machine learning lectures online too
Perfect explication of gradient descent. First with mathematical proof and then the associated code to prove it with graphical illustrations. Thank you
In code, error function is used to stop the algorithm. In the algorithm we have to input some threshold value, and if the difference between the successive error values becomes lesser than that threshold, the algorithm will be instructed to stop as the error gets converged.
I have a question: As we know that gradient descent is a nice algorithm for fastest convergence. But many a time this algorithm oscillates and also it needs a quite a number of iterations to reach at minima. Also many of time it stuck towards local minima then at global minima. So why don't we try hybrid algorithm for finding global minima and also at less iteration cost. I can suggest Genetic algorithm (for faster convergence even towards global minima) at first and then for gradient descent. What are your concern about it??
give me the feedback that we dont need basic stuff, i am gonna ignore that anyways!!!! , WOW moment for me as i was expecting a teacher just like this one.this one's a gem
real Gems of iits are those prof who know how to teach students so that he never have any confusion afterwards. rest are only researchers, they also do have knowledge but they aren't able to express in a proper manner.
The vector is still u and taking a transpose does not change the direction of the vector. Why did we take the transpose? To facilitate the matrix multiplication between vector u and ΔL(θ). That is all. The direction of u remains the same.
In the code shown in this lecture, eta value is chosen to be 1.0. However, in the earlier part, we said we can ignore the higher order terms of Taylor expansion because eta is usually very small and the higher order terms are negligible. But here higher-order terms of eta are not negligible. Why does our approach still work? In this kind of situtation, can we get a better convergence by including the higher order derivatives as well?
Gradient here refers to the size of the step itself. So, as the point traverses over the error surfaces and moves closer to the optimal point/surface, the size of the step will become smaller. The opposite direction only implies that the step size will decrease as we get closer.
look, it basically says that you have to go against the gradient of the slope. Imagine a parabola with its minima at the origin. If you are in the second quadrant, you have to move in the positive x direction. But at that point the slope is negative. Simillarly, when you are in the first quadrant, the slope there is positive, but you have to move towards the negative x-direction to reach the minima.
@@ruturajjadhav8905 step size is a dynamically estimated value, by which the algorithm moves over the error surface. gradient descent tries to minimize error by finding the optimal point which should be the lowest point on the surface
I understood the concept of taking a step opposite to the gradient. But what is the direction of the gradient ? I am not able to visualize that. Anyone?
@@shubhamkumar-nw1ui increase. Change can be either incremental or decremental. But gradient specifically means direction in which the fucntion value increases. It is same as the direction one follows while climbing a hill.
The way this guy maintains the positive atmosphere while teaching is very rare for teachers. Hope we get more courses from him
This guy is literally the most knowledgeable and knows the art and magic of how to teach. He is just like MIT Guy Gilbert Strang. Even better, because he knows where students trap points.
The positive atmosphere , crystal clear explanation, and a great voice.
The best course NPTEL
Top 20% might get pissed off but I have a philosophy that I teach for the bottom 20% students of the class.
Love you sir..
This is by far the best explanation of gradient descent I have ever come across! Thank you so much for this :D
This teacher is best of all, I spent 6 months after all these minute explainations but couldnt find it in any single place.
Please make his machine learning lectures online too
ruclips.net/p/PLH-xYrxjfO2VsvyQXfBvsQsufAzvlqdg9
@@thearindampal This is Deep learning series.
@@keshavsingh489 I am not sure if Prof. Khapra did a course on ML. This video is a part of DL course and the whole playlist is the one which I posted.
we need teachers like this in all best institutes of India, which is quite rare
Mitesh sir is one of the best teacher in nptel i have seen especially the way emphasis on the hidden maths is amazing
Perfect explication of gradient descent. First with mathematical proof and then the associated code to prove it with graphical illustrations. Thank you
jus loved to watch ur class ..............its amazing .....................the way u teach,,,,,,,
Best teacher touch very fundermetal to comprehensive best of luck
baap of all times
seriously man he is soo good in it
Excellent Lecture, Sir...the hessian and gradient explanation part was very helpful and made things more clear.
Thank you. All your lectures are great. Also teaching everything from basics helps to understand quickly.
Nice series of lectures about deep learning. Thank you so much!
Thank you so much sir. Best teaching .
Best teacher, i have never seen such a calm and positive teacher. 😍 we want more teachers like you sir.
I dont find any role of function error in last code implementation. As we already grad into x and y right?
In code, error function is used to stop the algorithm. In the algorithm we have to input some threshold value, and if the difference between the successive error values becomes lesser than that threshold, the algorithm will be instructed to stop as the error gets converged.
What an outstanding teaching skill with an amazing personality! I wish every university to be blessed with a teacher like him!
Sir please we want your ML series also
I have a question:
As we know that gradient descent is a nice algorithm for fastest convergence. But many a time this algorithm oscillates and also it needs a quite a number of iterations to reach at minima.
Also many of time it stuck towards local minima then at global minima.
So why don't we try hybrid algorithm for finding global minima and also at less iteration cost.
I can suggest Genetic algorithm (for faster convergence even towards global minima) at first and then for gradient descent.
What are your concern about it??
He teaches great. However, even after watching the video the math is incomprehensible for a poor math guy like me.
a very good explanation, Thank you Sir.
awesome explanation to one of the hardest part in ML
Thank you Dr. Khapra. Basics are as important as advanced concepts, I really enjoyed this video.
give me the feedback that we dont need basic stuff, i am gonna ignore that anyways!!!! , WOW moment for me as i was expecting a teacher just like this one.this one's a gem
It's very important for teachers to be sorted and calm and sir is real example of that!
real Gems of iits are those prof who know how to teach students so that he never have any confusion afterwards. rest are only researchers, they also do have knowledge but they aren't able to express in a proper manner.
at 27.48, moving in direction of gradient or opposite to direction of gradient ? Can anyone explain
The man the myth the legend Sir Mitesh Khapra. God of deep learning
Can you please tell more about how to chose the learning rate( eta ) here or you are treating that as a parameter itself?
super you are awesome
What textbook he mentioned at 9.04 ? hessian?
one heck of a lecture it is. Amazed ...
Best
One of the best explanations..Thank you very much sir..
What is U transpose, and angle b/w uT and gradient is 180 not u and gradient, please clarify sir
u = Δθ. It is just a notation for convenience.
The vector is still u and taking a transpose does not change the direction of the vector. Why did we take the transpose? To facilitate the matrix multiplication between vector u and ΔL(θ). That is all. The direction of u remains the same.
can someone tell me how we plot the error surface ?
Where to get notes of whole course
In the code shown in this lecture, eta value is chosen to be 1.0. However, in the earlier part, we said we can ignore the higher order terms of Taylor expansion because eta is usually very small and the higher order terms are negligible. But here higher-order terms of eta are not negligible. Why does our approach still work? In this kind of situtation, can we get a better convergence by including the higher order derivatives as well?
higher order terms are a dot product of eta_T grad(theta) evaluated as nth derivative, so the values in the derivatives will be smaller to be ignored.
Best teacher
At 14:46 ,beta is the angle between "u" and gradient L, instead of "u transpose"?
Yes. Thats what he corrected it to
OK , I understood the concept of taking a step opposite to the gradient. But what is the direction of the gradient ? I am not able to visualize that.
Gradient here refers to the size of the step itself. So, as the point traverses over the error surfaces and moves closer to the optimal point/surface, the size of the step will become smaller. The opposite direction only implies that the step size will decrease as we get closer.
look, it basically says that you have to go against the gradient of the slope. Imagine a parabola with its minima at the origin. If you are in the second quadrant, you have to move in the positive x direction. But at that point the slope is negative. Simillarly, when you are in the first quadrant, the slope there is positive, but you have to move towards the negative x-direction to reach the minima.
@@bharathraman2098 what is step size
@@ruturajjadhav8905 step size is a dynamically estimated value, by which the algorithm moves over the error surface. gradient descent tries to minimize error by finding the optimal point which should be the lowest point on the surface
@@bharathraman2098 what is step size the above video.?
I am not getting.
And how to visualise direction of the gradient?
it's amazing
I understood the concept of taking a step opposite to the gradient. But what is the direction of the gradient ? I am not able to visualize that. Anyone?
it tells the direction in which the increase in value of function is maximum.like in case of a function of one variable it is simply the tangent.
@@BRIJESHKUMAR-jn3lm Does gradient means direction in which "increase" is maximum or "change" is maximum ?
@@shubhamkumar-nw1ui increase.
Change can be either incremental or decremental. But gradient specifically means direction in which the fucntion value increases.
It is same as the direction one follows while climbing a hill.
how is loss function 1/2*(f(x)-y)^2 at 21:52 can some one help
It is taken for sake of simplicity, so that when you take derivative of f(x), the 2 gets cancelled out.
It should be without 1/2 as the number of points is 1 only for the mean squared error loss function
Have you uploaded ML lectures too?
Andrew Ng's version 2