Here is the code snippet, % Assume a quadratic function and ensure we get the first derivative. f = @(x) (x(1) -5).^2 + (x(2)+1).^2; % minimum points (5,-1) % find the gradient, the first derivative of the objective function. We keep 1st derivative w.r.t. x(1) and x(2) in separate columns gradf =@(x) [2*(x(1)-5);2*(x(2)+1)]; % Notice that gradient will be zero at point(5,-1) % Choose an initial guess or starting model point, iterations, and step size.; you can choose any guess and play with the initial guess, step size and iterations. intialGuess = [2;3]; iterations = 50; stepSize = 0.1; recordGuesses = [intialGuess]; nextGuess = intialGuess; for i = 1: iterations % we are going in a negative direction (Gradient Descent), so there will be -ve from the previous guess, not +ve nextGuess = nextGuess - stepSize*gradf(nextGuess); % record the guesses recordGuesses = [recordGuesses,nextGuess]; end display('Distance from the minimum') % This will tell how far we are from the original solution. i.e., minimum % point. Smaller the number better the convergence. display(norm([5;-1] - nextGuess)) figure; plot(recordGuesses(1,:),recordGuesses(2,:),'ro') hold on; plot(5,-1,'b^'); hold off; % NOTE: from the plot, it can be seen that as the gradient gets closer, the steps % between each solution also gets closer.
@@40NoNameFound-100-years-ago I don't know if it's a good reference, but this seems to have all the details about the methodology www.stat.cmu.edu/~ryantibs/convexopt-F18/scribes/Lecture_23.pdf
@@JoelRosenfeld Thank you. I appreciate your help. I was just reading about those two methods now. The projected gradient descent and the Frank Wolfe algorithm 😃👍
thank you , can you help me to solve my exercice ,it 's about ziggurat method how can i find the point (y2)&(x2) if i have only x(1) &(y1) with fuction =exp (-(x^2)/2) ,
Very helpful! I would now like to use this gradient in the form of a square instead of a line, would you know what I should change/ add to do this? Thank you!!
How can I represent data as function e.g I have ECG data (2014×1 double) in the other word( numeric matrix) data and I wanna represent it as function ??
At this point you just have a time series. Typical approaches to approximate an underlying function are to use polynomial approximations, B splines, and Fourier Series.
@@JoelRosenfeld thanks a lot, I have just amplitude and I ploted the signal without time but I can created time, could you give me more details about B spline....... take in considering I can't use python. More details please 🙏 and I have already gradient descent code but it's so long and I doesn't get it
If you want to approximate the gradient of an image, treating the image as samples of a function, then you can use any finite difference scheme you want for each partial derivative. I think what is usually done is to multiply by a vector like [-1 0 1]^T, but ultimately you could design any finite difference scheme to get an estimate.
I am having an issue in this field , I want to optimize an objective function of my interest using gradient based optimization techniques and facing some difficulties.If you could help me in this , I will be really greatul.
Very helpful video, intresting too, keep doing what you're doing
Thank you! I’m glad you liked it!
Here is the code snippet,
% Assume a quadratic function and ensure we get the first derivative.
f = @(x) (x(1) -5).^2 + (x(2)+1).^2; % minimum points (5,-1)
% find the gradient, the first derivative of the objective function. We keep 1st derivative w.r.t. x(1) and x(2) in separate columns
gradf =@(x) [2*(x(1)-5);2*(x(2)+1)]; % Notice that gradient will be zero at point(5,-1)
% Choose an initial guess or starting model point, iterations, and step size.; you can choose any guess and play with the initial guess, step size and iterations.
intialGuess = [2;3];
iterations = 50;
stepSize = 0.1;
recordGuesses = [intialGuess];
nextGuess = intialGuess;
for i = 1: iterations
% we are going in a negative direction (Gradient Descent), so there will be -ve from the previous guess, not +ve
nextGuess = nextGuess - stepSize*gradf(nextGuess);
% record the guesses
recordGuesses = [recordGuesses,nextGuess];
end
display('Distance from the minimum')
% This will tell how far we are from the original solution. i.e., minimum
% point. Smaller the number better the convergence.
display(norm([5;-1] - nextGuess))
figure;
plot(recordGuesses(1,:),recordGuesses(2,:),'ro')
hold on; plot(5,-1,'b^'); hold off;
% NOTE: from the plot, it can be seen that as the gradient gets closer, the steps
% between each solution also gets closer.
Thanks for such a great video.
How can we use constraints with this method?
It’s been a while, but I think you’d need to integrate in a projection to make sure that you stay within a constrained area
@@JoelRosenfeld could you recommend a good reference for that?
@@40NoNameFound-100-years-ago I don't know if it's a good reference, but this seems to have all the details about the methodology www.stat.cmu.edu/~ryantibs/convexopt-F18/scribes/Lecture_23.pdf
@@JoelRosenfeld Thank you.
I appreciate your help.
I was just reading about those two methods now. The projected gradient descent and the Frank Wolfe algorithm 😃👍
Awesome video!
Thank you! I’m glad you enjoyed it!
awesome, thanks for the video
You’re welcome!
thank you , can you help me to solve my exercice ,it 's about ziggurat method how can i find the point (y2)&(x2) if i have only x(1) &(y1) with fuction =exp (-(x^2)/2) ,
i want to know what model the keyboard is, would you be possible to let me know keyboard's model?
SteelSeries Apex 5 is the keyboard. Feels amazing.
@@JoelRosenfeld thankU
Very helpful! I would now like to use this gradient in the form of a square instead of a line, would you know what I should change/ add to do this? Thank you!!
How can I represent data as function e.g I have ECG data (2014×1 double) in the other word( numeric matrix) data
and I wanna represent it as function ??
At this point you just have a time series. Typical approaches to approximate an underlying function are to use polynomial approximations, B splines, and Fourier Series.
@@JoelRosenfeld thanks a lot, I have just amplitude and I ploted the signal without time but I can created time, could you give me more details about B spline....... take in considering I can't use python. More details please 🙏 and I have already gradient descent code but it's so long and I doesn't get it
@@arwasaed1025 I have a video talking about splines on my channel already. Just browse around. Should not be hard to find.
@@JoelRosenfeld okay, I really appreciate your efforts
Please provide code
Thanks sir for understanding this method 👍
Sure thing! I hope it helped!
can you find gradient of an image using three point formula ?
If you want to approximate the gradient of an image, treating the image as samples of a function, then you can use any finite difference scheme you want for each partial derivative. I think what is usually done is to multiply by a vector like [-1 0 1]^T, but ultimately you could design any finite difference scheme to get an estimate.
I am having an issue in this field , I want to optimize an objective function of my interest using gradient based optimization techniques and facing some difficulties.If you could help me in this , I will be really greatul.
What sort of objective function and what sort of data?
I have a problem How did you apply this method to the Traveling salesman problem?
hi, have you shared the codes you wrote in the video on github?