Thank you a lot! When I considered n linearly independent directions firstly, I was so confused. But after watching your example for 2 directions, I can fully understand Powell`s Algorithm and extend to n directions by myself. Very clear demonstration within 6 minutes!
Thanks, we just learned this method in my optimization course. However, we started differently. Instead of choosing a point and 2 directions, our professor started with 2 points and one direction to find the first couple of extrema. This different perspective is great because it showed me where the true conjugate algorithm lies, and that the method can be started in a couple different ways.
This would probably be an alright explanation if I had an understanding of the algorithm already, which I don't. Having no knowledge, you immediately jump into how to do the algorithm without explaining anything, like at all. Also if you're going to use an example, you should always use a numerical example. There's a lot of unanswered questions. How do you choose h1? How do you choose h2? How do you choose X0? What's the definition of conjugate in this specific context? When you say "if it's not quadratic it might be a little off", how off? Surely in some cases it would be massively off for the context?
I am sorry this video disappointed you. Did you watch the videos prior to this one? I think you are missing some key background information that would make this video much more understandable. In order to keep the videos as short as possible, I don't think I could make each one a stand-alone video simply because there is a lot of background information, like conjugate gradients and 1D optimization, that is needed to fully understand the method. I recommend accessing the course through the course website where you can see all of the notes and videos organized, as well as get links to the latest versions of all of this stuff and get access to other learning resources. You will Powell's method is 8th video in Topic 8. empossible.net/academics/emp4301_5301/ Hope this helps!
This algorithm was simpler than I thought it would be. I was curious how it worked since it's by far the best for least squares exponential curve fitting for stocks.
Awesome video! I am currently writing a program that performs a best fit procedure on a composite model of peaks (lorentzian and Gaussian). I was wondering if you knew at the top of your head what a good method for this would be. It doesn’t sound like Powell’s would be very good here. I am currently using the sum of least square, but I find it to be a little slow for large composite models. I am utilizing the Lmfit library in python which has a list of all methods available to me. Thank you!
Sounds more like a curve fitting problem. Have you looked at nonlinear regression? Checkout Topic 5 at the course website. empossible.net/academics/emp4301_5301/
This was well explained. A couple of questions... is the choice of h1 and h2 basically arbitrary, or are there good rules of thumb for determining these directions. Same with h4 I suppose. Next, are these direction defined as vectors? Since we're not moving parallel to one axis and holding the others constant, the function evaluations (to determine the extreme point long the direction of interest) must be made by changing all of the independent variables by some amount. Do we define a new function that describes the transect and then find the extreme point of that by something like Golden Section search? Curious how this might be implemented. Cheers!
Without any apriori knowledge of where the extrema is, there is not much you can do to choose better h1 and h2. Just be sure they are different enough to resolve an accurate direction. Maybe that means 30 degrees between them or so? The directions are defined as vectors since they are directions. Given a starting point s and a direction vector h, you can calculate the testing point as p = s + alpha*h. In this case, alpha is a single scalar quantity to be optimized. I think that was your confusion. AT some point, I will make a video showing the implementation in MATLAB. I have a lot of content on my "to create" list. LOL Hope this helps!
@@empossible1577 Thx much. I was thinking about it this am and figured I could make the x & y coords a function of a parameter (alpha in your example) and then use that line to "crawl" along to find an extreme point with golden section. I'm teaching myself numerical methods and using python as the tool of choice. Content like yours is invaluable as I have no tutor (know any?) for this kinda thing. So I definitely want to say "thanks". I'm gonna try to cook up a powell implementation today. Coding really is the best way to learn something IMO. Best of luck with the "to create" list! Looking forward to more great content!
@@pipertripp For computation, I dislike everything about Python. Julia is easy to use and extremely fast. It is by far the superior language for computation. The main thing is that is has a much smaller user community than Python.
Thank you so much. If anybody reads this, could you explain to me why Wikipedia's definition of the algorithm is different? "The new displacement vector becomes a new search vector, and is added to the end of the search vector list. Meanwhile, the search vector which contributed most to the new direction, i.e. the one which was most successful is deleted from the search vector list." Here, in this video, direction h1 is deleted, when h3 is added, even though the vector connecting x0 and x1 is not the search vector "which contributed most to the new direction".
You will find a lot of differences in how people implement different methods. I think that is because technology has evolved passed the original contribution of the methods and it takes on the flavor of whoever is implementing it. This particular implementation comes from Steven Chapra, although I do not recall which edition. Implementation can even change between editions! www.amazon.com/Numerical-Methods-Engineers-Steven-Chapra/dp/007339792X/ref=sr_1_4?crid=SEVQK23D7PJM&keywords=numerical+methods+for+engineers+chapra&qid=1644248418&sprefix=chapra+numeric%2Caps%2C102&sr=8-4
@@empossible1577 Actually I just checked Powell's paper himself and he proceeds like you, getting rid of the first search vector in the list of search vectors to add the newly created (conjugate) vector at the end of the search vector list. Wikipedia seems to have a mistake there and the source provided over there can not be found.
@@McDave1312 Thank you for looking in to this! In other methods, I have found quite a few differences depending on where I am reading it. It sure makes the learning process frustrating! Keep up the good work!
Let me point you to the course website: empossible.net/academics/emp4301_5301/ The applications are covered in a previous Lecture...Lecture 8a. In general, it is used in optimization to find the minimum or maximum of a function. For example, suppose you are designing a rectangular antenna. You could optimize the performance of the antenna by performing Powell's method where x and y in Powel's method would represent the two dimensions of the antenna. That is just one example.
Thank you a lot! When I considered n linearly independent directions firstly, I was so confused. But after watching your example for 2 directions, I can fully understand Powell`s Algorithm and extend to n directions by myself. Very clear demonstration within 6 minutes!
Awesome to hear! Thank you!!
Thank You! Very clear and informative lecture.
Thanks, we just learned this method in my optimization course. However, we started differently. Instead of choosing a point and 2 directions, our professor started with 2 points and one direction to find the first couple of extrema. This different perspective is great because it showed me where the true conjugate algorithm lies, and that the method can be started in a couple different ways.
Yeah, I think in my course we used gradient of the function.
Thank You so much for explaining it so simply!
Thank you so much! Great video, love to see it!
Thanks man...you made this a lot easy
This would probably be an alright explanation if I had an understanding of the algorithm already, which I don't. Having no knowledge, you immediately jump into how to do the algorithm without explaining anything, like at all. Also if you're going to use an example, you should always use a numerical example. There's a lot of unanswered questions.
How do you choose h1? How do you choose h2? How do you choose X0? What's the definition of conjugate in this specific context? When you say "if it's not quadratic it might be a little off", how off? Surely in some cases it would be massively off for the context?
I am sorry this video disappointed you. Did you watch the videos prior to this one? I think you are missing some key background information that would make this video much more understandable. In order to keep the videos as short as possible, I don't think I could make each one a stand-alone video simply because there is a lot of background information, like conjugate gradients and 1D optimization, that is needed to fully understand the method.
I recommend accessing the course through the course website where you can see all of the notes and videos organized, as well as get links to the latest versions of all of this stuff and get access to other learning resources. You will Powell's method is 8th video in Topic 8.
empossible.net/academics/emp4301_5301/
Hope this helps!
This algorithm was simpler than I thought it would be. I was curious how it worked since it's by far the best for least squares exponential curve fitting for stocks.
Interesting. I am always looking for new and interesting examples for the class so that I am not just working with generic functions. Thank you!
@@empossible1577 I've been using it specifically to predict future revenue, earnings and distributions for companies. Glad I could be of serivce.
Awesome video! I am currently writing a program that performs a best fit procedure on a composite model of peaks (lorentzian and Gaussian). I was wondering if you knew at the top of your head what a good method for this would be. It doesn’t sound like Powell’s would be very good here. I am currently using the sum of least square, but I find it to be a little slow for large composite models. I am utilizing the Lmfit library in python which has a list of all methods available to me. Thank you!
Sounds more like a curve fitting problem. Have you looked at nonlinear regression? Checkout Topic 5 at the course website.
empossible.net/academics/emp4301_5301/
This was well explained. A couple of questions... is the choice of h1 and h2 basically arbitrary, or are there good rules of thumb for determining these directions. Same with h4 I suppose. Next, are these direction defined as vectors? Since we're not moving parallel to one axis and holding the others constant, the function evaluations (to determine the extreme point long the direction of interest) must be made by changing all of the independent variables by some amount. Do we define a new function that describes the transect and then find the extreme point of that by something like Golden Section search? Curious how this might be implemented. Cheers!
Without any apriori knowledge of where the extrema is, there is not much you can do to choose better h1 and h2. Just be sure they are different enough to resolve an accurate direction. Maybe that means 30 degrees between them or so? The directions are defined as vectors since they are directions. Given a starting point s and a direction vector h, you can calculate the testing point as p = s + alpha*h. In this case, alpha is a single scalar quantity to be optimized. I think that was your confusion. AT some point, I will make a video showing the implementation in MATLAB. I have a lot of content on my "to create" list. LOL Hope this helps!
@@empossible1577 Thx much. I was thinking about it this am and figured I could make the x & y coords a function of a parameter (alpha in your example) and then use that line to "crawl" along to find an extreme point with golden section. I'm teaching myself numerical methods and using python as the tool of choice. Content like yours is invaluable as I have no tutor (know any?) for this kinda thing. So I definitely want to say "thanks". I'm gonna try to cook up a powell implementation today. Coding really is the best way to learn something IMO. Best of luck with the "to create" list! Looking forward to more great content!
@@pipertripp Finding tutors gets more difficult as you get higher in the course level. Have you looked at Julia instead of Python?
@@empossible1577 ha, tell me about it! I have not looked at Julia. What do you like about it?
@@pipertripp For computation, I dislike everything about Python. Julia is easy to use and extremely fast. It is by far the superior language for computation. The main thing is that is has a much smaller user community than Python.
Thanks a lot
Thank you so much. If anybody reads this, could you explain to me why Wikipedia's definition of the algorithm is different?
"The new displacement vector becomes a new search vector, and is added to the end of the search vector list. Meanwhile, the search vector which contributed most to the new direction, i.e. the one which was most successful is deleted from the search vector list."
Here, in this video, direction h1 is deleted, when h3 is added, even though the vector connecting x0 and x1 is not the search vector "which contributed most to the new direction".
You will find a lot of differences in how people implement different methods. I think that is because technology has evolved passed the original contribution of the methods and it takes on the flavor of whoever is implementing it. This particular implementation comes from Steven Chapra, although I do not recall which edition. Implementation can even change between editions!
www.amazon.com/Numerical-Methods-Engineers-Steven-Chapra/dp/007339792X/ref=sr_1_4?crid=SEVQK23D7PJM&keywords=numerical+methods+for+engineers+chapra&qid=1644248418&sprefix=chapra+numeric%2Caps%2C102&sr=8-4
@@empossible1577
Actually I just checked Powell's paper himself and he proceeds like you, getting rid of the first search vector in the list of search vectors to add the newly created (conjugate) vector at the end of the search vector list. Wikipedia seems to have a mistake there and the source provided over there can not be found.
@@McDave1312 Thank you for looking in to this! In other methods, I have found quite a few differences depending on where I am reading it. It sure makes the learning process frustrating! Keep up the good work!
Did anyone know use case of this method? Thanks!
Let me point you to the course website:
empossible.net/academics/emp4301_5301/
The applications are covered in a previous Lecture...Lecture 8a. In general, it is used in optimization to find the minimum or maximum of a function. For example, suppose you are designing a rectangular antenna. You could optimize the performance of the antenna by performing Powell's method where x and y in Powel's method would represent the two dimensions of the antenna. That is just one example.
@@empossible1577 thanks for your information!
+1 subscriber for you
@@KangJangkrik :-)