How to differentiate with respect to a vector - part 2
HTML-код
- Опубликовано: 6 фев 2025
- This video provides a description of how to differentiate a scalar with respect to a vector, which provides the framework for the proof of the form of least squares estimators in matrix form. Check out ben-lambert.co... for course materials, and information regarding updates on each of the courses. Quite excitingly (for me at least), I am about to publish a whole series of new videos on Bayesian statistics on youtube. See here for information: ben-lambert.co... Accompanying this series, there will be a book: www.amazon.co....
Man you are the best complement to my 50k a year education! God bless you! May the force be with you!, etc.
Hi, thanks for your message. You are right, this particular method is relevant to symmetric matrices. However, when we differentiate the matrix expressions we are only dealing with either: pure scalars such as y'Xb or quadratic form terms like b'X'Xb. The former can be dealt with using the methods from part 1. In the quadratic form term b'X'Xb, the matrix X'X is symmetric, so there's no problem there. If you have any more questions please let me know. Best, Ben
No problem. Ok y'xb is a scalar. This means that its transpose is just the same as itself. The transpose of y'xb is (y'xb)' which is b'x'y. Hence -y'xb-b'x'y =-2y'xb. To continue from there we just differentiate wrt b yielding -2x'y. Hope that helps! Thanks, Ben
Fantastic
I am a student studying advanced finance and economics in UQ. It really helps!
I would like to point out that at the end Ben confused row and column vector, corrected himself but then confused them in the reverse order. I had to relisten to make sure I got it right so in case anyone else is wondering if they heard right.
You explained it very clear. Thanks!
I was just cross referencing this with a Matrix derivatives cheat sheet - The Learning Method says "Every subject has it's own language and fundamental and advanced concepts, and, if I'm not performing in a subject then I must be missing one or more fundamentals" - so that A is symmetric is a fundamental here (it was missing from the cheatsheet and thus a little confusing). Another fundamental is that the function that we are talking the derivative of is a scalar. Are there any others I am missing?
super helpful. Thank you so much !!!
insightful! especially if you view matrix A as a transformation (essentially a function)
Thanks you for your apply! I have another question...I can't understand why at the OLS estimator we take the derivative of -y'xb equals to -b'x'y. Also i see in many books that they take the sum of them( -2y'xb).why?they are different and then how we can continue from that point? without using your video?? :) Thanks you again very much!!!
Very usefull.But i want to ask you something.It works only when A is symmetric i think...how you use it later at the OLS problem?there it's not A symmetric maxtrix...
can someone explain how it went from 6:00 to 6:10. How did you know that matrix equals to 2AX?
Hi, this just comes from the normal matrix product of a (2 x 2) matrix with a (2 x 1) vector. Hope that helps! Ben
In order to understand, you could go backwards: just calculate the term 2AX, and you will see that you end up with a 2 by 1 vector from 6:00.
Very helpful
The final result is wrong on the first sight. Theoretically, it should be (A+A^T)x, NOT 2Ax. However, since A is symmetric, it is CORRECT
+Tiger Deng Hi, thanks for your comment. The matrix is symmetric, hence (A+A^T) = 2A. Sorry that this wasn't clear. Best, Ben
+Ben Lambert Yeah, Sorry, my bad, I should read it more carefully
hello Ben...thanks for getting back. I know how to expand the matrices....but how do you contract it?
Hi, there is no simple way to contract matrices. Here I have just seen it because I can imagine that this product is what would be formed if one did the matrix multiplication. Hope that helps! Ben
notation error: the lower left hand element should be labeled a_2,1
@Meadowlark thanks for bringing in the notation error I too had this confusion @Tom and why would we want to do that ? I mean have an element 1,2 =2,1 ? for simplification reason ?
It's not an error. He wanted both the elements to be the same and hence the labelings are kept same. On expansion, you get the middle term with 2a_12 as the coefficient.
Not an error, it's to make the next step easier to see.
@@yashakhandelwal5235 I think otherwise he couldn't get the desired result
He said, take a SYMMETRIC matrix A, i.e a_{2,1} = a_{1_2}
I love you..