Derivative of a Matrix : Data Science Basics
HTML-код
- Опубликовано: 8 сен 2019
- What does it mean to take the derviative of a matrix?
---
Like, Subscribe, and Hit that Bell to get all the latest videos from ritvikmath ~
---
Check out my Medium:
/ ritvikmathematics
My Patreon:
www.patreon.com/user?u=49277905
"Please, take a minute to pause and convince yourself that everything on this board is accurate." So difficult to do when I was in school ("several" moon ago) madly scribbling down everything before it got wiped off the board, but now with the internet, with videos, and most importantly with a person who wants you to learn, this is so much easier to absorb. I'm looking forward to teaching my children and using your wise words. Thank you!
Don't homeschool your kids! you'll screw them up for life!
@@REALdavidmiscarriage And your evidence for this is..? Homeschool has issues, but so does regular schooling.
@@berylliosis5250 dude in my line of work I got to know a lot of people who have been homeschooled and they all show anti social tendencies and varying degrees of depression, but most of all they all hate their parents for forcing them into being homeschooled. most of them have an extremely hard time making friends or socialising with others. how are you supposed to learn to work in a group with kids in your age, if you don't have the social construct of a school. Also why not trust people who have studied a subject for years to teach your kids, over your own superficial knowledge of science and literatur. Also it's almost always the parents who want this whole homeschooling thing never the children. Cause they have serious attachment problems with their kids and can't let go of them because they are so obsessive. please get over yourselves hoomschooling parents!
@@REALdavidmiscarriage I know a bunch of people who've been homeschooled too. They've been socially capable, intelligent, mentally healthy (in one case, far more so than when they were in public school), and completely educated - potentially more so than their peers. They started homeschooling by mutual consent with their parents. Anecdotes don't prove anything here.
While I personally wouldn't want to be homeschooled or to homeschool myself, there are some people who thrive in that kind of system.
@@berylliosis5250 No shit. you just proved my point, exceptions prove the rule. Also you aren't bringing any evidence for it being as good as regular school or better. That's not how that works. You can't just say unicorns exist and ask me to disprove it. You are the one making a bold claim here in comparing homeschooling with regular schools you have to bring factual evidence but you are using anecdotes yourself. So why don't we just slow down a bit and treat this for what it is an argument based on anecdotes not some scientific research paper. Maybe 1 in 1000 students might thrive off of homeschooling. Yeah also maybe 1 in a few million people win the lottery ,so? Does that mean it is worth playing the lottery?
12:00 And if A isn't symmetric, the derivative could be represented as (A+At)x, where At is A transposed. Which also looks nice.
great point :)
Wow. I haven't taken calculus in years and this video made taking derivative of a matrix seem easy to do and understand. Well done as teaching well is an art form unto itself.
Glad you liked it!
Everyone is sleeping and I'm here watching derivatives of matrices
Relatable
'Everyone' includes all persons, presumably... that would include the observer, so this sentence is inadmissible or meaninglesss.. ps i am only a minor student of logic so I praise the observer's meaning...peace
@@doce7606 "except for me" is always implied
@@danielchmiel7787 not to a nit-picking logician, which normally I'm not, lol, i had just been reading Quine..
@@doce7606 "everyone" makes no statement about the one who said it
You're good at this ... extremely amazing....would you mind making a video on the following:
1. Maximum Likelihood Estimation
2. GMM
3. GLS
You are really good at what you do (i.e. making this simple and understandable). Hats off to you.
You are an absolute life-saver! I am a transfer student studying chemical engineering at UC Davis and your videos match up perfectly with what we are taught :) You have helped tremendously and have given me the knowledge to solve my overly complicated problem sets. Keep making videos and I'm certain you've helped many others as well. Brilliant instructor.
are you still studying chem eng?
So it’s basically a weird notation for a Jacobian?
I was thinking the same thing
@@ramakrishnaamitr10 even though he doesn't write them fancy, with how he does the math it looks like these are partial derivatives.
@@ramakrishnaamitr10 he isn't using the appropriate symbol, but he is indeed performing partial derivatives
Okay, so he looks at the function x -> Ax. This is a linear transformation, and the jacobian of any linear transformation is the linear transformation itself. This makes sense because you can think of the Jacobian as the best linear approximation for any function between R^n and R^m, whether it be linear or not.
Now, in some sense, yes you can say that the derivative of the matrix is the Jacobian, because a matrix, after all, represents a linear function. As already stated, the derivative of a linear function is basically the Jacobian.
I think the moral of this video is that it is best to actually think in terms of function from R^n -> R^m, (vector-valued functions)
Does this clarify things?
@Aletak 13 yeah, the Jacobian represents a local linear transformation, which describes how much you are stretching or squishing space. The determinant of the transformation gives you what the area is scaled by, which is why it comes up when you change variables :)
Absolutely love it! It was so useful to have the analogy between regular calculus and matrix calculus shown. Makes things much more intuitive.
amazing, love the energy.
Thank you!
Thanks for all your work ritvik! Especially explaining things with a PURPOSE, not just math porn with no applications in real world.
ruclips.net/video/XQIbn27dOjE/видео.html 💐💐
You are the man. I really appreciate your clear explanations.
Came here looking for LOWESS algorithm, and it turns out that the the derivative of xTAx plays a role in it. You helped me understand what matrix derivation is, plus solved my very particular need. Thanks.
You are a wizard, thanks.
I've watched the video 3 times and picked up few things that I didn't in the first time. I'm a little slow though.
ruclips.net/video/XQIbn27dOjE/видео.html 💐💐
One question: why the derivative of the second example is a column vector? (9:35) I thought it was a row vector, similar to the form in 3:38 (the first row: [df/dx1 df/dx2]. A great video! (It is the same problem as Ravi Shankar’s two months ago)
me too
I think you are correct, at 10:23 he does say that "if you had 3 different functions and 4 different variables you would have a 3 by 4 matrix, i.e. 3 rows and 4 columns". And the result would be 2*xt*A
Me too! I think it should be a row vector, and this pushed me to go back to see the video again
same! is there any answer?? was the instructor wrong?
yeah hes mixing numerator and denominator layout :/, in numerator layout a vector function by a scalar is a column vector, a scalar function by a vector is a row vector. In denominator layout a vector function by a scalar is a row vector and a scalar function by a vector is a column vector. (*By = derivatve with respect to)
man you deserve more spotlight. thank you from the bottom of my heart.
When you calculated the derivative of A over the vector x you add the partial derivatives of the function f1 and f2 as row vectors in the matrix. Then, when you calculated the gradient of f1 = x^{T}Ax then over x the results was a column vector. Shouldn't be in this case the first result A^{T}?
same thought bro
You are just AMAZING !! So clear and easy to get!
Math is so cool! I half suck at linear algebra but seeing all the crazy stuff you can do with it makes me want to go back and learn it really well.
Excellent explanation. Thank you!
I got this randomly from RUclips’s algorithm, and I’m gonna give this man a follow! I’m a math major
in like the first week or something? you see there is some weird shit going on right?
@Roman Koval everything is probability
@Roman Koval literally the very existence of an electron in a place in space is a probability, and electrons are building blocks for literally every material object
I thought of it on this semester.
If we consider the properties of the linearity of the derivative, I supposed that it must be distributed on the matrix.
I'm still taking the subject of Linear Algebra but the video showed off some neat tricks for this type of problem
ruclips.net/video/XQIbn27dOjE/видео.html 💐💐
This video is really helpful! Thanks for making this concept so clear!!!
This really simplifies the matrix derivative.
Thanks alot for making this so simple to understand!
Super easy to follow along and clearly explained, thank you!
Glad it was helpful!
Does this work only for symmetric matrices, or also for all square matrices?
Thank you so much for making all of these videos!
Great job clearing up this topic.
This channel is what we ALL needed, its great ur a genius. Should be a uni lecturer
This video helped me a lot! Love your energy, keep 'em coming!
Thank you! Will do!
Hi man, at the minute 9:28 there is one function on two variables, why have wrote a matrix with 1 column and two rows and not viceversa?
Very impressive! I like how the total derivative "emerges" from the xtAx form. It really shows how effectively linear algebra notation can be used to assemble new structures. One comment I would make is that as far as I know when taking the partial derivative it is common to use ∂ instead of d.
ruclips.net/video/XQIbn27dOjE/видео.html 💐💐💐
this video just saved me!!! Exactly what I need for my Econometrics assignment!
lol same
The derivative of Ax is A^T
Chandrashekar would be proud. I'm learning. Thanks
Sure we can take the derivative of a matrix! It just depends on what the function is. In this example shown in the video the function output is a vector. But it could have also been a matrix output. In that case we would have a rank 4 matrix as the derivative assuming inputs are two 2 dimensional tensors each. The main idea is to understand what a Jacobian matrix is and then you will see how all these are various special cases of that general idea. To rephrase, yes, we don' take a derivative of just any matrix as it makes no sense in the same way it doesn't make sense to take derivative of a vector. Derivative is defined for a function. But no matter what the output of a function is, be it scalar, vector, tensor or matrix, there is always a way to define its derivative.
A matrix inherently has discrete, integral indices, so it can't be differentiated, but you can differentiate a function whose coefficients are expressed by a matrix
I'd even go further and just say that you can identify a matrix with a vector in R^{nm} and use the idea of the Jacobian matrix like you talk about. I don't think it is necessary to go into the idea of rank unless you specifically care about tensor calculus.
Even still, In that case, it is still basically vectors, except you might be taking tensor products with elements in the dual space.
I absolutely agree that the main idea is to understand what a Jacobian matrix is
@@astrobullivant5908 The fact that the indices are discrete doesn't matter- a vector also has discrete indices! You don't differentiate with respect to the index number, but with respect to whatever variable each component depends on. If the matrix is constant, like [ 1 2 ; 3 4], then the derivative would just be the zero matrix.
@@seanki98 You're right, I'm wrong.
Great video, you have way with drilling the concept into people's heads. Just awesome.
I appreciate that!
an incredible easy to follow class, thanks a lot!
I love your energy in what you are doing.
I cheer for you and thank you for making great contents!
I appreciate that!
if i have a simple x^2
do i need to express it into x times x, where x is just a simple unknown,
and then i need to express one of the x in terms of matrix Ax like in this video
and the other x in term of vector ?
so that i can define what differentiation is to the computer,
or do i have other option?
You saves my warm quiz on Introduction to ML. Many thanks!
Thank you for being a tremendous help!
At@ 9:41, could you tell me why you took a column vector and not a row vector? Is it a rule that we should take it as a column vector ? How to know what would be the shape of the matrix or vector?
This is astonishingly easy to follow.
This is awesome!! 👍
Excellent! I was looking for the explanation of derivative of linear transformation for a long time!
ruclips.net/video/XQIbn27dOjE/видео.html 💐💐
Great video! You’re really awesome at explaining things clearly!
ruclips.net/video/XQIbn27dOjE/видео.html 💐💐
thanks for great video!
does the formula works even if in the case of when A is not symmertirc too?
You're a great communicator. Go Bruins!
go Bruins!
It is very helpful!
I am learning linear model.
But I am not familiar with derivatives of matries.
Thank you!
Earned a sub. Nice job man, thank you so much.
Amazing video. Thanks man. Subscribed right away.
Thanks for your awesome video!
Great video, thanks for sharing
At 10:25 you said for 3 different functions and 4 different variables you'd have a 3x4 matrix. But the one you solved above only had 1 function and 2 variables x1 and x2. Why then did you create a 2x1 matrix instead of a 1x2?
Thats very good content, helped me out alot, thank you good sir
Excellent video, thank you!
exactly what i was looking for !
Who are the 226 people who didn't like the video? Maybe the ones who didn't understand why the derivative of kx = k, and the derivative of kx^2 is 2kx. This is mind-blowingly intuitive. I've never heard a matrix being called a bunch of scalars in a box. All the videos made by ritvikmath are excellent videos. Although I have used Eigenvalues, Eigenvectors, and derivatives of linear combinations extensively, it never made this kind of intuitive sense.
dayyyyyyyyyum. So nicely explained. Thank you!
Very clearly explained.Subscribed.Thanks
13:15 Think of rearranging k*x^2 as x^T*k*x since x is a scalar. That is just the analog of quadratic form of x^T*A*x
Indeed :) - I've always thought of x^T*k*x as the vector form of quadratic too.
Nice!
This was real fun. Great analogies!
ruclips.net/video/XQIbn27dOjE/видео.html 💐💐💐
Excellently explained
this is awesome!!!! first time that I really get to know this!
ruclips.net/video/XQIbn27dOjE/видео.html 💐💐
Great job, thanks a lot from italy. Keep up the good work ;)
Wow all the way from Italy! Thank you :)
awesome sir :).
Great channel ! Thanks for sharing your knowledge and teaching skills.
ruclips.net/video/XQIbn27dOjE/видео.html 💐💐💐
It's easy to understand！！！Thank you
ruclips.net/video/XQIbn27dOjE/видео.html 💐💐
Thanks, very good video. helped me in understanding everything
Maaan really good video, thanks for that!
Interesting. I never learn this stuff from colleague nor it introduce it before.
ruclips.net/video/XQIbn27dOjE/видео.html 💐💐
Great video great work. Congrats
great video:) you're really good at explaining. Thank you very much!!
You're very welcome!
Thank you! This video really cleared things up for me :)
I'm so glad!
9:41 It’d be better to use the partial derivative notation since you have 2 variables essentially is that correct?
You are great ! great video, great representation, Thanks!
ruclips.net/video/XQIbn27dOjE/видео.html 💐💐
Actually, the calculations for \frac{\mathrm{d} Ax}{\mathrm{d} x} you use the numerator-layout notation and the result is A, but when you compute \frac{\mathrm{d} x^T Ax}{\mathrm{d} x}, you use the denominator-layout notation which the result is 2Ax, and if you use the numerator-layout notation, the result should be 2 x^T X.
Reference:
en.wikipedia.org/wiki/Matrix_calculus
I found the same thing. xTAx = 2xTA instead of 2Ax. The difference is the result is a row vector instead of a column vector. I also used the same wikipedia resource for definitions.
@@tissuewizardiv5982 Agreed.
Daym, this is very intuitive!!
ruclips.net/video/XQIbn27dOjE/видео.html 💐💐
Thank you so much for this amazing video!!! It was exactly what I needed
You're so welcome!
Dude I just wanted to let you know that your explanation is very intuitive and noice
thanks!
Suppose 2×2 matrix=A has a characteristic polynomial = C.P(A) = λ² - bλ + c
then dƒ/dλ = 2λ - b
Cayley Hamilton: A² - b•A + c•I
means dƒ/dA = 2A - b•A
which looks an awful lot like 2λ - bλ
Oh, that doesn't mean anything I'm just using power rule with A & λ instead of x.... right?
Well what is rhe definition of a derivative?
lim [ (ƒ(x+Δx)-ƒ(x))/Δx] = dƒ/dx
Δx→0
What about this?
lim [ (ƒ(λ+Δθ)-ƒ(λ))/Δλ] = dƒ/dλ?
Δλ→0
What about this?
lim [ (ƒ(A+ΔA)-ƒ(A))/ΔA] =dƒ/dA ?
ΔA→0
Ok, fine Im doing the same thing again with limits now. but suppose you define a 2×2 matrix=A with actual numbers and then you say ƒ[A] = A² = AA
and you speculate dƒ/dA = d/dA[A²] =2A
Right???
I mean you actually write entries in the matrix in
this limit below s.t.
I = Identity matrix
only instead of this:
lim [ (ƒ(A+ΔA)-ƒ(A))/ΔA]
ΔA→0
you cant ÷ a matrix, so you do this
lim [ ((A+ΔAI)² -A²)(ΔA)⁻¹ ] =
ΔA→0
lim [ A² + 2ΔAI + (ΔAI)² -A² (ΔA)⁻¹ ]
ΔA→0
ΔAI = ΔA•Identity matrix =
[ΔΑ 0]
[0 ΔΑ] = ΔΑΙ
(ΔΑΙ)⁻¹ = (1/det(ΔΑΙ))•adj(ΔΑΙ)=
[1/ΔΑ 0 ]
[ 0 1/ΔΑ] = (ΔΑΙ)⁻¹
We can get 2A.... right?
Or is it, as you say, just like taking the derivative of a constant?
(I leave this as an exercise for the reader to verify.)
Just playing... I'M DOING THIS!! NO CONSTANT, BABY!!
e.g.
Claim:
It is possible to take the derivative of at least one 2×2 matrix = A s.t. ƒ[A] = A²
& d/dA [A²] = 2A according to "the limit definition of a derivative" and the definition of a function, ƒ.
Proof of Claim:
Let
[ 1 1]
[ 0 2] = A
[ 1 3 ]
[ 0 4 ] =A²
[ 2 2 ]
[ 0 4 ] = 2A
[1/ΔΑ 0 ]
[ 0 1/ΔΑ] = (ΔΑΙ)⁻¹
lim [ A² + 2ΔAI + (ΔAI)² -A² (ΔA)⁻¹ ]
ΔA→0
lim [ A² + 2ΔAI + (ΔAI)² -A² (ΔA)⁻¹ ]
ΔA→0
oh, look at that boy!!
wait until that s**t cancels out
(I kneew they wouldn't line up, but you see it!!)
[1/ΔΑ 0]• ([1 3]+[2 2]+[ΔΑ 0]+[(ΔΑ)²0]-[1 3])
[0 1/ΔΑ] ([0 4] [0 4] [0 ΔΑ] [0(ΔΑ)²] [0 4])
as lim ΔA→0
Look at A² & -A²
gone! canceled
[1/ΔΑ 0]• ([2 2]+[ΔΑ 0]+[(ΔΑ)²0])
[0 1/ΔΑ] ([0 4] [0 ΔΑ] [0(ΔΑ)²])
as lim ΔA→0
now add those 3 matrices
[1/ΔΑ 0][2+ΔΑ+(ΔΑ)² 2ΔΑ]
[0 1/ΔΑ][ 0 4ΔΑ+(ΔΑ)²]
as lim ΔA→0
Multiply
[1/ΔΑ 0][2ΔΑ+(ΔΑ)² 2ΔΑ]
[0 1/ΔΑ][ 0 4ΔΑ+(ΔΑ)²]
as lim ΔA→0
=
[(2ΔΑ+(ΔΑ)²)/ΔΑ 2ΔΑ/ΔΑ]
[ 0/ΔΑ (4ΔΑ+(ΔΑ)²)/ΔΑ]
as lim ΔA→0
=
[(2+ΔΑ 2]
[ 0 4+(ΔΑ)] as lim ΔA→0
=
[(2+0 2]
[ 0 4+0] =
[ 2 2 ]
[ 0 4 ] = 2A = d/dA[A²]
therefore,
it is possible to take the derivative of at least one 2×2 matrix = A s.t. ƒ[A] = A²
& d/dA [A²] = 2A according to
"the limit definition of a derivative"
and the definition of a function, ƒ. ■
edit : I knew these wouldn't all line up, lol
Wha-what did you do?
Very good content. Democratizing linear algebra
Wow. Excellent video. Thanks!
Glad you liked it!
very easy to understand and thank you very much!
ruclips.net/video/XQIbn27dOjE/видео.html 💐💐💐
this is so awesome.
Your teaching quotient is very high.
Your concept is very clear and agood teacher
Brilliant, thank you!
You're very welcome!
thank you I find this very helpful
I have a question, in the first derivative d(Ax)/dx, why should we do it in row, while d(x'Ax)/dx, we do it in column? Thank you
Same question for this...
Hmm... Good question. I didn't notice that before I read your comment. It could because of the x' present at the beginning of x'Ax. I'm not sure though.
That's exactly what I thought.
Maybe he is writing the d(Ax)/dx in matrix notation while d(x^(T)Ax)/dx in vector notation? He does use square brackets for the former and parentheses for the latter, but I'm not too sure myself.
Because there's a difference between X and the transpose of X. X is a column vector and so X transpose is a row vector.
btw is it more appropriate to take partial derivative symbols for x1 and x2?
Sorry if my question is lame but at 10:24 you say that "If you have 3 different functions and 4 different variables, you have 3X4 matrix." Since we have 1 function and 2 different variables in the example, why don't we have 1X2 matrix instead of 2X1 matrix?
superb! .... Everyone is sleeping and I'm here watching derivatives of matrices
Wow, that was a brilliant video! I really like the teaching style. +1 Subscriber
Awesome, thank you!
Very informative!
god thanks very much, really an awesome work
you're a very good teacher
This means some diagonalisation of amplitude split-up merged by a third interactive may a transpose double the amplitude by a third interactive.Every crystal by interaction split-up the ampiltude by a phase difference interacted by a transpose dynamics of third interactive may produce twice the amplitude as obeserved by symmetry.
Sankaravelyudhan Nandakumar.
Very nice!
thanks for the great explain of matrix derivative!