Matrix trace isn't just summing the diagonal | Lie groups, algebras, brackets #5
HTML-код
- Опубликовано: 28 май 2024
- Part 6: • Lie algebras visualize...
Can we visualise this algebraic procedure of adding diagonal entries? What is really happening when we add them together? By visualising it, it is possible to almost immediately see how the different properties of trace comes about.
Files for download:
Go to www.mathemaniac.co.uk/download and enter the following password: traceisdiv
The concept of the whole video starts from one line the Wikipedia page about trace, and I am surprised this isn't on RUclips: "A related characterization of the trace applies to linear vector fields. Given a matrix A, define a vector field F on R^n by F(x) = Ax. The components of this vector field are linear functions (given by the rows of A). Its divergence div F is a constant function, whose value is equal to tr(A)."
Actually, this is one of the last concepts in linear algebra that I really wanted a visualisation for, the other being transpose, but this is already on the channel: • The deeper meaning of ...
Chapters:
00:00 Introduction
00:48 Matrix as vector field
02:24 Divergence
04:50 Connection between trace and divergence
10:12 Trace = sum of eigenvalues
13:32 Determinant and matrix exponentials
15:15 Trace is basis-independent
18:10 Jacobi's formula
Further reading:
Trace (the origin of the whole video): en.wikipedia.org/wiki/Trace_(...
Divergence (more qualitative, and subtly different from the video): en.wikipedia.org/wiki/Diverge...
Jacobi's formula (more formal proof): en.wikipedia.org/wiki/Jacobi%...
Other than commenting on the video, you are very welcome to fill in a Google form linked below, which helps me make better videos by catering for your math levels:
forms.gle/QJ29hocF9uQAyZyH6
If you want to know more interesting Mathematics, stay tuned for the next video!
SUBSCRIBE and see you in the next video!
If you are wondering how I made all these videos, even though it is stylistically similar to 3Blue1Brown, I don't use his animation engine Manim, but I use PowerPoint, GeoGebra, and (sometimes) Mathematica to produce the videos.
Social media:
Facebook: / mathemaniacyt
Instagram: / _mathemaniac_
Twitter: / mathemaniacyt
Patreon: / mathemaniac (support if you want to and can afford to!)
Merch: mathemaniac.myspreadshop.co.uk
Ko-fi: ko-fi.com/mathemaniac [for one-time support]
For my contact email, check my About page on a PC.
See you next time!
EDIT: It should be adjugate matrix, not adjoint. Can't understand why I called it adjoint...
There probably wouldn’t be another video until (maybe) Easter, and for some reason I have been quite efficient with this video, and so here you have another video within a week. I am surprised that this intuition of trace hasn’t been picked up on RUclips, because this is on the Wikipedia page.
Thank you and best of luck with your studies
It’ll take us that long to absorb it in spite of you brilliant elucidation!!
Wikipedia's math articles are the most abstract and general, so they are not really useful for a non-mathematician. You go to look up what a triangle is and get overwhelmed by non-Abelian algebras, Schwartzshild-Horowitz manifolds and such. Maybe nobody even bothered to look it up.
@@Stebanoid that's why it is better to learn from books
@@Stebanoid I'm just impressed you wrote non-Abelian algebras and Schwartzshild-Horowitz manifolds (terms I just copied)!
Very cool. I've never seen any of these interpretations before. Gonna have to watch this one again later. It's interesting how we're both studying Lie groups/algebras, and yet you're going down roads I've never seen or heard of.
Your relativity videos are best on internet. And you are the only one who understands twin paradox clearly thanks for videos
The visualisation of trace (and another concept of transpose) is something that I have been dying for ever since 3blue1brown makes the linear algebra series. So I have actually been thinking about this for a very long time - but trace relates to Lie algebra (like why does det = 1 condition for SU(n) means that the matrices in the Lie algebra needs to be traceless), and that's why I put it in this series.
@@mathemaniac heh heh I just commented the Special Unitary group 🤓😅🍜🌶️
@@motherisape "the only one who understands twin paradox"
🥺 I thought I was doing well
😳 😟 🤓😁😁😁🌶️🍜🉐🈹
@@misterlau5246 I mean he is only teacher on youtube who explain it with full derivation . all other videos are very loose or have few errors .
I cannot understand why we never were given an explanation of what Tr actually is. We were given its definition and properties and that's it. Until few minutes ago I thought it is some absolutely arbitrary useless thing that was given a name for some weird historical reason. Thank you for your service, good person who made this video!
It is such a blessing that we live in an era when math and physics are explained so clearly, from the fundamentals all the way to quantum field theory.
This is amazing. There was always so much mysticism around the trace. Just a seemingly random choice to study which happened to have all these nice properties. Now it's much clearer why it is important.
Super cool video, I've never seen this interpretation of a trace before
That's why I need to share this! In fact, this intuition is in the Wikipedia page, but somehow not on RUclips.
Thank you so much for the explanation! I have sought "physical intuition" behind trace for very long during my studies and eventually gave up. It is so satisfying to finally find this beautiful geometrial interpretation =)
This is beautiful. I once satisfied my curiosity with geometric interpretation of the trace as the directional derivative of a matrix A along the identity matrix, but I now see how much more beautiful and intuitive the trace can be. Thanks!
Interested to hear more about "directional derivative of a matrix A along the identity matrix" if convenient. Thanks
@@alexboche1349 lim_{t -> 0} (det(Id + tA) - det(Id))/t = trace(A)
which I would call instead "derivative along the direction A at the identity"
@@alexboche1349Look up Jacobi's formula if you are still looking for that.
Haven’t had my breath taken away like this since 3blue1browns linear algebra videos all those years ago😢
Love these videos. They make infinitely more sense than reading about the concepts. Especially with the asides along the way to say why a certain concept is important. A lot of texts just barrel forward without that explanation. it makes things less sticky.
i'm really enjoying this series, keep up the good work!
Very good explanation. It made me realize how much I like linear algebra but how much I still need to learn and understand (I haven't used it since around 20 years ago)
Awesome video, I never knew these connections and visualisations; really helped me understand all of this better.
This is absolutely brilliant. I once searched every mathstackexchange topic about trace in the hope of findind an interpretation like this, without success. The fact that you not only give it but also derive all of trace's classical properties seemingly "easily" with it really gives the impression that the true meaning of this operation lies here.
I've tried interpreting trace as related to the "perimeter" of the unit square under a matrix (in the case of diagonalizable matrices, the formula itself lead me there, by analogy with tthe determinant) , but it never was completely successful.
This video really feels like the missing piece of the puzzle. Thank you so much.
This is also what I have been thinking for a long time, and decide to put it into this series about Lie theory because it is related. Actually, this intuition is in Wikipedia: "A related characterization of the trace applies to linear vector fields. Given a matrix A, define a vector field F on R^n by F(x) = Ax. The components of this vector field are linear functions (given by the rows of A). Its divergence div F is a constant function, whose value is equal to tr(A)."
@@mathemaniac Wow, crazy that I never saw it on Wikipedia.
Trace now checks off my list of "interpretation needed" concepts.
The last one regarding linear algebra where I feel my interpretation isnt complete are complex eigenvalues. I guess when "scaling" is really scaling and rotating, things get messier...
@@TagRLCSI think that complex eigenvalues says to us that the eigenvector is in another dimension, implying that all the original plane is affected by the portion of the linear transformation related to rotation.
Complex eigenvectors I can provide an intuition for, but I am not sure about complex eigenvalues (it would be a lot more complicated). That will certainly be on my video idea list, and maybe after this video series about Lie theory.
@@mathemaniac Would truly be incredible. Looking forward to the next Lie theory videos too tho !
Representing the divergence as the volume rate of change per volume is interesting. I'm used to thinking of it as the flux per unit volume, that is, the flow rate through the boundary per volume enclosed. They're naturally very related concepts; a net flow out of the boundary naturally results in net expansion of a volume undergoing said flow, and a net flow into the boundary naturally results in net contraction of the volume, and in the limiting case approaching 0 flow time, those amounts must intuitively be equal.
Algebraically, the former can be expressed as the determinant of (1+εa, εb; εc, 1+εd)/ε, which as you showed in the video is a+d as ε tends to 0, while the latter can be expressed as the integral of (Av dot dB*) divided by the volume enclosed by B, where dB is a differential boundary element and so dB* is a normal vector of the boundary. In 2d, for a square, basis-vector-aligned choice of boundary of side length ε, this gives ((-(ε^2)c/2) + ((ε^2)a + (ε^2)b/2) + ((ε^2)d + (ε^2)c/2) + (-(ε^2)b/2))/(ε^2) -- intentionally leaving it unsimplified to show the process of finding the integral on each side of the square independently -- which then simplifies to a+d as expected.
Edit: I had briefly thought this method has a benefit over your method of not needing to take the limit as ε approaches 0; however I realized that I used ε for different quantities in each case. The limit as flow time goes to 0 was already performed implicitly in my expressing of the instantaneous flow rate at v as Av, so I repurposed ε to represent a parameter of the boundary, which is a property you had already covered by that the divergence of a field generated by a matrix is the same at all locations in the field. So I think your method is pretty much better in every way for thinking about divergence, in that framing it your way implies a much simpler calculation.
Actually the main reason why I am thinking of rate of change in area / area rather than flux is for the properties described in the video, especially the matrix exponential part. It would be difficult to use the flux intuition of divergence.
I wish the video showed the two interpretations were equivalent. I was confused initially
i'm just getting started with this trying to build a good intuition , and i must say i've had no previous background in calculating flux or divergence , so may i ask what would the dot product be explicitly here ? and what's the normal vector ? Thanks
@@adamboussif8035 the normal vector of a portion of a surface is perpendicular to the surface, and often is chosen to have a length corresponding to the area of the portion of the surface in question. the dot product of a vector field with the normal vector of a surface at a point is the product of the parallel components of the field and the normal vector, which, to phrase it more intuitively, means that it expresses how strongly the field is pointing directly through the surface at that point. summing up all those strengths over a closed surface gives how much the field tends to exit or enter the enclosed region overall, which is sorta like the average divergence in the enclosed region.
An interpretation of trace I don't see often enough: It is the dimension times the average amount that a linear transformation L preserves direction, i.e. it is dimension*(1/(unit sphere surface area))*(integral of v.L(v) over v in the unit sphere)
Do you any reading material related to this interpretation ?
@@TagRLCS No, but you can derive it from the connection between divergence and trace using the integral definition of divergence: en.m.wikipedia.org/wiki/Divergence#Definition
AWESOME!!! Never see this kind of thinking way
amazing animations, for the first time I really understood why we found Trace.
Very nice video, thank you. Looking forward to the next one.
Good video! I did not previously know the relation between trace and divergence!
Incredibly fantastic video!
Super good content. I didn't know most of these
Yet another time when thinking about matrices as grids of numbers leads one astray. I'd heard that the trace is the number of entries on the diagonal, but no explanation for why that should even be a meaningful operation. Thank you for actually explaining trace in a way that actually makes it seem important.
The idea of trace as divergence I hadn't considered, but makes a surprising amount of sense considering the matrix forms of elements from Clifford algebras. There, the "geometric" product of two vectors can be seen as a sum of their inner and exterior products. The exterior product product is easy to see in terms of matrices as it's ½(uvᵀ - vuᵀ) (or maybe the other way around), which neatly parallels how it's defined in Clifford algebra as ½(uv - vu) (with the geometric product rather than the matrix outer product). The inner product, which is what's more traditionally used to find divergences, however is harder to connect with the matrix representations. adapting ½(uv + vu) as ½(uvᵀ + vuᵀ) gives a diagonal matrix, but not a scaled identity matrix like you'd expect a scalar to be. However, if you add up the diagonal (and remove the factor of a half), you _do_ get the scalar that you'd normally get from the dot product. Adding up the diagonal in this case giving the trace. So algebraically, it does make sense that trace would be related to divergence, since the inner product is also related to divergence and trace appeared to be related to the inner product.
I was looking for the geometric algebra interpretation, somehow I knew it would be here. Thanks
Man, what a surprise!! After your phd story i was saden that there wont be many videos!!
Please don't expect another video next week - the next video might be somewhere during Easter.
This always felt more intuitive after I studied statistics, where for a covariance matrix the main diagonal elements are the variances of the data, so describing how "spread out" data is from that variable.
Really nicely done!
I think a better visualization of the rate of area change might be the following:
First create two parallellograms generated from A(t) and A(t +dt) with areas det(A(t)) and det(A(t + dt)). From the definition of the derivative and making dt very small, the rate of change becomes: d/dt(det(A(t))).
Then go back to the parallellograms. The vectors [A(t +dt) - A(t)](1,0) and [...](0,1) describe how the first parallellogram transforms into the second. Again, through the definition of the derivative and making dt very small we get: d/dt(A(t))(1,0) and d/dt(A(t))(0,1). These vectors don't act on the unit square but on the parallellogram generated by A(t). So we should first normalize - so to say - the area of the A(t) parallellogram to get the effect of the vertical and horizontal changes which generate the trace. So we can act A^-1(t) on it, but we shouldn't forget to multiply the result with det(A(t)) if we still want to measure the same thing. Here the fact is used that multiplying by A^-1(t) constitutes a factor det(A^-1(t)) on any area, which equals det(A(t))^-1. So we find that the same rate of change in area is det(A(t))*Tr(A^-1(t)d/dt(A(t))).
And since we already established that the rate of change of this area is equal to d/dt(det(A(t))), we get the desired result.
That is basically the last bit of the video!
@@mathemaniac No, my point is that you don't explain the change of area in terms of the literal change of one parallellogram towards the other, but keep the story in terms of vectors and formulas, which makes the explanation very algebraic.
You point to the formula and say "this term means this and that term means that, and if we cleverly rearrange it, you get the desired result". This is the opposite of an intuitive visualization; this is closing yourself off from reality, playing around with a formula, getting a result, and then afterwards checking back in with reality to see what you just did, i.e.: Algebra.
My method is that there are two ways to calculate the rate of area change: The difference in area of before and after; and the effect of an infinitesimal change in A on the unit square applied to the before parallellogram. And then just equate the two.
This way the explanation is kept on a very fundamental - dare I say high school level - understanding of dealing with areas of shapes.
@@7th808s can you update with some pictures? you probably have a specific visualization in mind which is easier to convey than with text
Great! At 8:05 I found it a little hard to understand, but I think what you're saying is the following.
Algebraically: adding columns doesn't change the determinant.
Geometrically: This can be seen by just drawing a parallelogram and showing it's area equals that of a square by repositioning the outer triangles to form a square.
More abstractly, when you move in the horizontal component, you're moving the vertical basis vector in the direction of the horizontal basis vector, so that the angle of the resulting vectors starts to become less than 90 degrees (closer to 45 degrees). But area of the parallelogram (i.e. the determinant) between vectors u,v (which equals |u||v|sinθ) is maximized when the vectors are perpendicular (so θ=90 degrees) so this is bad for the area. So on the one hand you're making the vector norm |u| longer (which increases area ), but on the other hand you're decreasing the angle θ so on balance the effect is zero.
Yet another channel worth to be subscribed.
I love this playlist!
wow i am on this topic in my course and you just made a vid on it thanks
Such a high quality content!
Great video! It would probably help the viewers if you made a blue dot in the (0,0), because some visual arguments you make seem to rely on (0,0) being mapped to itself. Or you could just draw axes for x and y.
Yeah, probably a blue dot here would help. The axes would have been very distracting from the already crowded picture of vectors.
Amazing video. It would be a prayer answered if you would cover the meaning of successive powers of a [square] matrix, particularly the trace of each successive power when the matrix is a graph adjacency matrix (and therefore triangular-symmetric). Big tie-in with quantum theory!!
I'm working on module theory right now and some of these results are paying the lighting bills in my head
Really cool and daring that you go into these topics which are generally a little bit more advanced than say 3b1b content
I wonder, how many watching this could really follow. Some of the underlying math was gently skipped to follow from one to the other step of argumentation. Took me some thoughts to fiddle the pieces together. But a nice visualization.
Might want to mention why the evolution of the unit square will remain a parallelogram with one corner at the origin (it all directly follows from linearity of A and the equation of evolution, but the same could be said for everything in the video).
Well done!
The last two videos are more entangled than the first ones. I couldn't follow a lot of details.
But i gonna watch the whole series.
That moment when you realize why the trace of the density matrix matters in statistical mechanics...
I would love to know more about statistical mechanics - but I have definitely seen trace of an operator somewhere in QM.
In that case we can wait!!
What do you mean by the vertical and horizontal components not mattering? I see they wouldn't matter for just a small area around the unit vectors, but we are considering the whol unit square.
very interesting
I studied linear algebra for a semester.
Noone explained us why we care about traces, determinants or eigenvalues/iegenvectors. "Shut up and compute" was the approach.
My University was good, but I wasn't in math department. Curious to know why these metrics are important.
Compare this explanation to the formal def, the key is to recogonize the "flux" as a "generalized area changing rate per unit area".
Isn't the trace a particular case of a convolution of a diagonal? Because you could use a more general convolution instead, but, since you generally want to have a square diagonalizable matrix, you usually don't need to, and this tool does enough. It's slightly circular, but the argument is: all info is in the vectors and values, and all else you need to store is the map that takes you from the old to the new matrix. Otherwise, you actually don't have cozy stuff like eigenvalues and eigenvectors and you have to settle on different tools, yes? And those can be defined like the sum of other diagonals and are occasionally useful. Or, at least, that's what I thought it was the thing we did it for, but I might have misunderstood it.
Wow ! !
Thank you
8:40 It's not clear to me that the horizontal component wouldn't change the area. For me, it looks like the both components would not change the area.
8:25 The red vertex I understand that it's changing to the right because of that vectors horizontal component, but the vertex above shouldn't change to the left, since the vector in that vertex has a horizontal component pointing to the left?
Regarding 8:40, it's a "second-order" effect (i.e. the effect is only zero at the margin, then it becomes non-zero). When you move in the horizontal component, you're moving the vertical basis vector in the direction of the horizontal basis vector, so that the angle of the resulting vectors starts to become less than 90 degrees (closer to 45 degrees). But area of the parallelogram between vectors u,v (which equals |u||v|sinθ) is maximized when the vectors are perpendicular (so θ=90 degrees) so this is bad for the area. So on the one hand you're making the vector norm |u| longer (which increases area ), but on the other hand you're decreasing the angle θ so on balance the effect is zero (at the margin).
I'm self-taught, so bear with my poor grasp on terminology. I suspect there's a relationship to this and the characteristic polynomial. Let K be a Krylov subspace of A. (K^-1 A K) yields a companion matrix C of A. C describes the matrix transformation of A as a recurrence relation defined by the characteristic polynomial. There's an implied leading 1 of the characteristic polynomial, so it's already in monic form. The first non-trivial coefficient is the second one, which happens to also be the trace of the matrix A. (It also happens to be the only nonzero term on the diagonal of C).
For example a companion matrix with characteristic polynomial x^n - r x^(n-1) will generate a geometric progression with a common ratio r. The trace of that companion matrix is r. So in (over)simplified terms, the trace acts like a common ratio/exponential base. There's might be a better way to describe this using the matrix exponential. This is actually my first time seeing that det(exp(t A)) = exp(t tr A) equation, but that appears that it might be describing what I'm struggling to explain.
It would appear that this is the same phenomenon described in a different way
For the third property you could have used the fact that eignevalues are basis-independent, so their sum (trace) must also be basis-independent.
Yes, but I want to use another, perhaps more direct perspective, which would be useful for showing the fourth property, and would be useful in developing intuition for what AB-BA actually means (but that's a story for another time).
I see, that makes sense
BTW, thanks for your work!
i'll have to look at these again at some point, because i had difficulty following halfway through (various local distractions compounding other factors, such as my dislike of matrix notation)
i've understood most properties of the trace, in the past, through observing that the trace can also be the delayed completion of a scalar product between two vectors, of which the matrix is the outer product; i have not checked how generalizable this is to square matrices larger than 2x2, but i have doubts
i'm also not positive how to compare it to my understanding of the determinant, the dimensionality of which is locked to n for an nxn matrix, but has more flexible equivalents in algebras i *do* like (particularly geometric algebra), but that's mostly down to severe executive dysfunction at this point
A smart take.
I have been around mathematics most of my life, and it was really hard understanding what you are trying to do. Then I saw the "Lie groups, algebras" in the title and everything fell into place. I think that you should really start such explanations with the simplest non trivial example, instead of some abstract numbers (maybe diagonal, or unipotent matrix).
Thanks!
Love this video, I am investing eigenvalues in my research, I would love to get some insight from you.
8:05 to anyone confused as to why the area is changing like that, remember that at the point (0,0) the vector *Ax* is zero, i.e it’s fixed.
My quantum mechanics classes would have made way more sense with this explanation and just “Welp, it’s trace because Von Neumann said so”
At 5:25 , what you are saying is, comparing x + epsilon A x, to x + d + epsilon A (x + d) ,
that the difference is d + epsilon A d,
correct?
Edit: wow, ok, so it seems like something pretty similar to “the trace is the derivative of the determinant” can be used pretty well as a definition of the trace?
I guess specifically, tr(A) := (d/dt)(det(1 + t A))|_{t=0}
(And then later getting the Jacobi formula from the properties you first get from that)
Yes.
Hey, I am planning to make educational modern physics videos. I'm amazed by your work. I have read your channel's description and I liked that you produce 3b1b quality animation just by power point, etc. please help by guiding the approach that you take on an animating diagram or maths part.
Pretty cool seeing the trace as a divergence! Is there an equivalent intuition for the curl of the vector field generated by the matrix?
Curl is a very 3-dimensional concept. You can actually compute yourself what the curl of such a vector field would be, but I'm not sure if the resulting vector has a particularly well-known meaning.
The reason why curl is a 3D concept is because it is the cross product of the grad operator with a vector. Cross product is a 3D concept. This is because in 3D there is only 1 direction orthogonal to both the vectors you are crossing. In 4D for example you have 2 directions orthogonal to the vectors you are crossing.
The Hodge Star operator sort of generalizes the cross product by saying let the "cross product" be the whole orthogonal (n-2)D hyperplane.
For example in 4D the "cross product" is the whole orthogonal plane. This object is a bivector.
You can use this Hodge Star Operator to generalize the curl but I'm not even sure how to begin visualizing it.
@@smolboi9659 I was recently learning the Hodge star operator. I wonder why it is not more widely taught? I think physicists have screwed up math ed by making everything 3-d when it really should be n-d.
@@alexboche1349 yea u really want to start with learning geometric calculus instead of standard multivariable calculus with curl and div. It makes so much more pedagogical sense. I linked some motivation videos for why we should start with geometric algebra/calculus.
ruclips.net/video/htYh-Tq7ZBI/видео.htmlsi=Ht06EnTdo6Ef5VWJ
ruclips.net/video/60z_hpEAtD8/видео.htmlsi=ppzdznrbtTf-zCTK
@@alexboche1349 yea it really makes more pedagogical sense to start with geometric calculus than vector calculus. Checkout "why can't you multiply vectors" by freya holmer and "a swift introduction to geometric algebra" by sudgylacmoe on youtube.
I have an idea. A linear transformation A can be written as (so the matrix of A is [a b; c d]).
Then, tr(A) is a + d
and div(A) = d/dx (ax + by) + d/dy (cx + dy) = a + d
so tr(A) = div(A)
no this doesn't type check. the divergence of a (0,2) tensor is a (0,1) tensor, but trace gives a (0,0) tensor
@@Iamfafafelthats because they are doing div(Av)
If trace is the divergence of a matrix's vector field, then what is its curl?
As I replied to another comment, curl is a very 3-dimensional concept, and even for a 3-dimensional matrix A, I don't think the curl of Ax (which is a vector) connects to a well-known thing in linear algebra.
Great
Where do you learn stuff like that idea of divergent? The books I normally use don't give me intuitions like that, so I have to search for videos like this one, but you probably know this because you read a book, right?
This intuition is actually on the Wikipedia page about trace: "A related characterization of the trace applies to linear vector fields. Given a matrix A, define a vector field F on R^n by F(x) = Ax. The components of this vector field are linear functions (given by the rows of A). Its divergence div F is a constant function, whose value is equal to tr(A)." In most cases (apart from complex analysis), my videos come from online resources or actually my own thoughts, not books.
As a teacher, I would love to learn how to create a video like this! Any guidance about where to start?
You can download the files following the link in the description.
For the first property you only proved it when A is diagonalisable but then it's kinda obvious from Prop 3 right?
Or did I miss something?
at 5:25 ， you gave displacement after time epsilon formula without proof. And used it to say that the divergence is independent of position x. This is circular reasoning if you don’t explain why you define displacement to be independent of x in the first place.
Are all these comments bots? There is an obvious hole in the explanation to anyone who studied maths at college level.
The displacement after time epsilon d + epsilon Ad pretty much directly follows from the definition of velocities. The argument is that if initially the displacement was d (and this is regardless of the position), then the final displacement is also independent of the position. Hope this clarifies.
The eigenvectors should be orthogonal to each other. I wonder why in the "Trace = sum of eigenvalues" visualization the eigenvectors are not orthogonal.
Thanks for the nice video!
Eigenvectors are not necessarily orthogonal to each other - in fact they might not even exist (in real numbers).
Such matrix (whose eigenvalues are orthogonal) are called hermition matrix for complex matrix, or symmetric matrix for real matrix.
@@Zealot0630not to be the “um actually” guy, but your matrix doesn’t need to be Hermitian or symmetric to have orthogonal eigenvectors. They need to be “normal,” which means that A^T commutes with A (choose dagger instead of T for a vector space over the complex numbers). Unitary matrices, for example, are also normal, and not all Unitary matrices are Hermitian.
And now I think about it, I can’t remember if it’s a biconditional statement or if normal implies orthogonal eigenvectors in one direction only.
@@joshuagrumski7459 If we can't "um actually" in mathematics, then when can we?
Not "orthogonal" per se, but normal. Commuting for example.
And, there can be orthogonality with a vector in Z too, and then we extend to planes, surfaces and we get to folds, 🤓😁🌶️🍜
15:30 Actually, the whole divergence argument was basis independent, so is a basis change argument even necessary?
The trace and determinant appear pretty naturally if you try solving a 2x2 matrix by hand :)
Is the proper name for adj(A) "adjoint" or "adjugate" ?
Yes, can't believe I called it adjoint. I have pinned the correction to it.
you're a real e^x person
it feels that trA is the partial derivative of Ax w.r.t x, where x is an manifold of space A.
Yes in a vague sense. More precisely, the divergence of Ax (which is in some sense "partial derivative w.r.t. x") is the trace of A.
I guess this also means that the trace of the Jacobian of a vector field is equal to its divergence.
I think he meant *adjugate* matrix not *adjoint* at 18:33 (tho the latter term used to mean that in old terminology). en.wikipedia.org/wiki/Adjugate_matrix en.wikipedia.org/wiki/Jacobi%27s_formula
cool stuff, but I lost the thread during the last part. maybe I'll take another look after my brain has had time to chew on the rest.
Regarding trace being basis-independent, that just follows automatically from the fact that the vector field is basis-independent (the actual arrows drawn in the plane are basis-independent, not their coordinate components), and therefore also its divergence. There is no need to actually transform the plane and the arrows, because that's not what a change-of-basis is. It just looks that way if you insist on drawing both the old basis and the new basis using a horizontal first axis and a vertical second axis. But the whole idea of basis change is that the plane is unchanged and you just put new axes on it.
The moment you start to turn and squish the arrows at 15:30, you have made everything a lot more complicated than it has to be. You proved that trace is the sum of eigenvalues, the proof of basis independence is basically identical, you can even use the exact same drawings. Heck, I would even consider proving them in the opposite order, and appeal to a diagonalization to show that trace is the sum of eigenvalues, at least for diagonalizable linear transformations (which, to be fair, was also assumed in your proof).
That being said, how have I never heard that trace is divergence? This is awesome! One more thing I can now freely talk about in linear algebra without ever appealing to matrices. Which is how it should be done in my opinion.
Yes, I briefly thought about doing it your way, but quickly abandoned this idea: the problem is that, at least for me, an active change is way easier to understand than a passive change (which you might call "change of basis"). This is also to establish the QAQ^(-1) order (active change) rather than P^(-1)AP (normally in change of basis, passive change) because it would be a lot more useful later on in Lie brackets.
@@mathemaniac I personally disagree about which is easier to understand (but then again, I have never actually *taught* linear algebra, so I don't actually know). But since you had a thought behind it and also since you want to use it later, I can respect your choice. Tradeoffs must be made some times.
sir do u self study to know this much?
I know the formula 'tr(AB)=tr(BA)' is valid even if B is not invertible. But I think this video couldn't explain this case. Is there any intuition which is helpful to understand this?
As said in the video, we will have to wait until the end of the video series as I need to explain the notion of Lie brackets. In short, the explanation would be demonstrating tr(AB - BA) = 0. AB - BA is what is called the Lie bracket of A and B, but what it really means would be the topic of a future video.
Thank you!❤
Here’s an argument (I don’t know whether to say it “provides intuition”) : let c be a number which is not an eigenvalue of A.
Then, A - c I is invertible (where I is the identity matrix)
Then, tr((A - c I) B) = tr(B (A - c I))
But also, tr((A - c I) B) = tr(A B - c B) (by distributing and I B = B), and this is equal to tr(A B) - c tr(B) , because the trace is linear.
meanwhile, on the other side, we have that tr(B (A - c I)) = tr(B A - c B) = tr(B A) - c tr(B)
And so, we have that tr(AB) - c tr(B) = tr(BA) - c tr(B)
So, adding a c tr(B) to both sides, we obtain
tr(AB) = tr(BA), regardless of whether A is invertible.
And, I suppose here is an intuition motivating the argument: a generic matrix is invertible, and so the claim holds for “almost all” matrices A, and like, on a dense subspace of the space of matrices. And, taking the trace is a continuous function.
And, if an equation holds on a dense subspace, and the equation uses only continuous functions, then it holds throughout.
Like, the equation tr(A B) = tr(B A) is equivalent to tr(A B) - tr(B A) = 0
and, tr(A B) - tr(B A) is a continuous function of A,
and is zero whenever A is invertible, which is in a dense subspace.
And, because the function is continuous, the preimage of {0} must be a closed set,
and because it contains a set which is dense in the space of matrices, it must contain this entire space.
Oops, that was another argument, perhaps not quite an intuition either.
The commonality between the arguments being: it is true for invertible A, and most A are invertible, specifically, you can always change A by just a little bit in order to make it invertible, and this lets you get that it works for all A.
@@drdca8263 Actually the first argument also works over arbitrary fields, as long as A has not all values of the fields as eigenvalue (which could happen with finite fields), while the second argument obviously only works on non-discrete topological fields (there might be additional conditions on the topology for the argument to work; obviously whatever conditions are needed are fulfilled by the real numbers).
@@__christopher__ Nice point, thanks!
At 5:20 you say "after a very short time epsilon", but that is the first mention of time. There have been no "t" variables or anything up to that point, just matrices and vectors, with the matrices being visualized as vector fields. I feel like you need to explain how time plays into this first, before we can know what is meant by "after a short time".
I have a question. Does your proof that tr(A)= sum of eigenvectors work for nondiagonalizable matrices? Because if the matrix isn’t diagonalizable, then there does not exist 2 eigenvectors, but there is an eigenvalue with multiplicity 2, so trace of the matrix will just be twice of that eigenvalue. I know I’ve done the proof that tr(A) is sum of eigenvalues. I think a nice one involves the characteristic polynomial, iirc, but I do wanna understand this visualization. It’s not like nondiagonalizable matrices have to be singular and/or have no meaningful vector field. Just take the classic example of the matrix with eigenvalue 1 and a 2D Jordan block.
I mean, even for matrices like A = (0,-1; 1,0), which describes a rotational vector field, the argument does not work, and you would have to think about how complex eigenvalues / complex eigenvectors work there. However, it would still be true that tr(A) is the divergence, but I am not entirely sure how to say in general tr(A) = sum of eigenvalues in those weird cases.
One thing you might be interested in is to look at a dynamical systems text. The vector fields as illustrated here can be thought as a linearisation of even more general vector fields (and the matrix that generates the linear vector field is actually the Jacobian matrix), and these are the objects of study in dynamical systems. For non-diagonalisable matrices, these are known to be quite difficult to deal with.
In short, I don't know how to interpret eigenvalues in those contexts, at least in this vector field interpretation. They are in some sense more "algebraic" in nature.
@@mathemaniac Yeah, I know from like diff geo that most of the time, Riemannian manifolds are a bit easier to deal with than symplectic manifolds, where the metric no longer needs to be Hermitian, and therefore diagonalizable. I have a professor who does research in dynamical and control systems, so I may ask him for text recommendations.
One thing that may help with understanding this is det(exp(tA))=exp(tr(tA)), so in particular, det(exp(tA))=exp(t*tr(A)), and you may be able to do a proof with this that is more "general" by taking some derivative? Idk, I don't have thaaaat much experience with dynamical systems, but this gives me the vibes of a less visual proof but one which I may see in a text on like diff geo.
And yeah, it makes sense that the Jacobian would be the the matrix that generates the linear vector field. We have that v(x)=Ax, so simply dv=A.
@@mathemaniac The matrix i has imaginary eigenvalues‽ :O So multiplying by i is like... multiplying by i‽
It still feels awkward for me to use ℂomplex scalars in matrices, when ℂomplex numbers themselves are already rotation matrices, and pure 2D rotation matrices are themselves ℂomplex numbers.
@@angeldude101sometimes it’s not useful to think of imaginary numbers as “rotation matrices,” but rather as “elements of the algebraic completion of the reals.” So, although the space of complex numbers is, for example, isomorphic to the field of skew symmetric matrices, it’s sometimes not useful to think of it in those terms. Sometimes we want to think of it in a more abstract sense. For example, real numbers can be understood to actually be sets (Dedekin Cuts), but I have no problem putting them in sets because I’m not really thinking of them as sets. We can use matrices to show that there is a nice construction for the complex numbers in terms of the real numbers, then say “ok, now, suppose you have some ‘representation’ of the complex numbers… what is possible?”
@@joshuagrumski7459I just typed out a response explaining how complex numbers operations are just matrix operations and that imaginary eigenvalues show up when rotation is involved, just like with complex numbers, but then I read that your comment basically said the same thing except written better
형 돌아와...
I got lost so early in this :( i used to understand this stuff
You are incredible! I'm a math Phd student right now and I'm blown away by this series
The geometric argument with the square doesnt quite work because you arent explaining all of the constraints youre assuming. You are assuming the 3rd and 4th corners of the parallelogram have a fixed distance and direction from the red anchor points.
Honestly...i am pretty lost with this video. That is more due to my weak foundation in math rather than your video i believe.
1 thing i don't understand is if trace is the sum of eigenvalues, trace is also the sum of diagonal. When exchange rows, it is said that eigenvalues are not affected, but given a square matrix, wouldn't that change the trace? since (e.g) row 7's 7i7j entry might not be the same as its 7i-2j entry if it exchanges with row 2, and may not be the same(?) as row 2's 2i-7j entry. the same/converse can be said of row 2's 2i-2j and 2i-7j entries?
So, the trace should be different? then how does the eigenvalues remain the same.
and it gets confusing(to me at least) because a matrix can represent a system of linear equations, where the rows are the corresponding equations, it can also be like(not sure the word for it) collection(?) of column vectors?
and then it can also be a linear transformation.
I am really lost.
I am not really sure where you got exchanging rows does not affect eigenvalues, because it is not true. Pick a matrix, say A = (1,0; 0,1) the identity matrix, then exchanging rows gives B = (0,1;1,0). The eigenvalue of A is 1 (repeated), and eigenvalues of B are -1 and 1.
As an Indian,I already learnt it in my high school and during Jee advanced Preparation😊
Cool! So, we have here unitary generators and the specific matrices which comply certain characteristics so an special unitary group is mandatory 🤓🤓🤓🤓🤓🌶️🌶️🌶️🌶️🌶️🌶️😈😈😈😈
first :)
hi
it's so stupid. seriously. adding these diagonal entries should have no meaning at all. it's like... wat?
I'm breaking.
following math is not about numbers, but alphabets + flamboyant vocabulary = simple ideas made complex.
there is nothing about Lie groups, algebras, or brackets...
I did make a very brief mention of Lie brackets in the video - but it is a part of a video series on Lie algebra, and this video is a requisite for the things ahead because I want to use the intuition in the video (mainly thinking about matrices as vector fields or flows of vector fields) to talk about Lie algebras and brackets afterwards.
I apologize in advance if I offend you or make you feel insecure with the comment.
I have a difficult time following your voice, you have some upspeak. Your voices raises at the end of many sentences and it is too irritating for me.
It feels like someone is interrupting your monolog at the end of every sentence for me, which makes it really hard to focus on what you are saying.
Your channel is quite successful so I was not sure should I give you feedback or would I just put unnecessary thoughts in your head.
Nothing to complain about the video but I just can watch it on mute and read the subtitles because else it is way to distracting for me.
hi