Dot products and duality | Chapter 9, Essence of linear algebra
HTML-код
- Опубликовано: 7 июн 2024
- Why the formula for dot products matches their geometric intuition.
Help fund future projects: / 3blue1brown
An equally valuable form of support is to simply share some of the videos.
Home page: www.3blue1brown.com/
Dot products are a nice geometric tool for understanding projection. But now that we know about linear transformations, we can get a deeper feel for what's going on with the dot product, and the connection between its numerical computation and its geometric interpretation.
Full series: 3b1b.co/eola
Future series like this are funded by the community, through Patreon, where supporters get early access as the series is being produced.
3b1b.co/support
Thanks to these viewers for their contributions to translations
Hebrew: Omer Tuchfeld
Vietnamese: @ngvutuan2811
------------------
3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with RUclips, if you want to stay posted about new videos, subscribe, and click the bell to receive notifications (if you're into that).
If you are new to this channel and want to see more, a good place to start is this playlist: goo.gl/WmnCQZ
Various social media stuffs:
Website: www.3blue1brown.com
Twitter: / 3blue1brown
Patreon: / 3blue1brown
Facebook: / 3blue1brown
Reddit: / 3blue1brown
You are a gift to mankind.
He is indeed saving lives here
Exactly!!!
100% agree. Something we just take for granted in University and never wondering why. This is not a healthy way to study. But he, He just light up my curiosity again.
I've been trying to understand linear algebra for the past 2 weeks now for my exam on Wednesday. I've hated every single second of it. I wish I had found this series sooner because it honestly sparked some interest in linear algebra within me and learning new stuff is so much easier when you're interested. Well, I'll try my best not to fail too badly on the exam, but at least I've got a fighting chance now.
glad to be the 1K like to the comment :)
Its a shame that mathematics has become such an infamous subject for the average person, otherwise this channel would be far more popular, which it really deserves.
Mathematicians are the secret society of people who know how the world works. We are the 1%. Consider this video series as an invitation to the silent ruling class.
Math has just become synonymous with computations
The problem is that most teachers suck.
Oobaga !! (Random drum sounds)
It's not completely bad that mathematicians are rare, it means we are much more valuable, which is nice.
"Unlearn what you've just learned" - finally something i can do
no you can't unlearn it if you haven't learned it in the first place.
@@ishan_murjhani lol. I mean these videos simplified all the concepts but still it is difficult to visualize it. Espcially duality ughhhg
ABo NaBiL Gamer ! It took me three watches with a couple of weeks in between to understand this fully. And even then it's not quite simple when you try to apply it on your own
@@ishan_murjhani vacuous truth
The first teacher to ever tell that intently to their students. Man that was even hard to write, what a bothersome alliteration :P
I have a phD in physics. I was really good at geometry during my study years... and I have NEVER seen this. And not one of my (very good) professors talked about this in this way. I am REALLY shocked, in a good way. You are the best Math Theacher on line that there is. We need one of you for every language on Earth!.
You learn this in undergrad quantum mechanics though (Riesz representation theorem).
Well it seems that my professors didn't make the nail go through my skull then. Or i am reaaally good at forgeting QM. Which is possible.
I say the same thing about the determinant analogy to area shifting! Through my bachelor and masters in physics! No one ever made that connection for me!
This series is basically perfect, congratulations.
It's strange that this subject is usually taught so badly. I attended a world class university and at no point did they properly discuss the intuition and motivation for the formal concepts.
Yep. We badly need reform in Mathematics education from "K through post-grad." I'm thrilled that 3B1B is calling attention to this issue.
The problem is, that visual understanding is seen as something, people should figure out themselves, mainly because it is faster (and easier for the lecturer) to show to rigorous proofs, if you _only_ show the proof, and don't show the intuition behind.
However, I guess, one could try and change some of the exercises to help with visual understanding. One could e.g. every once in a while instead of asking for the fourty-third inverse matrix ask the students to draw the effect of a matrix-vector multiplication into a coordinate system.
Or, one could leave out coordinates (to indicate their arbitrariness) and only give them vectors and their images drawn into space/the plane and ask for properties of the (or all) linear operation(s) which make the given vectors land on the indicated ones. (So, you could draw two basis vectors of the plane, let there image be on one line and without knowing any coordinates, students now know, that (e.g.) the determinant will be zero)
Well, the problem is that all of this stuff is impossible to show on a blackboard. If a program existed which was both powerful enough to illustrate these concepts and easy enough to use on the fly, mathematics understanding through using it would take off like a rocket.
That's why you keep seeing people what program 3B1B is using to make them. They're hoping for something they can play and experiment with.
carrionpigeons Don't entirely agree. For one thing, I think all of the visual concepts in this video could be comfortably presented on a blackboard. Secondly, lots of it isn't even visual, it's just explanations. For example, the video where matrix*vector multiplication was explained. Many courses just give you the definition in subscript notation and never motivate the concept. This course gave the "proper" explanation in terms of the intermediate step of linear combinations of matrix vectors, which makes everything a hundred times clearer. I have seen one course (MIT Linear Algebra) spend a long while explicating that idea, but most courses do not.
see also lem.ma for a great set of videos that (also) communicate these ideas well - especially the interplay of geometric and algebraic aspects - but using a traditional blackboard approach.
I wish there was a 3blue1brown of physics also
There are many good books to develop an intuition for physics though. Something lacking in hardcore math, I feel.
nothing beats quality animations for explaining concepts tho
@Bronn - you can say that again!
@@FullHouseFanatic I agree
@@Brono25 minutephysics had a series on special reletivity that was really good like this
I first watched this series 3 years ago. But each year, I rewatch it as I learn more and more maths. Each time, I realize that when I previously watched it, I missed something important and beautiful. It's amazing how much information there is in such a work, that can be easely be missed by the untrained eye, who can't appreciate the hidden connections of math.
I feel the same way
yup : there's quite a lot to think about in this particular episode. Had to stop a few times. Prob rewatch part of it tomorrow morning. Can't believe we weren't taught any of this in Math1...the formulas, yes, but zero intuition like this. No wonder most of us find Math really hard.
This highlights another thing that makes the video series good: The ideas are presented in a way where they can be accessed easily by people with different levels of experience and familiarity with the topic.
"Sometimes you realize that it's easier to understand it (a vector) not as an arrow in space, but as the physical embodiment of a linear transformation - it's as if a vector is a conceptual shorthand for a linear transformation." Definitely one of the most beautiful ideas I have ever learned, thank you for articulating it so well.
here I was thinking "understand vectors as coordinates and vice versa instead of arrows" was revolutionary... And now were back to arrows lol
at this point, i've been pavlov conditioned to associate that piano sound with "you gon learn some shit"
The much simpler real life application, I think, for finding a "dot product":
Imagine you and your friend are pulling a big rock, tied by 2 ropes. Your friend's rope is making an angle ø, with your rope, like V shape.
Now the *dot product* gives the actual amount of force your friend is adding to yours, given there is ø angle between you and your friend's direction of rope pulling.
If your friend stands in same line as you, pulls exactly in the same direction where you want to move the stone towards, (or both are pulling the same rope), angle ø=0, i.e Cos(0)=1, then thats the most efficient way of pulling the stone. Bcz all his force magnitude will absolutely add up to yours.
Otherwise, atleast, the lesser the angle ø between the forces, the higher the total magnitude will be.
But if your friends rope (force) direction is making some angle, ø with yours, then though he's putting some X amount of absolute Force, only a fraction of X (projection of your friend's on yours) will add up to yours.
If your friend makes an angle 90 degree with yours, i.e. he is pulling in perpendicular, then he's not adding up anything to your effort, cos(90)=0.
But if your friend is pulling the rope in opposite direction, cos(180)=-1==>Means your friend is working against you. Then the rock moves in the direction where the force is higher, but the amount of displacement would be much lesser, yourForce-friendForce.
This is also used in the Newton's second law of motion= F = m*a cos@.
So basically the Dot Product tells us, how much a vector is working FOR/AGAINST the other. It's called cosine similarities! This is also used in comparing 2 things (vectors) like how similar are they, do they add up to the entirety or negating each other..
Man.. How I wish I had a board to write down, draw and explain..
Excellent addition.
it s a cool explanation to give when explaining to someone else.
with example in real life it s always easier to understand thanks
This is a nice way to understand what projection does but it doesn't explain why the dot product is used, why it is used the way it is and why it works and makes sense at all.
really good
@@group555_ the dot product is basically asking this question, you and your friend pull a thing really heavy and you ask yourself: Does this guy help me at all or just faking, the value reflect the "help", and why projection , because the use of cos(x), cos(x) basically mean x direction loosely speaking, you can check more on Unit circle . Projection is basically mean take cos(x) of the other one and why cos(x) is used because it reflect the 'true' value of the other one ASSUME they are one the same vector, in this case x.
I'm saving this playlist for my children.
Hope you ain't to hard on them ;)
7:02 i feel stupid for just realizing that theres always "three blue and one brown" π in those talking animations
Oh my god I just realized that too
I feel more stupid now I understood this
It's also his channel symbol, a circle that's 3/4 blue and 1/4 brown (3blue1brown) which is all in turn referencing the fact that his eyes are 3/4ths blue and 1/4th brown. kinda neat lol
Actually it is 3 brown and 1 grey
Lol , I get it now 😅
This series of videos is going to last. I think presents an almost purely geometrical explanation of linear algebra that can't be found anywhere else at the moment.
I believe this could be done with every math subject. As an undergraduate student of mathematics, I often find myself struggling to translate what the books say into a more intuitive mental image of what's going on. Of course, some subjects (e.g. linear algebra) are easier to visualize than others. But then again, mathematicians always try to build mental images of what they're working on, even if it is a very abstract subject. These mental images are crucial, and in my opinion they should be taught along with the (undeniably necessary) formal arguments. These videos show that animations are a very powerful tool to do this.
All of this could be said about your Multivariable Calculus Course as well.
Now, I realize it is a lot of work, but it would be great if you did the same thing with some other undergraduate subjects. Have you already considered this? If so, what subjects?
Thank Alvaro! I'm definitely planning to do more of these. Long term, it would be cool to tackle all undergrad topics one by one, but the ones closer to the top of the list right now are calculus, real analysis, and probability.
Calculus and real analysis? What? I was under the impression that what we call analysis in our courses is called calculus in English. What is real analysis, then?
The word analysis, or analyse in english is just the process of understanding something, through the application of "pure" (i.e. undeniable) logic. Real analysis, in mathematics, refers to trying to define properties of the "real" numbers (i.e. numbers with potentially unending digits) and functions of those numbers. Calculus is the numerical/analytic description of change. Although often this results in functions of the real numbers, there are other uses for calculus outside of the real numbers. I imagine it must be quite difficult to learn about mathematics in another language, as the english part of mathematics, is often ill formed to say the least...
I'd love for you to go into hyperdeterminants and multidimensional matrices!
+Alex D. Bryan you seem to misunderstand my situation, or is it me misunderstanding your comment? I study maths in German (mostly) which is my mother tongue and the occasional course is in English, but that's not a problem for me. It seems, though, that our university lumps analysis and calculus (and topography) into one 2-year course named Analysis 1 through 4. So I didnt know how to distinguish between the two. The language is not the problem.
After viewed almost 10 times, finally I interpret the essence of this episode .
You got me beat. I watched it probaby 15 times to understand what he was saying lol
Yep, I'm going to try the fourth time.
I thought i was alone but now i am happy
哈哈哈
I did the same with the determinant video, I could never understand how the 3D determinant formula actually worked
For anyone who's lost like I was hours ago, I think I'm beginning to grasp it.
See the number line as a 1D vector space with basis vector u-hat, where u-hat has coordinates ux and uy. The basis vectors i-hat and j-hat define the 2D coordinate system. The coordinates ux and uy are projections of u-hat onto the x-axis and y-axis respectively.
Say you want to transform this 2D space into the 1D number line, such that L(i-hat) and L(j-hat) are projections of i-hat and j-hat onto the number line. You'll need a 1x2 matrix that will look like [L(i-hat) L(j-hat)]. As I wrote earlier, ux and uy are projections of u-hat onto the x-axis (i-hat) and y-axis (j-hat). Because of symmetry, the projections of i-hat and j-hat onto u-hat (the number line) will also be ux and uy. So, L(i-hat) = ux, L(j-hat) = uy! The matrix will be [ux uy].
Note that ANY vector, v, in the original 2D space is a linear combination of the basis vectors i-hat and j-hat. After the transformation, this still holds true: v = c1 i-hat + c2 j-hat and L(v) = c1 L(i-hat) + c2 L(j-hat). So, because L(i-hat) and L(j-hat) are projections of i-hat and j-hat, L(v) is a projection of v onto the number line!
Now, the vector equivalent of the transformation matrix [ux uy] is just u-hat. If you take the dot product between a vector v and u-hat, you do exactly the same as you do with the transformation; you project v onto u-hat!
So, to generalize: Taking the dot product of vectors v and w is equivalent to transforming the vector v by the matrix [wx wy]. But also equivalent to transforming the vector w by [vx vy].
Clear, simple explanation. Thank you!
That's it, it took me three replays to understand it lmao
Fantastic summary. Thanks! 💜
Your explanation makes it easier to understand. Thank you.
Excellent and clear explanation
To those who don't quite get the bit about the duality and how dot product could occur from that, after hours of thinking, I think I might have figured out a more direct explanation that could help you.
7:48 || 3b1b takes a copy of the number line and pastes it on the two-dimensional matrix grid, such that it is slanted and the "0" is at the origin. It is important to note that U-hat (the unit vector of the new number line) has the same length as I-hat and J-hat. We will see why in a moment.
7:50 || 3b1b explains why the new number line is a legal move, and that a function that converts 2-D vectors into a number on the new number line exists.
8:53 || 3b1b tells us that it is important to know where I-hat and J-hat land in the new number line. He explained why earlier, from 5:21 to 6:18. To add on to what 3b1b said, knowing where I-hat and J-hat lands in the new number line allows us to define ALL vectors that exist in the 2-D space just as they are in the form a * I-hat + b * J-hat. Suppose L(V) is the function to transform a given vector V onto the new number line, where V is a 2-D vector of the form a * I-hat + J-hat. Let L(I-hat) and L(J-hat) be where the I-hat and J-hat lands on the new number line respectively. Thus, L(V) = a * L(I-hat) + b * L(J-hat). a and b are given from vector V. If we can find what L(I-hat) and L(J-hat) are, then we know L(V) is possible.
8:59 || 3b1b explains to us that an arbitrary U-hat (with length equal to the lengths of I-hat and J-hat) with an x-component of Ux and y-component of Uy, has L(I-hat) and L(J-hat) as its x-component and y-component respectively. This proof can only work if U-hat has the same length as the lengths of I-hat and J-hat. Suppose otherwise, then the line of symmetry would no longer exist and the proof no longer exists.
10:07 || 3b1b shows us where the dot product comes in.
At this point, 3b1b has shown us the dot product between U-hat and another vector, say, V, is really just V projected onto the number line that U-hat sits on. We are meant to prove for two vectors, say V and W, and not one of them with an arbitrary unit vector. At 10:33, 3b1b tackles this by considering projections onto non-unit vectors, i.e, projecting V on W or vice versa.
He said that we should scale up the U-hat by a factor. In his example, he used 3. This took me a while to understand, but I think I finally got it. I don't think he explicitly stated this, but the new number line with U-hat, should lie on one of the vector you are trying to project on. Let's say that we want to project vector W on vector V, then the new number line should be on vector V. We need to use a unit vector, in this case U-hat, for this to work. This was explained in 8:59, I have written about it above as well.
Now, we have a projection of vector W on unit vector U-hat. However, we want the projection of vector W on vector V, and not its unit vector. This is where the scaling is necessary. Since U-hat is a unit vector of the number line that vector V sits on, that would mean that vector V is a multiple of U-hat. We know this is true because they are perfectly coincident, i.e, they share the same gradient and points. Now that we know that vector V is a multiple of U-hat, how much do we need to scale U-hat by to get vector V? The answer is just the length of vector V divided by the length of U-hat. It is like when a * b = c, and you know a and c, and b is the scaling factor, then you can divide a from both sides to get b = c / a. Recall that the length of U-hat is the same as the length of I-hat and J-hat. I-hat and J-hat both have a length of one, therefore, U-hat also has a length of one. Thus, we need to scale U-hat by the length of vector V divided by one, i.e length of vector V.
For the dot product, I am going to be using the asterisk (*). So A * B is the dot product between A and B.
Currently, we have (U-hat) * (vector W), or [Ux Uy] [A B] (A and B are meant to be on top of each other, A and B are arbitrary letters that I have picked to represent the x and y components of vector W respectively). Scaling up U-hat by length of vector V (denoted by |V|), |V| x (U-hat) * vector W. This becomes |V|(Ux x A + Uy x B) = |V| x Ux x A + |V| x Uy x B. Let C and D be the x and y components of vector V respectively. C = |V| x Ux and D = |V| x Uy. Finally, we have AC + BD, which is the outcome of vector V * vector W.
The new number line can be placed on vector V instead, and by the same arguments, we should have the same results.
To summarise, (1) get U-hat and place it on one of the vectors, say V. (2) Project other vector onto U-hat, so that it is projected onto the new number line. (3) Scale U-hat up to get the original vector V. If it helps anyone, I could have this down on paper and link it here.
Thanks bruh
Why did he call "u hat" a two dimensional vector when it is the unit vector of the numberline?
@@themangobui1474 because in the normal orthogonal coordinate system it's a 2 dimensional vector. We are relating the two dimensional space to the one dimensional number line, quich in the two dimensional space is pointing in direction u. To me it seems that the vector u in 2d space has the coordinates (0.5,sqrt(0.75)). Try to draw it and youll see!
Finally! Thank you very much for the explanation
Man, even with your fancy well-animated and explained examples, I was having a very hard time to understand this concept and took some days when I finally got it. No, it's not your fault. I think it's simply because english is not my main language so my learning curve rely more on visual examples than anything (my english is not poor, but it's just not fluent enough to understand everything at the same pace of natural english speakers).
However, I took this difficulty as good thing, as I REALLY learned all the subject of your linear algebra series by wathing every video multiple times.
I just want to thank you for your effort to bring us this different method of teaching math which is actually much more stimulant than the old fashion way most of us are used to (copy and paste formulas). And your passion about how cool is this subject gives me even more inspiration to learn. Thanks from a brazilian guy.
TheAwakeningMission É BR no bagulho! Tamo junto! xD
TheAwakeningMission same condition here bro ...I was just thinking that I'm too dum to under stand this concept but as u said I also have this problem...His linear transformation video I have to watched 4 times to fully understand. sometimes I feel to give up bcz I don't understand after so many times ...But truly I want to learn this concept .....
And that's okay, keep going :) Sometimes the brain needs a day or 2 to wrap itself around some idea.
If you don't get it in the beginning, that doesn't make you stupid. Stopping to try to understand is what makes you stupid.
LunnarisLP -- As someone who only speaks English, I can tell you that your last sentence is (unintentionally?) very funny.
have the same problem man. can you direct me on what to do when i have this problem. i mean what have you done to make it easier for yourself? have you just been watcing theese videos multiple times, or did you write something on paper?
Sincerely, this series of videos has taught me more than the whole Linear Algebra subject I took many years ago when I was studying Computer Science. It addresses precisely the most important problem I found back them and that is the lack of geometrical interpretation. Now I'm a Postdoc in Computer Graphics and still find myself finally understanding some details here and there. Thank you very much. I would gladly pay tons for something like this applied to more advanced concepts.
I think I am starting to understand it now. It really helps me to see it this way. I finally got more than halfway through your series and my brain is finally starting to put pieces together. I will keep watching and re-watching until it sticks. I'm gonna try to solve some problems now to see if I can solve them and also understand them better than I have before. Thank you for these!
I d say do khan academy first then watch these...
It took me a while to understand the point he was making(with regards to how projection occurs), but it makes sense in the context of previous chapters:
_(3-D Example below, but it generalizes pretty intuitively imo)_
1) *Linear Transformations and Matrix Multiplication→* Multiplying two matrices can be conceptualized as applying one linear transformation after another. I will make sense of the dot product in this context.
2) *Column Space→* The non-square vector on the right side of the dot product has column space one(rank = 1); thus, applying it as a linear transformation will shrink your original vector down into 1-dimensional space. → This is what Grant was getting at using û as the basis vector for 1-dimensional space.
3) *Basis→* The non-square matrix on the left can be conceptualized as the linearly-transformed î-intermediate, ĵ-intermediate, and k̂-intermediate coordinates(where the basis vectors land during the second linear transformation). The non-square matrix on the right can be conceptualized as just 1 basis vector(û) with 3 coordinates of landing.
4) *Dot Product→* Conceptualized as the projection of a 3-d vector onto 1-d space(the right matrix) where the product of transformed basis vectors(left matrix * right matrix) determines the extent to which each dimension contributes.
*What this means→ *
1) The dot product represents the extent to which 2-vectors combine in each dimension, represented by a singular value.
2) Column vectors can be conceptualized as a linear transformation from n-dimensional space onto 1-dimensional space.
I love the small pauses that give you a chance to reflect on maths' significance and beauty.
Yessss!❤
Hey pal, you did understand the topic. Would you mind to explain me since I'm having trouble?
I just stare into space thinking "maybe I should rewind a bit" 😂
Holy moly, this was deep. I feel as if I don't have to memorize anything for my upcoming linear algebra test -- it's all intuition! Thanks so much!
@Robin Rastle 👍
must have had an easy professor. it’s still all greek to me - and not only because Pi is teaching multiple Pi’s.
7:22 "unlearn what you have learned" makes me want to smash the like button more times than RUclips will allow, Master Yoda.
I appreciate that you use a black background. Its easy on my eyes because i get headaches very easily
It took me this much to realize why your function always makes
2
7
To [1.8]. It's e 2.71828...
so you chose the red one
Welcome to matrix
(7+2)/(7-2)
@@svsrkpraveen could you please explain the meaning of this quotient? Thanks!
@@f1cti 9/5 =1.8
You've done a truly excellent job with these videos. The geometric insight into linear algebra is wonderful, and I say that as someone who used linear algebra almost daily during my career at NASA.
i know this has been five years but out of curiosity what do you, sir, do at NASA?
If anyone's having trouble understanding, this is how I've come to think of it after watching the video a few times:
1. To turn 1 unit (u) of the number line into a vector, you'll have to project it from the number line onto the xy-plane. That projection will be marked by u's coordinates (ux, uy), as shown at 9:30
2. To go back to the number line from the xy-plane, move ux units along the x-axis, and then move uy units parallel to the y-axis, and you'll get back to u on the number line.
3. To turn any vector on the xy-plane into a number on the number line, you'll also have to mark its location in the xy-plane by taking note of its coordinates, (x,y). Then if you scale those coordinates by u's coordinates, you'll be able to get to the number line by walking x*ux units along the x-axis, then y*uy units parallel to the y-axis. And x*ux + y*uy is basically just the dot product of [x, y] and u.
Very good explanation. That does make everything a bit clearer.
The last part (you will be able to walk to the number line) is wrong, the dot product just gives the value on the number line directly.
I went through it for more than 1 hour pausing at each point made. I have never attended college and learned through textbooks recommended by first class colleges and the Khan Academy teachings (in particular) to get my engineering credentials for a living. I jammed into my head all these math proofs like an Ape and try to relate them to the real world of tangible things. Thank God that I found You to light up the bulbs and cast away the shadows (vague conjectures of realities).
Example on dot product. You sum it beautifully as a linear transformation from 2D space not defined as numerical vectors but projecting space onto a diagonal copy of a number line. I take this and others as wings to fly now.
Yes. I signed onto Patreon as a small token of appreciation.
The last four minutes of this lecture are absolutely beautiful!!! Thank 3B1B for visualizing these and making such beautiful connections between different concepts!!!
In computer graphics, I think there is a _far_ more intuitive way of understanding the dot product - it is a measure of similarity between two unit length vectors, 1 when identical, 0 when orthogonal, and -1 when perfectly opposite. (This can be intuited by the fact that a . b = |a||b|cos(theta)). This comes up _constantly_ when writing lighting shaders, since the amount of diffuse light that bounces off a surface is generally L . N, where L is the direction from the surface to the light source and N is the normal of the surface. (This is fantastic because it has a direct visual interpretation.) Also, interestingly enough, I believe Google employs a concept of "cosine similarity" on vectors of words when ranking websites for search.
Or just think of it as the length of a force vector.
Yep. I think of dot products as a measure of how much two vecors are going in the same direction, with the cosine of the angle between them acting sort of like a percentage that can also be negative.
Cosine Similarity is what used in many Recommendation Systems too!
The dot product can also rank the similarity of vectors in any dimension. For example: in natural language processing, you can create a vector full of numerical context values for each of the words in a sentence, or even a paragraph. Then you're taking the dot product of two vectors in dimensions upwards of 100, and using interpreting the result in the same way. The intuition comes very nicely with geometric interpretation, but spans much further.
Thanks. This made a lot more sense to me
Had to watch this multiple times to understand.
Congrats! You blew my mind.
I've watched it 5 times and I'm still not able to understand what is the use of taking a dot product. Please treat me like a 5-year child and give me some real world example. Thanks, brother
@@imrsk Sorry late comment but read @Bhargavi Suhas' comment down below.
For anyone who is still struggling to grasp, what finally made this concept click for me was to think of the "dual" of the vector as the UNIQUE LINE in space that now represents your 1D number line. A number line can live anywhere in a 2D space (a 1,0 number line would be the straight line along the X-axis, a 0,1 number line would be the straight line along the Y-axis, or the U-hat number line that was diagonal through the origin looked like coordinates of ~(0.5,0.5)) So, 2D space in a sense remains absolute. When we project onto a 1D number line, we're essentially saying "this is your PARTICULAR 1D "playing field" (number line) right now and it lives in a unique place within 2D space" but there are an truly infinite number of 1D number lines capable of being generated from the 2D space. The three examples I gave are just a subset of all the possibilities.
Thus, because each number line is unique (the 1,0 number line doesn't LOOK like the 0.5,0.5 number line in 2D space), each projection onto 1D space is unique, as represented by the unique 2D vector that codifies where the number line "originally lived" in 2D space.
i feel we are saying the same thing it but I've worded it different. Each vector can be seen as a line/arrow in 2D space, going on its merry way. A vector is also a transformation that can squish 2D space to 1D(by taking any other vector and dot product with it). Interesting to note that once we see the vector as a transformation, it no longer makes sense to think of it as a vector, since 2D space is now a line.
i like the "originally lived" - once that vector is seen as a transformation, there is only a number line... no space.
Quick caviat: not all 2d vectors transform into unique number lines. In fact, there are an infinite number of 2d vectors that could be transformed into EACH number line. For example, [1, 1] would be the same as [2,2]. For any two 2d vectors W-> and V-> where W-> = [Wx,Wy] and V->=[Vx,Vy], if Wy/Wx=Vy/Vx, they will transform into the same number line. This becomes intuitive if you consider that if you have a point (x,y) on any line that crosses the origin, you can calculate the slope of that line as y/x. So basically, if the "slope" of the two vectors is the same, they will transform into the same number line.
Thanks for this comment. Somehow this just helped me 'click'. Kudos!
How was the i and j projected onto a number line and got new basis (2, 1). Can anyone pls help me with this. I don't understand it.
Funny, how the things we have been taught for years finally makes sense. Beautiful work.
I've been a graphics programmer for over 8 years, I've also written countless collision systems, animation engines, physics engines, and your videos are still blowing my mind.
How is this not the standard way of teaching linear algebra in school/uni?
Simply because is a huge amount of work to make videos like these and teacher don't have time to make them at a large scale.
This is just . . . Beautiful! I just feel so lucky to have discovered your channel, because, no teacher, no professor, no matter how we advance in our fields that are related to maths, explains like this and shows the deep essence of the science (except if your main specialization is mathematics).
Gonna have to rewatch this one a few times before i understand what is going on.
I can’t know how to thank you more. Those videos are brilliant! As someone with a computer science PhD, linear algebra never looked that meaningful to me before. You cannot guess how many “aha” moments I got during your videos. Before anything else, these videos should be shown to students in every LA course all over the world. Thank you!
aaaaand this is where you lost me. lol. back to review i go.
aaaand a week later and i'm still lost.
lol yep it was all going so well until here..
same .I still dont know why i should project a vector onto anoter in the first place.To know if they point in the same direction.Y is it so useful or why do i have to project them
To Ismir Eghal, I hope this helps: Imagine you were imposed upon with the task of pushing a large boulder up a hill. You and I know the steeper the hill, the tougher the task. That's because gravity is exerting a force on the boulder. It turns out it makes more sense to think of gravity, not just as exerting any ol' force... but a "downward force." We can think of gravity here as a vector pointing downward with the size of the vector reflecting gravity's (CONSTANT) strength.
Notice something subtly odd here, changing the incline of the hill changes the difficulty of the task even though *gravity itself remains the same* .... What's going on here? Well, to push the boulder, you must apply a force to it and that force, like gravity, has a direction. Notice how, if you want to push the boulder directly up, you must fight 100% against gravity. If the hill lacks steepness at all, gravity plays no role in the task (it's essentially only friction making your job tough here).
You may begin to wonder if there's a precise way to describe this phenomenon. The more your force is against gravity, the more gravity fights it. If the ground is flat (so you're pushing *perpendicular* to gravity), gravity doesn't make your job tough at all....
Can you figure out how the dot product might be of some use here...?
So if we consider that the gravity vector and the vector along which we are applying the force to be perpendicular (meaning flat surface) we have to put in zero effort, so the dot product is zero. Correctme if i m wrong.
Hey Grant I know you're not gonna read this but you're my favorite person after Einstein
Thank you. Thank you so much! Dot products finally make sense on what they physically represent. However, I will say that this video is presented in a confusing order: it mentions that dot products relate to projections before proving why much later. I had to watch the second half of the video first in order to understand the first part's discussion of why order does not matter.
This one took some time to wrap my mind around but it is truly amazing! Thank you for this series, it's a great help to develop an intuition about linear algebra and all its applications.
Woow just wow
I have seen this video so many times already and it just keeps blowing my mind
This is so beautiful
I hope that second series is coming soon for all the Jordan, IP and spectral stuff
Thank you very much!
This is the most enlightening video I’ve ever watched on YT and probably in my entire life... I still can’t stop smiling :’) thanks a ton, Grant! 🤲🏼
Me too!!! 🤯✨🕊️
Every time I re-watch this video, I feel happier and more realized, I feel hope again, and I am not exaggerating. Thanks for your work, sir.
Your channel should trend monthly. I want to believe there is a reason why most people find math repulsive. A reason somehow more meaningful than the way it is taught in school. I think that knowing these types of things gives you a satisfying and powerful understanding of almost everything around you. Maybe through some divine phenomenon, that power is kept in the hands of few people, because if everyone found this stuff easy, well, they might not use it for good. But then again what am i saying think of how advanced we would be as a civilization there would be no room for evil. I dont kno
Man, this is the best explanation I have seen for Linear Algebra, excellent series, I can't wait for the next video. It felt great to find the series just at the moment I'm starting to learn Linear Algebra at the University, it has helped me to understand the true meaning of what I have been learning. Keep it up!
Sal introduced me to math, 3Blue1Brown made me appreciate and fall in love with it
I am surprised how these 15-20 minutes of videos are teaching me more than my high school courses that lasted for more than 9 months.
If this feels a bit overwhelming, ask drunken Grant to explain this to you. You reach him by setting the video speed at 0.5x. He's just as smart as regular Grant but has a more laid back style.
I love this comment, idk why it is on the top when it only has 4 likes, but i love this comment
You get my like
Great video! I'm now taking functional analysis where duality is a crucial topic. This video is incredibly helpful for building my understanding. Returning to intuitively simpler vector spaces and understanding duality here helps a lot.
Currently learning about this in much less detail in my trig class, but I really appreciate the number line translation you're explaining here. I typically need to hear/see a concept a few different ways to really get it hard wired into the brain and this number line translation helped a lot, thank you!
This is the BEST explanation I’ve ever seen. This is exactly what’s happening behind primal-dual optimization. Thank you!
Ok, I love the series so far. Chapter 7, however, took me by surprise. I spent three days and scribbled out 17 pages of notes trying to assimilate Chapter 7. I still catch myself daydreaming, trying to visualize the contortions those poor unit vectors must go through to exit 2-D space and find their new home on the number line. I admit I'm a little fearful of tackling cross products in Chapter 8.
SierraSlim1 Hi, bro! Can you send me a copy of your notes? I feel the same as you about this subject. Greetings from Brazil! Thanks!
It's really just about using an ordered group of numbers in two different ways.
You Sir/Ma'am seems to be a wonderful writer. And your comment is absolutely relatable. Until now I thought I was the only one spending entire couple of days on a single video, taking notes, uprooting hairs and slowly slowly visualizing every aspect of the explanations. Seems I am not alone. I hope you have completed the series by now and have absorbed all its beauty. Will take me couple more weeks to do that.
thanks for the commenting this. untill now i thought i was the only one who spent a lot of time in a single video but your comment gave me hope.
Yep chapter 7 was a little more difficult than the others and needed some time to comprehend.
These videos are amazing!
My only criticism is that you started using "projection" and "unit vector" without fulling explaining what they were.
As someone with no background in linear algebra, I had to look these up.
Similar here: I thought I was following this series quite well, but then he threw in projection without really explaining it, and I was lost for a bit. I did Maths A-Level but don't recall learning anything about projection. Better go look it up!
Yeah same here, although I think unit vector was explained earlier referring to i-hat and j-hat, but with u-hat I'm not sure what is means. I'm thinking it's a vector with a magnitude of 1.
@@inxiveneoy that’s exactly what it means yes
4+ years later...still my favorite video on RUclips.
I am simply overwhelmed by the clarity that I get after watching your video. Inspiring. Keep making these videos. Please.
I wish I watched your videos back when I was taking my linear algebra course. Life would have been really simple! You are an Ubermensch by all means, Cheers!
Suddenly, the way the dot product was first introduced in my maths textbook - the length of the first vector times the length of the second vector times the cosine of the angle between them - makes perfect sense!
Nillie this part still didn’t click to me
Mariusz Wiesiolek as we know, cosine is the ratio of adjacent side to hypotenuse. You can think of w as the hypotenuse and of its projection as of this adjacent side. Since we’re projecting w onto v, then the angle between v and w is the same as between w and its projection, therefore we can use the cosine. Hope you understand my attempt at explaining this since English isn’t my native language.
Good explanation Mark, I too was confused about that point. My understanding is that if the vectors are forces then the dot product is the resultant force (a*b*cos(theta)). The resultant force ends up somewhere between the two vectors (e.g. 2 horses pulling a barge), it doesn't end up going in the same direction as the other force - as it seems from the video.
@@helenkirby2539 I'm not sure if your comment refers to your prior understanding or your new understanding. If it's the latter, it is not correct: If the vectors represent forces, the resultant force is the sum of the vectors.
The great math class I never had in college, so insightful, amazed by the way you described those math concepts in a tangible way. Great work!
The most intuitive video I have ever seen in my life.Thank you
wow, just wow.
this was amazing. I never understood why scalar product actually works...
U demonstrate one thing: There are no bad students. Only bad teachers.
All the people who thought they hate math, will definitely change their mind after watching these videos.
Respect, for all the time and efforts u put into making this available 🙏
Well done guys, this just blew me away! I have struggled for a time to really intuitively understand the notion of a vector space and its dual. The way tensors and forms are framed in this language is something I've used for a Long time but never truly understood intuitively. This video (and the one on determinants) was an epiphany for me and everything I've struggled with (tensors, forms, differential geometry etc) just fell into place. I feel humbled that something I thought I knew so well, seen in a different way could illuminate so much more. I cannot recommend these videos enough.
Bravo sir. This was absolutely brilliant! Having a degree in math from one of the most trending universities, but seeing the first time in years this actually not-so-simple concept the way it was intended to be... Here is my deepest bow.
I push my fingers into my eyes
It's the only thing that slowly stops the ache
I’m not gonna make it
dat dat dat, dat dat, dat-dat-dat
@@Anthony-db7ou No... but it's made of all the things I have to take
WANYK
I'm no mathematician and I'm just a dilettante when it comes to the subject matter. But boy is it interesting. I always had respect for math and my math professors. Got me all the way to calc III. Love this series of videos.
Also completely approve of the Calvin and Hobbes dialogue at the beginning!
But maths is completely unlike religion.
When you have two apples and get two more, you have four apples. When you have two dollars and get two more, you have four dollars.
2+2=4 is just an abstraction over this observation. You don't need to believe anything. You don't even need to believe an axiom in order to work with it.
because all religions are burdened with superstitions, religion has almost become a synonymous of "believing blindly in something".
Actually religion means "binding together" "bridging yourself with the universe"/"realizing the unity of everything" (not believing in it, but Realizing it, experiencing it)
Pseudo-religions like Christianity have been preaching belief and have been exploiting people, and that is so stupid and ugly. Although blind belief and other such things have nothing to do with religion, they have been happening IN The NAME of religion, that's why the association (in our minds) is so strong.
You can observe that 2 apples + 2 apples = 4 apples , but essentially this is very mysterious. Everything in this world is quite mysterious and will remain mysterious no matter how much we sugar-coat everything something with explanations. Explanations are utilitarian. They are very useful since they can provide frameworks on which we can create technology, but explanations should not have the pretension that they represent ___the supreme absolute "truth" that must never be doubted___ or that they demystified existence.
These pretensions are dangerous, because then people will behave exactly like the religious fanatics do, but now IN THE NAME of science.
PS: Mathematical proof that you can't demystify existence:
If you want to explain what A is, you'll have to use another term B. But then you're caught in an infinite loop, then you'll have to demystify B as well. You'll have to describe what B is: "B is C and D in such and such way", and then you'll have to explain C and D and so on. All explanations of an object/phenomena X are expressed in terms of other object/phenomena Y1,Y2,...YN,... . If you managed to explain away an infinite number of phenomena, you'll have created an infinite more phenomena which are waiting to be explained.
PPS: I'd like to hear some criticisms of the above proof.
nobody needds your approval, you dumbass furry
completely disapprove mate!
I've had rough 4 years of undergraduate studies and got two more semesters extended. In my last semester before graduation, watching your videos on vectors gives me strength, 'cause I'm taking Vector Calculus. I would get nothing out of my course, if not for your beautiful explanation and eye-opening visuals. Now I am sure that if I can understand these videos, then practically anyone can enjoy, let alone learn, mathematics. Thank you so much!
P.S. I keep rewatching your course playlists to root the concepts and entertain, as well.
You, man...You, just make incredible videos on mathematics...I never thought all of this could ever have geometric interpretation. Hats off to you.
The definition you gave of dot product is only valid under the right hypotheses, which are not always validated in linear algebra since you could take any base of vectors for the set you're representing. Instead, the more general geometric definition gives a clearer view of the geometric properties of the dot product, and makes the formula you give trivial to find under the right circumstances. Great series, thanks a lot for your work.
Now I know why dot product of two perpendicular vectors is 0.
Such an amazing series, loving it.
This is absolutely great! I think I see for the first time "intuitive" explanation for what det and rank is. I took linear algebra course at the university but we were just using them to solve systems of equations. They just were. I feel almost betrayed that nobody told me this explanation then. (and nobody mentioned anything about eigenvalues !!!) This series is just wonderful! Keep doing the great stuff! And I really like it is strongly focused on essence not on computing: computing I can find anywhere.
Wow! This one was quite tough but really awesome once I stopped to think about it. The toughest bit was understanding why exactly duality holds for non-unit vectors, but I think I've finally got it. Thanks for this series, it's great and really approachable, like most of the stuff on your channel.
I really wish they'd emphasised these more in school in the 90's, given how important they are for game programming today.
This is real beauty , my eyes tears sometimes from such elegance !
Absolutely bloody amazing, it took me almost an hour to pause and rewind multiple times throughout the video, but now that I GET IT, i feel enlightened!
Your teaching is unparalleled. The best explanations, with the animations to build the best intuition. When I think mathematically, I think in terms of 3B1Bs animations.
Something eluded me the first time a saw this. However, after learning about "change of basis" in a rigorous manner, I think the explanations that used "symmetry" and "projections" finally clicked. The diagonal blue line that is spanned by u-hat is a vector (sub)space U; the projections are linear transformations. Since the length of u-hat, i-hat, and j-hat are all the same, then the length of the projection of u-hat to i-hat is equal to i-hat's projection to u-hat, i.e. u_x. Note that the projection of i-hat to u-hat is a projection of i-hat to U. Thus, in a "change of basis", i-hat has a "coordinate" of u_x in U with respect to U's basis which contains one vector, that is {u-hat}. Similarly, the coordinate of j-hat in U with respect to {u-hat} is u_y.
I'm not going to lie. This one fried my brain a little bit. I need to take a nap then come back to fully process this haha. It seems amazing though.
Only looked this up in preparation for my linear algebra intro course, now I am fully subsumed by the beauty in mathematical manipulations themselves..because of you Grant!!
I've come back to this video countless times, and each time it helps just as much
Riesz theorem spotted! An amazing video, always teaches me something new
I watched this video 3 years ago, am learning about it finally now. This gives me SO MUCH intuition
Thanks, now I understand.
Also, a vector is a n x 1 matrix projecting 1D space into n-D space with Rank one.
This is amazing! I've been wondering about these things for a while and this video just explains it so clearly.
you nailed this one! i also learned this concept almost at the end of my lineair algebra course, it just makes way more sense that way, and you can really dive into the concepts of inner product spaces
This is so unfair. All my life I suffered with linear algebra (I am into data science) and only now the does the author post these videos. :P
Saptarshi Mitra just curious .... r u watching this after completing University ?
Saptarshi Mitra: also curious. Did you start Linear Algebra in kindergarten? Cheers!
@@robinswamidasanUse common sense, dude. If the guy says he spent his entire life around linear algebra n is into data science(probably data scientist) & given that linear algebra course comes only after 12th, then *it's obvious that data science has something to do with linear algebra* 🙄
One easy way to understand dot product is to relate it with the formula of _cos(a-b)_ , i.e. _Cos(a-b)= Cos(a)Cos(b)+Sin(a)Sin(b)_ if you replace *Sin(a)Sin(b)* with *Cos(90-a)Cos(90-b)* which are essentially the angles the vectors make with y axes if a,b are angles with X-axis. This is exactly the same as dot product of the vector matrices of two unit vectors, as in both cases we are multiplying X and Y component of unit vectors and adding them. You can look up the geometric proof _Cos(a-b)= Cos(a)Cos(b)+Sin(a)Sin(b)_ and think of the line segments as unit vectors and you will understand that we are dong exactly the same thing in both the processes but under different names, that is line segments and vectors. You can do the same for higher dimensions.
That's a great way to remember the trig identity too!
Can you please tell me how to get the "length" of a vector, projected onto another vector in a plane???
@@-Momin literally just Google it. I would help you but I would have to Google it anyways lol. Just Google vector projection
I opened 12 tabs with different websites explaining dot product. 1 minute into this video, closed the other 11 tabs. A big fat thank you for helping us realize the 'why' part of it!
This video is a gem, this is what is needed in education, I understood each and every part and really say this is never taught it is also hard to teach these topics without actual visualization, there were parts where the graph was not meant to be scaled, I also like the simplicity and focus not giving irrelevant examples, when I will grow I will surely donate to this channel so that it can keep up with these videos for the future generation, I am still 16 and fully support this guy, Hats off huge respect from India sir🎉🎉🎉
you're a RUclips God, thank you
i found 9:30 super hard to understand visually with what is being said. What i see happening is that to project i-hat onto u you make a 90 degree line from u onto the head of i-hat. To project u onto i-hat you make a 90 degree line from i-hat that crosses the head of u.
Super great videos and representations. Its a very nice way of thinking about the cases
Me too
9:22 and 10:11 are awesome moments of realizations. This episode is by far the one that most challenges my abstract reasoning ability. Thank you so much for the videos!
Amazing, These videos really make concepts much easier to understand. Duality is really beautiful.
After completing this series, I might suffer from post anime depression
2:37
My intuition for "Why the order doesn't matter" was to consider the area of the triangle bounded by the 2 vectors. What all of this means is that you get the area of that triangle by 2 ways.
The area of the triangle bounded by the 2 vectors is half the absolute value of the determinant of the matrix of the two vectors (in 2-d) or magnitude of cross product (in 3-d).
You blow my mind, this series is gold
I love this playlist. Absolutely love. I downloaded the whole thing and I am re-watching it to help myself digest the topics of vectors and matrices. Beautiful stuff. I would pay you but I am broke and in school. Hopefully when I'm older I'll still be following you and remember to give thanks cause you'd've helped me gotten to where I will be. Majored in Maths and Actuarial Science. In fact, let me go and Subscribe. Amazing I haven't already. Keep up the great content. Unmistakably magnificent.
This topic was a bit more complex compared to the ones till now in this series! :(
Basically what he is saying is that you can imagine V to be one of two vectors (V and W), such that a Line is collinear with the vector V, such that after linear transformations, the 2d space that has vectors (like V and W) is converted into 1d space and that space will be that Line (Collinear with V).
And the basis vectors of 2d space will land on the points like (i lands on V_x and j lands on V_y). Now when we take linear transformation of vector W, we will write
W_x.(transformed i) + W_y (transformed j) = W_x.V_x + W_y.V_y
This is similar to dot product and matrix vector multiplication.
I couldn't understand the basics of Support Vector Machine, until I watched this video. Thanks, brilliant.
This is a unique, incredible, and amazing course that will without a doubt remain. This is how Linear Algebra should be taught and it astounds me it's not taught this way. I thank you again, for you're amazing and have turned Linear Algebra into one of my favourite subjects from my least favourite
This was so amazing understood after 4 rewatches and it was amazing 1 hint if u dont get it, like every 5 sec of the vid, pause and ponder