You know what's crazy. I was pronouncing it the correct way for my whole life. But I watched like one video in research that pronounced it the wrong way and for some reason decided that they were right and I was wrong. So I decided to intentionally change how I pronounced it to the wrong way. I'm actually so dumb lmao
@@copywright5635 that's crazy; on a similar note I'm convinced that Italian physicists can't pronounce "Dirac" Correctly; no professor at my Uni called the poor guy right. (Hasty generalization I know, kinda funny tho)
Runge-Kutta still leaks energy. For the equations of motion, the one integration scheme to rule them all is Velocity-Verlet. That conserves energy _exactly_ (apart from floating-point round-off), is 2nd order, and is just as computationally cheap as Euler.
22 дня назад+15
To make it easy for others: en.wikipedia.org/wiki/Verlet_integration#Velocity_Verlet
Yes, I decided to leave off Symplectic Integrators. But of course, Velrlet and Velocity Verlet would have been better options. Seeing a lot of comments about this, I will probably do a sequel to this video in the future
Idk man seems youtube glitch or something but it's missing 'k' after 898 in your sub count hope youtube fix the issue soon, once again thanks for a great video
Great video! It's funny that 3Blue1Brown's manim environment became the official animation framework of RUclips math videos. Every time I see some video made by manim, before watching it, I know that it is gonna be a good one. It never disappoints.
This is a great intro to approximation schemes! Really well explained and I love how you included all the animations to visualize what happens for each method. It was very helpful to see how drastic an effect the lack of energy conservation can have.
Nice video! However, at 9:38 it should be noted that the "order" of a method does not refer to the number of terms/stages (k1, k2), but rather the truncation of the Taylor series. This means that a 2nd order method will exacly match the taylor series up to the x^2/2! term within each time step, while the following terms (x^3/3!,...) are not exact or missing. For some fully implicit methods (Gauss-Legendre), the order can be two times the number of stages. (They're computationally expensive and I wouldn't recommend using them a lot but provide inpressive results for large time steps.)
Thank you for mentioning this. I could have mentioned this, however, since I didn't go through the Taylor series derivation, I thought it would just confuse most viewers.
Nothing in the universe is "powered by differential equations"! Differential equations CAN DESCRIBE a lot of things. This is very important to understand.
I love this kind of videos. I work on driveshafts, which are rotational mass-spring-damper systems to some degree. I loved doing the Taylor expansion and wrote some homework about how much accuracy you gain for increased compute, as you increased the order.
Another method (or sub-method) that maybe deserved a mention is the so-called leapfrog integration, where the average derivative for xI is taken from an average value of the previous tick acc and an extrapolated value for halfway towards the next one. It's sort of similar to RK2, but the samples are offset back by one(half) of a tick. It's relatively stable, and unlike RK2, you don't actually need to computer the derivative twice for each tick, as the first one is carried in from the previous iteration.
Yes, this is also a good method. I just wanted to keep the video simple. Perhaps I should have teased the sequel video at the end. I'm getting a lot of responses about this, so I think I'm going to make a sequel to this video covering Symplectic Integrators, along with some others like leapfrog
The RK algorithms are a very fascinating topic, and I've even implemented a few of them in a C++ application before specifically RK4. Yet, I still feel or think that FFTs and their inverses are some of the most interesting algorithms out there. Complex Vector and Field analysis, the Hamiltonians, especially the quaternions and octonions, and so much more are all interesting topics. We stand on the shoulder of giants! I truly enjoy videos like these, keep up the great work!
Thanks! FFTs are a topic I think deserve a longer and more dedicated video, but I'm considering doing one on them. Even for this, I barely touched the surface level of what RK is (didn't even derive RK4 lol), just wanted to provide some intuition for it. Both for the motivation, and why it would be more accurate than a simple Euler-Cromer method
@@copywright5635 Not exactly but similar to why Quaternions prove to work better than Euler Angles when performing rotations in 3D about the independent axes. When using Euler Angles there is the phenomenon of producing Gimbal Lock within the use of the rotation matrices. This where one axis ends up being rotated onto another where they become coincidental and from there you end up losing a degree of freedom as the two axes are locked and you can no longer differentiate between them. Quaternions helps to prevent this. Also, quaternions, even though the mathematical notations and expressions are fairly complex, implementing them in software is fairly trivial and they also have a very nice added benefit of being able to be calculated against other vectors and matrices as well as being converted to them. Because of this, they are also computationally cheap, very efficient and quite effective. It's not exactly the same, but it goes to show where a variety of Euler methods although is simpler to digest and work out by hand, also has their shortcomings. FFTs just provide a very good and efficient way to transpose from one system to another especially when working with wave patterns or anything with a frequency domain. Without FFTs, audio processing either being a wave file, a midi file, or even an MP3 file wouldn't be as efficient as they are today. Audio even when compressed requires a lot of information and can be fairly computationally expensive. FFTs reduces that by a couple orders of magnitude. Instead of trying to perform 20 thousand sine or cosine function calls per second for a 20kHz frequency. We can just sample it and use the sample rate to reconstruct a good enough approximation of the individual waves. Well sort of, as that's the abridged version. But yeah, I find it all to be very interesting and intriguing. I'm not just intrigued with this type of stuff either. I'm also intrigued by 3D Graphics Rendering, Game Engine - Physics Simulations, as well as actual CPU Hardware Design (ISA design). Then again, this gets into physics when you go beyond the logical device level and get into the actual structure of the transistors, resistors, etc. that are designed to manipulate electricity. And here we are again, with wave propagation. Right back to the use of wave functions and the power of FFTs lol! I just like things related to engineering. Factorio, Satisfactory, Dyson Sphere Program, Oxygen Not Included, Planet Crafter, Mindustry, Turing Complete, etc. there're all a part of my Steam Library and are my hobbies. And I'm no stranger to Music as I did play the Trumpet for close to 10 years back in my school days.
If you like this and Fourier methods, you should check out dispersion and dissipation analysis (sometimes referred to as "Fourier analysis") for ode solvers (and pde solvers too, but that's a bit more complex). It essentially allows someone to understand how a solver will respond to any initial condition of a linear problem.
Wow, I honestly had no idea these methods were even connected! Thank you for the straight-forward explanation and visualizations. Top notch content, Sir.... 👍
Amazing video ! I loved your pace and your little jokes, it really helps staying engaged with your presentation. The visualisations are of course really good too. :)
2 minutes in, I can already say: I like your style! Just one remark: I think it would be better to clarify that we can best describe nature with the help of differential equations, instead of saying that nature is governed by differential equations (even if it was true that we live in a simulation, this would be outside the scope of scientific thinking). This reminds me of talking about charged particles "feeling" a force and thereby intentionally reacting to it, or explaining darwinism by active adaptation to a changing environment, or even "lonely atoms which want to form bonds to share electrons" to fulfill some godly given octet rule so that all of them can live a long and happy life, and every other thing our teachers tought us "although we should keep in mind that this is just a simplification" - although the concepts you are about to explain (from my point of view at this moment in time, at minute 2 in your video) exactly oppose these views, of course. But keep in mind: Now the maths begins, and view counts will only drop from here on. To be clear: I really like your video! Narration, animation, general style: wonderful! And I had a look at your channel, and I will watch a few more of your videos.
Thank you for the feedback, yes I should have been more clear about that. I was mostly focused on the mathematics here, but of course, everything physical we describe with mathematics is just a model. I'll keep this mentality in mind for future videos.
i think metaphysics is a part of science. many great scientific insights came from studying metaphysics. i.e. what is true , how much can we know of truth, and what is that truth composed of? and there maybe a correspondence between our models and reality, i.e. reality = diff equations, and it's interesting enough for you to bring up, and I think it's helps strengthen our mind's talking about this... and it can give us insights into math from a point of view of physics and vice versa.
1:09 this sound is equivalent to undertaker's unexpected entry during an ongoing wrestling match😅 haven't watched the complete video, I'll leave a 'review' comment after watching it but it seems video will be awesome and informative.
0:21 eveything in nature isn't governed by differential equations (DE), DEs describe nature, they don't govern them. I know it can be seen as a nitpik, but I felt that the semantic difference between 'governing' and 'describing' were big enough to warrant the comment. The rest of the video was great!
From my days in college, when I took Numerical Analysis, I had an idea. Do the iterative equation that, with each iteration, produces Output values relative to increasing Time… BUT… for each Output value, use it as a STARTING VALUE for the iterative equation method known as *Picard’s Method.* When Picard’s Method iterates, it STAYS ON a single Time value (i.e. as you iterate, Picard’s Method doesn’t “move you” along the Time axis.) I never got to try my idea, but I always wanted to. Overall, the idea is that you keep switching back ‘n forth between an Increasing Time iterative equation AND a Picard’s Method iterative equation. For each “invocation” of Picard’s Method, you perform enough iterations until a suitable degree of convergence is achieved.
This is great. You seem to be getting a bit of flak from the very knowledgeable on the subject, but I just feel the video is directed to those like me, who *in theory * can follow the equations but have trouble with the 'where are we going with this' part. And in that respect the video succeeds with flying colours. So, thank you!
You might think my comment is mean or unsupportive of your work. But actually I really enjoy watching your video. The problem is you didn't answer your title. You didn't explain why the method is better than Euler. You didn't derive it or show any reason why it is more stable. You just showed graphically how it plays out but never actually proved anything about it
Thank you for the feedback. I think your concern comes from a differing opinion on what "why" means. I do concede that the title is not true in a strict mathematical sense (I didn't derive RK after all). However, I did provide reasoning for why RK2 (and by proxy RK4) seem to conserve energy better than an Euler method. I could have mentioned Symplectic Integrators, or even the Euler-Cromer method (which only changes one term, yet conserves energy for these problems). RK methods also don't inherently conserve energy, they simply converge much faster. I approached this video with the notion in mind that it is unclear why for certain systems, RK methods, even RK2, seem to simulate so much better than standard Euler Methods. I wanted to provide a sort of intuitive motivation, and I think I accomplished this. You are correct in saying I did not mathematically prove anything about Runge-Kutta methods. This was never the intention. I apologize if you found the title misleading. TL;DR, The "why" in the title is not the "why" of a mathematician. It's the "why" of an engineer or experimental physicist.
Oh this video is unfortunately a little bit late for me, just had my numerical simulation exam last month ;) still watching this video since it got recommended to me and its so fascinating how people came up with such things decades ago!!!
It's clear to see that the higher order algorithms are more exact per timestep but they're also more computationally expensive because of calculating multiple derivatives per timestep. It would've been nice to see how exact each algorithm is per derivative evaluation. Because it might be more efficient computationally to use a smaller time interval with a lower order algorithm than using a higher order algorithm.
Very clear! Implicit Gaussian Collocation for the win though! (For numerically fully conserving skew-symmetric use cases.) EDIT: But you already mentioned Sympletic integrators in one of your responses.
@@copywright5635 Thanks! Subbed as I’m looking forward to hearing more of (sympletic) integrators. I’ve used Gaussian Collocation for simulating with conservation accuracy down to machine-precision (with help of skew-symmetric PDE form), which is cool, but I can’t say I really ever understood what is sympleticity means nor how to derive, e.g., exponentially sympletic variants.
Another integration method that you can consider is Verlet one, third order error for position and second order error for velocity. It is highly used in games, since we care also about object interactions and with Verlet this is really easy. We can enforce non penetration constraints without necessarily applying a the force on those objects, but just displacing their positions and still not completely break the system. Obviously non physical correct, but robust and somewhat believable.
I recall Euler's method being introduced to me explicitly as a tool that does not produce good approximations, but rather convergent ones, which is useful for proving existence of actual solutions.
Nice! I coded RK4 and other methods in python for a 3 body problem simulation. RK4 and Velocity Verlet were way more stable than Euler or even a 2nd order Taylor series when we consider conservation of energy. Thank you for the video;
Thank you so much! I hear about Runge-Kutta so often at the lab but never understood it until now! But it bothers me that pretty much the same math (at least in my brain) has so many different names: finite difference, Euler, Runge-Kutta, Taylor expansion... I am bad with names :')
It's similar, but there are differences, as I outlined in the video. The point of this was mostly to show why we use "RK4", and what it is, since that term is often thrown around without actually understanding how it works
@@copywright5635 Yeah a lot of people in my field just say "we use Runge-Kutta" then use ODE45 without thinking about what's behind the scene. The video is great!
Nope. Though, it does sound similar I agree. All the music I use in this video is in the description. Even with classical music I'm trying to only use stuff that's either public domain or Creative Commons licensed
I would point out that even the "exact" answer is an approximation, because you have to approximate the value of sine or cosine in order to draw the graph or get a numerical result for the position of the object on the spring. Now I know that you can easily calculate the value of the trigonometric functions to far more accuracy than you need -- but those number are still calculate by an approximation algorithm.
Of course this is correct. I figured including this would be a bit off topic, as the approximation we're concerned with in the video is of the "initial value problem" type rather than for function values. Thank you for the comment though
Yes! I really wanted to incorporate this into the video, but I wanted to get it out before SoMEπ ended, so I ended up not incorporating it. I'm not sure where I'll do this, but maybe I'll make a video on my patreon or a second channel demonstrating this. Otherwise, a sequel covering symplectic integrators will be coming at some point!
For any linear system (such as the one modeled here) a discrete state-space model can be accurate even with a course time step. If you model a single iteration accurately, then you have a template that can be applied simply at each iteration.
yes this is of course true. I'm covering other systems in another video that should be out within ~1 month (maybe longer). That one will focus on symplectic integrators
I've been wondering for some time how to extend time step in Velocity Verlet. Could you extend Velocity Verlet using the same logic as here? How about VV extension which is higher order in force (VV kinda assumes that force doesnt change during the time step)?
During university I did a project on halo orbits and used a RK of order 10. During the exam the professor asked why I didn't used a symplectic method (one that preserves the energy): RK still had an energy error of machine's precision's order and was much faster.
yeah RK is really good for a lot of things. Also, symplectic integrators are also not 100% accurate anyways. Though, Velocity Verlet is faster than RK4 and is quite good as well
Cool [B)] There are other methods which use a constraint to ensure energy is exactly conserved (perhaps at the cost of accuracy in other ways, or computational cost? Not sure) right? Edit: nice bonus
Mhm. Though, Runge-Kutta methods are generally preferred in almost all cases. Euler's method is just generally a bit easier to implement. There are of course many other approximation schemes.
Actually, there are a class of ODE solvers called Symplectic Integrators which work incredibly well for Hamilton differential equations. For example, if you're doing simulations of satellite orbits, a Symplectic integrator will allow for accurate and stable simulations over very long periods of time.
@@jameswright4732 Yes, I was debating on including them in this video. I decided against it as I wanted to keep the focus narrow, and the video not too long. I may end up doing a sequel to this video covering Symplectic Integrators.
So is it correct to say that the Runge-Kutta method is essentially repeating the process of following a curve's tangent until another point on that curve, and taking that tangent too, repeat?
I think you have the right idea. If you want a more rigorous definition, here's an MIT article on that. web.mit.edu/10.001/Web/Course_Notes/Differential_Equations_Notes/node5.html A lot of approximation involves taking tangent lines (linearization), so it's a bit hard to distinguish between them if you think of it that way
There are Runge-Kutta methods with embedded schemes, this allows for two approximations of different orders to be made from the same stages. By comparing the two and knowing their orders the error can be approximated and thus controlled. I think one should always use such a method when using classical explicit methods. I don’t think RK4 is used widely due to its lack of error control, it is taught widely however, as it is the “classical” RK method. See e.g. en.m.wikipedia.org/wiki/List_of_Runge-Kutta_methods#Embedded_methods
Hm, could be a good topic, maybe as a sort of sequel to this video? I'm trying to not present topics in a super dry manner. I'd rather motivate them first, so perhaps continuing the conservation of Energy throughline (or Hamiltonian ig) would be good for that. Thanks for the suggestion.
This video is a clear example of the hubris of mathematicians and physicists. You say nature is governed by differential equations. I say nature is described (narrowly) by differential equations. Nature doesn't care about your pathetic little equations (yeah, even Navier-Stokes), and we find every day that nature is more complex than we thought yesterday.
holy fuck. I hate math, and i hate physics. I dont get any enjoyment out of it, but i get enjoyment out of self improvement. I have a weird obsession with the term "level up" and anything related to it. You just earned my subscribe because this is good stuff, could help me level up
Perhaps, though not a full video dedicated to it. I'm thinking about doing a sequel to this covering symplectic integrators, as that seems like the next logical step. Though, that's not for AT LEAST a month
I found this video fascinating, and very cool overall. [Subscribed] It is surprising how rare it is to see the words "Runge-Kutta" compared to Euler tho. However, with deep respect re: @0:20, NOTHING in nature is "governed" by differential equations, rather differential equations allow us to see how nature is governed. (nice bonus at the end!)
_Nothing_ is 'governed' by equations... this is the biggest and most pervasive error supposedly smart people make when discussing nature. Equations _describe_ they are models... and all models are wrong... some merely useful.
cool vid. tks may I offer these tidbits from the Popcorn Model of Nature's Reality. This is in the study of the Harmonics and the Harmony of Our Universe in the context of Everything: so, lets use a metaphor where 1 musical note, (*) , represents Nature's Reality; This note, (*) , represents the true existence of Nature's reality. This is the realm of the lord, the almighty GOOD (not a religion but an attitude). The real note in which everything resides. What follows are just harmonics of the supreme existence of reality. 1st harmonic of reality (hor)* the human mind and the MotherVerse. 2nd harmonic of reality * commonly referred to as our universe and where electromagnetic radiative force is dominant. 3rd hor * dark matter, the strong nuclear force dominates. 4th hor * the weak nuclear force dominates. 5th hor * gravity, where the popcorn really explodes. 6th hor * time, the here and now where the rubber meets the road. The 3rd, 4th, and 5th combine to create Dark Energy. This not everything. Undescribed harmonics extend, ad infinitum, above and below the note (*). The harmonics show that space that appears empty is never in fact empty. Between Nothing and Everything is Something :)
Yeah so I decided to not mention anything beyond RK4 as I didn't want to complicate the video any further. Granted, it wasn't complicated as is. But, I wanted to keep it very focused on the topic presented. For example. I didn't mention any Symplectic Integrators in the video, even though they would be probably even better for modeling a system like a simple harmonic oscillator.
Pretty good video, but you could've explained things better. Like really focusing on the fact that f(x_i, t_i) is the slope, and what that means. And saying "instead of x_i and t_i we use x_i+1 and t_i+1 in the implicit method" means nothing if you don't already have a good intuition for what it means. It would also have been helpful to see how implicit methods are used in practice (using Newton's method).
Well, I mean all of the equations I mention model it only approximately. you're probably right that i should have said "model" instead of "govern". That's an oversight on my part. I was mostly focused on the mathematics than the physics here.
// initialise s0 = 1; s1 = 0; // loop s0 -= w * s1; s1 += w * s0; an implament from a more sieveoliesed era. renormalise the leg every buffer, s0 is a half step behind, w= 2 * pi * freq / samplerate
please learn how to pronounce Runge.You don't have to be perfect, but at least try.
You know what's crazy. I was pronouncing it the correct way for my whole life. But I watched like one video in research that pronounced it the wrong way and for some reason decided that they were right and I was wrong. So I decided to intentionally change how I pronounced it to the wrong way.
I'm actually so dumb lmao
I've heard people pronounce it "rungie", so it's at least better than that. Haha
@@copywright5635 that's crazy; on a similar note I'm convinced that Italian physicists can't pronounce "Dirac" Correctly; no professor at my Uni called the poor guy right.
(Hasty generalization I know, kinda funny tho)
@@GeodesicBruhlet me guess, they say it like dee-rac?
@@copywright5635 English: /ˈrʊŋəˈkʊtɑː/
My favorite quote
from engineer courses
is: everything is linear
if you watch it really closely.
valid
Chaos Theory: "Am I a joke to you?"
@@alexandervorgias4812 also valid
100% everything has a linear relationship. If it wasn't for the additive identity: 1+0 = 1. No other field within mathematics would be possible.
@@alexandervorgias4812 Nope, even within Chaos Theory. There are linear relationships even if we don't have the ability to recognize them.
Runge-Kutta still leaks energy.
For the equations of motion, the one integration scheme to rule them all is Velocity-Verlet. That conserves energy _exactly_ (apart from floating-point round-off), is 2nd order, and is just as computationally cheap as Euler.
To make it easy for others: en.wikipedia.org/wiki/Verlet_integration#Velocity_Verlet
Yes, I decided to leave off Symplectic Integrators. But of course, Velrlet and Velocity Verlet would have been better options.
Seeing a lot of comments about this, I will probably do a sequel to this video in the future
I'm a big fan of Verlet systems too! So simple and so insanely stable.
@@akaHarvesteR Verlet was my professor at university for one year.
And RKF78 is nearly symplectic ;)
Idk man seems youtube glitch or something but it's missing 'k' after 898 in your sub count hope youtube fix the issue soon, once again thanks for a great video
there’s a k now
It's fixed
Great video! It's funny that 3Blue1Brown's manim environment became the official animation framework of RUclips math videos. Every time I see some video made by manim, before watching it, I know that it is gonna be a good one. It never disappoints.
This is a great intro to approximation schemes! Really well explained and I love how you included all the animations to visualize what happens for each method. It was very helpful to see how drastic an effect the lack of energy conservation can have.
Nice video!
However, at 9:38 it should be noted that the "order" of a method does not refer to the number of terms/stages (k1, k2), but rather the truncation of the Taylor series. This means that a 2nd order method will exacly match the taylor series up to the x^2/2! term within each time step, while the following terms (x^3/3!,...) are not exact or missing.
For some fully implicit methods (Gauss-Legendre), the order can be two times the number of stages. (They're computationally expensive and I wouldn't recommend using them a lot but provide inpressive results for large time steps.)
Thank you for mentioning this. I could have mentioned this, however, since I didn't go through the Taylor series derivation, I thought it would just confuse most viewers.
Nothing in the universe is "powered by differential equations"!
Differential equations CAN DESCRIBE a lot of things.
This is very important to understand.
this is true. the universe is powered by weed.
I think you meant to write a double negative sentence. Check again
I don't think it was meant literally.
CAN DESCRIBE imperfectly i might add
Hahaha, thanks for the Vsauce callback at 1:08
Manchurian candidate activation signal @ 1:08
I love this kind of videos. I work on driveshafts, which are rotational mass-spring-damper systems to some degree. I loved doing the Taylor expansion and wrote some homework about how much accuracy you gain for increased compute, as you increased the order.
Another method (or sub-method) that maybe deserved a mention is the so-called leapfrog integration, where the average derivative for xI is taken from an average value of the previous tick acc and an extrapolated value for halfway towards the next one.
It's sort of similar to RK2, but the samples are offset back by one(half) of a tick.
It's relatively stable, and unlike RK2, you don't actually need to computer the derivative twice for each tick, as the first one is carried in from the previous iteration.
Yes, this is also a good method. I just wanted to keep the video simple. Perhaps I should have teased the sequel video at the end. I'm getting a lot of responses about this, so I think I'm going to make a sequel to this video covering Symplectic Integrators, along with some others like leapfrog
@@copywright5635 nice! Looking forward to that one! 😄
Excellent video! The visuals and the voice over were spot on :) you’ve made a great addition to the set of SoME videos.
The RK algorithms are a very fascinating topic, and I've even implemented a few of them in a C++ application before specifically RK4. Yet, I still feel or think that FFTs and their inverses are some of the most interesting algorithms out there. Complex Vector and Field analysis, the Hamiltonians, especially the quaternions and octonions, and so much more are all interesting topics. We stand on the shoulder of giants! I truly enjoy videos like these, keep up the great work!
Thanks! FFTs are a topic I think deserve a longer and more dedicated video, but I'm considering doing one on them.
Even for this, I barely touched the surface level of what RK is (didn't even derive RK4 lol), just wanted to provide some intuition for it. Both for the motivation, and why it would be more accurate than a simple Euler-Cromer method
@@copywright5635 Not exactly but similar to why Quaternions prove to work better than Euler Angles when performing rotations in 3D about the independent axes.
When using Euler Angles there is the phenomenon of producing Gimbal Lock within the use of the rotation matrices. This where one axis ends up being rotated onto another where they become coincidental and from there you end up losing a degree of freedom as the two axes are locked and you can no longer differentiate between them.
Quaternions helps to prevent this. Also, quaternions, even though the mathematical notations and expressions are fairly complex, implementing them in software is fairly trivial and they also have a very nice added benefit of being able to be calculated against other vectors and matrices as well as being converted to them. Because of this, they are also computationally cheap, very efficient and quite effective.
It's not exactly the same, but it goes to show where a variety of Euler methods although is simpler to digest and work out by hand, also has their shortcomings.
FFTs just provide a very good and efficient way to transpose from one system to another especially when working with wave patterns or anything with a frequency domain.
Without FFTs, audio processing either being a wave file, a midi file, or even an MP3 file wouldn't be as efficient as they are today. Audio even when compressed requires a lot of information and can be fairly computationally expensive. FFTs reduces that by a couple orders of magnitude. Instead of trying to perform 20 thousand sine or cosine function calls per second for a 20kHz frequency. We can just sample it and use the sample rate to reconstruct a good enough approximation of the individual waves. Well sort of, as that's the abridged version.
But yeah, I find it all to be very interesting and intriguing. I'm not just intrigued with this type of stuff either. I'm also intrigued by 3D Graphics Rendering, Game Engine - Physics Simulations, as well as actual CPU Hardware Design (ISA design). Then again, this gets into physics when you go beyond the logical device level and get into the actual structure of the transistors, resistors, etc. that are designed to manipulate electricity. And here we are again, with wave propagation. Right back to the use of wave functions and the power of FFTs lol!
I just like things related to engineering. Factorio, Satisfactory, Dyson Sphere Program, Oxygen Not Included, Planet Crafter, Mindustry, Turing Complete, etc. there're all a part of my Steam Library and are my hobbies. And I'm no stranger to Music as I did play the Trumpet for close to 10 years back in my school days.
If you like this and Fourier methods, you should check out dispersion and dissipation analysis (sometimes referred to as "Fourier analysis") for ode solvers (and pde solvers too, but that's a bit more complex). It essentially allows someone to understand how a solver will respond to any initial condition of a linear problem.
@@jameswright4732 I've written a couple of simple ODEs.
My favourite astrophysics professor taught us to use symplectic integrators for orbital mechanics because they explicitly conserve energy.
perfect timing of you to post this video this semester
Wow, I honestly had no idea these methods were even connected! Thank you for the straight-forward explanation and visualizations. Top notch content, Sir.... 👍
That honestly is a great video, keep up the good work! (and so cool there's ondine in the beginning of the video)
Thanks! [ And Scarbo at the end :) ]
Amazing video ! I loved your pace and your little jokes, it really helps staying engaged with your presentation. The visualisations are of course really good too. :)
Loving Gaspard as the background❤
You've really made numerical methods an interesting area for me.
I really didn't expect to hear Gaspard de la Nuit :)
I do try to have good music lol
2 minutes in, I can already say: I like your style! Just one remark: I think it would be better to clarify that we can best describe nature with the help of differential equations, instead of saying that nature is governed by differential equations (even if it was true that we live in a simulation, this would be outside the scope of scientific thinking). This reminds me of talking about charged particles "feeling" a force and thereby intentionally reacting to it, or explaining darwinism by active adaptation to a changing environment, or even "lonely atoms which want to form bonds to share electrons" to fulfill some godly given octet rule so that all of them can live a long and happy life, and every other thing our teachers tought us "although we should keep in mind that this is just a simplification" - although the concepts you are about to explain (from my point of view at this moment in time, at minute 2 in your video) exactly oppose these views, of course. But keep in mind: Now the maths begins, and view counts will only drop from here on.
To be clear: I really like your video! Narration, animation, general style: wonderful! And I had a look at your channel, and I will watch a few more of your videos.
Thank you for the feedback, yes I should have been more clear about that. I was mostly focused on the mathematics here, but of course, everything physical we describe with mathematics is just a model. I'll keep this mentality in mind for future videos.
i think metaphysics is a part of science. many great scientific insights came from studying metaphysics. i.e. what is true , how much can we know of truth, and what is that truth composed of?
and there maybe a correspondence between our models and reality, i.e. reality = diff equations, and it's interesting enough for you to bring up, and I think it's helps strengthen our mind's talking about this...
and it can give us insights into math from a point of view of physics and vice versa.
1:09 this sound is equivalent to undertaker's unexpected entry during an ongoing wrestling match😅 haven't watched the complete video, I'll leave a 'review' comment after watching it but it seems video will be awesome and informative.
I wish you had made this video during my computational physics class 😅. Nevertheless, thanks for your clear explanation. Deserves more subs.👍
0:21 eveything in nature isn't governed by differential equations (DE), DEs describe nature, they don't govern them.
I know it can be seen as a nitpik, but I felt that the semantic difference between 'governing' and 'describing' were big enough to warrant the comment. The rest of the video was great!
I independently reinvented RK2 in high school. Very simple idea.
From my days in college, when I took Numerical Analysis, I had an idea. Do the iterative equation that, with each iteration, produces Output values relative to increasing Time… BUT… for each Output value, use it as a STARTING VALUE for the iterative equation method known as *Picard’s Method.* When Picard’s Method iterates, it STAYS ON a single Time value (i.e. as you iterate, Picard’s Method doesn’t “move you” along the Time axis.) I never got to try my idea, but I always wanted to. Overall, the idea is that you keep switching back ‘n forth between an Increasing Time iterative equation AND a Picard’s Method iterative equation. For each “invocation” of Picard’s Method, you perform enough iterations until a suitable degree of convergence is achieved.
Hey Vsauce, reference here!!!
The other end of the spring is fastened to a Hooke.
Carl Runge was a German mathematician and not the founder of Kurt Cobain's music genre.
This is great. You seem to be getting a bit of flak from the very knowledgeable on the subject, but I just feel the video is directed to those like me, who *in theory * can follow the equations but have trouble with the 'where are we going with this' part. And in that respect the video succeeds with flying colours. So, thank you!
You might think my comment is mean or unsupportive of your work. But actually I really enjoy watching your video. The problem is you didn't answer your title. You didn't explain why the method is better than Euler. You didn't derive it or show any reason why it is more stable. You just showed graphically how it plays out but never actually proved anything about it
Thank you for the feedback.
I think your concern comes from a differing opinion on what "why" means. I do concede that the title is not true in a strict mathematical sense (I didn't derive RK after all).
However, I did provide reasoning for why RK2 (and by proxy RK4) seem to conserve energy better than an Euler method. I could have mentioned Symplectic Integrators, or even the Euler-Cromer method (which only changes one term, yet conserves energy for these problems). RK methods also don't inherently conserve energy, they simply converge much faster.
I approached this video with the notion in mind that it is unclear why for certain systems, RK methods, even RK2, seem to simulate so much better than standard Euler Methods. I wanted to provide a sort of intuitive motivation, and I think I accomplished this.
You are correct in saying I did not mathematically prove anything about Runge-Kutta methods. This was never the intention. I apologize if you found the title misleading.
TL;DR, The "why" in the title is not the "why" of a mathematician. It's the "why" of an engineer or experimental physicist.
i always learn something new and interesting from your content!
Check out the semi-implicit Euler method. It's especially important because it preserves energy very well for small enough regular time steps.
Oh this video is unfortunately a little bit late for me, just had my numerical simulation exam last month ;) still watching this video since it got recommended to me and its so fascinating how people came up with such things decades ago!!!
Runge = Run-ge.
Nature is not governed by equations. It is modelled in equations.
its governed by equations.
As a Physics student, these videos are a great motivators.
Glad to hear it!
It's clear to see that the higher order algorithms are more exact per timestep but they're also more computationally expensive because of calculating multiple derivatives per timestep. It would've been nice to see how exact each algorithm is per derivative evaluation. Because it might be more efficient computationally to use a smaller time interval with a lower order algorithm than using a higher order algorithm.
Hm, well I didn't show this directly. But notice that the Euler time step with dt = 0.02 is still worse than RK4 with a 0.1 time step
Very clear! Implicit Gaussian Collocation for the win though! (For numerically fully conserving skew-symmetric use cases.) EDIT: But you already mentioned Sympletic integrators in one of your responses.
Mhm, sequel video covering them coming soon! There's so many though haha, we'll see if I do end up including Gaussian Collocation
@@copywright5635 Thanks! Subbed as I’m looking forward to hearing more of (sympletic) integrators. I’ve used Gaussian Collocation for simulating with conservation accuracy down to machine-precision (with help of skew-symmetric PDE form), which is cool, but I can’t say I really ever understood what is sympleticity means nor how to derive, e.g., exponentially sympletic variants.
Thanks man, fantastic explanation! Looking forward to more videos of yours.
Music choice is splendid. Love Ravel.
Another integration method that you can consider is Verlet one, third order error for position and second order error for velocity. It is highly used in games, since we care also about object interactions and with Verlet this is really easy. We can enforce non penetration constraints without necessarily applying a the force on those objects, but just displacing their positions and still not completely break the system. Obviously non physical correct, but robust and somewhat believable.
Thank you yes, I'm going to do a video soon on symplectic integrators
I recall Euler's method being introduced to me explicitly as a tool that does not produce good approximations, but rather convergent ones, which is useful for proving existence of actual solutions.
hm, well Euler's method can be covergent. However, as I showed in the video, for many systems errors will cause it to diverge quickly.
3 mins in and it's already getting interesting; never disappoints
Nice! I coded RK4 and other methods in python for a 3 body problem simulation. RK4 and Velocity Verlet were way more stable than Euler or even a 2nd order Taylor series when we consider conservation of energy. Thank you for the video;
I loved the video. Numerical methods are a super interesting topic.
Thank you so much! I hear about Runge-Kutta so often at the lab but never understood it until now! But it bothers me that pretty much the same math (at least in my brain) has so many different names: finite difference, Euler, Runge-Kutta, Taylor expansion... I am bad with names :')
It's similar, but there are differences, as I outlined in the video. The point of this was mostly to show why we use "RK4", and what it is, since that term is often thrown around without actually understanding how it works
@@copywright5635 Yeah a lot of people in my field just say "we use Runge-Kutta" then use ODE45 without thinking about what's behind the scene. The video is great!
Is that Rousseau’s piano? I swear it sounds just like his piano, I’m so used to his tuning
Nope. Though, it does sound similar I agree.
All the music I use in this video is in the description. Even with classical music I'm trying to only use stuff that's either public domain or Creative Commons licensed
I would point out that even the "exact" answer is an approximation, because you have to approximate the value of sine or cosine in order to draw the graph or get a numerical result for the position of the object on the spring. Now I know that you can easily calculate the value of the trigonometric functions to far more accuracy than you need -- but those number are still calculate by an approximation algorithm.
Of course this is correct. I figured including this would be a bit off topic, as the approximation we're concerned with in the video is of the "initial value problem" type rather than for function values. Thank you for the comment though
one nifty thing is you can use the difference between k2 and k3 to estimate the error and adapt your timestep dynamically.
Yes! I really wanted to incorporate this into the video, but I wanted to get it out before SoMEπ ended, so I ended up not incorporating it. I'm not sure where I'll do this, but maybe I'll make a video on my patreon or a second channel demonstrating this.
Otherwise, a sequel covering symplectic integrators will be coming at some point!
For any linear system (such as the one modeled here) a discrete state-space model can be accurate even with a course time step. If you model a single iteration accurately, then you have a template that can be applied simply at each iteration.
yes this is of course true. I'm covering other systems in another video that should be out within ~1 month (maybe longer). That one will focus on symplectic integrators
I've been wondering for some time how to extend time step in Velocity Verlet.
Could you extend Velocity Verlet using the same logic as here?
How about VV extension which is higher order in force (VV kinda assumes that force doesnt change during the time step)?
Incredible video!
I think the presentation went too fast in the few key moments, like when you define implicit scheme and how to actually calculate it
@@sheevys thanks for the feedback. I’ll be more careful about that next time!
During university I did a project on halo orbits and used a RK of order 10. During the exam the professor asked why I didn't used a symplectic method (one that preserves the energy): RK still had an energy error of machine's precision's order and was much faster.
yeah RK is really good for a lot of things. Also, symplectic integrators are also not 100% accurate anyways. Though, Velocity Verlet is faster than RK4 and is quite good as well
@@copywright5635 The simplectic I tried was leapfrog but second derivative computations for gravity in a rotating system were quite heavy.
Thank you for this beautiful explaination
Wish your channel existed more than a decade ago
Cool [B)]
There are other methods which use a constraint to ensure energy is exactly conserved (perhaps at the cost of accuracy in other ways, or computational cost? Not sure) right?
Edit: nice bonus
Mhm. Though, Runge-Kutta methods are generally preferred in almost all cases. Euler's method is just generally a bit easier to implement.
There are of course many other approximation schemes.
Actually, there are a class of ODE solvers called Symplectic Integrators which work incredibly well for Hamilton differential equations. For example, if you're doing simulations of satellite orbits, a Symplectic integrator will allow for accurate and stable simulations over very long periods of time.
@@jameswright4732 Yes, I was debating on including them in this video. I decided against it as I wanted to keep the focus narrow, and the video not too long. I may end up doing a sequel to this video covering Symplectic Integrators.
I would say the language of Nature is conservation, expressed in mathematical terms with differential equations.
Leapfrog KDK(or DKD) is generally a overall better pick in application, I think
So is it correct to say that the Runge-Kutta method is essentially repeating the process of following a curve's tangent until another point on that curve, and taking that tangent too, repeat?
I think you have the right idea. If you want a more rigorous definition, here's an MIT article on that.
web.mit.edu/10.001/Web/Course_Notes/Differential_Equations_Notes/node5.html
A lot of approximation involves taking tangent lines (linearization), so it's a bit hard to distinguish between them if you think of it that way
There are Runge-Kutta methods with embedded schemes, this allows for two approximations of different orders to be made from the same stages. By comparing the two and knowing their orders the error can be approximated and thus controlled. I think one should always use such a method when using classical explicit methods. I don’t think RK4 is used widely due to its lack of error control, it is taught widely however, as it is the “classical” RK method.
See e.g. en.m.wikipedia.org/wiki/List_of_Runge-Kutta_methods#Embedded_methods
yes, this is why for example ODE45 uses 4th and 5th order schemes
missing a delta t in the formula for k2 at 8:50?
where'd you find that background song from dawg?
11:27 holy shit matlab mentioned
How about symplecetic integrators?
Hm, could be a good topic, maybe as a sort of sequel to this video?
I'm trying to not present topics in a super dry manner. I'd rather motivate them first, so perhaps continuing the conservation of Energy throughline (or Hamiltonian ig) would be good for that. Thanks for the suggestion.
Clearly explained. Thanks.
Runge -- rung uh.
This video is a clear example of the hubris of mathematicians and physicists. You say nature is governed by differential equations. I say nature is described (narrowly) by differential equations. Nature doesn't care about your pathetic little equations (yeah, even Navier-Stokes), and we find every day that nature is more complex than we thought yesterday.
holy fuck. I hate math, and i hate physics. I dont get any enjoyment out of it, but i get enjoyment out of self improvement. I have a weird obsession with the term "level up" and anything related to it.
You just earned my subscribe because this is good stuff, could help me level up
thank you!
Man mathematicians can't even make sine graphs.
great!
This was fun! Want to do Crank-Nicholson next?
Perhaps, though not a full video dedicated to it.
I'm thinking about doing a sequel to this covering symplectic integrators, as that seems like the next logical step. Though, that's not for AT LEAST a month
@@copywright5635 I should have picked up that you're chasing Noether - superb!
I found this video fascinating, and very cool overall. [Subscribed]
It is surprising how rare it is to see the words "Runge-Kutta" compared to Euler tho.
However, with deep respect re: @0:20, NOTHING in nature is "governed" by differential equations, rather differential equations allow us to see how nature is governed.
(nice bonus at the end!)
Leapfrog is better.
The real reason I clicked on this video was to know how to pronounce Runge
I WAS WRONG DON'T TRUST ME. ITS "WRUNG - GEH" NOT "RUNJ"
2:18 shouldn't v(t) be with sin and not cos?
Oh that's an oversight on my part, good catch!
cool
Hello brother, I watch your video, I have some doubts related to Manim, will you come and talk to me? 😕
I could if u want. But I think there's a place better suited, the Manim Discord
discord.com/channels/581738731934056449/581743417999228938
_Nothing_ is 'governed' by equations... this is the biggest and most pervasive error supposedly smart people make when discussing nature. Equations _describe_ they are models... and all models are wrong... some merely useful.
The name Runge is pronounced slightly differently
ik im sorry check the pinned comment
"Runge"? As in Grunge? 😟
Carl Runge and Martin Kuta were German. Pronunciation would be like Roong-uh - Kooh-ta.
I"M SORRY OK, CHECK THE PINNED COMMENT REPLIES
@@copywright5635 Sorry! I somehow managed to miss those when I wrote my comment 🙏
cool vid. tks
may I offer these tidbits from the Popcorn Model of Nature's Reality.
This is in the study of the Harmonics and the Harmony of Our Universe in the context of Everything:
so,
lets use a metaphor where 1 musical note, (*) , represents Nature's Reality;
This note, (*) , represents the true existence of Nature's reality.
This is the realm of the lord, the almighty GOOD (not a religion but an attitude). The real note in which everything resides. What follows are just harmonics of the supreme existence of reality.
1st harmonic of reality (hor)* the human mind and the MotherVerse.
2nd harmonic of reality * commonly referred to as our universe and where electromagnetic radiative force is dominant.
3rd hor * dark matter, the strong nuclear force dominates.
4th hor * the weak nuclear force dominates.
5th hor * gravity, where the popcorn really explodes.
6th hor * time, the here and now where the rubber meets the road.
The 3rd, 4th, and 5th combine to create Dark Energy.
This not everything. Undescribed harmonics extend, ad infinitum, above and below the note (*). The harmonics show that space that appears empty is never in fact empty.
Between Nothing and Everything is Something :)
Because Euler was a mathematician, not a computer scientist.
Not mentioning Tsit45 is a huge oversight IMHO. It's strictly superior
Yeah so I decided to not mention anything beyond RK4 as I didn't want to complicate the video any further.
Granted, it wasn't complicated as is. But, I wanted to keep it very focused on the topic presented.
For example. I didn't mention any Symplectic Integrators in the video, even though they would be probably even better for modeling a system like a simple harmonic oscillator.
I don't think thats how you pronounce 'Range-Kutta'
RK is not strictly better. Sometimes Euler’s is stable when RK is not.
Is it? Perhaps you're talking about Euler-Cromer methods which are symplectic while RK methods aren't.
Not to be petty, but I think nature doesnt have a language.
Euler is indeed a Runge-Kutta method (first order). You should specify the higher order Runge-Kutta methods in the title.
Pretty good video, but you could've explained things better. Like really focusing on the fact that f(x_i, t_i) is the slope, and what that means. And saying "instead of x_i and t_i we use x_i+1 and t_i+1 in the implicit method" means nothing if you don't already have a good intuition for what it means. It would also have been helpful to see how implicit methods are used in practice (using Newton's method).
Differential equations don't govern nature. They model nature. And a lot of the equations you mention model nature only approximately.
Well, I mean all of the equations I mention model it only approximately.
you're probably right that i should have said "model" instead of "govern". That's an oversight on my part. I was mostly focused on the mathematics than the physics here.
Just use Laplace dude...
@@גיאדרי I mean for this situation it’s easy, but Laplace transform doesn’t always qork
Laplace _if it's linear_.
Runj
IM SORRY OK
// initialise s0 = 1; s1 = 0; // loop s0 -= w * s1; s1 += w * s0;
an implament from a more sieveoliesed era. renormalise the leg every buffer, s0 is a half step behind, w= 2 * pi * freq / samplerate
And after all that math and gobbledegook and fancy words, its basically just averaging