6 Months of Voxel Physics Engine Optimizations
HTML-код
- Опубликовано: 26 апр 2024
- Demos page (unoptimized): kotsoft.itch.io/
A summary of my past 6 months (or maybe more lol) of voxel physics engine optimizations to get massive simulations running in real-time on laptops and handhelds. Going through memory bandwidth optimization, sparse grid and adaptive particle resolution. 
Игры
![Chance The Rapper, DJ Premier - Together (2024) | [Official Music Video]](http://i.ytimg.com/vi/ZVwTX6WP-MM/mqdefault.jpg)
Super cool having a bit of a deeper dev log style video. Sadly the YT compression did the video a bit dirty...
I've reuploaded a 4K60 version. This was my first time doing a recap video so I used redownloaded footage from the past. But I have gone back and recorded new native 4K clips, not upscaled. In the future I will upload all videos in 4K.
RUclips Compression: "let me murder these particles"
I've reuploaded a 4K60 version. This was my first time doing a recap video so I used redownloaded footage from the past. But I have gone back and recorded new native 4K clips, not upscaled. In the future I will upload all videos in 4K.
It's amazing how many incredibly skilled programmers end up building voxel engines.
Would be cool if they got togheter and made a general voxal engine too see what games can be done with it.
I think that’s probably just because they can more closely approximate the real world than other types of digital simulations. Even Minecraft is more realistic in many ways than other game engines (while having a lower spatial resolution than other voxel engines, obviously). Most hyper-realistic simulations tends to have that atomization of digital objects in some form since you need to run the physics calculations on as many points in space as possible. Like even Microsoft Flight Simulator, which is arguably the best looking game/simulator out there, simulates aerodynamics by running the calculations on lots of different points on the body of the aircraft. Voxel engines kind of have that ability built in rather than having to lay polygonal models on top of the simulation. But, like always, there’s a trade-off between the volumetric resolution and the processing power required.
Wow, I can't believe this video was recommended to me. I was following your work... was it 10 years ago? More? When you were making the Grantophone instrument app on iPhone. Can't say I'm not sad you never made a working Android version (though I haven't looked since then), but it's amazing to see what you're up to these days.
Absolute breakthroughs man, congratulations and well done, keep up the good work!
Yeah, time flies, last Grantophone release on iOS was 2011. I think I'll just try to opensource it at some point or just make it webassembly. I actually don't have the code anymore but I can rewrite it.
@@GrantKot hello sir, this is amazing. If possible please opensource it, it would prpve to be really inspiring and helpful resource for new devs. Might as well stream the coding process live as well to showcase the challenges and struggles live while creating it. It would be helpful to learn the process of creating amazing things like this to other devs.
Thank you
(Again, I love the project!)
I love this. Beautiful video, peaceful vibe. Seeing the progress is very nice.
New 4K version of this video here: ruclips.net/video/3259aycYcek/видео.html
Sorry about the compression here. I usually upload 4K60 but because this is an overview over a long period of time I had to redownload some videos from YT and now they are triple compressed.
Great work on the physics aspect! I'm honestly shocked at the speed of your algorithms. Looking forward to further developments!
soo awseome ive been catching up occasionally and i rly cant wait to see how far this gets
Just, plain, "Wow!" as so many have already written below.
Deep respect for the dedication it takes to get all this achieved.
Keep up this awesome piece of work...
Wow, your demos look mesmerizing!
These simulations are some of the most impressive things ive ever seen! very fun to play with the demos.
Amazing! I can't even begin to comprehend how this works. So many particles, each with its own physics, all in real time... beautiful!
Wow! Fascinating how you've been optimising this. It's amazing to see it all moving around so smoothly.
I cant imagine the amount of work and motivation this took. Great progress!
What a great job man. Thank you for sharing.
Most of the technical stuff went over my head, but this is crazy impressive! Keep it up!
Wow! ❤
Incredible work, thx for sharing!
I hope you voxels evolve to be of different shapes like in space engineers, cheers.
you deserve a bigger audience, great work, i will follow your progress
that is genius work,
please keep sharing!!
this is dope - awesome work!
This is seriously underrated, this work is amazing! Thank you RUclips recommendations
Really enjoyed this vid. Subscribed
Loving watching this development. Just learning fluid basic sims myself and this is really inspiring.
Awesome work Grant!
Holy, this looks amazing
Amazing. You're very talented.
THIS IS SO INSANELY AWESOME!!
Damn , impressive. I have tried running simulations like this in parallel and it’s really hard ,nice work
This is cool, although with the combination of extreme compression artefacts (try rendering at 4k even if you only use 1080p footage to avoid YT compressing it to death) and the constant unpredictable camera movement I found it quite hard to watch or even see what was going on.
I've reuploaded a 4K60 version. This was my first time doing a recap video so I used redownloaded footage from the past. But I have gone back and recorded new native 4K clips, not upscaled. In the future I will upload all videos in 4K.
Instant subscribe! I am also a fan of luquid simulations. For now it seems you have achieved everything what I wanted. Although as they told me before It takes a lot of time.
It is good to know it is quite possible! Thank you for inspiration. It may help to fight my procrastination)
Beautiful. I am very impressed.
Just Amazing!! I am learning game development and this is the video that that shows how cool programming is..... Thanks a lot sir, Hope you will put move videos on optimizations.
I think I saw you on twitter, great video man :D
this man single-handedly working on making simulation theory real
Awesome work! Keep it up!
so cool man great work
This was amazing!
This is beautiful.
The ending with the ships looks like the making for a really fun game
Impressive, keep it coming!
dang this looks awesome
this amazing man.
This looks incredible. I’d love to see a Waverace style game with these simulated water physics.
looks amazing
Looks fantastic!
The work you've been doing has been absolutely inspiring. I hope my project can someday reach this level of quality.
holy crap this is amazing
Love the bits of humour in this video 😁
Very nice dude, insane coding skills!
This is more impressive that even I realize
Not a coder. But I'm an artist. Glad YT showed me this! This is art!
Upscaling to 4K (just a simple bilinear scaling) for a these kind of video could make it much more viewable! Great video
I've reuploaded a 4K60 version. This was my first time doing a recap video so I used redownloaded footage from the past. But I have gone back and recorded new native 4K clips, not upscaled. In the future I will upload all videos in 4K.
This is actually so fucking cool!!
i can't wait to play that game!
I would play the crap out of a pirate game with the sea and ships from the last scene. Really beautiful.
Very cool! The water looks so nice it made me thirsty. Hope you got to see Dune 2, that's the best movie I've seen in years.
Yup, I saw it in IMAX 70mm! I’ll get the digital version too to analyze everything in detail.
This is great study into how frigging fast modern hardware is if utilized well. And to do that needs knowledge of how the hardware works on quite low level. It's a shame that this skill isn't in high regards anymore... it should be. We could do amazing things and save energy as we go about it.
Amazing!
A voxel pirate boat sim game would be siiiiiiick
Once you get a first person character and some voxel building mode, you should definitely make this a game and put it on steam!
Amazing 👌 However, the camera movement is largely confusing - but better during the sand segment.
Sometimes, a static camera is the best choice for showing what's going on - and it might help improve a bit on the damage done by compression.
Slow motion might also be very cool for showing off this incredible work. Thanks for sharing! Instant like and subscribe ✌
Thanks! I have now completed full 4K60 redo of this video.
@@GrantKot I watched it as soon as I saw it pop up - great stuff, and thanks for sharing 👌
Videos like these deserve higher compression
I've reuploaded a 4K60 version. This was my first time doing a recap video so I used redownloaded footage from the past. But I have gone back and recorded new native 4K clips, not upscaled. In the future I will upload all videos in 4K.
@@GrantKot Legendary
This would be a good foundation for a sick waterbending game
Subd, looking forward to an applicable voxel engine with an easy to use UI..
Holy moly
Looks great! The water does feel a bit sticky on the boats at the end there though.
Thanks! Yeah, base Material Point Method can be a bit sticky I think in the future I can implement some of the newer papers.
this is insane.
This is nuts
I love it
6:06 is a great chunky flesh simulation lol
I see you have a Legion Pro 7, I too got a Legion 7 Pro, but I got the AMD variant with the 4090, and got a really good deal on the laptop through a sale. I have been thoroughly enjoying being able to do rather difficult to run things like racing in BeamNG Multiplayer with 16+ players, or pushing Minecraft to stupid render distances with various mods. I'm looking forward to games built on voxel engines like yours, it looks like so much fun to mess around with.
I do wanna say one thing, I really want to see flight simulation achieved via voxels. My dream would be a full game with a story, built around the idea of making ultra light aircraft, sculpted from wood, much like a balsa model aircraft, which the player can build and then fly in an incredible aerodynamic simulation, achieved utilizing purely voxels. I currently know nothing about coding, and this dream of a game made around this idea is impossible for me to achieve right now, but I think if I ever do get into coding and working with voxel physics simulations like this, that is what I'd do. I want to capture the beauty of flight, and specifically pure glider flight, in a physics simulation where the air flows accurately up and over mountains to replicate ridge lift, alongside thermal soaring and stuff like that. Idk if you'll read this, but I'd love to know where or how you started down your path in making these kinds of simulations. Your simulation with the ships I think is the closest I've seen to a simulation capable of achieving this idea of mine because the ships appear to be flexible, and destructible, which are both aspects crucial for making accurate glider flight simulation.
Yeah, I do want better air simulation at some point so I can handle gases and fire etc as well as wind to blow the sails, and hopefully support flight :). I got started in the beginning making games with GameMaker & Flash. Then 3d modeling and animation, and using the fluid simulation software, RealFlow. Then back into making my game and because we didn't have plugins then like nowadays with Niagara Fluids or Zibra Liquids decided to try rolling my own.
Tip: resample sub-1440p videos to at least 1440p so that RUclips retains most of the quality of the original upload (more bit rate granted for higher than 1080p uploads). Add also just a tad of sharpening for good measure.
I've used VirtualDub2 for sharpening and Lanczos3 resampling. Works well.
I've reuploaded a 4K60 version. This was my first time doing a recap video so I used redownloaded footage from the past. But I have gone back and recorded new native 4K clips, not upscaled. In the future I will upload all videos in 4K.
@@GrantKot Great news, and the "4K, 6 Months..." video quality looks really nice.
With voxels, there are always more boxes to tick!
that is so impressive! did you have an idea of what sort of game you wanted to make? i was imagining a cool ants nest game where you have to build a nest without it caving in while water comes in periodically lol
I started out with a game about cleaning oil spills but nowadays I see a lot of possible ways to go. I think there are a lot of engineering game possibilities. Ship simulator, monster truck simulator, bridge/building during storm/tsunami simulator. I also very much want to try out some evolution stuff, especially amphibious locomotion. It really just might go user-generated content mode and I would be making sure that it can handle all kinds of different physics phenomena.
@@GrantKot I think you're spot on. This engine tech has so much potential
in someone else his voxel optimizations he cut down on the floats bytes by just keeping track of the delta relative to a certain chunk saved him 6 to 9 bytes depending..
Yeah, could be some thing to explore, keeping track of the particle position relative to tile corner instead of global could let you use 16 bit float, or increase accuracy in 32 bit
Reminds me of John Lin, hm... very good looking.
the firestarter 9000
funny words, magic man. ;)
Stuff like this equally impresses and depresses me, because I can't begin to imagine how many years I would have to invest to build something even remotely as impressive as this. Very inspiring! Looking forward to future updates :)
Really good job! Have you thought about rendering a lot of simulations, storing them, rate them and use them to train an ai model? As far as I know nobody is currently working openly known on such a project (maybe for a reason). Start small however :) Cheers!
My main impression of AI models for fluid simulation is they look a bit like multigrid. If it is a CNN it seems actually a bit similar to what I have set up. Like the kernels are like the MPM kernels where velocity gradients are measured etc. What I feel is that I already have the minimum computation required to simulate materials. It might be useful to fit material parameters so that simulated material behaviors more closely match experimental behavior.
Wow. Nice work! I have a few questions about your fluid/solid simulation. How do you discretize the domain? Do you have particles like in SPH simulations? Are you actually solving mass and momentum balance equations to have a physically accurate simulation?
Thanks! What I use is called Material Point Method. It is a hybrid particle-grid method, others in this family are PIC or FLIP. Mass and momentum are taken into account and it is mostly quite accurate.
@@GrantKot Thanks a lot! I'll look into that.
If you export your videos with a higher bitrate it should preserve some of the details in the water particles and stuff even in 1080p
I've reuploaded a 4K60 version. This was my first time doing a recap video so I used redownloaded footage from the past. But I have gone back and recorded new native 4K clips, not upscaled. In the future I will upload all videos in 4K.
Watching the first 30 seconds its obvious you are using "fractal" simulations by using a few high quality particles and then using rng offsets for the others but I think its a smart choice.
very pretty water
teardown 2 incoming
16 times the detail!
Your work is amazing. I have never seen such a high-quality physical simulation. This may sound a bit rude, but is it possible to view the source code for educational purposes? It would help me a lot
That's some incredible real time particle simulations. Too bad RUclips compression murders all the detail out of it. I wonder how this would run in a game once everything else is calculated like lighting and other functions
Please make a sandbox game using this tech.
This is amazing sim work, but clarify one thing, what do you mean you were super intrigued by intel's cpus and their high core counts ? I really tried listening to that part 3 times but still can't figure out what you mean. I've been using pcs since the days of AMD k6, Duron and Celeron Days. And now I have a Ryzen cpu. The only reason why we have high core counts these days is because AMD broke the almost decade old intel grip on the 4core era. Back to the point :
What do you mean by being impressed with intel's higher core counts ? Perhaps you meant some sort of special register that may be unique to intel that helps your specific use case ? It makes very little sense to say intel has more cores since their top of the line desktop chips have a mix of bigLITTLE cores that top at 24threads/cpu whilst amd's top at 32. The enthusiast/workstation chips both Xeon W series and Threadrippers share the same story.
The only place where intel and amd share the same nr of threads (not cores) is the high end mobile market but then again amd's offer with 3d-Vcache may significantly help your use case.
All in all, you gained one subscriber 🥰 keep up the amazing work !
I'm mainly looking at what is available to most gaming consumers. When AMD goes above 16 cores there's a major jump in price right, with HEDT? The 13900K is available for $464. Anyway, once I did the testing my laptop 13900HX beat out my 12 core desktop 5900X. Could be due to DDR5 as well.
Anyway, I was able to get the E-cores performing about half a P-core. There was an earlier thing that said they were about 1/4 and I just wanted to see if I could do better by optimizing memory bandwidth. Sometimes I'm drawn to negatively reviewed hardware as well to see if I could do better. Like I was almost curious about the MSI claw to test out how it does compared to the AMD handhelds, but signs point to bad manufacturing by MSI as well. I currently have 3 AMD computers and 2 Intel so not trying to start any CPU wars. The sparse grid was developed also because I felt like it would help the grid data stay on v-cache more.
@@GrantKot ah I get it now, it makes sense to pick the best value option. And regarding ddr5, yes, In your memory bandwidth intensive usecase, it makes sense that ddr5 would outperform since it's got almost twice the bandwidth (on average). No CPU war intended btw 😅. Also, if you move your simulation from a dual channel to a quad or octa channel system, you should see serious perf jumps there too. Cheers
all this in 6 months ? pro
Cool,
But you can just implement the Quantum Mechanics and super optimise them instead (and than you will get all types of materials automatically!) hopefully, and hopefully you can do it , because it's gonna go complex
Actually 16 times the detail!
Do you have a mirror of this video somewhere? I'd love to see it on Vimeo or something, where the compression is less severe
I have a new 4K60 video where I did fresh screen captures this morning from all the available demos. I will try to see if I can also put up a better version of this video online somewhere.
looks amazing. but, wouldn't it be substantially faster if you did this on the GPU instead of the CPU?
It's a bit tough to say. With most gamer level machines, I think doing the simulation on CPU and rendering on GPU makes sense. Anything less than an Nvidia card ending in 70 or AMD equivalent. Especially on handhelds, or console, I think it's better to have them work as a team. And the memory is unified on APUs. There is also an expectation of raytracing in voxel engines on higher end cards.
i dream of swimming in that water in VR one day! it will happen. it's only a question of time.
Yeah, I am close to being able to go VR. And I do very much want to. Maybe once structures stuff is more developed and another optimization pass is done. I have already gotten multi-view rendering working with the previous demo.
@@GrantKot my closest experience of this in VR was a game called chroma lab on steam. I also played a game called VR falling sand that is more rudimentary but i enjoyed it.
6:40 😂😂😂
Hi, very cool! But unrelated question;
As someone who doesn't know too much about computers, i have the feeling that i the last 20 years or so computer technology hasn't really improved much. As in, we still use the same tech and principles as back then, but industry has made everything cheaper and we figured out to make things smaller and more efficient so things have in fact improved, while still using the same technology. Is this true? If so, then it only makes sense that technology improves incrementally (with diminished returns even) over time. The only place left then to truly improve things is with optimization. Like how is seen in the video. Fax? Also i wonder what AI is able to bring to the table but i will restrain myself from bringing up such a buzzword. Oops too late.
I think it is possible we found the bottom limit for price and new CPUs and GPUs will be faster, but also more expensive. So optimization will start coming more into play. But if computers continue to get faster and cheaper, then there isn't much need for optimization.
@@GrantKot Interesting. Thanks for your thoughts! So in other words there seems to be improvement going on still. Just not at a neck-breaking speed. I must mention this train of thought sparked by a recent video by 2kliksphilip (Release Day VS Hindsight: Computer Injustice) in which he talks about gpu's. It struck me as surprising that a certain older gpu (don't remember the name) is very reputable despite being old, and people regret not buying them back in the day when they were cheap.
Game dev graphics programmer here. There have been _numerous_ improvements in raw hardware speed, core counts, and algorithms.
CPUs have _tens_ of cores, GPUs have _thousands_ of cores.
What you may be referring to is that in the last 2 decades there has been decreasing returns in *core clock speeds* due to Silicon having a soft 5 GHz barrier. *You can’t afford the heating bill on CPUs that run at 500+ GHz.* Those are also made with GaAs due to Silicon’s inability to “scale”.
Intel held the industry back *for a decade* with quad core CPUs. AMD’s Ryzen and Threadripper brought us _”Cores for cheap”_ by giving us 6, 8, 12, 16, 24, 24, 48, 64 cores at affordable prices. Intel dropped the price of its 18-core CPU by a whopping 50% due to Threadripper!
Even “simple” algorithms can be sped up 2x to 8x by doing more calculations, doing branchless algorithms (trading latency for throughput), using small data tables, or some combination. RAM speed is SO slow compared to CPU speed that many data tables have been made obsolete - but one must measure your algorithm to know where this boundary is.
Multi threading has become mandatory for scalability of performance. I.e. compression/decompression, CPU rendering, physic’s simulation such as CFD.
Heterogeneous computing (CPU + GPU) where you try to keep both CPU & GPU workloads balanced to achieve high utilization and performance is a skill set.
Unreal Engine’s _5 _ showed us software rasterization is not dead and that it can even beat hardware rasterization. Hybrid utilization / hybrid rasterization is a game changer.
We finally have hardware raytracing where we can use custom data structures for walking a scene.
Deferred rendering and Physically Based Rendering are the norm for high quality photorealism.
We have pseudo real-time Global Illumination with multi bounce lighting.
We have path tracing in pseudo real-time.
GPUs have 12 GB, 16 GB, 24 GB, or even more VRAM allowing for bigger and bigger datasets to be used.
Not thrashing the data cache is _extremely_ important for high performance. One can have two algorithms doing the _exact_ work with the only difference being the way they access memory and one can be 8x times SLOWER!
A LOT of this is niche but there have a TON of changes to the way we approach algorithms, optimizations, you just have to look in the sub-fields of programming.
@@MichaelPohoreski Wow, well thanks for the long response!
Honestly i only grasp a small portion of what you just said, but i think i can make up of it that a whole lot has been going on, and i hardly understand the gist of it. Sorry if things fall on deaf ears haha.
It seems things have in fact improved alot. But like with any field there are many factors that influence the speed of change or progress. No such thing as linear progress! More like a bumpy line, but still upward?
@@merlinkater7756 Glad to help. If there is anything you would like clarification feel free to ask!
Computing is having a bit of "crisis". Moore's Law states that the **number of transistors** doubles roughly every 18 months. There is a direction correlation between transistor count and performance. It is becoming exceedingly more and more difficulty to use smaller and smaller distances for chip design due to "cross talk". Material Science is a whole another domain to study where the standard behavior of electricity starts to break down at these ultra TINY scales. It is WEIRD (in a good way.)
In the 90's x86 CPUs went "deep" by scaling up a single core GHz performance. That peaked when Silicon couldn't scale past 5 GHz cheaply, so x86 CPU's went "wide": Add more cores and figure out the synchronization issues at both the hardware and software level.
You may have noticed that for the last decade Thermal Management has played a HUGE part in chip design. i.e. If you have a 8 core cpu running at 5 GHz, the 16-core can only run at ~4 GHz due to all the extra heat the cores generate. If you throttle all the other cores down to 50% power usage you can run a single core at ~5.5 GHz. You may have even seen Overclockers set world records by running a CPU at 9+ GHz using Liquid Nitrogen. They basically have enough cooling to boot the computer up and take a screenshot before they are "out of juice". Thermal management and Power Efficiency drive CPUs these days.
You may have seen Apple's M1, M2, M3 chips. They are HYPER efficient using 1/4 to 1/2 of the watts that a normal x86 chip needs. If Apple would stop doing anti-consumer bone-headed decisions such as **soldering RAM onto the motherboard** they would be a _really_ nice alternative.
The funny thing is as fast as computers are they are STILL dog-slow. Once we have room temperature super conductors we are going to start to see REALLY FUN computers.
Bio-organic computing is another area to keep an eye on. The human body is basically a super-computer and Mother Nature has had Billions of years to research and develop the technology. =P
Sorry for the brain dump but I thought you would find it interesting to hear how computers have evolved in the last few decades at a high-level.
Maybe you can simulate my new theory of physics
Is it just me or do alot of people (including me) still run the 8700k, is it that solid?
Yeah it really is very solid. It had such a performance jump over my quad core that it inspired me to go 3D with my simulations. Then the 5900X forced me to learn how to multithread properly. 13900HX again has been amazing for me. Bummer about the desktop instabilities.
Houdini/blender/etc: your liquid would calculate for days
Some random guy on utub: 6:52 okay we made world scale realtime ocean
+_+
jeez dud