In 35 minutes you managed to completely explain systems that I have been trying to understand for years. No other video on the topic comes close for beginners, and a worthy follow up the Mario Galaxy video. This is extraordinary
Fun fact now that TotK is out: They actually tried fixing the Bridge of Hylia one, but there’s some of it that they missed and you can still see a cel-shadeless Link (although it is a bit more finicky). It seems some changes to Link’s textures were made (aside from just Rauru’s arm) as now you can see thick, black outlines around where Link’s abs would be
i don't think they missed it i just think they fixed it only where the decal is fully transparent, fixing it further probably is not possible with their engine
My mind was blown when you started playing with the lighting on a flat image in real time. I felt like a kid again, seeing something I never could have thought existed, like magic!
The secret of course is that it's not really a "flat" image at all, but a 3-D image encoded as colors for representation purposes. The normal buffer holds all the direction information necessary to reproduce the first light bounce off the object when combined with the material information such as the albedo (surface color) buffer. It's like you are storing a bas relief of the entire visible scene. The concept of modifying the light in an otherwise static scene is actually a whole field in computer graphics called "relighting" that is dedicated to finding ways to do this in real time for cases beyond the simple first bounce from point sources that's shown in the video.
I definitely feel like lighting is one of those things where if you do it right, people won't be sure you've done anything at all. it feels so natural when well executed it's easy to take for granted all the dedication it took behind the scenes to be that way. even coming into this video i didn't expect how many layers there were to it. fascinating stuff
A lot of these passes happen in parallel. The GPU is really good at doing a lot of similar operations in parallel. What the video failed to mention is that the techniques also evolve according to the hardware. In the past, these multi-pass techniques would be severely limited by the amount of parallelism available; it wouldn't necessarily slow down frame rate, but would introduce a lot of delay until the frame is ready for displaying. That would make the game feel like it has input lag, even though it has good frame rate overall; each frame takes a long time to bake in the GPU.
All of these calculations would have to happen even if they were still done separately for each object -- calculating the pixel's facing direction, computing stuff like shadows and fog and wrinkle maps, accumulating all the incoming light, and doing the material shading. It just feels like a lot more when each one of those stages is broken out to a lot of different buffers.
@@JasperRLZ It's like seeing the water is higher when it's poured into a narrower glass. It's the same amount of water, but it's taken in a different shape that may look like more.
@@JasperRLZ it is impressive that Nintendo got a high quality deferred rendering solution working so well on what is essentially mobile class hardware. A lot mobile games go with forward rendering because, for one, differed tends to require a lot more memory (need to store all the various buffers somewhere). Not to say the forward is limited to mobile. I believe Call of Duty goes with what they call a “Forward+” renderer.
@@TheCrewExpendable The switch uses the Nvidia tegra GPU. It's a true Nvidia graphics chip so it can handle the higher bandwidth. Also it's possible to do deferred rendering on mobile via tiled rendering - basically just breaking up the buffer into smaller sections. A lot of newer games are using this method.
The section around 16:36 where you demonstrated how a normal map can make a 2D image appear to have 3D lighting literally made me gasp. It seems so obvious having it explained this way, but I'm sure it took a lot of people a lot of work to get it to this point! Please keep making videos on whatever fascinates you most. You will have an audience :).
Indeed. I've known about Normal Maps for a while, knew what they do and how they look in their raw form but I never knew WHY they look like that or the specific details of how they work. This video explains it incredibly well.
@@Diwasho Maps like that make a lot more sense when you don't think of the channels as representing color! Just like the channels in an ordinary image are data representing color, the channels in a map represent all kinds of details relevant to rendering.
"When you do things right, people won't be sure you've done anything at all." Perhaps one of my most favourite things of all is seeing how something is done right, and learning to appreciate it all the more. It doesn't just make me appreciate the thing itself, but the people doing them, and I think it's made me appreciate people in general more. The things we're willing to do to entertain others can feel like such an unnecessarily nice gesture.
I don’t think I’ve ever learned as much about a single field within such a short time span. This video has led to me understanding many systems within game dev that I haven’t been able to graft just yet. I really hope you make more videos just like this one, analyzing the reason for something and meanwhile explaining so many other useful and interesting graphics tech solutions and standards. Mad respect! ❤
Fantastic video. Almost everything you went over there about rendering was unknown entirely to me, despite what i thought i knew about it. when you were going over the fan theories as to why it happened, I figured it would be some kind of mesh in the way with a transparent material over it, And all the leadup to you mentioning what it actually was, all just lined up perfectly as you were explaining exactly how it all fit together.
I've seen a lot of normal maps from my own decades of peeking behind curtains, but never *truly* understood them until this video. I felt totally dumbstruck when you made it sound almost basic- and it's such a cool technique! I'm thoroughly floored by how easy you made all of these incredibly complex concepts sound, and how naturally you explained them. Whatever you deep-dive into next: I can't wait to hear about it. And take your time- we can't imagine the amount of polish and edge-sanding that goes into making these videos seem so simple ;)
I sort-of understood deferred shading before, but this video made it super clear, and the glitch at the core of the video really helped me understand everything. In the demo at 16:37, are you also using the depth buffer? Otherwise it seems like the far wall would be illuminated, and the light wouldn't have the falloff on the ground.
Yes, that's correct. I didn't want to take the time to explain a depth buffer. My original video script used a directional light, and those two buffers would indeed be all that's needed for that case, but an animating point light looks a lot better and is easier to understand, so I took a small creative liberty and changed the directional light to a point light, even though I still don't mention the depth buffer. Good eye :)
@@JasperRLZ Just for directional lighting? Even for a point light I don't understand how the system would know the far wall is too far away to receive light without looking at the depth buffer.
My best guess for the buffer at 18:54 is probably a roughness buffer, or more likely some kind of specular multiplier/exponent buffer. The second one at 19:00 might be a mask for world weather/rain effects. (Link is likely black because the weather effects on characters is likely handled entirely separately from the world.)
This is probably the best video on introduction to buffers I've seen. After effects scene blew my mind even though I've programed shaders before. Congrats dude, this seems like it was a ridiculous amount of work, but well worth it!
what you said about lighting is also true when movies went from black and white to color: because there were no colors to worry about with b&w film, all you had to worry about was the lighting, and even tho those movies have no color many have aged well because film makers were paying attention to that, whereas when color film became available a lot of movies had very flat lighting because film makers became too focused on the colors in their shots and the cinematography suffered. it really wasn’t until movies like The Godfather that we started see really dynamic lighting in color movies like we did with b&w ones.
this video gets into all of the fun stuff about the rendering in an easy way to understand for me! and when it finally gets back to the main question, i really felt that "a-ha!" moment when every little detail you mentioned throughout fell into place, it was really satisfying to watch!
16:40 That's a really good explanation and I can't believe I never knew lighting was as efficient as this from the computer's perspective. That you can re-light an entire scene as though you hadn't even taken the picture yet is kind of a mindfuck when you break it down like that. Fascinating!
I just rewatched your Super Mario Galaxy video a few days ago and this was a nice treat. I love your deep dive videos, you manage to explain games in an incredibly visual way to the point where it's crazy you don't have a devkit or something. Keep em up!
Whoa! I actually found this lighting bug on my own a long time ago. I didn’t realize it was so significant to people online. Breath of the Wild is so big that it feels cool and rare that I found this organically.
Dude, that was amazing. I can't imagine the amount of time and effort you put into making this, but you nailed it. You made it simple and cohesive enough that even a dummy like me could follow without feeling completely lost, and that's a truly impressive feat. Thanks for all your hard work, and I look forward to the next video.
hahaa i literally typed this comment 5 seconds ago and now i see heaps of comments mentioning this exact thing. really goes to show how good this vid was
I've been looking forward to you showing off more BOTW since your short from last year. Nobody dives as deep or creates visuals like you Jasper. This blew me away! Thank you so much for all the work you put into these videos and your website.
Definitely more of this. Reflections could be an interesting deep dive. You could start with the Build engine (Duke 3D, Shadow Warrior, etc) requiring you to create a hollow room on the other side of a mirror to duplicate geometry into. Then move up to reflection maps, reflection cubes, dynamic reflections, screen space reflections, etc.
This is genuinely one of the most informative and entertaining videos I have ever found. You have an amazing talent that you should be proud of. Your channel is awesome and I can only hope you grow fast
This man deserves at least 1 million subs. This is the type of stuff that I find really interesting, but really intimidating so it’s nice to have something that is really easy to understand, with showing tons of visuals and examples, yet still teaches a lot and goes to over half an hour.
Fascinating. Some concepts are familiar to me, but to put into perspective all the different layers that an engine must render before showing you the final frame is insane. Now to do it 60 times or more each second sounds like a luxury.
This is one of the coolest videos I’ve ever seen… i never thought I’d ever be able to wrap my head around something like lighting and graphics in video games, it always seemed so complicated. Game development is so sick
I went through most of the video going "It's an occlusion pane!" Until you went SUPER indepth into deferred lighting, and I was half correct, but it was super interesting to learn why I was only half right, that something was being occluded, but not in a way an occlusion pane works. Super interesting and I love learning about rendering techniques in games. Great video as per usual :)
Something similar for me, from the start I had the feeling an invisible transparent object was causing the effect, but I never would have guessed that the effect was because Link was being rendered as part of the terrain.
This was an amazing video! There was so much dense information made very clear and easy to understand. It felt like it was perfectly paced so that you don’t have to pause/replay to fully understand but it was also moving at a break neck pace. I’d love more videos like these. You spoke of spending so long polishing a game that no one could can notice the hard work, and I know lots of that is going on in this video
I love people actually explaining what’s going on behind the scenes instead of giving me half-assed explanations Lighting is very scary for me because I don’t even know how you put all of that stuff together
I'm convinced that in a few years this channel will have measurably helped a generation of graphics programmers excel at their craft seriously you always do such a great job of explaining complicated concepts without using even a hint of allegory or metaphor. a mess of normals, front buffers, back buffers, vertex shaders, pixel shaders, whatever the fuck else, somehow made accessible with good rhetorical ability
Definitely not scared away by the technical details. As a technical artist that dabbles in rendering, it's extremely interesting! I like seeing the breakdown, because watching your process might actually help me get better at tackling rendering bugs that get thrown at me ha
I absolutely love how in depth your videos are and how much effort you put into making the visuals. It looks like you're using something like RenderDoc to get access to the buffers which is great. But it's absolutely impressive that you went the extra mile and showed the buffers getting written to and what they would look like before and after the decals. Especially on the glowing orb thing. That would've been so easy to just gloss over without going through the effort of getting a visual for it but you did the extra work and it's appreciated. I hope you take pride in this because you absolutely should.
Man, I have never seen any of your other vids before and this popped into my recommended, and this was INCREDIBLE! The explanation was in great detail, but presented in a way that didn't make it boring or too technical to understand to the average viewer, which is something that unfortunately you don't see too often these days. This obviously took a long time to make because of how polished it is, the choice of music goes well and isn't too intrusive, the custom examples in post-editing that you created just for the video, the comparisons to other games' visual systems, and everything in between! Plus, this is a topic that I haven't seen anywhere else much, so this was totally new to me and so interesting. I'd love more videos like this!
Blows my mind to see so many passes to get this game looking like it does. Major props to the artists and the devs that made it possible. And nice to see you back with another video! Always dig your breakdowns
This video taught me so much about stuff I technically knew about but didn't understand. From "what actually is deferred rendering besides something that tends to prevent devs from using MSAA" to "why is transparency crap or non existent in a bunch of newer games I've played", and "what are those material masks for in some PBR assets I've downloaded".
I love your technical videos the most. I'm surprised at how often the technique of building from first principle surprises me. Especially the things I already know. I've already used normal textures in a 2D environment, but seeing them manipulated manually in real-time in a video editor still gave me a youthful joy.
This all reminds me of my 8th grade math teacher. He would do this thing pretty often where he would answer someone's question with something seemingly unrelated, but as the explanation goes on he slowly shows ties with what he's explaining and the original question, until eventually the thing he was explaining which at first seemed completely unrelated is now the basis of the exact, concise answer to the student's question. Thanks for the great video :33
I've known about normals for a while, but this video did such a good job of explaining the how and why, and the after effects demonstration was brilliant. Also, OH MY GOD YES MORE PLEASE!
I feel proud of myself for correctly guessing the basic idea of the problem (overzealous occlusion shading cutting out an expensive step because it thinks the pixels are occluded for some purposes but not others), but I had no idea how complicated it was. Cool video, I always love learning about the thought and effort that goes into something I barely consciously noticed.
I remember finding out that new super Mario bros U is shaded with normal maps. I thought it was really interesting how they baked in the lighting data like that. Now I’m finding out that breath of the Wild simplifies shading by converting every rendered object into a normal map to then apply the lighting onto. What’s even crazier is that we have the capability of rendering the same frame like 20 times, then compiling all the separate information, and that all happens 60 times a second on Nintendo hardware. Takes me back to when I learned how CRT TV’s work.
BOTW targets 30, not 60. It barely hits 30 most of the time too. Deferred shading on switch is incredibly poorly performant since rendering to a pixel more than once basically obliterates the GPU, and you're extremely memory capped so it's hard to store much information in the gbuffer.
You don’t render the same frame 20 times. I doubt even a 4090 doesn’t have the room to do that. There’s a hardware feature called Multiple Render Target (MRT) allowing one render pass outputting to multiple buffers that stores different geometry data. Also switch is pretty limited on memory bandwidth. And a full-fledged deferred shading pipeline eats bandwidth like a monster. I’m actually very surprise to see how many Nintendo first party games managed to pull out deferred shading in switch - especially many with physically based materials
Incredibly well made video, actually made my day. Didn't think I'd be happily sat here watching 40 minutes of rendering techniques, but oh boy, I was fully engaged. Thank you Jasper!
TL:DR : There is a layer over the hole that forces everything behind it to be shaded like the wall, which they forgot to remove originally when they made the hole. So Link's cell shading disappears ,because it's replaced by the shading meant for the wall. This also applies to other post-processing effects, such as the glow of the bombs.
This notification popped up right as I got on RUclips to watch something. Guess I don't have to pick a video this time! With that and me re-watching the original Wind Waker cell-shading video a few days ago it almost feels like you're spying on me...
Man, all this information is amazing, but the visuals !! You went above and beyond showing all the render passes and how they are composited together, how did you even manage to get them? Amazing work !
To be fair in the village example, realistically forward rendering should be using occlusion culling to prevent the whole village from being rendered as well.
Yeah I think that line in particular undersells the main benefits of deferred lighting, which is mostly that adding more light sources is practically free
Even without occlusion culling, the opaque pass typically draws front to back and can early-out on fragments which are obscured... Buuuuut I think the example works well in a "lies to children" sort of way. For folks who know none of this stuff, it's a close-enough description of one of the main benefits of deferred lighting which is easy to understand.
@@HadynLander Doesn't that mean that the objects have to be sorted before being sent to the gpu? Is the sorting overhead less than what it would've costed to shade obscured pixels? What about large objects like terrain?
@@cylemons8099 nah, you can also do a depth prepass and then only draw pixels at equal depth in the main pass. Sorting would just make that a little more efficient, but a depth prepass makes sorting not necessary.
This is seriously incredible!! It's mind-boggling just how complicated lighting can be, even though it's something most people probably take for granted, and you explained it in a compelling, understandable way. Your custom lighting demo was really impressive too. Amazing work! ...Seeing all of this in action also makes me think about how Game Freak really needs to learn some lessons from it all.
lot's of people are confused why he'd make such a long video about a single bug, but I think the bug was really just an excuse to learn about the deffered lighting and I'm 100% fine with that. I love learning about this sort of stuff.
Awesome video :D I studied game development before Deferred Lighting was a mainstay thing so I've kinda always been scared/intimated by it, but this really summarised it quite well, and I got to learn about a fun glitch!
Your deep-dive videos are incredible. Your videos on Wind-Waker cell-shading and Super Mario Galaxy are what brought me to your channel. I learned a lot about modern graphics from this video and I hope you'll continue to make more videos like this. You explain these things in such a clear and effective way so much that you really help unwrap the magic of modern computer graphics.
I don't care how long it takes for you to upload a video. These videos are fascinating. Only you can explain how something works in a game in a manner everyone can understand. I hope I can work with 3D some day.
I had just recently discovered your channel thanks to the algorithm recommending me your video's on Galaxy and Wind Waker, and I LOVED THEM! I was a bit disappointed that you hadn't uploaded in a year so seeing this video in my recommendations made me happier than I expected. These kind of video's are just SO damn interesting, I loved every second of it! The way you explain everything is just so clean and understandable for someone who has very little technical knowledge about how video games are made, but would love to learn more about it. There's just something magical about understanding even the smallest things in how a game works that you've put hundreds of hours into. I'm already looking forward to any of your future video's!
Heya! I'm currently doing my final project in university on Breath of the Wild inspired lighting! This video is super helpful with understanding the techniques used in the game. Great work!
shhh i just didn't want to explain a depth buffer. my original script talked about a directional light which is really the 90% case but that just isn't as easy to understand visually so i changed it to a point light and told a white lie about that. but yes, the visualization used a depth buffer in that case. :)
I've never seen a more elegant explanation and demonstration of deferred lighting and normal mapping. Concepts I was aware of, but struggled to understand until now. This is one of the best videos on game development I've ever watched. Subscribed.
This video has been one of the most informative looks into modern 3D game engine lighting systems that I have ever seen. Everyone knows how well-engineered and well-developed Zelda BOTW is - with it's many dynamic systems all interacting together in beautiful ways - but taking technical deep dives into it really makes you respect the work they put into this game, even as someone who hasn't played it yet. This is what happens when a modern AAA game studio puts the time and effort into a game that it deserves, and doesn't release it until it's ready and complete.
Now compare this to Pokemon Scarlet and Violet (for those of you who don't know - despite the fact that Pokemon is one of the highest grossing franchises of any type of media, ever made in history - the games continue to be rushed messes; these recent releases are so unplayable performance wise that Nintendo has issued refunds to players disappointed with the game's performance. I just want to highlight that those games are being released years after BOTW did on the same hardware, after devs would have learned more about how to develop for the Switch, both open world games. Really shows you a lot, and this type of thing is why I stopped buying rushed corporate cash grab AAA games)
I absolutely love the effort you put into these videos. Not only do you explain why the bug happens. Not only do you say that fixing these bugs would have taken effort by the development team. No. You break down all of the technical issues in intricate detail. You program/visualize what exactly happens. You even talk about the development process for these games in a way that anyone would understand. Being a developer myself, this is mighty impressive. Good job!
Fantastic video, all of your videos are extremely informative Jasper, ever since the Mario Galaxy scrolling textures video. It blew my mind to see the faux-3D lighting being done with AE!
Mannnn, I really just wanna shout out the fact that, your videos out of any other, truly ride that ideal line of breaking these super intricate and clever systems down to be understandable and less intimidating, whilst *also* feeling no-holds-barred in terms of just HOW specific and deep you're willing to dive. I've watched more video essays about the mechanics of video games than I can count, but sooooo so many of them either go way too into the woods so it all blows over my head, or not deep enough to satisfy all the questions and details that I'd naturally want covered. The degree to which you're able to create INCREDIBLE supporting visuals and relevant examples of your topic, just to make sure the viewer is not just aware, but *familiar* with how a particular mechanic ticks, it blows my mind. Thank you so much for dedicating your time and passion to this, it really helps not just to satisfy curiosity, but make gamedev as a whole feel a *touch* less daunting.
PLEASE do more deep-dives into game rendering. I discovered your channel about a year ago with your Wind Waker video. It's fascinating learning about how much work goes into making a game look good beyond just art style, resolution, and textures, and you do such a great job explaining everything in a way that everyone can understand. I took a programming class once where we had to process image data to make a simple video player, and compared to that, I can only imagine how much more math is happening in these game engines under the hood that you have to "dumb down" for us. The fact that you reverse-engineer the game engine using rips from the exact game you're trying to explain is so helpful in seeing how everything works in real-time (or I guess slower than real time?), and I can't imagine how long that must take you to prepare. Really great work, and I look forward to your next video!
A video about a complicated topic that takes the time to fully, properly explain it so that I come away understanding what I watched? Incredible video man
This channel is a gem. Such a lovely video you've managed to create, it's pure pleasure to watch. So much information, presented in a hugely entertaining and educational way.
Live-demonstrating how deferred lighting works using Aftereffects tools blew my mind. I already had a sense of how modern lighting worked in separate passes but this is just a SUPERB explanation
I love it man. I just found out about your channel a couple a days ago, and remember being sad that your last video was a year ago. Nice to see new material so soon!
I have no prior knowledge on this, but you're so talented in explaining these things in a simple, but still highly accurate way. That's not something I take for granted, as I suck ass at explaining difficult stuff and know how challenging it can be to make it sound more simple, leaving out most of the very specific jargon and only explaining the most important things. Takes a lot of time.
Wow, I was just remembering your channel and how sad I was that you didn't upload more, and now here we are!! This was an amazing watch, thanks so much
This is honestly the first video I have ever watched that talks about video game lighting/rendering and makes the topic understandable and dare I say it, rather intuitive.
When I clicked on this, I wanted to know why the glitch happened. And then I just wanted to know everything about lighting in games. This video is really good.
This is a lovely video on how games handle lighting! Also.. the use of the deep forest music at 7:28 was so surreal, I adore that music and hearing it slowed down felt strange
Wow, what an excellent video on modern rendering techniques! Fascinating how much they can do with combining "flat" renders, and explained in such an accessible way. Kudos!
Love to see this kind of content. I’ve always seen glitches as a cool way to get behind the scenes of a game. Whether it be a t-posing model, a dev cube outside the map, or a lighting issue like this one. Replicable glitches show how systems work together by showing them fail to do so
I would absolutely love more technical deep dives, you do such a great job at visualizing all these complex systems in a way that makes them approachable to the average watcher. Glad to see you again!
Welcome back! I'm about to start a degree in game design and have been binging your videos. Thanks for breaking stuff down so interestingly and coherently!
Was expecting this to be a relatively simple explanation before I saw the video length, but I learned a lot about render passes and types of filters games use, while it was all easy to follow.
Amazing video as always! Your channel is turning more and more into a treasure trove for game devs!! Also, Elebits was underrated and your use of its music gives me nostalgic goosebumps! Thank you so much for everything you're giving us!!
Watching you use after effects to manually render a frame from the game blew my mind. I have never been able to wrap my head around the concept of a game engine until I saw you do that.
I've had the body of water you show at 21:47 glitch just like that! totally blew my mind to see all those beautiful marine plants w/o the rendered water surface.
This dude made a 35 minute video on this 🤦♂️
It's damn good too. Learned a lot about modern game renderering.
Coding be like
And it is so good!
@@chrispy5249 I’d make a bowl of Mac and cheese
and it held my attention better than any lecture i went to in school
In 35 minutes you managed to completely explain systems that I have been trying to understand for years. No other video on the topic comes close for beginners, and a worthy follow up the Mario Galaxy video. This is extraordinary
wait a minute? you here?
This is exactly how I felt about this.
Fun fact now that TotK is out: They actually tried fixing the Bridge of Hylia one, but there’s some of it that they missed and you can still see a cel-shadeless Link (although it is a bit more finicky). It seems some changes to Link’s textures were made (aside from just Rauru’s arm) as now you can see thick, black outlines around where Link’s abs would be
damn so link got more jacked?
@@alexgao2587He ain't no twink anymore. 🥶
@@serialdreamer8386Yeah, he's a himbo now.
i don't think they missed it i just think they fixed it only where the decal is fully transparent, fixing it further probably is not possible with their engine
@@serialdreamer8386 he's a twunk now
My mind was blown when you started playing with the lighting on a flat image in real time. I felt like a kid again, seeing something I never could have thought existed, like magic!
Same
Mine too!
Thanks for the video suggestion in the first place, Korri!
The secret of course is that it's not really a "flat" image at all, but a 3-D image encoded as colors for representation purposes. The normal buffer holds all the direction information necessary to reproduce the first light bounce off the object when combined with the material information such as the albedo (surface color) buffer. It's like you are storing a bas relief of the entire visible scene.
The concept of modifying the light in an otherwise static scene is actually a whole field in computer graphics called "relighting" that is dedicated to finding ways to do this in real time for cases beyond the simple first bounce from point sources that's shown in the video.
@@JasperRLZ Wait, did I?? Hahahaha! What a wonderful memory I have 🤣
I definitely feel like lighting is one of those things where if you do it right, people won't be sure you've done anything at all. it feels so natural when well executed it's easy to take for granted all the dedication it took behind the scenes to be that way. even coming into this video i didn't expect how many layers there were to it. fascinating stuff
Sound and music design too. They're not thankless, but they are easy to overlook when done well.
dude I love futurama
People won’t notice when it’s right but when something is off it’s jarring
@@IofSheikah totally, kind of an uncanny valley vibe
Only software engineers would think of making lighting and shading faster by rendering their frame 60 times each frame instead of once
A lot of these passes happen in parallel. The GPU is really good at doing a lot of similar operations in parallel. What the video failed to mention is that the techniques also evolve according to the hardware. In the past, these multi-pass techniques would be severely limited by the amount of parallelism available; it wouldn't necessarily slow down frame rate, but would introduce a lot of delay until the frame is ready for displaying. That would make the game feel like it has input lag, even though it has good frame rate overall; each frame takes a long time to bake in the GPU.
All of these calculations would have to happen even if they were still done separately for each object -- calculating the pixel's facing direction, computing stuff like shadows and fog and wrinkle maps, accumulating all the incoming light, and doing the material shading. It just feels like a lot more when each one of those stages is broken out to a lot of different buffers.
@@JasperRLZ It's like seeing the water is higher when it's poured into a narrower glass. It's the same amount of water, but it's taken in a different shape that may look like more.
@@JasperRLZ it is impressive that Nintendo got a high quality deferred rendering solution working so well on what is essentially mobile class hardware. A lot mobile games go with forward rendering because, for one, differed tends to require a lot more memory (need to store all the various buffers somewhere).
Not to say the forward is limited to mobile. I believe Call of Duty goes with what they call a “Forward+” renderer.
@@TheCrewExpendable The switch uses the Nvidia tegra GPU. It's a true Nvidia graphics chip so it can handle the higher bandwidth. Also it's possible to do deferred rendering on mobile via tiled rendering - basically just breaking up the buffer into smaller sections. A lot of newer games are using this method.
The section around 16:36 where you demonstrated how a normal map can make a 2D image appear to have 3D lighting literally made me gasp. It seems so obvious having it explained this way, but I'm sure it took a lot of people a lot of work to get it to this point!
Please keep making videos on whatever fascinates you most. You will have an audience :).
Indeed. I've known about Normal Maps for a while, knew what they do and how they look in their raw form but I never knew WHY they look like that or the specific details of how they work. This video explains it incredibly well.
@@Diwasho Maps like that make a lot more sense when you don't think of the channels as representing color! Just like the channels in an ordinary image are data representing color, the channels in a map represent all kinds of details relevant to rendering.
I'm just surpised that people dont know what normal maps are haha
@@snufhuffboy5101 Knowing how many people can't install a Wi-Fi printer will blow your mind then.
SAME !
"When you do things right, people won't be sure you've done anything at all."
Perhaps one of my most favourite things of all is seeing how something is done right, and learning to appreciate it all the more. It doesn't just make me appreciate the thing itself, but the people doing them, and I think it's made me appreciate people in general more. The things we're willing to do to entertain others can feel like such an unnecessarily nice gesture.
that philosophy warms my heart.
Nice Futurama reference
I don’t think I’ve ever learned as much about a single field within such a short time span. This video has led to me understanding many systems within game dev that I haven’t been able to graft just yet. I really hope you make more videos just like this one, analyzing the reason for something and meanwhile explaining so many other useful and interesting graphics tech solutions and standards. Mad respect! ❤
Fantastic video. Almost everything you went over there about rendering was unknown entirely to me, despite what i thought i knew about it. when you were going over the fan theories as to why it happened, I figured it would be some kind of mesh in the way with a transparent material over it, And all the leadup to you mentioning what it actually was, all just lined up perfectly as you were explaining exactly how it all fit together.
Did u hack?
@@lederp42 Im 4 133t h4ck3rm4n
@@lederp42 unlisted video, early view and comment, then publishing a day later (we only see the publish date, not upload date)
@@lederp42 Patreon exclusive early viewing.
I've seen a lot of normal maps from my own decades of peeking behind curtains, but never *truly* understood them until this video. I felt totally dumbstruck when you made it sound almost basic- and it's such a cool technique! I'm thoroughly floored by how easy you made all of these incredibly complex concepts sound, and how naturally you explained them. Whatever you deep-dive into next: I can't wait to hear about it.
And take your time- we can't imagine the amount of polish and edge-sanding that goes into making these videos seem so simple ;)
I sort-of understood deferred shading before, but this video made it super clear, and the glitch at the core of the video really helped me understand everything. In the demo at 16:37, are you also using the depth buffer? Otherwise it seems like the far wall would be illuminated, and the light wouldn't have the falloff on the ground.
That demo confused me as well, and depth would seem to be the missing piece of information.
Yes, that's correct. I didn't want to take the time to explain a depth buffer. My original video script used a directional light, and those two buffers would indeed be all that's needed for that case, but an animating point light looks a lot better and is easier to understand, so I took a small creative liberty and changed the directional light to a point light, even though I still don't mention the depth buffer. Good eye :)
@@JasperRLZ Ah, that makes sense. I must say the point light did make for a much more compelling visual! Loved the video!
@@JasperRLZ Just for directional lighting? Even for a point light I don't understand how the system would know the far wall is too far away to receive light without looking at the depth buffer.
I meant that directional lighting wouldn't require a depth buffer, since it has no distance falloff. Point lighting requires one.
My best guess for the buffer at 18:54 is probably a roughness buffer, or more likely some kind of specular multiplier/exponent buffer. The second one at 19:00 might be a mask for world weather/rain effects. (Link is likely black because the weather effects on characters is likely handled entirely separately from the world.)
This is probably the best video on introduction to buffers I've seen. After effects scene blew my mind even though I've programed shaders before.
Congrats dude, this seems like it was a ridiculous amount of work, but well worth it!
what you said about lighting is also true when movies went from black and white to color: because there were no colors to worry about with b&w film, all you had to worry about was the lighting, and even tho those movies have no color many have aged well because film makers were paying attention to that, whereas when color film became available a lot of movies had very flat lighting because film makers became too focused on the colors in their shots and the cinematography suffered. it really wasn’t until movies like The Godfather that we started see really dynamic lighting in color movies like we did with b&w ones.
I feel like a similar transition happened when we started utilizing CGI more as well, and we're only just now figuring it out
this video gets into all of the fun stuff about the rendering in an easy way to understand for me! and when it finally gets back to the main question, i really felt that "a-ha!" moment when every little detail you mentioned throughout fell into place, it was really satisfying to watch!
Truly the avengers endgame of video game lighting and rendering
16:40 That's a really good explanation and I can't believe I never knew lighting was as efficient as this from the computer's perspective. That you can re-light an entire scene as though you hadn't even taken the picture yet is kind of a mindfuck when you break it down like that. Fascinating!
I just rewatched your Super Mario Galaxy video a few days ago and this was a nice treat. I love your deep dive videos, you manage to explain games in an incredibly visual way to the point where it's crazy you don't have a devkit or something. Keep em up!
Whoa! I actually found this lighting bug on my own a long time ago. I didn’t realize it was so significant to people online. Breath of the Wild is so big that it feels cool and rare that I found this organically.
Dude, that was amazing. I can't imagine the amount of time and effort you put into making this, but you nailed it. You made it simple and cohesive enough that even a dummy like me could follow without feeling completely lost, and that's a truly impressive feat. Thanks for all your hard work, and I look forward to the next video.
hahaa i literally typed this comment 5 seconds ago and now i see heaps of comments mentioning this exact thing. really goes to show how good this vid was
I've been looking forward to you showing off more BOTW since your short from last year. Nobody dives as deep or creates visuals like you Jasper. This blew me away! Thank you so much for all the work you put into these videos and your website.
Definitely more of this. Reflections could be an interesting deep dive. You could start with the Build engine (Duke 3D, Shadow Warrior, etc) requiring you to create a hollow room on the other side of a mirror to duplicate geometry into. Then move up to reflection maps, reflection cubes, dynamic reflections, screen space reflections, etc.
The fact that all those images are generated and processed around 30 times per a single second is insane
This is genuinely one of the most informative and entertaining videos I have ever found. You have an amazing talent that you should be proud of. Your channel is awesome and I can only hope you grow fast
Completely agree. Was engrossed the entire time and the whole thing was both enlightening and fascinating.
@@kiwaminski Heh heh, "enlightening"
@@theSato Huehehe. Didn't even notice the pun, nice catch
This man deserves at least 1 million subs. This is the type of stuff that I find really interesting, but really intimidating so it’s nice to have something that is really easy to understand, with showing tons of visuals and examples, yet still teaches a lot and goes to over half an hour.
I agree with Squag
agreed
Fascinating. Some concepts are familiar to me, but to put into perspective all the different layers that an engine must render before showing you the final frame is insane. Now to do it 60 times or more each second sounds like a luxury.
Rendering that many frames is easy when they're so simple relative to what it would have to do otherwise.
This is one of the coolest videos I’ve ever seen… i never thought I’d ever be able to wrap my head around something like lighting and graphics in video games, it always seemed so complicated. Game development is so sick
I went through most of the video going "It's an occlusion pane!" Until you went SUPER indepth into deferred lighting, and I was half correct, but it was super interesting to learn why I was only half right, that something was being occluded, but not in a way an occlusion pane works. Super interesting and I love learning about rendering techniques in games. Great video as per usual :)
Something similar for me, from the start I had the feeling an invisible transparent object was causing the effect, but I never would have guessed that the effect was because Link was being rendered as part of the terrain.
@@amymason156 You could say he became one with the Earth!
This was an amazing video! There was so much dense information made very clear and easy to understand. It felt like it was perfectly paced so that you don’t have to pause/replay to fully understand but it was also moving at a break neck pace. I’d love more videos like these. You spoke of spending so long polishing a game that no one could can notice the hard work, and I know lots of that is going on in this video
I love people actually explaining what’s going on behind the scenes instead of giving me half-assed explanations
Lighting is very scary for me because I don’t even know how you put all of that stuff together
I'm convinced that in a few years this channel will have measurably helped a generation of graphics programmers excel at their craft
seriously you always do such a great job of explaining complicated concepts without using even a hint of allegory or metaphor. a mess of normals, front buffers, back buffers, vertex shaders, pixel shaders, whatever the fuck else, somehow made accessible with good rhetorical ability
Definitely not scared away by the technical details. As a technical artist that dabbles in rendering, it's extremely interesting!
I like seeing the breakdown, because watching your process might actually help me get better at tackling rendering bugs that get thrown at me ha
I absolutely love how in depth your videos are and how much effort you put into making the visuals. It looks like you're using something like RenderDoc to get access to the buffers which is great. But it's absolutely impressive that you went the extra mile and showed the buffers getting written to and what they would look like before and after the decals. Especially on the glowing orb thing. That would've been so easy to just gloss over without going through the effort of getting a visual for it but you did the extra work and it's appreciated. I hope you take pride in this because you absolutely should.
Man, I have never seen any of your other vids before and this popped into my recommended, and this was INCREDIBLE! The explanation was in great detail, but presented in a way that didn't make it boring or too technical to understand to the average viewer, which is something that unfortunately you don't see too often these days. This obviously took a long time to make because of how polished it is, the choice of music goes well and isn't too intrusive, the custom examples in post-editing that you created just for the video, the comparisons to other games' visual systems, and everything in between! Plus, this is a topic that I haven't seen anywhere else much, so this was totally new to me and so interesting. I'd love more videos like this!
Blows my mind to see so many passes to get this game looking like it does. Major props to the artists and the devs that made it possible.
And nice to see you back with another video! Always dig your breakdowns
This video taught me so much about stuff I technically knew about but didn't understand. From "what actually is deferred rendering besides something that tends to prevent devs from using MSAA" to "why is transparency crap or non existent in a bunch of newer games I've played", and "what are those material masks for in some PBR assets I've downloaded".
Incredible work, I'd love to see more of these! It felt like you were dissecting a luxury car engine and showing how all the pieces work together
I love your technical videos the most.
I'm surprised at how often the technique of building from first principle surprises me. Especially the things I already know. I've already used normal textures in a 2D environment, but seeing them manipulated manually in real-time in a video editor still gave me a youthful joy.
It’s been a while but your videos are always gonna be worth it.
This all reminds me of my 8th grade math teacher. He would do this thing pretty often where he would answer someone's question with something seemingly unrelated, but as the explanation goes on he slowly shows ties with what he's explaining and the original question, until eventually the thing he was explaining which at first seemed completely unrelated is now the basis of the exact, concise answer to the student's question. Thanks for the great video :33
I've known about normals for a while, but this video did such a good job of explaining the how and why, and the after effects demonstration was brilliant.
Also, OH MY GOD YES MORE PLEASE!
I feel proud of myself for correctly guessing the basic idea of the problem (overzealous occlusion shading cutting out an expensive step because it thinks the pixels are occluded for some purposes but not others), but I had no idea how complicated it was. Cool video, I always love learning about the thought and effort that goes into something I barely consciously noticed.
I remember finding out that new super Mario bros U is shaded with normal maps. I thought it was really interesting how they baked in the lighting data like that. Now I’m finding out that breath of the Wild simplifies shading by converting every rendered object into a normal map to then apply the lighting onto. What’s even crazier is that we have the capability of rendering the same frame like 20 times, then compiling all the separate information, and that all happens 60 times a second on Nintendo hardware. Takes me back to when I learned how CRT TV’s work.
BOTW targets 30, not 60. It barely hits 30 most of the time too. Deferred shading on switch is incredibly poorly performant since rendering to a pixel more than once basically obliterates the GPU, and you're extremely memory capped so it's hard to store much information in the gbuffer.
You don’t render the same frame 20 times. I doubt even a 4090 doesn’t have the room to do that.
There’s a hardware feature called Multiple Render Target (MRT) allowing one render pass outputting to multiple buffers that stores different geometry data.
Also switch is pretty limited on memory bandwidth. And a full-fledged deferred shading pipeline eats bandwidth like a monster. I’m actually very surprise to see how many Nintendo first party games managed to pull out deferred shading in switch - especially many with physically based materials
I can't IMAGINE how much work this video took to make. Absolutely phenomenal job.
Incredibly well made video, actually made my day. Didn't think I'd be happily sat here watching 40 minutes of rendering techniques, but oh boy, I was fully engaged. Thank you Jasper!
TL:DR : There is a layer over the hole that forces everything behind it to be shaded like the wall, which they forgot to remove originally when they made the hole. So Link's cell shading disappears ,because it's replaced by the shading meant for the wall. This also applies to other post-processing effects, such as the glow of the bombs.
This notification popped up right as I got on RUclips to watch something. Guess I don't have to pick a video this time!
With that and me re-watching the original Wind Waker cell-shading video a few days ago it almost feels like you're spying on me...
Man, all this information is amazing, but the visuals !! You went above and beyond showing all the render passes and how they are composited together, how did you even manage to get them?
Amazing work !
To be fair in the village example, realistically forward rendering should be using occlusion culling to prevent the whole village from being rendered as well.
Yeah I think that line in particular undersells the main benefits of deferred lighting, which is mostly that adding more light sources is practically free
Even without occlusion culling, the opaque pass typically draws front to back and can early-out on fragments which are obscured... Buuuuut I think the example works well in a "lies to children" sort of way. For folks who know none of this stuff, it's a close-enough description of one of the main benefits of deferred lighting which is easy to understand.
@@HadynLander Doesn't that mean that the objects have to be sorted before being sent to the gpu? Is the sorting overhead less than what it would've costed to shade obscured pixels? What about large objects like terrain?
@@cylemons8099 nah, you can also do a depth prepass and then only draw pixels at equal depth in the main pass. Sorting would just make that a little more efficient, but a depth prepass makes sorting not necessary.
The Return of the King, one of the only people I’ve seen talk about Galaxy’s graphics and I’m so glad to see you upload again!
Your videos are so high quality. I'm in a Computer Graphics course right now, so this is absolutely fascinating.
This is seriously incredible!! It's mind-boggling just how complicated lighting can be, even though it's something most people probably take for granted, and you explained it in a compelling, understandable way. Your custom lighting demo was really impressive too. Amazing work!
...Seeing all of this in action also makes me think about how Game Freak really needs to learn some lessons from it all.
lot's of people are confused why he'd make such a long video about a single bug, but I think the bug was really just an excuse to learn about the deffered lighting and I'm 100% fine with that. I love learning about this sort of stuff.
Awesome video :D
I studied game development before Deferred Lighting was a mainstay thing so I've kinda always been scared/intimated by it, but this really summarised it quite well, and I got to learn about a fun glitch!
Then, when the world needed him most, he came back!
Your deep-dive videos are incredible. Your videos on Wind-Waker cell-shading and Super Mario Galaxy are what brought me to your channel. I learned a lot about modern graphics from this video and I hope you'll continue to make more videos like this. You explain these things in such a clear and effective way so much that you really help unwrap the magic of modern computer graphics.
Hey, you're an incredible video creator man. The amount of effort you put in does not go unnoticed. Well done!
Your editing is insane genuinely. I love the effort put into this so much man
also love the choice of baba is you music!
I don't care how long it takes for you to upload a video. These videos are fascinating. Only you can explain how something works in a game in a manner everyone can understand. I hope I can work with 3D some day.
I had just recently discovered your channel thanks to the algorithm recommending me your video's on Galaxy and Wind Waker, and I LOVED THEM! I was a bit disappointed that you hadn't uploaded in a year so seeing this video in my recommendations made me happier than I expected. These kind of video's are just SO damn interesting, I loved every second of it!
The way you explain everything is just so clean and understandable for someone who has very little technical knowledge about how video games are made, but would love to learn more about it. There's just something magical about understanding even the smallest things in how a game works that you've put hundreds of hours into.
I'm already looking forward to any of your future video's!
As Arin Hansen would say: M O A R
DID U HACK?
@@Cliffordlonghead no, it was on his Patreon
Heya! I'm currently doing my final project in university on Breath of the Wild inspired lighting! This video is super helpful with understanding the techniques used in the game. Great work!
Your after effects relighting setup didn't require a depth buffer? How did that point light know to not light up pixels in outside of its radius?
shhh i just didn't want to explain a depth buffer. my original script talked about a directional light which is really the 90% case but that just isn't as easy to understand visually so i changed it to a point light and told a white lie about that. but yes, the visualization used a depth buffer in that case. :)
I've never seen a more elegant explanation and demonstration of deferred lighting and normal mapping. Concepts I was aware of, but struggled to understand until now. This is one of the best videos on game development I've ever watched. Subscribed.
This video has been one of the most informative looks into modern 3D game engine lighting systems that I have ever seen. Everyone knows how well-engineered and well-developed Zelda BOTW is - with it's many dynamic systems all interacting together in beautiful ways - but taking technical deep dives into it really makes you respect the work they put into this game, even as someone who hasn't played it yet. This is what happens when a modern AAA game studio puts the time and effort into a game that it deserves, and doesn't release it until it's ready and complete.
Now compare this to Pokemon Scarlet and Violet (for those of you who don't know - despite the fact that Pokemon is one of the highest grossing franchises of any type of media, ever made in history - the games continue to be rushed messes; these recent releases are so unplayable performance wise that Nintendo has issued refunds to players disappointed with the game's performance. I just want to highlight that those games are being released years after BOTW did on the same hardware, after devs would have learned more about how to develop for the Switch, both open world games. Really shows you a lot, and this type of thing is why I stopped buying rushed corporate cash grab AAA games)
I absolutely love the effort you put into these videos. Not only do you explain why the bug happens. Not only do you say that fixing these bugs would have taken effort by the development team.
No. You break down all of the technical issues in intricate detail. You program/visualize what exactly happens. You even talk about the development process for these games in a way that anyone would understand.
Being a developer myself, this is mighty impressive. Good job!
Opposite of clickbait. The video was more interesting than the title made it seem.
Oh man, I’m so glad this showed up in my feed. Incredibly interesting, very entertaining and really neat editing. Instant sub!
Fantastic video, all of your videos are extremely informative Jasper, ever since the Mario Galaxy scrolling textures video.
It blew my mind to see the faux-3D lighting being done with AE!
This explanation is absolutely insane! Really professional editing and narrative.
Mannnn, I really just wanna shout out the fact that, your videos out of any other, truly ride that ideal line of breaking these super intricate and clever systems down to be understandable and less intimidating, whilst *also* feeling no-holds-barred in terms of just HOW specific and deep you're willing to dive. I've watched more video essays about the mechanics of video games than I can count, but sooooo so many of them either go way too into the woods so it all blows over my head, or not deep enough to satisfy all the questions and details that I'd naturally want covered.
The degree to which you're able to create INCREDIBLE supporting visuals and relevant examples of your topic, just to make sure the viewer is not just aware, but *familiar* with how a particular mechanic ticks, it blows my mind. Thank you so much for dedicating your time and passion to this, it really helps not just to satisfy curiosity, but make gamedev as a whole feel a *touch* less daunting.
PLEASE do more deep-dives into game rendering. I discovered your channel about a year ago with your Wind Waker video. It's fascinating learning about how much work goes into making a game look good beyond just art style, resolution, and textures, and you do such a great job explaining everything in a way that everyone can understand.
I took a programming class once where we had to process image data to make a simple video player, and compared to that, I can only imagine how much more math is happening in these game engines under the hood that you have to "dumb down" for us. The fact that you reverse-engineer the game engine using rips from the exact game you're trying to explain is so helpful in seeing how everything works in real-time (or I guess slower than real time?), and I can't imagine how long that must take you to prepare.
Really great work, and I look forward to your next video!
A video about a complicated topic that takes the time to fully, properly explain it so that I come away understanding what I watched? Incredible video man
This channel is a gem. Such a lovely video you've managed to create, it's pure pleasure to watch. So much information, presented in a hugely entertaining and educational way.
Its crazy how engaging a video about lighting in game I haven’t played before be so engaging
Live-demonstrating how deferred lighting works using Aftereffects tools blew my mind. I already had a sense of how modern lighting worked in separate passes but this is just a SUPERB explanation
I love it man. I just found out about your channel a couple a days ago, and remember being sad that your last video was a year ago. Nice to see new material so soon!
I have no prior knowledge on this, but you're so talented in explaining these things in a simple, but still highly accurate way. That's not something I take for granted, as I suck ass at explaining difficult stuff and know how challenging it can be to make it sound more simple, leaving out most of the very specific jargon and only explaining the most important things. Takes a lot of time.
Wow, I was just remembering your channel and how sad I was that you didn't upload more, and now here we are!! This was an amazing watch, thanks so much
This is honestly the first video I have ever watched that talks about video game lighting/rendering and makes the topic understandable and dare I say it, rather intuitive.
When I clicked on this, I wanted to know why the glitch happened. And then I just wanted to know everything about lighting in games. This video is really good.
This is a lovely video on how games handle lighting! Also.. the use of the deep forest music at 7:28 was so surreal, I adore that music and hearing it slowed down felt strange
Man this is basically a "rendering 101" lecture and I am excited to be using this to explaining this stuff to my friends without gamedev experience :D
one of the best videos I've seen in a while. genuinely phenomenal work, loved the visualizations
Please do a whole series on rendering & game-dev techniques in general, I love how clearly you're able to explain difficult stuff :)
Wow, what an excellent video on modern rendering techniques! Fascinating how much they can do with combining "flat" renders, and explained in such an accessible way. Kudos!
Love to see this kind of content.
I’ve always seen glitches as a cool way to get behind the scenes of a game. Whether it be a t-posing model, a dev cube outside the map, or a lighting issue like this one. Replicable glitches show how systems work together by showing them fail to do so
Yes, please more in-depth videos, this one is exactly what I needed to finally understand how deferred lighting works!
I would absolutely love more technical deep dives, you do such a great job at visualizing all these complex systems in a way that makes them approachable to the average watcher. Glad to see you again!
Welcome back! I'm about to start a degree in game design and have been binging your videos. Thanks for breaking stuff down so interestingly and coherently!
Was expecting this to be a relatively simple explanation before I saw the video length, but I learned a lot about render passes and types of filters games use, while it was all easy to follow.
Amazing video as always! Your channel is turning more and more into a treasure trove for game devs!!
Also, Elebits was underrated and your use of its music gives me nostalgic goosebumps!
Thank you so much for everything you're giving us!!
At some point I completely forgot this video was about zelda and I was like "this is a very nice video about lighting in video games "
Watching you use after effects to manually render a frame from the game blew my mind. I have never been able to wrap my head around the concept of a game engine until I saw you do that.
The Baba is You music fits literally perfect to these videos, great choice
it is so good to have you back! i know these videos take an incredibly long time but i can only hope there isn't another 2 year wait for the next one
I've learned so much about computer graphics and you kept me hooked for the whole video, make more of these PLEASE
Found your channel just 2 days ago. Was sad to see it inactive, but now super glad it isn't!
I've had the body of water you show at 21:47 glitch just like that! totally blew my mind to see all those beautiful marine plants w/o the rendered water surface.
I’ve been waiting for a new video of yours! I always kept thinking about the scrolling textures and how well produced it was. Great work yet again!