It has been almost 10yrs that i have first wondered what the advancements of software and math optimizations could bring to a hardware as old as the N64, my prayers are finally being answered
”Fresnel and light-to-alpha for bump mapping” Holy sh*t, someone actually pulled off Bump Mapping on N64! I don’t even care if it’s done per vertex or per pixel, it’s still wild to see it in action.
This is amazing! To think that such old hardware would even be cabable of this. SGI and Nintendo made something that was ahead of its time. The other video showing ambient occlusion blew my mind aswell. Programming this to run in a performant manner must have been very difficult.
I won't say any of the microcode stuff is easy, but given the difficulty of the platform, implementing ambient occlusion was relatively easy. It just does light *= 1 - factor * (1 - alpha), and uses a different factor for ambient / directional / point lights.
@@contramachina354 Yes, Kaze will be using F3DEX3 for his upcoming project Return to Yoshi's Island. However, he'll only be using it for its performance improvements, and won't be using any of the new graphical features, as the new features are not compatible with the legacy HLE emulators most of his audience uses.
This really gives a 6th gen feeling to the N64, massive props to you for figuring this out! Now I'd love to see how Kaze Emanuar could possibly implement something similar into his custom SM64 engine
Kaze is planning to use F3DEX3 in RTYI for its performance improvements. However, he won't be using any of its new graphical features, as they are not compatible with legacy HLE emulators which most of his playerbase uses.
@@xdanic3 It's a chicken-and-egg problem. Part of the problem is that top quality LLE emulators are still too slow on low-end PCs or Android. In five years, that probably won't be a problem anymore.
@@debugempty Most emulators with LLE graphics. Ares or paraLLEl are recommended, but some others work too. Personally I rarely use emulators, this is developed on real hardware. I recently switched from an Everdrive 64 V3 to a SummerCart64, which has very fast USB upload.
when this is finished i hope it gets used to its full potential! with all the homebrew that going on for the n64 i think a new era of n64 games are on the horizon. this demo is amazing imo even if the process it "fairly simple" the final result is incredible for the n64 hardware!
The specular vertex lighting is very neat. Looks visually similar to the original unreal engine's vertex specular effect (though the light blending itself is different by far), but a really big advantage of the UE1 effect is that you can control a lot of the parameters between the width and strength, and an additional multiplier for it on top of the regular lighting. Impressive stuff for 1998, and it's very impressive that something close is being done on the N64 after all this time. Awesome job, man
I didn't talk about it in the video, but there is a "size" parameter here for each light when in specular mode. The strength is determined by the light color. I originally tried to implement diffuse + specular with a multiplier for each, but I ran out of instruction memory.
@@Sauraen Despite the limitations, I think it looks really nice. That's how it goes with either hardware or software limitations of this period though, you'll have to give ground to them eventually. Very nice job though, what you've managed to do with this hardware is wonderful
some people go to school, to become layers, doctors, researches. They put their energy to curing disease, fighting for people's rights in the legal system, or making innovations to our world. And then there's people like this (working hard to improve lighting in a 25 year old video game)
This is pretty cool, with some vertex colors on the ground to "absorb" red from the colors and some planning ahead the water could look very impressive, much like how mario sunshine's water on gamecube works
It won't get this implementation, this is based on Nintendo's libultra which is incompatible with libdragon. The homebrew community will have to implement fresnel in their own custom microcodes.
@@Sauraen hm well specifically I meant "an implementation with equivalent or better behavior", obviously I know libdragon and libultea have different features and apis
The two graphical features I'm most looking forward to seeing some makeshift/"good-enough" version of on the N64 are quality full-screen Motion Blur (effectively like either Camera+Object blur or the Per Pixel kinds) and dynamic Depth of Field. Not sure if that's on your radar at all or if you have any ideas how one would approach that, but this gives me hope that someone might implement that someday.
Majora's Mask has motion blur by mixing the last framebuffer with some alpha into the current one. Anything more complex than that is probably out of scope for the hardware, and this doesn't have much to do with custom microcode anyway. What custom microcode can do is vertex-based effects which end up producing a new shade color or alpha or texture coordinates for each vertex.
Are the following possible on the N64 through microcode? - n-Simplex interpolation/filtering - Summed Area Tables - Progressive Mesh Refinement (different from LOD, but has a similar goal) - MFAA (or something similar) - Quads - Textures using other colorspaces, like Y-CoCg - Raymarching - Blinn-Phong Shading - Directional Colored Shadows (to compliment Colored Lights)
The microcode is sort of the equivalent of a compute shader which runs where you would normally have a vertex shader in the pipeline. It can theoretically compute anything, but there are severe memory limitations. Anything which would be normally part of a fragment shader is handled by the RDP, which has a configurable but fixed-function pipeline--this has nothing to do with the microcode. - No idea what that is - No idea what that is - Theoretically yes, practically LoD-like stuff is mostly done on the CPU - This is a fragment related thing. The N64 has antialiasing which has a couple settings but it is not that capable. - Already supported as two tris. Having a real quad with an undistorted texture on it is supported by the hardware but no microcode implements this. - The RDP supports like a dozen texture formats, I've used several in this video (see my video on fast64 textures). For the kind of textures you're talking about, there's only one supported: YUYV. Games rarely used this and community tools don't currently support it. - Fragment related, also please, we're talking about a system designed in 1994. - Any lighting effect which you're willing to approximate as a per-vertex effect can be done, like I did here with specular. - OoT already uses dynamic shadows for some objects, there's no reason they could not be drawn with colors.
reggarding the environment mapping technique. While it is no more than a static texture, one could re-generate that static texture every frame by rendering it to an off-screen buffer and using its results as the env. map for the desired objects. That way, one could potentially render a cubemap into the env. texture, or even render a very simplified impostor of the scene with correct time of day lighting...
I never thought anyone would think it would be worth it to implement either ambient occlusion or fresnel on the N64, let alone both. But, here we are. 😅 Do you think the shade-to-alpha could be used in combination with some other techniques to produce an inexpensive(-ish) subsurface scattering effect? 🤔
Does Donkey Kong 64 also use the original texture-based crusty specular implementation for it's shine on the character models?, the sheen the kongs have is surely not true specular, but also look suspiciously smooth compared to the og crusty cubemap specular... I wonder if it's anything custom/special Rare did.
Is this available for the public? I've been making some mods for OoT and SM64 decomps with the Fast64 plugin. Is this integrated into that? Or is there some other way to load this microcode? I'm still pretty new to this...
All of this is public, check the links in the description. All the scenes shown in my videos are made with fast64, and the modifications to fast64 to support them are public. However, neither the microcode or the fast64 support is finished, so there are some incomplete or unsupported features and things may break in the future.
Not directly. Homebrew must not contain a single line of Nintendo code. So the OS and CPU rendering engine is completely different, and the microcodes are totally incompatible. However, I hope that F3DEX3 inspires homebrew microcode developers to support things like vertex colors and normals at the same time.
Imagine if the N64 had 16KB texture cache and memory without horrible latency issues. Would've had crystal clear textures and higher frame rates for maybe an extra $10-15 per console.
The texture cache could have been expanded for some small cost like that, but the memory system could not have been easily redesigned. The main constraint was development time--SGI made RCP V1, it had two major bugs, so they made V2 which took an extra 9 months. The console was then already behind schedule so they had to ship it. There are still a number of minor bugs in V2, plus supposedly one of their engineers found an issue causing memory inefficiency in V2 which they would have fixed if they did a V3. I also personally found a few changes they could make which would have improved performance by maybe up to 30% without significant silicon area cost, maybe I'll get a chance to talk about those someday. :)
I wanna make a game for the N64 now (but I wish I had the time). x'3 P.S. You should try to do the shadowmapping in Conker's Bad Fur Day and/or stencil shadow volumes for the shadows of Link next.
I'm not planning any F3DEX3 features for shadows. OoT's dynamic shadows system is about as good as you can do on N64 without major engine changes or a big perf hit. It is used in the final game for a handful of actors, plus there is beta code which can be re-enabled which draws the dynamic shadow for Link.
actually, OoT already had a dynamic projected shadow system similar to Conker's, though I've only noticed it beeing used for the shadows of the moveable mirrors in the Sand Temple, and for shadows of Ganon's cape in the last boss.
@@MyakoEventually there should be a HackerOoT release with the microcode already built-in. For someone who has a little experience with OoT romhacking, it's pretty straightforward to export the scene and models from Blender with fast64, and copy in the code which is also released. The target audience is romhack developers, not players--this demo isn't a game, it's only what you're seeing on screen here. In the near future, no--I'm interested in finishing F3DEX3 and the related tooling to improve the state-of-the-art in SM64 and OoT romhacking. If/when I finish it, I might switch to homebrew, though I probably would move on to non-N64 stuff.
Damn nintendo really had no idea what they had in their hands with the SGI gpu did they? I wish i could see the alternate world were sega of america menaged to convince japan to use the SGI gpu
I know you're kidding, but someone already did ray tracing on N64. It worked but only could handle a scene with a handful of basic primitives like spheres and cubes.
Not very taxing, it depends on how many lights and whether they are point or directional. But it only affects RSP time, and usually the RSP is not the bottleneck, the RDP is. There's a couple examples of timings with lighting on and off in my cel shading video.
Not directly, PC ports replace the whole graphics system with a PC graphics API, and this is a modification to the N64 graphics system which is the thing they've replaced. But of course, the PC port could implement the same effects, or much more advanced things.
Why anyone would be using crappy inaccurate HLE N64 emulators in 2024 when quality proper accurate emulators exist... Also, if this is based on original Nintendo microcode, how are you going to avoid Nintendo simply sending a take down notice?
There is no "right stick" in OoT, so if you want to orbit the camera around to show something from different perspectives, moving Link and pressing Z is the only option.
Seeing these graphical techniques being performed on real hardware feels like a promise fulfilled.
This seems to be pulling off tricks commonly seen on the PlayStation 2/Xbox (such as specular and bump mapping), which is crazy impressive.
bump mapping on PS2 was an extreme rarity even, to see any type of form of it running on an N64 is wild to me
Nothing better to show lighting effects than the zelda 64 day-night cycle. It always felt so warm.
OoT's environment system is very advanced for its time!
man I love your technical breakdowns. the specular highlight on the water from the sunset looks so nice.
This + Indigo LOL
Imagine seeing this back in 1998 looking over lake hylia 🤩🥳
It has been almost 10yrs that i have first wondered what the advancements of software and math optimizations could bring to a hardware as old as the N64, my prayers are finally being answered
”Fresnel and light-to-alpha for bump mapping” Holy sh*t, someone actually pulled off Bump Mapping on N64! I don’t even care if it’s done per vertex or per pixel, it’s still wild to see it in action.
wow, crazy how you’re able to make n64 graphics look better than some of the newer pokemon games, keep up the good work man :)
😆
That's doing a disservice to N64 graphics 😂
Lawl
Jesus Christ, N64 was GREATLY underutilized. Awesome stuff!
This is amazing! To think that such old hardware would even be cabable of this. SGI and Nintendo made something that was ahead of its time. The other video showing ambient occlusion blew my mind aswell. Programming this to run in a performant manner must have been very difficult.
I won't say any of the microcode stuff is easy, but given the difficulty of the platform, implementing ambient occlusion was relatively easy. It just does light *= 1 - factor * (1 - alpha), and uses a different factor for ambient / directional / point lights.
@@Sauraenwhich sounds to me like something that the RCP's vector unit could chew through like pac Man pellets.
Kaze Emannuar, I think his name is? Perhaps he could give it a try?
@@contramachina354 Yes, Kaze will be using F3DEX3 for his upcoming project Return to Yoshi's Island. However, he'll only be using it for its performance improvements, and won't be using any of the new graphical features, as the new features are not compatible with the legacy HLE emulators most of his audience uses.
This really gives a 6th gen feeling to the N64, massive props to you for figuring this out! Now I'd love to see how Kaze Emanuar could possibly implement something similar into his custom SM64 engine
Kaze is planning to use F3DEX3 in RTYI for its performance improvements. However, he won't be using any of its new graphical features, as they are not compatible with legacy HLE emulators which most of his playerbase uses.
@@SauraenEmulators will have to catch up, maybe that's easy if emulators devs get to see your microcode?
@@xdanic3 It's a chicken-and-egg problem. Part of the problem is that top quality LLE emulators are still too slow on low-end PCs or Android. In five years, that probably won't be a problem anymore.
@@Sauraen What emulators would this work on?
@@debugempty Most emulators with LLE graphics. Ares or paraLLEl are recommended, but some others work too. Personally I rarely use emulators, this is developed on real hardware. I recently switched from an Everdrive 64 V3 to a SummerCart64, which has very fast USB upload.
when this is finished i hope it gets used to its full potential! with all the homebrew that going on for the n64 i think a new era of n64 games are on the horizon. this demo is amazing imo even if the process it "fairly simple" the final result is incredible for the n64 hardware!
The specular vertex lighting is very neat. Looks visually similar to the original unreal engine's vertex specular effect (though the light blending itself is different by far), but a really big advantage of the UE1 effect is that you can control a lot of the parameters between the width and strength, and an additional multiplier for it on top of the regular lighting. Impressive stuff for 1998, and it's very impressive that something close is being done on the N64 after all this time. Awesome job, man
I didn't talk about it in the video, but there is a "size" parameter here for each light when in specular mode. The strength is determined by the light color. I originally tried to implement diffuse + specular with a multiplier for each, but I ran out of instruction memory.
@@Sauraen Despite the limitations, I think it looks really nice. That's how it goes with either hardware or software limitations of this period though, you'll have to give ground to them eventually. Very nice job though, what you've managed to do with this hardware is wonderful
@@OmegaRC59 Thanks!
Amazing! I just wish this was available back then. It would blow people's minds if games could have those effects back in the day.
The future of the N64 is bright, both figuratively and literally thanks to these new graphics techniques lol.
you have such a relaxing voice tbh, enjoyed the technical breakdown and the atmosphere of the island map too
Thanks! :)
This looks so impressive!
Thanks!
thats insane i never thought id ever see this happen
The water looks great!
i always wondered if the n64 could do specular light and now i know
some people go to school, to become layers, doctors, researches. They put their energy to curing disease, fighting for people's rights in the legal system, or making innovations to our world. And then there's people like this (working hard to improve lighting in a 25 year old video game)
This is pretty cool, with some vertex colors on the ground to "absorb" red from the colors and some planning ahead the water could look very impressive, much like how mario sunshine's water on gamecube works
its crazy how good the water looks in this you did a great job on this
The N64 RCP had Finewine technology before ATI/AMD did!
Silicon Graphics: AND THIS
IS TO GO FURTHER BEYOND
HAAAAAAAAAAAAHHHH
Looking spectacular, PS2 level indeed
Per-vertex lighting is actually on par with what the PS2 does in hardware.
watching and learning from this with a mind that is thinking about meta quest and vr development.
ok this is insane, this is a big step forward !
Man hopefully libdragon will get that fresnel and bumpmap implementation eventually (if it doesnt already have it)
It won't get this implementation, this is based on Nintendo's libultra which is incompatible with libdragon. The homebrew community will have to implement fresnel in their own custom microcodes.
@@Sauraen hm well specifically I meant "an implementation with equivalent or better behavior", obviously I know libdragon and libultea have different features and apis
Love your content!
this was an enjoyable video
Thanks!
Nice work. The constant switching of camera modes gives me a small headache lol
Sorry! That's how looking at things in OoT goes!
The two graphical features I'm most looking forward to seeing some makeshift/"good-enough" version of on the N64 are quality full-screen Motion Blur (effectively like either Camera+Object blur or the Per Pixel kinds) and dynamic Depth of Field. Not sure if that's on your radar at all or if you have any ideas how one would approach that, but this gives me hope that someone might implement that someday.
Majora's Mask has motion blur by mixing the last framebuffer with some alpha into the current one. Anything more complex than that is probably out of scope for the hardware, and this doesn't have much to do with custom microcode anyway. What custom microcode can do is vertex-based effects which end up producing a new shade color or alpha or texture coordinates for each vertex.
Are the following possible on the N64 through microcode?
- n-Simplex interpolation/filtering
- Summed Area Tables
- Progressive Mesh Refinement (different from LOD, but has a similar goal)
- MFAA (or something similar)
- Quads
- Textures using other colorspaces, like Y-CoCg
- Raymarching
- Blinn-Phong Shading
- Directional Colored Shadows (to compliment Colored Lights)
Progressive Mesh Refinement, IIRC, uses less memory than LOD but requires more computation time
The microcode is sort of the equivalent of a compute shader which runs where you would normally have a vertex shader in the pipeline. It can theoretically compute anything, but there are severe memory limitations. Anything which would be normally part of a fragment shader is handled by the RDP, which has a configurable but fixed-function pipeline--this has nothing to do with the microcode.
- No idea what that is
- No idea what that is
- Theoretically yes, practically LoD-like stuff is mostly done on the CPU
- This is a fragment related thing. The N64 has antialiasing which has a couple settings but it is not that capable.
- Already supported as two tris. Having a real quad with an undistorted texture on it is supported by the hardware but no microcode implements this.
- The RDP supports like a dozen texture formats, I've used several in this video (see my video on fast64 textures). For the kind of textures you're talking about, there's only one supported: YUYV. Games rarely used this and community tools don't currently support it.
- Fragment related, also please, we're talking about a system designed in 1994.
- Any lighting effect which you're willing to approximate as a per-vertex effect can be done, like I did here with specular.
- OoT already uses dynamic shadows for some objects, there's no reason they could not be drawn with colors.
Magnificent!
In this pace, in 5 years the comunity will be able to port doom 3 to the N64 :D.
reggarding the environment mapping technique. While it is no more than a static texture, one could re-generate that static texture every frame by rendering it to an off-screen buffer and using its results as the env. map for the desired objects. That way, one could potentially render a cubemap into the env. texture, or even render a very simplified impostor of the scene with correct time of day lighting...
I never thought anyone would think it would be worth it to implement either ambient occlusion or fresnel on the N64, let alone both. But, here we are. 😅
Do you think the shade-to-alpha could be used in combination with some other techniques to produce an inexpensive(-ish) subsurface scattering effect? 🤔
These effects are really nice.
you missed out on a perfect opportunity to make a vaporwave background
Does Donkey Kong 64 also use the original texture-based crusty specular implementation for it's shine on the character models?, the sheen the kongs have is surely not true specular, but also look suspiciously smooth compared to the og crusty cubemap specular... I wonder if it's anything custom/special Rare did.
Is this available for the public? I've been making some mods for OoT and SM64 decomps with the Fast64 plugin. Is this integrated into that? Or is there some other way to load this microcode? I'm still pretty new to this...
All of this is public, check the links in the description. All the scenes shown in my videos are made with fast64, and the modifications to fast64 to support them are public. However, neither the microcode or the fast64 support is finished, so there are some incomplete or unsupported features and things may break in the future.
I wonder if any of this could be of use to James Lambert's Portal 64?
Not directly. Homebrew must not contain a single line of Nintendo code. So the OS and CPU rendering engine is completely different, and the microcodes are totally incompatible. However, I hope that F3DEX3 inspires homebrew microcode developers to support things like vertex colors and normals at the same time.
@@Sauraen hold up so f3dex3 stuff is subject to copystrikes or smth
@@ssg-eggunner Of course, it is modded Nintendo content, they can take it down if they want to.
holy aliasing in the distant water
The N64 is capable of fixing that (mipmaps / lod frac), but sadly not at the same time as all the other stuff I wanted to do with this water.
also would love to see how far the 3ds could be pushed.
"link is stranded here"
"Then how do I click it?"
"Oh wait'
You can click him as much as you would like!
Imagine if the N64 had 16KB texture cache and memory without horrible latency issues. Would've had crystal clear textures and higher frame rates for maybe an extra $10-15 per console.
The texture cache could have been expanded for some small cost like that, but the memory system could not have been easily redesigned. The main constraint was development time--SGI made RCP V1, it had two major bugs, so they made V2 which took an extra 9 months. The console was then already behind schedule so they had to ship it. There are still a number of minor bugs in V2, plus supposedly one of their engineers found an issue causing memory inefficiency in V2 which they would have fixed if they did a V3. I also personally found a few changes they could make which would have improved performance by maybe up to 30% without significant silicon area cost, maybe I'll get a chance to talk about those someday. :)
imagine a fantasy derivative of n64 with that and faster RAMbus aswell
I wanna make a game for the N64 now (but I wish I had the time). x'3
P.S. You should try to do the shadowmapping in Conker's Bad Fur Day and/or stencil shadow volumes for the shadows of Link next.
I'm not planning any F3DEX3 features for shadows. OoT's dynamic shadows system is about as good as you can do on N64 without major engine changes or a big perf hit. It is used in the final game for a handful of actors, plus there is beta code which can be re-enabled which draws the dynamic shadow for Link.
actually, OoT already had a dynamic projected shadow system similar to Conker's, though I've only noticed it beeing used for the shadows of the moveable mirrors in the Sand Temple, and for shadows of Ganon's cape in the last boss.
@@buzinaocara Correct, it's used on those two actors, and also on the cuttable signs.
@@Sauraen such odd choices 😆
Oh wow, your work is truly amazing. Is there a chance that you eventually release ROM versions of all these demos? Please? 😁🙏🏻
I don't release ROMs for legal reasons, but the microcode source is public and all the assets are public-domain and linked in the description.
@@Sauraen: oh, that's a shame, I have no idea how to reproduce the ROM from the microcode and assets.
Have you considered trying libdragon?
@@MyakoEventually there should be a HackerOoT release with the microcode already built-in. For someone who has a little experience with OoT romhacking, it's pretty straightforward to export the scene and models from Blender with fast64, and copy in the code which is also released. The target audience is romhack developers, not players--this demo isn't a game, it's only what you're seeing on screen here.
In the near future, no--I'm interested in finishing F3DEX3 and the related tooling to improve the state-of-the-art in SM64 and OoT romhacking. If/when I finish it, I might switch to homebrew, though I probably would move on to non-N64 stuff.
Great explanation! Thanks.
this is amazing!!
Incredible !!!
Damn nintendo really had no idea what they had in their hands with the SGI gpu did they?
I wish i could see the alternate world were sega of america menaged to convince japan to use the SGI gpu
Wild!
hai sauraen cool stuff :3
Hai!
ay that's pretty cool
Hey Sauraen, has anyone integrated your F3DEX3 microcode in to an engine base like Hacker SM64 yet?
There's a PR open for it in HackerSM64 which should be merged soon.
@@Sauraen thanks man. This is SUPER exciting!!!! Really appreciate the work you've done.
So F3DEX4 will have raytracing then
I know you're kidding, but someone already did ray tracing on N64. It worked but only could handle a scene with a handful of basic primitives like spheres and cubes.
Not possible for much more than tech demos
I see
awesome job. How taxing is per vertex lighting vs unlits?
Not very taxing, it depends on how many lights and whether they are point or directional. But it only affects RSP time, and usually the RSP is not the bottleneck, the RDP is. There's a couple examples of timings with lighting on and off in my cel shading video.
so much to see and learn in your channel@@Sauraen
Fresnel can be used as rim light… huh…
Okay cool but isn't Specular... pretty common in later N64 games? I mean right off the top, Perfect Dark had tons of shiny surfaces.
I go over the vanilla "hilite" implementation in this video.
Need a time traveler to go back and teach these techniques to nintendo so nintendo couldnjust cone out the gates swinging with this shit
Is there anything the N64 cannot do?
Does this work with the pc ports?
Not directly, PC ports replace the whole graphics system with a PC graphics API, and this is a modification to the N64 graphics system which is the thing they've replaced. But of course, the PC port could implement the same effects, or much more advanced things.
@@Sauraen do you have any interest in working on the graphics pipeline on the PC ports? I'd like to commission someone to work on it
@@davurphy Not at this time, sorry!
Why anyone would be using crappy inaccurate HLE N64 emulators in 2024 when quality proper accurate emulators exist...
Also, if this is based on original Nintendo microcode, how are you going to avoid Nintendo simply sending a take down notice?
Watching this gave me ADD. Don’t shift focus so fast, this is a presentation not a speed run. Video could also be less than 5 minutes.
There is no "right stick" in OoT, so if you want to orbit the camera around to show something from different perspectives, moving Link and pressing Z is the only option.