Hey all! 👋 Just wanted to leave a comment to make a couple corrections/clarifications: Firstly, as many have pointed out, I mentioned the N64 outputting over composite with "a maximum resolution of 480p", but that's not quite accurate. While the footage in this video was captured at that resolution, N64 games were more typically presented in 240p or 288p, with the highest fidelity titles running at a max of 480i (interlaced). Secondly, when speculating about the future of video super sampling, I suggested that someone could theoretically use the Nvidia DLSS SDK to upscale video signals in realtime. In reality, DLSS uses additional data provided internally on a game-by-game basis to achieve its end result and could not be used as-is to upscale generic footage. However, Nvidia has just recently announced their RTX Video Super Resolution tech which is designed to do exactly that! Incidentally, 2kliksphillip has a new video on this tech, too, which I would highly recommend watching if you're interested in the topic. As always, thanks for watching! 😄
Don't you mean four syllable word not four letter word? Because emulation has more than four letters but it does have four syllables. This made me laugh!
i want to see my dream of cgi to fmv via ai , i was thinking we would have this like 15 years ago , like the ai does see wood and does fill it in real wood with images of wood and so on. so old games would look like real life. also upscale Sierra games , upscale does cause brakes in the lines so colors bleed , ai could fix the lines.
Perfect dark, and a few other games have a 480i mode but need the 8MB expansion pak. The 4MB Jumper Pak sucks dont Use it, as soon as you get the 8MB Pak, destroy that stupid and useless 4MB Jumper Pak, or sell it.
@@LennyQUMFIF Yep very true. Pokémon Stadium 2 is another one like that. Though there are some merits to staying at 240p if you don't like interlaced video.
the reason DLSS works, is that it captures as much information from the game as possible, like motion vectors. Without getting extra information (its impossible to get anything more than just the final image from an old console) something like DLSS is completly impossible, since it needs motion vectors to not become a smudgy mess. So no, u can't just use a software and a video capture to be able to upscale n64 games, because DLSS needs a lot more information than video footage to be able to upscale
Well, something like DLSS 2.0 and up. DLSS 1.0 was entirely framebuffer-based, and yeah, a lot of its output basically looked like Topaz or DALL-E 1 in the absolute worst ways. And yeah, trying to work 2.0 into the pipeline wouldn't really be an option without some kind of motion vector-producing software shim, which would probably have to be built on a per-game basis.
In all actuality, the fanmade pc ports of these older titles that have been popping up recently are the best answer. Natively upscales to whatever your pc can handle, and same for the aspect ratio. Sure it's not the most accessible option, but it's simultaneously the most cost efficient and gives you the best results. Not to mention that resolution and aspect ratio aren't the limit when it comes to making the games prettier, plus it allows us to push these versions of the games to other consoles. There's already a proper switch port for the fanmade pc version of OOT for example. Rebuilding the games from the ground up is likely going to be the future for these old titles.
Pretty neat, but I'd definitely take sharp, crisp pixels, or CRT simulation, from a scaler like the RetroTink over this. In its current state it really looks like a smeary mess, and can be quite unstable at times in motion. It does certainly have its impressive moments, but overall I would not want to play on this. Personally I play on a real CRT whenever possible, and firmly believe this to be the best way of viewing these low-res signals, but I understand that won't always be practical for everyone, especially as CRTs get scarcer over the decades. I'm hoping highly-accurate CRT emulation will be an accessible option down the road (or, better yet, my unrealistic dream of someone manufacturing new CRTs with retro gaming in mind).
I would definitely be interested in some kind of real-time upscaling device like that. Being able to do re-stylization would be an amazing option as well; being able to provide an image as a style input which is then automatically applied to the video output. That would be incredible on older and monochromatic consoles like the Gameboy.
I think using neural networks to remake old games in real time is gonna be possible, in the future. But right now cpus (gpu) are way to slow if you dont want a huge input lag. And that is with current models of NN for upscaling. Imagine like a shader that you put on Resident Evil (ps1) and it outputs graphics similar to the GC remake. The computing power would probably be more than ten times a rtx 4080 needed. But I can see it in the future. Until then, 240p lools best in 240p on a crt.
You could always learn C and refactor the entire cartridge like what that one German guy did with Mario 64. Using efficient C techniques you could not only get the textures to be as crystal clear as the hardware will allow but you could also increase the frame rate and keep it high.
For me personally, I'll just go for the HDMI adapter option (highest quality possible) and not get into supercharging it. That way I can also use the same HDMI adapter for my AV Famicom and Super Famicom systems too (if it works with all AV Nintendo systems anyway). Hopefully I can get one with 4:3 and 16:9 stretch options included, for those 16:9 supported games of course.
If you add the mClassic to any adapter you have it has a 4:3 mode switch right on the sucker and it polishes those edges and fixes up the color a little bit. I use a retrotynk adapter or EON Super 64 and slap the mClassic on the back of it for a very nice picture from my N64.
This is not how DLSS works. DLSS uses a lot of information beyond just the rendered video frame to achieve good upscaling results. Most notably, DLSS depends on the motion vectors provided by the game engine. It’s not a simple video upscaler.
I would prefer a line doubler (like the retrotink or EON Super 64) paired with the mClassic for a lag free polish on original hardware before I would wish to use an AI picture modification which would just be slightly too far from the original in my opinion. The nostalgia of knowing I am looking at the closest thing to true N64 graphics at the moment (when I'm using a modern TV vs my CRT of course) is kind of essential to my interest in playing the N64.
Maybe someone has already pointed this out, but the N64 also has S-Video which is much better quality as composite, which would help with the AI thingy. I believe you can also get an RGB mod for an N64 as well, so again that would improve the picture even more.
3:55 It gave Fox a weird sideways Sonic mouth. I think the AI upscale looks quite bad in most cases, unless the thing It's upscalling is filling most of the screen it just kinda takes a blurry image and makes a smudgy image, where some things look ok but the further back it is the worse of a job it does figuring out what to do with it.
What they really need to do is create a modern display for retro/composite/S-video that matches pitch and resolution of CRT that is displaying the 240P/480 signal. LCD displays have 1 fixed resolution so if you input a low resolution signal then it gets upscaled and turns into a blurry mess. If the panel was created so the native resolution matched the CRT resolution then perhaps it could look original again but probably not. It is for these reasons that I keep a Sony PVM Trinitron around.
i personally use a combination of a rgb modded n64, that passes through a hdmi line doubler (with optional smoother) and it makes the games look quite close to the footage you are showing as well :)
There's value in using AI to "upscale" textures. A friend of mine gave me an AI-upscaled version of the PC ports of Dino Crisis and (original) Resident Evil 2, which both looked very crisp. However, I think there's a bigger challenge for the N64, which has always had certain graphic issues when emulating game footage due its source hardware. Either way, it's a cool idea.
The N64 had universally enabled anti-aliasing and a blur filter applied to every game. The resolution isn’t that big of an issue for the video quality as can be seen by the PS1 and Saturn, but all the blurring really hurts the image. One can remove the blur using a hardware mod and remove anti-aliasing with game patches. And let me say, playing the games without that stuff enabled is a treat.
btw the most games on the n64 ran at 256x224 or 320x240 if you downscale your captured footage to the native resolution that game outputs then upscale it will look cleaner and less jaggy.
Wow that looks incredible! Better than any other attempts I've seen over the years to improve the graphics. Would need one powerful af FPGA to pack something like that in a Retrotink or OSSC
It gets even more impressive when you start using custom-designed models. I’ve been working on a custom model set for the Super Mario Bros Super Show (models available in the Animation Upscale Discord). The results on the animated portion aren’t as good as going back to the source 16mm film would be, but it’s still very impressive. Live action is taking a bit more work since it was shot directly to tape.
Yes and no, see they are not intended to look bad, the developers did as much as posible to look good, for exaple the wind waker and wind waker for wii u both are what the creator were thinking, the wii u just takes it to the next level but still represents the oficial way and also if back in the day the developers could they just straigth up would had use the renders for promotion
Your theory about DLSS will not work unless you are doing emulation. DLSS requires motion vectors for the pixels in each frame that are not exported by the hardware. It will not work without those. You could try doing motion vector estimation, but I have been told by others that does not work. I had a similar idea too.
Would frame interpolation be possible with emulators? A lot of early 3D games were programmed specifically to run at 30 fps (or 20 in Zelda's case) so being able to play all of them at 60 fps would be great.
@@TFSned This is a complicated question to answer. On one hand, yes it is very possible to make a game look 'smoother' by introducing additional render frames into a game and thereby make it have an effective higher frame rate. The problem is more related to how the games were programmed. Since many console games were designed with a specific set of criteria since they were expected to only be ran on a specific piece of hardware, many console games take shortcuts on game logic programming, and lock the game's logic to the expected frame rate of the game. So even if you can force the game to interpolate extra frames in between a typical game's 30-60 FPS runtime, the game will still only logically function at 30-60 frame intervals. This can mean many things, but usually all of them lead to the game feeling more jank or unresponsive in spite of the smooth gameplay, or in some cases, can entirely break the game's logic due to heavily relying on frame by frame logic. This is why if you uncap the frame rate on an emulator, it will speed the game up instead of simply just having an uncapped frame rate with the game running at the speed it was before. There are certain games on emulators that can change the frame rate cap thanks to hacks or cheats (Such as being able to play Mario Sunshine on Dolphin at 60 FPS using a cheat code), but those are typically the exception and usually only work because the game's engine was designed to be able to theoretically run at that higher frame rate, and it was likely locked at a lower frame rate because the actual consumer hardware couldn't handle it, or because it introduced more bugs and issues that weren't easy to fix in a timely manner, thereby capping the FPS was the easier answer. In short, yes it is theoretically possible, but the result may end up worsening the experience instead of enhancing it, or may just plain and simple break the game you're playing.
Please get an OSSC or Retrotink for your original hardware presentation, the m cable is only "decent" after the video signal has already been processed by a good scaler like the aforementioned. Also N64 was never 480p, it was 240p with a select few games supporting 480i with the 8mb ram expansion, which is even more reason for you to use a real scaler
the easiest way of doing it would to build a signal processor that intercepts the signal from the console into a small pc with a gpu and use AMD's FSR to uprender the signal before putting it out to a tv or monitor. the only issue with trying to do that now is that FSR would have to be trained on the full library of N64 games for it to work, or any game for that matter since it does not just upscale any video feed it is given. what we need is a AI that is powerful enough to upscale any footage it is given and do it in real time.
What we need is a proper 2.0 HDMI upscaler box with Scart input that can properly handle scanlines. Many of the upscalers we have available introduce some type of issue.
the composite video signal and 480p is not why the N64 looks blurry, the N64 uses 3-point bilinear filtering on it's hardware, you'd literally have to solder new hardware for it to be nearest neighbour.
@@mirabilis it looks truer and it would give a more accurate and truer source for AI image upscaling. Bilinear just makes it look blurry which is why the image upscaling represented on the video doesn't look the best, but klikflilip's Nvidia upscaling (DLSS) looks better. his 240p upscaled footage to 720p looked much more accurate.
I upscale all the longplays on my channel! Games like parasite Eve I increase the resolution and fps and use all kinds of other effects! Got an old game Drakerider from 2012 uploading now that’s 12 hours long all upscaled. Takes forever but super big difference!
Glad your touching on the (very large) retro subject! Nowadays it's all framemeister, OSSC and retrotink2x/5x and I would love to see more videos on thoses by you!
While it’s one of more harmless uses for AI, it still wouldn’t be that practical. For people that want to use emulation there’s far better texture packs and upscaling options that get the job done, and for people that want to play on the original hardware, I doubt they would want to play through something that is so heavily changing the “original intent” of the graphics. Plus HDMI and upscalers already exist. Interesting I guess, but like a lot of AI things, it feels like they come up with solutions for problems that already have better solutions.
I found you touched on this before with the N64 combining the Mclassic with the E.O.N Super 64. I use this for my current set up and though it's not perfect it's damn good compared to the original and essentially the MClassic's anti aliasing AI combined with the E.O.N's Smoothing.
1000000% more of these kinds of videos and discussions. We have seen the gadgets of yesteryear and today, but how we preserve and enjoy our favorite pastimes in the future? That's what I've enjoyed in this bit.
basically the ai video conversion brings the console video capture to emulation quality, maybe one day in real-time from console capture :D neat vid and idea!
I have an M-Classic and I regulary use it with my Switch. It makes a big difference for me considering my TV is 4K and it doesn't upscale a 720p image very well, but a 1080p image looks just fine.
This is exactly the sort of look I’d like to get on real hardware. I know some folks like that blurry, pixelated and scan line look, but it’s not for me. I would much rather have the cleaned up and crisp look of this video. But how close to that can you get with current external solutions like the retrotink 5x (which I have) and some other smoother like a MClassic (which I also have)? I’ve tried the 5x alone, and it’s certainly better than plugging it in directly…but it still looks pretty blurry and pixelated. I wonder about the other smoother Drew mentioned in this video. Either way, I hope it will be possible sooner than later to get this kind of look on real hardware with minimal to no input lag.
I prefer those old games running in their original blurry resolution as they were designed around it when you scale it up or render it at a higher resolution you reveal all the imperfections that the low resolution originally hid so while the game looks sharper the overall image quality ends up looking worse
I tend to agree. It's even worse when it's a game that has pre-rendered backgrounds because the the rendered models get up-scaled, but the background images stay blurry. At the original resolution the pre-rendered backgrounds blend well with the rendered elements and create a pretty convincing effect, but once they're up-scaled it just looks like a bad green screen. Some games handle the up-scaling better than others though, games with a more cartoon-like aesthetic like Mario 64 don't look as bad when up-scaled.
Another great video on an interesting topic. I too am a retro gamer and I love playing my games on original hardware. Topics like this give me hope for retro gaming's future. Thank you.
for original hardware, honestly the best option is soldering a HDMI output directly into the motherboard. not that it's easy or non-time consuming, but currently that's the best option. tho you can't really do much for textures, which is a byproduct of the severely limited texture pipeline the n64 has anyways. if textures are an absolute essential upgrade tho, emulation is the only way currently. lot's of community-made texture packs and mods offer a variety of options, which is nice :) it sucks that decompilation takes so much time, because running the games natively on modern devices would be the best way to preserve and "supercharge" them, but alas that's out of the question unfortunately.
Kliksphillip mentioned. Love to see his very niche style of videos get some spotlight outside those circles. I wonder if it will be possible in the future to hijack the video output directly from the console in this type of manner. Considering there are already console mods for HDMi converters, maybe.
Easiest nowadays is still use a CRT and if you want to push it get an s-video adapter. Also with a flash cart there are rom hacks that reduce the blurriness.
Emulation is great for when you want more than the console can give, personally i have a powered RCA splitter so i have 1 signal going to a 9 or 11 inch CRT, the others going into my PC fir streaming and a cheap 27 inch LCD. I just need a HDMI budget method for putting it on my big TV. Works great and has 0 input latency so yay!
Those games weren't meant to run on those clear image screens. The scan lines and the blurryness makes a huge difference. The more high definition that are implemented on those games, the more ugly they look. Now we can count every pixel and every polygon just by looking at it. In CRT TVs you see less geometric figures in the polygons and less sharpness of the pixels.
I was theorizing this when I first saw stable diffusion drop a couple years ago. Filter all the games with a prompt to change the art style in real time. Can't wait to be honest
I was emulating some ps1 games, started at 4k but found myself turning it back down to 2x resolution because i only wanted an original look but with a little clean up. Im happy with that
I'm personally in the camp that I want to keep the original experience alive. But that's only for video games. For movies I want the highest fidelity so I can understand why people would want games in that way too. I'm sure there are some movie buffs out there that say film projection is the only way to experience a film.
if there's something I wanna see for a HDMI upscaling box, it's probably one that uses FSR to upscale HDMI inputs to higher resolutions, I wonder if anyone is attempting this, because I'm most certainly not the first person to think of the idea, but it seems like such a smart one.
DLSS is a temporal upscaler. You could not just take it and throw it in an external device or use it at all. You need actual data from the game itself. This is why DLSS is only available in games that implement it. AMD's FSR 1 should work though, but it could be hard to implement in hardware. Emulators already make use of it.
There is a great video called "enhancing photorealistic enhancement" which uses Ai to make GTA footage look like real life. With the progression of AI image models there is the possibility of style transfer. For example you could feed the AI images of the pre-rendered models of the star fox characters and have an AI use them as a base to enhance the N64 footage. Of course, this requires for models with more temporal coherence and we would need much more powerful hardware, but as you said the point of the video was to look at least a decade into the future.
What I would be interested to see is one of these adapters/AIs that is specifically trained on something like low res n64 footage and high res emulator footage in an attempt to have a more specialized dataset for upscaling those old polygonal n64 games.
I don't think DLSS frame gen works on the output it works by analysing the input geometry so it would not work for your use case but could work for emulation
I actually thought of a idea like this 2 years ago. I said to myself if their was a way for someone to figure out how to get one of these A.I upscalers to work for retro consoles then it would be golden! Imagine being able to have PS2, GAMECUBE, Wii or even PS3 and XBOX360 console games Upscale to a convincing 4K!
I would expect AI 10 years into the future to be so completely beyond what we currently have that it's unrecognizable. The progress is happening so fast.
It all depends as of right now AI is not self aware and just like Siri can only be programmed meaning it has a human bias behind how it gives/takes info, if they really did manage to get an ai that was even slightly self aware though I'd say things would change in like a few months lol, it would be writing code , upgrading itself , what an amazing time we live in when these are the things we talk about lol
I don't mind the low resolution, it's the blur that I don't like. Which I take care of with a Gameshark and codes to disable the blur/anti-aliasing. Also get an fps boost on games with unlocked framerates.
So real time ai upscaling for videos exist on the Nvidia shield and it is pretty good. For retro games, I can't imagine you'll ever get reliably good and stable results without access to a depth buffer, it's a big part of why DLSS works so well and why FSR 1.0 was so bad by comparison
In principal I don't think any screen-video upscale could ever catch up with enhancements that has access to the whole pipeline, and could use upscaled textures, anti-aliasing, less-lossy interpolation and much more. and in the former case too, an algorithm that was trained on live action and high-poly CGI is not ideal.
Nvidia do something similar already with the 2019 shield. It receives a stream from your gaming pc and upscales it to 4k. Im not sure how good it is however.
What if you connect all these converters and adapters together and see if that improves the quality of the output. Would be interesting to see especially for the wii and switch.
Hello Nintendrew! Yeah, I suffer from that blurriness... On a CRT (or those early 4:3 LCDs, like the one I have) with S-Video, the N64 looks good, acceptable. But in modern TVs, I can't stand... What's your actual setup nowadays? Do you plug with those upscalers? Is it HDMI modded?... I think in a near future I may use a modern TV, but I don't want that future right now...
I've wondered if this is possible for quite a while, and I do think it could be something great if refined enough. I like it because it doesn't fundamentally change the graphics, just give them a clarity they might've theoretically always had if resolution standards of the time allowed it. It's like putting on a new pair of glasses. Or, even if that's not what you want, AI could possibly be used to simply simulate a nice CRT look without a CRT.
I’m still looking forward to the edge fill when a movie is full screen native. And the AI fills in the edges with something better than what we have now.
To be fair, I don't think it would be possible to use DLSS to do this kind of upscaling, at least not trivially, because DLSS needs more internal data from the game engine, like movement vectors and stuff like that, not just the raw output image. AMD's FSR on the other hand could, I believe. Plus, I think it's open source.
Hey all! 👋 Just wanted to leave a comment to make a couple corrections/clarifications:
Firstly, as many have pointed out, I mentioned the N64 outputting over composite with "a maximum resolution of 480p", but that's not quite accurate. While the footage in this video was captured at that resolution, N64 games were more typically presented in 240p or 288p, with the highest fidelity titles running at a max of 480i (interlaced).
Secondly, when speculating about the future of video super sampling, I suggested that someone could theoretically use the Nvidia DLSS SDK to upscale video signals in realtime. In reality, DLSS uses additional data provided internally on a game-by-game basis to achieve its end result and could not be used as-is to upscale generic footage. However, Nvidia has just recently announced their RTX Video Super Resolution tech which is designed to do exactly that! Incidentally, 2kliksphillip has a new video on this tech, too, which I would highly recommend watching if you're interested in the topic.
As always, thanks for watching! 😄
Don't you mean four syllable word not four letter word? Because emulation has more than four letters but it does have four syllables. This made me laugh!
@@KevinBlue18 "Four-letter word" was a bit of a joke, but it's used in American English to refer to any word that is crass/rude/offensive. 😛
i want to see my dream of cgi to fmv via ai , i was thinking we would have this like 15 years ago , like the ai does see wood and does fill it in real wood with images of wood and so on. so old games would look like real life. also upscale Sierra games , upscale does cause brakes in the lines so colors bleed , ai could fix the lines.
i want a software in my PC where i can play old games with bit graphic and just let them get updated by my own dlss !!!!!
Antworten
It's worth noting that the N64 isn't actually 480p. It's usually 240p and has a maximum resolution of 480i.
Perfect dark, and a few other games have a 480i mode but need the 8MB expansion pak.
The 4MB Jumper Pak sucks dont Use it, as soon as you get the 8MB Pak, destroy that stupid and useless 4MB Jumper Pak, or sell it.
@@LennyQUMFIF Yep very true. Pokémon Stadium 2 is another one like that. Though there are some merits to staying at 240p if you don't like interlaced video.
@RaniaIsAwesome Yes that's true as well.
Would this AI technology work well with GBA ports. I would love to see Astrix and Oblesk on the GBA
@@LennyQUMFIF Some games break when there's an 8 MB pak installed
I know people love this but it just looks like melted crayons to me. then again, I need glasses so I could use more pixels in my games.
Nah you’re right. It looks pretty bad. Not surprised though since it’s AI lmao.
Nintendrew is allergic to crts lol
Crayons is on point, on top of wobbly edges like the PlayStation.
AI interpolation needs higher base resolution for good results.
the reason DLSS works, is that it captures as much information from the game as possible, like motion vectors. Without getting extra information (its impossible to get anything more than just the final image from an old console) something like DLSS is completly impossible, since it needs motion vectors to not become a smudgy mess.
So no, u can't just use a software and a video capture to be able to upscale n64 games, because DLSS needs a lot more information than video footage to be able to upscale
Well, something like DLSS 2.0 and up. DLSS 1.0 was entirely framebuffer-based, and yeah, a lot of its output basically looked like Topaz or DALL-E 1 in the absolute worst ways.
And yeah, trying to work 2.0 into the pipeline wouldn't really be an option without some kind of motion vector-producing software shim, which would probably have to be built on a per-game basis.
Hell no, fuck AI. I'll take my upscaler until the day I die.
I swear, every time you use the Game Boy Camera music it puts me in an amazing mood.
It does 😄 nice touch lol
amazing video..however, the footage from the upscaled version looks like someone dumped a bunch of Vaseline and grease on the n64 footage
In all actuality, the fanmade pc ports of these older titles that have been popping up recently are the best answer. Natively upscales to whatever your pc can handle, and same for the aspect ratio. Sure it's not the most accessible option, but it's simultaneously the most cost efficient and gives you the best results. Not to mention that resolution and aspect ratio aren't the limit when it comes to making the games prettier, plus it allows us to push these versions of the games to other consoles. There's already a proper switch port for the fanmade pc version of OOT for example. Rebuilding the games from the ground up is likely going to be the future for these old titles.
I wish there were fanmade native PC ports of Banjo-Kazooie and Rare's other 3D platformers, because as of now, there aren't any.
@@gamestation2690 Boy do I have news for you.
1:14 ah yes, my favorite 4 letter word: emulation
Always great to sit down at lunch and a new nintendrew episode drops
Absolutely
I just sat down to drop one myself. Post lunch.
Indeed
Lol it’s been like a month tho
Hahaha, my microwave just beeped, - ready to watch this episode.
I was surprised to see the red device for the switch. I backed that and haven’t heard anyone talk about it yet
Pretty neat, but I'd definitely take sharp, crisp pixels, or CRT simulation, from a scaler like the RetroTink over this. In its current state it really looks like a smeary mess, and can be quite unstable at times in motion. It does certainly have its impressive moments, but overall I would not want to play on this.
Personally I play on a real CRT whenever possible, and firmly believe this to be the best way of viewing these low-res signals, but I understand that won't always be practical for everyone, especially as CRTs get scarcer over the decades. I'm hoping highly-accurate CRT emulation will be an accessible option down the road (or, better yet, my unrealistic dream of someone manufacturing new CRTs with retro gaming in mind).
A reasonable person would think with the thumbnail and title that you actually played it at this resolution, not ran it through topaz. Clickbait.
Totally agree. Just emulate it for resolution. The Nintendo Switch is in hd.
Cool story
@@mlassz009 yeah that's not how you use "cool story"
@@mapsgoonthewall5396 Thanks for the hot tip
3:30 What a squandered opportunity to use a WaveRace 64’s “MAXIMUM POWER!” sound byte! C’mon Drew! Haha 😂
I would definitely be interested in some kind of real-time upscaling device like that. Being able to do re-stylization would be an amazing option as well; being able to provide an image as a style input which is then automatically applied to the video output. That would be incredible on older and monochromatic consoles like the Gameboy.
I think using neural networks to remake old games in real time is gonna be possible, in the future. But right now cpus (gpu) are way to slow if you dont want a huge input lag. And that is with current models of NN for upscaling. Imagine like a shader that you put on Resident Evil (ps1) and it outputs graphics similar to the GC remake. The computing power would probably be more than ten times a rtx 4080 needed. But I can see it in the future. Until then, 240p lools best in 240p on a crt.
You could always learn C and refactor the entire cartridge like what that one German guy did with Mario 64. Using efficient C techniques you could not only get the textures to be as crystal clear as the hardware will allow but you could also increase the frame rate and keep it high.
For me personally, I'll just go for the HDMI adapter option (highest quality possible) and not get into supercharging it. That way I can also use the same HDMI adapter for my AV Famicom and Super Famicom systems too (if it works with all AV Nintendo systems anyway).
Hopefully I can get one with 4:3 and 16:9 stretch options included, for those 16:9 supported games of course.
If you add the mClassic to any adapter you have it has a 4:3 mode switch right on the sucker and it polishes those edges and fixes up the color a little bit. I use a retrotynk adapter or EON Super 64 and slap the mClassic on the back of it for a very nice picture from my N64.
This is not how DLSS works. DLSS uses a lot of information beyond just the rendered video frame to achieve good upscaling results. Most notably, DLSS depends on the motion vectors provided by the game engine. It’s not a simple video upscaler.
Love the video Drew! Great insight into what’s coming in the near future
I would prefer a line doubler (like the retrotink or EON Super 64) paired with the mClassic for a lag free polish on original hardware before I would wish to use an AI picture modification which would just be slightly too far from the original in my opinion. The nostalgia of knowing I am looking at the closest thing to true N64 graphics at the moment (when I'm using a modern TV vs my CRT of course) is kind of essential to my interest in playing the N64.
Maybe someone has already pointed this out, but the N64 also has S-Video which is much better quality as composite, which would help with the AI thingy. I believe you can also get an RGB mod for an N64 as well, so again that would improve the picture even more.
3:55 It gave Fox a weird sideways Sonic mouth.
I think the AI upscale looks quite bad in most cases, unless the thing It's upscalling is filling most of the screen it just kinda takes a blurry image and makes a smudgy image, where some things look ok but the further back it is the worse of a job it does figuring out what to do with it.
What they really need to do is create a modern display for retro/composite/S-video that matches pitch and resolution of CRT that is displaying the 240P/480 signal. LCD displays have 1 fixed resolution so if you input a low resolution signal then it gets upscaled and turns into a blurry mess. If the panel was created so the native resolution matched the CRT resolution then perhaps it could look original again but probably not. It is for these reasons that I keep a Sony PVM Trinitron around.
i personally use a combination of a rgb modded n64, that passes through a hdmi line doubler (with optional smoother) and it makes the games look quite close to the footage you are showing as well :)
Oh yeah same here with my Retrotink 5X Pro & set the Phase to Bypass.
Ah, and here I thought you actually, physically did something to your Nintendo 64 to get that footage.
There's value in using AI to "upscale" textures. A friend of mine gave me an AI-upscaled version of the PC ports of Dino Crisis and (original) Resident Evil 2, which both looked very crisp. However, I think there's a bigger challenge for the N64, which has always had certain graphic issues when emulating game footage due its source hardware. Either way, it's a cool idea.
The N64 had universally enabled anti-aliasing and a blur filter applied to every game. The resolution isn’t that big of an issue for the video quality as can be seen by the PS1 and Saturn, but all the blurring really hurts the image. One can remove the blur using a hardware mod and remove anti-aliasing with game patches. And let me say, playing the games without that stuff enabled is a treat.
btw the most games on the n64 ran at 256x224 or 320x240 if you downscale your captured footage to the native resolution that game outputs then upscale it will look cleaner and less jaggy.
Wow that looks incredible! Better than any other attempts I've seen over the years to improve the graphics. Would need one powerful af FPGA to pack something like that in a Retrotink or OSSC
4 letter word....
It gets even more impressive when you start using custom-designed models. I’ve been working on a custom model set for the Super Mario Bros Super Show (models available in the Animation Upscale Discord). The results on the animated portion aren’t as good as going back to the source 16mm film would be, but it’s still very impressive. Live action is taking a bit more work since it was shot directly to tape.
Where did you get the training data for the Mario show?
@@No_True_Scotsman I got stills from the Shout Factory DVD sets released in the 00's.
Thanks for sharing this!
I want my old games to look like ass because that's how they were intended to look.
fr fr
Yes and no, see they are not intended to look bad, the developers did as much as posible to look good, for exaple the wind waker and wind waker for wii u both are what the creator were thinking, the wii u just takes it to the next level but still represents the oficial way and also if back in the day the developers could they just straigth up would had use the renders for promotion
Your theory about DLSS will not work unless you are doing emulation. DLSS requires motion vectors for the pixels in each frame that are not exported by the hardware. It will not work without those. You could try doing motion vector estimation, but I have been told by others that does not work. I had a similar idea too.
Would frame interpolation be possible with emulators? A lot of early 3D games were programmed specifically to run at 30 fps (or 20 in Zelda's case) so being able to play all of them at 60 fps would be great.
@@TFSned This is a complicated question to answer. On one hand, yes it is very possible to make a game look 'smoother' by introducing additional render frames into a game and thereby make it have an effective higher frame rate. The problem is more related to how the games were programmed. Since many console games were designed with a specific set of criteria since they were expected to only be ran on a specific piece of hardware, many console games take shortcuts on game logic programming, and lock the game's logic to the expected frame rate of the game. So even if you can force the game to interpolate extra frames in between a typical game's 30-60 FPS runtime, the game will still only logically function at 30-60 frame intervals. This can mean many things, but usually all of them lead to the game feeling more jank or unresponsive in spite of the smooth gameplay, or in some cases, can entirely break the game's logic due to heavily relying on frame by frame logic. This is why if you uncap the frame rate on an emulator, it will speed the game up instead of simply just having an uncapped frame rate with the game running at the speed it was before. There are certain games on emulators that can change the frame rate cap thanks to hacks or cheats (Such as being able to play Mario Sunshine on Dolphin at 60 FPS using a cheat code), but those are typically the exception and usually only work because the game's engine was designed to be able to theoretically run at that higher frame rate, and it was likely locked at a lower frame rate because the actual consumer hardware couldn't handle it, or because it introduced more bugs and issues that weren't easy to fix in a timely manner, thereby capping the FPS was the easier answer.
In short, yes it is theoretically possible, but the result may end up worsening the experience instead of enhancing it, or may just plain and simple break the game you're playing.
Please get an OSSC or Retrotink for your original hardware presentation, the m cable is only "decent" after the video signal has already been processed by a good scaler like the aforementioned. Also N64 was never 480p, it was 240p with a select few games supporting 480i with the 8mb ram expansion, which is even more reason for you to use a real scaler
the easiest way of doing it would to build a signal processor that intercepts the signal from the console into a small pc with a gpu and use AMD's FSR to uprender the signal before putting it out to a tv or monitor. the only issue with trying to do that now is that FSR would have to be trained on the full library of N64 games for it to work, or any game for that matter since it does not just upscale any video feed it is given. what we need is a AI that is powerful enough to upscale any footage it is given and do it in real time.
That's awesome thank you
What we need is a proper 2.0 HDMI upscaler box with Scart input that can properly handle scanlines. Many of the upscalers we have available introduce some type of issue.
the composite video signal and 480p is not why the N64 looks blurry, the N64 uses 3-point bilinear filtering on it's hardware, you'd literally have to solder new hardware for it to be nearest neighbour.
Ew, that would make it look worse.
@@mirabilis it looks truer and it would give a more accurate and truer source for AI image upscaling. Bilinear just makes it look blurry which is why the image upscaling represented on the video doesn't look the best, but klikflilip's Nvidia upscaling (DLSS) looks better. his 240p upscaled footage to 720p looked much more accurate.
@@xehP Honestly I kind of like how it looks. I typically find N64 games more graphically a pleasing than PS1.
I upscale all the longplays on my channel! Games like parasite Eve I increase the resolution and fps and use all kinds of other effects! Got an old game Drakerider from 2012 uploading now that’s 12 hours long all upscaled. Takes forever but super big difference!
Glad your touching on the (very large) retro subject! Nowadays it's all framemeister, OSSC and retrotink2x/5x and I would love to see more videos on thoses by you!
While it’s one of more harmless uses for AI, it still wouldn’t be that practical. For people that want to use emulation there’s far better texture packs and upscaling options that get the job done, and for people that want to play on the original hardware, I doubt they would want to play through something that is so heavily changing the “original intent” of the graphics. Plus HDMI and upscalers already exist. Interesting I guess, but like a lot of AI things, it feels like they come up with solutions for problems that already have better solutions.
why go through all this trouble, when its cheaper, and easier, to get a CRT, which will always look better...
I found you touched on this before with the N64 combining the Mclassic with the E.O.N Super 64. I use this for my current set up and though it's not perfect it's damn good compared to the original and essentially the MClassic's anti aliasing AI combined with the E.O.N's Smoothing.
1000000% more of these kinds of videos and discussions. We have seen the gadgets of yesteryear and today, but how we preserve and enjoy our favorite pastimes in the future? That's what I've enjoyed in this bit.
Yes New Video! I hope it's good!
This was very interesting, and the technology could be used to revitalize vintage gameplay! I'd like to experience this myself.
basically the ai video conversion brings the console video capture to emulation quality, maybe one day in real-time from console capture :D neat vid and idea!
Bro having emu quality graphics on OG 64 hardware is amazing that's always been goals for me.
I have an M-Classic and I regulary use it with my Switch. It makes a big difference for me considering my TV is 4K and it doesn't upscale a 720p image very well, but a 1080p image looks just fine.
Another killer video bro! This is the future of retro gaming right here. Wow!!
Fascinating stuff! Thank you for painting the future as a bright one for retro gamers!
This is exactly the sort of look I’d like to get on real hardware. I know some folks like that blurry, pixelated and scan line look, but it’s not for me. I would much rather have the cleaned up and crisp look of this video. But how close to that can you get with current external solutions like the retrotink 5x (which I have) and some other smoother like a MClassic (which I also have)? I’ve tried the 5x alone, and it’s certainly better than plugging it in directly…but it still looks pretty blurry and pixelated. I wonder about the other smoother Drew mentioned in this video. Either way, I hope it will be possible sooner than later to get this kind of look on real hardware with minimal to no input lag.
I prefer those old games running in their original blurry resolution as they were designed around it when you scale it up or render it at a higher resolution you reveal all the imperfections that the low resolution originally hid so while the game looks sharper the overall image quality ends up looking worse
I tend to agree. It's even worse when it's a game that has pre-rendered backgrounds because the the rendered models get up-scaled, but the background images stay blurry. At the original resolution the pre-rendered backgrounds blend well with the rendered elements and create a pretty convincing effect, but once they're up-scaled it just looks like a bad green screen. Some games handle the up-scaling better than others though, games with a more cartoon-like aesthetic like Mario 64 don't look as bad when up-scaled.
Another great video on an interesting topic. I too am a retro gamer and I love playing my games on original hardware. Topics like this give me hope for retro gaming's future. Thank you.
for original hardware, honestly the best option is soldering a HDMI output directly into the motherboard. not that it's easy or non-time consuming, but currently that's the best option. tho you can't really do much for textures, which is a byproduct of the severely limited texture pipeline the n64 has anyways.
if textures are an absolute essential upgrade tho, emulation is the only way currently. lot's of community-made texture packs and mods offer a variety of options, which is nice :)
it sucks that decompilation takes so much time, because running the games natively on modern devices would be the best way to preserve and "supercharge" them, but alas that's out of the question unfortunately.
Kliksphillip mentioned. Love to see his very niche style of videos get some spotlight outside those circles.
I wonder if it will be possible in the future to hijack the video output directly from the console in this type of manner. Considering there are already console mods for HDMi converters, maybe.
Easiest nowadays is still use a CRT and if you want to push it get an s-video adapter. Also with a flash cart there are rom hacks that reduce the blurriness.
i would just love to use this tech on my DVD collection! 480p to 4k would be incredible.
Emulation is great for when you want more than the console can give, personally i have a powered RCA splitter so i have 1 signal going to a 9 or 11 inch CRT, the others going into my PC fir streaming and a cheap 27 inch LCD. I just need a HDMI budget method for putting it on my big TV.
Works great and has 0 input latency so yay!
Those games weren't meant to run on those clear image screens. The scan lines and the blurryness makes a huge difference. The more high definition that are implemented on those games, the more ugly they look. Now we can count every pixel and every polygon just by looking at it. In CRT TVs you see less geometric figures in the polygons and less sharpness of the pixels.
I was theorizing this when I first saw stable diffusion drop a couple years ago. Filter all the games with a prompt to change the art style in real time. Can't wait to be honest
First part of the video the music is from Gameboy Camera, such good memories, good choice 👍👌
Great video man!
I’d love to see an adapter that decompresses the sound.
Ai in the future of retrogames. Nooooppeeeeee please
I was emulating some ps1 games, started at 4k but found myself turning it back down to 2x resolution because i only wanted an original look but with a little clean up. Im happy with that
I'm personally in the camp that I want to keep the original experience alive. But that's only for video games. For movies I want the highest fidelity so I can understand why people would want games in that way too. I'm sure there are some movie buffs out there that say film projection is the only way to experience a film.
It's an interesting time to watch this with the HDMI solutions that are about now.
Retrotink family of products are the answer to your question
A cheap solution for up-scaling video console footage sounds great.
Love the game room and this video rocks! Thanks for sharing!
if there's something I wanna see for a HDMI upscaling box, it's probably one that uses FSR to upscale HDMI inputs to higher resolutions, I wonder if anyone is attempting this, because I'm most certainly not the first person to think of the idea, but it seems like such a smart one.
I already used FSR on PS1 emulators which already improved the image quality greatly
i've thought about this a lot for a long time now. analog AI chips trained to upscale retro games. gib now
DLSS is a temporal upscaler. You could not just take it and throw it in an external device or use it at all. You need actual data from the game itself. This is why DLSS is only available in games that implement it. AMD's FSR 1 should work though, but it could be hard to implement in hardware. Emulators already make use of it.
There is a great video called "enhancing photorealistic enhancement" which uses Ai to make GTA footage look like real life. With the progression of AI image models there is the possibility of style transfer. For example you could feed the AI images of the pre-rendered models of the star fox characters and have an AI use them as a base to enhance the N64 footage. Of course, this requires for models with more temporal coherence and we would need much more powerful hardware, but as you said the point of the video was to look at least a decade into the future.
What I would be interested to see is one of these adapters/AIs that is specifically trained on something like low res n64 footage and high res emulator footage in an attempt to have a more specialized dataset for upscaling those old polygonal n64 games.
....just get a retrotink
I don't think DLSS frame gen works on the output it works by analysing the input geometry so it would not work for your use case but could work for emulation
Imagine if emulation could render each frame as a.i upscaling, it would suck but be cool!
I actually thought of a idea like this 2 years ago. I said to myself if their was a way for someone to figure out how to get one of these A.I upscalers to work for retro consoles then it would be golden! Imagine being able to have PS2, GAMECUBE, Wii or even PS3 and XBOX360 console games Upscale to a convincing 4K!
I would expect AI 10 years into the future to be so completely beyond what we currently have that it's unrecognizable. The progress is happening so fast.
Yep and sadly be used for evil (you're already seeing examples), humans truly cant have nice things without corrupting it somehow.
10 years?
Try 3-4
It all depends as of right now AI is not self aware and just like Siri can only be programmed meaning it has a human bias behind how it gives/takes info, if they really did manage to get an ai that was even slightly self aware though I'd say things would change in like a few months lol, it would be writing code , upgrading itself , what an amazing time we live in when these are the things we talk about lol
No! DLSS works completely different! It has a lot more to work with than only an output picture signal. It has 3D information etc.
it's even funnier when you take into account that the emulators themselves already offer the option to increase the internal render resolution
I bought a Jungle Green N64 & 35 games out the blue with my settlement money years ago & I've been waiting for something like this to go back 🤙
I don't mind the low resolution, it's the blur that I don't like. Which I take care of with a Gameshark and codes to disable the blur/anti-aliasing. Also get an fps boost on games with unlocked framerates.
This looks fucking horrible LMAO
Yes
After awhile we could have something like dreams where the A.I. could draw whole new assets and even animations to use on top of existing ones.
So real time ai upscaling for videos exist on the Nvidia shield and it is pretty good. For retro games, I can't imagine you'll ever get reliably good and stable results without access to a depth buffer, it's a big part of why DLSS works so well and why FSR 1.0 was so bad by comparison
In principal I don't think any screen-video upscale could ever catch up with enhancements that has access to the whole pipeline, and could use upscaled textures, anti-aliasing, less-lossy interpolation and much more. and in the former case too, an algorithm that was trained on live action and high-poly CGI is not ideal.
Nvidia do something similar already with the 2019 shield. It receives a stream from your gaming pc and upscales it to 4k.
Im not sure how good it is however.
What if you connect all these converters and adapters together and see if that improves the quality of the output. Would be interesting to see especially for the wii and switch.
It looks super smooth
Hello Nintendrew!
Yeah, I suffer from that blurriness... On a CRT (or those early 4:3 LCDs, like the one I have) with S-Video, the N64 looks good, acceptable. But in modern TVs, I can't stand... What's your actual setup nowadays? Do you plug with those upscalers? Is it HDMI modded?... I think in a near future I may use a modern TV, but I don't want that future right now...
I'd rather play my N64 with low graphics and not have it look like an ai oil-painting
I've wondered if this is possible for quite a while, and I do think it could be something great if refined enough. I like it because it doesn't fundamentally change the graphics, just give them a clarity they might've theoretically always had if resolution standards of the time allowed it. It's like putting on a new pair of glasses. Or, even if that's not what you want, AI could possibly be used to simply simulate a nice CRT look without a CRT.
I’m still looking forward to the edge fill when a movie is full screen native. And the AI fills in the edges with something better than what we have now.
I've played around with the proteus fine tune on Topaz on some of my crappy phone footage. And the artemis
On you go Nintendrew, I have no doubt what you have said is correct for the future - but have to press doubt on you doing it yourself.
To be fair, I don't think it would be possible to use DLSS to do this kind of upscaling, at least not trivially, because DLSS needs more internal data from the game engine, like movement vectors and stuff like that, not just the raw output image. AMD's FSR on the other hand could, I believe. Plus, I think it's open source.