I supercharged my Nintendo 64 with AI | Nintendrew
HTML-код
- Опубликовано: 27 сен 2024
- Get started on your 2023 learning goals with Brilliant and get 20% off an annual premium subscription! ▶ brilliant.org/...
BECOME A MEMBER! ▶▶ bit.ly/drewcrew
Patreon: / nintendrew
Facebook: / nintendrew
Twitter: / nintendrew_
Twitch: / nintendrew_
Instagram: / nintendrewyt
Discord: / discord
Recording/Live Stream Gear I use:
- Game Capture HD60 S
- Stream Deck
- Elgato Green Screen
Check them all out right here!: e.lga.to/Ninten...
Thanks to Loeder for the use of his sick tunes: / @loedermusic
Looking for links to some of the decor, tools, or accessories you see in my videos? Check out my Amazon recommendations page! www.amazon.com...
#N64 #AI #artificialintelligence
Hey all! 👋 Just wanted to leave a comment to make a couple corrections/clarifications:
Firstly, as many have pointed out, I mentioned the N64 outputting over composite with "a maximum resolution of 480p", but that's not quite accurate. While the footage in this video was captured at that resolution, N64 games were more typically presented in 240p or 288p, with the highest fidelity titles running at a max of 480i (interlaced).
Secondly, when speculating about the future of video super sampling, I suggested that someone could theoretically use the Nvidia DLSS SDK to upscale video signals in realtime. In reality, DLSS uses additional data provided internally on a game-by-game basis to achieve its end result and could not be used as-is to upscale generic footage. However, Nvidia has just recently announced their RTX Video Super Resolution tech which is designed to do exactly that! Incidentally, 2kliksphillip has a new video on this tech, too, which I would highly recommend watching if you're interested in the topic.
As always, thanks for watching! 😄
Don't you mean four syllable word not four letter word? Because emulation has more than four letters but it does have four syllables. This made me laugh!
@@KevinBlue18 "Four-letter word" was a bit of a joke, but it's used in American English to refer to any word that is crass/rude/offensive. 😛
i want to see my dream of cgi to fmv via ai , i was thinking we would have this like 15 years ago , like the ai does see wood and does fill it in real wood with images of wood and so on. so old games would look like real life. also upscale Sierra games , upscale does cause brakes in the lines so colors bleed , ai could fix the lines.
i want a software in my PC where i can play old games with bit graphic and just let them get updated by my own dlss !!!!!
Antworten
It's worth noting that the N64 isn't actually 480p. It's usually 240p and has a maximum resolution of 480i.
Perfect dark, and a few other games have a 480i mode but need the 8MB expansion pak.
The 4MB Jumper Pak sucks dont Use it, as soon as you get the 8MB Pak, destroy that stupid and useless 4MB Jumper Pak, or sell it.
@@LennyQUMFIF Yep very true. Pokémon Stadium 2 is another one like that. Though there are some merits to staying at 240p if you don't like interlaced video.
@RaniaIsAwesome Yes that's true as well.
Would this AI technology work well with GBA ports. I would love to see Astrix and Oblesk on the GBA
@@LennyQUMFIF Some games break when there's an 8 MB pak installed
7:47 To be honest I never grew up with the n64 but when I got one in late 2016, I found something uniquely appealing about the jagged edges on the environments and especially the characters of most games and the low resolution, blurry picture quality. I still can’t explain why I like that
I know why i like it it gives that console that nostalgia feel many people look for in emulators but let's be honest it doesn't give that feel doesn't it
In emulators anyway
@@fangzgaming5510 With emulators, you can turn all of the processing and upscaling off, but you have to remember, if you're connecting your antique console to a modern TV, you're still not going to get that feel.
Antique consoles produce a signal meant for antique televisions, even using a BVM is not quite true to the original. The jagged edges and blurry picture quality doesn't show up on a 14" Matsui from 1989.
I feel this way about PS1 games I don't care for the super clean look like I do on 64 games. For people saying emulators can't give that old school look and feel though should try PSX FIN that emulator looks as blurry as it would on the actual system.
They were designed to be enjoyed that way. They never even dreamed of all of this upscaling etc. Nobody had even heard of HD back then.
I know people love this but it just looks like melted crayons to me. then again, I need glasses so I could use more pixels in my games.
Nah you’re right. It looks pretty bad. Not surprised though since it’s AI lmao.
Nintendrew is allergic to crts lol
Crayons is on point, on top of wobbly edges like the PlayStation.
AI interpolation needs higher base resolution for good results.
I was surprised to see the red device for the switch. I backed that and haven’t heard anyone talk about it yet
the reason DLSS works, is that it captures as much information from the game as possible, like motion vectors. Without getting extra information (its impossible to get anything more than just the final image from an old console) something like DLSS is completly impossible, since it needs motion vectors to not become a smudgy mess.
So no, u can't just use a software and a video capture to be able to upscale n64 games, because DLSS needs a lot more information than video footage to be able to upscale
Well, something like DLSS 2.0 and up. DLSS 1.0 was entirely framebuffer-based, and yeah, a lot of its output basically looked like Topaz or DALL-E 1 in the absolute worst ways.
And yeah, trying to work 2.0 into the pipeline wouldn't really be an option without some kind of motion vector-producing software shim, which would probably have to be built on a per-game basis.
Pretty neat, but I'd definitely take sharp, crisp pixels, or CRT simulation, from a scaler like the RetroTink over this. In its current state it really looks like a smeary mess, and can be quite unstable at times in motion. It does certainly have its impressive moments, but overall I would not want to play on this.
Personally I play on a real CRT whenever possible, and firmly believe this to be the best way of viewing these low-res signals, but I understand that won't always be practical for everyone, especially as CRTs get scarcer over the decades. I'm hoping highly-accurate CRT emulation will be an accessible option down the road (or, better yet, my unrealistic dream of someone manufacturing new CRTs with retro gaming in mind).
Always great to sit down at lunch and a new nintendrew episode drops
Absolutely
I just sat down to drop one myself. Post lunch.
Indeed
Lol it’s been like a month tho
Hahaha, my microwave just beeped, - ready to watch this episode.
I swear, every time you use the Game Boy Camera music it puts me in an amazing mood.
It does 😄 nice touch lol
amazing video..however, the footage from the upscaled version looks like someone dumped a bunch of Vaseline and grease on the n64 footage
I would definitely be interested in some kind of real-time upscaling device like that. Being able to do re-stylization would be an amazing option as well; being able to provide an image as a style input which is then automatically applied to the video output. That would be incredible on older and monochromatic consoles like the Gameboy.
I think using neural networks to remake old games in real time is gonna be possible, in the future. But right now cpus (gpu) are way to slow if you dont want a huge input lag. And that is with current models of NN for upscaling. Imagine like a shader that you put on Resident Evil (ps1) and it outputs graphics similar to the GC remake. The computing power would probably be more than ten times a rtx 4080 needed. But I can see it in the future. Until then, 240p lools best in 240p on a crt.
You could always learn C and refactor the entire cartridge like what that one German guy did with Mario 64. Using efficient C techniques you could not only get the textures to be as crystal clear as the hardware will allow but you could also increase the frame rate and keep it high.
In all actuality, the fanmade pc ports of these older titles that have been popping up recently are the best answer. Natively upscales to whatever your pc can handle, and same for the aspect ratio. Sure it's not the most accessible option, but it's simultaneously the most cost efficient and gives you the best results. Not to mention that resolution and aspect ratio aren't the limit when it comes to making the games prettier, plus it allows us to push these versions of the games to other consoles. There's already a proper switch port for the fanmade pc version of OOT for example. Rebuilding the games from the ground up is likely going to be the future for these old titles.
I wish there were fanmade native PC ports of Banjo-Kazooie and Rare's other 3D platformers, because as of now, there aren't any.
@@gamestation2690 Boy do I have news for you.
For me personally, I'll just go for the HDMI adapter option (highest quality possible) and not get into supercharging it. That way I can also use the same HDMI adapter for my AV Famicom and Super Famicom systems too (if it works with all AV Nintendo systems anyway).
Hopefully I can get one with 4:3 and 16:9 stretch options included, for those 16:9 supported games of course.
If you add the mClassic to any adapter you have it has a 4:3 mode switch right on the sucker and it polishes those edges and fixes up the color a little bit. I use a retrotynk adapter or EON Super 64 and slap the mClassic on the back of it for a very nice picture from my N64.
There's value in using AI to "upscale" textures. A friend of mine gave me an AI-upscaled version of the PC ports of Dino Crisis and (original) Resident Evil 2, which both looked very crisp. However, I think there's a bigger challenge for the N64, which has always had certain graphic issues when emulating game footage due its source hardware. Either way, it's a cool idea.
The N64 had universally enabled anti-aliasing and a blur filter applied to every game. The resolution isn’t that big of an issue for the video quality as can be seen by the PS1 and Saturn, but all the blurring really hurts the image. One can remove the blur using a hardware mod and remove anti-aliasing with game patches. And let me say, playing the games without that stuff enabled is a treat.
i personally use a combination of a rgb modded n64, that passes through a hdmi line doubler (with optional smoother) and it makes the games look quite close to the footage you are showing as well :)
Oh yeah same here with my Retrotink 5X Pro & set the Phase to Bypass.
3:55 It gave Fox a weird sideways Sonic mouth.
I think the AI upscale looks quite bad in most cases, unless the thing It's upscalling is filling most of the screen it just kinda takes a blurry image and makes a smudgy image, where some things look ok but the further back it is the worse of a job it does figuring out what to do with it.
It gets even more impressive when you start using custom-designed models. I’ve been working on a custom model set for the Super Mario Bros Super Show (models available in the Animation Upscale Discord). The results on the animated portion aren’t as good as going back to the source 16mm film would be, but it’s still very impressive. Live action is taking a bit more work since it was shot directly to tape.
Where did you get the training data for the Mario show?
@@No_True_Scotsman I got stills from the Shout Factory DVD sets released in the 00's.
Your theory about DLSS will not work unless you are doing emulation. DLSS requires motion vectors for the pixels in each frame that are not exported by the hardware. It will not work without those. You could try doing motion vector estimation, but I have been told by others that does not work. I had a similar idea too.
Would frame interpolation be possible with emulators? A lot of early 3D games were programmed specifically to run at 30 fps (or 20 in Zelda's case) so being able to play all of them at 60 fps would be great.
@@TFSned This is a complicated question to answer. On one hand, yes it is very possible to make a game look 'smoother' by introducing additional render frames into a game and thereby make it have an effective higher frame rate. The problem is more related to how the games were programmed. Since many console games were designed with a specific set of criteria since they were expected to only be ran on a specific piece of hardware, many console games take shortcuts on game logic programming, and lock the game's logic to the expected frame rate of the game. So even if you can force the game to interpolate extra frames in between a typical game's 30-60 FPS runtime, the game will still only logically function at 30-60 frame intervals. This can mean many things, but usually all of them lead to the game feeling more jank or unresponsive in spite of the smooth gameplay, or in some cases, can entirely break the game's logic due to heavily relying on frame by frame logic. This is why if you uncap the frame rate on an emulator, it will speed the game up instead of simply just having an uncapped frame rate with the game running at the speed it was before. There are certain games on emulators that can change the frame rate cap thanks to hacks or cheats (Such as being able to play Mario Sunshine on Dolphin at 60 FPS using a cheat code), but those are typically the exception and usually only work because the game's engine was designed to be able to theoretically run at that higher frame rate, and it was likely locked at a lower frame rate because the actual consumer hardware couldn't handle it, or because it introduced more bugs and issues that weren't easy to fix in a timely manner, thereby capping the FPS was the easier answer.
In short, yes it is theoretically possible, but the result may end up worsening the experience instead of enhancing it, or may just plain and simple break the game you're playing.
1:14 ah yes, my favorite 4 letter word: emulation
Love the video Drew! Great insight into what’s coming in the near future
1000000% more of these kinds of videos and discussions. We have seen the gadgets of yesteryear and today, but how we preserve and enjoy our favorite pastimes in the future? That's what I've enjoyed in this bit.
it's even funnier when you take into account that the emulators themselves already offer the option to increase the internal render resolution
I was theorizing this when I first saw stable diffusion drop a couple years ago. Filter all the games with a prompt to change the art style in real time. Can't wait to be honest
Let me read some of the comments I want to find the dude saying, "Oh but Ai, is not Ok and it doesn't look as "intended." 😂
Fascinating stuff! Thank you for painting the future as a bright one for retro gamers!
Wow that looks incredible! Better than any other attempts I've seen over the years to improve the graphics. Would need one powerful af FPGA to pack something like that in a Retrotink or OSSC
What we need is a proper 2.0 HDMI upscaler box with Scart input that can properly handle scanlines. Many of the upscalers we have available introduce some type of issue.
btw the most games on the n64 ran at 256x224 or 320x240 if you downscale your captured footage to the native resolution that game outputs then upscale it will look cleaner and less jaggy.
DLSS uses motion vectors from the game code, unless you also had some sort of high-speed code transport from the N64 to the "DLSS device" it would not be as effective at up-scaling. That said, AMD FSR and CAS algorithms do a pretty good job without motion vectors or AI so who knows what the future might bring. AI will certainly get faster and better at this. You may even get the option to replace all graphics with newly generated realistic ones similar to who AI art tools can take a child's crayon drawing and turn it into a realistic painting.
While it’s one of more harmless uses for AI, it still wouldn’t be that practical. For people that want to use emulation there’s far better texture packs and upscaling options that get the job done, and for people that want to play on the original hardware, I doubt they would want to play through something that is so heavily changing the “original intent” of the graphics. Plus HDMI and upscalers already exist. Interesting I guess, but like a lot of AI things, it feels like they come up with solutions for problems that already have better solutions.
This was very interesting, and the technology could be used to revitalize vintage gameplay! I'd like to experience this myself.
Those games weren't meant to run on those clear image screens. The scan lines and the blurryness makes a huge difference. The more high definition that are implemented on those games, the more ugly they look. Now we can count every pixel and every polygon just by looking at it. In CRT TVs you see less geometric figures in the polygons and less sharpness of the pixels.
the composite video signal and 480p is not why the N64 looks blurry, the N64 uses 3-point bilinear filtering on it's hardware, you'd literally have to solder new hardware for it to be nearest neighbour.
Ew, that would make it look worse.
@@mirabilis it looks truer and it would give a more accurate and truer source for AI image upscaling. Bilinear just makes it look blurry which is why the image upscaling represented on the video doesn't look the best, but klikflilip's Nvidia upscaling (DLSS) looks better. his 240p upscaled footage to 720p looked much more accurate.
@@xehP Honestly I kind of like how it looks. I typically find N64 games more graphically a pleasing than PS1.
Yes New Video! I hope it's good!
First part of the video the music is from Gameboy Camera, such good memories, good choice 👍👌
There is a great video called "enhancing photorealistic enhancement" which uses Ai to make GTA footage look like real life. With the progression of AI image models there is the possibility of style transfer. For example you could feed the AI images of the pre-rendered models of the star fox characters and have an AI use them as a base to enhance the N64 footage. Of course, this requires for models with more temporal coherence and we would need much more powerful hardware, but as you said the point of the video was to look at least a decade into the future.
Kliksphillip mentioned. Love to see his very niche style of videos get some spotlight outside those circles.
I wonder if it will be possible in the future to hijack the video output directly from the console in this type of manner. Considering there are already console mods for HDMi converters, maybe.
Imagine if emulation could render each frame as a.i upscaling, it would suck but be cool!
Thanks for sharing this!
I actually thought of a idea like this 2 years ago. I said to myself if their was a way for someone to figure out how to get one of these A.I upscalers to work for retro consoles then it would be golden! Imagine being able to have PS2, GAMECUBE, Wii or even PS3 and XBOX360 console games Upscale to a convincing 4K!
I don't think I'll ever touch an analogue to digital converter or upscaler. Either emulate in HD or grab a CRT from grandmas house
They look really good on crt
I'm personally in the camp that I want to keep the original experience alive. But that's only for video games. For movies I want the highest fidelity so I can understand why people would want games in that way too. I'm sure there are some movie buffs out there that say film projection is the only way to experience a film.
It's an interesting time to watch this with the HDMI solutions that are about now.
There is literally a $99 device that does this all, and works great with retro consoles. Sold at BestBuy, Amazon, and honestly does what you are trying to do without all this extra work
I'd rather play my N64 with low graphics and not have it look like an ai oil-painting
Bro having emu quality graphics on OG 64 hardware is amazing that's always been goals for me.
That's awesome thank you
1:16 It's not 4 letter word, it's a 4 syllable word! :) Great video!
Was looking for this
Regardless of OG hardware vs emulation, when AI models fully understand how to remaster a game frame by frame in real time, we could be in store for things like UE5 graphics for 6th, 7th and 8th gen games.
So real time ai upscaling for videos exist on the Nvidia shield and it is pretty good. For retro games, I can't imagine you'll ever get reliably good and stable results without access to a depth buffer, it's a big part of why DLSS works so well and why FSR 1.0 was so bad by comparison
4:38 Bro Saria crawled out the obby
On you go Nintendrew, I have no doubt what you have said is correct for the future - but have to press doubt on you doing it yourself.
A cheap solution for up-scaling video console footage sounds great.
I already used FSR on PS1 emulators which already improved the image quality greatly
This is exactly the sort of look I’d like to get on real hardware. I know some folks like that blurry, pixelated and scan line look, but it’s not for me. I would much rather have the cleaned up and crisp look of this video. But how close to that can you get with current external solutions like the retrotink 5x (which I have) and some other smoother like a MClassic (which I also have)? I’ve tried the 5x alone, and it’s certainly better than plugging it in directly…but it still looks pretty blurry and pixelated. I wonder about the other smoother Drew mentioned in this video. Either way, I hope it will be possible sooner than later to get this kind of look on real hardware with minimal to no input lag.
I did buy a plug-in device that upscales my games. These kinds of things aren't new, and advancements are constantly being made to improve them. Just get a RetroTink 5X Pro for more accurate upscaling than the AI did that video
No 480p on the N64, actually. Just 240p or 480i.
I bought a Jungle Green N64 & 35 games out the blue with my settlement money years ago & I've been waiting for something like this to go back 🤙
DLSS is basically black sorcery, god tech, game changer in every sense of the word.
With that said, that's only for modern games designed for higher resolutions in my opinion. For retro games, nothing will beat a good CRT (or complex CRT shader in an OLED panel).
That's just me though.
Thing is what you describe as the "original experience" isn't the original experience either. You're playing games specifically made to be displayed on CRT TVs on modern LCD and LED displays.
No! DLSS works completely different! It has a lot more to work with than only an output picture signal. It has 3D information etc.
Clickbait title. I was expecting to see some clever hack or new device. But no, you didn't supercharge your N64. All you did is upscale some footage with topaz ai which has been around for years now.
i got fooled. i pulled out my wallet ready to order something 😂
Easiest nowadays is still use a CRT and if you want to push it get an s-video adapter. Also with a flash cart there are rom hacks that reduce the blurriness.
In principal I don't think any screen-video upscale could ever catch up with enhancements that has access to the whole pipeline, and could use upscaled textures, anti-aliasing, less-lossy interpolation and much more. and in the former case too, an algorithm that was trained on live action and high-poly CGI is not ideal.
After awhile we could have something like dreams where the A.I. could draw whole new assets and even animations to use on top of existing ones.
DLSS is a temporal upscaler. You could not just take it and throw it in an external device or use it at all. You need actual data from the game itself. This is why DLSS is only available in games that implement it. AMD's FSR 1 should work though, but it could be hard to implement in hardware. Emulators already make use of it.
why go through all this trouble, when its cheaper, and easier, to get a CRT, which will always look better...
I like how the N64 need something like this for its low res textures but we see the ps1 being imitated with modern games even today.
What modern games are imitating PS1? Is there like an indie sub-genre of games made to look like PS1 games?
I have an M-Classic and I regulary use it with my Switch. It makes a big difference for me considering my TV is 4K and it doesn't upscale a 720p image very well, but a 1080p image looks just fine.
medium textures with clean color is the best for all retrogames!!!
Yes, I'd buy such an AI upscaling device. I'd use it to upscale Game Boy and Game Boy Advance game video. 🙂
As someone who plays even the PS4 with a downscaler, fuzzy images don't bother me all that much. The highest the TV I'm using only goes to 1080i, and not only my Wii U and PS3 are both plugged into the only available component arrays, my PS4 doesn't seem to like running in 1080i when it boots up. The TV itself is capable of S-video, but I don't know if my PS4 will play nice with that, either.
I’d love to see an adapter that decompresses the sound.
To be fair, I don't think it would be possible to use DLSS to do this kind of upscaling, at least not trivially, because DLSS needs more internal data from the game engine, like movement vectors and stuff like that, not just the raw output image. AMD's FSR on the other hand could, I believe. Plus, I think it's open source.
Now combine this with an HD texture pack
Hello Nintendrew!
Yeah, I suffer from that blurriness... On a CRT (or those early 4:3 LCDs, like the one I have) with S-Video, the N64 looks good, acceptable. But in modern TVs, I can't stand... What's your actual setup nowadays? Do you plug with those upscalers? Is it HDMI modded?... I think in a near future I may use a modern TV, but I don't want that future right now...
meanwhile me who has played on a pc for years......
looks good bro, totally hd
Nvidia do something similar already with the 2019 shield. It receives a stream from your gaming pc and upscales it to 4k.
Im not sure how good it is however.
Take the best CRT from yesterday and take this image (via camera) as comparison and train the ai with this. So de CRT Look how it supposed to look will be burned into the Model and then can be upscaled to 4K
I've played around with the proteus fine tune on Topaz on some of my crappy phone footage. And the artemis
i would just love to use this tech on my DVD collection! 480p to 4k would be incredible.
Man I actually play and stream a ton of PS2 games online and have been thinking about how we might have a way to greatly improve the image quality while playing on original hardware for a while. I have messed with topaz for a couple of my youtube videos but not worth the time at the moment. But I do have an RTX card and run all my video through my PC so it would be awesome if we could somehow do something to improve the quality. Currently I use a MClassic which not only works well with PS2 but PS3, xbox, 360 and my switch. It helps alot with the jaggies which is a big deal to me! Hope we see ai upscaling as a simple hdmi adapter in the future.
Man I have feeling that because of those blurry ass pictures and textures gaming through ps1/n64 days from 95-2005 is the reason why I'm wearing glasses 😂 I'd rather use emulator with sharper cleaner images to go easy on my aging eyes
Another killer video bro! This is the future of retro gaming right here. Wow!!
I think this tech would be very good and fun!
The RetroTINK 4K is the best at upgrading graphics to modern TV.
I got 12 CRT's, but when they eventually die will be looking for some kind of solution like this
Yes!
Ai in the future of retrogames. Nooooppeeeeee please
I wish I could do this on my own N64
Real-time isn't likley going to be necessary though. With AI it should be possible to take a rom and remaster it, even with personal request. Maybe a wait for the dozens of hyper focused "minds" to load but...
I love the N64. I run it on a CRT. It bugs out my eyes. I don't remember that happening 25 years ago.
When are you planning to do the Photofast review? You should try it with the EON 64, mClassic and Photofast all connected to see the output.
You didn’t play in 4K so you didn’t supercharge the n64. Also I use topaz and just rendered 12 seconds of footage and that took 35 minutes so best of luck with rendering an hour.
You can get much more consistent and better results RIGHT NOW with original hardware by using high quality video out cables and something like a retrotink 5x or the upcoming 4k edition, and then layering CRT filters over the video. Games from the n64 commonly output 240p and were designed to take advantage of a CRT's display quirks, and while the filters don't capture all of them perfectly, itll look a lot better than the ugly smeared look of consumer AI upscaling tech.
In my opinion, AI upscaling is great for use with texture packs (especially when used as a "first pass", and then a human tweaks the final result to remove errors or inconsistencies), but looks absolutely awful for upscaling the raw video output. Selling this as a "supercharged" potential future tech is clickbait when there's better options right now.
You say. Neural networks might play a roll in a few years time. But it definitely will and does already. I have a feeling the next 2 years are gonna change the world as a whole. Leg alone retro gaming. Definitely more excited about the later though lol
The topaz algorithm could use an accelerator. Since it's their algorithm they could sell it. To do that they either make custom silicon or program an FPGA (which they probably need to do first as a proof of concept/prototype).
I approve of the thumbnail!
Game Boy Camera music