As for your remarks about zeroing all sound effects and record without sounds: this was the case for gamecube games where only one *.aw contained the instrument sounds. For brsar all sounds were played by rseq. Often they were rseq files with one note playing. Rseq was not reverse engineered yet, but changing all notes data to value 255 stopped the sounds. rseq files contain labels, so i could easily find out which one contained sound effects. Animal Crossing contained more than 10000 rseq files so i had to write a script for modifying the brsar to only play songs back then
@@champagnesupernova1839...On what device have you watched this video on? RUclips usually has a frequency cutoff around that area. In fact, the video had no audio output at that moment for me using the RUclips app. How did you hear that?
I personally can never overstate how much I appreciate someone like you who actually does the needed research on console's audio specs and functionalities. Rather than the laughable latter half of people who bluntly assumes and mislead with "MIDI this and Soundfont that" and thinks that's the "only formats that's ever existed." Thank you so much for making this video and thank you for reading in on the topic unlike everyone else. (Everyone else who just reads the first sentence on Wikipedia.)
@@literallywhons4s if you mean VGMTrans, it *converts* to midi and dls the best it can, but the actual ds sound data is stored in a different format that's only mostly compatible
@@literallywhons4s The data is not stored as MIDI data and SF2 soundfonts internally. What you get from using such tools is simply an _attempt_ to convert the sequence and sample data into more common and more-documented formats. However, due to the differences in native capabilities between the formats, the conversion is often not completely perfect or faithful; examples of this can be inaccuracies in the volume of certain tracks or instruments, or missing DSP effects. If you're looking for for faithful playback of sequenced NDS audio, then you may want to look into the 2SF and NCSF formats instead.
City folk's title screen was streamed at 44.1khz and I think the gamecube had some 48khz music in the service disc and some unused tracks in sonic mega collection
A little bit of extra history/trivia -- you briefly touched on Super Smash Bros Brawl using BRSTMs. Music replacement was really big really early in SSBB's hacking days, which is why the BrawlBRSTMs RUclips channel got so big, and why their 30-minute extension format became so iconic for game music uploads on RUclips -- they made it that long so that anyone seeking to replace music in Brawl wouldn't have to futz around with figuring out the correct loop point for what they're converting to BRSTM.
That's not the reason why the videos were made that long. All of the 30-minute long videos of music posted on BrawlBRSTMs were effectively previews of pre-made BRSTMs hosted on the website "Smash Custom Music" (formerly known as "Brawl Custom Music"), a site originally ran by the same person who ran the BrawlBRSTMs channel on RUclips: Segtendo. Each song hosted on this site was made available to download as a BRSTM (in addition to several other audio formats commonly-used by games) with their loop points and other metadata already written in the file. Each video on the channel featured a link to the song's respective page on the website in their descriptions.
2:50 worth noting that on mobile, sounds above like 15khz are cut off by the RUclips app/phone. So not hearing it here doesn’t necessarily mean you can’t hear that frequency.
I played the 16kHz+ example from this video on the RUclips app out of my phone speakers into my microphone, and it was able to be picked up fully up to 20kHz imgur.com/a/FQn5Aw9 Not to say there couldn't be some case where that happens.
Eu lamento lhe dizer, mas o problema não é com o app não. Eu consegui ouvir claramente a frequência. Pode ser que alguns telefones, muito dificilmente, não reproduzam, mas o app do RUclips entrega sim a reprodução.
This video is amazing, dude. Love your piano covers to bits, and it's such a blessing hearing you gush about how the Wii stores its sounds. Can't wait to see more stuff like this!
It was actually super surprising to hear that this was the first video of this kind that you've made, it was super well-written, edited, and presented, and very interesting! You definitely have a knack for this and we would love to see more!
I didn't expect such a kind of video coming from you (I don't mean that in a negative way), but it was very interesting to watch. I really enjoy music analysis videos so thank you for making one too and putting so much effort into it. Now, I may not be the best at understanding what all the different files or file types do, but I did learn at least a bit more about them from you explaining them. Looking forward to more stuff like this! :)
I should clarify that this is exactly how music works on every Nintendo console that came afterwards: 3DS, Wii U and Switch. They kept reusing NintendoWare. Also, the DSP is a hardware feature (which all of the aforementioned consoles have). It has nothing to do with NintendoWare itself.
PCM isn't compressed. It's plain sampled amplitudes. ADPCM uses differentials rather than absolute amplitudes. Interesting that they didn't provide FLAC (the library is BSD licensed) and Ogg Vorbis. FLAC would cut audio size approximately in half. A more modern approach could include software or hardware synthesis, with FM and some analog synthesis modeling being pretty lightweight, while physical modeling is extremely computationally-intensive.
I was looking for a comment about PCM not being compression :) Those compression formats you mentioned would probably use considerably more CPU or require dedicated extra hardware, maybe that’s the reason they didn’t include them?
@@danielschenk91 Short consideration: the real-time synthesis is also computationally costly, and the below about cache considerations applies. Long consideration: Decompression is generally faster than compression, and with FLAC it's like 50-ish CPU cycles per sample, so maybe 2.5 million cycles, so maybe 0.32%. That comes alongside disc and flash access times (which create interrupts) and storage in memory. Reading across uncompressed PCM audio consumes more CPU cache and triggers more cache misses, which are expensive (they can be hundreds of CPU cycles to service). Reading across compressed audio like FLAC and decoding into a ring buffer is nicer for CPU cache and so can increase the amount of work the CPU can do in total. This is also an issue with binary trees vs. hash tables vs. arrays, and other considerations when programming where doing more computations can actually be faster because the CPU stalls less often. Be mindful, a cache line is 64 bytes, which is 2/3 of a millisecond of uncompressed audio, so with a 50% compression ratio streaming 20ms chunks of audio you're crossing either 30 (uncompressed) or 15 (compressed) cache lines-if the working memory for the decompressor stays at the same address to avoid more cache invalidation, of course. Increasing the size of the buffered audio stream doesn't increase the working memory needed for decompressing. Point is you gain some processing power from avoiding cache misses, and lose some processing power from doing decoding computations.
Mario Kart Wii's title and main menu themes are found as wave format and the wifi + all the Mario Kart Channel themes are sequence music inside the game's BRSAR. There are also unused options menu instruments only, but those go unused as they decided to make it a BRSTM instead.
You have a very good voice for these kinds of videos. I don’t think I understood even half of what you were talking about, yet I still felt compelled to listen through in its entirety
I'm only 7 minutes in and this video is mind blowingly high quality. Like these animations around the waveform and muted colors are the perfect selection for highlighting the snippets of audio as you're talking about them. Instant subscriber!
I mod Wii games, so I didn't really learn anything here, but I was very pleasantly surprised that you actually knew what you were talking about. Like, I expected an extremely broad video that was like "they used BRSTM files and sometimes they have multiple tracks lol isn't that cool" but this is genuinely a good overview of the NintendoWare audio pipeline. Wish there was more stuff on RUclips like this.
Leaving "Change Player List" back into the options will play the options half of "STM_TITLE_RET.brstm" from the beginning, which skips the short intro section present in "STM_TITLE.brstm" that I showed in this video. Loop points in both are also past the intro.
Would love to see a playlist dedicated to content like this! Btw, your voice is vaaaguely reminiscent of one of the narrators on How It's Made. Also, show Nintendo who really owns the intellectual property! Take that!
As someone who has been obsessed with this stuff for years, this is such a good video, I love it so much, even down to the tiny tidbits like pointing out how SPM is in 22.05k (aside from a couple songs for some reason). I remember when it was super hard to rip modern sequenced nintendo music (B*SEQ) and I'd just wonder what the heck was going on with them, lol. I remember messing around with a really old command line tool to concert brseqs from the photo channel to midis and just listening to them for hours I feel like sequenced music is a lost art in a lot of modern games, the only games I can think of in recent memory are the warioware games. I think there's some indie outliers too (including maybe something I'd like to do myself in the future...) Anyway, really good video! Instant sub, glad this got recommended to me
I've never found video game music super interesting (as in how it works, there are soooo many good soundtracks I adore) but this is such a good video I was interested all the way through. Very nice explanation!
Growing up, I always felt the music in Mega Man 9’s Wii port sounded slightly different than the 360/PS3/Official Soundtrack versions. I was able to kind of hear it in Jewel Man’s Stage, or Dr. Wily’s Stage 2 when you do a side by side comparison. Another game that I think also sounds slightly different than it’s high quality soundtrack version is Sonic 4 on the Wii. I’m happy to see that this explanation is most likely why this happens!
If you like music that changes dynamically with gameplay, try looking for a game that uses FMOD. it's an adaptive music engine that mostly uses streamed audio, but it also supports sequences. If you didn't know, celeste uses FMOD, and there's even a publicly available download for celeste's FMOD project. Celeste uses both streamed and sequenced, and there's also live processing. you can hear the sequences when you're in a room that has "cassette blocks", which are pink and blue blocks that toggle their collisions to the beat. you can tell it's sequenced because it pauses for a bit when you die, for it to then start playing once it's synced to the blocks after respawning. the most notable example of processing would probably be in chapter 9, where a screen ends up being longer than you were lead to believe, and in that part of the screen where you have a bunch of anxiety inducing unexpected gameplay, a lowpass filter kicks in
the wii (and gamecube for that matter) could have supported 48 khz if a dev had really really wanted to, the audio DAC itself is 48 khz, it's just that both of the audio microcodes nintendo released/used (the AX microcode provided to 3rd parties and the zelda microcode used internally) only operated at 32 khz for processed audio. (streaming audio was natively 48 khz, and was mixed into the rest of the game's audio after the audio processing was finished, but due to its limitations it could only really be used in cutscenes). there were a couple of third party titles where the devs wrote their own audio microcode from scratch and got 48 khz for everything, but they weren't really significant.
I love this type of video!! It reminds me of scruffy’s style of content - not in a bad way, it’s hard to find such entertaining well edited videos explaining topics like these. Much love from someone who’s watched your older videos for a pretty long time, now :)
Thanks for the interesting video! I would've liked to hear short examples of the effects you talked about in greater depth/the ones that caused issues, but I realize that might have been a lot of work. When you did do it I felt like I fully understood the effect/problem. Hope more people see this video!
the technical complexity behind video game music is fascinating, and that’s before we even start talking about the music theory behind it. this video is great! thanks for researching and created awesome visuals.
It's actually streamed audio, just layered on top of each other. The files are all wav files located in the music folders - no sequence files in the game files (not even the compressed pak files). Just well timed scripts firing.
That was very educational, thank you. You reference songs all throughout the video though that I wish you had showed examples of. Like when talking about sequence music you mention the live orchestra in galaxy. That would have been neat to hear the example you were referencing.
This was an incredibly interesting video. Got me in the mood to give ripping Wii Fit Plus songs another shot. Also i find that your voice is perfect for this kind of content.
(copied reply to similar comment) I played the 16kHz+ example from this video on the RUclips app out of my phone speakers into my microphone, and it was able to be picked up fully up to 20kHz imgur.com/a/FQn5Aw9 Not to say there couldn't be some case where that happens.
@@dublincalifCannot hear it in app with phone speakers. Still cannot hear it on app with headphones. CAN hear it on phone speakers on phone's browser.
@@dublincalifBeen doing the last 3 weeks checking out my music playlist. The audible audio quality loss between phone app and phone browser is INSANE. Why is app bad?
@@DarkonFullPower Maybe it's the video quality? The audio quality scales with video quality. It's very noticeable if you drop to like 144p. Edit: It doesn't seem to be that, though the audio quality is noticeably worse in 144p.
Oh rhis is absolutely fantastic! I thought this was going to be super basic and dismissive like "did you know the Wii used MIDI?!!!" but it's awesome seeing specifics as well as actual examples of the dev kits and tools they used. The editing is also super sharp!
I loved this. I am begging you to do a video on the DS's audio. Its music and soundscape feels so unique and distinct from everything else as this interesting compromise between modern capabilities and old qualities. Maybe I'm just nostalgic, though.
I'm a video game audio engineer myself, and this was a great video. One thing I will say is that modern audio middleware allows for full dynamic and interactive systems- the benefit of sequencers has diminished with the exception of file size and player generated content
Sequenced music for games is so good, I’ve been working on a game and I’ve been making the soundtrack from sequenced music because it’s able to respond to what’s going on in-game at a finer detail than simply having the option to mix between channels, for instance looping is nice but with sequenced music you can convert a song into a Markov chain and have it loop in a weighted random path and these paths can have connections to play a bridging piece of music to a different song seamlessly - those can be random, or controlled random (for instance you can require a song play for at least 2 minutes and after then it can leave the song, and if it’s been over 5 minutes it will take the next exit), or completely controlled (such as a main menu theme which has bridges to all songs in every level as soon as you select a level the music is told that it needs to get to a certain song). You can also tweak the tempo and the layers to fit different scenarios, when you pause the game I’ve currently got it set to mute some channels and slow it down (and also the music will prefer to stay on the current song for longer, although give it 15 minutes paused and it will move on; with just the backing track it’s not annoying) No doubt future games I’ll make will have sequenced music too because being able to tie the music close to the game like other assets feels essential and gamers can feel the extra effort put in
The one thing I hate people saying “the motion controls suck” No kyle. You are sitting/laying down on your couch when you play. Stand up. Get invested. It works so much better when you play the wii like its, yk, a wii
Great video, I enjoyed learning more of the science behind BRSTMs, as I used to convert songs to that format for Smash Bros Brawl / Project M back in the day for fun. Also, I love that you sampled the soundtrack from Fluidity in the background, I adore that game.
Weird that the MM9 audio cares so much about square falloff and ignores the triangle's aliasing that was on the NES. Completely changes its sound. Even more jarring when you hear the prerecorded triangle "tom drums" that DO include the aliasing.
I got recommended your latest video, and then I found this one. Peak content, very informative, love that you tread into this much detail. Also, sequenced music my beloved.
23:00 This is actually what has happened, "streamed" music is so much more complex nowadays it goes even beyond what sequenced music could do, dynamic music is crazy
i was expecting to see atleast 200k views on this, but when i saw 30k i was shocked! this video is well thought and explained, and the visuals are wonderful. thanks so much for this, i learned a lot and it held my attention!
HOLY CRAP!! WAIT!! I watched this whole video and didnt realize it was YOU until the end!! Amazing vid, man! If you enjoy it, keep it up! Huge props :)) hope you're doing well
I really enjoyed this. I am a big sequenced music enjoyer. the one thing I noticed was that at 2:35 you say "[x] thousand kilohertz" twice, where I think actually you're referring to 32kHz or 32,000Hz. 32,000kHz would be 32MHz lol. I would not have known this is your first video like this though I absolutely loved it. thanks for sharing
I think its a shame that the modern age of terabyte harddrives has resulted in games no longer bothering to optimise things like music, especially when it actually lets you do super cool stuff you just don't see anymore, like in monkey island where the tracks seamlessly intertwine as you go from room to room
You can still do things like that. The game Balatro, an indie game which released earlier this year, has one song play throughout which cycles through different versions with different instruments. It’s more a case of people not bothering to implement cool transitions in music than a case of being unable to do so
Plenty, I would say almost all modern games have fully dynamic and interactive music systems that go far beyond simply transitioning the music, I encourage you to pay attention more when you play.
You can and do get smooth audio transitions in modern games, it isn't some lost technique. The modern Doom games handle this when transitioning in and out of combat.
Yeah, all the storage space and memory capacity improvements have made optimization a lost art. Piracy issues aside, it's why I love the repacking community's work, proving that games can be cut down to fractions of their installation size without any real compromise -- often even improving performance.
15:48 "The source of the drum loop is actually a well-known-" yup, it's gonna be the Amen break. Love the level of detail here. What's the song during the Closing segment?
The lack of playing examples of the various tracks and instead of talking about them is a bit frustrating… eg. The Mewtwo theme is your favorite, but you didn’t play any of it 😢 similarly it would have been nice to _hear_ the bowling theme with the ‘speaker in bowling alley’ effects, as captured from the game itself. No doubt this was to mitigate the risk of copyright strikes, but it would have been really helpful for context.
I have a real soft spot for sequenced music. So much variety, and once you have the samples, it's much easier to create! It's a very accessible option for newcomers to make music that reminds them of familiar games and sounds
The Futurama end credits are awesome, I love making my little mouse cursor dance along with it after I watch a legally acquired copy of an episode of Futurama on my personal computer! The song I was referring to? It goes a little something like this: Doo-doo, doo-doo! Doo-doo, do-do-do-do-do-do. Bwom ba bah bah bah [Billy West's name appears onscreen] Ba hahaha hahahaaaa!!!!!!!!! I shared this video with 10 of my friends
In a similar way, a lot of early computer games used sequenced music on FM synthesizer chips made by Yamaha (e.g. MSX, NEC PC98, IBM PC). Later many PC games, including the original Unreal & Unreal Tournament, switched to using soundtracker modules (MODs) which originated on the Commodore Amiga and are still a staple of the demoscene along with more traditional chiptunes, e.g. Commodore 64 SID chip.
Great video and very interesting! I am working at House of Elias (previously known as Elias Software) and we are working hard on bringing midi back for video games and make it easy to use with high quality samples. The sampler and instrument libraries can sound really good today, so the possibilities will be endless. :)
Honestly the Zelda Switch games BOTW and TOTK using steamed audio do a fantastic job making the music dynamic, especially in combat. It's really something special what they did with the sound design in that game.
Ehhhhhh.... I actually had the DAW Evanescence recorded their albums on around this time, and this stuff isn't even close. It really isn't about the DAW, but the available plugins.
WAIT A MINUTE your that guy. i ditn even recognise you because usually ur style of video is different. havn't seen ur channel in a really long time. im glad ur uploading these now, ima subscribe :)
This was one of the notable selling points that the PlayStation 1 hinged on versus the N64. Since games were on discs, PlayStation games easily had 10x-20x (or more) storage over an N64 cartridge. This comes at the sacrifice of load times of course.
This reminded me how the music in the Wii version of Sonic 4: Episode 1 is quite different from the PC/X360/PS3 versions, I always thought it was because Sega used some compression system due to the space limits of the WiiWar games, it's really cool to know the real reason for this, it made me curious if other multiplatform games did this.
Funny to see my AC: CF playlist in a youtube movie ^_^
As for your remarks about zeroing all sound effects and record without sounds: this was the case for gamecube games where only one *.aw contained the instrument sounds. For brsar all sounds were played by rseq. Often they were rseq files with one note playing. Rseq was not reverse engineered yet, but changing all notes data to value 255 stopped the sounds. rseq files contain labels, so i could easily find out which one contained sound effects. Animal Crossing contained more than 10000 rseq files so i had to write a script for modifying the brsar to only play songs back then
Wow, surprised to see you're still around
You're the goat, I've downloaded so many CF songs from your channel, so ty!!
Been listening to Super KK Slider soundtrack for 15 years now... wild...
ur vids made my childhood. hope ur doing well
"...losing this range of frequencies:" *[SCREAMING BUG SOUND]*
oh no, my hearing really is bad isn’t it?
@@shan8130 could also just be low quality speaker systems
@@champagnesupernova1839 Yeah! Yeah... Let's uh, let's go with that.
@@champagnesupernova1839...On what device have you watched this video on? RUclips usually has a frequency cutoff around that area. In fact, the video had no audio output at that moment for me using the RUclips app. How did you hear that?
@@champagnesupernova1839 yeag im not surprised my phone cant do *usually* out of range frequencies
I LIVE for sequenced songs, I don't know why they're so fascinating to me. Thank you for making this video!
They're basically tracker modules, except split up. The Mod Archive has thousands of original ones.
Have you seen Ahoy's video on tracker music? If not, I think you'd like it.
@@TheYoshieMaster Oh, yeah. That's a good one. I'll second the recommendation!
Nitpick, but the frequencies aren't in *thousands* of kHz. The k means thousand already, so you'd be up in the MHz range.
Despite me being aware of this common mistake in the past, I can't seem to stop making it myself
@@dublincalif we all knew what you meant. I just want to look smart on the internet.
Yeah
and also PCM isn't a compression scheme. It's just a way of storing an analog signal in a digital file
@@tehrobotjesus Me, too :/
I personally can never overstate how much I appreciate someone like you who actually does the needed research on console's audio specs and functionalities. Rather than the laughable latter half of people who bluntly assumes and mislead with "MIDI this and Soundfont that" and thinks that's the "only formats that's ever existed."
Thank you so much for making this video and thank you for reading in on the topic unlike everyone else.
(Everyone else who just reads the first sentence on Wikipedia.)
I get more annoyed than I should at people saying theyre looking for a soundfont for a ds game as if that exists
@@literallywhons4syeah I’ve done that a few times
@@literallywhons4s if you mean VGMTrans, it *converts* to midi and dls the best it can, but the actual ds sound data is stored in a different format that's only mostly compatible
@@literallywhons4s The data is not stored as MIDI data and SF2 soundfonts internally. What you get from using such tools is simply an _attempt_ to convert the sequence and sample data into more common and more-documented formats. However, due to the differences in native capabilities between the formats, the conversion is often not completely perfect or faithful; examples of this can be inaccuracies in the volume of certain tracks or instruments, or missing DSP effects.
If you're looking for for faithful playback of sequenced NDS audio, then you may want to look into the 2SF and NCSF formats instead.
@gubmen6969 many people make soundfonts based on these game's sequence instruments so they probably only know those exist, and nothing else
City folk's title screen was streamed at 44.1khz and I think the gamecube had some 48khz music in the service disc and some unused tracks in sonic mega collection
super monkey ball's music is 48khz
A little bit of extra history/trivia -- you briefly touched on Super Smash Bros Brawl using BRSTMs. Music replacement was really big really early in SSBB's hacking days, which is why the BrawlBRSTMs RUclips channel got so big, and why their 30-minute extension format became so iconic for game music uploads on RUclips -- they made it that long so that anyone seeking to replace music in Brawl wouldn't have to futz around with figuring out the correct loop point for what they're converting to BRSTM.
That's not the reason why the videos were made that long.
All of the 30-minute long videos of music posted on BrawlBRSTMs were effectively previews of pre-made BRSTMs hosted on the website "Smash Custom Music" (formerly known as "Brawl Custom Music"), a site originally ran by the same person who ran the BrawlBRSTMs channel on RUclips: Segtendo.
Each song hosted on this site was made available to download as a BRSTM (in addition to several other audio formats commonly-used by games) with their loop points and other metadata already written in the file.
Each video on the channel featured a link to the song's respective page on the website in their descriptions.
Do any of you guys know the music being played at 6:35 ?
the music was also available on their website directly as a brstm with correct loops
“Are you able to hear this?”
..silence..
I can hear it hehe
@@zapowwsame
Maybe it does play something but our ears are just built differently. Gonna ask my dog if he can hear it.
I'm blaming it on my phone
It felt tingly
2:50 worth noting that on mobile, sounds above like 15khz are cut off by the RUclips app/phone. So not hearing it here doesn’t necessarily mean you can’t hear that frequency.
I played the 16kHz+ example from this video on the RUclips app out of my phone speakers into my microphone, and it was able to be picked up fully up to 20kHz imgur.com/a/FQn5Aw9
Not to say there couldn't be some case where that happens.
I felt some pressure from my headset on my ears but didn't hear a thing
Eu lamento lhe dizer, mas o problema não é com o app não. Eu consegui ouvir claramente a frequência. Pode ser que alguns telefones, muito dificilmente, não reproduzam, mas o app do RUclips entrega sim a reprodução.
@@dublincalifThis link was deleted and may not be accessed
Dang, just last year I could still hear 17.5khz. Don’t even listen to music anymore, seems strange I can’t hear above 16khz anymore. Aging sucks.
This video is amazing, dude. Love your piano covers to bits, and it's such a blessing hearing you gush about how the Wii stores its sounds. Can't wait to see more stuff like this!
It was actually super surprising to hear that this was the first video of this kind that you've made, it was super well-written, edited, and presented, and very interesting! You definitely have a knack for this and we would love to see more!
agree!!
I didn't expect such a kind of video coming from you (I don't mean that in a negative way), but it was very interesting to watch. I really enjoy music analysis videos so thank you for making one too and putting so much effort into it. Now, I may not be the best at understanding what all the different files or file types do, but I did learn at least a bit more about them from you explaining them. Looking forward to more stuff like this! :)
I should clarify that this is exactly how music works on every Nintendo console that came afterwards: 3DS, Wii U and Switch.
They kept reusing NintendoWare.
Also, the DSP is a hardware feature (which all of the aforementioned consoles have). It has nothing to do with NintendoWare itself.
Google says that the GameCube had a DSP, but Wii does not?
@@ArneChristianRosenfeldt nope. GameCube, Wii, 3DS, Wii U and Switch all have DSPs, and all of them use the same ADPCM encoding.
PCM isn't compressed. It's plain sampled amplitudes. ADPCM uses differentials rather than absolute amplitudes. Interesting that they didn't provide FLAC (the library is BSD licensed) and Ogg Vorbis. FLAC would cut audio size approximately in half. A more modern approach could include software or hardware synthesis, with FM and some analog synthesis modeling being pretty lightweight, while physical modeling is extremely computationally-intensive.
I was looking for a comment about PCM not being compression :)
Those compression formats you mentioned would probably use considerably more CPU or require dedicated extra hardware, maybe that’s the reason they didn’t include them?
@@danielschenk91 Short consideration: the real-time synthesis is also computationally costly, and the below about cache considerations applies.
Long consideration: Decompression is generally faster than compression, and with FLAC it's like 50-ish CPU cycles per sample, so maybe 2.5 million cycles, so maybe 0.32%. That comes alongside disc and flash access times (which create interrupts) and storage in memory. Reading across uncompressed PCM audio consumes more CPU cache and triggers more cache misses, which are expensive (they can be hundreds of CPU cycles to service). Reading across compressed audio like FLAC and decoding into a ring buffer is nicer for CPU cache and so can increase the amount of work the CPU can do in total. This is also an issue with binary trees vs. hash tables vs. arrays, and other considerations when programming where doing more computations can actually be faster because the CPU stalls less often.
Be mindful, a cache line is 64 bytes, which is 2/3 of a millisecond of uncompressed audio, so with a 50% compression ratio streaming 20ms chunks of audio you're crossing either 30 (uncompressed) or 15 (compressed) cache lines-if the working memory for the decompressor stays at the same address to avoid more cache invalidation, of course. Increasing the size of the buffered audio stream doesn't increase the working memory needed for decompressing. Point is you gain some processing power from avoiding cache misses, and lose some processing power from doing decoding computations.
@@johnmoser3594 Very interesting stuff!
Mario Kart Wii's title and main menu themes are found as wave format and the wifi + all the Mario Kart Channel themes are sequence music inside the game's BRSAR. There are also unused options menu instruments only, but those go unused as they decided to make it a BRSTM instead.
The option instruments aren’t unused, the Options menu in the Mario Kart channel uses them.
@@JohnP55 They are unused, MKC uses different ones.
@@PlayersPurityYes, it’s a different BRSAR, but they’re the same samples.
You did such a great work ! Please make other videos like that !
You have a very good voice for these kinds of videos. I don’t think I understood even half of what you were talking about, yet I still felt compelled to listen through in its entirety
I did NOT expect siivagunner to show up here
Now im interested, time to make my own high quality rips
I'm only 7 minutes in and this video is mind blowingly high quality. Like these animations around the waveform and muted colors are the perfect selection for highlighting the snippets of audio as you're talking about them. Instant subscriber!
I mod Wii games, so I didn't really learn anything here, but I was very pleasantly surprised that you actually knew what you were talking about. Like, I expected an extremely broad video that was like "they used BRSTM files and sometimes they have multiple tracks lol isn't that cool" but this is genuinely a good overview of the NintendoWare audio pipeline. Wish there was more stuff on RUclips like this.
This might be the best video I've seen about how video game music works. Really good job
The beginning of the Wii sports resort options menu can be heard if you go into the player change list and leave or wait long enough for it to loop.
Leaving "Change Player List" back into the options will play the options half of "STM_TITLE_RET.brstm" from the beginning, which skips the short intro section present in "STM_TITLE.brstm" that I showed in this video. Loop points in both are also past the intro.
@@dublincalif I may have misremembered it ._.
Oh my gosh an analysis video!? I am literally shivering in me timbers!
Ok wow this is a well researched video
Would love to see a playlist dedicated to content like this!
Btw, your voice is vaaaguely reminiscent of one of the narrators on How It's Made. Also, show Nintendo who really owns the intellectual property! Take that!
I would literally click back & forth the different apps just to listen to their music. That ethereal abstract spacey vibe was so good
As someone who has been obsessed with this stuff for years, this is such a good video, I love it so much, even down to the tiny tidbits like pointing out how SPM is in 22.05k (aside from a couple songs for some reason). I remember when it was super hard to rip modern sequenced nintendo music (B*SEQ) and I'd just wonder what the heck was going on with them, lol. I remember messing around with a really old command line tool to concert brseqs from the photo channel to midis and just listening to them for hours
I feel like sequenced music is a lost art in a lot of modern games, the only games I can think of in recent memory are the warioware games. I think there's some indie outliers too (including maybe something I'd like to do myself in the future...)
Anyway, really good video! Instant sub, glad this got recommended to me
Your explanation of Streamed music just helped me shave off hundreds of megabytes of memory use on my own game. Thanks!
I'm so glad someone finally dove into this topic. I'm curious how other past consoles produced their music as well now
please do more of these. i LOVE deep dives in obscure/rarely talked about technologies
I've never found video game music super interesting (as in how it works, there are soooo many good soundtracks I adore) but this is such a good video I was interested all the way through. Very nice explanation!
Growing up, I always felt the music in Mega Man 9’s Wii port sounded slightly different than the 360/PS3/Official Soundtrack versions.
I was able to kind of hear it in Jewel Man’s Stage, or Dr. Wily’s Stage 2 when you do a side by side comparison.
Another game that I think also sounds slightly different than it’s high quality soundtrack version is Sonic 4 on the Wii.
I’m happy to see that this explanation is most likely why this happens!
my favorite nintendo-songs-with-a-piano-rendition youtuber made a video essay on something nerdy let's go
Do any of you guys know the music being played at 6:35 ?
This video was fire dude please don't stop making content like this
If you like music that changes dynamically with gameplay, try looking for a game that uses FMOD. it's an adaptive music engine that mostly uses streamed audio, but it also supports sequences. If you didn't know, celeste uses FMOD, and there's even a publicly available download for celeste's FMOD project.
Celeste uses both streamed and sequenced, and there's also live processing. you can hear the sequences when you're in a room that has "cassette blocks", which are pink and blue blocks that toggle their collisions to the beat. you can tell it's sequenced because it pauses for a bit when you die, for it to then start playing once it's synced to the blocks after respawning. the most notable example of processing would probably be in chapter 9, where a screen ends up being longer than you were lead to believe, and in that part of the screen where you have a bunch of anxiety inducing unexpected gameplay, a lowpass filter kicks in
the wii (and gamecube for that matter) could have supported 48 khz if a dev had really really wanted to, the audio DAC itself is 48 khz, it's just that both of the audio microcodes nintendo released/used (the AX microcode provided to 3rd parties and the zelda microcode used internally) only operated at 32 khz for processed audio. (streaming audio was natively 48 khz, and was mixed into the rest of the game's audio after the audio processing was finished, but due to its limitations it could only really be used in cutscenes). there were a couple of third party titles where the devs wrote their own audio microcode from scratch and got 48 khz for everything, but they weren't really significant.
I cannot believe how in depth this video was...way more information than I ever thought I'd learn about this 😂 great job!
I love this type of video!! It reminds me of scruffy’s style of content - not in a bad way, it’s hard to find such entertaining well edited videos explaining topics like these. Much love from someone who’s watched your older videos for a pretty long time, now :)
Thanks for the interesting video! I would've liked to hear short examples of the effects you talked about in greater depth/the ones that caused issues, but I realize that might have been a lot of work. When you did do it I felt like I fully understood the effect/problem. Hope more people see this video!
Insanely well crafted video. Keep up the great work!
This is the kind of content I dream about! I’ve never seen your other videos but I love technical videos about games!
the technical complexity behind video game music is fascinating, and that’s before we even start talking about the music theory behind it.
this video is great! thanks for researching and created awesome visuals.
Don't know if I'll ever use this information, but I like hearing how things work. Great video.
First video? It's such a good researched and explained video. I enjoyed it a lot.
This was fascinating af
Wow I really enjoyed this video! I'm happy to see you branch out your content!!
The amount of care put into the Wii's OS and interface was unprecedented.
Left 4 Dead and Left 4 Dead 2 also use sequenced music to add and remove instruments during Hoarde events!
It's actually streamed audio, just layered on top of each other.
The files are all wav files located in the music folders - no sequence files in the game files (not even the compressed pak files). Just well timed scripts firing.
You should totally do a video on portal’s music! I know it’s not Nintendo but the way it handles its music is fascinating
That was very educational, thank you. You reference songs all throughout the video though that I wish you had showed examples of. Like when talking about sequence music you mention the live orchestra in galaxy. That would have been neat to hear the example you were referencing.
This was an incredibly interesting video. Got me in the mood to give ripping Wii Fit Plus songs another shot. Also i find that your voice is perfect for this kind of content.
2:47 Note for RUclips mobile users. Mobile caps sound at 15-16khz, so anything above that cannot be heard. Play the video on PC to hear it.
(copied reply to similar comment) I played the 16kHz+ example from this video on the RUclips app out of my phone speakers into my microphone, and it was able to be picked up fully up to 20kHz imgur.com/a/FQn5Aw9
Not to say there couldn't be some case where that happens.
I can hear it clearly on Android.
@@dublincalifCannot hear it in app with phone speakers. Still cannot hear it on app with headphones. CAN hear it on phone speakers on phone's browser.
@@dublincalifBeen doing the last 3 weeks checking out my music playlist. The audible audio quality loss between phone app and phone browser is INSANE. Why is app bad?
@@DarkonFullPower Maybe it's the video quality? The audio quality scales with video quality. It's very noticeable if you drop to like 144p.
Edit: It doesn't seem to be that, though the audio quality is noticeably worse in 144p.
I may have also played around with Wii sound files and seeing you talk about it just makes me happy, good luck to you ✌️
man i've been watching you for years now, and this new style of content is great!! love it. keep it up
Oh rhis is absolutely fantastic! I thought this was going to be super basic and dismissive like "did you know the Wii used MIDI?!!!" but it's awesome seeing specifics as well as actual examples of the dev kits and tools they used. The editing is also super sharp!
I loved this. I am begging you to do a video on the DS's audio. Its music and soundscape feels so unique and distinct from everything else as this interesting compromise between modern capabilities and old qualities. Maybe I'm just nostalgic, though.
This was amazing. Thank you for taking the time to go so in depth with everything.
I'm a video game audio engineer myself, and this was a great video. One thing I will say is that modern audio middleware allows for full dynamic and interactive systems- the benefit of sequencers has diminished with the exception of file size and player generated content
Very fascinating breakdown of the Wii's aural interior! Keep up the good work, mate.
Sequenced music for games is so good, I’ve been working on a game and I’ve been making the soundtrack from sequenced music because it’s able to respond to what’s going on in-game at a finer detail than simply having the option to mix between channels, for instance looping is nice but with sequenced music you can convert a song into a Markov chain and have it loop in a weighted random path and these paths can have connections to play a bridging piece of music to a different song seamlessly - those can be random, or controlled random (for instance you can require a song play for at least 2 minutes and after then it can leave the song, and if it’s been over 5 minutes it will take the next exit), or completely controlled (such as a main menu theme which has bridges to all songs in every level as soon as you select a level the music is told that it needs to get to a certain song). You can also tweak the tempo and the layers to fit different scenarios, when you pause the game I’ve currently got it set to mute some channels and slow it down (and also the music will prefer to stay on the current song for longer, although give it 15 minutes paused and it will move on; with just the backing track it’s not annoying)
No doubt future games I’ll make will have sequenced music too because being able to tie the music close to the game like other assets feels essential and gamers can feel the extra effort put in
The one thing I hate people saying “the motion controls suck”
No kyle. You are sitting/laying down on your couch when you play. Stand up. Get invested. It works so much better when you play the wii like its, yk, a wii
9:28 saving my spot ignore this
such a great video! Specially loved the technical details and the quality of the speaking, keep going!
Great video, I enjoyed learning more of the science behind BRSTMs, as I used to convert songs to that format for Smash Bros Brawl / Project M back in the day for fun. Also, I love that you sampled the soundtrack from Fluidity in the background, I adore that game.
Thank you for making a great video on such an underrated subject!
Weird that the MM9 audio cares so much about square falloff and ignores the triangle's aliasing that was on the NES. Completely changes its sound. Even more jarring when you hear the prerecorded triangle "tom drums" that DO include the aliasing.
Genuinely fantastic video, I would love to see more like this.
I got recommended your latest video, and then I found this one. Peak content, very informative, love that you tread into this much detail.
Also, sequenced music my beloved.
Amazing content bro! Im a huge music guy, and this was awsome. Ive been messing with soundphonts and its so fun!
23:00 This is actually what has happened, "streamed" music is so much more complex nowadays it goes even beyond what sequenced music could do, dynamic music is crazy
i was expecting to see atleast 200k views on this, but when i saw 30k i was shocked! this video is well thought and explained, and the visuals are wonderful. thanks so much for this, i learned a lot and it held my attention!
HOLY CRAP!! WAIT!! I watched this whole video and didnt realize it was YOU until the end!! Amazing vid, man! If you enjoy it, keep it up! Huge props :)) hope you're doing well
I really enjoyed this. I am a big sequenced music enjoyer. the one thing I noticed was that at 2:35 you say "[x] thousand kilohertz" twice, where I think actually you're referring to 32kHz or 32,000Hz. 32,000kHz would be 32MHz lol.
I would not have known this is your first video like this though I absolutely loved it. thanks for sharing
I think its a shame that the modern age of terabyte harddrives has resulted in games no longer bothering to optimise things like music, especially when it actually lets you do super cool stuff you just don't see anymore, like in monkey island where the tracks seamlessly intertwine as you go from room to room
You can still do things like that. The game Balatro, an indie game which released earlier this year, has one song play throughout which cycles through different versions with different instruments. It’s more a case of people not bothering to implement cool transitions in music than a case of being unable to do so
Plenty, I would say almost all modern games have fully dynamic and interactive music systems that go far beyond simply transitioning the music, I encourage you to pay attention more when you play.
You can and do get smooth audio transitions in modern games, it isn't some lost technique. The modern Doom games handle this when transitioning in and out of combat.
Yeah, all the storage space and memory capacity improvements have made optimization a lost art. Piracy issues aside, it's why I love the repacking community's work, proving that games can be cut down to fractions of their installation size without any real compromise -- often even improving performance.
Shush, the Devs put gay people into the game/j
15:48 "The source of the drum loop is actually a well-known-" yup, it's gonna be the Amen break.
Love the level of detail here. What's the song during the Closing segment?
Yo. This was great man. Would love to see more like it. Your approach is great
I didn't know who you were, but I like technical video game related videos so am glad the algorithm sent this my way
The lack of playing examples of the various tracks and instead of talking about them is a bit frustrating… eg. The Mewtwo theme is your favorite, but you didn’t play any of it 😢 similarly it would have been nice to _hear_ the bowling theme with the ‘speaker in bowling alley’ effects, as captured from the game itself.
No doubt this was to mitigate the risk of copyright strikes, but it would have been really helpful for context.
Great video. Appreciate the technical knowledge share, even if some of the details were a little off, I hugely appreciated this!
This was very insightful, thanks for uploading!
That was fascinating, would love to watch more
Awesome video man love to see new people trying to make people’s world better
I have a real soft spot for sequenced music. So much variety, and once you have the samples, it's much easier to create! It's a very accessible option for newcomers to make music that reminds them of familiar games and sounds
I’m making a game for Gameboy with sequenced music
The Futurama end credits are awesome, I love making my little mouse cursor dance along with it after I watch a legally acquired copy of an episode of Futurama on my personal computer!
The song I was referring to? It goes a little something like this:
Doo-doo, doo-doo! Doo-doo, do-do-do-do-do-do.
Bwom ba bah bah bah [Billy West's name appears onscreen]
Ba hahaha hahahaaaa!!!!!!!!!
I shared this video with 10 of my friends
In a similar way, a lot of early computer games used sequenced music on FM synthesizer chips made by Yamaha (e.g. MSX, NEC PC98, IBM PC). Later many PC games, including the original Unreal & Unreal Tournament, switched to using soundtracker modules (MODs) which originated on the Commodore Amiga and are still a staple of the demoscene along with more traditional chiptunes, e.g. Commodore 64 SID chip.
Great video and very interesting!
I am working at House of Elias (previously known as Elias Software) and we are working hard on bringing midi back for video games and make it easy to use with high quality samples. The sampler and instrument libraries can sound really good today, so the possibilities will be endless. :)
Great video, mabye you'll tackle the music of the DS ?
The editing is top tier!
Honestly the Zelda Switch games BOTW and TOTK using steamed audio do a fantastic job making the music dynamic, especially in combat. It's really something special what they did with the sound design in that game.
this is super great! I'd love to see more stuff like this, keep up the good work!
Amazing work, please do more of these!
Kinda hard to believe that the Wii could have been the best DAW at the time. Truly mind blowing….
Ehhhhhh.... I actually had the DAW Evanescence recorded their albums on around this time, and this stuff isn't even close. It really isn't about the DAW, but the available plugins.
WAIT A MINUTE your that guy. i ditn even recognise you because usually ur style of video is different. havn't seen ur channel in a really long time. im glad ur uploading these now, ima subscribe :)
Criminally underrated video.
This was one of the notable selling points that the PlayStation 1 hinged on versus the N64. Since games were on discs, PlayStation games easily had 10x-20x (or more) storage over an N64 cartridge. This comes at the sacrifice of load times of course.
16:35 So *that's* what the -almond break- Amen break is!!
Don’t even get me started on that tank mini game. I felt like I was blind and deaf trying to tell people how awesome that music is.
2:54 I couldn't hear that for I had my volume off.
This reminded me how the music in the Wii version of Sonic 4: Episode 1 is quite different from the PC/X360/PS3 versions, I always thought it was because Sega used some compression system due to the space limits of the WiiWar games, it's really cool to know the real reason for this, it made me curious if other multiplatform games did this.
Such a fascinating video, amazing job!