As for your remarks about zeroing all sound effects and record without sounds: this was the case for gamecube games where only one *.aw contained the instrument sounds. For brsar all sounds were played by rseq. Often they were rseq files with one note playing. Rseq was not reverse engineered yet, but changing all notes data to value 255 stopped the sounds. rseq files contain labels, so i could easily find out which one contained sound effects. Animal Crossing contained more than 10000 rseq files so i had to write a script for modifying the brsar to only play songs back then
@@champagnesupernova1839...On what device have you watched this video on? RUclips usually has a frequency cutoff around that area. In fact, the video had no audio output at that moment for me using the RUclips app. How did you hear that?
I personally can never overstate how much I appreciate someone like you who actually does the needed research on console's audio specs and functionalities. Rather than the laughable latter half of people who bluntly assumes and mislead with "MIDI this and Soundfont that" and thinks that's the "only formats that's ever existed." Thank you so much for making this video and thank you for reading in on the topic unlike everyone else. (Everyone else who just reads the first sentence on Wikipedia.)
@@literallywhons4s if you mean VGMTrans, it *converts* to midi and dls the best it can, but the actual ds sound data is stored in a different format that's only mostly compatible
@@literallywhons4s The data is not stored as MIDI data and SF2 soundfonts internally. What you get from using such tools is simply an _attempt_ to convert the sequence and sample data into more common and more-documented formats. However, due to the differences in native capabilities between the formats, the conversion is often not completely perfect or faithful; examples of this can be inaccuracies in the volume of certain tracks or instruments, or missing DSP effects. If you're looking for for faithful playback of sequenced NDS audio, then you may want to look into the 2SF and NCSF formats instead.
A little bit of extra history/trivia -- you briefly touched on Super Smash Bros Brawl using BRSTMs. Music replacement was really big really early in SSBB's hacking days, which is why the BrawlBRSTMs RUclips channel got so big, and why their 30-minute extension format became so iconic for game music uploads on RUclips -- they made it that long so that anyone seeking to replace music in Brawl wouldn't have to futz around with figuring out the correct loop point for what they're converting to BRSTM.
That's not the reason why the videos were made that long. All of the 30-minute long videos of music posted on BrawlBRSTMs were effectively previews of pre-made BRSTMs hosted on the website "Smash Custom Music" (formerly known as "Brawl Custom Music"), a site originally ran by the same person who ran the BrawlBRSTMs channel on RUclips: Segtendo. Each song hosted on this site was made available to download as a BRSTM (in addition to several other audio formats commonly-used by games) with their loop points and other metadata already written in the file. Each video on the channel featured a link to the song's respective page on the website in their descriptions.
City folk's title screen was streamed at 44.1khz and I think the gamecube had some 48khz music in the service disc and some unused tracks in sonic mega collection
This video is amazing, dude. Love your piano covers to bits, and it's such a blessing hearing you gush about how the Wii stores its sounds. Can't wait to see more stuff like this!
I didn't expect such a kind of video coming from you (I don't mean that in a negative way), but it was very interesting to watch. I really enjoy music analysis videos so thank you for making one too and putting so much effort into it. Now, I may not be the best at understanding what all the different files or file types do, but I did learn at least a bit more about them from you explaining them. Looking forward to more stuff like this! :)
It was actually super surprising to hear that this was the first video of this kind that you've made, it was super well-written, edited, and presented, and very interesting! You definitely have a knack for this and we would love to see more!
2:50 worth noting that on mobile, sounds above like 15khz are cut off by the RUclips app/phone. So not hearing it here doesn’t necessarily mean you can’t hear that frequency.
I played the 16kHz+ example from this video on the RUclips app out of my phone speakers into my microphone, and it was able to be picked up fully up to 20kHz imgur.com/a/FQn5Aw9 Not to say there couldn't be some case where that happens.
Eu lamento lhe dizer, mas o problema não é com o app não. Eu consegui ouvir claramente a frequência. Pode ser que alguns telefones, muito dificilmente, não reproduzam, mas o app do RUclips entrega sim a reprodução.
I should clarify that this is exactly how music works on every Nintendo console that came afterwards: 3DS, Wii U and Switch. They kept reusing NintendoWare. Also, the DSP is a hardware feature (which all of the aforementioned consoles have). It has nothing to do with NintendoWare itself.
I mod Wii games, so I didn't really learn anything here, but I was very pleasantly surprised that you actually knew what you were talking about. Like, I expected an extremely broad video that was like "they used BRSTM files and sometimes they have multiple tracks lol isn't that cool" but this is genuinely a good overview of the NintendoWare audio pipeline. Wish there was more stuff on RUclips like this.
I'm only 7 minutes in and this video is mind blowingly high quality. Like these animations around the waveform and muted colors are the perfect selection for highlighting the snippets of audio as you're talking about them. Instant subscriber!
You have a very good voice for these kinds of videos. I don’t think I understood even half of what you were talking about, yet I still felt compelled to listen through in its entirety
Mario Kart Wii's title and main menu themes are found as wave format and the wifi + all the Mario Kart Channel themes are sequence music inside the game's BRSAR. There are also unused options menu instruments only, but those go unused as they decided to make it a BRSTM instead.
Leaving "Change Player List" back into the options will play the options half of "STM_TITLE_RET.brstm" from the beginning, which skips the short intro section present in "STM_TITLE.brstm" that I showed in this video. Loop points in both are also past the intro.
PCM isn't compressed. It's plain sampled amplitudes. ADPCM uses differentials rather than absolute amplitudes. Interesting that they didn't provide FLAC (the library is BSD licensed) and Ogg Vorbis. FLAC would cut audio size approximately in half. A more modern approach could include software or hardware synthesis, with FM and some analog synthesis modeling being pretty lightweight, while physical modeling is extremely computationally-intensive.
I was looking for a comment about PCM not being compression :) Those compression formats you mentioned would probably use considerably more CPU or require dedicated extra hardware, maybe that’s the reason they didn’t include them?
@@danielschenk91 Short consideration: the real-time synthesis is also computationally costly, and the below about cache considerations applies. Long consideration: Decompression is generally faster than compression, and with FLAC it's like 50-ish CPU cycles per sample, so maybe 2.5 million cycles, so maybe 0.32%. That comes alongside disc and flash access times (which create interrupts) and storage in memory. Reading across uncompressed PCM audio consumes more CPU cache and triggers more cache misses, which are expensive (they can be hundreds of CPU cycles to service). Reading across compressed audio like FLAC and decoding into a ring buffer is nicer for CPU cache and so can increase the amount of work the CPU can do in total. This is also an issue with binary trees vs. hash tables vs. arrays, and other considerations when programming where doing more computations can actually be faster because the CPU stalls less often. Be mindful, a cache line is 64 bytes, which is 2/3 of a millisecond of uncompressed audio, so with a 50% compression ratio streaming 20ms chunks of audio you're crossing either 30 (uncompressed) or 15 (compressed) cache lines-if the working memory for the decompressor stays at the same address to avoid more cache invalidation, of course. Increasing the size of the buffered audio stream doesn't increase the working memory needed for decompressing. Point is you gain some processing power from avoiding cache misses, and lose some processing power from doing decoding computations.
I love this type of video!! It reminds me of scruffy’s style of content - not in a bad way, it’s hard to find such entertaining well edited videos explaining topics like these. Much love from someone who’s watched your older videos for a pretty long time, now :)
As someone who has been obsessed with this stuff for years, this is such a good video, I love it so much, even down to the tiny tidbits like pointing out how SPM is in 22.05k (aside from a couple songs for some reason). I remember when it was super hard to rip modern sequenced nintendo music (B*SEQ) and I'd just wonder what the heck was going on with them, lol. I remember messing around with a really old command line tool to concert brseqs from the photo channel to midis and just listening to them for hours I feel like sequenced music is a lost art in a lot of modern games, the only games I can think of in recent memory are the warioware games. I think there's some indie outliers too (including maybe something I'd like to do myself in the future...) Anyway, really good video! Instant sub, glad this got recommended to me
If you like music that changes dynamically with gameplay, try looking for a game that uses FMOD. it's an adaptive music engine that mostly uses streamed audio, but it also supports sequences. If you didn't know, celeste uses FMOD, and there's even a publicly available download for celeste's FMOD project. Celeste uses both streamed and sequenced, and there's also live processing. you can hear the sequences when you're in a room that has "cassette blocks", which are pink and blue blocks that toggle their collisions to the beat. you can tell it's sequenced because it pauses for a bit when you die, for it to then start playing once it's synced to the blocks after respawning. the most notable example of processing would probably be in chapter 9, where a screen ends up being longer than you were lead to believe, and in that part of the screen where you have a bunch of anxiety inducing unexpected gameplay, a lowpass filter kicks in
I've never found video game music super interesting (as in how it works, there are soooo many good soundtracks I adore) but this is such a good video I was interested all the way through. Very nice explanation!
Would love to see a playlist dedicated to content like this! Btw, your voice is vaaaguely reminiscent of one of the narrators on How It's Made. Also, show Nintendo who really owns the intellectual property! Take that!
Thanks for the interesting video! I would've liked to hear short examples of the effects you talked about in greater depth/the ones that caused issues, but I realize that might have been a lot of work. When you did do it I felt like I fully understood the effect/problem. Hope more people see this video!
Weird that the MM9 audio cares so much about square falloff and ignores the triangle's aliasing that was on the NES. Completely changes its sound. Even more jarring when you hear the prerecorded triangle "tom drums" that DO include the aliasing.
the technical complexity behind video game music is fascinating, and that’s before we even start talking about the music theory behind it. this video is great! thanks for researching and created awesome visuals.
15:48 "The source of the drum loop is actually a well-known-" yup, it's gonna be the Amen break. Love the level of detail here. What's the song during the Closing segment?
(copied reply to similar comment) I played the 16kHz+ example from this video on the RUclips app out of my phone speakers into my microphone, and it was able to be picked up fully up to 20kHz imgur.com/a/FQn5Aw9 Not to say there couldn't be some case where that happens.
@@dublincalifCannot hear it in app with phone speakers. Still cannot hear it on app with headphones. CAN hear it on phone speakers on phone's browser.
@@dublincalifBeen doing the last 3 weeks checking out my music playlist. The audible audio quality loss between phone app and phone browser is INSANE. Why is app bad?
@@DarkonFullPower Maybe it's the video quality? The audio quality scales with video quality. It's very noticeable if you drop to like 144p. Edit: It doesn't seem to be that, though the audio quality is noticeably worse in 144p.
This was an incredibly interesting video. Got me in the mood to give ripping Wii Fit Plus songs another shot. Also i find that your voice is perfect for this kind of content.
HOLY CRAP!! WAIT!! I watched this whole video and didnt realize it was YOU until the end!! Amazing vid, man! If you enjoy it, keep it up! Huge props :)) hope you're doing well
the wii (and gamecube for that matter) could have supported 48 khz if a dev had really really wanted to, the audio DAC itself is 48 khz, it's just that both of the audio microcodes nintendo released/used (the AX microcode provided to 3rd parties and the zelda microcode used internally) only operated at 32 khz for processed audio. (streaming audio was natively 48 khz, and was mixed into the rest of the game's audio after the audio processing was finished, but due to its limitations it could only really be used in cutscenes). there were a couple of third party titles where the devs wrote their own audio microcode from scratch and got 48 khz for everything, but they weren't really significant.
I really enjoyed this. I am a big sequenced music enjoyer. the one thing I noticed was that at 2:35 you say "[x] thousand kilohertz" twice, where I think actually you're referring to 32kHz or 32,000Hz. 32,000kHz would be 32MHz lol. I would not have known this is your first video like this though I absolutely loved it. thanks for sharing
Oh rhis is absolutely fantastic! I thought this was going to be super basic and dismissive like "did you know the Wii used MIDI?!!!" but it's awesome seeing specifics as well as actual examples of the dev kits and tools they used. The editing is also super sharp!
Growing up, I always felt the music in Mega Man 9’s Wii port sounded slightly different than the 360/PS3/Official Soundtrack versions. I was able to kind of hear it in Jewel Man’s Stage, or Dr. Wily’s Stage 2 when you do a side by side comparison. Another game that I think also sounds slightly different than it’s high quality soundtrack version is Sonic 4 on the Wii. I’m happy to see that this explanation is most likely why this happens!
That was very educational, thank you. You reference songs all throughout the video though that I wish you had showed examples of. Like when talking about sequence music you mention the live orchestra in galaxy. That would have been neat to hear the example you were referencing.
I loved this. I am begging you to do a video on the DS's audio. Its music and soundscape feels so unique and distinct from everything else as this interesting compromise between modern capabilities and old qualities. Maybe I'm just nostalgic, though.
Great video, I enjoyed learning more of the science behind BRSTMs, as I used to convert songs to that format for Smash Bros Brawl / Project M back in the day for fun. Also, I love that you sampled the soundtrack from Fluidity in the background, I adore that game.
I got recommended your latest video, and then I found this one. Peak content, very informative, love that you tread into this much detail. Also, sequenced music my beloved.
Sequenced music for games is so good, I’ve been working on a game and I’ve been making the soundtrack from sequenced music because it’s able to respond to what’s going on in-game at a finer detail than simply having the option to mix between channels, for instance looping is nice but with sequenced music you can convert a song into a Markov chain and have it loop in a weighted random path and these paths can have connections to play a bridging piece of music to a different song seamlessly - those can be random, or controlled random (for instance you can require a song play for at least 2 minutes and after then it can leave the song, and if it’s been over 5 minutes it will take the next exit), or completely controlled (such as a main menu theme which has bridges to all songs in every level as soon as you select a level the music is told that it needs to get to a certain song). You can also tweak the tempo and the layers to fit different scenarios, when you pause the game I’ve currently got it set to mute some channels and slow it down (and also the music will prefer to stay on the current song for longer, although give it 15 minutes paused and it will move on; with just the backing track it’s not annoying) No doubt future games I’ll make will have sequenced music too because being able to tie the music close to the game like other assets feels essential and gamers can feel the extra effort put in
2:49 I felt like I was hearing something but wasn't entirely sure because of how we don't have actual silence in the video that is temporally near that sound clip to immediately compare to. But upon further inspection, yes I certainly did hear something similar to a dentist drill.
It's actually streamed audio, just layered on top of each other. The files are all wav files located in the music folders - no sequence files in the game files (not even the compressed pak files). Just well timed scripts firing.
WAIT A MINUTE your that guy. i ditn even recognise you because usually ur style of video is different. havn't seen ur channel in a really long time. im glad ur uploading these now, ima subscribe :)
I think its a shame that the modern age of terabyte harddrives has resulted in games no longer bothering to optimise things like music, especially when it actually lets you do super cool stuff you just don't see anymore, like in monkey island where the tracks seamlessly intertwine as you go from room to room
You can still do things like that. The game Balatro, an indie game which released earlier this year, has one song play throughout which cycles through different versions with different instruments. It’s more a case of people not bothering to implement cool transitions in music than a case of being unable to do so
Plenty, I would say almost all modern games have fully dynamic and interactive music systems that go far beyond simply transitioning the music, I encourage you to pay attention more when you play.
You can and do get smooth audio transitions in modern games, it isn't some lost technique. The modern Doom games handle this when transitioning in and out of combat.
Yeah, all the storage space and memory capacity improvements have made optimization a lost art. Piracy issues aside, it's why I love the repacking community's work, proving that games can be cut down to fractions of their installation size without any real compromise -- often even improving performance.
I'm a video game audio engineer myself, and this was a great video. One thing I will say is that modern audio middleware allows for full dynamic and interactive systems- the benefit of sequencers has diminished with the exception of file size and player generated content
2:50 this demonstration doesn't work on YT because YT already cuts off frequencies past 15 kHz it's quite literally impossible to hear anything because of that I can hear 16-18 kHz relatively well (unless my hearing has degraded significantly in the last 2-3 years), yet I have to max out my headphones to hear anything for it, which I blame on the filtering not being 100% on youtube's end edit: yep I just confirmed using Audacity to get the raw audio, I can hear up to 18 kHz decently. After that the sound volume gets too quiet for me to hear without blasting my headphone volume up by an unreasonable amount. I can kinda hear 20 kHz, but honestly it feels more like pressure than actual audio
RUclips videos with audio encoded in OPUS (such as this one) support up to 20kHz. I have recorded output of this RUclips video from multiple browsers and devices on all quality settings, and content up to 20kHz was present on every test I have made. If you're curious, you could capture the loopback of your output device playing that example in audacity and check it in a spectrum, like this i.imgur.com/dVZh9oD.png
@@dublincalif huh I think opus is a newer format for RUclips My info is from a few years ago, and I don't remember RUclips providing data in an opus format previously when checking the available audio codecs
23:00 This is actually what has happened, "streamed" music is so much more complex nowadays it goes even beyond what sequenced music could do, dynamic music is crazy
i was expecting to see atleast 200k views on this, but when i saw 30k i was shocked! this video is well thought and explained, and the visuals are wonderful. thanks so much for this, i learned a lot and it held my attention!
not only is this something I'd want to watch, I had a lot of trouble finding something that actually pulled my interest for hours today till I ran across your video here. I miss Trackers era music, and to this day I still go through and listen to the Unreal Gold soundtrack because there's just a certain feel to having the entertainment system itself be an instrument that plays over the game itself. What other games would you say have some of the best soundtracks using this same methodology?
This is very interesting to me as an EDM producer, i have hardware that could simulate these sequences and effects and im gonna try and make a drum and bass track with this knowledge. Thanks for this great insight into how the processing of sound used to work. Im gonna hunt for old tech to use now
hi, I'm French, so sorry if my English is not perfect. Could you please make a tutorial for the Wii u mii editor (a video for the editor, and if possible, a video for the editor menu), thank you very much if you do ! (I love this new type of video)
awesome video production quality! very comprehensive and the A/B demos are much appreciated. dunno if you have interest in PC games, but maxis game Spore has a pretty cool music engine. could be an interesting video topic. like when designing creatures/vehicles, it's tied into the GUI so you often hear changeups in the music even just by hovering the mouse over various options.
2:41 wait, i thought that for pcm audio the sample rate was just the amount of samples of the audio signal that were being taken per second, how does that affect the range of audio frequencies that get encoded? Is it because the timing with which the samples are taken aligns with the audio frequency or something?
2:47 I heard both, on phone, S21U, it was an interesting topic to research on a while back when I first knew about it, some people can't hear different sound frequencies based on a lot of factors, I believe younger people can hear more with a maximum of about 20khz
Great video and very interesting! I am working at House of Elias (previously known as Elias Software) and we are working hard on bringing midi back for video games and make it easy to use with high quality samples. The sampler and instrument libraries can sound really good today, so the possibilities will be endless. :)
Loved this video! You explained this stuff very thoroughly, and I found a program that can convert BRSARs to MIDI, and even extracts the soundfonts. It's so fun to play around with :D
2:37 I tested this with a tone generator, so I know I can hear 18,000 hertz just fine. But the the audio is silent when he plays the high frequency sound. It might have been compressed out of existence.
What were you playing back the video with? Every device I've tested so far outputs this example out of RUclips up to 20kHz exactly, but I'd be curious if something's cutting it shorter for some people.
That sequenced form sounds a lot like a tracker module, except split up into separate files for pattern and sample data. Tracker modules put both in the one file. I listen to a lot of music in various module formats, mostly composed by the Demoscene, though with a few game soundtracks thrown in as well, as they did get a bit of use in PC games. Examples include Jazz Jackrabbit, Epic Pinball, Stargunner, Terminal Velocity, Unreal 1 and Unreal Tournament, Deus Ex, and Ion Fury. If you want to find original ones, try The Mod Archive. They've even got an in-browser player.
The lack of playing examples of the various tracks and instead of talking about them is a bit frustrating… eg. The Mewtwo theme is your favorite, but you didn’t play any of it 😢 similarly it would have been nice to _hear_ the bowling theme with the ‘speaker in bowling alley’ effects, as captured from the game itself. No doubt this was to mitigate the risk of copyright strikes, but it would have been really helpful for context.
Funny to see my AC: CF playlist in a youtube movie ^_^
As for your remarks about zeroing all sound effects and record without sounds: this was the case for gamecube games where only one *.aw contained the instrument sounds. For brsar all sounds were played by rseq. Often they were rseq files with one note playing. Rseq was not reverse engineered yet, but changing all notes data to value 255 stopped the sounds. rseq files contain labels, so i could easily find out which one contained sound effects. Animal Crossing contained more than 10000 rseq files so i had to write a script for modifying the brsar to only play songs back then
Wow, surprised to see you're still around
You're the goat, I've downloaded so many CF songs from your channel, so ty!!
Been listening to Super KK Slider soundtrack for 15 years now... wild...
ur vids made my childhood. hope ur doing well
I LIVE for sequenced songs, I don't know why they're so fascinating to me. Thank you for making this video!
They're basically tracker modules, except split up. The Mod Archive has thousands of original ones.
Have you seen Ahoy's video on tracker music? If not, I think you'd like it.
@@TheYoshieMaster Oh, yeah. That's a good one. I'll second the recommendation!
"...losing this range of frequencies:" *[SCREAMING BUG SOUND]*
oh no, my hearing really is bad isn’t it?
@@shan8130 could also just be low quality speaker systems
@@champagnesupernova1839 Yeah! Yeah... Let's uh, let's go with that.
@@champagnesupernova1839...On what device have you watched this video on? RUclips usually has a frequency cutoff around that area. In fact, the video had no audio output at that moment for me using the RUclips app. How did you hear that?
@@champagnesupernova1839 yeag im not surprised my phone cant do *usually* out of range frequencies
I personally can never overstate how much I appreciate someone like you who actually does the needed research on console's audio specs and functionalities. Rather than the laughable latter half of people who bluntly assumes and mislead with "MIDI this and Soundfont that" and thinks that's the "only formats that's ever existed."
Thank you so much for making this video and thank you for reading in on the topic unlike everyone else.
(Everyone else who just reads the first sentence on Wikipedia.)
I get more annoyed than I should at people saying theyre looking for a soundfont for a ds game as if that exists
@@literallywhons4syeah I’ve done that a few times
@@literallywhons4s if you mean VGMTrans, it *converts* to midi and dls the best it can, but the actual ds sound data is stored in a different format that's only mostly compatible
@@literallywhons4s The data is not stored as MIDI data and SF2 soundfonts internally. What you get from using such tools is simply an _attempt_ to convert the sequence and sample data into more common and more-documented formats. However, due to the differences in native capabilities between the formats, the conversion is often not completely perfect or faithful; examples of this can be inaccuracies in the volume of certain tracks or instruments, or missing DSP effects.
If you're looking for for faithful playback of sequenced NDS audio, then you may want to look into the 2SF and NCSF formats instead.
@gubmen6969 many people make soundfonts based on these game's sequence instruments so they probably only know those exist, and nothing else
Nitpick, but the frequencies aren't in *thousands* of kHz. The k means thousand already, so you'd be up in the MHz range.
Despite me being aware of this common mistake in the past, I can't seem to stop making it myself
@@dublincalif we all knew what you meant. I just want to look smart on the internet.
Yeah
and also PCM isn't a compression scheme. It's just a way of storing an analog signal in a digital file
@@tehrobotjesus Me, too :/
A little bit of extra history/trivia -- you briefly touched on Super Smash Bros Brawl using BRSTMs. Music replacement was really big really early in SSBB's hacking days, which is why the BrawlBRSTMs RUclips channel got so big, and why their 30-minute extension format became so iconic for game music uploads on RUclips -- they made it that long so that anyone seeking to replace music in Brawl wouldn't have to futz around with figuring out the correct loop point for what they're converting to BRSTM.
That's not the reason why the videos were made that long.
All of the 30-minute long videos of music posted on BrawlBRSTMs were effectively previews of pre-made BRSTMs hosted on the website "Smash Custom Music" (formerly known as "Brawl Custom Music"), a site originally ran by the same person who ran the BrawlBRSTMs channel on RUclips: Segtendo.
Each song hosted on this site was made available to download as a BRSTM (in addition to several other audio formats commonly-used by games) with their loop points and other metadata already written in the file.
Each video on the channel featured a link to the song's respective page on the website in their descriptions.
Do any of you guys know the music being played at 6:35 ?
the music was also available on their website directly as a brstm with correct loops
City folk's title screen was streamed at 44.1khz and I think the gamecube had some 48khz music in the service disc and some unused tracks in sonic mega collection
super monkey ball's music is 48khz
This video is amazing, dude. Love your piano covers to bits, and it's such a blessing hearing you gush about how the Wii stores its sounds. Can't wait to see more stuff like this!
I didn't expect such a kind of video coming from you (I don't mean that in a negative way), but it was very interesting to watch. I really enjoy music analysis videos so thank you for making one too and putting so much effort into it. Now, I may not be the best at understanding what all the different files or file types do, but I did learn at least a bit more about them from you explaining them. Looking forward to more stuff like this! :)
It was actually super surprising to hear that this was the first video of this kind that you've made, it was super well-written, edited, and presented, and very interesting! You definitely have a knack for this and we would love to see more!
agree!!
2:50 worth noting that on mobile, sounds above like 15khz are cut off by the RUclips app/phone. So not hearing it here doesn’t necessarily mean you can’t hear that frequency.
I played the 16kHz+ example from this video on the RUclips app out of my phone speakers into my microphone, and it was able to be picked up fully up to 20kHz imgur.com/a/FQn5Aw9
Not to say there couldn't be some case where that happens.
I felt some pressure from my headset on my ears but didn't hear a thing
Eu lamento lhe dizer, mas o problema não é com o app não. Eu consegui ouvir claramente a frequência. Pode ser que alguns telefones, muito dificilmente, não reproduzam, mas o app do RUclips entrega sim a reprodução.
@@dublincalifThis link was deleted and may not be accessed
Dang, just last year I could still hear 17.5khz. Don’t even listen to music anymore, seems strange I can’t hear above 16khz anymore. Aging sucks.
“Are you able to hear this?”
..silence..
I can hear it hehe
@@zapowwsame
Maybe it does play something but our ears are just built differently. Gonna ask my dog if he can hear it.
I'm blaming it on my phone
It felt tingly
Oh my gosh an analysis video!? I am literally shivering in me timbers!
I should clarify that this is exactly how music works on every Nintendo console that came afterwards: 3DS, Wii U and Switch.
They kept reusing NintendoWare.
Also, the DSP is a hardware feature (which all of the aforementioned consoles have). It has nothing to do with NintendoWare itself.
Google says that the GameCube had a DSP, but Wii does not?
@@ArneChristianRosenfeldt nope. GameCube, Wii, 3DS, Wii U and Switch all have DSPs, and all of them use the same ADPCM encoding.
I mod Wii games, so I didn't really learn anything here, but I was very pleasantly surprised that you actually knew what you were talking about. Like, I expected an extremely broad video that was like "they used BRSTM files and sometimes they have multiple tracks lol isn't that cool" but this is genuinely a good overview of the NintendoWare audio pipeline. Wish there was more stuff on RUclips like this.
I'm only 7 minutes in and this video is mind blowingly high quality. Like these animations around the waveform and muted colors are the perfect selection for highlighting the snippets of audio as you're talking about them. Instant subscriber!
You have a very good voice for these kinds of videos. I don’t think I understood even half of what you were talking about, yet I still felt compelled to listen through in its entirety
This might be the best video I've seen about how video game music works. Really good job
Mario Kart Wii's title and main menu themes are found as wave format and the wifi + all the Mario Kart Channel themes are sequence music inside the game's BRSAR. There are also unused options menu instruments only, but those go unused as they decided to make it a BRSTM instead.
The option instruments aren’t unused, the Options menu in the Mario Kart channel uses them.
@@JohnP55 They are unused, MKC uses different ones.
@@PlayersPurityYes, it’s a different BRSAR, but they’re the same samples.
The beginning of the Wii sports resort options menu can be heard if you go into the player change list and leave or wait long enough for it to loop.
Leaving "Change Player List" back into the options will play the options half of "STM_TITLE_RET.brstm" from the beginning, which skips the short intro section present in "STM_TITLE.brstm" that I showed in this video. Loop points in both are also past the intro.
@@dublincalif I may have misremembered it ._.
PCM isn't compressed. It's plain sampled amplitudes. ADPCM uses differentials rather than absolute amplitudes. Interesting that they didn't provide FLAC (the library is BSD licensed) and Ogg Vorbis. FLAC would cut audio size approximately in half. A more modern approach could include software or hardware synthesis, with FM and some analog synthesis modeling being pretty lightweight, while physical modeling is extremely computationally-intensive.
I was looking for a comment about PCM not being compression :)
Those compression formats you mentioned would probably use considerably more CPU or require dedicated extra hardware, maybe that’s the reason they didn’t include them?
@@danielschenk91 Short consideration: the real-time synthesis is also computationally costly, and the below about cache considerations applies.
Long consideration: Decompression is generally faster than compression, and with FLAC it's like 50-ish CPU cycles per sample, so maybe 2.5 million cycles, so maybe 0.32%. That comes alongside disc and flash access times (which create interrupts) and storage in memory. Reading across uncompressed PCM audio consumes more CPU cache and triggers more cache misses, which are expensive (they can be hundreds of CPU cycles to service). Reading across compressed audio like FLAC and decoding into a ring buffer is nicer for CPU cache and so can increase the amount of work the CPU can do in total. This is also an issue with binary trees vs. hash tables vs. arrays, and other considerations when programming where doing more computations can actually be faster because the CPU stalls less often.
Be mindful, a cache line is 64 bytes, which is 2/3 of a millisecond of uncompressed audio, so with a 50% compression ratio streaming 20ms chunks of audio you're crossing either 30 (uncompressed) or 15 (compressed) cache lines-if the working memory for the decompressor stays at the same address to avoid more cache invalidation, of course. Increasing the size of the buffered audio stream doesn't increase the working memory needed for decompressing. Point is you gain some processing power from avoiding cache misses, and lose some processing power from doing decoding computations.
@@johnmoser3594 Very interesting stuff!
You did such a great work ! Please make other videos like that !
I love this type of video!! It reminds me of scruffy’s style of content - not in a bad way, it’s hard to find such entertaining well edited videos explaining topics like these. Much love from someone who’s watched your older videos for a pretty long time, now :)
please do more of these. i LOVE deep dives in obscure/rarely talked about technologies
As someone who has been obsessed with this stuff for years, this is such a good video, I love it so much, even down to the tiny tidbits like pointing out how SPM is in 22.05k (aside from a couple songs for some reason). I remember when it was super hard to rip modern sequenced nintendo music (B*SEQ) and I'd just wonder what the heck was going on with them, lol. I remember messing around with a really old command line tool to concert brseqs from the photo channel to midis and just listening to them for hours
I feel like sequenced music is a lost art in a lot of modern games, the only games I can think of in recent memory are the warioware games. I think there's some indie outliers too (including maybe something I'd like to do myself in the future...)
Anyway, really good video! Instant sub, glad this got recommended to me
I did NOT expect siivagunner to show up here
Now im interested, time to make my own high quality rips
If you like music that changes dynamically with gameplay, try looking for a game that uses FMOD. it's an adaptive music engine that mostly uses streamed audio, but it also supports sequences. If you didn't know, celeste uses FMOD, and there's even a publicly available download for celeste's FMOD project.
Celeste uses both streamed and sequenced, and there's also live processing. you can hear the sequences when you're in a room that has "cassette blocks", which are pink and blue blocks that toggle their collisions to the beat. you can tell it's sequenced because it pauses for a bit when you die, for it to then start playing once it's synced to the blocks after respawning. the most notable example of processing would probably be in chapter 9, where a screen ends up being longer than you were lead to believe, and in that part of the screen where you have a bunch of anxiety inducing unexpected gameplay, a lowpass filter kicks in
I've never found video game music super interesting (as in how it works, there are soooo many good soundtracks I adore) but this is such a good video I was interested all the way through. Very nice explanation!
Would love to see a playlist dedicated to content like this!
Btw, your voice is vaaaguely reminiscent of one of the narrators on How It's Made. Also, show Nintendo who really owns the intellectual property! Take that!
Thanks for the interesting video! I would've liked to hear short examples of the effects you talked about in greater depth/the ones that caused issues, but I realize that might have been a lot of work. When you did do it I felt like I fully understood the effect/problem. Hope more people see this video!
Weird that the MM9 audio cares so much about square falloff and ignores the triangle's aliasing that was on the NES. Completely changes its sound. Even more jarring when you hear the prerecorded triangle "tom drums" that DO include the aliasing.
This is the kind of content I dream about! I’ve never seen your other videos but I love technical videos about games!
man i've been watching you for years now, and this new style of content is great!! love it. keep it up
Insanely well crafted video. Keep up the great work!
First video? It's such a good researched and explained video. I enjoyed it a lot.
Wow I really enjoyed this video! I'm happy to see you branch out your content!!
I cannot believe how in depth this video was...way more information than I ever thought I'd learn about this 😂 great job!
This was amazing. Thank you for taking the time to go so in depth with everything.
the technical complexity behind video game music is fascinating, and that’s before we even start talking about the music theory behind it.
this video is great! thanks for researching and created awesome visuals.
Ok wow this is a well researched video
I would literally click back & forth the different apps just to listen to their music. That ethereal abstract spacey vibe was so good
Your explanation of Streamed music just helped me shave off hundreds of megabytes of memory use on my own game. Thanks!
15:48 "The source of the drum loop is actually a well-known-" yup, it's gonna be the Amen break.
Love the level of detail here. What's the song during the Closing segment?
2:47 Note for RUclips mobile users. Mobile caps sound at 15-16khz, so anything above that cannot be heard. Play the video on PC to hear it.
(copied reply to similar comment) I played the 16kHz+ example from this video on the RUclips app out of my phone speakers into my microphone, and it was able to be picked up fully up to 20kHz imgur.com/a/FQn5Aw9
Not to say there couldn't be some case where that happens.
I can hear it clearly on Android.
@@dublincalifCannot hear it in app with phone speakers. Still cannot hear it on app with headphones. CAN hear it on phone speakers on phone's browser.
@@dublincalifBeen doing the last 3 weeks checking out my music playlist. The audible audio quality loss between phone app and phone browser is INSANE. Why is app bad?
@@DarkonFullPower Maybe it's the video quality? The audio quality scales with video quality. It's very noticeable if you drop to like 144p.
Edit: It doesn't seem to be that, though the audio quality is noticeably worse in 144p.
I'm so glad someone finally dove into this topic. I'm curious how other past consoles produced their music as well now
This video was fire dude please don't stop making content like this
This was an incredibly interesting video. Got me in the mood to give ripping Wii Fit Plus songs another shot. Also i find that your voice is perfect for this kind of content.
Don't know if I'll ever use this information, but I like hearing how things work. Great video.
9:28 saving my spot ignore this
HOLY CRAP!! WAIT!! I watched this whole video and didnt realize it was YOU until the end!! Amazing vid, man! If you enjoy it, keep it up! Huge props :)) hope you're doing well
the wii (and gamecube for that matter) could have supported 48 khz if a dev had really really wanted to, the audio DAC itself is 48 khz, it's just that both of the audio microcodes nintendo released/used (the AX microcode provided to 3rd parties and the zelda microcode used internally) only operated at 32 khz for processed audio. (streaming audio was natively 48 khz, and was mixed into the rest of the game's audio after the audio processing was finished, but due to its limitations it could only really be used in cutscenes). there were a couple of third party titles where the devs wrote their own audio microcode from scratch and got 48 khz for everything, but they weren't really significant.
Amazing content bro! Im a huge music guy, and this was awsome. Ive been messing with soundphonts and its so fun!
such a great video! Specially loved the technical details and the quality of the speaking, keep going!
I really enjoyed this. I am a big sequenced music enjoyer. the one thing I noticed was that at 2:35 you say "[x] thousand kilohertz" twice, where I think actually you're referring to 32kHz or 32,000Hz. 32,000kHz would be 32MHz lol.
I would not have known this is your first video like this though I absolutely loved it. thanks for sharing
Very fascinating breakdown of the Wii's aural interior! Keep up the good work, mate.
Oh rhis is absolutely fantastic! I thought this was going to be super basic and dismissive like "did you know the Wii used MIDI?!!!" but it's awesome seeing specifics as well as actual examples of the dev kits and tools they used. The editing is also super sharp!
my favorite nintendo-songs-with-a-piano-rendition youtuber made a video essay on something nerdy let's go
Do any of you guys know the music being played at 6:35 ?
Growing up, I always felt the music in Mega Man 9’s Wii port sounded slightly different than the 360/PS3/Official Soundtrack versions.
I was able to kind of hear it in Jewel Man’s Stage, or Dr. Wily’s Stage 2 when you do a side by side comparison.
Another game that I think also sounds slightly different than it’s high quality soundtrack version is Sonic 4 on the Wii.
I’m happy to see that this explanation is most likely why this happens!
I may have also played around with Wii sound files and seeing you talk about it just makes me happy, good luck to you ✌️
That was very educational, thank you. You reference songs all throughout the video though that I wish you had showed examples of. Like when talking about sequence music you mention the live orchestra in galaxy. That would have been neat to hear the example you were referencing.
I loved this. I am begging you to do a video on the DS's audio. Its music and soundscape feels so unique and distinct from everything else as this interesting compromise between modern capabilities and old qualities. Maybe I'm just nostalgic, though.
Great video, I enjoyed learning more of the science behind BRSTMs, as I used to convert songs to that format for Smash Bros Brawl / Project M back in the day for fun. Also, I love that you sampled the soundtrack from Fluidity in the background, I adore that game.
I got recommended your latest video, and then I found this one. Peak content, very informative, love that you tread into this much detail.
Also, sequenced music my beloved.
Great video, mabye you'll tackle the music of the DS ?
You should totally do a video on portal’s music! I know it’s not Nintendo but the way it handles its music is fascinating
Yo. This was great man. Would love to see more like it. Your approach is great
Sequenced music for games is so good, I’ve been working on a game and I’ve been making the soundtrack from sequenced music because it’s able to respond to what’s going on in-game at a finer detail than simply having the option to mix between channels, for instance looping is nice but with sequenced music you can convert a song into a Markov chain and have it loop in a weighted random path and these paths can have connections to play a bridging piece of music to a different song seamlessly - those can be random, or controlled random (for instance you can require a song play for at least 2 minutes and after then it can leave the song, and if it’s been over 5 minutes it will take the next exit), or completely controlled (such as a main menu theme which has bridges to all songs in every level as soon as you select a level the music is told that it needs to get to a certain song). You can also tweak the tempo and the layers to fit different scenarios, when you pause the game I’ve currently got it set to mute some channels and slow it down (and also the music will prefer to stay on the current song for longer, although give it 15 minutes paused and it will move on; with just the backing track it’s not annoying)
No doubt future games I’ll make will have sequenced music too because being able to tie the music close to the game like other assets feels essential and gamers can feel the extra effort put in
2:49 I felt like I was hearing something but wasn't entirely sure because of how we don't have actual silence in the video that is temporally near that sound clip to immediately compare to. But upon further inspection, yes I certainly did hear something similar to a dentist drill.
Left 4 Dead and Left 4 Dead 2 also use sequenced music to add and remove instruments during Hoarde events!
It's actually streamed audio, just layered on top of each other.
The files are all wav files located in the music folders - no sequence files in the game files (not even the compressed pak files). Just well timed scripts firing.
Great video. Appreciate the technical knowledge share, even if some of the details were a little off, I hugely appreciated this!
WAIT A MINUTE your that guy. i ditn even recognise you because usually ur style of video is different. havn't seen ur channel in a really long time. im glad ur uploading these now, ima subscribe :)
Genuinely fantastic video, I would love to see more like this.
I think its a shame that the modern age of terabyte harddrives has resulted in games no longer bothering to optimise things like music, especially when it actually lets you do super cool stuff you just don't see anymore, like in monkey island where the tracks seamlessly intertwine as you go from room to room
You can still do things like that. The game Balatro, an indie game which released earlier this year, has one song play throughout which cycles through different versions with different instruments. It’s more a case of people not bothering to implement cool transitions in music than a case of being unable to do so
Plenty, I would say almost all modern games have fully dynamic and interactive music systems that go far beyond simply transitioning the music, I encourage you to pay attention more when you play.
You can and do get smooth audio transitions in modern games, it isn't some lost technique. The modern Doom games handle this when transitioning in and out of combat.
Yeah, all the storage space and memory capacity improvements have made optimization a lost art. Piracy issues aside, it's why I love the repacking community's work, proving that games can be cut down to fractions of their installation size without any real compromise -- often even improving performance.
Shush, the Devs put gay people into the game/j
I'm a video game audio engineer myself, and this was a great video. One thing I will say is that modern audio middleware allows for full dynamic and interactive systems- the benefit of sequencers has diminished with the exception of file size and player generated content
Thank you for making a great video on such an underrated subject!
22:51 many years ago! I still do😂
this is super great! I'd love to see more stuff like this, keep up the good work!
The amount of care put into the Wii's OS and interface was unprecedented.
2:50
this demonstration doesn't work on YT because YT already cuts off frequencies past 15 kHz
it's quite literally impossible to hear anything because of that
I can hear 16-18 kHz relatively well (unless my hearing has degraded significantly in the last 2-3 years), yet I have to max out my headphones to hear anything for it, which I blame on the filtering not being 100% on youtube's end
edit: yep I just confirmed using Audacity to get the raw audio, I can hear up to 18 kHz decently. After that the sound volume gets too quiet for me to hear without blasting my headphone volume up by an unreasonable amount. I can kinda hear 20 kHz, but honestly it feels more like pressure than actual audio
RUclips videos with audio encoded in OPUS (such as this one) support up to 20kHz. I have recorded output of this RUclips video from multiple browsers and devices on all quality settings, and content up to 20kHz was present on every test I have made. If you're curious, you could capture the loopback of your output device playing that example in audacity and check it in a spectrum, like this i.imgur.com/dVZh9oD.png
@@dublincalif huh
I think opus is a newer format for RUclips
My info is from a few years ago, and I don't remember RUclips providing data in an opus format previously when checking the available audio codecs
23:00 This is actually what has happened, "streamed" music is so much more complex nowadays it goes even beyond what sequenced music could do, dynamic music is crazy
i was expecting to see atleast 200k views on this, but when i saw 30k i was shocked! this video is well thought and explained, and the visuals are wonderful. thanks so much for this, i learned a lot and it held my attention!
not only is this something I'd want to watch, I had a lot of trouble finding something that actually pulled my interest for hours today till I ran across your video here.
I miss Trackers era music, and to this day I still go through and listen to the Unreal Gold soundtrack because there's just a certain feel to having the entertainment system itself be an instrument that plays over the game itself.
What other games would you say have some of the best soundtracks using this same methodology?
This was very insightful, thanks for uploading!
I didn't know who you were, but I like technical video game related videos so am glad the algorithm sent this my way
This is very interesting to me as an EDM producer, i have hardware that could simulate these sequences and effects and im gonna try and make a drum and bass track with this knowledge. Thanks for this great insight into how the processing of sound used to work. Im gonna hunt for old tech to use now
21:45 What is the name of the song playing here? I think it's some cover of the Air: Cottage 1 theme from Wii Sports Resort, but who's it by?
2:54 I couldn't hear that for I had my volume off.
hi, I'm French, so sorry if my English is not perfect. Could you please make a tutorial for the Wii u mii editor (a video for the editor, and if possible, a video for the editor menu), thank you very much if you do !
(I love this new type of video)
awesome video production quality! very comprehensive and the A/B demos are much appreciated.
dunno if you have interest in PC games, but maxis game Spore has a pretty cool music engine. could be an interesting video topic. like when designing creatures/vehicles, it's tied into the GUI so you often hear changeups in the music even just by hovering the mouse over various options.
Amazing work, please do more of these!
11:27 For anyone not aware, the reason for the odd pronunciation is that the "l" is actually a capitalized i.
That was fascinating, would love to watch more
2:41 wait, i thought that for pcm audio the sample rate was just the amount of samples of the audio signal that were being taken per second, how does that affect the range of audio frequencies that get encoded? Is it because the timing with which the samples are taken aligns with the audio frequency or something?
2:47 I heard both, on phone, S21U, it was an interesting topic to research on a while back when I first knew about it, some people can't hear different sound frequencies based on a lot of factors, I believe younger people can hear more with a maximum of about 20khz
Great video and very interesting!
I am working at House of Elias (previously known as Elias Software) and we are working hard on bringing midi back for video games and make it easy to use with high quality samples. The sampler and instrument libraries can sound really good today, so the possibilities will be endless. :)
Loved this video! You explained this stuff very thoroughly, and I found a program that can convert BRSARs to MIDI, and even extracts the soundfonts. It's so fun to play around with :D
This was fascinating af
2:37 I tested this with a tone generator, so I know I can hear 18,000 hertz just fine. But the the audio is silent when he plays the high frequency sound. It might have been compressed out of existence.
What were you playing back the video with? Every device I've tested so far outputs this example out of RUclips up to 20kHz exactly, but I'd be curious if something's cutting it shorter for some people.
Same here. I know I hear past 20k, but youtube never works past 16k for me. (Iphone, gchrome on Windows)
That sequenced form sounds a lot like a tracker module, except split up into separate files for pattern and sample data. Tracker modules put both in the one file.
I listen to a lot of music in various module formats, mostly composed by the Demoscene, though with a few game soundtracks thrown in as well, as they did get a bit of use in PC games. Examples include Jazz Jackrabbit, Epic Pinball, Stargunner, Terminal Velocity, Unreal 1 and Unreal Tournament, Deus Ex, and Ion Fury. If you want to find original ones, try The Mod Archive. They've even got an in-browser player.
The lack of playing examples of the various tracks and instead of talking about them is a bit frustrating… eg. The Mewtwo theme is your favorite, but you didn’t play any of it 😢 similarly it would have been nice to _hear_ the bowling theme with the ‘speaker in bowling alley’ effects, as captured from the game itself.
No doubt this was to mitigate the risk of copyright strikes, but it would have been really helpful for context.