Great video! That's a lot of work to collect all that data. I wanted to confirm if your data actually looks random (since it's hard to tell, as a human) so I ran some code to generate 346 sets of 3 unique tracks (to simulate 346 track selections as you did), and then ran this simulation over and over to see what happened. If the track selection is properly random, then the most frequent track would usually show up 18-22 times. The least frequent is usually 3-5 times. Everything else is distributed how you'd expect, with 10 and 11 times being the most common frequencies. So it looks like the randomness is correct in-game. I simulated the test from reddit 2018, and I got very different results from that one. It's very rare for a track to show up 0 times. So if that test was accurate, then the online track selection was not very random back then.
Wow, thank you! Glad to know our test is backed up by a simulation as well. 😅Yeah I'm surprised it seems to be fixed. Now the Wii U version on the other hand may be a different story, though the online for that is going off soon...
@@rye9270I do have too much time, but this one was quick! I asked chatgpt to code it, then I just ran the simulation a few times. That all took 2 minutes. Then a few more minutes to write the comment
i remember that itunes had to change how their shuffle works because it being truly random made people feel like they were playing the same tracks over and over, so we can definitely trick ourselves into thinking there is a pattern when there is none.
Banger video! And honestly, time of day isn't really off the table. Splatoon is very clear that it only has some maps online at a time, and while this was so long in the past, Mario Kart DS only had certain tracks online, showing again there's only a limited selection up.
@@OfficialGlitchDoctor3:31 if you dont believe in god why do you have to swear to one for those of us that do? swearing to god is breaking the 3rd commandment
As someone who's played long enough to accrue 65k VR I'm honestly shocked to find out that it's legitimately random. I would have sworn that certain tracks were exponentially more likely to show up, but I guess it turns out that I'm just more likely to notice when the game offers the tracks that I've previously suspected to be "more likely". It must be some sort of mental bias on my part after all! Great work man.
The results in the data have a p value of 0.2814 under the chi squared test, thus the data does not significantly suggest that the course selection is not random
I'd say that the tracks that repeat the most and actually stand out to us are either overwhelmingly picked due to popularity like Wii Rainbow Road(or for some reason everyone accidentally clicked Water Park) or the tracks that you just hate playing on. It also doesn't help if those 2 categories overlap because I definitely hate some tracks that are popular(not because they're popular, I just don't like their visual and/or track design). Also, banger video.
This is what I was thinking. I personally really dislike dry dry desert so I would definitely feel like it wasn't as random as it should be if I got it too often.
Regarding when the course gets picked after the vote, I will never forget one time when 11 of us picked the same track, and the 1 player picked a different track, and the game picked the 1 outlier. I was livid
I think what makes it feel that way for me is that the tracks that show up more often tend to be ones I don’t like (electrodrome, dry dry desert, donut plains 3)
My theory is that although it is random, there are only a select 32 or so tracks that are always voted for because of popularity. Therefore, those tracks are played more often, tricking us into thinking the selection isn’t random.
this could be another case of people tricking themselves into believing something isn't random when it really is, but i feel like it's way more common (like 1/6 or something instead of 1/12) for my track to be chosen when it's the first race i've played that session, as opposed to the 1/12 chance it should be.
What a cool study! Although I found the result unsurprising, this video does an excellent job showcasing the way that pattern recognition in the human brain can sometimes misfire. We've all had those lobbies where we can swear we've played the same track so many times that we can swear that it feels like John Nintendo himself is willing it unto us. In reality, understanding true randomness is understanding that sometimes, weird luck is sometimes all it takes to encounter repetition. And with 96 whole tracks in the game, a few of them are BOUND to become an outlier
i love this! i recently started keeping track in a notebook too because i felt like there were some tracks i had almost forgotten. my data is very rough but i'm glad to see someone else was doing this!
I honestly think that everyday, random tracks have higher chances of appearing, thus causing the problem you mentioned at the beginning. So it is random, but the pool of tracks and their chances to appear constantly changes.
The tracks I very frequently get: Cheep Cheep Beach GBA Mario Circuit Amsterdam Drift SNES Rainbow Road DK Jungle Dragon Driftway will edit if I think of more
That Mount Wario joke was as gold as the stuff he'd pay you. Good work on the video, although I kinda want to see a whole statistical breakdown (or see the data and do it myself).
When playinhg local multiplayer, I find that the tracks that we like to play often become really boring after a while, so I would guess Nintendo would probably try to make people pick more varied courses to not get tired of any specific ones. It would make sense to push the less popular courses more so that the popular courses don't get played so much that people get bored of them.
It’s not generic, I actually think it’s very unique. However, it is very annoying to play. You need to brake drift basically every turn, and don’t get me started on 200cc
This research has a big flaw. Assuming that by at some point, you should see all of the courses. From the sound of it, you assume that the more you play, the more likely you'll run into a specific track. In reality tho, it doesn't increase your chances. Assuming there isn't an algorithm and it is conpletely random, your chances wont change. It'll be the same throughout
Nintendo doesn’t want us to repeatedly pick tracks in MK8 (like in MK7 & MKWii where you can pick any track). However, once a race has played on a specific track (ex. Mount Wario) it can’t be shown again on the next race until after at least 1 race on a different track (ex. N64 Royal Raceway after the Mount Wario race) has been played.
I did a Q-Q plot of the data against a Poisson distribution with a rate of 10.8125 (how many times each course should show up on average over 1038 selections) and it can modeled really nicely with the line y=x which means it's a good fit, so the data is indeed randomly distributed.
What you mentioned about Mirror and 200cc showing up only once is strange. Usually when I play online, I tend to only get Mirror Mode when I play online
I know this is a far reach, but for who interests and has the knowledge, the May 2014 USA kiosk disc demo for Mario Kart 8 shipped with debug symbols (not sure if stripped or non-stripped ELF, most games accidentally ship with stripped, so safe to assume Nintendo EAD did too). I don't think the variables used are noted, so the process to know if the random generator is actually random is gonna take weeks to discover, anyways the functions for it are called "UpdateMenuDecide" and "UpdateRoulette", since they are present in Wireless Play/LAN Play too. Outside nerd stuff, for me it seems like the selection of courses seems to be an internal table where courses are ranked like ELO or another system like that, I don't think the courses from the DLC get a higher score because they're just DLC courses, you just had a lil bit of luck getting that unequal percentage, I do believe though that from the random pool of courses it chooses at first, the ones that are not selected just get a higher and higher rating and that's why they don't go away. For the screen after that when it picks the 10 random courses, yeah that's 100% random, no string attached, it's just a roll of the dice.
The thing about randomness is that, seemingly counterintuitively, you'd expect long runs of the same option every so often. What you need to look for is not that there *are* some courses favored, but whether or not a certain amount of favoritism being *likely*
@GlitchDoctor: They are a few problem with does test, if you stay in the exact same room for 346 tracks, without re-setting the room, or being force into another room. Since it would be imposible to do this because you can't stay awake for over 48 hours, and don't ruin your health just to test this. Unless you used an A.I. on emulator to play 365 random race, but if all people leave that mean it will switch group and you don't know where the random begin from when you auto switch group.
This kind of thing is exactly what MatPat was talking about when he said it's _"our_ turn to be the Theorists". You had a hypothesis, you gathered data, you got a result. Science! Also the only way to control for the "selection bias" of which track gets _played,_ would be to have a private lobby of 12 players all "in on the experiment", three sets of four people each picking a specific slot every time (first/second/third) to give equal odds of selection, maybe also writing down which position the selection cursor starts and stops on.
4:02 being the rare exemption it's my favorite track, but however your test I can say from over 2,000 hours on the game, some tracks do repeat sometimes and when I play with friends or teammates from competieve messing around in a worldwide I can say that some track do pick a lot more we had big blue over 7 times in around 13 races and had bone dry dunes 6 times in around 18, I feel like at random in each room has some sort of pool of tracks it like to select from the exemption is the random one because it's random
I have a feeling that this is like apples shuffle feature: when it was really random it was not liked, but when they tipped the scales in some ways, it felt more random, i prefer a rigged thing that feels random than a random thing that feels rigged
I also feel like once you notice a track appearing a lot, you’re more likely to focus on that track more making it feel like it appears even more then it does. This could also apply to tracks you strongly like or dislike, for me it feels like my absolute favourites never appear but that’s probably because I focus specifically on those tracks and other tracks just never cross my mind. Like, I never think about Shy Guy Falls much so I’m less likely to spot a “pattern” with it then a track I strongly like or dislike.
I wonder if this is the same case for the wii u version as well because I remember back in the day playing big blue and yoshi circuit what felt like every race haha, very nice video!
Thank you! 😄 I did briefly try Wii U again, though not nearly as much as Deluxe. Originally I was gonna write it into the script but didn't wanna jump to conclusions with the limited data I got.
As someone who just played over 200 races on the Wii U version during the last few days and got every course picked at least once, they picked the DLC courses WAY more than the pre-DLC ones, both on the course select screen and the races, I don’t think there was a single time the course select screen didn’t have a DLC course even once, and I really did count every time we raced on each course
Nice video, I think your data does show a random distribution, but I wish you had gone into a little bit of statistical analysis to show this rather than just eyeballing it. Like shortcat's sim data shows, even with perfectly random data you are likely to see data points in the 3-5 and 18-22 range. You can test your data and put confidence intervals on the expected range of appearances and make a stronger statement on whether the appearance count is randomly distributed or not. You briefly alluded to this, so otherwise great vid! Also, did you separately track the result from the 'random' option too? Was there anything interesting there?
The CC of an online lobby is randomized weighted based on the average MMR of the lobby overall, so the higher you rank up the higher the CC you play on average.
When people say they want "randomness", what they actually want is variety. But true randomness often produces long strings of the same outcome. Instead, systems of "smart randomness" can be used to measure previous outcomes and ensure high variety. In other words: if the game manipulated which courses showed up, it would feel more random.
I wish you could have the entire map selection or make a literal wheel and spin every track in the game then once it’s done replace it, remove it, or move it up the line and once all tracks are done shuffle them into a true random mode
Hey there, just wanted to say this was a great video, i liked it and subscribed and i hope you had as much fun making this and i had watching it. God bless!
Thank you very much! I really do appreciate it. 😄 Yes it was fun doing this experiment! Maybe not so much fun sorting out the data 😂 but ya gotta do what ya gotta do.
Its ez, if everyone is picking idk Dk Summit or mushroom gorge but there is one single person choosing freaking waterpark.. we all know which one is gonna be picked.. Tbh i dont understand if its a 50% 50% or if the game really likes pickin the track that was voted the least
i remember when wave 5 released, everyone voted the wave 5 tracks except for this one guy who would always only pick base game tracks just to be a contrarian (he also had a Rick Mii, so he's probably also a Redditor). And the game picked his tracks SO OFTEN! It was always Thwomp Ruins as well...
With Dry Dry Desert and City tracks being the most appearing, I have a theory that they make courses that don't get chosen so much appear more often in order to try to even out the number of times each track gets played online.
Mk ten needs an online mode overhaul with it like the battle arenas in smash ultimate where the host select the track and other players have a preferred tracks preset and the game would rotation in order and once all preferred tracks are played the lobby host can pick a track or select random
My initial assumption: each time a course is shown it sticks around for a while in order to give prople who saw it a chance to pick it. This means you only get a true random roll every 10 or so races. Furthermore when tracks are chosen it may deprioritize them
Great video! I appreciate someone doing this since I've been noticing this as well. However, I was wondering if you noticed the frequency of certain tracks changing after around 2 weeks or so? For example, in the session you played, you said Dry Dry Desert was most frequent, but there have been times when I play almost everyday for a few hours for 2 weeks, and I swear some courses only appear no more than once or twice. This then changes later after those two weeks where I'll see a new batch of courses that appear more frequently. Essentially what me and my friends have noticed is that the game tends to cycle through which courses will get in the selection the most, and this has been happening for well over a year of watching this. It's so consistent that I just can't get myself to personally believe all of this is entirely random. But that's me.
When you process statistical data like this you **need** to include the standard deviation. Your average number of appearances from the data you find to be 10.8 and hypothesize 10-11 appearances. However if the standard deviation is 0.1 or less it's very possible that the fact that your mean lands in the 10-11 range is due to random chance when in actuality it /is/ rigged. The standard deviation communicates how far from the average you can expect a sample to be, so if the standard deviation is low enough then even a seemingly small deviation from the mean could be meaningful, even if to us we think it feels normal or expected. Standard deviations are the other side of the statistics coin, and averages without reported standard deviations are extremely prone to false conclusions. If I say "the average house is worth 1 million dollars, so if I randomly select a house, I can expect it to be around 1 million dollars". This wouldn't really be a great conclusion if the standard deviation was, say also 1 million, because I should also expect any randomly selected sample to be 1 standard deviation away, ie: between 0 and 2 million dollars (which like yeah, no shit, that's not actually that meaningful). So saying that "10.8 is close enough to be within random error" only works if the standard deviation is large enough; if we should expect random samples to be a lot further from the average, which we can only know from the standard deviation. And the be even more technical, you probably want to do an equal proportion goodness of fit Chi Squared hypothesis test on this data, but considering that the average is 10.8 and say we take the hypothesized average at the midpoint of 10 and 11 to mean 10.5 for our hypothesis, the fact that one standard deviation is just over 3.41, that would mean that over 60% of the samples should lie between around 7 and 13.5, which 10.8 is very definitively inside, so it's probably fine......... But never ever ever trust averages (or conclusions based on such averages) without a standard deviation present
@@SnoFitzroy it's where you try to measure how good of a fit a bunch of different groups of data fit and expected proportion, in this case the proportions are equal. If you're testing that all of the colors of m&ms occur at the same frequency, that's what you'd use. So here you collect your samples, say 1000 races, and count the number of occurrences for each track as X. Then you calculate the expected number of times that each track should have appeared, E=1000/#tracks, and calculate (E-X)²/X² for each track. Sum those up, and calculate the p-value in the correct Chi Squared Distribution :)
Kind of related to this, but spotify's random shuffle isn't random at all. They tried making it random, but you'd randomly get some related songs in a row and people would say it's not random. So they programmed it to reduce that from happening, making it appear "random."
Randomness usually wont reault in an even spread. I once read a story about a college professor who had his studenta flip a coin 100 times and log the results. In class he then proceeded to name all the students who actually did the assignment as most just randomly wrote results down. The giveaway was that fakers strongly underestimated how long a streak can go for
From the video it was not clear to me if they actually did the math or not, anyways math checks out. There is no statistically significant deviation in the data (chi² test). In fact the probability that the deviation is larger then this is about 50%.
Great video! In computing there’s no such thing as random, we just have to find out how it picks ‘random’ which we might never know. Having played Mario maker 2 a lot I notice some patterns in how it picks courses in endless mode. For example a speedrun level will always be followed by another speedrun level. I’m trying to find a pattern like you did here with Mario Maker. Maybe Nintendo uses a similar strategy for every game?
Im not that surprised with this result. I play games with heavy RNG elements and one thing I've noticed is that the average player is terrible with statical analysis even when provided all the data let alone when guessing part of it. I think the best example is with just a simple die roll. If something has lets say a 1/6 chance to occur, most players get mad when it takes more than 6 attempts because they feel like "statistically" they should've had it occur. But if you actually did the calculations the chance you would fail all 6 rolls is a 33% chance which is pretty high.
One thing you said at 7:21 that a “completely random” selection would have each track appearing 10-11 times isn’t exactly true. That would be accurate if the track selection was perfectly DISTRIBUTED (similar to how Spotify Shuffle works) Truly RANDOM data naturally can have outliers and even in your larger data collection there was a sizeable chance that a track would never come up, as seen in the previous experiment. Would have loved to see some mention of standard deviations to quantify how “random” the selections truly were!
You could possibly speed up the data collection process by disconnecting at the start of each race so you don't have to spend time playing it. Although it would be interesting to know if there was bias in place based on time of day or region. Or any other factors. Maybe the MK Modding community would be able to pull the code out and find the information directly.
>Don't you feel it selects certain courses too much YES OH MY GOD - it keeps selecting DLC courses that I don't recognize (which is all of them, I don't have the DLC) and it feels like everyone else is on a "home field advantage" and I'm drifting the wrong way, wasting items, getting bombarded by players who know where the shortcuts are, etc, etc. I've seen Dragon Driftway (the map I'm best with) ONCE while playing online and there were exactly TWO other players. The one time I get my second favorite map and there's no real competition. Sure, it was a fun easy win but it didn't feel earned, what with me being Dry Bowser against two Baby characters lmao. even if the data shown here implies no bias, I'd give anything to see Electrodrome ONCE online. It's my favorite course in the whole series.
You know, interestingly enough, the Wii U version of this game had a pretty clear bias toward it's 16 DLC tracks, if you chose to play with them on. With how many Tour courses there are, I'm glad that the balancing seems even now. One thing I also wonder about this experiment is if the "random" select option is also truly random. It'd probably take a lot more testing to get data from that. I've wondered if it acts as a hidden 4th track, but I'm also pretty sure I've seen random select courses that appeared on the track select screen as well. It'd be interesting to see how that would impact the overarching data - clearly, it would be hard to ensure that random is selected, and you'd likely have less samples as well. One thing you could do is use a tournament mode tournament, and not give everyone the code. Technically you can leave the lobby very quickly to get new courses a bunch I think, and it'd only require your friend to cooperate to see what random selects as well. I definitely think that'd be faster than racing normally if you were to do further tests. Interesting video though! I am surprised that some tracks showed up 4 or 21 times, but perhaps with more trials, those numbers would level out. I've always felt Dry Dry Desert common is super common online for some reason too - it could make sense since it is kind of the silliest track and a bit RNG dependent. Nintendo might feel it gives more players a chance, or makes them feel like they're more in. But then again, it could also just be purely coincidental.
I’m surprised it’s random. I love bone dry dunes despite it being a more obscure map, I don’t like playing online because it pretty much never shows up.
Something that I'm interested to know is what your points/level is and if you where in Global or Region lobbies. If you are in an advanced lobby, it could change the type of races giving you more mirror and 200 games based on skill level. Also, high level players tend to prefer different types of courses
i get SNES RAINBOW ROAD, Excitebike Alpine Pass, wario stadium and DOLPHIN SHOALS OVER AND over & i think it's honestly to keep people on the game for the better tracks 😂
Each different course swlected has an equal chamce of being picked, so if 11 people choose one and 1 person chooses another its a 50/50, and random just randomly picks one of the other choices, it's just letting the other people in the match decide instead
Personally i think maybe the game looks at what stages all players haven't raced on for a while and gives those stages higher odds of appearing on the selection. This would explain unpopular stages appearing a lot because people avoid racing on them.
Surely the feeling of non-randomness comes from the fact that players are given an option to pick their favourite from a selection of 3? In the section where you're talking about tracks with low pick rates pretty much everything excluding Music Park is a Certifiably Mid track. Your Coconut Malls, Mount Warios, and apparently DK Jungles, are going to feel weighted towards during play as people will gravitate to those when they appear in a random selection of 3. It's going to feel like you never see Bone Dry Dunes because you don't, because nobody picks it. Also while we're here - why does everyone seem to love the tour city tracks in online lobbies? I've had lobbies before when I'm the only one voting for something like Squeaky Clean Sprint against a sea of London Loops, and it actually boggles my mind.
I don't think it's totally random. When you said that Bone Dry Dunes was in last place, I knew it was skewed in some way. I haven't played the Switch version that much, but in the Wii U version of the game, that track almost never showed up. Absolutely the least common one. I've played that game for hundreds of hours, and it has shown up less than 10 times in online for me. Also, I just thought of something. Maybe some tracks are made to show up first, such as Dry Dry Desert and Electrodrome. Which would explain why it seems those show up very often. Maybe Bone Dry Dunes is a track that only shows up if the lobby has been open for a long time.
I'm willing to bet the Wii U version was a bit more biased. I remember getting really pissed off that Ice Ice Outpost was showing up so often. Sure, it was a newly released DLC, but the number of times it showed up was staggering.
Not directly related to this, but I’ve never liked the “random” button in the course selection screen. If everyone else chooses course A, but I don’t care and choose random, that leaves the chance for course B to be selected anyway. Shouldn’t that button be replaced with something along the lines of “I don’t care, pick whatever the others want”
That *highly* depends on the Tour tracks. If it’s Paris, London, and Tokyo then I kinda see what you mean… If it’s Berlin, Athens, and Vancouver then it’s a better selection. Be grateful you can play the Tour cities in a console game.
I don’t think this is a Mario kart specific problem, is just computer generated randomness isn’t actually that random in small amounts. I run into issues like this with all sorts of other games, it the more you increase the sample size, the more the numbers even out NOW NINTENDO I HAVENT GOTTEN CHEESE LAND IN A WEEK WHYYYYY
Haven't watched the video yet but all I know is Big Blue isn't up for selection enough and when it is it isn't actually selected enough, all that time trialling for nothing (jk)
Great video! I had a great time playing with you guys & helping with the doc, I was not expecting those results
Hood emojis 🤯
@@braziliantsar
Great video! That's a lot of work to collect all that data. I wanted to confirm if your data actually looks random (since it's hard to tell, as a human) so I ran some code to generate 346 sets of 3 unique tracks (to simulate 346 track selections as you did), and then ran this simulation over and over to see what happened.
If the track selection is properly random, then the most frequent track would usually show up 18-22 times. The least frequent is usually 3-5 times. Everything else is distributed how you'd expect, with 10 and 11 times being the most common frequencies. So it looks like the randomness is correct in-game.
I simulated the test from reddit 2018, and I got very different results from that one. It's very rare for a track to show up 0 times. So if that test was accurate, then the online track selection was not very random back then.
Wow, thank you! Glad to know our test is backed up by a simulation as well. 😅Yeah I'm surprised it seems to be fixed. Now the Wii U version on the other hand may be a different story, though the online for that is going off soon...
you have way too much time on your hands
@@rye9270they’re job is to play mariokart and make videos. They have as much time as they want ☠️
@@rye9270I do have too much time, but this one was quick! I asked chatgpt to code it, then I just ran the simulation a few times. That all took 2 minutes. Then a few more minutes to write the comment
@@rye9270and they still make good money. Shortcat is a goat
i remember that itunes had to change how their shuffle works because it being truly random made people feel like they were playing the same tracks over and over, so we can definitely trick ourselves into thinking there is a pattern when there is none.
Banger video! And honestly, time of day isn't really off the table. Splatoon is very clear that it only has some maps online at a time, and while this was so long in the past, Mario Kart DS only had certain tracks online, showing again there's only a limited selection up.
Thank you brother. 🙂 Yeah honestly there were a ton of things we thought about, but was very surprised when everything checked out.
WTF are you doing here???
@@Solscalr Chillin'
@@OfficialGlitchDoctor3:31 if you dont believe in god
why do you have to swear to one for those of us that do?
swearing to god is breaking the 3rd commandment
@@blob5907it's called a saying, just because some people dont believe in god doesnt mean we arent allowed to say it.
200cc and mirror mode appear less when you're VR is low and more when you're VR is high
Sorry, what does VR mean? I tried searching it up, and all I got was virtual reality.
hate to be _that guy,_ but
*your
@@Damon_Blue vs rating
@@ChordettesFan325 Thanks
As someone who's played long enough to accrue 65k VR I'm honestly shocked to find out that it's legitimately random. I would have sworn that certain tracks were exponentially more likely to show up, but I guess it turns out that I'm just more likely to notice when the game offers the tracks that I've previously suspected to be "more likely". It must be some sort of mental bias on my part after all! Great work man.
Yeah we were very surprised by the outcome as well! Felt like it was giving a lot of the same tracks over and over for sure. Thank you! 😄
4:50 You heard him. Mount.
5:58 “Every option was a different Rainbow Road”
*Music playing isn’t from any of the ones onscreen*
I swear it always picks the track NO ONE wanted. Literally its always the person that only ONE person votes
Isn't it an equal chance of courses, so that one person could have a 50% chance
edit:i meant track, not person
@@arzfan29yes it is
The results in the data have a p value of 0.2814 under the chi squared test, thus the data does not significantly suggest that the course selection is not random
I'd say that the tracks that repeat the most and actually stand out to us are either overwhelmingly picked due to popularity like Wii Rainbow Road(or for some reason everyone accidentally clicked Water Park) or the tracks that you just hate playing on.
It also doesn't help if those 2 categories overlap because I definitely hate some tracks that are popular(not because they're popular, I just don't like their visual and/or track design).
Also, banger video.
This is what I was thinking. I personally really dislike dry dry desert so I would definitely feel like it wasn't as random as it should be if I got it too often.
There was one time that Boo Lake showed up 5 times within 20 races, one of the Boo Lakes appeared from random and in total we played 3/5 Boo Lakes
4:52 Mount Wario joke 10 years later 10/10
Regarding when the course gets picked after the vote, I will never forget one time when 11 of us picked the same track, and the 1 player picked a different track, and the game picked the 1 outlier. I was livid
I think what makes it feel that way for me is that the tracks that show up more often tend to be ones I don’t like (electrodrome, dry dry desert, donut plains 3)
My theory is that although it is random, there are only a select 32 or so tracks that are always voted for because of popularity. Therefore, those tracks are played more often, tricking us into thinking the selection isn’t random.
this could be another case of people tricking themselves into believing something isn't random when it really is, but i feel like it's way more common (like 1/6 or something instead of 1/12) for my track to be chosen when it's the first race i've played that session, as opposed to the 1/12 chance it should be.
I know the made the iTunes shuffle option more random by making it less random. In other words, it shuffles a little less than normal
What a cool study!
Although I found the result unsurprising, this video does an excellent job showcasing the way that pattern recognition in the human brain can sometimes misfire.
We've all had those lobbies where we can swear we've played the same track so many times that we can swear that it feels like John Nintendo himself is willing it unto us. In reality, understanding true randomness is understanding that sometimes, weird luck is sometimes all it takes to encounter repetition.
And with 96 whole tracks in the game, a few of them are BOUND to become an outlier
i love this! i recently started keeping track in a notebook too because i felt like there were some tracks i had almost forgotten. my data is very rough but i'm glad to see someone else was doing this!
When I play Mario Kart 8 on Wii U, I notice that the DLC courses get prioritized more than the base game courses.
I was just doing similar research, glad someone went all the way through with it
I honestly think that everyday, random tracks have higher chances of appearing, thus causing the problem you mentioned at the beginning. So it is random, but the pool of tracks and their chances to appear constantly changes.
in our high school sports team, in a round of 6 races where everyone has to vote random, we got yoshi valley THREE TIMES
The tracks I very frequently get:
Cheep Cheep Beach
GBA Mario Circuit
Amsterdam Drift
SNES Rainbow Road
DK Jungle
Dragon Driftway
will edit if I think of more
I'm using tilt controls!
Tilt Control gang go hard 😎🤙
That Mount Wario joke was as gold as the stuff he'd pay you. Good work on the video, although I kinda want to see a whole statistical breakdown (or see the data and do it myself).
When playinhg local multiplayer, I find that the tracks that we like to play often become really boring after a while, so I would guess Nintendo would probably try to make people pick more varied courses to not get tired of any specific ones.
It would make sense to push the less popular courses more so that the popular courses don't get played so much that people get bored of them.
0:19 Correction: MarioKart Tour actually had the most courses with over 500 tracks.
On a technicality
not mainline
@@alouby Yes it fucking well is. Nintendo said so
@@DryPaperHammerBro Nintendo said smash 3ds is smash 4 and smash Wii U is smash 5, we don't care
ain’t no way he used the SpongeBob holographic meatloaf sfx 1:45
😂😂
Extremely disappointed you didn’t actually show the bell curve
This is a new video type that I’ve never seen before and it’s awesome thanks for covering this!
150cc is most common because of your rating.
Your online rating plays a big role in what lobbies you play.
Wow!! Great video! But you can’t convince me Excitebike Arena doesn’t show up 10 out of 10 times 😂
Justice for Bone-Dry Dunes! It's not a "generic desert", it's a very innovative track.
It’s not generic, I actually think it’s very unique. However, it is very annoying to play. You need to brake drift basically every turn, and don’t get me started on 200cc
The original had the tornadoes that made it more interesting.
@@prestonandtom9337 wrong track
@@Elfilinfan oh yeah
@@prestonandtom9337disenchantment pfp W
This research has a big flaw. Assuming that by at some point, you should see all of the courses. From the sound of it, you assume that the more you play, the more likely you'll run into a specific track. In reality tho, it doesn't increase your chances. Assuming there isn't an algorithm and it is conpletely random, your chances wont change. It'll be the same throughout
honey wake up glitchdoctor posted another banger
Thank you brother. 🤙
Spin doctor
I don’t know why, but ninja hideaway never appears on the course selection. Mario kart is truly rigged
Nintendo doesn’t want us to repeatedly pick tracks in MK8 (like in MK7 & MKWii where you can pick any track). However, once a race has played on a specific track (ex. Mount Wario) it can’t be shown again on the next race until after at least 1 race on a different track (ex. N64 Royal Raceway after the Mount Wario race) has been played.
I was in a bad mood the other day and it kept giving the lobby Cheese Land as an option I just quit because it’s my least favorite course 😭
I did a Q-Q plot of the data against a Poisson distribution with a rate of 10.8125 (how many times each course should show up on average over 1038 selections) and it can modeled really nicely with the line y=x which means it's a good fit, so the data is indeed randomly distributed.
What you mentioned about Mirror and 200cc showing up only once is strange. Usually when I play online, I tend to only get Mirror Mode when I play online
I know this is a far reach, but for who interests and has the knowledge, the May 2014 USA kiosk disc demo for Mario Kart 8 shipped with debug symbols (not sure if stripped or non-stripped ELF, most games accidentally ship with stripped, so safe to assume Nintendo EAD did too). I don't think the variables used are noted, so the process to know if the random generator is actually random is gonna take weeks to discover, anyways the functions for it are called "UpdateMenuDecide" and "UpdateRoulette", since they are present in Wireless Play/LAN Play too.
Outside nerd stuff, for me it seems like the selection of courses seems to be an internal table where courses are ranked like ELO or another system like that, I don't think the courses from the DLC get a higher score because they're just DLC courses, you just had a lil bit of luck getting that unequal percentage, I do believe though that from the random pool of courses it chooses at first, the ones that are not selected just get a higher and higher rating and that's why they don't go away.
For the screen after that when it picks the 10 random courses, yeah that's 100% random, no string attached, it's just a roll of the dice.
4:54 caught me so off guard 😂
The thing about randomness is that, seemingly counterintuitively, you'd expect long runs of the same option every so often. What you need to look for is not that there *are* some courses favored, but whether or not a certain amount of favoritism being *likely*
Its either GBA Mario Circuit or Royal Raceway. Pick your poison.
@GlitchDoctor: They are a few problem with does test, if you stay in the exact same room for 346 tracks, without re-setting the room, or being force into another room. Since it would be imposible to do this because you can't stay awake for over 48 hours, and don't ruin your health just to test this. Unless you used an A.I. on emulator to play 365 random race, but if all people leave that mean it will switch group and you don't know where the random begin from when you auto switch group.
I knew for a fact that Bone Dry Dunes was gonna be on the low end. It rarely ever showed up even before the Booster Pass came along lol.
This kind of thing is exactly what MatPat was talking about when he said it's _"our_ turn to be the Theorists". You had a hypothesis, you gathered data, you got a result. Science!
Also the only way to control for the "selection bias" of which track gets _played,_ would be to have a private lobby of 12 players all "in on the experiment", three sets of four people each picking a specific slot every time (first/second/third) to give equal odds of selection, maybe also writing down which position the selection cursor starts and stops on.
Good to know im just extremely unlucky then when i play for multiple days without seeing shy guy falls or a shock a single time
4:02 being the rare exemption it's my favorite track, but however your test I can say from over 2,000 hours on the game, some tracks do repeat sometimes and when I play with friends or teammates from competieve messing around in a worldwide I can say that some track do pick a lot more we had big blue over 7 times in around 13 races and had bone dry dunes 6 times in around 18, I feel like at random in each room has some sort of pool of tracks it like to select from the exemption is the random one because it's random
I have a feeling that this is like apples shuffle feature: when it was really random it was not liked, but when they tipped the scales in some ways, it felt more random, i prefer a rigged thing that feels random than a random thing that feels rigged
I also feel like once you notice a track appearing a lot, you’re more likely to focus on that track more making it feel like it appears even more then it does. This could also apply to tracks you strongly like or dislike, for me it feels like my absolute favourites never appear but that’s probably because I focus specifically on those tracks and other tracks just never cross my mind. Like, I never think about Shy Guy Falls much so I’m less likely to spot a “pattern” with it then a track I strongly like or dislike.
I wonder if this is the same case for the wii u version as well because I remember back in the day playing big blue and yoshi circuit what felt like every race haha, very nice video!
Thank you! 😄 I did briefly try Wii U again, though not nearly as much as Deluxe. Originally I was gonna write it into the script but didn't wanna jump to conclusions with the limited data I got.
As someone who just played over 200 races on the Wii U version during the last few days and got every course picked at least once, they picked the DLC courses WAY more than the pre-DLC ones, both on the course select screen and the races, I don’t think there was a single time the course select screen didn’t have a DLC course even once, and I really did count every time we raced on each course
The mount wario joke took me a second but that was hilarious
Mario kart tour still has more tracks than 8dx
Mario kart tour still has not become better than 8 duluxe
My opinion says otherwise 😂
I think offline choice is random too. Did teams in local co op with some friends and our team got elected to pick the track like 10 times in a row
4:51 this part took me a second to get but then I cracked up 😂
It felt like a Scott the Woz joke ngl 😂
It's probably a combination of the choices in a group and the most popular tracks by the players
Nice video, I think your data does show a random distribution, but I wish you had gone into a little bit of statistical analysis to show this rather than just eyeballing it. Like shortcat's sim data shows, even with perfectly random data you are likely to see data points in the 3-5 and 18-22 range. You can test your data and put confidence intervals on the expected range of appearances and make a stronger statement on whether the appearance count is randomly distributed or not. You briefly alluded to this, so otherwise great vid!
Also, did you separately track the result from the 'random' option too? Was there anything interesting there?
The CC of an online lobby is randomized weighted based on the average MMR of the lobby overall, so the higher you rank up the higher the CC you play on average.
When people say they want "randomness", what they actually want is variety. But true randomness often produces long strings of the same outcome. Instead, systems of "smart randomness" can be used to measure previous outcomes and ensure high variety. In other words: if the game manipulated which courses showed up, it would feel more random.
I wish you could have the entire map selection or make a literal wheel and spin every track in the game then once it’s done replace it, remove it, or move it up the line and once all tracks are done shuffle them into a true random mode
Hey there, just wanted to say this was a great video, i liked it and subscribed and i hope you had as much fun making this and i had watching it. God bless!
Thank you very much! I really do appreciate it. 😄 Yes it was fun doing this experiment! Maybe not so much fun sorting out the data 😂 but ya gotta do what ya gotta do.
@@OfficialGlitchDoctor Yeah data mining kind of sucks lol
Its ez, if everyone is picking idk Dk Summit or mushroom gorge but there is one single person choosing freaking waterpark.. we all know which one is gonna be picked.. Tbh i dont understand if its a 50% 50% or if the game really likes pickin the track that was voted the least
i remember when wave 5 released, everyone voted the wave 5 tracks except for this one guy who would always only pick base game tracks just to be a contrarian (he also had a Rick Mii, so he's probably also a Redditor). And the game picked his tracks SO OFTEN! It was always Thwomp Ruins as well...
With Dry Dry Desert and City tracks being the most appearing, I have a theory that they make courses that don't get chosen so much appear more often in order to try to even out the number of times each track gets played online.
Mk ten needs an online mode overhaul with it like the battle arenas in smash ultimate where the host select the track and other players have a preferred tracks preset and the game would rotation in order and once all preferred tracks are played the lobby host can pick a track or select random
No fr tho, online is just Dry Dry, Cheese land, Electrodrome, its rare you get something else. I think MKWii did course selection right
My initial assumption: each time a course is shown it sticks around for a while in order to give prople who saw it a chance to pick it. This means you only get a true random roll every 10 or so races. Furthermore when tracks are chosen it may deprioritize them
You know, it may be helpful to have the community lend a hand for this data as well, perhaps regions can mess with the distribution
Great video! I appreciate someone doing this since I've been noticing this as well. However, I was wondering if you noticed the frequency of certain tracks changing after around 2 weeks or so? For example, in the session you played, you said Dry Dry Desert was most frequent, but there have been times when I play almost everyday for a few hours for 2 weeks, and I swear some courses only appear no more than once or twice. This then changes later after those two weeks where I'll see a new batch of courses that appear more frequently. Essentially what me and my friends have noticed is that the game tends to cycle through which courses will get in the selection the most, and this has been happening for well over a year of watching this. It's so consistent that I just can't get myself to personally believe all of this is entirely random. But that's me.
When you process statistical data like this you **need** to include the standard deviation. Your average number of appearances from the data you find to be 10.8 and hypothesize 10-11 appearances. However if the standard deviation is 0.1 or less it's very possible that the fact that your mean lands in the 10-11 range is due to random chance when in actuality it /is/ rigged. The standard deviation communicates how far from the average you can expect a sample to be, so if the standard deviation is low enough then even a seemingly small deviation from the mean could be meaningful, even if to us we think it feels normal or expected. Standard deviations are the other side of the statistics coin, and averages without reported standard deviations are extremely prone to false conclusions. If I say "the average house is worth 1 million dollars, so if I randomly select a house, I can expect it to be around 1 million dollars". This wouldn't really be a great conclusion if the standard deviation was, say also 1 million, because I should also expect any randomly selected sample to be 1 standard deviation away, ie: between 0 and 2 million dollars (which like yeah, no shit, that's not actually that meaningful). So saying that "10.8 is close enough to be within random error" only works if the standard deviation is large enough; if we should expect random samples to be a lot further from the average, which we can only know from the standard deviation.
And the be even more technical, you probably want to do an equal proportion goodness of fit Chi Squared hypothesis test on this data, but considering that the average is 10.8 and say we take the hypothesized average at the midpoint of 10 and 11 to mean 10.5 for our hypothesis, the fact that one standard deviation is just over 3.41, that would mean that over 60% of the samples should lie between around 7 and 13.5, which 10.8 is very definitively inside, so it's probably fine.........
But never ever ever trust averages (or conclusions based on such averages) without a standard deviation present
"equal proportion goodness of fit Chi Squared hypothesis test"
ok you have to br trolling wtfff is that
@@SnoFitzroy it's where you try to measure how good of a fit a bunch of different groups of data fit and expected proportion, in this case the proportions are equal. If you're testing that all of the colors of m&ms occur at the same frequency, that's what you'd use. So here you collect your samples, say 1000 races, and count the number of occurrences for each track as X. Then you calculate the expected number of times that each track should have appeared, E=1000/#tracks, and calculate (E-X)²/X² for each track. Sum those up, and calculate the p-value in the correct Chi Squared Distribution :)
Kind of related to this, but spotify's random shuffle isn't random at all. They tried making it random, but you'd randomly get some related songs in a row and people would say it's not random. So they programmed it to reduce that from happening, making it appear "random."
Randomness usually wont reault in an even spread. I once read a story about a college professor who had his studenta flip a coin 100 times and log the results. In class he then proceeded to name all the students who actually did the assignment as most just randomly wrote results down. The giveaway was that fakers strongly underestimated how long a streak can go for
From the video it was not clear to me if they actually did the math or not, anyways math checks out. There is no statistically significant deviation in the data (chi² test). In fact the probability that the deviation is larger then this is about 50%.
Great video! In computing there’s no such thing as random, we just have to find out how it picks ‘random’ which we might never know.
Having played Mario maker 2 a lot I notice some patterns in how it picks courses in endless mode. For example a speedrun level will always be followed by another speedrun level.
I’m trying to find a pattern like you did here with Mario Maker. Maybe Nintendo uses a similar strategy for every game?
Also, did you investigate the track selections correlation to the cup it’s in? Could be something to do with that too
Im not that surprised with this result. I play games with heavy RNG elements and one thing I've noticed is that the average player is terrible with statical analysis even when provided all the data let alone when guessing part of it.
I think the best example is with just a simple die roll. If something has lets say a 1/6 chance to occur, most players get mad when it takes more than 6 attempts because they feel like "statistically" they should've had it occur. But if you actually did the calculations the chance you would fail all 6 rolls is a 33% chance which is pretty high.
One thing you said at 7:21 that a “completely random” selection would have each track appearing 10-11 times isn’t exactly true. That would be accurate if the track selection was perfectly DISTRIBUTED (similar to how Spotify Shuffle works) Truly RANDOM data naturally can have outliers and even in your larger data collection there was a sizeable chance that a track would never come up, as seen in the previous experiment. Would have loved to see some mention of standard deviations to quantify how “random” the selections truly were!
This is odd to me… Athens appears often for me! I got it 2 times in 5 races today! I’m not complaining as I really like it.
You could possibly speed up the data collection process by disconnecting at the start of each race so you don't have to spend time playing it. Although it would be interesting to know if there was bias in place based on time of day or region. Or any other factors. Maybe the MK Modding community would be able to pull the code out and find the information directly.
>Don't you feel it selects certain courses too much
YES OH MY GOD - it keeps selecting DLC courses that I don't recognize (which is all of them, I don't have the DLC) and it feels like everyone else is on a "home field advantage" and I'm drifting the wrong way, wasting items, getting bombarded by players who know where the shortcuts are, etc, etc. I've seen Dragon Driftway (the map I'm best with) ONCE while playing online and there were exactly TWO other players. The one time I get my second favorite map and there's no real competition. Sure, it was a fun easy win but it didn't feel earned, what with me being Dry Bowser against two Baby characters lmao.
even if the data shown here implies no bias, I'd give anything to see Electrodrome ONCE online. It's my favorite course in the whole series.
You know, interestingly enough, the Wii U version of this game had a pretty clear bias toward it's 16 DLC tracks, if you chose to play with them on. With how many Tour courses there are, I'm glad that the balancing seems even now.
One thing I also wonder about this experiment is if the "random" select option is also truly random. It'd probably take a lot more testing to get data from that. I've wondered if it acts as a hidden 4th track, but I'm also pretty sure I've seen random select courses that appeared on the track select screen as well. It'd be interesting to see how that would impact the overarching data - clearly, it would be hard to ensure that random is selected, and you'd likely have less samples as well.
One thing you could do is use a tournament mode tournament, and not give everyone the code. Technically you can leave the lobby very quickly to get new courses a bunch I think, and it'd only require your friend to cooperate to see what random selects as well. I definitely think that'd be faster than racing normally if you were to do further tests.
Interesting video though! I am surprised that some tracks showed up 4 or 21 times, but perhaps with more trials, those numbers would level out. I've always felt Dry Dry Desert common is super common online for some reason too - it could make sense since it is kind of the silliest track and a bit RNG dependent. Nintendo might feel it gives more players a chance, or makes them feel like they're more in. But then again, it could also just be purely coincidental.
I’m surprised it’s random. I love bone dry dunes despite it being a more obscure map, I don’t like playing online because it pretty much never shows up.
Something that I'm interested to know is what your points/level is and if you where in Global or Region lobbies. If you are in an advanced lobby, it could change the type of races giving you more mirror and 200 games based on skill level. Also, high level players tend to prefer different types of courses
i get SNES RAINBOW ROAD, Excitebike Alpine Pass, wario stadium and DOLPHIN SHOALS OVER AND over & i think it's honestly to keep people on the game for the better tracks 😂
Each different course swlected has an equal chamce of being picked, so if 11 people choose one and 1 person chooses another its a 50/50, and random just randomly picks one of the other choices, it's just letting the other people in the match decide instead
Personally i think maybe the game looks at what stages all players haven't raced on for a while and gives those stages higher odds of appearing on the selection. This would explain unpopular stages appearing a lot because people avoid racing on them.
Surely the feeling of non-randomness comes from the fact that players are given an option to pick their favourite from a selection of 3? In the section where you're talking about tracks with low pick rates pretty much everything excluding Music Park is a Certifiably Mid track. Your Coconut Malls, Mount Warios, and apparently DK Jungles, are going to feel weighted towards during play as people will gravitate to those when they appear in a random selection of 3. It's going to feel like you never see Bone Dry Dunes because you don't, because nobody picks it.
Also while we're here - why does everyone seem to love the tour city tracks in online lobbies? I've had lobbies before when I'm the only one voting for something like Squeaky Clean Sprint against a sea of London Loops, and it actually boggles my mind.
I don't think it's totally random. When you said that Bone Dry Dunes was in last place, I knew it was skewed in some way. I haven't played the Switch version that much, but in the Wii U version of the game, that track almost never showed up. Absolutely the least common one. I've played that game for hundreds of hours, and it has shown up less than 10 times in online for me.
Also, I just thought of something. Maybe some tracks are made to show up first, such as Dry Dry Desert and Electrodrome. Which would explain why it seems those show up very often. Maybe Bone Dry Dunes is a track that only shows up if the lobby has been open for a long time.
Great analysis! Kudos on all the research and testing! I'm glad I'm not the only one who doesn't like Bone-Dry Dunes! 😅
I'm willing to bet the Wii U version was a bit more biased. I remember getting really pissed off that Ice Ice Outpost was showing up so often. Sure, it was a newly released DLC, but the number of times it showed up was staggering.
The lack of 200cc is the main reason I don’t like playing Mario Kart online
This blew my mind
Great video, that is actually a lot of data to collect
it wouldn't be random, if it was evenly distributed. There are always bumps to be expected
Your not factoring in the fact player 1 is what causes the rng seed, and player 1 is the top left player as you start a track.
I swear i get bone dry dunes and choco mountain all the time lol
Not directly related to this, but I’ve never liked the “random” button in the course selection screen. If everyone else chooses course A, but I don’t care and choose random, that leaves the chance for course B to be selected anyway. Shouldn’t that button be replaced with something along the lines of “I don’t care, pick whatever the others want”
3 TOUR TRACKS IN THE TRACK SELECTION = The worst experience an human being can have
That *highly* depends on the Tour tracks.
If it’s Paris, London, and Tokyo then I kinda see what you mean…
If it’s Berlin, Athens, and Vancouver then it’s a better selection.
Be grateful you can play the Tour cities in a console game.
I don’t think this is a Mario kart specific problem, is just computer generated randomness isn’t actually that random in small amounts. I run into issues like this with all sorts of other games, it the more you increase the sample size, the more the numbers even out NOW NINTENDO I HAVENT GOTTEN CHEESE LAND IN A WEEK WHYYYYY
Haven't watched the video yet but all I know is Big Blue isn't up for selection enough and when it is it isn't actually selected enough, all that time trialling for nothing (jk)