to see the full, unedited "clear rate" experiment, check it out on my vod channel here: ruclips.net/video/JTA0wkdl9M0/видео.html ✨ also, big shoutouts to dsteves for all of the help with this project! a lot of the data and experiments i used in this video wouldn't have been possible without his help!! check out dsteves' channel here: www.twitch.tv/dsteves 😎
It would have been cool to go back in time and track the average clear percent over time. I have a feeling that since the game released, less people are playing and the people that are still playing are really good at the game. So perhaps the levels got harder to accommodate that crowd thus the average clear percent would go down over time.
I think the initial difficulty seeding being higher than expected makes a lot of sense. As the creator, you know all the tricks needed for a level, so it's easier for you than it would be for a new player. Having the initial seed be biased toward higher difficulty is actually pretty smart game design.
Yeah. I always wonder how one of my levels was initially placed after upload since it took me one attempt to clear check it, but found out later that most people who played it struggled and gave up.
That said, I still feel like you could get a better starting estimate by taking into accounts the # of failures before a successful attempt when clear checking it. It would make it so that harder to clear levels tend to be higher difficulty on average.
And yet I've made many levels that took me dozens of tries to clear check and yet they have a 5%-10% clear rate. Many creators aren't the best or most skilled and yet make levels that test them to their limits, whereas there are significantly better players out there that are so much better and can beat the level easier. Beyond that, in my experience most levels are straight forward. No tricks, tech, or lore needed. Just standard levels. I think the point you brought up is very cogent in troll levels, little Timmy levels with random hidden blocks, and very hard levels by amazingly skilled players/creators. For the rest of levels, which might be the larger portion, it doesn't necessarily hold true.
Initial clear check just shouldn't affect difficulty at all, actually. Bad game design.
2 года назад+7
@@Syranovæ If initial clear check didn't matter at all, it could be that most of the people would never get their levels in the random pool, because nobody could find them in the first place. In SMM2 you need to get at least 1 play from somebody else before you get a new level into the random pool, and as a rookie level creator finding even that first player can be tough. I'm not exactly sure if the first play needs to finish the level. Then again, when writing this it occurred to me that they could just use that first other player's clear rate to recalibrate the difficulty before putting the level into the correct random pool. They might actually do that, this video doesn't really confirm or deny anything related to that.
Comment from one verified user with 9 likes placed over multiple comments of unverified users with so much more likes? Clearly, RUclips is falling apart in terms of ratings of videos.
all songs used are from the kirby 64 ost: 0:00 - Mountain Stream 1:23 - Pop Star 2:51 - Rock Star 3:36 - Aqua Star 5:15 - Above the Clouds 5:57 - Boss Theme 6:45 - Neo Star 7:54 - Bumper-Crop Bump 9:47 - Ripple Star 10:49 - Shiver Star 12:29 - Factory Inspection 13:09 - Training
One of the things we also talked about briefly was if levels were reuploaded exactly the same way if a level would differ in initial difficulty, then after the video was in production I remembered about title screens and how I've seen the same level in easy and normal and sometimes even expert, the spike test now does show that this is the case. All in all the goal is to find some sort of algorithm that combines all the pieces together. While initial difficulty on clear check is one piece of the algorithm it's figuring out a way to combine that, possibly the clear rate, and the ratio of who flagged the level as cleared over who has not, as well as figuring out any other overlooked part of the equation that is the key here to getting a sure fire algorithm going.
Since the number of tries it takes the level maker comes into play, I wonder how that has affected the categorization of my own uploaded levels. I often quit a clear check and nerf my levels a little (because I don’t have the patience to beat them lol) until I end up beating them on the first or second try after failing to beat earlier versions of them. So I wonder if Nintendo factors in ALL attempts on a saved level, or if the counter starts over when the level maker quits a clear check and changes the level slightly. One of my levels like this has a 4% clear rate, but I ultimately beat its final form in one try (after failing to beat earlier versions of it). So I wonder which endless mode it ended up in initially because of that.
I think that the algorithm might ignore any clear attempts of a person after the person beat the level for the first time. Another theory I have is that clears of different persons count differently based on some internal skill level value of a player. Somewhat similar to ELO
I'm very curious if skips also factor into the Nintendo algorithm. For instance: take the puzzle level that was in Expert yet had a 17% clear rate, or the exceedingly long levels in Super Expert. Both would likely have very high skip rates, as many people simply do not like puzzle levels or lengthy levels. Someone may die once in a 2 minute long level and skip to the next one rather than attempt it again. This wouldn't impact the clear rate much, but it may have a bearing on the difficulty level. Just a thought!
I had this same question but typically a lengthy level already indicates its difficulty. So if we take that into account I surmise they may have a separate yet similar algorithm to deaths that involve skips. Since every skip would be unique to a level, if a player didn't participate in the level there must be a reason why(which I would surmised levels in Expert and Super Expert get skipped over 5/10 to 9/10 times respectively)
If the player never clears the level like exit the software, it won't save like if you never played it, also The clear rate of 199,000 out of 400,000 is only showing every player's attempt to succeed, all the times the characters looses a life it counts as a failure point(meaning that is where the player's character fails there because of enemy, hitbox of icicle or liquid hazard),The shoe icon only shows how many people played it, and also the clear rate is everyone's attempt to reach the goal point there. Never overestimate the clear rate if the player doesn't know what to do, give it a chance if you think you can complete it.
I had the same question while watching the twitch vod and I was really hoping someone would have brought it up live in twitch chat. If I had to theorize, skips wouldn't affect difficulty as strongly as deaths but it would make sense to factor them in to a degree. On one hand a skip could mean the player died once or twice and determined it was too difficult and skipped it or came across some difficult tech or some precise jump they knew they couldn't do so they skipped it before dying at all. In those cases the skip would represent the amount of difficulty the player WOULD have had if they had kept playing until they cleared the level. Either a great amount of time in the case of difficult tech without a way to die, or many many deaths in the case of precise jumps. On the other had a skipped level may not be difficult but instead be too long, boring, or super annoying. The player could easily beat it, they just can't be bothered wasting their time on something they aren't enjoying. So these skips would be the opposite; they represent a hypothetical first try clear rather than an amount of deaths or struggle clearing the level. So basically skips represent an unknown variable in the difficulty because we can't know the reason for every individual skip. In a hypothetical world where skips didn't exist and a level had to be played until cleared, a level's clear rate could potentially be drastically different than same real world level. The more I think about it, the more curious I am on just how the algorithm DOES consider skips. Does have a way of predicting the reason why a specific player skipped a specific level? Does it gauge the player's own skill level based every other level they've attempted? Does it consider the amount of spent time in the level before skipping? Does it just give all skips a blanket weight and be done with it? Or did the programmers just say "hecc it" and leave skips out of the algorithim altogether?? 🤔🤔🤔 It's so interesting to think about and we will likely never 100% understand how the algorithim works. Unless someone at Nintendo leakes it out...
I think this would be grouped into the how many clear vs plays it has. Not the clear rate, but how many DIFFERENT players actually clear it. If the theory that levels that get cleared more often, even if the clear rate is low, are easier in difficulty, then skipping a level and never going back to clear it likely does increase the difficulty rating
What would be interesting is if they factored in player experience. An experienced player dying on a level could indicate harder level. An inexperienced player 1st trying a level could be strong evidence the level is easy
That was one of the things we were looking at also, but unfortunately there wasn't an easy way to test that - that's definitely a possibility in factoring in level difficulty though!
Agree with this. Someone who has beaten a lot of Super Expert levels should have a different weight in how a level gets classified from a newbie and how hard the level is for them. The algorithm should be looking at the reputation of WHO beats the level.
@Raysfire If the SMM2 scraped datasets contain the players who cleared each level, a correlational analysis could maybe be performed, but obviously this is well beyond the scope of a Twitch stream :)
The thing that the high clear rate super expert levels had in common is that none of them had any way of dying other than the time limit, so many people getting the level and skipping affects it (since skipping without dying doesn't count as an attempt)
Thats basically 0% chance. There are so many levels that some are going to have the same tags as other but be completely different difficulties. There have to be some easy levels with the puzzle tag
I figure your deaths to set the difficultly are probably reflective of the deaths per completion in general. Easy levels are ones where people generally win on the first or second try, normal on the third or fourth, and expert more. The reason the clear rates are different is because of people who skip after failing the first time or who retry for world record. Only people who go on to clear the level or who die more than the expected attempts factor into the difficulty. This would explain why some levels that force a death to respawn from a checkpoint appear in easy or normal despite having ~10% clear rates, the levels consistently get 1-2 deaths per successful attempt and almost no-one dies more often(because it'd need timeouts or similar), but a lot of people quit when they realise the level is a net negative to lives.
The difficulty rating of chess puzzles can be calculated using a modified form of ELO from the players who succeed or fail at it. It would make sense for Mario Maker to use a similar system
Very insightful into the algorithm, thank you. I would like to clarify the conclusions we’ve drawn though- the only thing we’ve proven from our own experiments is that creator clear check attempts noticeably affects the algorithm’s placement. The data science studies were useful, noticing trends of lower clear rates and higher world records tending toward higher difficulty placements; however, we cannot say that the algorithm necessarily checks for these things. In order to show causation we would have to do experiments similar to the spike test but with multiple other players/accounts, keeping the creator clear checks constant.
I got that first ballon level in SE endless too! Also I lost all my lives because I kept having one coin left and the clouds left. Also thanks for this video I’ve noticed too and have been wondering! Sometimes I would get a level with one person who played it, and I don’t know how it Nintendo knew for sure that it was hard. It’s very interesting.
Rays this is an amazing video that you clearly spent a lot of time making. I loved the creative style of it! Keep up the amazing work keeping everyone here entertained and informed on everything Mario maker. ❤
All right, I study mathematics and the graph at 13:10 ringed a bell: a hyperbolic curve. If this graph is the way things work behind the scenes, then you can tell that the _product_ of the number of attemtps and the time per attempt is what counts - which is *the total time spent for the creator to beat the level.* I would love to have this theory tested. Amazing video! I highly enjoyed it. More science, please!
What I learned: When uploading a Super Expert level that you want to trick Nintendo into thinking it's Easy: 1. Make a copy of the level *before* trying to upload it. 2. Try to beat the level. 3. If you win, great, go to step 5, because you just uploaded it! If you lose, then delete the level. 4. If you're here, then go back to step 1, but with the copy of the level. 5. Watch as your level quickly gets moved to Super Expert because Nintendo's algorithm isn't stupid, and will notice when people are struggling. Or just dev door it I dunno.
And now all I need is to get Mario maker 2! YOU'RE GOING INTO SUPER EXPERT, SINGULAR SPIKE! This could actually be used to create break-levels for the harder difficulties. You know, give 'em a free win.
Thank goodness for the sub! I forgot about Quiz day! Awesome Vid Mr. Shud! It's really interesting that the creator in a sense can partially decide where their level will show up!
Wow this was an amazing video! I think I have to throw when doing clear checks on my kaizo's I build. I practice them so much in the editor, that I clear check them in a couple of tries. Then they get put into categories where people won't possibly like or be able to finish them. They're dead on arrival.
other ideas i can think of for possible factors might be things like hazards in the level, level size, time spent creating the level, amounts of inputs on clear attemps (for example just holding right, holding right and jumping some, or having something more complex than that) also maybe the level creator's previous levels have an effect?
Man, it’s so weird hearing you make documentary-style videos. I never thought they were your thing. But I think you do really well, I enjoy these videos!
This is actually really interesting how you can see super expert levels within the expert category - if the creator of the level of their level practices segements before attempting to upload it they will usually clear it relatively quickly possibly allowing the level to get misassigned
I have once encountered a very hard course with a high clear rate. It was an SMB course which was very short (one screen) and contained a giant pit filled with lots of boo rings and falling goombas and I had to jump on the goombas to the flagpole while avoiding the millions of boos, which is extremely precise. I couldn't complete it.
Wait so does that mean it is possible to get a super expert difficulty level on easy if the creator beat it first try? My that sounds scuffed. Anyway, great job on the video. I appreciate the amount of time you've put into research.
@raysfire This seems like a good machine learning task. Have you tried grabbing a programmer who knows a bit about data science and get them to run a regression algorithm on some of the data you collected? It could be a simple linear combination of some of those variables you identified. If I were developing a game and wanted to check clear difficulty (and had lots of time) I think it'd be really smart to take a recording of the user's clear and try to mutate it by throwing in random delays / extra button presses / skipping button presses to get a sense for how "precise" the level was. I really doubt that's what they're doing though (it'd be expensive / take a long time to upload a level most likely).
This was an amazing video, as you were explaining what your experiments were and what they ment I also found my self think "well what if..." "then that means..." and its as if you answered my thoughts! Kinda creepy XD but great minds think alike though. The script was awesome and kept me intrigued and interested the whole way through. I also loved how you showed the stats, really helped me follow along. Was worth the wait, I hope this video gets into the algorithm, it deserves it!
another potential theory: real ones remember the time alpharad deluxe did a co-op super world filled with a bunch of easy levels, but because of the ways they were set up (typically refreshing or auto levels) and the way co-op respawn works, it was near IMPOSSIBLE for them to beat the levels that way. they attempted an auto level for nearly THREE HOURS before giving up, switching back to single player, and clearing it normally. so it's possible that playing in co-op can also skew levels' difficulties, especially if co-op attempts count towards the clear rate. idk if it does, but it's just a theory. a g-
i remember this level was used by someone who made a video about how the clear rate works, and yeah basically if you die a ton on your level while trying to beat it to upload it, it gets ranked harder
My theory is that it starts out as the creator's clear rate, but after it's uploaded, it doesn't just determine the difficulty primarily off the level's clear rate, but instead the game might do 2 things: 1. Check for player outliers via their individual clear rate, and find the clear rate without those outliers. and/or 2. Take the average time to clear into account.
Wow, I like how raysfire is dressed up as an actual scientist. Edit: I subbed so I don't need to take part in the quiz because I won't wake up early for stream.
See I find normal significantly easier than expert. My normal record is currently well over 200 but my expert record is only 6. This is with skipping, which I use pretty liberally. I do agree though. Some normal levels could almost be super expert and I have played expert levels that are easier than your average normal level.
If you click on a users profile, the game keeps track of how many clears they've had in each respective difficulty - ie it tells you how many easy levels theyve beaten, how many normal, etc. Is it possible that these stats are taken into consideration? Like maybe if a player has beaten a ton of super expert levels before, the game considers them skilled at the game and how much of an effect they have on a levels chosen difficulty is altered.
The levels I make that I rush through to make a record for people to try an beat gets dunked on in sheer amount of plays from my one level I took a leisurely stroll through and took my time to beat.
to see the full, unedited "clear rate" experiment, check it out on my vod channel here: ruclips.net/video/JTA0wkdl9M0/видео.html ✨
also, big shoutouts to dsteves for all of the help with this project! a lot of the data and experiments i used in this video wouldn't have been possible without his help!!
check out dsteves' channel here: www.twitch.tv/dsteves 😎
Nice video btw
Great vid
nice vid keep up the good work
It would have been cool to go back in time and track the average clear percent over time. I have a feeling that since the game released, less people are playing and the people that are still playing are really good at the game. So perhaps the levels got harder to accommodate that crowd thus the average clear percent would go down over time.
@@cak5171 Check around 8:59 to see that DSteves cleared K. Wonka's level, too. XD
I think the initial difficulty seeding being higher than expected makes a lot of sense. As the creator, you know all the tricks needed for a level, so it's easier for you than it would be for a new player. Having the initial seed be biased toward higher difficulty is actually pretty smart game design.
Yeah. I always wonder how one of my levels was initially placed after upload since it took me one attempt to clear check it, but found out later that most people who played it struggled and gave up.
That said, I still feel like you could get a better starting estimate by taking into accounts the # of failures before a successful attempt when clear checking it. It would make it so that harder to clear levels tend to be higher difficulty on average.
And yet I've made many levels that took me dozens of tries to clear check and yet they have a 5%-10% clear rate.
Many creators aren't the best or most skilled and yet make levels that test them to their limits, whereas there are significantly better players out there that are so much better and can beat the level easier.
Beyond that, in my experience most levels are straight forward. No tricks, tech, or lore needed. Just standard levels.
I think the point you brought up is very cogent in troll levels, little Timmy levels with random hidden blocks, and very hard levels by amazingly skilled players/creators. For the rest of levels, which might be the larger portion, it doesn't necessarily hold true.
Initial clear check just shouldn't affect difficulty at all, actually. Bad game design.
@@Syranovæ If initial clear check didn't matter at all, it could be that most of the people would never get their levels in the random pool, because nobody could find them in the first place.
In SMM2 you need to get at least 1 play from somebody else before you get a new level into the random pool, and as a rookie level creator finding even that first player can be tough. I'm not exactly sure if the first play needs to finish the level.
Then again, when writing this it occurred to me that they could just use that first other player's clear rate to recalibrate the difficulty before putting the level into the correct random pool. They might actually do that, this video doesn't really confirm or deny anything related to that.
So what we've learned is that we need to throw when clear checking automatic levels to help no skip runners.
Comment from one verified user with 9 likes placed over multiple comments of unverified users with so much more likes? Clearly, RUclips is falling apart in terms of ratings of videos.
yo
And also throw triple shell jump, mashing, and near-pixel perfect precision levels
@@Fowlware fun fact: likes are not the only thing that RUclips looks at to bump a comment.
comment placement is decided based on likes and how often the user has a interaction (and maybe subscribers)
all songs used are from the kirby 64 ost:
0:00 - Mountain Stream
1:23 - Pop Star
2:51 - Rock Star
3:36 - Aqua Star
5:15 - Above the Clouds
5:57 - Boss Theme
6:45 - Neo Star
7:54 - Bumper-Crop Bump
9:47 - Ripple Star
10:49 - Shiver Star
12:29 - Factory Inspection
13:09 - Training
Thank you
based
Would’ve been a sin to leave Kirby 64 out of this 😝
Chad
The Kirby series has the best soundtrack, with the classic Sonic games being the second best.
One of the things we also talked about briefly was if levels were reuploaded exactly the same way if a level would differ in initial difficulty, then after the video was in production I remembered about title screens and how I've seen the same level in easy and normal and sometimes even expert, the spike test now does show that this is the case. All in all the goal is to find some sort of algorithm that combines all the pieces together. While initial difficulty on clear check is one piece of the algorithm it's figuring out a way to combine that, possibly the clear rate, and the ratio of who flagged the level as cleared over who has not, as well as figuring out any other overlooked part of the equation that is the key here to getting a sure fire algorithm going.
Since the number of tries it takes the level maker comes into play, I wonder how that has affected the categorization of my own uploaded levels. I often quit a clear check and nerf my levels a little (because I don’t have the patience to beat them lol) until I end up beating them on the first or second try after failing to beat earlier versions of them. So I wonder if Nintendo factors in ALL attempts on a saved level, or if the counter starts over when the level maker quits a clear check and changes the level slightly. One of my levels like this has a 4% clear rate, but I ultimately beat its final form in one try (after failing to beat earlier versions of it). So I wonder which endless mode it ended up in initially because of that.
Thank-you! I’ve been on the lookout for resources that would allow me to see stats on my own levels. My curiosity can now be satiated.
Hey man! It's me, hamstercol from twitch, how are you?
I think that the algorithm might ignore any clear attempts of a person after the person beat the level for the first time.
Another theory I have is that clears of different persons count differently based on some internal skill level value of a player. Somewhat similar to ELO
Hi biggest maker
I'm very curious if skips also factor into the Nintendo algorithm. For instance: take the puzzle level that was in Expert yet had a 17% clear rate, or the exceedingly long levels in Super Expert. Both would likely have very high skip rates, as many people simply do not like puzzle levels or lengthy levels. Someone may die once in a 2 minute long level and skip to the next one rather than attempt it again. This wouldn't impact the clear rate much, but it may have a bearing on the difficulty level.
Just a thought!
I had this same question but typically a lengthy level already indicates its difficulty. So if we take that into account I surmise they may have a separate yet similar algorithm to deaths that involve skips. Since every skip would be unique to a level, if a player didn't participate in the level there must be a reason why(which I would surmised levels in Expert and Super Expert get skipped over 5/10 to 9/10 times respectively)
If the player never clears the level like exit the software, it won't save like if you never played it, also The clear rate of 199,000 out of 400,000 is only showing every player's attempt to succeed, all the times the characters looses a life it counts as a failure point(meaning that is where the player's character fails there because of enemy, hitbox of icicle or liquid hazard),The shoe icon only shows how many people played it, and also the clear rate is everyone's attempt to reach the goal point there.
Never overestimate the clear rate if the player doesn't know what to do, give it a chance if you think you can complete it.
I had the same question while watching the twitch vod and I was really hoping someone would have brought it up live in twitch chat.
If I had to theorize, skips wouldn't affect difficulty as strongly as deaths but it would make sense to factor them in to a degree.
On one hand a skip could mean the player died once or twice and determined it was too difficult and skipped it or came across some difficult tech or some precise jump they knew they couldn't do so they skipped it before dying at all. In those cases the skip would represent the amount of difficulty the player WOULD have had if they had kept playing until they cleared the level. Either a great amount of time in the case of difficult tech without a way to die, or many many deaths in the case of precise jumps.
On the other had a skipped level may not be difficult but instead be too long, boring, or super annoying. The player could easily beat it, they just can't be bothered wasting their time on something they aren't enjoying. So these skips would be the opposite; they represent a hypothetical first try clear rather than an amount of deaths or struggle clearing the level.
So basically skips represent an unknown variable in the difficulty because we can't know the reason for every individual skip. In a hypothetical world where skips didn't exist and a level had to be played until cleared, a level's clear rate could potentially be drastically different than same real world level.
The more I think about it, the more curious I am on just how the algorithm DOES consider skips. Does have a way of predicting the reason why a specific player skipped a specific level? Does it gauge the player's own skill level based every other level they've attempted? Does it consider the amount of spent time in the level before skipping? Does it just give all skips a blanket weight and be done with it? Or did the programmers just say "hecc it" and leave skips out of the algorithim altogether?? 🤔🤔🤔
It's so interesting to think about and we will likely never 100% understand how the algorithim works. Unless someone at Nintendo leakes it out...
I think this would be grouped into the how many clear vs plays it has. Not the clear rate, but how many DIFFERENT players actually clear it. If the theory that levels that get cleared more often, even if the clear rate is low, are easier in difficulty, then skipping a level and never going back to clear it likely does increase the difficulty rating
What would be interesting is if they factored in player experience. An experienced player dying on a level could indicate harder level. An inexperienced player 1st trying a level could be strong evidence the level is easy
That was one of the things we were looking at also, but unfortunately there wasn't an easy way to test that - that's definitely a possibility in factoring in level difficulty though!
Probably wouldn't work as I am good at smm1 (so the skills carry over) but i have barely played smm2.
Agree with this. Someone who has beaten a lot of Super Expert levels should have a different weight in how a level gets classified from a newbie and how hard the level is for them. The algorithm should be looking at the reputation of WHO beats the level.
@Raysfire If the SMM2 scraped datasets contain the players who cleared each level, a correlational analysis could maybe be performed, but obviously this is well beyond the scope of a Twitch stream :)
@Bezray There would be outliers, but with enough data it would average out. And they could weight players less if they haven’t beaten many levels
The thing that the high clear rate super expert levels had in common is that none of them had any way of dying other than the time limit, so many people getting the level and skipping affects it (since skipping without dying doesn't count as an attempt)
I made the "Level 49*", thanks for playing and including this level in your experiment :)
I'm so glad you like the level.
Regards.
Something to consider: tags could also potentially have an influence. Many of your outlier examples were puzzle levels.
Thats basically 0% chance. There are so many levels that some are going to have the same tags as other but be completely different difficulties. There have to be some easy levels with the puzzle tag
I think the issue with puzzle levels is that people just skip the level if they can't solve the puzzle, lowering its clear rate
I figure your deaths to set the difficultly are probably reflective of the deaths per completion in general. Easy levels are ones where people generally win on the first or second try, normal on the third or fourth, and expert more.
The reason the clear rates are different is because of people who skip after failing the first time or who retry for world record.
Only people who go on to clear the level or who die more than the expected attempts factor into the difficulty.
This would explain why some levels that force a death to respawn from a checkpoint appear in easy or normal despite having ~10% clear rates, the levels consistently get 1-2 deaths per successful attempt and almost no-one dies more often(because it'd need timeouts or similar), but a lot of people quit when they realise the level is a net negative to lives.
The difficulty rating of chess puzzles can be calculated using a modified form of ELO from the players who succeed or fail at it. It would make sense for Mario Maker to use a similar system
Hello, I am Mr. Fusion Man. There are no fusions in this video. I hope a versus challenge comes soon or I'm gonna be out of a job.
Lol
Idk the twister and galoomba at 0:15 - 0:19 seem pretty fused to me
@@rorycannon7295 sadly, the twister is closer than the galoomba, they are not fused.
@@EnderrBoi i mean they could be shadoingled along the z axis?
@@EnderrBoi but tru
So that's why I could play endless easy and end up on a brand new level that's incredibly difficult, only to find a dev door several attempts later.
Very insightful into the algorithm, thank you.
I would like to clarify the conclusions we’ve drawn though- the only thing we’ve proven from our own experiments is that creator clear check attempts noticeably affects the algorithm’s placement.
The data science studies were useful, noticing trends of lower clear rates and higher world records tending toward higher difficulty placements; however, we cannot say that the algorithm necessarily checks for these things.
In order to show causation we would have to do experiments similar to the spike test but with multiple other players/accounts, keeping the creator clear checks constant.
I got that first ballon level in SE endless too! Also I lost all my lives because I kept having one coin left and the clouds left. Also thanks for this video I’ve noticed too and have been wondering! Sometimes I would get a level with one person who played it, and I don’t know how it Nintendo knew for sure that it was hard. It’s very interesting.
Lol I remember making that level, 0:20 didn’t think it would get on super expert
Fr?
Hey Rays. Loved the content. It's really nice to see someone put so much effort into their content. Keep up the good work Mr Shud :)
12:00 you can also try to match the WR time for easy,medium,expert,super expert
Like wait 120s to finish the lvl for super expert.
Idk tho
I’m glad you overcame your pc issues to upload this masterpiece! Love all the content you’ve been putting out
Rays this is an amazing video that you clearly spent a lot of time making. I loved the creative style of it! Keep up the amazing work keeping everyone here entertained and informed on everything Mario maker. ❤
I have always wanted someone to make a video just like this, thank you Raysfire!
All right, I study mathematics and the graph at 13:10 ringed a bell: a hyperbolic curve. If this graph is the way things work behind the scenes, then you can tell that the _product_ of the number of attemtps and the time per attempt is what counts - which is *the total time spent for the creator to beat the level.* I would love to have this theory tested.
Amazing video! I highly enjoyed it. More science, please!
What I learned:
When uploading a Super Expert level that you want to trick Nintendo into thinking it's Easy:
1. Make a copy of the level *before* trying to upload it.
2. Try to beat the level.
3. If you win, great, go to step 5, because you just uploaded it! If you lose, then delete the level.
4. If you're here, then go back to step 1, but with the copy of the level.
5. Watch as your level quickly gets moved to Super Expert because Nintendo's algorithm isn't stupid, and will notice when people are struggling.
Or just dev door it I dunno.
The way that all these new videos are amazing. Keep up the good work!
raysfire's content is top notch. you should have way more subs! keep up the good work
little timmies might leave their switch on at night and it counts as 1000-2000 of deaths in speedruns
Through the power of SCIENCE!
Holy quality batman, this is so much different than what I remember from watching you a year or two ago. And I'm all for it!
2:15 I look in newest and see like 15 comments saying spindel but nobody said
beeBobble
There you go, AComputerMouse
And now all I need is to get Mario maker 2!
YOU'RE GOING INTO SUPER EXPERT, SINGULAR SPIKE! This could actually be used to create break-levels for the harder difficulties. You know, give 'em a free win.
spindel XD (someone said say spindel in the comments 1:33 )
Why does the Easy text disappear at 13:36
really great video! was interested throughout the whole thing, and loved the methodology used to discover things. keep up the great work :)
I love how Rays is truly just a lad living his life
Spicy video rays! 👀 And awesome editing, loved the graph :D
Thank goodness for the sub! I forgot about Quiz day! Awesome Vid Mr. Shud! It's really interesting that the creator in a sense can partially decide where their level will show up!
Wow this was an amazing video!
I think I have to throw when doing clear checks on my kaizo's I build. I practice them so much in the editor, that I clear check them in a couple of tries. Then they get put into categories where people won't possibly like or be able to finish them. They're dead on arrival.
other ideas i can think of for possible factors might be things like hazards in the level, level size, time spent creating the level, amounts of inputs on clear attemps (for example just holding right, holding right and jumping some, or having something more complex than that)
also maybe the level creator's previous levels have an effect?
Love the video. The whiteboard needs to be featured more. Possibly for the Mario Party CPU tourney?
Man, it’s so weird hearing you make documentary-style videos. I never thought they were your thing. But I think you do really well, I enjoy these videos!
I wont lie Rays, Ive been loving this new type of content on your channel. Keep it up 👍. Also no test for me baby WOOOOOO
These new vids have been really great, you deserve way more attention man
This is actually really interesting how you can see super expert levels within the expert category - if the creator of the level of their level practices segements before attempting to upload it they will usually clear it relatively quickly possibly allowing the level to get misassigned
I have once encountered a very hard course with a high clear rate. It was an SMB course which was very short (one screen) and contained a giant pit filled with lots of boo rings and falling goombas and I had to jump on the goombas to the flagpole while avoiding the millions of boos, which is extremely precise. I couldn't complete it.
I love the nostalgic Kirby 64 themes in the background
Wait so does that mean it is possible to get a super expert difficulty level on easy if the creator beat it first try? My that sounds scuffed.
Anyway, great job on the video. I appreciate the amount of time you've put into research.
@raysfire This seems like a good machine learning task. Have you tried grabbing a programmer who knows a bit about data science and get them to run a regression algorithm on some of the data you collected? It could be a simple linear combination of some of those variables you identified.
If I were developing a game and wanted to check clear difficulty (and had lots of time) I think it'd be really smart to take a recording of the user's clear and try to mutate it by throwing in random delays / extra button presses / skipping button presses to get a sense for how "precise" the level was. I really doubt that's what they're doing though (it'd be expensive / take a long time to upload a level most likely).
This was an amazing video, as you were explaining what your experiments were and what they ment I also found my self think "well what if..." "then that means..." and its as if you answered my thoughts! Kinda creepy XD but great minds think alike though.
The script was awesome and kept me intrigued and interested the whole way through. I also loved how you showed the stats, really helped me follow along.
Was worth the wait, I hope this video gets into the algorithm, it deserves it!
another potential theory: real ones remember the time alpharad deluxe did a co-op super world filled with a bunch of easy levels, but because of the ways they were set up (typically refreshing or auto levels) and the way co-op respawn works, it was near IMPOSSIBLE for them to beat the levels that way. they attempted an auto level for nearly THREE HOURS before giving up, switching back to single player, and clearing it normally. so it's possible that playing in co-op can also skew levels' difficulties, especially if co-op attempts count towards the clear rate. idk if it does, but it's just a theory. a g-
THE WHITEBOARD AND LAB COAT REVEAL! I don't know if imma pass the test on Thursday? Is there any retakes?
It may be half 11 in the Queensland of England but rays has priority over sleep
I’ve always wondered this. Such a good video idea and well executed
1:31
Spindel
WindWaver426 you are welcome
Amazing video rays! Thanks for the science!
i remember this level was used by someone who made a video about how the clear rate works, and yeah basically if you die a ton on your level while trying to beat it to upload it, it gets ranked harder
these videos just keep getting better and better
One of the coolest raysfire videos yet! And the use of the Kirby 64 soundtrack makes it all the better!
11:06 if you want to do *SCIENCE,* THEN PLEASE CALL IT A FUCKING HYPOTHESIS AND NOT A THEORY 😭😭😭😭😭😭
such a high quality vid earned my sub good stuff
I remember playing a level never beaten before or two in normal mode. Always wondered how that worked
This is really neat! Thanks for the science, rayfsfire!
My theory is that it starts out as the creator's clear rate, but after it's uploaded, it doesn't just determine the difficulty primarily off the level's clear rate, but instead the game might do 2 things:
1. Check for player outliers via their individual clear rate, and find the clear rate without those outliers.
and/or
2. Take the average time to clear into account.
I always have great respect for Dr. Shud
1:36 Spindel
Wow, I like how raysfire is dressed up as an actual scientist.
Edit: I subbed so I don't need to take part in the quiz because I won't wake up early for stream.
i feel like this guy had a science assignment where he could do whatever topic and chose this
Spindel! (look at 1:35 for context)
Outside of your mutiplayer with red and panga this type of content is some of your best. Keep it up!
Woah very cool experiment! Love these videos, always good content and high quality!!
2:47 just over*
Joke's on you, I walked into the wrong classroom!
Worth the wait! Get this man a new chair!
10:09 does anyone have the level code for this level? I don’t think I can find it in the VOD.
To @AComputerMouse at the 2:15 timestamp, I gotchu buddy: beeBobble
this video is really good, and the kirby 64 ost just makes it better
I would consistently get levels I would call expert and super expert when playing normal. My normal record is exactly the same as my expert record.
Me too!
See I find normal significantly easier than expert. My normal record is currently well over 200 but my expert record is only 6. This is with skipping, which I use pretty liberally.
I do agree though. Some normal levels could almost be super expert and I have played expert levels that are easier than your average normal level.
Hype new rays video, hope it goes as well as the last couple
Thanks for validating me in my feeling that normal was way harder than easy.
Okay I have studied all night for the quiz on Tuesday, I think I'm ready
Great vid!!! hope this one pops off just like the last one
11:13 I like this video, but hearing you say that in a "science" video physically hurt me.
When you see your tool being used in a video: 😇
2:14 beeBobble
epic, keep up the great content.
I really hope I can skip that test on thursday! I subbed, I promise!
1:05 KITCHEN REVEAL!!!!1!!
Always a good day went you got content from ya boy rays
You looked like EazySpeezy when you had those goggles on 😂
I got a "First Clear" in Super Expert once. The stage was literally so easy
BANGER VIDEO RAYSFIRE ‼️🗣
1:35 chat boy spindel
If you click on a users profile, the game keeps track of how many clears they've had in each respective difficulty - ie it tells you how many easy levels theyve beaten, how many normal, etc. Is it possible that these stats are taken into consideration? Like maybe if a player has beaten a ton of super expert levels before, the game considers them skilled at the game and how much of an effect they have on a levels chosen difficulty is altered.
2:07 🤨🤨🤨🤨🤨🤨🤨🤨raysfire?????🤨🤨🤨🤨🤨🤨🤨🤨
i'm 11 months late... is the quiz still on thursday?
The levels I make that I rush through to make a record for people to try an beat gets dunked on in sheer amount of plays from my one level I took a leisurely stroll through and took my time to beat.
scanning for shudheads 📡
shudhead located 📍
I love videos like this
“Spindel”
13:36 'Easy' has left the chart
shudman scientist cosplay is juicing