Activision's SBMM paper analyzed by scientific researcher

Поделиться
HTML-код
  • Опубликовано: 4 дек 2024

Комментарии • 42

  • @Hybred
    @Hybred  4 месяца назад +6

    If you liked this video check out my analysis on how TTK affects SBMM: ruclips.net/video/3ob7NV830V8/видео.html
    Also allow me to clarify & summerize some thoughts - I feel like I could've explained some things better since I did this all in one take (I'm too lazy to video edit)
    - Their ping data is obfuscated too much. They don't show exact ping values, they have the data in aggregate form, and the study is misrepresentive of player experiences by only speaking about averages when theirs plently of people from low populated areas like New Zealand who get 150 ping because SBMM refuses to match them with nearby players. This feels intentionally misleading, because it presents itself as if no one could have this sort of experience with their system when a lot of people do.
    - The study was short term, but Activision claims these results would compound overtime with no evidence. How did they extrapolate this? With no data and no study being done, it's safe to assume it's their assumption based off the current trend before they stopped the study. But theirs no way to actually know this, the only real examples we have are from past Call of Duty games with different SBMM systems, and those systems have the inverse effect of what Activision claims will happen in current titles where the population does not become only too good / sweaty players within its expected life cycle (Which I consider 3 years since that's when the studio releases their next game).
    - Biased PR language, not going in depth or showing graphs and data for information that may show any evidence to the contrary. If this paper was published under any professional body it would fail on so many levels for this, it's called selective data presentation where any data points with unfavorable results are omitted. They have so much data but show so little, if all the data in their tests proved their point they would be showing you all the data, why wouldn't they? It only benefits them, so you should be skeptical, and this is why it wouldn't pass peer review.

  • @primepete
    @primepete 4 месяца назад +7

    EXCELLENT video. Watched and took notes on the whole thing. This will definitely help down the line in future videos. If I end up using this info I'll cite you. Great work.

    • @Hybred
      @Hybred  4 месяца назад +4

      Thank you! I'll summarize some things because I feel like I got tongue tied at a few parts and could've explained better since I did this all in one take (too lazy to edit)
      - Their ping data is obfuscated too much. They don't show exact ping values, they have the data in aggregate form, and the study is misrepresentive of player experiences by only speaking about averages when theirs plently of people from low populated areas like New Zealand who get 150 ping because SBMM refuses to match them with nearby players. This feels intentionally misleading, because it presents itself as if no one could have this sort of experience with their system when a lot of people do.
      - The study was short term, but Activision claims these results would compound overtime with no evidence. How did they extrapolate this? With no data and no study being done, it's safe to assume it's their assumption based off the current trend before they stopped the study. But theirs no way to actually know this, the only real examples we have are from past Call of Duty games with different SBMM systems, and those systems have the inverse effect of what Activision claims will happen in current titles where the population does not become only too good / sweaty players within its expected life cycle (Which I consider 3 years since that's when the studio releases their next game).
      - Biased PR language, not going in depth or showing graphs and data for information that may show any evidence to the contrary. If this paper was published under any professional body it would fail on so many levels for this, it's called selective data presentation where any data points with unfavorable results are omitted. They have so much data but show so little, if all the data in their tests proved their point they would be showing you all the data, why wouldn't they? It only benefits them, so you should be skeptical, and this is why it wouldn't pass peer review.

  • @AbcdEf-lz6oe
    @AbcdEf-lz6oe 4 месяца назад +6

    Good vid on analyzing the research. It’s nice hearing the application of research methods being applied to the paper and showing its flaws instead of conspiracy rants.

  • @Davud-nb7yh
    @Davud-nb7yh 4 месяца назад +4

    great video!
    Playing COD games nowadays, feels like the game gives you JUST ENOUGH dopamine to keep you playing but not enough to have fun. so while you'd be playing more ( in short term at least) you're not enjoying much & you're being kept in this miserable situation only for a chance of buying something.
    and i would say being kept playing while miserable makes it so the only way to have some fun is to buy something. it feels very very malicious and it's certainly not fun, that's why i haven't bought since cold war.

  • @maserati_lukas801
    @maserati_lukas801 4 месяца назад +1

    As a student of political science, thanks for making a video with a Scientific view instead of a 10min rant video 🤝😂

  • @Tealdragon204
    @Tealdragon204 4 месяца назад +2

    To be fair. As much as some people play COD competitively, it has always been a game that appeals to a more casual playerbase. So it is not a surprise that they would dislike a lack of SBMM. I just think many COD players are in denial about how good they are
    I'm saying this as someone who doesn't play COD but I play competitive shooters (I'm above average there but I'm not terribly skilled yet). All my friends would go on and on about how great they are at COD and whenever I joined their lobbies I'd do well (2 K/D is I assume at least pretty good for COD) and they'd consistently go negativde
    They didn't enjoy playing COD with me to say the least. But I think that goes to show that a lot of COD players (perhaps due to SBMM) have a inflated sense of ego when it comes to gunskill. I'm not even an amazing FPS player in the first place. I'm maybe top 15% in any game that I play (and top 10% in the games I play a lot) but I think you have a lot of people in the 25-75 percentiles who think they are far better players than they are
    I think COD is right. Casual players enjoy SBMM. But the inherent issue with SBMM is that you never get better if it's hidden. At least with ranking systems there is something to push towards and an attempt at accurately assessing your skill. Instead ghost SBMM makes it feel like any given match is just down to luck and not skill. People never improve, they just complain about the other players that beat them and think they're God when they face bad players
    But as the XDefiant dev said. If you can't find any bad players to kill, that's because you are the bad player

    • @Hybred
      @Hybred  4 месяца назад +3

      COD has always been a casual shooter correct, but the casual nature of COD existed MW2019 and arguably those periods of time were more casual than now with stronger SBMM.
      That's why the video highlights stabilization over time as point, amongst other things.

    • @elchinuoficial
      @elchinuoficial 4 месяца назад +2

      Back in the Black Ops 2 days we had League Play when it was as competitive as it gets, most games were close because it had players of equal skill. I agree COD was never competitive worthy compare to other FPS games, but back then you had a fair SBMM and not this rigged and manipulative matches with EOMM.

  • @bryanbruebaker1507
    @bryanbruebaker1507 4 месяца назад +4

    Exactly what I thought when I read all of the SBMM papers. They're tailoring the data to present it to their argument. They aren't highlighting the negatives or the fact that regardless of SBMM or not, all games lose players over time. Stating that 90% of players leave the game after blowout matches is redundant information. The franchise is done for me. It's not a special game like it used to be. It used to have a unique identity but since they've tried to take up the Battlefield market, it doesn't feel unique anymore. The game should have stuck with hitscan and a realistic tone with no SBMM. The seasonal model is tiring on top of a $70.00 purchase. There is no reason to buy the games anymore outside of Zombies (when it's good, unlike Vanguard and MWIII) or the fact that it has a massive playerbase.

    • @primemachine146
      @primemachine146 16 дней назад +1

      Just wanted to state something, bo6 uses a hybrid system, it’s used hit scan but only to a certain range then it switches to projectile, I’d assume this is done for keeping multiplayer mostly hit scan and having projectiles for those long engagements in warzone

  • @caseycameron5370
    @caseycameron5370 4 месяца назад +1

    I came here from prime petes channel

  • @NicolasPowell-wl2ch
    @NicolasPowell-wl2ch 4 месяца назад +3

    I was a top 100 SnD player on BO3, when I was playing solo (90% of the time I was solo queue) I would suffer with high ping games in off peak time 10pm onwards. I’m from Australia and at the time was living in a smaller town. I loved the feel of the game, but the lobbies were so swayed. I would often play against full teams, which is hard even if they’re terrible because SnD is dominated by map control which comes easily with coordination.
    SBMM seems like it suits the bottom 40% and gives diminishing quality of returns as you go up the skill ladder. Top 1% is hell without at least 1 person of even skill or a full team

  • @mobarrera5850
    @mobarrera5850 3 месяца назад

    Nice video

  • @israeldavila27
    @israeldavila27 4 месяца назад +11

    You want to know what it's REALLY going to be like for an average player in this SBMM hellscape? Picture this: You're a hamster on a wheel, running faster and faster, but never actually getting anywhere. That's SBMM in a nutshell. See, Activision's throwing out this non-peer-reviewed "study" like it's the gospel truth, but it's about as scientifically sound as using a Ouija board to predict stock market trends. They're cherry-picking data faster than a kid in an orchard, conveniently ignoring the long-term effects and the ACTUAL experiences of players.
    You, the average Joe Gamer, are caught in the crossfire. One match you're stomping noobs like you're the second coming of Shroud, the next you're getting your virtual pants handed to you by a squad that seems to have more fingers than the rest of us. It's a rollercoaster of emotion that'd make even the most hardened COD veteran reach for the Dopamine. And don't even get me started on trying to play with friends. It's like trying to fit square pegs in round holes while blindfolded and standing on one foot. The system just BREAKS. Activision's patting themselves on the back, thinking they've created some kind of matchmaking utopia. Meanwhile, the rest of us are stuck in what feels like a never-ending sweat lodge, wondering if we'll ever see a casual match again.
    The harsh, unvarnished truth about SBMM, served up with a side of skepticism and a heaping helping of "this needs to be fixed YESTERDAY." But hey, at least we're all suffering together, right? ...Right?

    • @bryanbruebaker1507
      @bryanbruebaker1507 4 месяца назад

      I think it's time for us to move on. I hate to, because this franchise means so much to me. It helped me get through my awkward high school years. It allowed me to feel good at something. It allowed me and all of my friends to relate to something and have something to do when we weren't hanging out at each other's houses. I used to look forward to the annual releases. But I now see that as an above average player that has been playing this franchise for fifteen years it will never be like how we wish it to be. I keep hoping that they will fix the franchise but since Modern Warfare 2019, I see that they have their direction, and are not deviating. I guess the reason why I think we should move on is because it's preventing us from fully enjoying other games out there and also, if we miss the old games so badly, we really should just populate those ones. Like what happened with Black Ops II three years later with 300,000 people playing it on XBOX. Anymore, if I want to play Call of Duty, I will play the ones that I know I will enjoy.

    • @MatthewCenance
      @MatthewCenance 4 месяца назад +2

      That doesn't sound like "skill based" match making. Lurching around from single-handedly defeating unskilled players to being pummeled by experts doesn't sound right. Real skill based match making would be trying to match you with players of similar skill most of the time, not mainly players with drastically higher or lower skill than you.

    • @juzedhere1984
      @juzedhere1984 4 месяца назад +1

      @@MatthewCenance that s how their SBMM/EOMM is working, if you have a kd of 2 for a few matches, then they give you harder matches where the enemy team is 2 times better than u, while you get people that either use their feet to play the game or you get people that have the same situation as you in ur team so that u get a kd lower then 1, for example when u get a 0.5 kd for a few matches while facing better players then u, they give u easier lobbies so that you can come back to that 2 kd and feel good about it
      this is a never ending cycle

    • @MatthewCenance
      @MatthewCenance 4 месяца назад +1

      @@juzedhere1984 Skill based matchmaking is not supposed to work like that. It's supposed to be gradual. Online chess matchmakers don't match you with players at ELO rank ~2000 because you beat 3 ELO rank ~1500 players in a row easily, unless you're very new to the matchmaking system and it needs to find out quickly what ELO rank you should be at for accurate rankings and matchmaking.

    • @vytah
      @vytah 4 месяца назад

      ​@@bryanbruebaker1507fun fact: BO2 also has SBMM

  • @hydb801
    @hydb801 4 месяца назад

    they discuss what makes up skill and how they average out skill in a party in the whitepaper

    • @Hybred
      @Hybred  4 месяца назад +3

      That's what makes up skill, not how they define skill. That's what I said.
      Its like a vaccine study defining the term "fully vaccinated", some just mean being vaxxed others may mean vaxxed + boosted. From a scientific standpoint, a paper about skill based matchmaking needs to give a definition of skill.
      Because despite calling it SBMM, they justify their findings based off engagement, their SBMM is tailored for best engagement. So it seems to be EOMM that considers skill as apart of its engagement process, not SBMM itself.

  • @TennessseTimmy
    @TennessseTimmy 3 месяца назад +1

    Cod match making is not skill based, skill is just one of the input values in their match making.

  • @aquadragon7045
    @aquadragon7045 4 месяца назад

    Wonder if game crashes were factored into people leaving. I get suspended for 2hr every time on ranked because crashing is STILL an issue and has been an issue for a long time with this game.

  • @MGrey-qb5xz
    @MGrey-qb5xz 4 месяца назад

    do you think this game would be on par or near black ops 2 level?

    • @Hybred
      @Hybred  4 месяца назад

      Which game?

    • @MGrey-qb5xz
      @MGrey-qb5xz 4 месяца назад

      @@Hybred oh lol , the upcoming black ops 6. should have said that

    • @Hybred
      @Hybred  4 месяца назад +6

      @@MGrey-qb5xz I think the TTK is slightly too long for having advanced mobility like omni-movment. It needs reduced by at least one shot. So gameplay wise that will be less fun pairing those two together.
      The other thing is recognizing what made old games like BO2 so special, and the answer to that is social features. I think modern censorship and disbanding lobbies ruin social features of a game, everyone's brand new each match, no one talks because their communication banned or afraid they will be, when they do no one can banter with each other, etc. This kills the charm of the game, sense of community and makes it harder to make fonde memories. I made so many friendships on BO2, never had any on a newer COD title.
      So in other words - even if gameplay is as good, you will never recapture that magic. Gaming is too corporate and out of touch. The gameplay may be fun, graphics pretty, but the important elements, the ones most neglect to consider because it's not obvious are gone.

    • @MGrey-qb5xz
      @MGrey-qb5xz 4 месяца назад +1

      @@Hybred well said more people need to spread this kind of awareness

  • @TennessseTimmy
    @TennessseTimmy 3 месяца назад +2

    This entire thing is stupid.
    Anyone who has played cod in the last 5 years knows that it's engagement optimized Mm.
    Play the game and see the clear pattern of rigged games

  • @MatthewCenance
    @MatthewCenance 4 месяца назад +2

    Is modern Skill based match making properly skill based, or an attempt to force sweaty matches?

    • @Davud-nb7yh
      @Davud-nb7yh 4 месяца назад +2

      to me it's not about skill at all, if it was about skill the matches would be relatively the same, but what happens is you have 1 great game and then 3 terrible games or sth like that.
      feels very very manipulative to keep you playing (whether you enjoy or not). i think the ONLY thing they care about engagement which translates to a chance of buying something.
      (and hell maybe not enjoying the core game makes you more likely to buy)

  • @sharkymb8552
    @sharkymb8552 2 месяца назад

    On top of that, the first few games you play everyday are always against low skill opponents.
    This is very obvious in apex legends, firdt 2 games are almost always wins then they put you in the sweat lobbies
    This is to milk thier casual playerbase where they make the most money
    This also includes artifical lowering the skill ceiling of the game because it helps them protect the casual player base

  • @sudd3660
    @sudd3660 4 месяца назад

    any online game must have strict/accurate SBMM, you can not let the skilled player play with the beginners!
    i am not feeding the high skilled players for years, and no one likes to be farmed. and the casual and beginners are what makes up a game, those are the people that matter.

    • @Hybred
      @Hybred  4 месяца назад +2

      This is just a data driven video not an opinionated one, but strict SBMM isn't as good as it seems. You see no personal improvement as you get better at the game. It's like if you get better at your job so they give you a raise, but then you're taxed the exact amount of extra money you're given thus it's as if you got no raise at all. While it's great when you're starting, its punishing when you're no longer a beginner.
      A SBMM system that protects new players is probably ideal for your purposes, so you can experience the game for the first time with others and have room for personal growth

    • @sudd3660
      @sudd3660 4 месяца назад

      @@Hybred You see personal improvement as you get better at the game. sbmm just makes you see players closer to your skill level.
      and we can not let only the few percent at the top get a good experience and the rest is farmed.
      for personal experience i have thousands of hours in apex legends, i still need accurate matchmaking to have a good experience, i also have teammates that needs to be on my level. and not encounter one player that farms the server and kills my whole team.

    • @Hybred
      @Hybred  4 месяца назад +2

      @@sudd3660 you don't see personal improvement if your competition gets better with you, unless its loose SBMM which I'm okay with. You specified strict SBMM, which to me means it will try to make it as perfect as it can within reasonable queue times, no wiggle room otherwise. In that scenario you would perform the same despite getting better, unless you're a top 1% player because then theirs not anyone better than you really so you'd still have people to "farm".
      I also don't understand these anecdotes since pre-MW2019 which is all COD fans want it's not like COD was some sort of noob unfriendly, hardcore competitive video game with only sweats. Most people just say remove SBMM as a trend but most actually just want the old system back, which was fine. No one complained it wasn't strict enough.

    • @spyrothedragon5057
      @spyrothedragon5057 4 месяца назад +1

      I disagree. If you have a separation between casual and competitive play in your game, I do not think there should be strict, if any, sbmm in casual(almost all cods have ranked modes, if not at launchg for whatever reason). Casual, is meant to be just that, casual. You aren't meant to take it super seriosuly. So, when strict sbmm comes into casual and forces you to always play against people really close to your skill level, it will always feel sweaty and competitive because you have to do your absolute best to win against players of the same skill level. And because most gamers are competitive, in the sense that they will want to win, even in a casual mode, you won't ever be able to relax. If you have loose sbmm or none at all, you can get matches where you can chill and get carried, or where you are so doomed to lose that there's no point in trying to win, just learning from the better players(another point, the best way to learn a skilled game is to play against people of higher skill than you). Basically, sbmm actually makes casual games/modes feel much more sweaty overall than if you were to play with no sbmm.