@@deefeeeeefeeeeeeeeee I'm sure he's aware that random numbers generated by computer aren't actually random because computers aren't capable of generating random data, but it's close enough for the purposes of game design. Very little that we can actually observe is actually random; we just can't see all the variables. The point, however, which you seem to have missed in your attempt to look clever, is that in a distribution of randomly determined numbers where no determination influences nor is influenced by any other, the results often do not feel random, even though they actually are, and in order to get results that feel random, you have to reduce the randomness of the determination.
@@deefeeeeefeeeeeeeeee he doesn't assume anything, he specifies what 'random' is in PCs right at the beginning. I dunno who hurt you but you should stop projecting.
@@deefeeeeefeeeeeeeeee he isn’t talking about the mathematical concept of true randomness, he’s talking about game design. He’s using the term to describe a game design philosophy. If you had better common sense you would know that.
This really touches on a bigger issue in not only game design, but so many other jobs that involve different people needing things from each other within the company. _"Understanding what people want, rather than what they are asking for."_ I work as a Data Engineer/Analyst, figuring out how to get data together into a usable space, and then also figure out what we can get from that data and put it into a report that someone can use for their job. Almost every single project starts with "I need you to get me ". A lot of analysts or engineers will hear that and immediately delve into whether we have it, how to get it, how we'll get it working and get it into a reportable and understandable way, etc. However, what should be done next is to ask "What are you needing this data for?", or a better phrase _"What question are you trying to answer with this data?"_ This is crucial, because there are so many nuances that could dictate how you proceed.. and I've seen so many projects get extended over and over as the analysts keep presenting their work and the person requesting it keeps sending it back with changes needed because it _doesn't answer the real question._ A good project manager who has the authority to say "No" and ask the real questions can be a MASSIVE change in how projects get completed, and the final product. Spending a single sit down at the start, figuring out the real need in question, and then properly directing resource to fulfilling that need. When I see supposedly finished games coming out these days with their disjointed mechanics, dropped features and half-baked third acts.. I sense that there were a lot of siloed development teams that spent a lot of time spinning their wheels because they had no one to properly manage the projects and bring things together. I tend to feel that the small team or solo dev indie games (the legit ones, not money making churn), while smaller in scope, tend to be tighter and more cohesively designed because there's usually one guy who directs his vision all together properly. Less chance of making what they "asked for" instead of what they really "wanted", when it's all the same person, lol. Developing the skill to figure out what people actually need instead of what they asked for (either through a deeper understanding, or via good diplomatic social skills) is invaluable in any part of your life that involves "someone asking you for something". Which is almost any job, and a lot of inter-personal relationships. Parenting too, now that I think about it, lol.
I usually trust the math without question until it happens to me. Sometimes it just feels SO bad when it happens that your brain is like "No, this can't happen! Not to me!"
@@cullenclark8581 I think the XCOM devs have said that players found true 50/50 odds unfair. If they were given a true 50/50 chance and told it was 50/50, they complained that it wasn't 50/50 and felt stacked against them. The devs found that the odds had to be significantly stacked towards a positive result to match what players felt 50/50 should be.
One thing I've learned about randomness and probability rolls is that the dice will do whatever they want until it reaches infinity rolls. When I dabbled in board game design, you learn a lot about randomness in so many different contexts. Tabletop and Board game design is something I wish a lot of video game designers would even take a month of playing around in, just to experience the differences of randomness mechanisms directly rather than it being obscured by the computer. Random draw and random roll, as you demonstrated, are vastly different.
Man, the internet is so cool. Your games made my childhood I never thought I'd get the chance to just hear you talk about the processes that went into them
Reminds me of when I played Candyland for the first time as an adult and realized that the instant you shuffle the deck, the game is decided and the whole thing is just drawing cards and seeing who already won. Barely even a game there
in the 80s, tetris used to be pure random but somewhere in the 90s they swapped to something the fans call a seven bag shuffle. it creates fewer moments where it feels like the game is actively working against you, but I miss the fun thrill of getting 3 or even 4 of the same tetromino in a row.
Tim, all these talks have made me really regret my decision to leave the gaming industry (I worked as a sound designer). I've even gone so far as to re-download Wwise and FMOD in order to refresh my memory with those tools, and I'm thinking about updating my resume/portfolio. I don't know if I'll go through with it yet, but truly, thank you for re sparking my passion.
Similar with me, these vids re-invigorated my interest in software development. As he was describing the demands he was receiving in this vid regarding “random” selection, I already started mapping how I would write that in my head. I’ve already started working on personal projects.
Tim, I have talked to so many irrational people about this over the years. The people who rage about RNG generally are just not capable of understanding or accepting it. I've found that no matter how you explain it, they still think the generator is broken.
one of the best, funniest and at the same time truest stories I've ever heard.. as a programmer I come across this a lot, I've even been yelled at when I did exactly what they asked me to do.. the absolute highlight was when I wrote working code, my boss rewrote it (of course he made an error), then he fixed the error and came to me saying that I had an error in the code, but he already fixed it for me :D and the biggest takeaway from this is: never do what they ask you to do, always try to find out what they want you to do.. applies to everything: life, work, partners, etc..
"Well, let's do math. I love math!" Reminds me of how Jonathan Blow is (apparently still) mad at Sid Meier for suggesting that RNG shouldn't be exact because it makes players feel bad. And you're right, it's absolutely a communication issue. I've spent a lot of time in my career translating between programmers and non-programmers and sometimes there wasn't even anything to translate; both sides wanted the exact same thing, but the language they were using was causing the other party to think the opposite. Some of my favorite moments involved programmers telling me something wasn't possible, and then watching management spin their wheels endlessly trying to get what they want rather than accepting they can't have it their way. Management behaving like children...I feel like there's a corollary in here somewhere 😂
I've spent a lot of time translating, that's an important skill. _especially_ when dealing with programmers, but it applies to a lot of fields and jobs. If group A member talks directly to group B member, effective communication is not achieved. It takes a translator, as you say, to understand what group A wants/needs from group B, and then being able to put that into the right language and context for group B member to understand. It seems like it happens bajillions of times, and a lot of people _don't_ understand that this is so frequent, and it leads to less effectiveness, less efficiency, lots of hiccups in whatever business or goal that everyone shares.
This is a great example of other departments not knowing what they want or not knowing how to express (logically) what they want. Interesting stories! I think a lot of times these happen naturally as an outcome of "changing their mind" when a team is developing a feature or set of features which are not always implemented perfectly on the first iteration. Adding to that, changes also tend to happen when the feature has been tested or played around with enough to know how it "feels" when played in "developer brain mode" versus "player brain mode" as a designer.
Excellent video, Tim! I run into this all the time at the studios I work with. The last part especially resonated a lot with me. I often describe it as "listen to their issues, but ignore their solutions", because most of the time those would be "patches" that would fix the symptom, but not the disease, and it would still cause problems in other parts of the game.
9:44 I was like "Oh my God...."... and then Tim did that facepalm at the same time... XD... All I want to say is that all your hard work was not in vain! Keep up the good work (and the patience hahahahahahah)..
I'll never forget the test case I wrote after coding a randomizer for a tech tree at work. It said, "Verify that the random results match your internal model of randomness."
I play old school runescape where many items are highly rare drops with odds of 1/500 or so on a boss. Many people will start to complain once they have 500 kills, feeling that they are "dry" and should have gotten the item by then. But the reality is that going up to 1500 kills, ~5% of players will still have not gotten that item. When you consider that hundreds of thousands of people have played this game over the years, 5% is a VERY large number of players. I personally prefer it compared to having some sort of upper limit or increased odds per kill, and I always feel better when I start looking at the math behind it.
I rememver a D&D session my mate had where he, as DM, was rolling for an NPC and he rolled a 1 and then rolled another 1 (to see how bad the results of the critical failure would be). Two Critical Fails in a row. This is why randomness is good.
saw this a lot during my time designing webpages people dont know how to express or describe the kind of website they want, how they want it laid out, how they want it to look, etc. what ive found to be most successful is work through how they want the experience to be, and then draw up a prototype for confirmation. i know that option isnt always practical in a development environment, but it saved me time from having to go back and forth. ive heard its pretty much the exact same for artists, too.
YES!! I love your examples. lol!!! I wish I had your videos 10~20 years ago. lol One of my math trick for "random" is circular increments. Let's say I'm simulating a D20 roll. Previous roll had the number n. For the next roll, I'd do "(n + rand(19)) % 20 + 1)". That achieves the non-repeating roll without needing control statement or a loop. :D
I love this topic so much, it revolved around the perception of fairness which sadly isn't a healthy topic to speak to everyone on because it tends to bleed into the Confirmation Bias territory. Also in discrete math, calculating your chances of a winning hand in blackjack changes depending on if you calculate it before or after the cards are initially dealt.
Very interesting. I've seen something similar on Q and A sites, where people ask for very specific solutions instead of explaining what they are trying to achieve with it.
Classic! This exactly happened in our first game, Vaporum. One of my colleagues came and said: "Your random system is bullshit! I have an 85% hit chance and I missed 3 times in a row!" I tried to explain that that's how randomness works, but to no avail. And this is universal, not just my colleague -- most people think that the scenario I described is BS. It simply feels unfair. I then later reworked it to mostly what Tim describes here, which is not random, but "feels" better, and everybody was happy.
But why did you let them fall into the trap? You should have known that and be more supportive. This: "Make feel better" is what is expected from a game. The simplest step would have been to ask whether a mathematical correct random generator is really wanted.
As you were telling your story, I'm thinking to myself "that's not random. Oh, neither is that..." and ways I would design it. I like randomness as you designed it, but with a caveat. When loot doesn't drop or you miss when you are likely to hit, add a modifier that adds a percentage or point into loot quality or critical hit chance that resets after a hit or loot drop. It sucks missing a shot or not getting loot or bad loot, but getting that rare drop or that juicy critical after a bunch of nothing makes it much more satisfying. Mitigating disappointment is better than "designed randomness" in my opinion because once you learn how it was designed you can start gaming the system and takes the fun out of it.
it depends. for the music playback issue, curated randomness is exactly what you want. hearing the same song twice in a row gets boring or feels artificial, and same for hearing the same sequence of songs twice in a row, to a lesser degree. most things we like are a mix of determinism to rule out uninteresting outcomes, and randomness to add novelty and excitement.
As an RNG designer, none of this came as a surprise. Random shuffles and Ornstein-Uhlenbeck processes (en.wikipedia.org/wiki/Ornstein%E2%80%93Uhlenbeck_process) are your friend for natural feeling event creation.
Really enjoying hearing about the more technical programming aspects of game development. It's nice following along with your thought process and what you're doing in the code even with simple things like arrays and RNGs.
"When almost anyone says to you: 'I want randomness', no they don't; they want shuffled permutations. This is smart people using a word that doesn't mean what they think it means. I learned to listen to what people wanted, and not what they were asking for. Which, by the way, is a very good skill when you are reading reviews about games you made. People will tell you they want something, but it's not what they want, they want something else. Either they don't have the vocabulary, or the self-reflection, or they just can't even put into words what it is they want. But frequently, people ask for things, and it's not what they want. They want something else, and it takes a while to extract that from them. What you need to learn is how to ask for things which are exactly what you want. Not: 'this is what I'm gonna ask for, and you should figure out what it is I want.'"
Tim is 100% right about this being a gap in word meaning. Colloquial meaning vs technical meaning is one of the problems that plagues most languages in general. And, in this case, there's one particular synonym of "random" that I think might actually help pinpoint the communication gap even more: "designless".
The "colloquial" meaning of random is just incorrect. Randomness is mathematically well-defined and fits an intelligent person's intuitive idea of what randomness is. "Designless" does not describe it either because the examples Tim gives are clearly "designed". Some people just don't understand basic maths.
@@slynt_ However, Tim could have asked before implementation whether the game designers really wanted a mathematically correct random generator. Especially when it is known that terms are understood differently and you want a random generator for a game that is ultimately supposed to be fun.
I personally find true randomness in games a very frustrating thing to deal with since it normally leaves players feeling screwed over with things like hit chances and drop rates. I find myself wishing more games would use your shuffled permutation example. I usually think of it as a 'randomly' generated playlist that gets recreated on completion because my old CD player used that terminology :). I realise the point of the video wasn't about the merits of true randomness but I did wonder how many other people came to that terminology due to old CD players haha.
The first permutation itself is random. There are games (Colony Ship comes to mind) that offer different methods (true random vs shuffled vs some other skewed random).
If I've learned anything from going out in the professional world it's that most people don't understand basic math concepts. I invite people to events for my job. We've had people say they only want managers to be invited, fine... but then they only want managers from this state, and only if they are in this specific kind of field... Not realizing that every qualifier they add is like another circle in a venn diagram and the people that fit all of these categories are exponentially reduced each time they add another. So I went from being able to invite thousands of people potentially to like... 10.
Hey Tim. Both Fallout 1 & 2's random factor regarding combat was hilariously brutal. As frustrated as I have been while playing the games for all these years, as much fun has it been to "reminisce" with other old fallout players about exactly _that._ Not just missing 3 shots in a row. But having bullets "boomerang" around and hit a companion/a companion hitting me standing in the background. Ian was notorious for shooting you in the back in F1. Sulik was notorious for boomeranging burst-fire with sub-machine guns in F2. Taking on a companion in Fallout was always a bit of a 'Monkeys Paw' deal. :p
Critical failures were such an interesting way to add difficulty to combat. Not just having the enemies with more health, or adding more enemies, no getting around it by min maxing damage or exploiting AI. Now you have to manage wasting ammo and reloading your clip early, or having to run and pick up your weapon in the middle of combat. Very unique, and something that always left a fondness behind.
3 times in a row isn't even that much. 1/20*1/20*1/20=1/8000 ... but people think they are going to win the lottery :D I mean these are things I knew already about people, but this gives me another perspective on how we humans think sometimes :D
Save scumming was hard in xcom but not impossible. With a few save points that give you space to maneuver before getting trapped into predetermined rolls, you can avoid calamity. That or bomb the heck out of everything and let RNGesus sort them out.
It's really funny, I just watched that talk you mentioned yesterday. These stories prove that programmer brains just work differently than everybody else's XP
On a related note, your rng seed handling is top tier. More than 20 years after Arcanum release, BG3 just dropped with loot in chests being regenerated after a reload. Unbelievable.
it's intentional so that people dont feel bad if they dont got what they wanted and reload the game to get better result, and _"honest"_ players wont do it, it's not _"Unbelievable"_ it is 100% intentional not to store the random seed in a save file to keep the game player friendly most modern games track the player's success and increases it in case of series of bad result so that the player feel better and not being cheated by the game, the randomness Tim talks about basically does not exist in modern games because developers dont want people get too upset, in xcom 2 on normal difficulty if you miss a shot the game secretly increases your hit chance (while displaying the old lower value) so you will feel you being soo good 😃
How many "reload sets" are there though? There are so many items in BG3 and DOS that I wouldn't be too quick to shit on it as it might be rather inconsequential. Also, I forget, is Luck in BG3? I've only gotten a few hours into it so far. I would also say that reloading for a new set would be pretty boring and tedious for the reason I stated initially.
@@Arkanthrall Probably because most players don't care enough to reload for some arbitrary chest. I guess maybe if it was one at the end of a dungeon or something it might be worth it. Personally I wouldn't care to reload just for loot anyway.
the stories about the devs you do are very fun to listen too! its kinda funny to hear about how a common problem seems to be developers knowing what they want(i want the consumer experience to FEEL random, but not give the negative feelings that true random would give), but not really have the language to describe it well without going back and forth.
I knew which story you would tell from the video title and again when you mention reboot & i still watched to the end. Dear Tim i really love your stories, yourself and josh sawyer are in inspiring!
Its so fascinating to me that so many humans struggle with what true randomness really is. Once you started telling the stories are started bracing myself hahaha
do you think regarding the randomness with wildstar, you walkthrough their expectations to get exactly what they want from the get go? the best coders I've walked with can interpret the intent of a designer and walk through the end results of what a designer asks for? The worse thing you can do is take a designer literally sometimes
In a perfect world, designers and programmers agree on the intent of a design doc. Designers can help that happening by being more precise in their work and by including a goal list for their design.
What I was taught in school was wants and needs. Wants are what the client literally says, and needs is what they ACTUALLY want. They want randomness but what they need is a shuffled order or permutations.
Great topic. I'd say a better way to do random is to roll a number at the beginning then create a variable based on the thing chosen. Like if it's song one that's playing the variable would be one. Then for "randomness" you have it add a number between one and five for example. Which ever number is rolled determines which song plays. If the number is five you have it subtract a number between 1 and 4 and plays the corresponding song. If the number/song ever lands on the same song it simply repeats the roll and chooses again. Thus random but varied.
6:06 The composer wanted songs to be shuffled in random order. This is a feature CD players had in the 1980s. The composer may have been satisfied with one shuffled sequence played continuously, which would have been simpler than what Tim eventually implemented.
One thing I've always wanted to see with classic D&D and other dice/random based games, was the option of which 'RNG' to use. True Random (the originally mentioned version), Permutation Shuffle (the music example), and weighted expected outcome (Loot drop/accuracy/criticals example). With the Permutation Shuffle it would be a neat idea to have an item for ally and enemy 'cards', that allows you to advance 1 'card', or reshuffle the entire 'deck'. Overall though in video games I think weighted expected outcome is the most enjoyable for most players. For enemies they only get the bad side, so that a hit makes the next attack more likely to miss. Maybe a difficulty slider affects who gets what degree of the effects. I also think people dislike missing on a 95%, because they aren't thinking in d20 terms, but that is the 'max chance'. When most people think 'max chance', they think 100%, and the idea of missing doesn't enter their mind. That makes people autoconvert 95s into 100s in their mind, and think the game is at fault for them missing. When you actually play on the table with the ever present dread of a Nat 1, it makes that more visible. Showing the dice results in a combat log helps, but actually showing the 'physical' dice every roll (maybe off in the corner) seems to help more.
Missing 3 times in a row... happens. (0.0125% of the time) When it happens in 4 separate encounters (mobs) in a row is when you start questioning the RNG programming. ;)
In statistics there's an idea of sampling with replacement vs sampling without replacement. You're describing sampling with replacement. When you take something out, you put a replacement in (or the same thing goes back in). Which one makes more sense depends on context. For an accounting auditor, they would want to use sampling without replacement. There's no need to examine the same document twice, when you're testing for accuracy in financial statements. Both these methods are still random.
Both way have random aspect but only the first one is truly random, because random means, you can't know the event and sampling without replacement give you information after every try.
Love to go "oh, they actually want the tracks shuffled" before you got to that part. The way they actually wanted the systems to work makes total sense, since it's about creating a satisfying variety, but... yeah, people need to learn to think things through and ask for what they actually want.
On the flip side, developers need to build a sense of empathy and a sensibility about what feels nice rather than taking people's requests at face value and executing them verbatim. I shouldn't have to tell you that a playlist driven by uniform probability distributions is not fun to listen to.
@@cornoc You can't exactly assume that team members want something other than what they're asking for, though. That's going to look extremely bad if you get it wrong, and you're suddenly accused of not being able to follow instructions. Both sides need to make sure that they communicate clearly, and it's on the person making a request to actually ask for what they want. If you have your doubts about the request, you can't do much better than explaining how it will work and asking for a confirmation. If the request is still wrong at that point... well...
Great and funny video! I immediately got numerical recipes in c++ after watching your reboot talk. The wildstar story is hilarious 😂 Thank you so much Tim!!
There is a good video explaining spotify's shuffle feature and why it isnt truly random, which touches similar subject to this. I just think, on instinctual level, it's just hard for people to comprehend probability
Was curious about what kind of algorithm fallout used after randomness being mentioned a few times in your videos. Found some RE'd code, it's a type of Lehmer generator (MCG) called "schrage's method". At a glance, this type appears to produce better randomness than the common MCG at the cost of having to use division. Additionally, there are 32 "streams" of this PRNG, making the state very big. Seems good to me (with *very* basic testing)!
Lol good video. Apparently people don't know what random means. Also, love your game engine vids. As for your back and forth with manager(i guess) is hilarious. Been through this when those making requirements, but they don't really know what they want from beginning. Youdesign based on initial reqs, but they keep changing it based on not expected output
(Spiros) Ah, "Numerical Recipes in C". Takes me back to my University days :) That, and the "Art of Programming", quiet valuable back then. Really nice programming story. 👍
A little communication goes a long way here. I was fortunate to work side by side with a couple great devs on bug fixes, QA, and feature request in a non-games industry and a follow up to this video could be titled 'are you sure?'. Our devs were great at taking what they were working on for the week/month, getting to a critical juncture or bottleneck, and come to me and others and say 'here's what's going to happen if we do this exactly like you asked, are you sure you want to go forward like that?'. And sometimes the answer was yes, but many times it was 'no' or a compromise on the original vision. I'm not sure this is a 'skill' so much as it is a level of communication and trust you can build between your co-workers which stems from caring about the final product, and being proactive enough to take a second and reach out (a lot easier at a small company I'm sure).
This the part of all software engineering that you don't learn in school. People can't communicate to you what they actually want. You will be iterating with them multiple times before they are satisfied. I also think these situations where two worlds collide are where great software is potentially created. Programmers sometimes look down on other stakeholders even though we are often only valuable because we collaborate with others.
We were supposed to implement a Tic Tac Toe game in a computer science class. The requirement was an AI that didn't make mistakes. It was therefore a very poor AI from a gaming point of view.
2:01 as someone who plays gatcha nowadays, every % chance in math is now 50/50: or you hit or you don't, there's no in between. 3:38 when you played so much that your brain doesn't even work well anymore "something is wrong with that" lmao. 4:35 this story is crazy, people really don't know what they want. Actually they don't know sh*t, if he couldn't even recognized it wasn't random when you explained what random is. 7:55 I love how in this story both where so casually and polite "ok" "okay" "ok".
I would argue that the generalized definition of "random" has changed and it's important to keep an eye on things like that. I would suggest that what YOU define as "random" should be redefined as "true random". I work in multimedia (audio/video) and we are going through some changing terminology thanks to everybody thinking AI does everything. Your radio station story reminded me of doing the EXACT SAME THING with a Fallout 4 custom radio mod I produced (Jetfuel Radio). Lots of push and pull with the players about how often songs were repeating. My solution for the 20 songs was to hold 8 of them in a queue and once the song was finished, to clear it from the queue and select another which meant including the radio banter, you couldn't hear a song repeat for 1.5 hrs. When they still said it was repeating too often i could remind them with certainty it was at least 90 mins since they last heard the song repeat. Less randomless wouldn't be a radio structure, it's a "playlist" lol.
yeah, designers want a *bit* of unpredictability and variety. It's from the lens of "experience". They don't want true mathematical randomness LOL Of course it seems stupid, but the language is just wrong. Randomness has it's own definition. Surprise and/or variety is most likely what a game designer wants. Most people don't have the words "shuffle without replacement", "permutation" in their dictionary :D I mean I learn this words before uni math either :D But this was a hell of an episode. Really funny!
I really wish there was a setting in Spotify where it would always pick the song that hasn't been played in the longest time. Even across playlists and listening sessions.
I actually kind of wish there were less randomness in most games. Like, I don't really get why so many games cap hit chance at 95% rather than just letting it go to 100%. That's just a recipe for frustration.
I always hear incorrectly that he says "Hi everyone, it's your boy Tim". I'm pretty sure it's because of Jawbreakers guy with his "Hi everyone, it's your boy Zack".
I suppose "random order" or in other cases "probabilistic" are better words here than "random". It makes me think of early NES RNG tables, where [1..256] were shuffled in random order, selected sequentially in this order, and compared against a predefined number to determine outcomes.
ive seen at least one game (wish i could remember the name) where it actually had an option to use randomness or 'intuitive randomness' which i think was if your chance to miss was 20% you would miss roughly every 5 attacks, so it'd feel more like how people apparently assume randomness works. and it did feel better as far as i recall.
If you have 10% chance to critical hit on your abilities, that is 10% crit chance per cast of an ability. Not every 10 hits you're guaranteed to critical. Randomness typically on surface level accounts for the singular input and doesn't take in an array of data, like every 10 then this happens.
this is an extremely common problem with all programming issues because computers do literally what you tell them to do, and humans generally have a vague concept of what they want. They know what it is when they see it, but they can't easily describe it in the detail necessary to have a machine replicate it. It's why AI isn't going to replace programmers any time soon. The programmers job more or less is to take the input from creative types and interpret it so that machines can execute, and AI generated code has the benefit of vague inputs into concrete outputs but that doesn't mean it isn't buggy/doesn't do what you want it to.
With AI, the user simply learns to specify his request. And the AI then very quickly generates new code with each subsequent request until the result suits the user.
Same thing for animation for advertising, when the client or art director asks for a 20% bigger logo, I usually make it 30, 35%, any less than that and they're not satisfied.
It is based on the characters name, race and background. Roll the same character and it is the same loot. Change name and it is new. Repeat old name it is same (if race background etc kept)
I think further from "people want something but it's not what they say they want", the answer is "they want fun challenges that don't break the game", but what one group consider fun others may consider grind. For instance true randomness can be brutally grindy. You have to accept that some people who write bad reviews are simply playing a game that doesn't suit them, and are upset that it wasn't what they assumed it would be. You can do a lot of damage control simply with transparent marketing.
Hi Tim, really insightful video - especially interesting observation that people tend to respond better to a 'shuffled deck' of results rather than true randomness. As a tabletop GM, it's gotten me wondering how I might use a literal shuffled deck of cards instead of dice for future games. Anyway, hope you don't mind an observation/question. In tabletop RPG systems (i.e. GURPS), outcomes are typically resolved with physical dice rolls. Something I've noticed as a GM is that players tend to be much more accepting of negative outcomes when I throw my dice in the open, rather than behind a screen. Same action, same outcome - but the ability for players to see it seems to have a placating effect. So, question; was there ever a point in the development of a randomness-based system where you considered showing the specific results of a roll to players, rather than just the outcome? To go back to your colleague's example - missing three 95% shots in a row is obviously frustrating, but might it have helped to see a '98, 99, 96' flash up on the screen rather than just 'failed, failed, failed'?
I think the way it went is still the best approach to implement what the designer wants. If you try to work out exactly what they want from the start, or take initiative yourself doing the extra work. You're still gonna be wrong and have to rework the system when the designer isn't happy with what you worked out. So it's best to do the minimum possible solution at first that technically fulfills the request, then iterate. You just have to accept that normal people don't do math, science or logical reason of any kind. Normal people feel stuff and use words colloquially. It would take a very sophisticated psychological process and extensive education to make designers/customers use technically correct language. They would never accept going through that and would only be aggravated by it. It's a million times easier for us technically minded people to adapt instead.
No, the better solution would be to simply ask again before implementing. For example, you can ask whether the random generator should really be mathematically random and then describe the problems that can occur and are described in the video above. This saves time and money. And the time saved can then be invested in other things. Be supportive should be the way.
When I was recently poking around Fallout 3's radio logic to add additional songs I was surprised how overly complicated it is to ensure perceived randomness. It's 9x as long as it would be if the songs were truly random.
Not sure if this is the intended lesson, but what I'm taking away is: Feelings differ from the precise, definitive metaphors we use to represent those feelings. Code relies on precision and hews closely to dictionary definitions. The greater the delta between feeling and dictionary definition, the greater the need for iteration.
Things can get even more crazy as when shuffling a list of certain things, someone may complain that they don't what the same sequence of x elements in a row. So you start creating several buckets, distribute all items into 3 or more buckets and the bucket size is the maximum sequence that may be repeated. Though you always iterate through one bucket before you switch to the next. You only shuffle between all the "not active" buckets before switching to a new bucket. It's a pain in terms of bookkeeping but ensures a "certain pattern free" suffled sequence. Now imagine the suffle shouldn't be just random but a weighted random..... In the end there's no room left for any randomness at all. You have one item with a 50% chance, so it "has to" come up every second time anyways. If there are only 3 other items, one with 30% and two with 10 each there is almost nothing left to "shuffle". Game designers just don't understand randomness at all. If the number of items in a list is 2 and they don't want to have the same twice in a row, they must just alternate. There's no way around that. Most humans are bad at picking random numbers because they think of the world to be somehow dependend on the previous rolls which simply is not the case. That's why humans are extremely bad in creating "random" sequences. This Numberphile video shows this beautifully: ( tP-Ipsat90c )
I want to bring up the pity system I see in New World (Amazon's MMO). Rabbit event 2022, you hunt rabbit for a cool drop. Some people get it on the first 10 kills, others still see no sight of it after 8000 kills. There was a lot of discontent and flamming on forum and reddit. Rabbit event 2023, you hunt rabbit for a cool drop that is guaranteed to drop on the 200th kill. The pity mechanic gives players a sense of progression when they're not hitting that good roll. More game should implement this simply because from the player's stand point, they would always assume 50% means they should see 1 in 2. In this case I would add a pity to give them the good roll after 4 failed rolls, even thought statiscally you can fail way more than that in a row but it's not a good player experience.
There have been lots of case studies, uniform distributions feel unfair to people. Other ways feel better, there's lots of other ways to do things that end up with the same actual odds, but aren't IID uniform random. What people want is something that is random enough to not be totally predictable, but that also feels fair, that's always going to be hard.
What you did with the critical hit makes me doubt if the tooltip is even correct anymore. Like is "Critical Hit Chance 20%" even a wholly accurate statement anymore? I don't know what explanation you could put into that though without the tooltip becoming an essay. XD Great video. This had me smiling from ear to ear because I've gone through more or less the exact same conversations and solutions.
what are you talking about? games secretly increase the parameters so that the player will FEEL better but in the tooltip they lie to you, many many (if not all) games work like this, modern games built around player's satisfaction, games lie to you all the time to feel you are a better player than you are
![Jack Harlow - Hello Miss Johnson [Official Music Video]](http://i.ytimg.com/vi/Q86_nlRoIGw/mqdefault.jpg)
This is such a good example of "true" randomness vs "perceived" randomness. Such a key cornerstone of general game design. Great video Tim!
No because he assumes that the PC randomness is a "true" randomness when it's not. If he had good PC knowledge he would know that.
@@deefeeeeefeeeeeeeeee I'm sure he's aware that random numbers generated by computer aren't actually random because computers aren't capable of generating random data, but it's close enough for the purposes of game design. Very little that we can actually observe is actually random; we just can't see all the variables.
The point, however, which you seem to have missed in your attempt to look clever, is that in a distribution of randomly determined numbers where no determination influences nor is influenced by any other, the results often do not feel random, even though they actually are, and in order to get results that feel random, you have to reduce the randomness of the determination.
@@deefeeeeefeeeeeeeeee he doesn't assume anything, he specifies what 'random' is in PCs right at the beginning. I dunno who hurt you but you should stop projecting.
@@deefeeeeefeeeeeeeeee
he isn’t talking about the mathematical concept of true randomness, he’s talking about game design. He’s using the term to describe a game design philosophy. If you had better common sense you would know that.
Seems like for non math people arbitrary and random are switched. Im no math guy and i would say thats how it makes sense in my head.
This really touches on a bigger issue in not only game design, but so many other jobs that involve different people needing things from each other within the company.
_"Understanding what people want, rather than what they are asking for."_
I work as a Data Engineer/Analyst, figuring out how to get data together into a usable space, and then also figure out what we can get from that data and put it into a report that someone can use for their job.
Almost every single project starts with "I need you to get me ". A lot of analysts or engineers will hear that and immediately delve into whether we have it, how to get it, how we'll get it working and get it into a reportable and understandable way, etc. However, what should be done next is to ask "What are you needing this data for?", or a better phrase _"What question are you trying to answer with this data?"_
This is crucial, because there are so many nuances that could dictate how you proceed.. and I've seen so many projects get extended over and over as the analysts keep presenting their work and the person requesting it keeps sending it back with changes needed because it _doesn't answer the real question._
A good project manager who has the authority to say "No" and ask the real questions can be a MASSIVE change in how projects get completed, and the final product. Spending a single sit down at the start, figuring out the real need in question, and then properly directing resource to fulfilling that need.
When I see supposedly finished games coming out these days with their disjointed mechanics, dropped features and half-baked third acts.. I sense that there were a lot of siloed development teams that spent a lot of time spinning their wheels because they had no one to properly manage the projects and bring things together.
I tend to feel that the small team or solo dev indie games (the legit ones, not money making churn), while smaller in scope, tend to be tighter and more cohesively designed because there's usually one guy who directs his vision all together properly. Less chance of making what they "asked for" instead of what they really "wanted", when it's all the same person, lol.
Developing the skill to figure out what people actually need instead of what they asked for (either through a deeper understanding, or via good diplomatic social skills) is invaluable in any part of your life that involves "someone asking you for something". Which is almost any job, and a lot of inter-personal relationships. Parenting too, now that I think about it, lol.
Getting people to actually communicate what they really want versus what they say they want is a woeful errand indeed.
I usually trust the math without question until it happens to me. Sometimes it just feels SO bad when it happens that your brain is like "No, this can't happen! Not to me!"
Because people are simply not innately good with probability no matter how clear or simple it is, gamblers fallacy and all that.
"Oh for f'sake."
- Me, when I re-try something for the many n'th time because I want something 'just right' as a combination of probabilities.
@@cullenclark8581 I think the XCOM devs have said that players found true 50/50 odds unfair. If they were given a true 50/50 chance and told it was 50/50, they complained that it wasn't 50/50 and felt stacked against them. The devs found that the odds had to be significantly stacked towards a positive result to match what players felt 50/50 should be.
One thing I've learned about randomness and probability rolls is that the dice will do whatever they want until it reaches infinity rolls.
When I dabbled in board game design, you learn a lot about randomness in so many different contexts. Tabletop and Board game design is something I wish a lot of video game designers would even take a month of playing around in, just to experience the differences of randomness mechanisms directly rather than it being obscured by the computer.
Random draw and random roll, as you demonstrated, are vastly different.
Man, the internet is so cool. Your games made my childhood I never thought I'd get the chance to just hear you talk about the processes that went into them
Reminds me of when I played Candyland for the first time as an adult and realized that the instant you shuffle the deck, the game is decided and the whole thing is just drawing cards and seeing who already won. Barely even a game there
I love learning through this series. Thanks, Tim.
in the 80s, tetris used to be pure random but somewhere in the 90s they swapped to something the fans call a seven bag shuffle.
it creates fewer moments where it feels like the game is actively working against you, but I miss the fun thrill of getting 3 or even 4 of the same tetromino in a row.
Tim, all these talks have made me really regret my decision to leave the gaming industry (I worked as a sound designer). I've even gone so far as to re-download Wwise and FMOD in order to refresh my memory with those tools, and I'm thinking about updating my resume/portfolio. I don't know if I'll go through with it yet, but truly, thank you for re sparking my passion.
Similar with me, these vids re-invigorated my interest in software development. As he was describing the demands he was receiving in this vid regarding “random” selection, I already started mapping how I would write that in my head. I’ve already started working on personal projects.
Tim, I have talked to so many irrational people about this over the years. The people who rage about RNG generally are just not capable of understanding or accepting it. I've found that no matter how you explain it, they still think the generator is broken.
one of the best, funniest and at the same time truest stories I've ever heard.. as a programmer I come across this a lot, I've even been yelled at when I did exactly what they asked me to do.. the absolute highlight was when I wrote working code, my boss rewrote it (of course he made an error), then he fixed the error and came to me saying that I had an error in the code, but he already fixed it for me :D
and the biggest takeaway from this is: never do what they ask you to do, always try to find out what they want you to do.. applies to everything: life, work, partners, etc..
"Well, let's do math. I love math!" Reminds me of how Jonathan Blow is (apparently still) mad at Sid Meier for suggesting that RNG shouldn't be exact because it makes players feel bad. And you're right, it's absolutely a communication issue. I've spent a lot of time in my career translating between programmers and non-programmers and sometimes there wasn't even anything to translate; both sides wanted the exact same thing, but the language they were using was causing the other party to think the opposite. Some of my favorite moments involved programmers telling me something wasn't possible, and then watching management spin their wheels endlessly trying to get what they want rather than accepting they can't have it their way. Management behaving like children...I feel like there's a corollary in here somewhere 😂
I've spent a lot of time translating, that's an important skill. _especially_ when dealing with programmers, but it applies to a lot of fields and jobs. If group A member talks directly to group B member, effective communication is not achieved. It takes a translator, as you say, to understand what group A wants/needs from group B, and then being able to put that into the right language and context for group B member to understand.
It seems like it happens bajillions of times, and a lot of people _don't_ understand that this is so frequent, and it leads to less effectiveness, less efficiency, lots of hiccups in whatever business or goal that everyone shares.
This was incredible haha, I absolutely love stories like this. The 1 in 8000 over a week was so interesting.
This is a great example of other departments not knowing what they want or not knowing how to express (logically) what they want. Interesting stories! I think a lot of times these happen naturally as an outcome of "changing their mind" when a team is developing a feature or set of features which are not always implemented perfectly on the first iteration. Adding to that, changes also tend to happen when the feature has been tested or played around with enough to know how it "feels" when played in "developer brain mode" versus "player brain mode" as a designer.
This is hilarious. "I want randomness!" Tim: "Not this again..."
Excellent video, Tim! I run into this all the time at the studios I work with. The last part especially resonated a lot with me. I often describe it as "listen to their issues, but ignore their solutions", because most of the time those would be "patches" that would fix the symptom, but not the disease, and it would still cause problems in other parts of the game.
9:44 I was like "Oh my God...."... and then Tim did that facepalm at the same time... XD... All I want to say is that all your hard work was not in vain! Keep up the good work (and the patience hahahahahahah)..
I'll never forget the test case I wrote after coding a randomizer for a tech tree at work. It said, "Verify that the random results match your internal model of randomness."
I play old school runescape where many items are highly rare drops with odds of 1/500 or so on a boss. Many people will start to complain once they have 500 kills, feeling that they are "dry" and should have gotten the item by then. But the reality is that going up to 1500 kills, ~5% of players will still have not gotten that item. When you consider that hundreds of thousands of people have played this game over the years, 5% is a VERY large number of players. I personally prefer it compared to having some sort of upper limit or increased odds per kill, and I always feel better when I start looking at the math behind it.
I rememver a D&D session my mate had where he, as DM, was rolling for an NPC and he rolled a 1 and then rolled another 1 (to see how bad the results of the critical failure would be). Two Critical Fails in a row. This is why randomness is good.
saw this a lot during my time designing webpages
people dont know how to express or describe the kind of website they want, how they want it laid out, how they want it to look, etc.
what ive found to be most successful is work through how they want the experience to be, and then draw up a prototype for confirmation. i know that option isnt always practical in a development environment, but it saved me time from having to go back and forth.
ive heard its pretty much the exact same for artists, too.
YES!! I love your examples. lol!!! I wish I had your videos 10~20 years ago. lol
One of my math trick for "random" is circular increments. Let's say I'm simulating a D20 roll. Previous roll had the number n. For the next roll, I'd do "(n + rand(19)) % 20 + 1)". That achieves the non-repeating roll without needing control statement or a loop. :D
Thank you for sharing all your precious knowledge!
I love this topic so much, it revolved around the perception of fairness which sadly isn't a healthy topic to speak to everyone on because it tends to bleed into the Confirmation Bias territory. Also in discrete math, calculating your chances of a winning hand in blackjack changes depending on if you calculate it before or after the cards are initially dealt.
Very interesting. I've seen something similar on Q and A sites, where people ask for very specific solutions instead of explaining what they are trying to achieve with it.
Classic! This exactly happened in our first game, Vaporum. One of my colleagues came and said: "Your random system is bullshit! I have an 85% hit chance and I missed 3 times in a row!" I tried to explain that that's how randomness works, but to no avail. And this is universal, not just my colleague -- most people think that the scenario I described is BS. It simply feels unfair. I then later reworked it to mostly what Tim describes here, which is not random, but "feels" better, and everybody was happy.
But why did you let them fall into the trap? You should have known that and be more supportive. This: "Make feel better" is what is expected from a game. The simplest step would have been to ask whether a mathematical correct random generator is really wanted.
As you were telling your story, I'm thinking to myself "that's not random. Oh, neither is that..." and ways I would design it. I like randomness as you designed it, but with a caveat. When loot doesn't drop or you miss when you are likely to hit, add a modifier that adds a percentage or point into loot quality or critical hit chance that resets after a hit or loot drop. It sucks missing a shot or not getting loot or bad loot, but getting that rare drop or that juicy critical after a bunch of nothing makes it much more satisfying.
Mitigating disappointment is better than "designed randomness" in my opinion because once you learn how it was designed you can start gaming the system and takes the fun out of it.
it depends. for the music playback issue, curated randomness is exactly what you want. hearing the same song twice in a row gets boring or feels artificial, and same for hearing the same sequence of songs twice in a row, to a lesser degree.
most things we like are a mix of determinism to rule out uninteresting outcomes, and randomness to add novelty and excitement.
As an RNG designer, none of this came as a surprise. Random shuffles and Ornstein-Uhlenbeck processes (en.wikipedia.org/wiki/Ornstein%E2%80%93Uhlenbeck_process) are your friend for natural feeling event creation.
Really enjoying hearing about the more technical programming aspects of game development. It's nice following along with your thought process and what you're doing in the code even with simple things like arrays and RNGs.
Important words on "communication" ... Loved this episode. This could be it's own talk.
"When almost anyone says to you: 'I want randomness', no they don't; they want shuffled permutations. This is smart people using a word that doesn't mean what they think it means. I learned to listen to what people wanted, and not what they were asking for. Which, by the way, is a very good skill when you are reading reviews about games you made. People will tell you they want something, but it's not what they want, they want something else. Either they don't have the vocabulary, or the self-reflection, or they just can't even put into words what it is they want. But frequently, people ask for things, and it's not what they want. They want something else, and it takes a while to extract that from them. What you need to learn is how to ask for things which are exactly what you want. Not: 'this is what I'm gonna ask for, and you should figure out what it is I want.'"
9:50 My favorite reaction from this video. I really like how you tell these stories, very relatable even if you haven't worked in that field.
That song shuffling logic development was hilarious.
Tim is 100% right about this being a gap in word meaning. Colloquial meaning vs technical meaning is one of the problems that plagues most languages in general. And, in this case, there's one particular synonym of "random" that I think might actually help pinpoint the communication gap even more: "designless".
The "colloquial" meaning of random is just incorrect. Randomness is mathematically well-defined and fits an intelligent person's intuitive idea of what randomness is. "Designless" does not describe it either because the examples Tim gives are clearly "designed". Some people just don't understand basic maths.
@@slynt_ However, Tim could have asked before implementation whether the game designers really wanted a mathematically correct random generator. Especially when it is known that terms are understood differently and you want a random generator for a game that is ultimately supposed to be fun.
I personally find true randomness in games a very frustrating thing to deal with since it normally leaves players feeling screwed over with things like hit chances and drop rates. I find myself wishing more games would use your shuffled permutation example. I usually think of it as a 'randomly' generated playlist that gets recreated on completion because my old CD player used that terminology :). I realise the point of the video wasn't about the merits of true randomness but I did wonder how many other people came to that terminology due to old CD players haha.
The first permutation itself is random. There are games (Colony Ship comes to mind) that offer different methods (true random vs shuffled vs some other skewed random).
This was brilliant. Got a few chuckles out of me. Even the part about browsing forums is also true.
I could watch hours of you telling stories of randomness, great stuff!
These videos have been keeping me sane whilst I work on my masters degree in games design, god bless you sir lmao
If I've learned anything from going out in the professional world it's that most people don't understand basic math concepts. I invite people to events for my job. We've had people say they only want managers to be invited, fine... but then they only want managers from this state, and only if they are in this specific kind of field... Not realizing that every qualifier they add is like another circle in a venn diagram and the people that fit all of these categories are exponentially reduced each time they add another. So I went from being able to invite thousands of people potentially to like... 10.
Hey Tim.
Both Fallout 1 & 2's random factor regarding combat was hilariously brutal.
As frustrated as I have been while playing the games for all these years, as much fun has it been to "reminisce" with other old fallout players about exactly _that._
Not just missing 3 shots in a row. But having bullets "boomerang" around and hit a companion/a companion hitting me standing in the background.
Ian was notorious for shooting you in the back in F1. Sulik was notorious for boomeranging burst-fire with sub-machine guns in F2.
Taking on a companion in Fallout was always a bit of a 'Monkeys Paw' deal. :p
Critical failures were such an interesting way to add difficulty to combat. Not just having the enemies with more health, or adding more enemies, no getting around it by min maxing damage or exploiting AI. Now you have to manage wasting ammo and reloading your clip early, or having to run and pick up your weapon in the middle of combat. Very unique, and something that always left a fondness behind.
3 times in a row isn't even that much. 1/20*1/20*1/20=1/8000 ... but people think they are going to win the lottery :D I mean these are things I knew already about people, but this gives me another perspective on how we humans think sometimes :D
As an XCOM player I know the ways of RNGesus all too well
All hail/curse RNGesus...
As a severely unlucky player of video games, RNGesus and I are well acquainted. Bastard owes me way to many hours of my life.
You too, Commander.
This game was in the back of my mind as I watched this video. It all correlates so well.
Save scumming was hard in xcom but not impossible. With a few save points that give you space to maneuver before getting trapped into predetermined rolls, you can avoid calamity.
That or bomb the heck out of everything and let RNGesus sort them out.
It's really funny, I just watched that talk you mentioned yesterday. These stories prove that programmer brains just work differently than everybody else's XP
Deck of cards approach to random events is the way to go on how people understand randomness and % chance for event to occur.
Love these videos. Thank you for sharing your experiences and knowledge. Playing Fallout was a life changer for me.
On a related note, your rng seed handling is top tier. More than 20 years after Arcanum release, BG3 just dropped with loot in chests being regenerated after a reload. Unbelievable.
I think they did the same in Divinity Original Sin 1 and 2. I don't understand why they keep doing that,
it's intentional so that people dont feel bad if they dont got what they wanted and reload the game to get better result, and _"honest"_ players wont do it, it's not _"Unbelievable"_ it is 100% intentional not to store the random seed in a save file to keep the game player friendly
most modern games track the player's success and increases it in case of series of bad result so that the player feel better and not being cheated by the game, the randomness Tim talks about basically does not exist in modern games because developers dont want people get too upset, in xcom 2 on normal difficulty if you miss a shot the game secretly increases your hit chance (while displaying the old lower value) so you will feel you being soo good 😃
How many "reload sets" are there though? There are so many items in BG3 and DOS that I wouldn't be too quick to shit on it as it might be rather inconsequential. Also, I forget, is Luck in BG3? I've only gotten a few hours into it so far. I would also say that reloading for a new set would be pretty boring and tedious for the reason I stated initially.
@@Arkanthrall Probably because most players don't care enough to reload for some arbitrary chest. I guess maybe if it was one at the end of a dungeon or something it might be worth it. Personally I wouldn't care to reload just for loot anyway.
@@fixpontt That is awful. I can't believe how much game devs pander to crybaby gamers these days. All for the bottom line, I suppose.
I have been watching your videos for a while and you have given me a different perspective on game development in general . thànk you
the stories about the devs you do are very fun to listen too! its kinda funny to hear about how a common problem seems to be developers knowing what they want(i want the consumer experience to FEEL random, but not give the negative feelings that true random would give), but not really have the language to describe it well without going back and forth.
I knew which story you would tell from the video title and again when you mention reboot & i still watched to the end.
Dear Tim i really love your stories, yourself and josh sawyer are in inspiring!
I loved the original talk when you mentioned this topic. It inspired me to create a playlist with various videos about RPG development.
Its so fascinating to me that so many humans struggle with what true randomness really is.
Once you started telling the stories are started bracing myself hahaha
do you think regarding the randomness with wildstar, you walkthrough their expectations to get exactly what they want from the get go? the best coders I've walked with can interpret the intent of a designer and walk through the end results of what a designer asks for?
The worse thing you can do is take a designer literally sometimes
In a perfect world, designers and programmers agree on the intent of a design doc. Designers can help that happening by being more precise in their work and by including a goal list for their design.
Saving this for next time when I have to explain agile methodology to someone.
What I was taught in school was wants and needs. Wants are what the client literally says, and needs is what they ACTUALLY want. They want randomness but what they need is a shuffled order or permutations.
Great topic. I'd say a better way to do random is to roll a number at the beginning then create a variable based on the thing chosen. Like if it's song one that's playing the variable would be one. Then for "randomness" you have it add a number between one and five for example. Which ever number is rolled determines which song plays. If the number is five you have it subtract a number between 1 and 4 and plays the corresponding song. If the number/song ever lands on the same song it simply repeats the roll and chooses again. Thus random but varied.
6:06 The composer wanted songs to be shuffled in random order. This is a feature CD players had in the 1980s. The composer may have been satisfied with one shuffled sequence played continuously, which would have been simpler than what Tim eventually implemented.
One thing I've always wanted to see with classic D&D and other dice/random based games, was the option of which 'RNG' to use. True Random (the originally mentioned version), Permutation Shuffle (the music example), and weighted expected outcome (Loot drop/accuracy/criticals example). With the Permutation Shuffle it would be a neat idea to have an item for ally and enemy 'cards', that allows you to advance 1 'card', or reshuffle the entire 'deck'. Overall though in video games I think weighted expected outcome is the most enjoyable for most players. For enemies they only get the bad side, so that a hit makes the next attack more likely to miss. Maybe a difficulty slider affects who gets what degree of the effects.
I also think people dislike missing on a 95%, because they aren't thinking in d20 terms, but that is the 'max chance'. When most people think 'max chance', they think 100%, and the idea of missing doesn't enter their mind. That makes people autoconvert 95s into 100s in their mind, and think the game is at fault for them missing. When you actually play on the table with the ever present dread of a Nat 1, it makes that more visible. Showing the dice results in a combat log helps, but actually showing the 'physical' dice every roll (maybe off in the corner) seems to help more.
Missing 3 times in a row... happens. (0.0125% of the time) When it happens in 4 separate encounters (mobs) in a row is when you start questioning the RNG programming. ;)
In statistics there's an idea of sampling with replacement vs sampling without replacement. You're describing sampling with replacement. When you take something out, you put a replacement in (or the same thing goes back in). Which one makes more sense depends on context. For an accounting auditor, they would want to use sampling without replacement. There's no need to examine the same document twice, when you're testing for accuracy in financial statements. Both these methods are still random.
Both way have random aspect but only the first one is truly random, because random means, you can't know the event and sampling without replacement give you information after every try.
Love to go "oh, they actually want the tracks shuffled" before you got to that part.
The way they actually wanted the systems to work makes total sense, since it's about creating a satisfying variety, but... yeah, people need to learn to think things through and ask for what they actually want.
On the flip side, developers need to build a sense of empathy and a sensibility about what feels nice rather than taking people's requests at face value and executing them verbatim. I shouldn't have to tell you that a playlist driven by uniform probability distributions is not fun to listen to.
@@cornoc You can't exactly assume that team members want something other than what they're asking for, though. That's going to look extremely bad if you get it wrong, and you're suddenly accused of not being able to follow instructions.
Both sides need to make sure that they communicate clearly, and it's on the person making a request to actually ask for what they want. If you have your doubts about the request, you can't do much better than explaining how it will work and asking for a confirmation. If the request is still wrong at that point... well...
@@JediMB Yes, empathy and good communication solves all of this.
@@cornoc "empathy" lol that's the main reason the game industry is terrible nowadays but sure
@@slynt_ You think being able to see through a player's eyes is ruining the industry? lol..... love to hear how you figure that
Great and funny video! I immediately got numerical recipes in c++ after watching your reboot talk. The wildstar story is hilarious 😂
Thank you so much Tim!!
There is a good video explaining spotify's shuffle feature and why it isnt truly random, which touches similar subject to this. I just think, on instinctual level, it's just hard for people to comprehend probability
Was curious about what kind of algorithm fallout used after randomness being mentioned a few times in your videos. Found some RE'd code, it's a type of Lehmer generator (MCG) called "schrage's method". At a glance, this type appears to produce better randomness than the common MCG at the cost of having to use division. Additionally, there are 32 "streams" of this PRNG, making the state very big. Seems good to me (with *very* basic testing)!
Thanks for sharing this information in a way thats easy to look up. Was looking for a decent number generator for my own project
The most basic library generators are random enough for games. Better quality is only really needed in cryptography.
Lol good video. Apparently people don't know what random means. Also, love your game engine vids. As for your back and forth with manager(i guess) is hilarious. Been through this when those making requirements, but they don't really know what they want from beginning. Youdesign based on initial reqs, but they keep changing it based on not expected output
(Spiros) Ah, "Numerical Recipes in C". Takes me back to my University days :)
That, and the "Art of Programming", quiet valuable back then.
Really nice programming story. 👍
A little communication goes a long way here. I was fortunate to work side by side with a couple great devs on bug fixes, QA, and feature request in a non-games industry and a follow up to this video could be titled 'are you sure?'. Our devs were great at taking what they were working on for the week/month, getting to a critical juncture or bottleneck, and come to me and others and say 'here's what's going to happen if we do this exactly like you asked, are you sure you want to go forward like that?'. And sometimes the answer was yes, but many times it was 'no' or a compromise on the original vision.
I'm not sure this is a 'skill' so much as it is a level of communication and trust you can build between your co-workers which stems from caring about the final product, and being proactive enough to take a second and reach out (a lot easier at a small company I'm sure).
This the part of all software engineering that you don't learn in school. People can't communicate to you what they actually want. You will be iterating with them multiple times before they are satisfied. I also think these situations where two worlds collide are where great software is potentially created. Programmers sometimes look down on other stakeholders even though we are often only valuable because we collaborate with others.
We were supposed to implement a Tic Tac Toe game in a computer science class. The requirement was an AI that didn't make mistakes. It was therefore a very poor AI from a gaming point of view.
2:01 as someone who plays gatcha nowadays, every % chance in math is now 50/50: or you hit or you don't, there's no in between.
3:38 when you played so much that your brain doesn't even work well anymore "something is wrong with that" lmao.
4:35 this story is crazy, people really don't know what they want. Actually they don't know sh*t, if he couldn't even recognized it wasn't random when you explained what random is.
7:55 I love how in this story both where so casually and polite "ok" "okay" "ok".
I would argue that the generalized definition of "random" has changed and it's important to keep an eye on things like that.
I would suggest that what YOU define as "random" should be redefined as "true random". I work in multimedia (audio/video) and we are going through some changing terminology thanks to everybody thinking AI does everything.
Your radio station story reminded me of doing the EXACT SAME THING with a Fallout 4 custom radio mod I produced (Jetfuel Radio).
Lots of push and pull with the players about how often songs were repeating. My solution for the 20 songs was to hold 8 of them in a queue and once the song was finished, to clear it from the queue and select another which meant including the radio banter, you couldn't hear a song repeat for 1.5 hrs. When they still said it was repeating too often i could remind them with certainty it was at least 90 mins since they last heard the song repeat.
Less randomless wouldn't be a radio structure, it's a "playlist" lol.
Nice video. And I guess I am at least honest enough to know I actually don't always want random.
yeah, designers want a *bit* of unpredictability and variety. It's from the lens of "experience". They don't want true mathematical randomness LOL Of course it seems stupid, but the language is just wrong. Randomness has it's own definition. Surprise and/or variety is most likely what a game designer wants. Most people don't have the words "shuffle without replacement", "permutation" in their dictionary :D I mean I learn this words before uni math either :D But this was a hell of an episode. Really funny!
I really wish there was a setting in Spotify where it would always pick the song that hasn't been played in the longest time. Even across playlists and listening sessions.
I actually kind of wish there were less randomness in most games. Like, I don't really get why so many games cap hit chance at 95% rather than just letting it go to 100%. That's just a recipe for frustration.
By God's sake, Tim, give this poor man his REAL random song list :D !
By the way, I love how you explained the math stuff.
I always hear incorrectly that he says "Hi everyone, it's your boy Tim". I'm pretty sure it's because of Jawbreakers guy with his "Hi everyone, it's your boy Zack".
Story of my life :)
Loving the Tim Bits!
I suppose "random order" or in other cases "probabilistic" are better words here than "random". It makes me think of early NES RNG tables, where [1..256] were shuffled in random order, selected sequentially in this order, and compared against a predefined number to determine outcomes.
This is my favorite video so far!
The music topic reminds me of how Apple “fixed” their shuffle feature
ive seen at least one game (wish i could remember the name) where it actually had an option to use randomness or 'intuitive randomness' which i think was if your chance to miss was 20% you would miss roughly every 5 attacks, so it'd feel more like how people apparently assume randomness works. and it did feel better as far as i recall.
If you have 10% chance to critical hit on your abilities, that is 10% crit chance per cast of an ability. Not every 10 hits you're guaranteed to critical. Randomness typically on surface level accounts for the singular input and doesn't take in an array of data, like every 10 then this happens.
this is an extremely common problem with all programming issues because computers do literally what you tell them to do, and humans generally have a vague concept of what they want. They know what it is when they see it, but they can't easily describe it in the detail necessary to have a machine replicate it.
It's why AI isn't going to replace programmers any time soon. The programmers job more or less is to take the input from creative types and interpret it so that machines can execute, and AI generated code has the benefit of vague inputs into concrete outputs but that doesn't mean it isn't buggy/doesn't do what you want it to.
With AI, the user simply learns to specify his request. And the AI then very quickly generates new code with each subsequent request until the result suits the user.
Ooof, that crit hit case is particularly egregious. Yeouch.
Same thing for animation for advertising, when the client or art director asks for a 20% bigger logo, I usually make it 30, 35%, any less than that and they're not satisfied.
In Arcanum the chest near Kite shaman (near the wreckage, at the very start) had also randomly generated loot.
It is based on the characters name, race and background.
Roll the same character and it is the same loot. Change name and it is new. Repeat old name it is same (if race background etc kept)
What a plethora of great topics. Great job. I'm biased towards anything Arcanum so thank you for your videos. Meaningful history for me.
I think further from "people want something but it's not what they say they want", the answer is "they want fun challenges that don't break the game", but what one group consider fun others may consider grind. For instance true randomness can be brutally grindy. You have to accept that some people who write bad reviews are simply playing a game that doesn't suit them, and are upset that it wasn't what they assumed it would be. You can do a lot of damage control simply with transparent marketing.
The worst kind of bug is come from their randomness, hardest to find and sometime the hardest to debug it...
Hi Tim, really insightful video - especially interesting observation that people tend to respond better to a 'shuffled deck' of results rather than true randomness. As a tabletop GM, it's gotten me wondering how I might use a literal shuffled deck of cards instead of dice for future games. Anyway, hope you don't mind an observation/question.
In tabletop RPG systems (i.e. GURPS), outcomes are typically resolved with physical dice rolls. Something I've noticed as a GM is that players tend to be much more accepting of negative outcomes when I throw my dice in the open, rather than behind a screen. Same action, same outcome - but the ability for players to see it seems to have a placating effect.
So, question; was there ever a point in the development of a randomness-based system where you considered showing the specific results of a roll to players, rather than just the outcome? To go back to your colleague's example - missing three 95% shots in a row is obviously frustrating, but might it have helped to see a '98, 99, 96' flash up on the screen rather than just 'failed, failed, failed'?
Yes, we did that in Temple. You could see your rolls if you expanded the hyperlinks in the description box
"You keep using that word. I do not think it means what you think it means!"
Have a great day, Sir! o7
I think the way it went is still the best approach to implement what the designer wants. If you try to work out exactly what they want from the start, or take initiative yourself doing the extra work. You're still gonna be wrong and have to rework the system when the designer isn't happy with what you worked out. So it's best to do the minimum possible solution at first that technically fulfills the request, then iterate. You just have to accept that normal people don't do math, science or logical reason of any kind. Normal people feel stuff and use words colloquially. It would take a very sophisticated psychological process and extensive education to make designers/customers use technically correct language. They would never accept going through that and would only be aggravated by it. It's a million times easier for us technically minded people to adapt instead.
No, the better solution would be to simply ask again before implementing. For example, you can ask whether the random generator should really be mathematically random and then describe the problems that can occur and are described in the video above. This saves time and money. And the time saved can then be invested in other things. Be supportive should be the way.
When I was recently poking around Fallout 3's radio logic to add additional songs I was surprised how overly complicated it is to ensure perceived randomness. It's 9x as long as it would be if the songs were truly random.
Maybe it was already asked somewhere, but would not be possible make a podcast of these videos as well? Would love to listen to it while driving.
Not sure if this is the intended lesson, but what I'm taking away is: Feelings differ from the precise, definitive metaphors we use to represent those feelings. Code relies on precision and hews closely to dictionary definitions. The greater the delta between feeling and dictionary definition, the greater the need for iteration.
Have I watched this video before? I'm having severe deja vu :O
hilarious and insightful video. Thanks. :)
This is becoming a daily ritual.
Things can get even more crazy as when shuffling a list of certain things, someone may complain that they don't what the same sequence of x elements in a row. So you start creating several buckets, distribute all items into 3 or more buckets and the bucket size is the maximum sequence that may be repeated. Though you always iterate through one bucket before you switch to the next. You only shuffle between all the "not active" buckets before switching to a new bucket. It's a pain in terms of bookkeeping but ensures a "certain pattern free" suffled sequence.
Now imagine the suffle shouldn't be just random but a weighted random..... In the end there's no room left for any randomness at all. You have one item with a 50% chance, so it "has to" come up every second time anyways. If there are only 3 other items, one with 30% and two with 10 each there is almost nothing left to "shuffle".
Game designers just don't understand randomness at all. If the number of items in a list is 2 and they don't want to have the same twice in a row, they must just alternate. There's no way around that. Most humans are bad at picking random numbers because they think of the world to be somehow dependend on the previous rolls which simply is not the case. That's why humans are extremely bad in creating "random" sequences. This Numberphile video shows this beautifully: ( tP-Ipsat90c )
I want to bring up the pity system I see in New World (Amazon's MMO).
Rabbit event 2022, you hunt rabbit for a cool drop. Some people get it on the first 10 kills, others still see no sight of it after 8000 kills. There was a lot of discontent and flamming on forum and reddit.
Rabbit event 2023, you hunt rabbit for a cool drop that is guaranteed to drop on the 200th kill. The pity mechanic gives players a sense of progression when they're not hitting that good roll. More game should implement this simply because from the player's stand point, they would always assume 50% means they should see 1 in 2. In this case I would add a pity to give them the good roll after 4 failed rolls, even thought statiscally you can fail way more than that in a row but it's not a good player experience.
There have been lots of case studies, uniform distributions feel unfair to people. Other ways feel better, there's lots of other ways to do things that end up with the same actual odds, but aren't IID uniform random. What people want is something that is random enough to not be totally predictable, but that also feels fair, that's always going to be hard.
What you did with the critical hit makes me doubt if the tooltip is even correct anymore. Like is "Critical Hit Chance 20%" even a wholly accurate statement anymore? I don't know what explanation you could put into that though without the tooltip becoming an essay. XD
Great video. This had me smiling from ear to ear because I've gone through more or less the exact same conversations and solutions.
what are you talking about? games secretly increase the parameters so that the player will FEEL better but in the tooltip they lie to you, many many (if not all) games work like this, modern games built around player's satisfaction, games lie to you all the time to feel you are a better player than you are
@@fixpontt why ask what are you talking about if you already know EXACTLY what I am talking about?