The "Hot Hand" IS Real - Classic Fallacy Debunked
HTML-код
- Опубликовано: 11 сен 2024
- Probability and statistics rarely get this kind of attention. But this story is special. It’s about debunking a long-held statistics result that itself was about debunking sports lore.
A 1985 study found that basketball players do not have a "hot hand." That is, players don't go on hot streaks or cold streaks. Our impression they do became known as the "hot hand fallacy."
A new working paper now says the "hot hand" actually exists--the "hot hand fallacy" is actually a fallacy! The paper claims the 1985 paper made a subtle statistical mistake. This video gives the intuition about the paper and why its argument appears to be sound.
Blog post: mindyourdecisio...
Papers cited
Miller, Joshua Benjamin and Sanjurjo, Adam, Surprised by the Gambler’s and Hot Hand Fallacies? A Truth in the Law of Small Numbers (September 15, 2015). IGIER Working Paper #552. Available at SSRN: ssrn.com/abstra... or dx.doi.org/10.2...
Gilovich, Thomas; Tversky, A.; Vallone, R. (1985). "The Hot Hand in Basketball: On the Misperception of Random Sequences." www.researchgat...
Further reading
Rinott, Yosef and Bar-Hillel, Maya, Comments on a ‘Hot Hand’ Paper by Miller and Sanjurjo (2015) (September 6, 2015). Available at SSRN: ssrn.com/abstra... or dx.doi.org/10.2...
Wall Street Journal www.wsj.com/art...
Andrew Gelman andrewgelman.co...
If you like my videos, you can support me at Patreon: / mindyourdecisions
Connect on social media. I update each site when I have a new video or blog post, so you can follow me on whichever method is most convenient for you.
My Blog: mindyourdecisio...
Twitter: / preshtalwalkar
Facebook: / 168446714965
Google+: plus.google.co...
Pinterest: / preshtalwalkar
Tumblr: / preshtalwalkar
Instagram: / preshtalwalkar
Patreon: / mindyourdecisions
Newsletter (sent about 2 times a year): eepurl.com/KvS0r
My Books
"The Joy of Game Theory" shows how you can use math to out-think your competition. (rated 4/5 stars on 23 reviews) www.amazon.com...
"The Irrationality Illusion: How To Make Smart Decisions And Overcome Bias" is a handbook that explains the many ways we are biased about decision-making and offers techniques to make smart decisions. (rated 5/5 stars on 1 review) www.amazon.com...
"Math Puzzles Volume 1" features classic brain teasers and riddles with complete solutions for problems in counting, geometry, probability, and game theory. Volume 1 is rated 4.5/5 stars on 11 reviews. www.amazon.com...
"Math Puzzles Volume 2" is a sequel book with more great problems. www.amazon.com...
"Math Puzzles Volume 3" is the third in the series. www.amazon.com...
"40 Paradoxes in Logic, Probability, and Game Theory" contains thought-provoking and counter-intuitive results. (rated 4.9/5 stars on 7 reviews) www.amazon.com...
"The Best Mental Math Tricks" teaches how you can look like a math genius by solving problems in your head (rated 4.7/5 stars on 3 reviews) www.amazon.com...
"Multiply Numbers By Drawing Lines" This book is a reference guide for my video that has over 1 million views on a geometric method to multiply numbers. (rated 5/5 stars on 1 review) www.amazon.com...
![MISSING: SLIM SHADY [Expanded Mourner’s Edition Trailer]](http://i.ytimg.com/vi/xh8qgHrOO1g/mqdefault.jpg)
Of course the "hot hand" is real. People aren't coins. A coin has no memory, no psyche; it's nothing but a piece of metal. A person has memory and emotions and feelings. When I'm playing a game, or working on a project, a success will boost my confidence and give me a more positive outlook. That will give me a slightly higher chance of success on whatever I try next. Not because my actual skill increased, but because positive thinking is a powerful thing, and when things are tense, a success will cause us to focus our skills, leading to more successes. Same goes for failures. A failure will bring us down just a little bit, meaning we have a higher chance of subsequent failures. This is why coaches give teams and players pep talks. It's to put them into a positive state of mind. It's not just to hear themselves talk.
Somewhat- but I actually say skill is not a simple quantity and they get more skillful in a practical sense- they "know" unconsciously how to do it right and their brain clicks so they can do it right again . Coins are deterministic not truly random- it is merely that measuring the input forces is way to hard to measure. Theoretical mathematicians have massive egos and think the theoretical result- ignoring any information effects- is always right. You can see it in the problem posting on the internet.
In your example there are 24 "hits" that are followed by shots. 12 of those are hits and 12 are misses. You gave the HHHH the same weight as the MHMM. I don't think that measures the exposures correctly (to use an actuarial term). I'm not disputing the conclusions of the paper, but I'm suspicious about your simplification.
+greyareabeyond The observations should be "pooled" like you pointed out in which case there is a 50% rate. What happened in the original study did not pool the observations. It instead did something like averaging the trials, which gives a biased estimate. This example comes from the paper as well, and I like how the authors explain it.
First off, doing the study by only tracking confidence on 'previous shot made' is incredibly flawed. A player's confidence can be affected by an infinite amount of immeasurable things. Some things are measurable, such as how his team is performing. The biggest problem with only looking at previous shot is that you have no measure of TIME---when that last shot was taken in relation to the next shot. A player is ABSOLUTELY more likely to make a shot if he took the last shot very recently because he is warmed up and his MUSCLE MEMORY is more recent and accurate. That's why Steph Curry can hit 70 three pointers in a row if you feed him one after the next. However, if you had Steph Curry, instead, take 1 three pointer per day for 70 days (no warmup), there's no chance he shoots the same accuracy.
Love this channel but I have to take exception to this explanation. Shooting hoops is not a 50/50 Hit or Miss chance as in the case of a coin toss because of the skill bias of the thrower. If I were to take 1000 shots and Michael Jordan were to take 1000 shots, the probability of a Hit or Miss would be wildly different for each of us. What must be done is to account for the skill bias to define the individual hit rate, which is then the "norm". Lets stick with MJ, as I'm rubbish, so we would then be able to define his hit rate. 3 key things affect this - shot risk (i.e. throw from half-way vs. static free-throw), continuity (i.e. first game back having been out injured for several weeks vs. a run of 15 games) and finally confidence (at the peak of self-belief that you will make the shot - often connected with continuity and affected by game pressure). If we remove the shot risk and just focus on Free Throws, MJ's career success is actually about 75% - way above my average and way above 50%. What we can then track thru statistical analysis is the trend of success, where we see the impact of continuity and confidence. We cannot apply the same cold analysis to a purely 50/50 occurrence - coin toss - to this scenario. The appearance of "hot hands" is a combination of continuity, confidence and natural statistical variance. However, what we should look at is the opposite effect. In theory MJ should make 100% of FT's, as he has total control of the outcome. So the starting point is not 50/50. We then need to analyse the vast number of influences on the accuracy of his shots to cause his average to drop below 100%. Using traditional statistical analysis is ultimately flawed because this is not a closed system of equal probability.
Taking on other apparently "chance" occurrences like card games, the key difference is how much influence the skill bias has on the outcome. In MJ's case, he has 100% control in at the FT line. In cards though, no matter how skilled the player, they have to use the cards dealt, so the influence of skill bias is diminished.
In your example there are 24 H's that have anything after it, 12 of which have an H after them. Thats 50/50. The only problem is splitting the occurences up in groups of four, but then weighting them equally when taking the average.
+John Galmann greyareabeyond allready pointed this out I see
7:48 This is where you lost me. "So there is actually a bias in the sample estimate because a lot of times when you just make one shot, there's a miss right after it." So what's wrong exactly with just averaging these cases? And what does that have to do with the formulas for sample variance versus population variance? I mean, I would kinda buy an edge effect but for some reason the sentence you said did not make any sense to me.
+Penny Lane We want to estimate the probability of a hit after a hit--this is P(H | H). The sample estimate found by averaging the trials. We found this to be 40%, which is less than 50% because of all those 0% cases. The reason is the sample formula is biased--similar to how the formula for sample variance is biased unless you adjust it.
+MindYourDecisions Hm. Maybe we have just different ways to look at it. The way I see it, this is an artifact of looking at only series of four throws and therefore ignoring the last throw because it has no throw following it. That's what I meant by "edge effect." The average definitely approaches 50% if you increase the length of the series you look at. But as I said, maybe that's your idea as well, you just think of it differently. Or maybe I'm just wrong, who knows? ;)
Yeah that's not a problem with the formula, the fact here is that they have considered every sequence to be equally relevant while clearly they should have made a total average.
HHMH is actually [HH], [HM], [MH]. Only ONE out of THREE is a double hit. Shouldn't the probability be 33.33%
+Cesar Coll Only [HH] and [HM] count and it's 50% that the second is a hit. The observation [MH] has a miss in its first shot--we are counting only hits in the first shot.
Got it. 50% of the times that we go "OMG... last shot was a hi... will he make this one???" will be followed by "Yes! Score!!"
This shows that sometimes we oversimplify things when using statistics. If you watch sports closely, different players are different in when their hot hands tend to occur. Some tend to be early in the game, some towards the middle, and others toward the end. The typical expectation of the hot hand tends to be that it will kick in somewhere in the middle because the player has reached a point of being warmed up.
That being said, let this be a warning to all looking at statistics. Be careful of limiting yourself in factors involved in your statistics because that can lead you to errors in the analytical end.
I agree, there are many subjective factors such as illness, adrenaline, crowd noise, coaching, defensive alignments...I can go on and on. It seems out of place to apply statistics to such a subjective scenario (unlike flipping a fair coin).
To those who've ever played sports outside of school gym class, the claim that the hot-hand is a myth has always been laughable. Macro-statistics of randomness and chance have nothing to say about the mechanics of the body being hyper-coordinated and in greater sync for brief or extended periods, to accomplish sporting tasks more effectively than at other times; statistics is more fit to calculate how often and regularly such periods occur, but not whether a play or a shot is affected by a previous one or whether in a string of plays/shots, those plays/shots are related at all.
+bronzenrule On a related note, there is a huge backlash against the use of analytics in basketball and sports in general. To the extent we recognize patterns, it's because our brains are acting like computing machines. So analytics suggests IF we can figure out how our brains recognize patterns, then we can do a similar analysis using computers, but we'll make the assumptions explicit. The hard part is creating a good model and collecting proper data.
+bronzenrule Hear hear. People aren't coins.
+Paul Vanukoff +bronzenrule Don't forget, though, that the original paper did not reach its conclusion by imagining that people were coins. The original paper examined actual performances by actual people, and appeared to conclude that people were no different from coins for the purposes of trend analysis. For the reasons bronzenrule points out, this result was always surprising and perhaps it should have been more closely examined sooner. If, as now seems likely, the true analysis is more in line with our expectation, then that actually demonstrates that statistics, when applied correctly, DOES calculate whether plays or shots are affected by previous ones and whether plays and shots in strings of plays or shots are related -- if it doesn't, it's being done wrong.
Perhaps we need to always keep in mind what we're dealing with when we're doing mathematics. In this case, we're dealing with human apes playing a game of moving a ball.
There might be no reason to suppose that past events (which fundamentally are evolutionary) are not going to change the probability of animals moving a ball to one location or another.
People aren't flipping coins when shooting hoops. If you tell someone they have been shooting better recently, perhaps they will try harder.
Or have the authors of the papers explicitly accounted for this idea?
+Jesse Cummins Psychology can play a role in sports. You could say this is a meta-discussion: it's about the psychology in regards to the results. We think the hot hand exists, but do statistics bear that out? Teams are now using basketball analytics to make sure they maximize the value of their players and possessions. If they knew players can get a hot hand, they would design more plays for them. It might also help the player's confidence to know that, but these papers are about analyzing the results rather than the player's mental state.
+Jesse Cummins The study did account for this idea. It didn't model people as coins, it took actual performances of actual human apes playing actual games of moving an actual ball. If there was an error -- which now seems likely -- it wasn't a conceptual error of trying to apply statistical analysis to player behaviour, it was a functional error of applying the analysis incorrectly.
+Jesse Cummins Statistics is not mathematics!
Humans hitting hitting shot is not the result of random probability, and the definition of "hot hand" as being more likely to make a shot after making a shot is not reflective of what basketball players would call a hot hand.
topics like this is why i’m a stats major and why i want to be a scout after i graduate. super exciting stuff. personally i think that the hot hand is real and the streakyness seen in individual shots has more to do with controllable factors (like the coaching, shooters positioning and defense and stuff like that) than just random sequences
I don't need to write a paper on it, here you go, a scientific debunk:
get a "cold hand" on purpose = "cold hand" exists = "hot hand" exists
Great video, I love this channel so far! My one suggestion would be to space out the letters evenly at 6:00 just to made it easier to read. Otherwise, nice job! :D
+hdog679 Fair point! Sorry I should have used a different font on that.
I didn't read the methods section of the original paper, but... Wouldn't a hot hand have to exist just due to simple learning and feedback effects from shooting shots? If you looked the same player at different stages in his career, if he or she has improved his success rate over time, then P(hit|hit) will of course be higher than P(hit) overall because his earlier, lower hit rate is pulling down his average. As someone else already mentioned, a human shooting a basketball is not a memoryless process. Your brain is adjusting synaptic strengths after each shot you observe, depending on whether you make it or not. The visual feedback you received from making a shot 20 shots ago may indeed be exerting causal effects on the current shot through learning, no?
If you are into baketball and math, watching a couple of games, and noting, misses and scores for every shooter, should give you a pretty good idea of whats going on. Do that for everymatch for the entire season, and you have more than enough data to know whats going on.
Practise does make perfect.
I recall seeing this paper being referenced in the book "thinking fast and slow" and by then it really made sense to me! It's surprising to find out that they have been wrong for such a long time :) Great video, BTW!
+Rochester Oliveira Thanks! Yeah this paper is mentioned time and again. The only book I recall that ever criticizes it is "Thinking Strategically" by Dixit and Nalebuff. They point out that if a player has a hot hand, then the defense is more likely to crowd out that player. That could open up possibilities for other players. So a true test is not whether a player misses shots after making them--it's about whether a team does better. But no one has measured this. Plus this new paper says even more: players actually do get the hot hand--and that's seen even if the defense steps it up.
ps. just to point out MJ's career average FT's is actually about 83.5%...!!!!
So???
Wouldn't the bias of a last shot not followed by any shots be avoided by considering every 4-shot series as a band where the last shot is followed by the first shot?
+Guepardo Guepárdez Yes, if the 4 shot trials were "pooled" that would remove the bias. I've gotten a couple questions about this, and I understand people aren't going to read all the comments. So I'm going to copy/paste part of my reply in case people read this thread: "What happened in the original study did not pool the observations. It instead did something like averaging the trials, which gives a biased estimate. This example comes from the paper as well, and I like how the authors explain it."
My intuition is to do a hypothesis testing on if autocorrelation equals 0
Great video please do one on the mid range shot it isn’t a low Percentage shot. the closer you get to the basket the more likely you’ll hit the shot. Both DeRosen and Dwade where great mid range shooters
3:10 I think you got your mords wixed up. You said sample of the average when you wrote average of the sample.
+Jordan Kloosterman Yes I misspoke there: I meant average of the sample.
+Jordan Kloosterman
" I think you got your mords wixed up " did you mean to switch the m and w in "words switched"
+Shoco462 Yeah, it works much better when you say it instead of writing it out. Its just a fun little habit I have.
+Jordan Kloosterman This is called a spoonerism, if you just so happened to not know that ^^
3:42 in other words, the only difference if you considering the universe or 3 planets in the universe
I'm both a math teacher who understands basic probability and a basketball player who's played ball for over 30 years (even at collegiate level). IMHO the 'hot hand' exists. I've experienced it myself several times and played AGAINST players who have torched me. I'd heard of the first paper that claimed that the hot hand was an illusion but I didn't agree. Anyone who's played competitive basketball long enough knows that the hot hand exists!
This paper is false, and I can show you a few reasons why.
1. Given a string of "hot hand" returns, you'll find that this changes the overall average. For example, let's look at Blackjack (because sports player statistics can change over time and basic strategy is a strict set), we will assert that in this game we have a 40% chance of winning the hand and we don't split or double to make it easier on the math. Given 100 hands, we can conclude that 40 of them were winners and 60 were losers on average. Now, take the same thing that is shown in this video and apply it and you may find that the chance of you winning after a win would be supposedly 52%. I ran it for several million and came up with a statistic close to 44.9%....which means out of 100 hands, 45 of them would be winners. This is false to the first assertion. Depending on which statistic to "ignore due to bias" will change the outcome % for the next "hot hand," which changes the overall statistic completely, unless it is extremely close to the overall to begin with.
2. I say "extremely close" because in order for the overall average to match, any data along the way must dictate the next outcome due in fashion to offset the local average. Meaning....instead of a "hot hand" it's actually "negative bias" which is opposite of the wanted result. For example, if blackjack gives you a 40% overall win average, then if you were to win 50% of the first 100 hands, you are set to win only 30% of the next 100. This isn't to say that you are guaranteed to win, it is only an estimate in order to keep the overall average proper. As you increase the local gap, you decrease the variance from the nominal until you reach infinite gap where the variance is 0% change (because after infinite hands, the next infinite hands should be exactly the same average of 40%). Because this becomes a limit problem toward 0 result, this means the very next hand after a win would be 40% - X and the next hand after a loss would be 40% + X, where X is equal to a infinitecimal amount. Notice the directions of chance are opposite that of what came out before, but the amount is effectively 0.
The problem here is that basketball players aren't random binary number generators. There are other human variable factors.
accuracy probability seems to be an impoartant factor
I watched Klay Thompson of the Golden State Warriors score 37 points in the 3rd quarter of a game last year! He did not miss a SINGLE shot the entire quarter!!!! He even took and made a 3 pointer that did not count. Please don't tell me that the hot hand is a myth!
+salsadancer00 How many player-quarters have been played in NBA history? And how many times has such an occurrence as you described (or better, without misses) happened? I don't know, but probably EXTREMELY infrequently. If what you described (or better) happened, say, on average by one player in one quarter per season, then you'd have a point. I don't think that happens.
sigh, but what if he actually does increase his hit chance sometimes because he's in better form etc pp?
I think the error actually occurs in the compensation for this 'form effect'. I haven't read either study, but what I think is going on is this: They wanted to know whether after a hit a player would have a better chance than would be expected given their form of the day. So what they did was comparing players' hitting percentage in a game with those players' hitting percentage after a hit in *the same game*. However, if there is no hand effect, in a game where a player has h hits and m misses, the chance of a shot after a hit being a hit will not be h/(h+m), which is their shot percentage, but (h-1)/(h+m-1) - the next shot could then have been any of the players' shots *except the one that it is after*. The error occurred because they compared with the first number rather than the second.
Why do MHHH and MMHH have 100%? And why do some of them have 50%? 50% isn't a possible outcome; the only possible outcomes should be 0%, 33%, 66%, and 100%.
BTW, *My* answer to 5:30 is 25% of the time. 50% if it's *at least one* H after H.
Because there is no "HM" (hit, miss) in them. There are only "HH"s (hit, hit). We can disregard any MM and MH because we're not interested in what happens after a miss. It's the same reason some have 50%. One case is dismissed because it starts with a miss, Leaving one HM and one HH.
You're not supposed to "dismiss" any cases, just reduce them from the total. MHHH is 2/3, and MMHH is 1/3. Additionally, MMMH and MMMM shouldn't be ignored. They still affect the grand total.
***** The initial question is about what happens _after a hit_. A MMMM and a MMMH both give you zero information about what happens after a hit. If you have a HM, you can deduce that this case resulted in a miss (0%). If you have a HH, you can deduce that this case resulted in a hit (100%). But with a MH or MM, you can't deduce anything.
+moismyname MMMM and MMMH should be considered 0%, not ignored completely. Probability is the ratio of the number of outcomes that meet criteria to the number of ALL outcomes.
The hot hand ben Cohen brought me here
this is true when basketball is played by spherical robots in a vaccum.
irl shots are not probability or random.
The video never made that sort of assumption about basketball players -- they used a simplified example to show off a particular form of analytical error in a more accessible way.
why does HHHH have 100%? the last H has neither after it, so shouldn't it be 75%?
+mrBorkD The last H has no shot after it, so it can't contribute to our counts either way. We have no information as to whether the next shot, following that H, *would have been* or *would not have been* an H or an M.
Dave Clark well it's going to be a 50 50 chance on it(on the next one)
This is kind of a problem in how you calculate the probability, since if you don't take it into account, you only take 3 of the Hs into account, whereas in sequences that end in M, you've got 4 letters.
@7:24
Shouldn't the 50% chances be 33%?
Isn't the 50% only true for infinitely many throws? Since with 4 shots, if the last one is a hit, you would have the opportunity to get another hit?
+Shadow4707 I think thats the point of the video and the exact problem of samples. Sure, 4 is a very small sample size but it shows the effect on the extreme end of it
+Shadow4707 The paper does talk about this point. I think they said the bias does go down when you increase the sample size. But the issue is the analysis done is often with small sample sizes--which is what we get in practice.
Why this?
MHHM: 50%
MMHH: 100%
+Cesar Coll In the first case, one time there was an H it was followed by another H, the other time there was an H it was followed by an M, so overall 50% of the time that there was an H it was followed by another H. In the second case, the only time there was an H followed by another shot that other shot was an H, so overall 100% of the time that there was an H it was followed by another H. This might seem paradoxical, because both sequences contain exactly two Hs and two Ms, but in the second case we simply have no information as to whether the shot following the second H would have been an H or an M, so it can't contribute to our counts.
We're interested in measuring how often we see the results of [HH] and [HM] in any four shot combination.
MHHM consists of [MH], [HH], [HM]. [MH] is not a result we're interested in so we ignore it. The next result is [HH] which we are interested in. The next result is [HM] which we are interested in. We have two interesting results: one [HH] and one [HM]. Thus we have a 50% for the [MHHM] combination.
MMHH consists of [MM], [MH], and [HH]. [MM] and [MH] are not results we're interested in. [HH] is an interesting result. We have one result and it's [HH]. Thus we have a 100% for the [MMHH] combination.
Make sense?