Daniel Kahneman - Noise: A Flaw in Human Judgment
HTML-код
- Опубликовано: 5 сен 2024
- Imagine that two doctors in the same city give different diagnoses to identical patients. Now imagine that the same doctor making a different decision depending on whether it is morning or afternoon, or Monday rather than Wednesday. This is an example of noise: variability in judgments that should be identical.
Shermer speaks with Nobel Prize winning psychologist and economist Daniel Kahneman about the detrimental effects of noise and what we can do to reduce both noise and bias, and make better decisions in: medicine, law, economic forecasting, forensic science, bail, child protection, strategy, performance reviews, and personnel selection.
SUPPORT THE PODCAST
If you enjoy the podcast, please show your support by making a donation. Your patronage will ensure that sound scientific viewpoints are heard around the world.
www.skeptic.co...
SPONSOR
Wondrium
wondrium.com/s...
SPONSOR
Brilliant
brilliant.org/...
#michaelshermer
#skeptic
Listen to The Michael Shermer Show via Apple Podcasts, Spotify, Amazon Music, Google Podcasts, Stitcher, iHeartRadio, and TuneIn.
www.skeptic.co...
Never seen a more humble person than Kahneman!!
The beginning of the conversation is a clear reminder not to believe something simply because there is a study (or even a group of studies) that reaches a certain conclusion.
jEven more, many scientific studies are controlled (simplified) to such a degree that real-world interactions are removed or ignored leading to false conclusions.
It’s true; but albeit imperfect it’s a best way we have to approximate the truth. The alternative is chaos based on each individual’s intuition or blind beliefs.
Thanks for this, Michael. Love Kahneman. Bought the audiobook!
Thanks! Shared the interview with my behavioral economics students. Hope they will enjoy as well!
And a strong ending.
At last someone who refuses to go down the free will and consciousness rabbit hole! 👍
Thank you i am grateful
Great stuff. Got me thinking again. Thanks, Michael and Daniel.
Somehow Daniel Kahneman makes me feel at home, it is like his researchers ressonate to me, easily. Maybe because he is noble prize!
Good job Michael, very hard interview to conduct. I'm a bit of a fan of Kahneman, but he pretty much gave the same answer to most of the questions. Still helps to drive the point home.
Yes, it got repetitive. I listened on audio and came here to see the reactions. There were some points where I wish Kahneman had expanded further, especially in terms of practical applications. He also seemed to fence-sit quite a bit when drilled on specifics. I suppose that could be his wisdom on display, but a bit frustrating nonetheless.
@@wodenravens the interview was only repetitive because Michael was consistently bringing up “red herring” examples of decision making based on biases and NOT noise. He missed the entire point of this mans work!
Interesting concept, but regarding the decisions of judges, I think we might be better off with noise than the mandatory minimum sentencing guidelines that have stuffed the prisons with non-violent drug offenders. That is the reason many if not most judges object to them.
@@unitewithch A large part of what we know about biases is also due to Kahneman's work (with Tversky). The distinction between bias and noise is subtle and not easy to understand without a little training in statistics. I think this video helps with the distinction.
@@crazierthan-u7571 Noise is variation; some judgements will be harsher or more lenient in non predictable ways. It makes the system less fair.
Where to set sentencing guidelines is an important issue which may help to reduce noise. Unfortunately due to everything in the US of A being politicized these guidelines are affected by bias.
The smart thing to do would be to study countries/states with low prison populations and low rates of recidivism to look for ideas to trial.
0.7% of your population in prison seems crazy.
I'm guessing this is not an easy problem.
*noise & bias both nvr applied to me,* but i do worry about my colleagues. gr8 work! -JC
Luckily, a good sound quality! But please remember to enable automatic captions/subtitles, it does a lot to help foreigners and hearing disabled persons. Thank you! Now, I'll be listening and trying to get all the words in the dialogue!
I am able to see the closed-captions and the transcript. Granted it's voice-to-text so the words that are hard to hear might be inaccurate in the transcript but it's pretty close.
I think Michael still hasn't fully understood the differences between bias and noise... he kept referring to biases over and over again. Hence why Daniel had to stress the differences so many times.
I don’t understand it either
Great interview thank you
Noise in Kahneman definition is that of a mistake that is built on the variability of a result or judgment. It is not systematic (as a bias) but just build on random variation. The surprise is that the variation is to great.
we believe in organised world
while luck decides most of details of big picture and we pretend only in zoomed out picture as system governing everything
we are biased focusing on what we assume is true forgetting what is true but in the dark often more importantly to what we decide or pretend to be in power of
btw thank you for summarising what's this material is actually about
always helpful
Machine learning/statistics practitioners have dealt with this topic for decades. In ML lingo: you can think of humans as predictive models. They make errors; these can be attributed to bias and variance. Ensembles of humans will have less variance than individual humans.
I’ve learned so much and have so much to think upon. Aldo isn’t Dr K such a lovely gentleman? I imagine studying under him would be a life enhancing experience.
Much psychological knowledge.
Best crossover ever!
This reminds me of Paul Watzlawick...The Invented Reality: How Do We Know What We Believe We Know?
Just circulating a year later, after observing behavior from a new environment . Bias and Noise, reception and perception of colour. Individual judgment hygiene can reduce bias and noise; statistical analysis enhances cognitive accuracy, unfortunately humans are fallible repeating the same mistake over again. Disastrous outcomes then become a norm, however, we can praise applied science in algorithm and rules that are noise free. HOPEFUL from applied Science which is not biased but factual and offers Accuracy. Thank you Daniel Kahneman and Michael Shermer.
your inborn mentality also limits you whole life to one side and then the bubble you choose to feel comfortable in too amplifying it
your diet can change functioning of brain
your mom's diet in pregnancy
maybe these are very general thoughts but hint on mechanics we often ignore and even if are aware hard to know if can objectify
even becoming a scientist is a filter already
and it would be good to have a minority report for that too
Come on, why did you not get the difference between bias and noise at 39:55 ??? Kahneman's answer is just polite. Of cause all people can have the same bias (e.g. WIRED-bias), put they do not produce the same noise.
52:00 or worse; anything that can be measured goes into diagnostic guideline. -JC
Thanks for great episode. May 12 '24 10:15 GMT
Somehow this stream reminded me of the "precision of the well-known" concept, originally created with the counter-naturalistic texts of the Russian formalists. The Russian formalists were a group of linguists and literary scholars who, in close collaboration with artists, reformulated aesthetic principles that had been known for a few centuries, but were -- and now again are -- about to be forgotten. In the years after the Russian Revolution, they sought to formulate what they considered "revolutionary literary theory and literary practice." Against this backdrop, another group, the Prague School, in its 1929 manifesto briefly and precisely formulated the "precision of the well-known", as the difference between automatic and actual use of language, and as an emphasis on the special social function of poetry and literature -- to 'de-automate' language by drawing attention to the words we use almost unconsciously and which are used against us because we do not think about them. The groups came to share a nasty fate -- both fell victim to dictators' manipulation with the "precision of the well-known" -- worn-out ideas and rigid 'truths' taken over without suspicion that they might never have been true. The efforts of the Russian formalists were stifled in the late 1920s and early 1930s, when Stalinist literary theorists introduced the only permissible doctrine of the Soviet Union: Socialist realism. The Prague School was stopped by the Nazi regime in 1939.
Thanks for sharing! I tried to google it but no relevant search results turned up. Do you know of any websites, researchpapers, books etc. that discuss this more in depth? Thanks
@@AndreasJansson2010 I do have a number of texts about it, but in books such as Gero von Wilpert's "Sachwörterbuch der Literatur" (German), so I'll try digging up something in English for you. Btw. are you part of the Skeptic team? (am ever so sorry to hear what happened to Pat Linse -- she was splendid working with).
@@AndreasJansson2010 Uf, you're right, it is very difficult to dig up anything about it in English language. You should, however, be able to find something in (admitted, I haven't read any of these works, as I mainly read in Danish and German):
Erlich, Victor. "Russian Formalism: In Perspective." The Journal of Aesthetics and Art Criticism 13:2 (1954): 215-25.
Mandelker, Amy. "Russian Formalism and the Objective Analysis of Sound in Poetry." The Slavic and East European Journal 27:3 (1983): 327-38.
Steiner, Peter. "Russian Formalism." The Cambridge History of Literary Criticism. Ed. Raman Selden. Vol. 8. Cambridge: Cambridge University Press, 1995. 11-29. 8 vols.
Seems to me as if the Prague School's manifesto was published in the journal Travaux du Cercle Linguistique de Prague, 1929. Perhaps you can find a translation -- or related writing -- in:
Steiner, Peter: "The Prague School: Selected Writings, 1929-1946", University of Texas Press Slavic Series, 1982.
Loved it.
Thought this would literally be about noise and how it affects us. Listening to all the sound systems and motorcycles that got bought with my unborn children's paycheck last year is definitely a problem.
Thank you.
Regarding how averaging can eliminate the variability when many people measure the *SAME* object.. and how that doesn't cancel t the noise when people are measuring different cases..
If the this noise between people is caused by individual biases, previous nights sleep, pre/post lunch, sports team winning, etc... how is that different than the (defunct?) studies on priming (smaller plates, classical music at the wine store, more choices leading to fewer purchases, etc)? For example, the Scarcity book by Sendhil Mullainathan talks about how psychology priming works in relation to scarcity of time and money.
Also, I imagine that noise (std-deviation) could be minimized by showing how one's judgement differs from the peers' average?
Like the first subject or priming indicates, I think we do not know where OR WHEN the noise that affects us has come from or what affect it is having on us as we seek to rationalize the universe, and maybe we can never know that or never know that for certain. That kind of explains how our unconscious can control our behavior and we can even know that and expect it and see it, but it still does. And why the world's government are so bad - or just good enough to maintain what seems like a stable forest for predators to hunt in, but not enough for them to be seen or found or dealt with.
This was an interesting interview, and it managed to keep my attention more than the book did. "Noise" wasn't that great, it was long winded, and hyper-focused on rather unimportant or less valuable bits of information. I haven't read 'Thinking, Fast and Slow', but I hope it's much better Noise.
The problem is people judge between what they consider good and evil rather than try to adjust towards balance.
effects of monotheism and centralism
there is only one objective truth
but it doesn't mean what we know is that to any degree
zero humility in this civilization
Hiring is wrong much of the time- I worked hard to get certain people that did not satisfy, while I reluctantly accepted a person who did much better at achieving what I expected others to succeed at!
You haven't edited the part he told after he took the affirmation from you that it's edited Dr Shermer?
I remember that he said the example somewhere else too I don't know why he would want it to be edited
Also check Robert Salposky explanation on behaviour (or some aspects of it).
TRuth is subconscious. Beauty too.
20:00 kahnemann here seems to use a different definition of noise and bias of what the general measurement theory says, what he had just stated before and even what's in his own book if i remember it correctly.
Here they seem to conclude that noise is among different people and bias within the person.
But as for measurement theory, if you see a judge inexplicably varies the sentences for the same crime along the day ( 3 - 6 years, say, for the same crime) and you cannot pinpoint when and how, that should be noise; if you eventually manage to notice that the judge is statistically consistent in being grumpier and more severe in her sentences _before lunch_ , that's bias. (Than it would be arbitrary to say that the 6 years grumpier sentence is the standard, by defining a full stomach bias of -3 years, or the other way around).
I'm not entirely sure kahnemann uses this same definition consistently.
In light of Lisa Feldman Barrett's research and theory that the brain is in service of regulating the body and predicting best actions based on prior learning, experience, and memory as a network constructing a model of the body in the world....the reality of noise seems unavoidable. Maybe a degree of mitigation can be imposed... but not elimination.
Did anyone ever think that all people see the world the same? Even the two headed lady can’t agree between them.
Does anyone else see noise in how they assign Nobel Prices.!
haha perfect comment
This whole conversation reduces to Kahneman trying to get Shermer to think on terms his particular conception of noise and Shermer falling back to think and talk on terms of bias. I really didn't get the desire to read Kahneman's book from this conversation, it seems to me that he'll be just enumerating a list of places where there's noise and telling us to average to get rid of it.
I think it is more like Shermer is not understanding the definition of noise. Having read Thinking Fast and Slow, and listening to the first part of Noise, I am pretty confident that Shermer isnt fully grasping the differences between noise and bias. It isn't the smoothest interview. 2 other books this topic brings to mind are Moneyball and The Worm at the Core. They all touch on the same problems from different perspectives.
I think it is rather good as it really explains the difference between the two concepts. I found this conversation very helpful.
I don’t understand how bias and noise, as variables are quantified? What are the standards that go into the judgment of quantifying each variable?
Ok, decision hygiene.
Delay intuition.
Bias, in prediction, can be measured by the difference between the mean prediction and the true outcome e.g. 20 analysts predict Google's stock price in the next quarter and the mean (average) of their predictions is $3,000. The next quarter rolls around and the stock price is $3,100. The bias is -$100. The standard deviation of their estimates, let's say $500, is the noise - the measure of the differences of each of the estimates from the true outcome.
Lemme know if that answered your question
dude on the right is the coolest turtle ever
The guest is brilliant and the host is nice, but one would expect that a host would have read the book before an interview like this. Which obviously was not the case.
This video contains advertisement I actually watch.
Great Micheal. Fascinating and illuminating!
Pete Woodroffe Music Pete Woodroffe Music Pete UK
The 'religion of science', now that's something to chew on.
36:00
1:06:00
Crimes against authority. Punching a cop.
Wait ...
There is no climate crisis. It's actually perfect example of Daniel book think fast think slow for biased political views mixing with "science"
Stanovich is the pioneer, I would say, but maybe I'm splitting hairs.
It’s true that religious people and scientists have prior beliefs, which are perhaps mostly intuitive, but only scientists have posterior beliefs, which take data into account. That’s why a belief in the credibility of scientific orgs regarding climate change isn’t the same as belief in the credibility of a church
(Obvi non-scientists can have posterior beliefs about non-religious issues)
Discussion of climate change around 41:00 needs to take into account a few variables which counter the idea of “independent researchers”: (1) computer modeling is a commonly used with their underlying biases and “noise”; (2) the source of funding (Uncle Sam) itself has a inbuilt bias where reinforcing correlations/hypotheses are funded more often than, say, a proposed “skeptic”/“denier” (interesting choice of words btw) project. Not to mention the third type of bias discussed earlier: how we view the world varies from person to person. Interesting, thought provoking interview.
Amazing interview. I have preordered the book long time ago and watched this interview as well. Even after all of this i still don’t get why he is reducing the effect of bias in favor of noise while those variations that happens are because people are biased, and because they are those variations exist. So how come he keeps on saying its not bias and its noise while noise are caused because of it! I understand that they are different things and I understand that noise is much more useful to deal with in term of managing and mitigating but still we can’t say that noise is more than bias as it does not make sense!
In one response he mentioned because of their personality! So is in it effected by bias as well! The mood and weather are all factors that contribute to different heuristics and biases and thats why those variations occurs! Someone please correct me if im wrong.
because it's variability that's always there
no matter if anybody is biased or not
when you aggregate people their views on average get close to objective average
but every measurement is not done by average
biases are often shared systematic from their sources spreading like infection
this is about general mathematical distribution
and zomming always on only part of picture
maybe this distinction makes no difference
noise is not personal and won't trigger anyone 😂
@@szymonbaranowski8184 I agree, that when it comes to calculation which i'm sure its much more easier to get a useful results from, but, what im trying to say is those variability are caused because people are biased in so many ways in the first place. And its not accurate to say "biased or not" cause everyone is biased, its systematic. For example, one of the examples in the book is about doctors diagnoses, so many biases can contribute to this variability such availability, ostrich, optimism, choice supportive,..etc. So here im referring to the cause of variability and not its effect or result.
Also we do have many tools for noise reduction Izotope, Waves and many other plugins really do the job of reducing Noise, at least if you are trying to eliminate noise as much as possible, never possible to eliminate all noise, in many cases people want Noise, they want Fuzz or Distortion or reverberation in their works as it maybe a artistic preference to increase difference by making more noise than others make. Increasing differences of opinion and so forth maybe desirable to the artist. The errors are not just unwanted....we want the errors, the errors make it stand out or above the fray of noise reduction and perfection in sound...you can take tape compression, artists will introduce as much noise as possible to make the recording "warm". IN other words in terms of judges some will use that desire to increase the noise where they can in the ways they want to. Errors maybe desirable in some cases.
Not sure I follow. Are suggesting that a judge may desire variability in the harshness of a sentence, perhaps as an added deterrent, or perhaps to make a name for themselves?
This was a great conversation and like the prior commentator I thank you for it. I am sorry to say, however, that in my view you just don't get Dr. Kahneman's point. I see that your PhD is in the history of science. You are a humanist with a faith in human exceptionality and h. sapiens' capacity to "think" themselves onto what you have decided is a proper path. I wonder if you remember Leonard Mlodinow's statement when you interviewed him in September of last year when he said: “As a way of understanding the natural world, philosophy is dead." Your approach, as with most of those with faith in humanism, is more philosophical than sceientific. For me, Anil Seth is correct in saying that “We are biological flesh and blood animals whose conscious experiences are shaped at at all levels by the biological mechanisms that keep us alive” and has data to support that position.
I enjoy many of the the Skeptic talks you moderate and have been a subscriber off and on (OK. mostly off. I am an easyrider). I have found that like most humanists today, you are not a scientist (I do not consider Social Science real science) and your conversation with Dr. Khaneman made this manifest. As he himself said, he has come a long way since prior book. He was a real gentleman in the way he tried to correct you each time you cited an example of what you understood noise to be. Again and again he said, "That's bias, not noise."
After a few years working as a civil rights infused poverty lawyer, I turned coat and worked as, first, a prosecutor in King County, Washington, and,, after a few years, as a federal prosecutor in the East. In my experience, Kahneman's observations about what is going on in the legal systems in the US extremely accurate. I have read quite a bit of David Eagleman's writing and think his views, especially with regard to the problems with current legal systems, interfaces well with Kahneman's. I retired fully from the law in 2005 and feel sad that the area of endeavor to which I devoted so much of my adult life was so flawed in its approach to human behavior. But in my defense I will say that our knowledge of the human brain and its function was primitive untli the last decade or two. It is still quite limited but orders of magnitue better than it was even in the mid to late 20th century.
Kahneman is my most favourite scientist rn, however the interviewer seems like to jump to many subject, including politics, that I think daniel is trying to avoid getting fall into politic pitfall
Wondrium should partner with Curiosity Stream.
Why would a man or women lie in bed and describe someone/stranger of the opposite sex as "sexy" is the question!!!???
Why not? You assume it wasn't elicited by the Doctor. Not everyone has to pretend they are blind.
Did he not once tell this story but it took place in his car, while driving home?
A book about Noise without any reference to Merzbow, Throbbing Gristle or Whitehouse seems to me not a comprehensive work, maybe someone needs to incorporate all the various theories on noise into one volume. Of course my version of noise maybe not the same as your version of noise, you might prefer some other group out of the thousands to choose from, maybe Fennesz maybe more to your liking than Merzbow. We even have preferences in terms of the kinds of noise we like.
Noise as used hear has little to do with auditory noise. Kahneman could have used error or variance.
🐋
Faith and confidence, isn't that hair splitting? You can have faith in science and confidence in Jesus.
In Ghostbusters, Bill Murray says, "Back off, I'm a scientist." Kahneman is, thankfully, not one of those scientists. You, by the way, talk much too much, trying to make yourself important and knowledgeable. Doesn't work though. You're not in the same league as Kahneman.
I guess we dont watch this comedy often enough
Technology does not impress me.
Shermer: "If you want to get a spacecraft to Mars with people onboard you use science not astrology." Interesting that you bring that topic up, let's say you want healthy people on Mars, well you probably don't want people born in October or December being onboard if you want healthy people, those 2 months were found as the worst birth months to be born in terms of health. Way over-represented in illnesses and diseases.
How does this scan between hemispheres? Could seasonal weather at time of birth or during gestation not play a larger factor than, say, mercury being retrograde?
That kind of was the point that variations in health have to do with the climate you maybe born in. Winter tends to possibly result in lower Vit D etc. You'd want to take this into consideration in terms of who would be a colonist. Whom was born at a time that makes them more sickly vs whom was born in the healthiest time around May.
@@econogate funny - I’ve often wondered how much of a factor geography/ climate plays in who colonises/ explores
Can you cite where this was found?
What metric of health was used?
Which illnesses and disease?
That is very wrongheaded. The birth month is irrelevant as you will use other metrics to judge health and well-being. You are not taking a random cohort of people from the population. You would do health checks. Those are the data you would judge.