Number Needed to Treat: Treatments Don't Work Like You Think They Work
HTML-код
- Опубликовано: 5 июл 2014
- One of the problems with the way we discuss health interventions is that we see them in black and white. Something is either good for you or bad for you. Things are rarely that simple, though. Moreover, there's "good for you" and "GOOD FOR YOU". How do you tell the difference? Watch and learn.
Almost all of the data for this came from the amazing website TheNNT (thennt.com). You can go there to see those and more.
Additionally, Aaron's new book is out! Please consider buying a copy. He'd really appreciate it! dontputthatinthere.com/#buy_th...
John Green -- Executive Producer
Stan Muller -- Director, Producer
Aaron Carroll -- Writer
Mark Olsen -- Graphics
/ aaronecarroll
/ crashcoursestan
/ realjohngreen
/ olsenvideo
Debunking long held beliefs is one of my favorite types of videos. Please keep doing these!
He has so much beautiful math and research too :'3 Oh and he's fair about it too, don't forget that! :3
He's just going one step further but still being black-and-white.
If a pill prevents cancer in 1/50 people, it doesn't necessarily do nothing in the others.
Cancer is also not a perfectly discrete event, and there are markers of ill health that precede it, just like prediabetes. If the pill prevents cancer in 1 guy, then it might prevent the transitional stage in, say, 20 guys. And that would be a large QOL improvement.
Aspirin is a great example of this. It's use for preventing heart attacks is slim, but it has a massive plethora of other beneficial effects, and almost zero downside of taken under the care of a doctor
Please, we should put up this RUclips channel on TVs at waiting rooms in hospitals and clinics in Canada and the US. Gives waiting patients something interesting to watch, and they could actually make informed decisions. Patients have better expectations, doctors waste less time explaining the same thing over again, literally everyone wins!!
Plus, streaming the video costs nothing.
Please Healthcare Triage , do your own small study with some local clinics, and see how it affects patient outcomes! :D
When people talk about number needed to treat, they usually talk about how the therapy has little benefit for the individual. (Such as the NNT of 61 for Mediterranean Diet). They often neglect to talk about the huge benefit when applied over a large population (such as if 1 million people ate it then there would be 16 thousand fewer heart attack, strokes, or deaths). That is a serious savings of money, lives and disability when applied to a state or country.
Healthcare Triage I'd recommend being avoiding statements along the lines of "no benefit whatsoever." We really mean no quantifiable benefit. There may be interactions with other drug, diet, lifestyle changes that could magnify the benefit to meet the criterium. Also, while the patient may have still had the negative outcome (say, heart attack) it may be milder or they may have better recovery.
Saying "no benefit whatsover" is a similar binary categorization that you are aiming to correct. Rhetorically, it's very easy to fall into these binary traps when trying to make a point. I really appreciate your videos, thanks!
Facts are facts thou. You're just sugar coating because you feel it's bullshit
NNT really is a great way to quickly determine whether something is clinically significant or not, even though things are statistically significant, still.
Fortunately, number needed to harm has the same calculations as NNT. You can really quickly find both numbers given the absolute risk reduction of a therapy and the attributable risk (the harm) to see how much NNT vs NNH there is.
The rest of that is a judgment call.
i work in the "natural" section of a grocery store. If i could get everyone one in the world to watch one video, it would be this one. we get tons of people chasing super expensive diets and supplements that almost certainly aren't worth their money.
This is really helpful to me. I have had doctors try to convince me to do things I'm unwilling to do or cautious about, and I haven't had the language to have a productive discussion about it. Now I will know what sorts of questions to ask.
This was excellent, might be the most informative and simple video on the efficacy of drugs I've ever seen.
I just want to say that I really appreciate your videos here. Despite my personal issues with medicine, I really think your focus on evidence-based medicine and the emphasis on research has made you one of the best resources on the internet with regards to medical knowledge. Keep up the good work!
I just found this channel this week and I'm loving it! So much great information in short bits that I can understand and actually apply to my life. A lot better than all these articles about a "study that was done that shows" that leaves me with more questions than I had in the first place. Thank you so much for the great work you are doing.
Thanks!
Your videos are amazing and it shocks me how little I really know about health and healthcare. Thanks for the info and keep up the good work. I really wish this stuff was taught to the general public.
This show just keeps getting more educational!!! Keep it up guys!
Your videos are brilliant and fascinating. So informative! This NNT and the other NNH are both particularly helpful. Thanks.
Im revising for medical finals and you've saved my life! THANKYOU! Please keep making stats easier to understand
Thanks as an MD I've never had a good concept on number to treat; this was quite helpful !
This should have a billions views. Absolutely essential to understand risk versus benefit.
Wish a lot more people would understand this. Especially when dealing with cancer.
this is the best video that i have watched in years , this is the first i watch a video with out fast forwarding it!
The problem with absolute risk reduction is that it depends on the original risk: for instance, a person's chances of getting a heart attack in the first place. Since people's risks for different conditions vary, relative risk reduction is much more helpful if you're trying to evaluate a drug, and absolute risk reduction would be helpful on a person-to-person basis. A drug with a 50% relative risk reduction for heart attacks would be helpful for someone with a 50% chance of a heart attack, but it would not be recommended for someone with a 1% chance.
This channel is awesome. Great discovery!
I’ve been trying to understand this for a long time and finally I do! Thank you!
Really great video and breakdown - thank you so much!
Absolutely fantastic video and I can't wait for the next one. This is a simple and sound idea with applications all over the place, and getting it out of the ivory tower is a true public service.
Also, I find myself wondering how this might apply to non-medical interventions like food stamps or microfinance. Some simplifying assumptions have to be made to talk about treatment as discrete, but having a NNT for "lifted out of poverty" or the like could be a nice on-the-box number to motivate discussion. To the research(/blogosphere)!
Wow , the topic has been made so easy & simple .
Love this series. Just posted this to my KHIT blog.
Omg, I would like to thank you so much, I have an exam about epidemiology and I finally understood this topic with your video !
Am an EMT in PA. The State added CPAP to the EMT Scope of Practice because (so I've been told) the NNT for CPAP to avoid an intubation is 6. That makes it one of the most effective treatments in EMS, possibly all of medicine.
Thanks for telling us about this stuff. This is very helpful (:
I just discovered this channel and it's quite good.
This is wonderful, thank you!
This is one of the best damn channels on RUclips.
the more i watch your shows the more i get to know about my moms field as a Nurse. and what goes on in the medical field.
Ya, it's...a lot more going on under the hood than you'd expect.
Great stuff as always
Brilliant and mathematical reasoning. Thank you HCT for existing.
Great Video! Love it.
This helped me so much, thank you!
Great video. We need this kind of material playing across colleges on the first day of classes
Man this just reconfirms my terrible no-drug bias. I ever get a doc trying to sell me a drug I need to remember to ask for the absolute reduction figure.
Thanks for your explanation! it helps a lot
Thanks for explaining NNT
5:35- I doubt anyone in this comment section would know the channel Paul Harrell, but if I couldn’t see them, just audio, I’d swear that they were the same person for this line. The manner is exactly the same- professional
Thank you for the video.
Great explanation.
Great video.
Just got looking at your videos, I feel it would be interesting if you got doctors thinking contrary to your research and knowledge and debated different topics, so people can see all sides of the story. This is really cool though, I really like the whole "No, that's not quite correct, here is why, but you should take care anyway" approach to things. It is a really nice way to display information and lay it out on a platter for people to take or leave.
it bothers me to no end, that we don't live in a society that doesn't actively make decisions on clear and established evidence based studies.
There's almost always a study for everything (even outside of healthcare), and yet we all make decisions predominantly emotionally or intuitively, based on some soundbite and antiquated but widespread common knowledge (however flawed).
Maybe in other fields, but in medicine there is a huge emphasis on evidence-based medicine. My 4 years of medical school hammered in evidence-based medicine as the holy grail of medicine. Core measures for many different chronic diseases make sure that every doctor is held to some measurable standard by which evidence backs the standard.
Thing is, even such standards isn't always upheld due to existing circumstances, and it is in these circumstances that truly make a doctor shine. Anybody can know to follow standards, but it is those when the standard isn't quite the best fit for the patient where a doctor can make it better.
Medicine is definitely one of the better ones. I'm not in the field so I can't comment on an individual or personal front, but I meant more macro problems. Eg. in healthcare: preventative vs reparative care, money vs treatment, location vs. quality of treatment, fractured health networks and inefficiency, etc.
Specifically I was thinking about creating policy in a social sense, about fields outside of science, like economics, justice, or politics. They seem to base their plans on segregate theories, and biased beliefs, rather than multifaceted cause and effect, which is primarily driven by human behaviour.
but I could be a cynic.
This video really got me to think, but I probably wouldn't take a drug unless absolutely necessary; especially if it has a ton of side effects like most tend to have! Can't wait for next week's video :)
This is amazing. I feel like I have been kept in the dark all of my life.
Very interesting video! I really appreciate the ones that deal with odds and statistics. Two of my siblings are biostatisticians, and these help me understand their conversations. :D
Thanks for this. Statistics are hard for most people to understand intuitively and this video seems like it should help people understand.
Excellent!
TY for this and the NNH video. After watching them I was able to ask the right questions to help me decide whether or not to start Tamoxifen recently. When doctors throw around relative numbers, especially the "your risk would be cut by X%" it can be very misleading!
thanks for clearing doubt
You pissed me off with the GMO episode but you redeemed yourself with this one! Good job!
This is excellent
an excellant lecture
great lesson
Brilliant!
Very good episode. I fear I've been one of those people swayed by good vs. bad logic here quite a bit more than I would like. :p Time to change!
"To the research" should be part of every educational video on RUclips. Anecdotes will get you nowhere if you're actually trying to convince someone of your viewpoint.
Anecdotes should get you nowhere, unfortunately we live in a world where for a large percentage of the population anecdotes will get you farther than research will.
kanklez Unfortunately an uncomfortable number of people accept the multiple of anecdote as "data"...
People listen to stories. If you can present research as a good story, more people will listen.
There was a study several years ago that found that if you show someone clear evidence that something they believe is wrong, they are likely to believe the wrong thing even more firmly than before.
Gareth Field Yes. hearts AND minds.
I'm surprised this video did not come back from dead after vaccine efficacy discussions. Man, it is awesome to find an explanation years before covid hit. Because this video and its conclusions is not biased due to covid-19 debates and so on. However, it might be biased due to other problems, but at the moment I don't care about those problems. Thanks.
Thank you so much!!!
ahhhhhhh-mazing. by far my favorite explanation.
It should be noted that in the aspirin example that he specifically said FIRST heart attack. The numbers for secondary prevention of heart attack and stroke are different.
You should get your units straight. The absolute risk reduction is: Old risk - New risk = Absolute risk reduction.
If you really want to express it in percentages, do so on _both_ ends of the equals sign. The dividing by 100 is really dividing by 1 (100%) in disguise, so don't do that. The results would be the same, but it would be so much more understandable.
E.g. 50% - 25% = 25% absolute risk reduction (or 0.50 - 0.25 = 0.25).
Love how this channel makes me think c:. Thank you
Awesome!
Amazing thanks
Thank you
Hey Healthcare Triage. Love this channel. Ever consider doing a episode on combating the claims of the anti-psychology movement? Not that the field of psychology is without room for criticism but I think your way of investigating such sensationalized views would be helpful.
I know the difference between good for you and GOOD for you!
Well then, good for YOU. :)
Hey Healthcare Triage! Could you guys do an episode on antidepressants and, if possible, antipsychotics? There's some conspiratorial controversy on these medications and I would enjoy a thorough video on this topic. Thanks!
Quick question: speaking as a doctor, what was your opinion of the staff hotdog eating contest?
Remember what he means by "no benefit". If you eat a Mediterranean Diet and don't have a heart attack, then that doesn't necessarily mean that the diet is what made the crucial difference, it means that there is a 1.7% chance (assuming that statistic is perfectly accurate) that you were really high risk for a heart attack this year but that there was just enough collective lack of unhealthiness in your life that you barely didn't have a heart attack, and that the Mediterranean diet was one of those parts.
So helpful
Wow, fascinating particularly from a public policy stand point!
Thank you ! You must be one of the bravest people in the world taking on billionaires like this.
I'm getting a new doctor ! ( just because of the aspirin).
And for further clarification, the number needed to treat is also just a rough estimate, and not the real number. This is because if four people are using a drug that has a 25% success rate, it is actually about a 70% chance that one of them will receive successful results. However, those odds decrease with size. If you had infinite people and a correlating infinitely small chance, there is a 63% chance (1-(e^-1) to be precise) that one person should see successful results. Of course, though, because there is a 40% chance that two people would see success, the average amount of people seeing success in a trial (with X people and a 1/X chance of success, and infinities subbed in) seems to be around 1.7 or 1.8. I haven't done all the math ;). Basically the numbers will be close to that with any sample size over a hundred.
Conclusion: 1.7 people would likely win in a lottery with 100 people entering and 1% odds. So, the Number Needed To Treat; the number of people needed to use a drug for one person to have it work, would actually be calculating the number of people (if the odds of it working are only a few percent above zero or less) needed to use a drug for it to work on 1.7 people, a fairly major difference.
numbers may seem small, but when you blow them up to a population level.... we are talking about a lot of lives
A related Idea from David Friedman: "From Someone who studies the effects of different diets comes up with evidence that consuming more salt causes high blood pressure and associated medical risks and concludes that people should eat less of it. Another researcher repeats the study looking at mortality from all causes and finds that people who eat more than the average amount of salt are no more likely to die early than people who do not.
Arguably you not only should not be surprised at the result, you should expect it."
Great video-NNT is a hugely important concept. But he’s making some statistical simplifications when he says everyone else “received NO benefit.” That makes it sounds like waste. That’s not how probability works. Everyone received a small probability of benefit, and probably were helped somewhat.
great video! thank you. are there any good criticisms of this sort of statistical analysis?
For NNT, it isn't exactly that 1 person out of 'x' will benefit and the others won't at all. It's more like, 1 person out of 'x' will for sure benefit (as defined by those giving the treatment) and others will experience a range of effects from harm to benefit and in-between. Other things like safety profile of the drug will determine whether those other people experience no change, mild improvement or worse actual harm
could you do a series on the benefits and weaknesses of different studies: RCTs, cross sectional, cohort studies?
Can you do a video on Australia's healthcare system?
If this was common knowledge, I think people could feel more in charge of their medical decisions and plans. But as time has shown, it's easier to rely on what marketing waters down and tells us because it's on our TVs, internet ads, and radios, not somewhere on the internet that you have to search for, a doctor's desk drawer, or deep down in PI sheets.
Could you do an episode on fluoride in water? I think it would be interesting for folks on both sides of the issue.
Parentheses! (50-25)/100
The order of operations applies to physicists and physicians alike :)
There's a bit of bad math in this, the chance that you will have one or more successful trials out of 4 trials when the chance of success is 25% (see around 4:30) is calculated with cumulative binomial probability, not with a simple division of 100 and the probability of success. In reality, across 4 trials the probability you will have 1 or more successes is (4 choose 1) * ((1 - 0.25)^3 * (0.25^1) or roughly 68%
-- Edited for clarification and grammar.
I would encourage you to consider the real, and well documented effect of placebo in an educational video such as this. Even though you cannot put hard numbers to that effect as easily as you can on these other statistics...
Multiplier effect of "good diet", blood pressure pills and statins over lifetime makes it worthwhile. Treat arteries in early stage of disease and will get better results as there will probably not be residual risk seen in treating late stage disease. Statins cost $40/ yr. Safe after 27 yrs experience. Good lecture.
Do a video about phone radiation Health effects.
And knuckle cracking.
Scishow (Made under the same basic group of people) Did a episode on that, it does nothing to the body in a negative way.
Sci Show just did an episode on knuckle cracking. Spoiler alert: it's harmless.
Peter Fitzgerald Hold my beer. Gonna go crack my knuckles in my mother's face.
Our healthcare system tends to describe things as black and white, true. But, this video also reduced things to black and white. For example, in the Mediterranean diet scenario, 60 have "no benefit at all" and 1 person wins the lottery and is saved from their "major cardiovascular event". But, as in the rest of life, there's many other benefits that weren't measured in that NNT. A healthy diet can have a profound effect on increased quality of life. Drugs too can have other benefits than what they are primarily marketed/intended for. So "no benefit at all" is not really an accurate statement. A better phrase than "no benefit at all" would be "no measured benefit".
+Healthcare Triage Hey, mind if I caption this video? (Or to be more precise, if I captioned it, would you apply the captions?) I've got an assignment in my ASL class and an interest in telling people how to use math to improve their lives. You've got a Deaf audience who probably struggle through some of the awkward RUclips machine captions. I'd fix that. Win/win/win!
can you do a video (sorry if you already have... this is the first one I've watched) on clinical vs statistical significance? It dovetails from this video nicely
Can you do a video on the advertising of drugs and how it affects our drug usage?
1% is a lot though, when talking about death. Some studies have suggested that people tend to value a 1/10,000 annual chance (0.01%) of death at roughly a $1,000. For example, to take on that sort of additional risk at a "dangerous" workplace, workers tend to receive a $1,000 premium in salary. Small risks do matter to most people.
So yes, be mindful of the true benefits and costs of any treatment. But don't dismiss a risk reduction measure out of hand just because 1% sounds small.
Whilst I agree NNT is useful for indicating the degree of benefit, the NNT presumes all the benefit is seen in one patient. So if we treat 4 people, and only 3 get a disease within the year vs 4 if untreated, there is a NNT of 4. But in reality some people get no benefit from a drug, some get a lot, but there is a continuum here - some people may get partial benefit. So if instead consider NNT of 4 with the benefit spread out, then instead of 100% prevention to 1 patient, one has 25% beneft amongst 4 patients. So if all 4 of our patients still get heart disease eventually, but at a 25% longer time interval than previously, then the NNT is still 4 (if one counts patients from a larger sample size), but that does not mean 3 out of the 4 gain no benefit at all (or that only one has just a 25% chance of benefit). Now of course however one divides/spreads out benefits of a low-number NNT, a NNT of 1000 is going to benefit few, or at least only very slightly to most people.
So is there a special name for the ratio of number needed to harm over number needed to treat(NNH/NNT[or nnt/nnh])?
Would you do this calculation for some other recommendations? Stop smoking, never start smoking, limiting calorie intake, exercise, laproscopic surgery, weight loss, etc.