Correction: 15:21 The left hand side of the equation should be “log(odds Obesity)” instead of “size”. NOTE: In statistics, machine learning and most programming languages, the default base for the log() function is 'e'. In other words, when I write, "log()", I mean "natural log()", or "ln()". Thus, the log to the base 'e' of 2.717 = 1. Support StatQuest by buying my books The StatQuest Illustrated Guide to Machine Learning, The StatQuest Illustrated Guide to Neural Networks and AI, or a Study Guide or Merch!!! statquest.org/statquest-store/
I am very emotional when writing this. I was struggling to learn logistic regression until I came to this channel and it has really transformed my understanding and confidence. I love how this channel uses visual tools and graphs to explain the concepts instead of some heavy dose of equations. This channel is a blessing to students like us who struggled during pandemic with classes.
Thi is sooooo true, this channel is a bless for students like us and the way your r explaining with practical example is like I've found my heaven. Many many thanks for these videos. You are one of my best teacher in stats.
im with u on this, the pandemic has made me lose so much of motivation in my studies but we can always bounce back :) lets get thru the semester together!!!
your entire story underlines the question of why do we ACTUALY need a university .. it costs a fortune and the professors tend to explain it worse than a YT video ... ofc .. there are fields where it's much much harder to put all the stuff into a short well done video .. but oof .. the more i learn on the job from other fields than mine .. the more i get the impression universities should be used to support learning not to be the be all and end all of education .. especialy not when it's increddibly expensive and or utter bs people study (f.nist glaciology for example..)
yea man, I took biostats three times in my life (undergrad, masters, PhD) and the only time things REALLY made sense at the intuitive level is watching StatQuest explanations.
@@MrSpiritmonger I have a graduate degree in pure math and I'm on here watching these so I can learn how to succinctly summarize things to my nontechnical boss (in this case I actually don't know the concept at all and this time it's for me, but that's how I found the channel)
Josh, ur my stats savior. Everyone start teaching very serious "let's get to the topic" but u soothe us with that gentle guitar beat and make us feel so comfortable. I just don't know how to express my gratitude, still thanks a ton and I mean it
You're an absolute gem of human being. It takes a special talent and surely a lot of work to be able to explain these concepts so clearly ! Thank you so much for sharing all of this with us :)
Can’t believe I’m saying this. Right after I finished watching a video of someone doing fancy snowboarding tricks, I started binge-watching your statquest videos and I got so addictive... I started hating stats for the last term because of a boring prof but you saved my butts!!!
4:50 in linear regression, y axis can have any value (which makes it easier to solve) but for logistic regression y values confined between 0 and 1 as it represents the probability. To deal with this, we transform the y axis to be log odds instead of probability Transforming probability to log odds using the logit function, logit = log(p/(1-p)) 5:50 how the axis changes The old y axis (in probability) that went from 0.5 to 1 goes to 0 to infinity (in log odds) 8:00 the log odds y axis transforms the curved plot to a linear plot 8:20 the coefficients of logistic regression are for the linear plot of log odds axis 8:48 you get coefficients of the kind just like linear regression 9:30 don’t understand, check odds ratio stat quest
StatQuest, you are absolutely the best video material on RUclips!! It's funny but it's also in-depth and complete. I wish I could learn all of my academic courses with you.
Absolute Stunner! I sat through hours of classroom lectures and mentoring yet couldn't understand it simple enough like you explained on this video. Thank you!
Your channel is truly excellent. I watch these videos, then read my textbook, then my lecture, finally complete / code my results. It’s proving very helpful.
Could not resist joining your channel after seeing this video. Damn, you are genius Josh and indeed a blessing for people like us who get overwhelmed by Statistics due to the heavy theory prevailing everywhere. Absolutely loved the numbers approach which shows how the logistic regression is calculated!
Respected sir. I teach econometrics to scholars with minimal fees to support them in research. For every batch in mle and logit model I share your videos with full confidence. We all are thankful to you. I can see your dedication and love for your work. Thank you so much.
Over the past couple months I have been self learning stats and data science and struggled a lot even though I genuinely loved learning about them. I am a visual learner so it was so hard to comprehend many simple concepts sometimes. I spend many hours a day on my own accord studying, and most of them are wasted on trying hard to understand a concept visually. Even though ChatGPT saved me a lot, they have so much trouble visualising concepts. Now I chanced upon your channel, I am just honestly dumbfounded by how visual your explanations and how good they are. Just speechless, and I just wanna leave a comment of appreciation. I sometimes feel proud of myself being able to visualise simple concepts myself, but those are nothing compared to your visualisations. This amount of visualisations especially for complex topics 1000% requires a certain talent and an extreme understanding of the concepts. Wishing for your continued success ahead
@@statquest Thanks! 2 days of watching several videos, I couldn't help it but decide to support you by just placing an order for your book from amazon today. :') ps: i genuinely dont buy books often, maybe once in every few years. That's how grateful i am. Thank you Josh
Josh, your videos can never give me a break. Always have something new to note down. I've been practicing Logistic Reg and Deep Learning for a while but I never knew Logistic Reg is grounded in log(odds). I do not think any ML/DL books or courses actually cover log(odds), at least in all that i've read/studied. They only mentioned the activation function which is the sigmoid. Thank you for the awesome video as always.
Hi Josh, I'm from Vietnam. I have read a lot of literature related to econometrics but it is only after watching this video that I really understand what maximum likelihood estimation is. This is really the pinnacle of Mathematics!
These videos totally help in getting a glance again from starting!! when you study so deep that you forget where you started1! totally solves Occam's razor problem!!BAM
i am currently doing a degree in Comp Sci and AI and am regularly referring to your videos as they explain the concepts so well. Really useful channel!
Hi, I just wanted to say thank you for these videos. I have tried to help a friend with this for a while because her teacher isn't very good at explaining things and it's been hard because it's been almost a decade since I took a statistics course. Your videos are extremely helpful and your explanations are so good, I think both me and my friend have learned more in two videos than we have during the entirety of her course. Thank you ❤
When I first watched your linear reg videos, I think they are not as good as probability videos. After I watched your linear + logistic reg videos, I know I was totally wrong. You link those two brilliant things together in such an easy and intuitive way. So BRILLIANT!!!
@@statquest If you could add assumptions explanations about linear and/or logistic regression, that would be fantastic. A lot of interviews, from financial quant to tech industry, they love to ask those.....
I was doing the DATA SCIENCE CAPSTONE COURSE on linear regression! horrible teaching! Man you killed it! I finally understand log odds and how it manifests as log odds ratio! thank You! Statquest! The comparison to linear regression! My god! Great stuff and Great teaching!
I really love the text that displayed when you explained. I'm not good in English. Sometime I cannot catch up the word you said but I still clearly understand your explanation because of helping from that text. Thank you.
I see why you made a wonderful video about statistics. It explained by the song in the beginning of this video!! You made it by joy and passion. Thanks so much for your videos
00:02 Logistic regression coefficients and their interpretation 02:20 Logistic regression is a type of generalized linear model used for predicting obesity based on weight. 04:46 Logistic regression transforms the y-axis to log odds of obesity 07:05 The log of one divided by zero equals positive infinity 09:43 Logistic regression coefficients explained 11:59 We fit two lines to the data and use them to predict the size of mice. 14:31 Two lines are fitted to the data to represent the log of the odds of obesity for mice with normal and mutated genes. 16:54 Linear models and logistic regression have similar concepts, but the coefficients in logistic regression are in terms of log odds.00:02 Logistic regression coefficients and their interpretation 02:20 Logistic regression is a type of generalized linear model used for predicting obesity based on weight. 04:46 Logistic regression transforms the y-axis to log odds of obesity 07:05 The log of one divided by zero equals positive infinity 09:43 Logistic regression coefficients explained 11:59 We fit two lines to the data and use them to predict the size of mice. 14:31 Two lines are fitted to the data to represent the log of the odds of obesity for mice with normal and mutated genes. 16:54 Linear models and logistic regression have similar concepts, but the coefficients in logistic regression are in terms of log odds.
At 8:02, you say the log-odds transforms the squiggly line into a straight line but isn't the log-odds also a squiggly line? The log-odds is, after all, just the inverse of the logistic function (sigmoid). I must be missing something.
@@statquest Oh I see what I'm misunderstanding. I was thinking the squiggly line was the logistic function and the straight line was the loggit function when really we're just applying the log(odds) to the y-axis. Thank you so much for taking the time to respond to my question. I'm very grateful for your content.
I love the way you explain complex concepts in such a simple and understandable way! I'm currently doing a professional training in machine learning and many things that are just brought to me by complex equations are so much easier to grasp with your videos! If I could wish for something, it would be some additional Python videos. You do a lot with R, but I feel like Python is requested more often in the non-academic world.
In summary, there are five graphs/function within logistic regression. 1. p-odds: y = odds = p/(1-p). this is a part of inverse function p belongs to [0,1] and odds belongs to [0,inf). Two parts are p1/2 2. odds-log(odds): y = log(odds) this is a normal log function where odds belongs to [0,inf) and log(odds) belongs to (-inf,+inf). Corresponding two parts are 0
A very informative video. A slight observation re.terminology. In any regression modelling, Let’s take your example of predicting size of a mouse using weight as a predictor, what you’re actually predicting is not the size of a mouse, but the expected size of a mouse given its weight E[size /weight] = intercept + b0*weight. It’s crucial to introduce that probabilistic reasoning at an early stage. The idea that we’re trying to predict the expected value of the target variable given the value of a predictor, independent variable, betrays the method that we want to find the regression coefficient of weight that would render observing the size most likely, hence the famous MLE - Maximum Likelihood Estimation.
One request... All the videos available are randomly watched.... But can you suggest the sequence to watch.. So that the content becomes more structured for the learning path of ML
I all the time watching the videos of StatQuest and just wondering for two things: 1) Why there is no actual course, after which you can get a certificate as the teaching methods, explanation and instructor are awesome? 2) There are few, but still there are some people who click on 'dislike' sign to the video. I was wondering why, as there is no doubt that the instructor is explaining like for kids. For example, I have checked the PCA model for data reduction and StatQuest's explanation was the only one from which I finally got the idea of what is going on. So, BIG thanks to StatQuest...
Thank you very much! I would love to make an actual course - and maybe one day I will. Right now I'm spending all of my time making videos - that's my favorite part, so that's what I do. :)
Hi. I am trying hard to find out the calculation for arriving at the standard error of the co-efficient and the intercept. Would be helpful if this can be shared.
Josh, need bother you again. two questions: 1) 6:15 you map probability [0,1] to logs(odds), but how do you get the probability for each observation at the first place? 2) once you map probability to log(odds) 6:15 and 15:32, so y-axis is log(odds), how do you interpret it? at 15:32, you put "size" as y-axis, is that how interpret log(odds)? isn't it here the possibility to have obese? Thanks a lot in advance!
Let's start by just making sure the definition of the two axes are clear: At 6:15, I'm showing how different probabilities map to the log(odds) axis. So p=0.5 translates to log(odds) = 0, p=731 translates log(odds) = 1. Thus, each point on the probability axis translates to something on the log(odds) axis. OK, now that we have that part clear, let's talk about the probability that each mouse is obese. At 1:37 in the video I say that the blue dots represent obese mice and the red dots represent mice that are not obese. So the probability that blue dot mouse is obese = 1, since the blue dots represent obese mice. The probability that a red dot mouse is obese = 0, since the red dots represent mice that are not obese. Does that make sense? As for the log(odds) axis, this represents the change in the log(odds) of obesity for every unit change in weight (or in genotype). So, in the weight vs obesity example, if you have a mouse that is one unit heavier than another mouse, then the log(oods) increases by 1.83. Does that make sense?
@@statquest Great Thanks for confirming the log(odds) axis. But I am still unclear about the first question: so in 1:37 and 6:15, your red dot have probability 0 (to have obese), and blue dots have probability 1. But at 6:15, how do you get fraction probabilities (those p=0.73, p=0.88)? are they the result from your logistic regression curve?
@@Tyokok Oh, I think I see the confusion. p=0.73 and p=0.88 are not from the data or the curve. They are just example probabilities, values between 0 and 1, that I use to show how a point on the y-axis (probability) for the logistic regression curve relates to a point on the log(odds) axis. In other words, I just wanted to show how the formula log(p/(1-p)) = log(odds) worked, so I picked some numbers and plugged them in. Even though I could have picked any number between 0 and 1 for the demonstration, I picked p=0.73 and p=0.88 because I knew they would translate to nice, round numbers on the log(odds) axis. Does that make sense?
@@statquest Yes that's what I learnt from your video. What I don't understand is that when you mapping the observations to log(odds), ( since observations are binary, either p=0 or p=1 probability,) your log(odds) will end up with only position and negative infinity value in log(odds) space. Then it's not a slope line. Or I am missing something?
@@Tyokok One use of the transformation from probability to log(odds) is once you fit a curve to the data, someone can tell you that they have a mouse that weighs "x".... You can then plot it on the graph (along the x-axis) and use the curve to see what the probability is that that mouse is obese. You can the use the log(p/(1-p)) transformation to say what the log(odds) are that that mouse is obese. Does that make sense?
Notes for future revision. 08:10 *Squiggly line = Probability view.* (Probability vs variable values) Y = 1 / (1+exp(-(βo+β1x1+..))) *Straight line = Coefficient view.* (log(odd) vs variable values) Y = βo + β1x1 + ... Squiggly line Y=0.5 ⇄ Straight line Y=0 Straight line slope ⇄ Squiggly line ?? ---- 14:26 Categorical variable Still, *Straight line = Coefficient view.* (log(odd) vs variable values) But, because the X is categorical, we use delta between the variables for X. Linear Regression Continuous Y vs categorical X ⇄ Continuous Y vs categorical X and X delta Y = B1 x mean_var1 + B2 x delta_mean_var12 Logistic Regression: Probability Y vs categorical X variables ⇄ Log Odd Y vs categorical X variables ⇄ Log Odd Y vs categorical X and X delta In log world, delta ⇄ ratio Y = B1 x log(Odd_Obesity_var1) + B2 x delta_log(Odd_var1)_log(Odd_var2) = B1 x log(Odd_Obesity_var1) + B2 x log(Odd_var1/Odd_var2 ratio) 17:45 Therefore, in logistics regression results, the coefficient β2 applies to log(odd_var1 / odd_var2) i.e. log(odd ratio). X1 is log(odd ratio), not var2, count of var2, mean of var2 etc.
Hi bam, at 4.13 timestamp if the weight of the mouse is 0 we are getting some value right. What does it mean? Can we think like if there is no weight mouse still we can say generalised obesity of the mouse or what?? I'm requesting you to give reply
The critical part is the link between a normal variable, the probability and the log odds results. You explained a bit of it and i didnt see any better video, but i'd lie if i said i understsand that step already to a degree that i could explain it to someone myself.
18:36 Any reason why the scale for the coefficients is log(odds)? I don't quite follow why we have to do everything in log(odds) for logistic regression.
The log(odds) makes the problem solvable. A linear shape can only move in certain ways, and those constraints make it relatively easy to decide if we've found an "ideal" fit. In contrast, the squiggle can move in infinitely many ways, and trying all of them would take forever, making it impossible to decide if we had found an "ideal" fit.
In 16:15 we see it's easy to calculate the coefficients when X axis has discrete values (e.g. normal gene, mutated gene). Can you show how to calculate coefficients when X axis values are continuous (e.g. weight)? The next video says we just rotate the line to get the best fit when the values are continuous, but doesn't show any coefficient calculation like it is done for discrete values in this video. Or it's not possible to do such calculation, that's why we start with random coefficients (a random straight line)?
Logistic Regression is solved using an iterative optimizing procedure like Gradient Descent, which I explain here: ruclips.net/video/sDv4f4s2SB8/видео.html
Hi, @5:00, why did you say it's a problem in Log regression that y-value is confined to [0,1]. And why's it required or important to convert the y-axis from probability to log-odds?
Because the straight line (which is easy to fit to data) does not have limits on the maximum and minimum value. In contrast, the squiggle has limits on the maximum and minimum values and that makes it much harder to fit to data. In this specific case, the squiggly line has too many degrees of freedom.
I talk about that in my video on Odds Ratios and Log(Odds Ratios) at 11 minutes and 24 seconds in. Here's the link: ruclips.net/video/8nm0G-1uJzA/видео.html
@@statquest I must confess I am puzzled too. The Odds & Logs video shows calculating SE from the observations. But what are the observations here? We have one observation of slope. Are we to infer that there are multiple observations of slope when finding the best fit curve and these are what are used? Or am I missing something? Otherwise, great videos!!!
Hi Josh, thanks so much. Your videos are awesome. At 16:30, as an alternative or in addition to the Wald's Test, could we have calculated a McFadden's Psuedo R-squared value by comparing the model with 2 fitted lines to a simple model that only has an 'overall log(odds of obesity)' line fitted? I assume we could calculate it for a logistic regression with discrete data (like we can for continuous data as demonstrated in Part 3)?
In your video 13:36, I have 2 questions, 1- What does B1 and B2 means in the design matrix? 2nd question which is relevant to question 1 is that why we labeled mutated gene as (1,1) instead of (0,1)?
best explain ever. you have ability to make us so interested in stat. this channel is just my favorite. THANK YOU. i want to know if you have a book in this topic or an article?!
Great explanation, thank you! I have a quick question. If a feature is not statistically significant it means that is not good for the prediction right? So in the example at 10:38 it means that weight is not a good feature for predicting if a mouse is obese or not (isn't that counter-intuitive in some sense) correct me please if I am wrong. And what about the intercept? It means that the intercept found by training the model isn't the best one? Thank you!
In this case, the reason the p-value for weight is not significant is that we don't have enough measurements. To get a better idea on how increasing the sample size will help, see: ruclips.net/video/Rsc5znwR5FA/видео.html As for the p-value for the intercept. That simply means that the intercept value is not significantly different from 0 (given the small sample size). Generally speaking, the intercept, regardless of whether or not it is significantly different from 0, is of no interest to the researcher.
I'm not sure what time point you are asking about since the video isn't that long, however, I'm pretty sure you are getting odds confused with probabilities. They are not the same thing. Here's a video that explains the difference: ruclips.net/video/ARfXDSkQf1Y/видео.html
This is from models based on linear regression. To learn more, see: ruclips.net/video/nk2CQITm_eo/видео.html and ruclips.net/video/NF5_btOaCig/видео.html
One question, At 6:05, why are we trying to put different values to new Y axis from old X axis.. Are these the values from data set.. Sorry for such a basic question..
All we're doing is translating the y-axis on the left, which represents probability and goes from 0 to 1, to the axis on the right, which represents the log(odds) and goes from negative infinity to positive infinity. The reason we're doing this is that, by changing the y-axis, the squiggly line on the left becomes a straight on the right. It's much easier to solve for the optimal straight line than the optimal squiggly line. So, for computational reasons, we transform the y-axis from probability to log(odds). Does that make sense?
@@statquest Thanks for the explanation. I understand for getting 0 of new Y axis,we started with taking center of old Y axis which is 0.5 and put it in log function. But how the other values 0.731,0.88 are taken as an example here ?
@@shantanukathale7210 I picked 0.731 and 0.88 because they would result in "nice" numbers on the log(odds) y-axis on the right side. They are just examples of how the transformation works. In practice, the transformation (back and forth between the two graphs), is done for you, so you don't have to do it yourself.
I think the problem of my course is, it doesn't use the terms such as Wald's test, Fisher's Exact Test, but it only showed in mathematical terms and symbol. At first, I feel like I never learned it before, but when I revised it, I actually has learned this. I don't know why my uni syllabus doesn't use this global statistics terms in the course. They could be useful especially when the learning phase has entered machine learning phase.
The function on the left, which is curved, is the logistic function. The function on the right is a straight line and corresponds to the standard linear equation y = ax + b.
Great video again! However, I am still a bit confused about the Z values of the coefficients, after watching the other suggested videos linked to this one. How's the Wald Test play a role for the coefficients of the continuous variables explained at around 9:23?
Wald's test gives us a way to test the hypothesis that the coefficient is equal to 0. In other words, Wald's test approximates a normal distribution of random log(odds) centered on 0 and the further (and more standard deviations away) our estimate is away from 0, the smaller the p-value and more significant the difference.
@@statquest Thank you! I think I can get the high-level ideas in behind now. I've always struggled to understand which test corresponds to which distribution. I may need to delve deeper into this.
Hello At 3:25.... you wrote as SIZE = 0.86+0.7*weight... What I know is y = mx+c I am unable to compare these 2 equations..you said Y = 0.86 , m = 0.7 that is OK. (How did you calculate m). But What about " x " and " c " ...Please help.
What time point are you asking about? If you are asking about something around 14:00 , then the answer is in my linear models videos: ruclips.net/p/PLblh5JKOoLUIzaEkCLIUxQFjPIlapw8nU
at 9:55 it talks about the estimated intercept: I don't get why this is estimated or what the standard normal curve that you go on to show represents (WHAT has a mean of zero?). don't we definitively know the intercept of a fixed line? or is it a "translation" of the standard normal curve for weights (or odds of weights?) into one of coefficients (seems nontrivial that it would remain a standard normal curve given the log aspect)?
One of the main ideas behind statistics is to use a relatively small set of data to get an idea of what things would be like if we had a very large dataset. For example, we might measure the height of 10 people in a town and use the average height for the 10 people as an estimate for the average height for everyone who lives in that town. We then have statistics that give us a sense of how good that estimate is. If the 10 measurements for height were all over the place (some people were very short and some were very tall), then we might not have a lot of confidence in the estimated mean. In contrast, if all 10 measurements were very similar, then we might have more confidence that the estimate does a pretty good job representing the overall mean for the town. That same mentality is applied to logistic regression. We are using a relatively small dataset to make predictions about a larger population. Thus, the line that we fit to the relatively small dataset is an estimate of the line that would fit to a larger dataset. So the intercept in the line we estimated is, in turn, an estimate of the intercept that would belong to the line fitted to the larger dataset. Because the estimated intercept tells us something (in this case, it tells us that the log(odds) of obesity in the population is -3.48), we want to know how confident we can be in that estimate. So we do a test to see if the estimate is significantly different from 0. If the estimated log(odds) is not significantly different from 0, then we can't put much faith in it.
The most helpful thing would be if you named what sort of distribution we're talking about (the mathematical name), what the statistic in question is, and what the population and sample are. I'm probably just missing something in the video. *Issue A* If the coeff is the statistic, how can this vary at all, given that sigma and logit are bijections and we're transforming a particular value (or rather, function, but there's an associated uniquely determined coefficient that will result from the mapping of each function) rather than a random variable via logit? Are we talking about a sampling distribution of the coefficient, where we have a single sample of size = number of mice, and we're trying to estimate the "population fitted-line-coefficient"? * but can you even create a sampling distribution from one sample? And how would the size of the sample factor into that since they're all getting boiled down into a single coefficient? And we don't know the population size, right? (I know all of these things are factored in in an algorithmic way depending on the type of distribution, but I'm not confident that I know what type of distribution we're doing.) *Issue B)* But why do we expect the mean to be zero? I get that the first graph was showing probabilities so it's bounded by 0 and 1 (and in fact boolean), and that the mean of 0 and 1 is .5, then logit(.5) = 0. But just bc .5 is the mean of 0 and 1, the only two classification values that can be taken, that doesn't make it the mean of the original sample of the mice; in fact in this example it is not. Thanks for the reply, it must be a huge hassle to untangle all of these comments.
@@SpecialBlanket I'm not sure I fully understand your question, however, I get the feeling that you might want to take a step back and learn about how normal linear regression is done before learning about this, since normal linear regression sheds a lot of light on what we're doing here. Here's the link: ruclips.net/p/PLblh5JKOoLUIzaEkCLIUxQFjPIlapw8nU
Let me try to rephrase it. You are considering the coefficient from one sample. How are you calculating its standard error? Why do you expect the distribution of all possible samples' coefficients (that's the distribution with respect to which we're calculating the p-value for this particular coefficent, right?) to be normal? Or is this what is covered by Wald's?
The red dot at -3.48 refers to the y-axis intercept of the straight line on the right hand side of the screen. Sure, we then plot the red dot on the x-axis of a normal distribution, but it wasn't calculated the way we normally calculate a z-value (in other words, we didn't calculate -3.48 subtracting the mean and then dividing by the standard deviation).
Umm... This video explains how to calculate the coefficients for logistic regression. Here's how maximum likelihood is applied to do the estimation: ruclips.net/video/BfKanl1aSG0/видео.html Here's how R-squared and p-values are calculated: ruclips.net/video/xxFYro8QuXA/видео.html Here's how deviance statistics are calculated: ruclips.net/video/9T0wlKdew6I/видео.html ...and here's how deviance residuals are calculated ruclips.net/video/JC56jS2gVUE/видео.html
In your video 7:19, log(0) is infinity and therefore log(1)- log(0) should be log(1)-infinity = negative infinity. How did you get something - negative infinity instead of "something - infinity"?
16:50 Up to this point everything made perfect sense to me, but how is it possible to get a Std. Error value for the log odds ratio when it is derived purely from counting in a single sample set? I'd think it is impossible. You know, if you compare it to the t-test example: there each datapoint has its own value on the y-axis which allows for the calculation of the Std. error, but in the log reg you don't have anything similar.
The z-value can be used to calculate a p-value, which, in this case, will be greater than > 0.05. We know this because 95% of the area under the curve (95% of the probability) is within 2 standard deviations of the mean (and the mean is 0 for a z-distribution).
At 7:13 you've mentioned (log(1) - log(0)) as (something - negative infinity) == positive infinity.. but log(1) value is zero right..then it becomes (something(0) - negative infinity) == negative infinity right
17:45 In terms of interpreting the p-values of the intercept and the slope, does that mean having the Normal Gene does not have a significant effect on a mouse being obese or not, while having the Mutated Gene make a mouse significantly more likely to be obese?
When I tried to calculate log(2/9) using my calculator it displays -0.65757 but in the video at 14:52, it shows the result is -1.5. Can anyone helps me out finding the reason behind of it?
@@statquest Thanks a lot for this information. OMG! You're too good at explaining things. Already became one of your biggest fans. Looking forward to having a jam with you ^_^
Good video and good content as usual. However i feel i am still one step away from truly understanding what do the "Coefficients" actually mean, that is to say none of us think in chance in terms of log(odds). I assume, and googled, and i get the impression that using e^x with log odds will give me probability. And in your explanation i guess it means for the continuous variable that by increasing X by 1 the log(odds) increases by said coefficient. I appreciate your work and good teaching skills, also your singing is, surprisingly, good. Update: RUclips suggested a video you made on Log odds which has taken me further on understanding this log odds concept. The final stepping stone i needed to understand it was that both odds and probability can be defined using simply succesful outcomes and unsuccessful outcomes.
![Seungmin "그렇게, 천천히, 우리(As we are)" | [Stray Kids : SKZ-PLAYER]](http://i.ytimg.com/vi/kAzmhLHePqU/mqdefault.jpg)
Correction:
15:21 The left hand side of the equation should be “log(odds Obesity)” instead of “size”.
NOTE: In statistics, machine learning and most programming languages, the default base for the log() function is 'e'. In other words, when I write, "log()", I mean "natural log()", or "ln()". Thus, the log to the base 'e' of 2.717 = 1.
Support StatQuest by buying my books The StatQuest Illustrated Guide to Machine Learning, The StatQuest Illustrated Guide to Neural Networks and AI, or a Study Guide or Merch!!! statquest.org/statquest-store/
Josh.. thanks for these videos man.. one question did you mean obese = log(odds normal_gene) x B1 + ... instead of size = log(...???
@@borispenaloza6788 What time point in the video (minutes and seconds) are you talking about?
@@statquest Starting at 15:23.. you mentioned obesity but the equation shows size...
@@borispenaloza6788 Ahh, I see. That's a typo.
@@statquest yes.. but it was a great explanation!
I am very emotional when writing this. I was struggling to learn logistic regression until I came to this channel and it has really transformed my understanding and confidence. I love how this channel uses visual tools and graphs to explain the concepts instead of some heavy dose of equations. This channel is a blessing to students like us who struggled during pandemic with classes.
Hooray! I'm glad the video was helpful! :)
Thi is sooooo true, this channel is a bless for students like us and the way your r explaining with practical example is like I've found my heaven. Many many thanks for these videos. You are one of my best teacher in stats.
im with u on this, the pandemic has made me lose so much of motivation in my studies but we can always bounce back :) lets get thru the semester together!!!
your entire story underlines the question of why do we ACTUALY need a university .. it costs a fortune and the professors tend to explain it worse than a YT video ...
ofc .. there are fields where it's much much harder to put all the stuff into a short well done video .. but oof .. the more i learn on the job from other fields than mine .. the more i get the impression universities should be used to support learning not to be the be all and end all of education .. especialy not when it's increddibly expensive and or utter bs people study (f.nist glaciology for example..)
@@statquest DOUBLE BAM !!
Thank You for the beautiful and relaxing videos.
TRIPLE BAM!!! Thank you so much for supporting StatQuest!!! :)
Josh, I just want you to know you are the only reason I will graduate from my MS in Stats program
Wow! Good luck!
yea man, I took biostats three times in my life (undergrad, masters, PhD) and the only time things REALLY made sense at the intuitive level is watching StatQuest explanations.
@@MrSpiritmonger I have a graduate degree in pure math and I'm on here watching these so I can learn how to succinctly summarize things to my nontechnical boss (in this case I actually don't know the concept at all and this time it's for me, but that's how I found the channel)
Saaame!
How was it?
Josh, ur my stats savior. Everyone start teaching very serious "let's get to the topic" but u soothe us with that gentle guitar beat and make us feel so comfortable. I just don't know how to express my gratitude, still thanks a ton and I mean it
Thank you very much! :)
You're an absolute gem of human being. It takes a special talent and surely a lot of work to be able to explain these concepts so clearly ! Thank you so much for sharing all of this with us :)
Thank you very much! I really appreciate your support - it means a lot to me. :)
Can’t believe I’m saying this. Right after I finished watching a video of someone doing fancy snowboarding tricks, I started binge-watching your statquest videos and I got so addictive... I started hating stats for the last term because of a boring prof but you saved my butts!!!
BAM! :)
This is insane. Logistic Regression is one of the fundamental regression algorithms, yet no one is able to explain it with such clarity.
Thanks!
4:50 in linear regression, y axis can have any value (which makes it easier to solve) but for logistic regression y values confined between 0 and 1 as it represents the probability. To deal with this, we transform the y axis to be log odds instead of probability
Transforming probability to log odds using the logit function, logit = log(p/(1-p))
5:50 how the axis changes
The old y axis (in probability) that went from 0.5 to 1 goes to 0 to infinity (in log odds)
8:00 the log odds y axis transforms the curved plot to a linear plot
8:20 the coefficients of logistic regression are for the linear plot of log odds axis
8:48 you get coefficients of the kind just like linear regression
9:30 don’t understand, check odds ratio stat quest
6 years down the line, and your videos are still the best:) Thanks Josh❤
Thank you!
StatQuest, you are absolutely the best video material on RUclips!! It's funny but it's also in-depth and complete. I wish I could learn all of my academic courses with you.
Thank you! :)
JOSH, just like your videos, your music is incredible. Thank you for all the efforts you put in. Quadruple BAM !!!
Thank you very much! :)
Absolute Stunner! I sat through hours of classroom lectures and mentoring yet couldn't understand it simple enough like you explained on this video. Thank you!
Thank you very much! :)
Your channel is truly excellent. I watch these videos, then read my textbook, then my lecture, finally complete / code my results. It’s proving very helpful.
Thank you! I'm glad my videos are so helpful. :)
Could not resist joining your channel after seeing this video. Damn, you are genius Josh and indeed a blessing for people like us who get overwhelmed by Statistics due to the heavy theory prevailing everywhere. Absolutely loved the numbers approach which shows how the logistic regression is calculated!
Thank you so much!!! I really appreciate your support. :)
Respected sir. I teach econometrics to scholars with minimal fees to support them in research. For every batch in mle and logit model I share your videos with full confidence. We all are thankful to you. I can see your dedication and love for your work. Thank you so much.
Thank you very much! :)
Over the past couple months I have been self learning stats and data science and struggled a lot even though I genuinely loved learning about them. I am a visual learner so it was so hard to comprehend many simple concepts sometimes. I spend many hours a day on my own accord studying, and most of them are wasted on trying hard to understand a concept visually. Even though ChatGPT saved me a lot, they have so much trouble visualising concepts. Now I chanced upon your channel, I am just honestly dumbfounded by how visual your explanations and how good they are. Just speechless, and I just wanna leave a comment of appreciation. I sometimes feel proud of myself being able to visualise simple concepts myself, but those are nothing compared to your visualisations. This amount of visualisations especially for complex topics 1000% requires a certain talent and an extreme understanding of the concepts. Wishing for your continued success ahead
Thank you very much and good luck with your studies!
@@statquest Thanks! 2 days of watching several videos, I couldn't help it but decide to support you by just placing an order for your book from amazon today. :') ps: i genuinely dont buy books often, maybe once in every few years. That's how grateful i am. Thank you Josh
@@hgr126 Awesome! Thank you very much for your support!
Josh, your videos can never give me a break. Always have something new to note down. I've been practicing Logistic Reg and Deep Learning for a while but I never knew Logistic Reg is grounded in log(odds). I do not think any ML/DL books or courses actually cover log(odds), at least in all that i've read/studied. They only mentioned the activation function which is the sigmoid. Thank you for the awesome video as always.
Thank you very much! :)
Hi Josh, I'm from Vietnam. I have read a lot of literature related to econometrics but it is only after watching this video that I really understand what maximum likelihood estimation is. This is really the pinnacle of Mathematics!
Wow, thank you!
Am on a StatQuest Marathon.Addicted to this videos.Glad I found them. Asante sana.Much love from Kenya
Wow! Thank you very much! :)
Thank you so much! Your clarity is brilliant! If all the teachers in the world explained like you, there wouldn't exist such a thing as bad students.
Wow, thank you!
These videos totally help in getting a glance again from starting!! when you study so deep that you forget where you started1!
totally solves Occam's razor problem!!BAM
kudos to you, not everyone learns the same, you show everything explicit and on simple terms. Thank you for doing this.
Glad it was helpful!
i am currently doing a degree in Comp Sci and AI and am regularly referring to your videos as they explain the concepts so well. Really useful channel!
Glad you like them!
Statquest =infinite positivity
BAM! :)
Finally we have an awesome tutorial on RUclips on this topic.
Big fan😊
Hooray!!! I'm glad you like this video so much! :)
Hi, I just wanted to say thank you for these videos. I have tried to help a friend with this for a while because her teacher isn't very good at explaining things and it's been hard because it's been almost a decade since I took a statistics course. Your videos are extremely helpful and your explanations are so good, I think both me and my friend have learned more in two videos than we have during the entirety of her course. Thank you ❤
I'm glad my videos are helpful! :)
When I first watched your linear reg videos, I think they are not as good as probability videos. After I watched your linear + logistic reg videos, I know I was totally wrong. You link those two brilliant things together in such an easy and intuitive way. So BRILLIANT!!!
Wow! Thank you very much! :)
@@statquest If you could add assumptions explanations about linear and/or logistic regression, that would be fantastic. A lot of interviews, from financial quant to tech industry, they love to ask those.....
I'm from Chile, and i just want to say you: YOU ARE AN AMAZING TEACHER, PERIOD!
BAM! Thanks! :)
I JUST LOVE UR DRY HUMOR AND UR FUNNY INTROS. just watched the vids today and uve gained urself a new subscriber!! love from singapore :D
Hooray! Thank you so much! :)
I was doing the DATA SCIENCE CAPSTONE COURSE on linear regression! horrible teaching! Man you killed it! I finally understand log odds and how it manifests as log odds ratio! thank You! Statquest! The comparison to linear regression! My god! Great stuff and Great teaching!
This is the best song I heard from watching your channel so far :D Anyway, your clear explanation is awesome. Keep up the great work!
Thank you so much! :)
Thank you so much! You're one of the coolest and most talented teachers I've ever had!
Wow, thank you!
I really love the text that displayed when you explained.
I'm not good in English. Sometime I cannot catch up the word you said but I still clearly understand your explanation because of helping from that text.
Thank you.
Hooray! I'm glad you like the text. :)
I see why you made a wonderful video about statistics. It explained by the song in the beginning of this video!! You made it by joy and passion. Thanks so much for your videos
thank you! you really are a great teacher, there is nothing like a teacher who can explain something in such a simple way!
Wow, thank you!
Josh you are a teaching Rockstar the size of Walter Levin. Thank you so much! Just bought the book. Loving it!
Thank you so much! :)
00:02 Logistic regression coefficients and their interpretation
02:20 Logistic regression is a type of generalized linear model used for predicting obesity based on weight.
04:46 Logistic regression transforms the y-axis to log odds of obesity
07:05 The log of one divided by zero equals positive infinity
09:43 Logistic regression coefficients explained
11:59 We fit two lines to the data and use them to predict the size of mice.
14:31 Two lines are fitted to the data to represent the log of the odds of obesity for mice with normal and mutated genes.
16:54 Linear models and logistic regression have similar concepts, but the coefficients in logistic regression are in terms of log odds.00:02 Logistic regression coefficients and their interpretation
02:20 Logistic regression is a type of generalized linear model used for predicting obesity based on weight.
04:46 Logistic regression transforms the y-axis to log odds of obesity
07:05 The log of one divided by zero equals positive infinity
09:43 Logistic regression coefficients explained
11:59 We fit two lines to the data and use them to predict the size of mice.
14:31 Two lines are fitted to the data to represent the log of the odds of obesity for mice with normal and mutated genes.
16:54 Linear models and logistic regression have similar concepts, but the coefficients in logistic regression are in terms of log odds.
Hi josh i am from somalia your videos helped me to understand logist regressions , your explanation is very good thank you😊
Thank you! :)
Greetings from the University of Texas at Austin! I am going through a Machine Learning class and your classes are totes useful! Thanks!
Totes Bam!
Josh you're explanations are spot on!
Thank you!
This litterally could not have been any clearer. I have now 0 questions about this topic. Amazing teaching skills.
Hooray!!!! Thank you very much. :)
@@statquest Pam Pam!
At 8:02, you say the log-odds transforms the squiggly line into a straight line but isn't the log-odds also a squiggly line? The log-odds is, after all, just the inverse of the logistic function (sigmoid). I must be missing something.
When the log(odds) are used on the y-axis, than the squiggly line on the left (which has probability on the y-axis), becomes straight.
@@statquest Oh I see what I'm misunderstanding. I was thinking the squiggly line was the logistic function and the straight line was the loggit function when really we're just applying the log(odds) to the y-axis. Thank you so much for taking the time to respond to my question. I'm very grateful for your content.
@@nak6608 That's right. All we're doing is changing the scale on the y-axis from probability (squiggle) to log(odds) (straight).
I learned a lot watching your logistic regression playlist!
! Thanks for making these videos!
My pleasure!
I love the way you explain complex concepts in such a simple and understandable way! I'm currently doing a professional training in machine learning and many things that are just brought to me by complex equations are so much easier to grasp with your videos!
If I could wish for something, it would be some additional Python videos. You do a lot with R, but I feel like Python is requested more often in the non-academic world.
Thanks! I'm starting to add more python videos.
In summary, there are five graphs/function within logistic regression.
1. p-odds: y = odds = p/(1-p). this is a part of inverse function p belongs to [0,1] and odds belongs to [0,inf). Two parts are p1/2
2. odds-log(odds): y = log(odds) this is a normal log function where odds belongs to [0,inf) and log(odds) belongs to (-inf,+inf). Corresponding two parts are 0
Bam!
3:08 I can't stop laughing at your hilarious voice tone when you say "shameless self-promotion"! It's cracking me up!
Bam! :)
Mr Statquest, you are the best sir!
Wow, thanks!
i recommend your channel whenever people ask me where did i study from
Thank you very much! Sharing my videos is the best complement. :)
A very informative video. A slight observation re.terminology. In any regression modelling, Let’s take your example of predicting size of a mouse using weight as a predictor, what you’re actually predicting is not the size of a mouse, but the expected size of a mouse given its weight E[size /weight] = intercept + b0*weight. It’s crucial to introduce that probabilistic reasoning at an early stage. The idea that we’re trying to predict the expected value of the target variable given the value of a predictor, independent variable, betrays the method that we want to find the regression coefficient of weight that would render observing the size most likely, hence the famous MLE - Maximum Likelihood Estimation.
One request... All the videos available are randomly watched.... But can you suggest the sequence to watch.. So that the content becomes more structured for the learning path of ML
You can go to the playlists option on the home page and watch them in order by topic
@@wizeguy9285 but there will not be correct order
OMG find a full path on Internet for ML. Statquest is one of the best channels on RUclips but ML its much more than just a playlist to follow.
Go to play list you can see in order. You need to subscribe for that
I all the time watching the videos of StatQuest and just wondering for two things:
1) Why there is no actual course, after which you can get a certificate as the teaching methods, explanation and instructor are awesome?
2) There are few, but still there are some people who click on 'dislike' sign to the video. I was wondering why, as there is no doubt that the instructor is explaining like for kids. For example, I have checked the PCA model for data reduction and StatQuest's explanation was the only one from which I finally got the idea of what is going on.
So, BIG thanks to StatQuest...
Thank you very much! I would love to make an actual course - and maybe one day I will. Right now I'm spending all of my time making videos - that's my favorite part, so that's what I do. :)
@@statquest while no course, will follow your videos :)
for 13:53, we only care about the mutated mouse, so why do you have to set 1 to the normal gene mouse
Because that is how design matrices are created. For details, see: ruclips.net/video/NF5_btOaCig/видео.html
Brilliant and clear. Makes a relatively complex topic really simple. Thank you
Thank you! :)
You saved my Day, best videos 100stars 4 u. Even though my test will be in german, I understood it in english better. Thank u!!
discover this channel is the best thing happened to me!!! GRACIAS!!!!
Bam!
Hi. I am trying hard to find out the calculation for arriving at the standard error of the co-efficient and the intercept. Would be helpful if this can be shared.
Literally same here
Josh, need bother you again. two questions: 1) 6:15 you map probability [0,1] to logs(odds), but how do you get the probability for each observation at the first place? 2) once you map probability to log(odds) 6:15 and 15:32, so y-axis is log(odds), how do you interpret it? at 15:32, you put "size" as y-axis, is that how interpret log(odds)? isn't it here the possibility to have obese? Thanks a lot in advance!
Let's start by just making sure the definition of the two axes are clear: At 6:15, I'm showing how different probabilities map to the log(odds) axis. So p=0.5 translates to log(odds) = 0, p=731 translates log(odds) = 1. Thus, each point on the probability axis translates to something on the log(odds) axis.
OK, now that we have that part clear, let's talk about the probability that each mouse is obese. At 1:37 in the video I say that the blue dots represent obese mice and the red dots represent mice that are not obese. So the probability that blue dot mouse is obese = 1, since the blue dots represent obese mice. The probability that a red dot mouse is obese = 0, since the red dots represent mice that are not obese. Does that make sense?
As for the log(odds) axis, this represents the change in the log(odds) of obesity for every unit change in weight (or in genotype). So, in the weight vs obesity example, if you have a mouse that is one unit heavier than another mouse, then the log(oods) increases by 1.83. Does that make sense?
@@statquest Great Thanks for confirming the log(odds) axis. But I am still unclear about the first question: so in 1:37 and 6:15, your red dot have probability 0 (to have obese), and blue dots have probability 1. But at 6:15, how do you get fraction probabilities (those p=0.73, p=0.88)? are they the result from your logistic regression curve?
@@Tyokok Oh, I think I see the confusion. p=0.73 and p=0.88 are not from the data or the curve. They are just example probabilities, values between 0 and 1, that I use to show how a point on the y-axis (probability) for the logistic regression curve relates to a point on the log(odds) axis. In other words, I just wanted to show how the formula log(p/(1-p)) = log(odds) worked, so I picked some numbers and plugged them in. Even though I could have picked any number between 0 and 1 for the demonstration, I picked p=0.73 and p=0.88 because I knew they would translate to nice, round numbers on the log(odds) axis. Does that make sense?
@@statquest Yes that's what I learnt from your video. What I don't understand is that when you mapping the observations to log(odds), ( since observations are binary, either p=0 or p=1 probability,) your log(odds) will end up with only position and negative infinity value in log(odds) space. Then it's not a slope line. Or I am missing something?
@@Tyokok One use of the transformation from probability to log(odds) is once you fit a curve to the data, someone can tell you that they have a mouse that weighs "x".... You can then plot it on the graph (along the x-axis) and use the curve to see what the probability is that that mouse is obese. You can the use the log(p/(1-p)) transformation to say what the log(odds) are that that mouse is obese. Does that make sense?
Good man! It's easy to underestimate how much is in logistic regression.
Thanks! :)
Notes for future revision.
08:10
*Squiggly line = Probability view.*
(Probability vs variable values)
Y = 1 / (1+exp(-(βo+β1x1+..)))
*Straight line = Coefficient view.*
(log(odd) vs variable values)
Y = βo + β1x1 + ...
Squiggly line Y=0.5 ⇄ Straight line Y=0
Straight line slope ⇄ Squiggly line ??
----
14:26 Categorical variable
Still,
*Straight line = Coefficient view.*
(log(odd) vs variable values)
But,
because the X is categorical, we use delta between the variables for X.
Linear Regression
Continuous Y vs categorical X
⇄
Continuous Y vs categorical X and X delta
Y = B1 x mean_var1 + B2 x delta_mean_var12
Logistic Regression: Probability Y vs categorical X variables
⇄
Log Odd Y vs categorical X variables
⇄
Log Odd Y vs categorical X and X delta
In log world, delta ⇄ ratio
Y = B1 x log(Odd_Obesity_var1) + B2 x delta_log(Odd_var1)_log(Odd_var2)
= B1 x log(Odd_Obesity_var1) + B2 x log(Odd_var1/Odd_var2 ratio)
17:45
Therefore, in logistics regression results, the coefficient β2 applies to log(odd_var1 / odd_var2) i.e. log(odd ratio).
X1 is log(odd ratio), not var2, count of var2, mean of var2 etc.
:)
Our prof speaks broken English and does not explain this stuff very well. You are a life-saver!
Thanks!
Hi bam, at 4.13 timestamp if the weight of the mouse is 0 we are getting some value right. What does it mean? Can we think like if there is no weight mouse still we can say generalised obesity of the mouse or what?? I'm requesting you to give reply
It's just that the straight line that we are using to make predictions crosses the y-axis at x=0.
Best logistic regression tutorial.
Thanks so much!!! I'm really happy to hear you like this one. I was worried it would be too obscure.
StatQuest with Josh Starmer It is not obscure at all; just perfect.
Thank you! :)
Thanks Professor Josh Harmer (I have a presentation tmrw!!) sending gratitude and good vibes!
Best of luck!
@@statquest I passed! ^^
@@zukofire6424 TRIPLE BAM!!! Congratulations :)
At 9:52 why do we use the probability z1.47. Why not just calculate the p value for z
Because two-sided p-values are the way to go. For details, see: ruclips.net/video/JQc3yx0-Q9E/видео.html
Thanks Josh. I’m hooked. MS Data Analytics & Viz. in view 🙏🏿
bam! :)
Triple Bam!!! Thanks a lot Josh.This really clears up a lot of confusions.
The critical part is the link between a normal variable, the probability and the log odds results. You explained a bit of it and i didnt see any better video, but i'd lie if i said i understsand that step already to a degree that i could explain it to someone myself.
For more details on that step, see: ruclips.net/video/BfKanl1aSG0/видео.html
18:36 Any reason why the scale for the coefficients is log(odds)? I don't quite follow why we have to do everything in log(odds) for logistic regression.
The log(odds) makes the problem solvable. A linear shape can only move in certain ways, and those constraints make it relatively easy to decide if we've found an "ideal" fit. In contrast, the squiggle can move in infinitely many ways, and trying all of them would take forever, making it impossible to decide if we had found an "ideal" fit.
In 16:15 we see it's easy to calculate the coefficients when X axis has discrete values (e.g. normal gene, mutated gene). Can you show how to calculate coefficients when X axis values are continuous (e.g. weight)? The next video says we just rotate the line to get the best fit when the values are continuous, but doesn't show any coefficient calculation like it is done for discrete values in this video. Or it's not possible to do such calculation, that's why we start with random coefficients (a random straight line)?
Logistic Regression is solved using an iterative optimizing procedure like Gradient Descent, which I explain here: ruclips.net/video/sDv4f4s2SB8/видео.html
Hi, @5:00, why did you say it's a problem in Log regression that y-value is confined to [0,1]. And why's it required or important to convert the y-axis from probability to log-odds?
Because the straight line (which is easy to fit to data) does not have limits on the maximum and minimum value. In contrast, the squiggle has limits on the maximum and minimum values and that makes it much harder to fit to data. In this specific case, the squiggly line has too many degrees of freedom.
Triple Bam!! Good video!!!! Greets from. Germany 🇩🇪😍
Danke! :)
This was great until the moment with the Standard Error, which just.. came no idea from where? How do I calculate it?
I talk about that in my video on Odds Ratios and Log(Odds Ratios) at 11 minutes and 24 seconds in. Here's the link: ruclips.net/video/8nm0G-1uJzA/видео.html
@@statquest I must confess I am puzzled too. The Odds & Logs video shows calculating SE from the observations. But what are the observations here? We have one observation of slope. Are we to infer that there are multiple observations of slope when finding the best fit curve and these are what are used? Or am I missing something? Otherwise, great videos!!!
Hi Josh, thanks so much. Your videos are awesome.
At 16:30, as an alternative or in addition to the Wald's Test, could we have calculated a McFadden's Psuedo R-squared value by comparing the model with 2 fitted lines to a simple model that only has an 'overall log(odds of obesity)' line fitted?
I assume we could calculate it for a logistic regression with discrete data (like we can for continuous data as demonstrated in Part 3)?
Sure!
In your video 13:36, I have 2 questions, 1- What does B1 and B2 means in the design matrix? 2nd question which is relevant to question 1 is that why we labeled mutated gene as (1,1) instead of (0,1)?
For more information on design matrices, see: ruclips.net/video/NF5_btOaCig/видео.html and ruclips.net/video/CqLGvwi-5Pc/видео.html
best explain ever. you have ability to make us so interested in stat. this channel is just my favorite. THANK YOU.
i want to know if you have a book in this topic or an article?!
Hooray! I'm so glad to here you like the video!!! :) However, all I have are the videos - no book or articles.
Great explanation, thank you! I have a quick question. If a feature is not statistically significant it means that is not good for the prediction right? So in the example at 10:38 it means that weight is not a good feature for predicting if a mouse is obese or not (isn't that counter-intuitive in some sense) correct me please if I am wrong. And what about the intercept? It means that the intercept found by training the model isn't the best one? Thank you!
In this case, the reason the p-value for weight is not significant is that we don't have enough measurements. To get a better idea on how increasing the sample size will help, see: ruclips.net/video/Rsc5znwR5FA/видео.html
As for the p-value for the intercept. That simply means that the intercept value is not significantly different from 0 (given the small sample size). Generally speaking, the intercept, regardless of whether or not it is significantly different from 0, is of no interest to the researcher.
@@statquest Thank you. Love your videos.
At 19:02 why it is log(2/9)?
When total samples are 11(2 obese and 9 not obese) out of which 2 are obese.
So it can be log (2/11)??
I'm not sure what time point you are asking about since the video isn't that long, however, I'm pretty sure you are getting odds confused with probabilities. They are not the same thing. Here's a video that explains the difference: ruclips.net/video/ARfXDSkQf1Y/видео.html
14:17 May I ask like when to reject and when to accept the hypothesis that, that coefficient is equal to 0 ? Thank you very much!
This is from models based on linear regression. To learn more, see: ruclips.net/video/nk2CQITm_eo/видео.html and ruclips.net/video/NF5_btOaCig/видео.html
@ okay, noted with thanks!
The best explanation ever. Thank you Sir.
bam! :)
One question, At 6:05, why are we trying to put different values to new Y axis from old X axis.. Are these the values from data set.. Sorry for such a basic question..
All we're doing is translating the y-axis on the left, which represents probability and goes from 0 to 1, to the axis on the right, which represents the log(odds) and goes from negative infinity to positive infinity. The reason we're doing this is that, by changing the y-axis, the squiggly line on the left becomes a straight on the right. It's much easier to solve for the optimal straight line than the optimal squiggly line. So, for computational reasons, we transform the y-axis from probability to log(odds). Does that make sense?
@@statquest Thanks for the explanation. I understand for getting 0 of new Y axis,we started with taking center of old Y axis which is 0.5 and put it in log function. But how the other values 0.731,0.88 are taken as an example here ?
@@shantanukathale7210 I picked 0.731 and 0.88 because they would result in "nice" numbers on the log(odds) y-axis on the right side. They are just examples of how the transformation works. In practice, the transformation (back and forth between the two graphs), is done for you, so you don't have to do it yourself.
I think the problem of my course is, it doesn't use the terms such as Wald's test, Fisher's Exact Test, but it only showed in mathematical terms and symbol. At first, I feel like I never learned it before, but when I revised it, I actually has learned this. I don't know why my uni syllabus doesn't use this global statistics terms in the course. They could be useful especially when the learning phase has entered machine learning phase.
noted!
why is the logit-function a straight line at 6:02 and not a curve? on Wikipedia the logit-function is curved
The function on the left, which is curved, is the logistic function. The function on the right is a straight line and corresponds to the standard linear equation y = ax + b.
Great video again! However, I am still a bit confused about the Z values of the coefficients, after watching the other suggested videos linked to this one. How's the Wald Test play a role for the coefficients of the continuous variables explained at around 9:23?
Wald's test gives us a way to test the hypothesis that the coefficient is equal to 0. In other words, Wald's test approximates a normal distribution of random log(odds) centered on 0 and the further (and more standard deviations away) our estimate is away from 0, the smaller the p-value and more significant the difference.
@@statquest Thank you! I think I can get the high-level ideas in behind now. I've always struggled to understand which test corresponds to which distribution. I may need to delve deeper into this.
Such a good, simple, clear video explanation!
Thanks! :)
Hello
At 3:25.... you wrote as SIZE = 0.86+0.7*weight...
What I know is y = mx+c
I am unable to compare these 2 equations..you said Y = 0.86 , m = 0.7 that is OK. (How did you calculate m).
But What about " x " and " c " ...Please help.
What time point are you asking about? If you are asking about something around 14:00 , then the answer is in my linear models videos: ruclips.net/p/PLblh5JKOoLUIzaEkCLIUxQFjPIlapw8nU
at 9:55 it talks about the estimated intercept: I don't get why this is estimated or what the standard normal curve that you go on to show represents (WHAT has a mean of zero?). don't we definitively know the intercept of a fixed line? or is it a "translation" of the standard normal curve for weights (or odds of weights?) into one of coefficients (seems nontrivial that it would remain a standard normal curve given the log aspect)?
One of the main ideas behind statistics is to use a relatively small set of data to get an idea of what things would be like if we had a very large dataset. For example, we might measure the height of 10 people in a town and use the average height for the 10 people as an estimate for the average height for everyone who lives in that town. We then have statistics that give us a sense of how good that estimate is. If the 10 measurements for height were all over the place (some people were very short and some were very tall), then we might not have a lot of confidence in the estimated mean. In contrast, if all 10 measurements were very similar, then we might have more confidence that the estimate does a pretty good job representing the overall mean for the town.
That same mentality is applied to logistic regression. We are using a relatively small dataset to make predictions about a larger population. Thus, the line that we fit to the relatively small dataset is an estimate of the line that would fit to a larger dataset. So the intercept in the line we estimated is, in turn, an estimate of the intercept that would belong to the line fitted to the larger dataset. Because the estimated intercept tells us something (in this case, it tells us that the log(odds) of obesity in the population is -3.48), we want to know how confident we can be in that estimate. So we do a test to see if the estimate is significantly different from 0. If the estimated log(odds) is not significantly different from 0, then we can't put much faith in it.
The most helpful thing would be if you named what sort of distribution we're talking about (the mathematical name), what the statistic in question is, and what the population and sample are. I'm probably just missing something in the video.
*Issue A* If the coeff is the statistic, how can this vary at all, given that sigma and logit are bijections and we're transforming a particular value (or rather, function, but there's an associated uniquely determined coefficient that will result from the mapping of each function) rather than a random variable via logit? Are we talking about a sampling distribution of the coefficient, where we have a single sample of size = number of mice, and we're trying to estimate the "population fitted-line-coefficient"?
* but can you even create a sampling distribution from one sample? And how would the size of the sample factor into that since they're all getting boiled down into a single coefficient? And we don't know the population size, right? (I know all of these things are factored in in an algorithmic way depending on the type of distribution, but I'm not confident that I know what type of distribution we're doing.)
*Issue B)* But why do we expect the mean to be zero? I get that the first graph was showing probabilities so it's bounded by 0 and 1 (and in fact boolean), and that the mean of 0 and 1 is .5, then logit(.5) = 0. But just bc .5 is the mean of 0 and 1, the only two classification values that can be taken, that doesn't make it the mean of the original sample of the mice; in fact in this example it is not.
Thanks for the reply, it must be a huge hassle to untangle all of these comments.
@@SpecialBlanket I'm not sure I fully understand your question, however, I get the feeling that you might want to take a step back and learn about how normal linear regression is done before learning about this, since normal linear regression sheds a lot of light on what we're doing here. Here's the link: ruclips.net/p/PLblh5JKOoLUIzaEkCLIUxQFjPIlapw8nU
@@statquest I did watch that one =/
Let me try to rephrase it. You are considering the coefficient from one sample. How are you calculating its standard error? Why do you expect the distribution of all possible samples' coefficients (that's the distribution with respect to which we're calculating the p-value for this particular coefficent, right?) to be normal? Or is this what is covered by Wald's?
In 9:28 why estimated intercept is being said (instead of z value) away from 0 in normal distribution ?
The red dot at -3.48 refers to the y-axis intercept of the straight line on the right hand side of the screen. Sure, we then plot the red dot on the x-axis of a normal distribution, but it wasn't calculated the way we normally calculate a z-value (in other words, we didn't calculate -3.48 subtracting the mean and then dividing by the standard deviation).
Hi, Can you explain how to calculate intercept and coefficient values using logistic regression.
Umm... This video explains how to calculate the coefficients for logistic regression.
Here's how maximum likelihood is applied to do the estimation:
ruclips.net/video/BfKanl1aSG0/видео.html
Here's how R-squared and p-values are calculated:
ruclips.net/video/xxFYro8QuXA/видео.html
Here's how deviance statistics are calculated:
ruclips.net/video/9T0wlKdew6I/видео.html
...and here's how deviance residuals are calculated
ruclips.net/video/JC56jS2gVUE/видео.html
Any formula are there for calculate the coefficient value and intercept values
@@arundasari773 No. It is done iteratively using an optimization algorithm.
Hey Josh, awesome stuff.Landed on your channel after 3b1b ....Quadruple BAM !!!
Hooray!!! I'm glad you like my stuff! :)
You are absolutely an amazing stat teacher :)
Thank you very much! :)
In your video 7:19, log(0) is infinity and therefore log(1)- log(0) should be log(1)-infinity = negative infinity. How did you get something - negative infinity instead of "something - infinity"?
log(0) is defined as -infinity. Thus, log(1) - log(0) = 0 - (-infinity) = +infinity.
16:50 Up to this point everything made perfect sense to me, but how is it possible to get a Std. Error value for the log odds ratio when it is derived purely from counting in a single sample set? I'd think it is impossible. You know, if you compare it to the t-test example: there each datapoint has its own value on the y-axis which allows for the calculation of the Std. error, but in the log reg you don't have anything similar.
The good news is that I show how to calculate the std. error with log(odds) in this video: ruclips.net/video/ARfXDSkQf1Y/видео.html
10:18 i don't understand. Does the standard deviation have to equal 2? I mean why compare z value to 2?
The z-value can be used to calculate a p-value, which, in this case, will be greater than > 0.05. We know this because 95% of the area under the curve (95% of the probability) is within 2 standard deviations of the mean (and the mean is 0 for a z-distribution).
❤ this channel. Completed my understanding...
Awesome, thank you!
At 7:13 you've mentioned (log(1) - log(0)) as (something - negative infinity) == positive infinity.. but log(1) value is zero right..then it becomes (something(0) - negative infinity) == negative infinity right
0 - (-1*infinity) = 0 + infinity = infinity.
17:45 In terms of interpreting the p-values of the intercept and the slope, does that mean having the Normal Gene does not have a significant effect on a mouse being obese or not, while having the Mutated Gene make a mouse significantly more likely to be obese?
Correct
@@statquest Thanks a lot!!!
When I tried to calculate log(2/9) using my calculator it displays -0.65757 but in the video at 14:52, it shows the result is -1.5. Can anyone helps me out finding the reason behind of it?
What base logarithm are you using? Generally the natural logarithm, the log base ‘e’, is used for statistics and machine learning. Does that help?
@@statquest Thanks a lot for this information. OMG! You're too good at explaining things. Already became one of your biggest fans. Looking forward to having a jam with you ^_^
Rayhanul Islam Hooray! :)
U must b having log base 10 in calC. use natural log i.e. log base 'e'
Good video and good content as usual. However i feel i am still one step away from truly understanding what do the "Coefficients" actually mean, that is to say none of us think in chance in terms of log(odds). I assume, and googled, and i get the impression that using e^x with log odds will give me probability. And in your explanation i guess it means for the continuous variable that by increasing X by 1 the log(odds) increases by said coefficient.
I appreciate your work and good teaching skills, also your singing is, surprisingly, good.
Update: RUclips suggested a video you made on Log odds which has taken me further on understanding this log odds concept. The final stepping stone i needed to understand it was that both odds and probability can be defined using simply succesful outcomes and unsuccessful outcomes.
I'm glad you figured it out! :)
Vary clear! thank you! a little question: what is the use for the intercept when both variables are dichotomic ( = -1.5041; min 17:34 )?
The intercept tells us the log(odds) of the normal gene's association with cancer.