👋🏼 Hello there! In this statistics lecture we learn how to test a hypothesis using the bootstrap approach. we will also explore where and why we would use a #bootstrapping approach for hypothesis testing. 👉🏼Related Video: Hypothesis Testing by Bootstrapping in #R Programming (bit.ly/2K6lFH0); Like to support us? You can Donate (bit.ly/2CWxnP2), Share our Videos, Leave us a Comment and Give us a Like 👍🏼! Either way We Thank You! 🦄
thanks! we're adding new videos just about every week, so keep your eyes open for them, or click the BELL to receive notifications every time we upload new content
Great course. I am learning a lot. One: when do the bootstrap samples, shouldn't you keep value and diet connected? For example, 390 was observed in the meat group. However, it occurs as last value of bootstrap sample #1 , but assigned to the casein group. Thanks.
If you’re building a confidence interval, then yes, as you want to preserve that relationship. But for a hypothesis test, where you assume a null hypothesis is true...that the weight and diet are independent, then you resample under that assumption. Hope that came out clear
@@marinstatlectures I'm still not sure about this. If the null hypothesis was that there is no relationship between diet and weight gain, then mixing the feed types wouldn't be an issue. But in this case we are trying to test whether there is a difference between weight gain on one diet vs another. In order to test this, shouldn't the weight gain be kept separated in order to compare the difference?
@@username-videos The null hypo is: "weight gain same under M or C", so we can think M == C, then actually there is only one group. Therefore, we can assign any weight value to any feed value. e.g. we can assign 257 to M or we can also assign 257 to C. Moreover, if 257 appear twice in one sample, they can be assigned to different groups. In other words, the weight value is not relevant to the feed type, so they can be in any feed group.
@@username-videos In this case of bootstrapping, we're creating the bootstrap pseudo samples with the null hypothesis in mind, i.e. assuming that there is no difference in the diets. We get a t-value for this case and then compare it to the t-value of our sample - which has separate values for the diets. It is in essence, if there was no difference in the diets, then we can just mix everything up in one pile as it 's all the same and then we compare this scenario with the reality of our observed sample and look if it looks similar or statistically different. I realise it is an old question, replying for future readers too.
Excellent tutorial sir. I have two questions: (1) Can the "test statistic" also be a financial ratio such as the Sharpe Ratio? If so, would we treat it exactly the same way as you have treated the test statistics of Mean and Median for finally arriving at the P-values? i.e. compare the difference between the Sharpe Ratios of C & M for each bootstrap? (2) Is there an academic paper that you can point us to? There's one by Boos (2003) that does mention the same approach, but it does not sufficiently clarify what a "test statistic" can potentially be, which also leads me back to the first question.
"The first sarcastic answer I'll give you there is 'yes they do [differ]', one of them starts with M, one of them starts with C." That humor just made my day 😂 Thank you for another excellent explanation, your videos are fantastic!
I do not find it theoretically convincing, if we take bootstrapped sample for one category, i.e. N , from another diet category, i.e. C in the main sample. Could you please look at it @ 10:38 minutes. You took 390 in the first bootstrap sample and wrote in the category N, which actually belongs to the diet category C. Isn't it theoretically invalid? Thanks in advance
Nice, but shouldn't we generate boostrap distribution of t statistic from shifted means, to get the sampling distribution under the null hypothesis? Or is just enough to assume the null by concatenating the two samples like in your example?
Hi, in this approach we do get the sampling distribution under the null. by putting the 2 samples "together" we are assuming that the two samples were drawn from the "same population" or that the "grouping variable" is irrelevant. similar to how in a 2-sample t-test you assume that Ho: there is no difference in means", here by assuming that the "grouping variable" is irrelevant, it makes the same sort of assumption
Thanks for posting this video! What's the advantage of using the absolute difference over the simple difference in test statistics? If we were interested in using the bootstrap equivalent of a 1 sided t test, would the simple difference be more appropriate?
Yes, if we are interested in using the bootstrap equivalent of a 1 sided t-test, then no abs sign will be more appropriate! But the research question that he is interested in is whether the two diets are different, not which one produces more weight gain.
LOVE all your lectures Mike! I've sent your link to everyone in my lab. Question: can you perform a bootstrap with two independent variables? For instance, if you had the same example you worked with here, but you also wanted to determine the effect of sex on weight of chicks. Thanks!
yes you can bootstrapping for virtually any statistical test - bootstrapping is just a method of empirical analysis for tests that would otherwise be based in theory
Why we looking bootstrap value grater than p-value(observing bootstrap area is more extreme when the p value is big it is good) and when this value is smaller than 0.05, what is the mean of there is significant difference( significant difference between what?) =) Thank you so much 😊
Hi! I know this may have been an old post but I appreciate it. I would like to ask... you mentioned small sample... Say for example: computed sample using Cochran should be 385 but after data collection you came up short and only got 170. Would bootstrap hypothesis testing be ok than just using the traditional way knowing that you are short of the required sample size?
Hello! Thanks for the lecture! I have a question. When you derive the p value, you seem to adopt the one-sided hypothesis test (by assuming that the difference must be positive based on prior knowledge). Is that correct? Why don't you adopt two-sided hypothesis test? Thanks!
He DID NOT assume the difference must be positive based on prior knowledge. Notice the absolute sign in the test statistics - If we assume the difference must be positive, then he wouldn't put abs signs around the difference.
Yeah, right now we have one class on bootstrapping, but I plan on expanding it to a few more. For each major topic covered, I will have a few where we cover a bootstrap alternative approach, and others where I make an activity for bootstrap available to students as an optional exercise, if they want to explore this more. Then next addition after this is to include a class or two on the Bayesian approach. I think it’s good for students to see that there are more than just the “classical” approaches to these sorts of problems.
anything you can think of...that is the beauty of the bootstrap. maybe you want to compare the 90th percentile for the two groups...or the range of the 10th-25th percentile for each group, or what ever was of interest in the context of your study. the bootstrap frees things up to first think of an estimate that is meaningful, and then that can be worked with using the bootstrap
no, because in a bootstrap hypothesis test, we are assuming that the observations could have come from either of the groups (that the labels are meaningless)....so we take all of the observations, and then resample with replacement to build a bootstrap sample for each of the groups. in other words, we assume that the 380 (and all other observations) could belong to the casein or the meat meat group. (under the null hypothesis)
you just pick that value...it is the number of resamples you will take. that can be as large as you want, and the only issue will be computing time/power. id suggest to use B=10,000 at minimum. it's worth mentioning that using a larger B will NOT get you more info, it will just make the bootstrap-estimate more stable
It’s a course I teach at The University of British Columbia. It’s a graduate course, although it covers mostly intro stats material, but at a slightly higher level. The course name is Statistics For Health Research
👋🏼 Hello there! In this statistics lecture we learn how to test a hypothesis using the bootstrap approach. we will also explore where and why we would use a #bootstrapping approach for hypothesis testing. 👉🏼Related Video: Hypothesis Testing by Bootstrapping in #R Programming (bit.ly/2K6lFH0); Like to support us? You can Donate (bit.ly/2CWxnP2), Share our Videos, Leave us a Comment and Give us a Like 👍🏼! Either way We Thank You! 🦄
Finally a complete example. Thumbs up
Very nice explained! thanks!!
I am loving these videos...keep 'em coming.
thanks! we're adding new videos just about every week, so keep your eyes open for them, or click the BELL to receive notifications every time we upload new content
Great course. I am learning a lot. One: when do the bootstrap samples, shouldn't you keep value and diet connected? For example, 390 was observed in the meat group. However, it occurs as last value of bootstrap sample #1 , but assigned to the casein group. Thanks.
If you’re building a confidence interval, then yes, as you want to preserve that relationship. But for a hypothesis test, where you assume a null hypothesis is true...that the weight and diet are independent, then you resample under that assumption. Hope that came out clear
Thanks. Got your point. I really appreciate your videos.
@@marinstatlectures I'm still not sure about this. If the null hypothesis was that there is no relationship between diet and weight gain, then mixing the feed types wouldn't be an issue. But in this case we are trying to test whether there is a difference between weight gain on one diet vs another. In order to test this, shouldn't the weight gain be kept separated in order to compare the difference?
@@username-videos The null hypo is: "weight gain same under M or C", so we can think M == C, then actually there is only one group. Therefore, we can assign any weight value to any feed value. e.g. we can assign 257 to M or we can also assign 257 to C. Moreover, if 257 appear twice in one sample, they can be assigned to different groups. In other words, the weight value is not relevant to the feed type, so they can be in any feed group.
@@username-videos In this case of bootstrapping, we're creating the bootstrap pseudo samples with the null hypothesis in mind, i.e. assuming that there is no difference in the diets. We get a t-value for this case and then compare it to the t-value of our sample - which has separate values for the diets. It is in essence, if there was no difference in the diets, then we can just mix everything up in one pile as it 's all the same and then we compare this scenario with the reality of our observed sample and look if it looks similar or statistically different.
I realise it is an old question, replying for future readers too.
Excellent tutorial sir. I have two questions:
(1) Can the "test statistic" also be a financial ratio such as the Sharpe Ratio? If so, would we treat it exactly the same way as you have treated the test statistics of Mean and Median for finally arriving at the P-values? i.e. compare the difference between the Sharpe Ratios of C & M for each bootstrap?
(2) Is there an academic paper that you can point us to? There's one by Boos (2003) that does mention the same approach, but it does not sufficiently clarify what a "test statistic" can potentially be, which also leads me back to the first question.
"The first sarcastic answer I'll give you there is 'yes they do [differ]', one of them starts with M, one of them starts with C."
That humor just made my day 😂
Thank you for another excellent explanation, your videos are fantastic!
Lol, you’re welcome ;) glad you’re enjoying them!
how would you use bootstrap to compare two samples with more than one variable? BTW, Excellent video! And I love your explanation with R too
Thanks for very nice explanation. Is it possible to calculate the power with bootstrap hypothesis testing?
I do not find it theoretically convincing, if we take bootstrapped sample for one category, i.e. N , from another diet category, i.e. C in the main sample. Could you please look at it @ 10:38 minutes. You took 390 in the first bootstrap sample and wrote in the category N, which actually belongs to the diet category C. Isn't it theoretically invalid?
Thanks in advance
that's exactly what was questioning while watching this video.. all calculation processes were good to go though
Nice, but shouldn't we generate boostrap distribution of t statistic from shifted means, to get the sampling distribution under the null hypothesis? Or is just enough to assume the null by concatenating the two samples like in your example?
Hi, in this approach we do get the sampling distribution under the null. by putting the 2 samples "together" we are assuming that the two samples were drawn from the "same population" or that the "grouping variable" is irrelevant. similar to how in a 2-sample t-test you assume that Ho: there is no difference in means", here by assuming that the "grouping variable" is irrelevant, it makes the same sort of assumption
@@marinstatlectures thanks!
Thanks for posting this video! What's the advantage of using the absolute difference over the simple difference in test statistics? If we were interested in using the bootstrap equivalent of a 1 sided t test, would the simple difference be more appropriate?
Yes, if we are interested in using the bootstrap equivalent of a 1 sided t-test, then no abs sign will be more appropriate! But the research question that he is interested in is whether the two diets are different, not which one produces more weight gain.
Amazing !
LOVE all your lectures Mike! I've sent your link to everyone in my lab. Question: can you perform a bootstrap with two independent variables? For instance, if you had the same example you worked with here, but you also wanted to determine the effect of sex on weight of chicks. Thanks!
yes you can bootstrapping for virtually any statistical test - bootstrapping is just a method of empirical analysis for tests that would otherwise be based in theory
best!
Why we looking bootstrap value grater than p-value(observing bootstrap area is more extreme when the p value is big it is good) and when this value is smaller than 0.05, what is the mean of there is significant difference( significant difference between what?) =) Thank you so much 😊
Hi! I know this may have been an old post but I appreciate it. I would like to ask... you mentioned small sample...
Say for example: computed sample using Cochran should be 385 but after data collection you came up short and only got 170. Would bootstrap hypothesis testing be ok than just using the traditional way knowing that you are short of the required sample size?
Hello! Thanks for the lecture! I have a question. When you derive the p value, you seem to adopt the one-sided hypothesis test (by assuming that the difference must be positive based on prior knowledge). Is that correct? Why don't you adopt two-sided hypothesis test? Thanks!
He DID NOT assume the difference must be positive based on prior knowledge. Notice the absolute sign in the test statistics - If we assume the difference must be positive, then he wouldn't put abs signs around the difference.
Interesting... Is it covered in SPPH400 now?
Yeah, right now we have one class on bootstrapping, but I plan on expanding it to a few more. For each major topic covered, I will have a few where we cover a bootstrap alternative approach, and others where I make an activity for bootstrap available to students as an optional exercise, if they want to explore this more.
Then next addition after this is to include a class or two on the Bayesian approach. I think it’s good for students to see that there are more than just the “classical” approaches to these sorts of problems.
What other different test we can use instead of mean and median?
anything you can think of...that is the beauty of the bootstrap. maybe you want to compare the 90th percentile for the two groups...or the range of the 10th-25th percentile for each group, or what ever was of interest in the context of your study.
the bootstrap frees things up to first think of an estimate that is meaningful, and then that can be worked with using the bootstrap
Did you make a mistake by using 380 in BOOT-SAMP2?
no, because in a bootstrap hypothesis test, we are assuming that the observations could have come from either of the groups (that the labels are meaningless)....so we take all of the observations, and then resample with replacement to build a bootstrap sample for each of the groups. in other words, we assume that the 380 (and all other observations) could belong to the casein or the meat meat group. (under the null hypothesis)
How do we calculate "B"?
you just pick that value...it is the number of resamples you will take. that can be as large as you want, and the only issue will be computing time/power. id suggest to use B=10,000 at minimum. it's worth mentioning that using a larger B will NOT get you more info, it will just make the bootstrap-estimate more stable
what is SPPH400?
It’s a course I teach at The University of British Columbia. It’s a graduate course, although it covers mostly intro stats material, but at a slightly higher level. The course name is Statistics For Health Research