Hi Colin, at the end of your video, you asked for a suggestion for a video. I'd be interested in seeing brms used to fit an ordinal mixed-effects logistic model. In particular, I'd like to see how you check the proportional odds assumption. I think I can probably deal with this aspect of fitting this type of model myself: there are postings in the Stan forum about how to use cs() to check this assumption (which I haven't read carefully yet). However, I'd be most interested in a tutorial on this. Although I'm a committed Bayesian, for my own interest, I tried to use mixor() to check this assumption within a frequentist framework, and I just couldn't get mixor() to do it.
while I was watching this video, I find it funny that I find your voice familiar and turns out you're the guy who thought me about Regex a year ago lol Thanks for the tutorial! Such an underrated channel.
Hi Colin, thanks for this awesome work. I've got two questions: 1. Doesn't bayesian regressions have some sort of parameter for significance of an effect like the p value or a bayes factor? and if not, how can I tell, wheter an effect is likely to exist? Shell I watch out for the cis whether they include a 0, so that the effect is significant, when the 0 isn't included like in bootstrap? 2. Is there any way you can do model comparisons in bayesian regressions with brms? If I would build a non-bayesian logistic regression in r, I`ld probably use glmmTMB, build like one or two alternative models and compare them via the common anova() function. Is something like that possible here? best wishes man, keep up that good work!
I would check out this paper: link.springer.com/article/10.3758/s13423-016-1221-4 The framework laid out is basically that in both frequentist and bayesian statistics you can choose a testing approach (p-values, bayes factors) or an estimation approach (confidence intervals, credible intervals). The goal in an estimation approach is not to state whether an effect exists or not, but to simply estimate a reasonable range for the size of an effect; you should try to avoid the immediate comparision to 0 as the only goal with a credible interval. If comparision to 0 is actually what you want, a bayes factor is probably better; that said, credible intervals do have the same property where you can make conclusions based on whether 0 is included in the range. For many experimenters, this is sacrilege as often the goal is just to determine effect vs no effect, but in fact when you start to open yourself up to the possibility of estimation as an approach you will find there are places where estimation is preferred. But it of course does depend on your research, I do a lot of work now in my insustry job that involves p-values because frankly we just want a quick answer, we aren't doing a deep examination like I would for a paper. As for your second question, there are a ton of ways to compare models in bayesian statistics generally, brms specifically has "loo_compare()" which I would look into. It basically just spits out the precomputed information criterion values (I used waic) which are based on leave one out performance. I don't know if I mention this in the video as it's been a while, but I highly recommend McElreath's Rethinking Statistics if you want to learn more about Bayesian stats. He has lectures up on youtube.
Does anyone know any resources about why and to what to change the default prior information? In every tutorial or description it is sold as logical (here the default is a student t distribution with certain parameters, but we want it more narrow, so we change one of them). But why do we want that and how do I know what to change mine to?
I highly recommend checking out the Stan guide to priors: github.com/stan-dev/stan/wiki/Prior-Choice-Recommendations McElreath's Rethinking Statistics is also great for a deep dive
I do have a blog but on different topics: colinquirk.com/ The code is available here (github.com/colinquirk/LivestreamCode ), but sadly without commentary. Maybe one day! I hope to start up again in a few months when I have a bit more time.
Hi Colin, at the end of your video, you asked for a suggestion for a video. I'd be interested in seeing brms used to fit an ordinal mixed-effects logistic model. In particular, I'd like to see how you check the proportional odds assumption. I think I can probably deal with this aspect of fitting this type of model myself: there are postings in the Stan forum about how to use cs() to check this assumption (which I haven't read carefully yet). However, I'd be most interested in a tutorial on this. Although I'm a committed Bayesian, for my own interest, I tried to use mixor() to check this assumption within a frequentist framework, and I just couldn't get mixor() to do it.
while I was watching this video, I find it funny that I find your voice familiar and turns out you're the guy who thought me about Regex a year ago lol Thanks for the tutorial! Such an underrated channel.
Glad it's helpful! I hope to get back to making content some time in the future when I have more time.
Thanks Colin.
I have a question. How to get the predicted values of test data? Or how do check the accuracy of the model?
Hi Colin, thanks for this awesome work. I've got two questions: 1. Doesn't bayesian regressions have some sort of parameter for significance of an effect like the p value or a bayes factor? and if not, how can I tell, wheter an effect is likely to exist? Shell I watch out for the cis whether they include a 0, so that the effect is significant, when the 0 isn't included like in bootstrap? 2. Is there any way you can do model comparisons in bayesian regressions with brms? If I would build a non-bayesian logistic regression in r, I`ld probably use glmmTMB, build like one or two alternative models and compare them via the common anova() function. Is something like that possible here? best wishes man, keep up that good work!
I would check out this paper: link.springer.com/article/10.3758/s13423-016-1221-4
The framework laid out is basically that in both frequentist and bayesian statistics you can choose a testing approach (p-values, bayes factors) or an estimation approach (confidence intervals, credible intervals). The goal in an estimation approach is not to state whether an effect exists or not, but to simply estimate a reasonable range for the size of an effect; you should try to avoid the immediate comparision to 0 as the only goal with a credible interval. If comparision to 0 is actually what you want, a bayes factor is probably better; that said, credible intervals do have the same property where you can make conclusions based on whether 0 is included in the range. For many experimenters, this is sacrilege as often the goal is just to determine effect vs no effect, but in fact when you start to open yourself up to the possibility of estimation as an approach you will find there are places where estimation is preferred. But it of course does depend on your research, I do a lot of work now in my insustry job that involves p-values because frankly we just want a quick answer, we aren't doing a deep examination like I would for a paper.
As for your second question, there are a ton of ways to compare models in bayesian statistics generally, brms specifically has "loo_compare()" which I would look into. It basically just spits out the precomputed information criterion values (I used waic) which are based on leave one out performance.
I don't know if I mention this in the video as it's been a while, but I highly recommend McElreath's Rethinking Statistics if you want to learn more about Bayesian stats. He has lectures up on youtube.
@@colinquirkDS Well thank you very much! That was a fast answer to my questions. I will definitly check out both the sources you mentioned!
How do you specify hyper parameters using brms?
Colin thanks for the video. Please use a script next time.
Does anyone know any resources about why and to what to change the default prior information? In every tutorial or description it is sold as logical (here the default is a student t distribution with certain parameters, but we want it more narrow, so we change one of them). But why do we want that and how do I know what to change mine to?
I highly recommend checking out the Stan guide to priors: github.com/stan-dev/stan/wiki/Prior-Choice-Recommendations
McElreath's Rethinking Statistics is also great for a deep dive
Colin do you have a blog in which everything written and explained ?
I do have a blog but on different topics: colinquirk.com/
The code is available here (github.com/colinquirk/LivestreamCode ), but sadly without commentary. Maybe one day! I hope to start up again in a few months when I have a bit more time.
@@colinquirkDS Colin thanks a lot!
Where can I get the chimp data?
Almost certainly from here, though I may have edited it a bit for the video
rdrr.io/github/rmcelreath/rethinking/man/chimpanzees.html
@@colinquirkDS Thanks
@@colinquirkDS I have tried to access this data but it has been in vein. Would you send a direct link please
@@collinskimtaimchonje5716 This should be it: github.com/rmcelreath/rethinking/blob/master/data/chimpanzees.csv
@@colinquirkDS Thank you.