Great video! I actually have one question. Is the variance of the intercept the same for all the journals? I was expecting to see a random effect of the intercept for each journal. I'd appreciate if you can clarify this. Thanks in advance
Thank you so much for your videos, as mentioned before, I also had a hard time logistic multilevel models in R. This video is very helpful! I would like to ask you a brief question. I am working with a panel data observing 55 provinces at the monthly level for 11 years. My variable for time is called "date" and includes the year and month. Therefore my question would be if the following code would be correct for my case: modelo
It is extremely useful! Thank you! I have a question regarding running glmer(mention.outlier~1+(1|Journal), data = master, family = binomial, control = glmerControl(optimizer = "bobyqa"),nAGQ = 0). The result gives random effect on the Journal (Intercept), 0.472, and this can be regarded as the between Journal variance. I am curious that how can I get the within journal variance? As without the within Journal variance, I cannot calculate ICC (Intraclass Correlation Coefficient). Thank you!
What if your ratio is really low? I have like 7 million data points about units that were picked in a warehouse and added a 0 or 1 based on whether an error was made in that pick. However, only 3000 of the 7 million contained an error..=
Great question - I honestly haven't figured that one out yet. I have some videos for multinomial log, but haven't tried a multinomial MLM yet. I'll do some investigating!
First of all - Love these videos! Also that you use a lot of food examples :) Thanks! I have a question regarding an experiment where we want to test the effect of different learning strategies across time. We used four different conditions manipulated between subjects. Each participant answered the same set of binary questions across 3-time points. If I'm thinking correctly participants would be nested within each condition since one participant only can belong to one condition(?). When I graphed the data (mean for each condition) I can see that all conditions are equal at time0 (pretest) then move(learn) very differently across time1 and time2. Since my hypothesis is that intercepts and slopes would be different across time depending on first condition, second question and third ID I set up my model like this: model
certainly, if you wanted a random slope of time, with a nested intercept of condition by question by participant. I'd actually test if all that nesting is necessary. Otherwise it seems like a good approach to me.
@@StatisticsofDOOM Thanks! I slowed down a bit and compared models with simpler models, first only random intercepts (e.g., glmer(Outcome ~ Condition * Time + (1 | ID) + (1 | Question), ended up with a glmer(Outcome ~ Condition * Time + (Time | Question/ID). One thing I haven't fully understood is when to use "1+" e.g., (1+Time|ID)?
What is your DV is not equally distributed under the null hypothesis? I have a DV where the response chance level is .16. Is there a way to specify the null distribution instead of assuming a 50/50 DV ratio? Thank you!
Great question - I'm not sure that's possible without using Bayes. The null model would be that IV does not predict categories better than chance, which is a coin flip in a dichotomous outcome. The problem with highly unlikely outcomes is that just guessing the larger category gets you better than chance. I didn't cover it in this video, but you could investigate how the outcome is categorized to see if they are actually being predicted well or not (like a frequency table of hits and misses, sometimes called a confusion matrix). You could also sample the larger category to provide a similar sample size and run that comparison - if you take this route, I would definitely run multiple samples and average the effects together, just to ensure any "weird" samples are not the only thing presented.
@@StatisticsofDOOM Ah, that makes sense. Here is my situation: I have an array of objects (6 items): 5 items are deemed the distractors and 1 item is the target. I want to know whether participants are more likely to execute their first eye movement towards the target in two different conditions (preferred vs. non-preferred targets). So, the null would predict that participants' first eye movements have about .16 chance to be directed to any one of the 6 items. Would I be able to use a multinomial MLM where the DV is the location of each participant's first eye movement for each trial....eg.: Part location trial condition Par 1 distractor 1 1 A Par 1 distractor 2 2 A Par 1 target 3 B and so on.... Would a multinomial regression with 6 levels force chance to be .16? Phew, thank you!
@@michaeladebolt3116 Right exactly. You could do a multinomial in that case, which would show you preference for any item in particular but might be a wild set of categorical predictors (like distractor 1 versus 2 for condition, 1 v 3 for condition, etc.) - it might be easier to collapse to distractor versus not for the outcome. I honestly do not know how to do MLM multinomial log ... maybe glmer does this, and I'm just not aware but I've always found multinomial to be a small nightmare to do.
@@StatisticsofDOOM Argh. it's never easy! :P This is now something my labmates and I are debating about. Would collapsing across distractor/target technically be incorrect since the chance of looking to any one item is .16? My vote is that the *behavior* (the eye movement) can do one of two things: look at the target or not.... thus, chance is 50/50 and a binomial would be best suited. What's your vote? :)
@@michaeladebolt3116 If you don't have any reason to believe that one distractor is better (more distracting?) than the others, then I would vote to collapse. That's much easier to analyze and understand as well.
Thak you so much for all the videos you do with R. I have a question about having multiple binary outcomes as IV. How would that work? In my survey, participants had to pick or not credits from a pool of 21 credits. I did a logit for each credit but I am looking for a more elegant way to include the binary output for all 21 credits in on unique model. Is it possible to do this?
@@StatisticsofDOOM I asked my participants to pick 7 out of 21 credits (to extend an airport), where each credit was either related to sustainability, ressources or community/social. I have data on demographics (age, sex, political views and education). I model it as a binary logit model where each credit was an IV (same way you showed in your other video onbinary logit models). Doing this, I could check the effect of each predictor on the selection of credit. I am wondering if I could include everything in one model?
@@juliemilovanovic2679 the only thing I could suggest is maybe creating a sum total for the categories the credits fit into, rather than each credit individually.
Thank you so much for making this video! I have had a very hard time finding information of logistic multilevel models in r.
Thank you for the kind words!
Hi! Thanks for this. What's your take on calculating for ICCs to estimate if the random effects are meaningful to include in the model?
Definitely - also check out the MuMIn package - I've been using it after this video was made, it's really great!
Great video!
I actually have one question. Is the variance of the intercept the same for all the journals? I was expecting to see a random effect of the intercept for each journal. I'd appreciate if you can clarify this. Thanks in advance
It would be - it's calculating the variance of all the intercepts, rather than the variance on each intercept.
@@StatisticsofDOOM thank you. It makes sense now
Thank you so much for your videos, as mentioned before, I also had a hard time logistic multilevel models in R. This video is very helpful! I would like to ask you a brief question. I am working with a panel data observing 55 provinces at the monthly level for 11 years. My variable for time is called "date" and includes the year and month. Therefore my question would be if the following code would be correct for my case: modelo
Certainly could work if you just want to control for time!
It is extremely useful! Thank you! I have a question regarding running glmer(mention.outlier~1+(1|Journal), data = master, family = binomial, control = glmerControl(optimizer = "bobyqa"),nAGQ = 0). The result gives random effect on the Journal (Intercept), 0.472, and this can be regarded as the between Journal variance. I am curious that how can I get the within journal variance? As without the within Journal variance, I cannot calculate ICC (Intraclass Correlation Coefficient). Thank you!
For between versus within, you could try centering and using level 1 and level 2 journal predictors on the outcome (and not a random intercept).
What if your ratio is really low? I have like 7 million data points about units that were picked in a warehouse and added a 0 or 1 based on whether an error was made in that pick. However, only 3000 of the 7 million contained an error..=
That would be nearly impossible to predict. Try running small subsets versus your smaller group multiple times and average the results together.
Thank you so much for this excellent video! I would like to ask how to add multiple confounder to m3?
You could just do + variable name in the formula for the model!
Hello professor, how can we calculate the odds ratio for the factors? Thank you.
What do you mean? It includes them as the coefficient?
How about the case if I need to do MLM when DV is multinominal? Are there any examples/videos that I can study about it?
Great question - I honestly haven't figured that one out yet. I have some videos for multinomial log, but haven't tried a multinomial MLM yet. I'll do some investigating!
Are you using NLME for binary logistic MLM? THanks
You can! I think here I’m using glmer here because I find it easier which is in lme4.
First of all - Love these videos! Also that you use a lot of food examples :) Thanks!
I have a question regarding an experiment where we want to test the effect of different learning strategies across time. We used four different conditions manipulated between subjects. Each participant answered the same set of binary questions across 3-time points. If I'm thinking correctly participants would be nested within each condition since one participant only can belong to one condition(?).
When I graphed the data (mean for each condition) I can see that all conditions are equal at time0 (pretest) then move(learn) very differently across time1 and time2. Since my hypothesis is that intercepts and slopes would be different across time depending on first condition, second question and third ID I set up my model like this:
model
certainly, if you wanted a random slope of time, with a nested intercept of condition by question by participant. I'd actually test if all that nesting is necessary. Otherwise it seems like a good approach to me.
@@StatisticsofDOOM Thanks! I slowed down a bit and compared models with simpler models, first only random intercepts (e.g., glmer(Outcome ~ Condition * Time + (1 | ID) + (1 | Question), ended up with a glmer(Outcome ~ Condition * Time + (Time | Question/ID). One thing I haven't fully understood is when to use "1+" e.g., (1+Time|ID)?
@@LeSchmac Usually that's used to make sure you include the intercept in the model.
How do we add interaction terms when we are also doing random effects? The equation gets mind boggling ~_~
Using the : operator in the equation (all R models work this way), DV ~ X:X will give you DV is approximated by X1 + X2 + X1*X2
What is your DV is not equally distributed under the null hypothesis? I have a DV where the response chance level is .16. Is there a way to specify the null distribution instead of assuming a 50/50 DV ratio? Thank you!
Great question - I'm not sure that's possible without using Bayes. The null model would be that IV does not predict categories better than chance, which is a coin flip in a dichotomous outcome. The problem with highly unlikely outcomes is that just guessing the larger category gets you better than chance. I didn't cover it in this video, but you could investigate how the outcome is categorized to see if they are actually being predicted well or not (like a frequency table of hits and misses, sometimes called a confusion matrix). You could also sample the larger category to provide a similar sample size and run that comparison - if you take this route, I would definitely run multiple samples and average the effects together, just to ensure any "weird" samples are not the only thing presented.
@@StatisticsofDOOM Ah, that makes sense.
Here is my situation: I have an array of objects (6 items): 5 items are deemed the distractors and 1 item is the target. I want to know whether participants are more likely to execute their first eye movement towards the target in two different conditions (preferred vs. non-preferred targets). So, the null would predict that participants' first eye movements have about .16 chance to be directed to any one of the 6 items.
Would I be able to use a multinomial MLM where the DV is the location of each participant's first eye movement for each trial....eg.:
Part location trial condition
Par 1 distractor 1 1 A
Par 1 distractor 2 2 A
Par 1 target 3 B
and so on....
Would a multinomial regression with 6 levels force chance to be .16?
Phew, thank you!
@@michaeladebolt3116 Right exactly. You could do a multinomial in that case, which would show you preference for any item in particular but might be a wild set of categorical predictors (like distractor 1 versus 2 for condition, 1 v 3 for condition, etc.) - it might be easier to collapse to distractor versus not for the outcome. I honestly do not know how to do MLM multinomial log ... maybe glmer does this, and I'm just not aware but I've always found multinomial to be a small nightmare to do.
@@StatisticsofDOOM Argh. it's never easy! :P This is now something my labmates and I are debating about. Would collapsing across distractor/target technically be incorrect since the chance of looking to any one item is .16? My vote is that the *behavior* (the eye movement) can do one of two things: look at the target or not.... thus, chance is 50/50 and a binomial would be best suited. What's your vote? :)
@@michaeladebolt3116 If you don't have any reason to believe that one distractor is better (more distracting?) than the others, then I would vote to collapse. That's much easier to analyze and understand as well.
Muito bom. Faz um de gráfico de tukey
can you share data set
Yes the data is here: github.com/doomlab/Outliers/blob/master/paper/outliers%20complete.csv as indicated in the video.
Thak you so much for all the videos you do with R. I have a question about having multiple binary outcomes as IV. How would that work? In my survey, participants had to pick or not credits from a pool of 21 credits. I did a logit for each credit but I am looking for a more elegant way to include the binary output for all 21 credits in on unique model. Is it possible to do this?
I mean you could include it as one variable, but you will get 20 IVs for that inclusion. I think this is what you are asking?
@@StatisticsofDOOM I asked my participants to pick 7 out of 21 credits (to extend an airport), where each credit was either related to sustainability, ressources or community/social. I have data on demographics (age, sex, political views and education). I model it as a binary logit model where each credit was an IV (same way you showed in your other video onbinary logit models). Doing this, I could check the effect of each predictor on the selection of credit. I am wondering if I could include everything in one model?
@@juliemilovanovic2679 the only thing I could suggest is maybe creating a sum total for the categories the credits fit into, rather than each credit individually.