Thanks a lot for such a detailed explanation. One question: I am running MLM for repeated measures where measures are taken for four different scenarios for all participants and I want to control the effect of scenarios (inter-dependence). As per my understanding, I should be using 1|scenario instead of 1|partno for random intercept model. Is it correct?
You could also do both list(1|scenario, 1|partno) or nest them together (1|partno/scenario) - this last one would be participants in scenarios (since they saw all of them).
@@StatisticsofDOOM Thank you so much. A followup question regarding the random part: if I remove scenario from the random part just keep the participant as random then the model will not control for the order of the scenario (as the interpretation of the results will be easier if we keep them fix than randomize)?
Hi Erin Thanks for your great videos. Hope to see something about Mixed-effects modeling one day. The R package was developed by Bates, 2005; Bates & Sarkar, 2007. Although I ran it but totally confused about what to report? Interesting application of the package was also done by Cunnings, I. and Finlayson, I. (2015) Mixed effects modelling and longitudinal data analysis. Again thanks.
Hi Erin, Quick question. Say we had a repeated measured dependent variable like arthritis pain score. And we wanted to see if a treatment altered the progression of the score, adjusting for initial pain and other variables like sex, BMI, economic status etc. The repeated measure would be nested in the individual variable. So level one would contain the repeated measures. Level two would contain patient ID. Level three would be treatment. Where would we put the controlling variables like Sex. My intuition is that these would go in at level two as these vary on the individual level because they weren't repeated measures. Is this correct? Also, would you have a random intercept and random slope for individuals? By the way great channel!
Yes that happens when your participants/lines are not the same across both models. So if you have X1 in model 1 and X1 + X2 in model 2, ... if some participants are missing X2, they were in model 1 but not model 2 - try excluding NAs with something like complete.cases first.
Dear, I would like if it was possible that the Lord would help me in a review, lest it incur in error. If it were a simple logistic regression, I could use The Area Under an ROC Curve, but since I'm using a generalized linear mixed model with reply Binaria, can analyze the same way this output? If not, what would you suggest? Thank you
I'd suggest taking a quick watch of one of the moderation videos I have ... for example, this one might be good ruclips.net/video/BR050BbER6s/видео.html
Great video! Very interesting! I did have one question though. After the model has been generated, is there a way to see if the values from ranef() are significantly different than each other? Thank you so much!
Amazing videos Erin! I am so glad to have found them, as I have been struggling for a while with my multilevel models :) I tried applying your screening technics for the mlm, but I'm not sure they can work for my data. I am working with public opinion data from the European Election Studies and there are quite a few missing cases in most of the variables of interest. For example, a variable like positioning a party on a left-right scale is not answered by 1/3 of the respondents. As the respondents who do not answer are changing for each party, mahalanobis ends up ignoring almost all of the dataset. My question is whether I actually need to run the screening as most of it does not give me meaningful results and since R can handle missing data in multilevel models. Thank you for all the effort! It is much appreciated!
Yes and no. If you are going to use the variable that is mostly missing, the lines with that missing data will be dropped (even if the other variables there). MLM can interpolate missing data (like say, if you have time 1 and 3, you'll still get an estimate, rather than losing the whole person), but it is still regression and any person/line without complete data will be excluded.
Hi Erin, do you have an equivalent guide for a multilevel model in SPSS (otherwise I'm stuck running 5 different one-way repeated measures ANOVAs, due to having a small sample size on my 2nd and 3rd repeated measure) or know of anyone who has a guide as straightforward as yours? I am afraid my university does not provide access to 'R'. Honestly, your videos have helped me so, so, much and I thank you for getting me this far.
R is totally free, along with R studio (www.r-project.org/) and (rstudio.com/products/rstudio/download/). You could try checking the hlm guide in this folder - don't remember how good it is because it's old: github.com/doomlab/statsofdoom-files/tree/master/advanced/SPSS/Other%20Files
Great videos, Erin! Quick question: You mentioned that you have video on imputation discussing various techniques and the MICE package for handling missing data. Could you provide the link. Many thanks!
Hi Erin, Thank you again. Is it safe to say when you only allow the intercepts to vary between participant in MLM, it is (sort) of equivalent to Mixed RM ANOVA? When you would allow slopes to vary, it's not? Apart from this, maybe it is because youtube deleted comments for correction, but in the beginning of this video you said that you wanted to predict comp_anx based on the rest, but i see you are predicting experience (for two conditions, hence that the two experience measurements (first level) are clustered within participants (second level), right?) I got a bit confused. Some how I expected an IV dummy for the kind of experience (if you keep experience as DV). Thanks again for your time making these videos, there are great and i learn a lot of them, Take Care!
I think random slopes only make sense if they make sense with your data / hypothesis. I don't really ever have a good explanation for them, so I try not to use them personally. Not sure on your second question, I am predicting experience and seeing how anxiety plays into that (maybe I said this backwards). I didn't control for the second level of nesting in this video to keep it simple, but you could certainly test for it or use it as a separate categorical variable.
Thanks a lot Erin, but I was wondering if this model can be used for an experimental design with 2 repeated-measure factors? for example, if my design is 2 by 2, so each participant has 4 values for 4 conditions, and I want do create a mixed model with a random interact of participant, and the fixed effect of interaction of A and B, so the model might be like this: model = lmer(RT~ A*B + (1|subject), data = dataset), but I'm not sure if this correct? thanks a lot for your any comments and suggestions?
Yes! You would still control for participant error as the random intercept (and making the dataset in long format might be hard - you would need column A and column B to be able to use both of them in the model).
I was not aware that you could put labels on factors! Yet I do not know any other person knowing R in real life to discuss and share experiences. mahalanobis I didn't know that either. symnum I didn't know that either. It seems to me that one needs to take notes when watching your videos. Do you have your source anywhere available?
The code and datasets are found on our website (which we are structuring so people can navigate it easier), but these MLM examples are here right now: statstools.com/learn/advanced-statistics-examples/
Very nice. Please keep in mind that you would make me ecstatically happy if you could provide working examples of multilevel (hierarchical) structural equation modelling in R.
Wonderful video! I'm curious if you have a video using lme4 for generalized linear mixed effects models with repeated measures in R? I browsed your videos but perhaps its by a different name :).
@@StatisticsofDOOM okay great I will look into using nlme! Thanks! Oh and do you happen to have an example of syntax where repeated measures are nested in participants? Oh and with a binary outcome hahaha.
The multilevel is because each participant is measured multiple times - when we switch to long format, we do need to control for correlated error of repeated subjects.
Do you still have the data set for this example? I am working on a project for my stats class and I would like to use your example for a presentation. I will be sure to reference you!
This is the best stats channel I have ever seen. I am so happy to have found it! Thank you so much for being so very intelligent and intelligible.
Thanks for this! I've used a few mixed/multi-level modeling packages in R, and this is a very clear tutorial.
Thanks a lot for such a detailed explanation. One question: I am running MLM for repeated measures where measures are taken for four different scenarios for all participants and I want to control the effect of scenarios (inter-dependence). As per my understanding, I should be using 1|scenario instead of 1|partno for random intercept model. Is it correct?
You could also do both list(1|scenario, 1|partno) or nest them together (1|partno/scenario) - this last one would be participants in scenarios (since they saw all of them).
@@StatisticsofDOOM Thank you so much. A followup question regarding the random part: if I remove scenario from the random part just keep the participant as random then the model will not control for the order of the scenario (as the interpretation of the results will be easier if we keep them fix than randomize)?
@@TheAthakur right, it wouldn't control for the items in the scenarios, just the participants.
Thanks a lot... Erin. I am just started working with R. Seems your video will help me a lot...
Appreciate the video and efforts. Autocompletion feature by tab key in RStudio will ease typing.
I was not able to trace the dataset that you used. Please help.
Trace? You mean find?
statstools.com/learn/advanced-statistics-examples/
Hi Erin
Thanks for your great videos. Hope to see something about Mixed-effects modeling one day. The R package was developed by Bates, 2005; Bates & Sarkar, 2007. Although I ran it but totally confused about what to report? Interesting application of the package was also done by Cunnings, I. and Finlayson, I. (2015) Mixed effects modelling and longitudinal data analysis. Again thanks.
Hi Erin
Thanks for your reply. It does help.
Hi Erin,
Quick question.
Say we had a repeated measured dependent variable like arthritis pain score. And we wanted to see if a treatment altered the progression of the score, adjusting for initial pain and other variables like sex, BMI, economic status etc.
The repeated measure would be nested in the individual variable. So level one would contain the repeated measures. Level two would contain patient ID. Level three would be treatment. Where would we put the controlling variables like Sex. My intuition is that these would go in at level two as these vary on the individual level because they weren't repeated measures. Is this correct? Also, would you have a random intercept and random slope for individuals?
By the way great channel!
Thank you for the reply! It didn't "throw up on me". Data looks good :)
#first model
library(nlme)
model1=gls(value~1, data=mlmlong, method="ML", na.action="na.omit")
#randomintercept
model2=lme(value~1, data=mlmlong, method="ML", na.action="na.omit", random = ~1|id)
#Predictors
model3=lme(value~sex+time+variable, data=mlmlong, method="ML", na.action="na.omit", random = ~1|id)
summary(model3)
anova(model2,model3)
> summary(model3)
Linear mixed-effects model fit by maximum likelihood
Data: mlmlong
AIC BIC logLik
243.293 253.4263 -115.6465
Random effects:
Formula: ~1 | id
(Intercept) Residual
StdDev: 0.0001043667 4.35892
Fixed effects: value ~ sex + time + variable
Value Std.Error DF t-value p-value
(Intercept) 11.371354 2.481424 27 4.582591 1e-04
sex1 7.014989 1.482935 27 4.730477 1e-04
time 5.702880 1.452973 27 3.924972 5e-04
variable -21.821207 1.452973 27 -15.018312 0e+00
> anova(model2,model3)
Model df AIC BIC logLik Test L.Ratio p-value
model2 1 3 322.0145 327.0811 -158.0072
model3 2 6 243.2930 253.4263 -115.6465 1 vs 2 84.72145
when i anova two model by anova(model1,model2) it says models were not fitted from the same dataset.
Yes that happens when your participants/lines are not the same across both models. So if you have X1 in model 1 and X1 + X2 in model 2, ... if some participants are missing X2, they were in model 1 but not model 2 - try excluding NAs with something like complete.cases first.
Dear,
I would like if it was possible that the Lord would help me in a review, lest it incur in error.
If it were a simple logistic regression, I could use The Area Under an ROC Curve, but since I'm using a generalized linear mixed model with reply Binaria, can analyze the same way this output?
If not, what would you suggest?
Thank you
Thanks a lot, it is truly helpful. Quick questions: if there are interaction items in the model, how to interpret it.
I'd suggest taking a quick watch of one of the moderation videos I have ... for example, this one might be good ruclips.net/video/BR050BbER6s/видео.html
Great video! Very interesting! I did have one question though. After the model has been generated, is there a way to see if the values from ranef() are significantly different than each other? Thank you so much!
I would get the confidence intervals of those modes and see if they overlap. I’m unsure how to do that but maybe the condVar function?
Amazing videos Erin! I am so glad to have found them, as I have been struggling for a while with my multilevel models :)
I tried applying your screening technics for the mlm, but I'm not sure they can work for my data. I am working with public opinion data from the European Election Studies and there are quite a few missing cases in most of the variables of interest. For example, a variable like positioning a party on a left-right scale is not answered by 1/3 of the respondents. As the respondents who do not answer are changing for each party, mahalanobis ends up ignoring almost all of the dataset. My question is whether I actually need to run the screening as most of it does not give me meaningful results and since R can handle missing data in multilevel models.
Thank you for all the effort! It is much appreciated!
Yes and no. If you are going to use the variable that is mostly missing, the lines with that missing data will be dropped (even if the other variables there). MLM can interpolate missing data (like say, if you have time 1 and 3, you'll still get an estimate, rather than losing the whole person), but it is still regression and any person/line without complete data will be excluded.
Hi Erin, do you have an equivalent guide for a multilevel model in SPSS (otherwise I'm stuck running 5 different one-way repeated measures ANOVAs, due to having a small sample size on my 2nd and 3rd repeated measure) or know of anyone who has a guide as straightforward as yours? I am afraid my university does not provide access to 'R'.
Honestly, your videos have helped me so, so, much and I thank you for getting me this far.
R is totally free, along with R studio (www.r-project.org/) and (rstudio.com/products/rstudio/download/).
You could try checking the hlm guide in this folder - don't remember how good it is because it's old: github.com/doomlab/statsofdoom-files/tree/master/advanced/SPSS/Other%20Files
Thank you for this great tutorial.
Glad it helps!
Great videos, Erin! Quick question: You mentioned that you have video on imputation discussing various techniques and the MICE package for handling missing data. Could you provide the link. Many thanks!
ruclips.net/channel/UCMdihazndR0f9XBoSXWqnYgsearch?query=r+data+screening
Here are a bunch of videos about data screening!
Hi Erin,
Thank you again. Is it safe to say when you only allow the intercepts to vary between participant in MLM, it is (sort) of equivalent to Mixed RM ANOVA? When you would allow slopes to vary, it's not? Apart from this, maybe it is because youtube deleted comments for correction, but in the beginning of this video you said that you wanted to predict comp_anx based on the rest, but i see you are predicting experience (for two conditions, hence that the two experience measurements (first level) are clustered within participants (second level), right?) I got a bit confused. Some how I expected an IV dummy for the kind of experience (if you keep experience as DV). Thanks again for your time making these videos, there are great and i learn a lot of them, Take Care!
I think random slopes only make sense if they make sense with your data / hypothesis. I don't really ever have a good explanation for them, so I try not to use them personally.
Not sure on your second question, I am predicting experience and seeing how anxiety plays into that (maybe I said this backwards). I didn't control for the second level of nesting in this video to keep it simple, but you could certainly test for it or use it as a separate categorical variable.
Thanks a lot. Very clear explanation. Subscribed!
Thanks a lot Erin, but I was wondering if this model can be used for an experimental design with 2 repeated-measure factors? for example, if my design is 2 by 2, so each participant has 4 values for 4 conditions, and I want do create a mixed model with a random interact of participant, and the fixed effect of interaction of A and B, so the model might be like this: model = lmer(RT~ A*B + (1|subject), data = dataset), but I'm not sure if this correct? thanks a lot for your any comments and suggestions?
Yes! You would still control for participant error as the random intercept (and making the dataset in long format might be hard - you would need column A and column B to be able to use both of them in the model).
@@StatisticsofDOOM Thanks a lot Erin!
I was not aware that you could put labels on factors! Yet I do not know any other person knowing R in real life to discuss and share experiences. mahalanobis I didn't know that either. symnum I didn't know that either. It seems to me that one needs to take notes when watching your videos. Do you have your source anywhere available?
The code and datasets are found on our website (which we are structuring so people can navigate it easier), but these MLM examples are here right now: statstools.com/learn/advanced-statistics-examples/
Very nice. Please keep in mind that you would make me ecstatically happy if you could provide working examples of multilevel (hierarchical) structural equation modelling in R.
Great video!!!! Extremely helpful - thank you!
Glad to be of help!
Wonderful video! I'm curious if you have a video using lme4 for generalized linear mixed effects models with repeated measures in R? I browsed your videos but perhaps its by a different name :).
Most of my examples are with nlme, which I find to be a bit easier than lme4 - gives me approximately the same results.
@@StatisticsofDOOM okay great I will look into using nlme! Thanks! Oh and do you happen to have an example of syntax where repeated measures are nested in participants? Oh and with a binary outcome hahaha.
Shoot me an email ... I think have some binary log code that uses glmer that is nested. buchananlab at gmail
@@StatisticsofDOOM awesome, sent an email, thanks so much!
Can you explain where the multilevel is, in this example? Is it the melting?
The multilevel is because each participant is measured multiple times - when we switch to long format, we do need to control for correlated error of repeated subjects.
Hi. Do you have a link for the mlm example csv dataset? Thanks.
statstools.com/learn/advanced-statistics-examples/
It's under the MLM examples.
Hi Erin, great lecture. I would like to see the dataset while learning from it, do you have a link for the dataset (the above link is not working)?
Do you still have the data set for this example? I am working on a project for my stats class and I would like to use your example for a presentation. I will be sure to reference you!
Thank you!
Error in reStruct(random, REML = REML, data = NULL) :
object 'partno' not found
I might need a bit more context to help here - but sounds like you didn't define a participant number in the dataframe.
@@StatisticsofDOOM Curiously I did that and partno is working, in the rest of the code. Strange.
Please could you send the R code?
All information can be found here: statstools.com/learn/advanced-statistics/
Our code is at the OSF page linked on our website.
Thank you very much :-)