Thank you for the video, have you ever done a MGA moderated mediation model? I saw you have one but the moderation is on the a path and only has one moderating term
I've never done a moderated mediation that is also part of a multigroup comparison. It should be as "simple" as adding a grouping variable to a moderated mediation input file.
I am running a multigroup mediation model on Mplus. After checking the Chi-Square difference between constrained and unconstrained models by sex, the result is significant suggested that pathways may differ for males and females. However, when constraining individual pathways sequentially, there does not appear to be any significant differences between males and females. All pathways are significant, however there is only a difference in magnitude. How would I report this?
@@Gaskination Thanks! Is it normal for that to happen though (i.e., significant difference between constrained and unconstrained pathways but no significant difference in any individual pathway)?
Hi James! This is so helpful; thank you so much. I have a question if you're willing to help. When I run this, for my unconstrained model, I get a chi-square of 0 and degrees of freedom are also 0. For the constrained, the chi-square is 13.305 and degrees of freedom are 16. Nevertheless, when I conduct the chi-square test of differences, the models are not significantly different. Is it okay to report a chi-square of 0 or should I do some kind of follow-up? It seems odd that it would come up as 0.
The chi-square is zero because the DF are zero. That means all potential parameters are accounted for. Therefore, the proposed model (yours) perfectly matches the observed model (implied by the covariance matrix). If you want a degree of freedom, include some control variables, or eliminate non-critical paths.
Hi James, Sorry to bother you again -- I am running a multigroup analysis with latent variables. I can either find an example with only "BY" statements or only "ON" statements, however, I have both. Is it possible to run this model and which parameters do I need to constrain (all the BY and all the ON statements ?) ? Thank you!
Hi James, In your video (toward the end), feedback on JOBSAT is nonsignificant in CS group (p = 0.087) but significant in the BS group (p = 0.001). So why does the chi-square difference test you ran in the end show NO significant difference? In other words, when we test each path invariance, the chi square test result (either sig or nonsig) is dependent on the estimate value (coefficient) in each group, not the p value for that path in each group? Thank you very much!
The non-sig p-value for the difference is likely due to high error in one group (CS) making it difficult to determine with confidence that the two groups are different. Nevertheless, we can still claim moderation such that the predictor is a good predictor for one group and not the other.
@@Gaskination Sounds good! Thank you so much for your timely reply! These have confused me for days and no other resources can give me an answer. I will keep learning from your channel!
I was just keeping it simple by testing a single hypothesis (i.e., that the model's regression weights were invariant). However, if you would like to test all these differences at the same time, I believe the literature supports that approach.
Thank you so much for this video! Can you explain why you chose the right-tailed chi-square model fit value instead of the value reported lower down in the output? I'm having an issue with the lower chi-square value keeping my df the same between the constrained and unconstrained model, but not in the right-tailed chi-square value. Any help would be appreciated!
It's just a one-tailed vs two-tailed test. I used one-tailed because I like to be extra sensitive to false negatives when testing moderation. I also didn't know about the p-value for the model test at the bottom :) I tried checking it right now, but my Mplus is not working properly...
Hello, great video. I'm running a path analysis and am checking whether the model fits across different demographic control variables in the same way. Would it appropriate to test each in turn whilst controlling for the others? For example, I have gender, caring responsibility, and ethnicity - would it be appropriate when running the multigroup for gender that I still include caring responsibility and ethnicity as controls (as per the overall general model) or should I take those out so it focuses on the core hypothesized paths? The other question I have is can multigroup analysis be conducted when there are more than 2 groups or would you have to run separate ones, for example if you wanted to look at uk versus usa versus france, would you look at uk vs usa, uk vs france, usa vs france, or could you combine as one overall multigroup analysis?
Good questions: 1. It is usually best practice to include all variables so that the model is as complete as possible and so that we can have greater confidence in the observed effects. 2. You should test them in pairs as you've noted, or you can also do USA vs other, France vs other, UK vs other
Hi, thank you for this wonderful video. Can you test wether indirect effects are different across groups and how would you do that? Same way? I am not sure if indirect effects need to be constraint since the individuals paths are already being constraint. Thank you again!
If they are not categorical/nominal, then just include them as you would any other independent variable. If they are categorical, then you'll need to break them into binary dummy variables. With the three you chose, age and sex (if using binary) can be included like normal. Race would need to be split into many binary variables (e.g., European, Japanese, Chinese, Indian, Mexican each have their own binary variable to indicate whether the value is true for that row).
Thank you for your videos! I'm having a hard time interpreting a difference I've found. I compared the fully constrained model fit to a model where one of my paths are unconstrained (I'm testing for multigroup invariance by racial group). There was a significant difference (as indicated by the p-val of the difference in chi-square test); however, the path is non-significant for both groups. How do I interpret the difference given the non-significance of the relationships between the variables for both groups? Thanks in advance!
I would report it this way. Say that although the path is significantly different across groups, it is not significant for either group independently. So, any conclusions about the difference should be taken with caution.
Thank you for this educational video! This is really helpful! Now I have a question about an analysis that contains three groups. If I unconstrain one path and I find the chi-square different from that of the fully constrained model, what should I do next to know which two groups or if it is all three groups that make this difference?
@@Gaskination Thank you for your reply! So please let me assume I have three countries (country A, country B, and country C). When I do one of the three pairwise comparisons, in the syntax do I type GROUPING IS COUNTRY (0=A 1=B), and then I compare the constrained model to the model which has the path I am interested in unconstrained? Is it okay not to put 2=C in the syntax? Will Mplus know what I try to do?
Thanks for the explanation -- when I'm running a grouping syntax, I am not getting any model fit in the output. Do you have any idea why that'd be the case? Thanks in advance!!!!
If the fully constrained model IS significantly different from a free model, should I then proceed from the free model (and start to constrain path by path) or should I proceed from the fully constrained model (and release constraints path by path)?
I've seen both approaches taken. If you start with the constrained model, and unconstrain one path at a time, essentially this says that the model is assumed to be invariant (constrained to be equal) except for the path that is unconstrained. When the whole model is constrained, if the chi-square difference test is significant, then we have not observed invariance. So, we check each path. If we observe a non-significant chi-square difference test after unconstraining a single path, then we know that this path was the cause of non-invariance when the whole model was fully constrained. If we do it the other way, constraining only one path at a time, rather than unconstraining, then we are not making any assumptions about invariance except regarding the individual path. This means that even if we do observe a significant chi-square difference test (i.e., the path is not invariant across groups), we'll still need to test the full model for invariance while unconstraining the specific path; otherwise we cannot claim invariance due to that path alone. Hope this makes sense.
@@Gaskination Yes it makes sense. Thank you for the explanation! My follow-up question is: if I go from the constrained model and release a path at a time, what do I do with the intercepts and residual variances? Do i release some of those correspondingly or do these stay constrained across groups when just 1 path is unconstrained. (i'm using MGA for moderated mediation)
@@yumengf I would recommend having all or none (of the intercepts and residual variances) constrained for path by path analysis. The assumptions in these cases are as stated before. In your case, I would recommend a fully constrained model (including intercepts and residual variances).
@@Gaskination In my understanding, we would like to see if constraining one specific path to be equal diminishes the unconstrained model fit significantly (i.e., the chi-squared difference becomes significant). If it does, then that specific path is not invariant (different between groups); otherwise, it is invariant (equal between groups). I can see in your AMOS tutorial that you used this procedure (constraining paths one at a time). Is this any different in Mplus? ruclips.net/video/w5ikoIgTIc0/видео.html
@@benjaminghasemi9843 it is true that I have used both approaches. the concepts are the same, but the assumptions are different. The more conventional assumptions recommend constraining the whole model except the path under scrutiny. However, the simpler to understand model is the other way around.
Essentially you'll want to subtract the two indirect effects from each other during a bootstrap and then form confidence intervals around that difference to see if it is significantly different from zero (i.e., does zero fall within the interval).
Thanks for this helpful video. Just a question: I have a mediation model which is identified. How can I compare it with the constrained model made equal by gender? I just have the chisq for the unconstrained model which is zero..
Good question. In this case, when DF=0, you can just make a rough comparison of the regression weights. You can also test whether those regression weights are different by using a difference of slopes test: www.danielsoper.com/statcalc/calculator.aspx?id=103
@@Gaskination Thanks Doctor Gaskin. Very interesting. However, I am also interested in comparing the six indirect effects I have in the model, and see if they change in significance between males and females,. Is it sufficient to inspect the significance of each one separately for each group? And say in the paper that I adopted the subgroup approach because I started with a just-identified model? Thanks in advance
@@Paolo-tu2ft If in Mplus, then you can label the paths and then set a new variable to be equal to the subtraction of one indirect path from another. This should produce a confidence interval and p-value for the difference. However, I can’t double check my advice right now, as I’m on my iPad and not near a device that can run mplus. Best of luck to you.
Thanks James. Having run these analyses before, it's easy to get tripped up trying to examine moderation with complex models. This is very clear.
James need video on within-person and between-group centering and also on multiple group path analysis using that centred data.
Looking forward to having your video on Multilevel analysis with Mplus too.
Thank you for the video, have you ever done a MGA moderated mediation model? I saw you have one but the moderation is on the a path and only has one moderating term
I've never done a moderated mediation that is also part of a multigroup comparison. It should be as "simple" as adding a grouping variable to a moderated mediation input file.
I am running a multigroup mediation model on Mplus. After checking the Chi-Square difference between constrained and unconstrained models by sex, the result is significant suggested that pathways may differ for males and females. However, when constraining individual pathways sequentially, there does not appear to be any significant differences between males and females. All pathways are significant, however there is only a difference in magnitude. How would I report this?
I would recommend reporting it just like that. Give all the evidence.
@@Gaskination Thanks! Is it normal for that to happen though (i.e., significant difference between constrained and unconstrained pathways but no significant difference in any individual pathway)?
@@brettholfeld4913 It's not too common, but it can happen without any mistake.
@@Gaskination Great!
Thank you for a clear explanation! It is very helpful!
Hi James! This is so helpful; thank you so much. I have a question if you're willing to help. When I run this, for my unconstrained model, I get a chi-square of 0 and degrees of freedom are also 0. For the constrained, the chi-square is 13.305 and degrees of freedom are 16. Nevertheless, when I conduct the chi-square test of differences, the models are not significantly different. Is it okay to report a chi-square of 0 or should I do some kind of follow-up? It seems odd that it would come up as 0.
The chi-square is zero because the DF are zero. That means all potential parameters are accounted for. Therefore, the proposed model (yours) perfectly matches the observed model (implied by the covariance matrix). If you want a degree of freedom, include some control variables, or eliminate non-critical paths.
Hi James,
Sorry to bother you again -- I am running a multigroup analysis with latent variables.
I can either find an example with only "BY" statements or only "ON" statements, however, I have both.
Is it possible to run this model and which parameters do I need to constrain (all the BY and all the ON statements ?) ?
Thank you!
check out sections 5.8 and 5.11 here: www.statmodel.com/download/usersguide/Chapter5.pdf
Hi James,
In your video (toward the end), feedback on JOBSAT is nonsignificant in CS group (p = 0.087) but significant in the BS group (p = 0.001). So why does the chi-square difference test you ran in the end show NO significant difference? In other words, when we test each path invariance, the chi square test result (either sig or nonsig) is dependent on the estimate value (coefficient) in each group, not the p value for that path in each group? Thank you very much!
The non-sig p-value for the difference is likely due to high error in one group (CS) making it difficult to determine with confidence that the two groups are different. Nevertheless, we can still claim moderation such that the predictor is a good predictor for one group and not the other.
@@Gaskination Sounds good! Thank you so much for your timely reply! These have confused me for days and no other resources can give me an answer. I will keep learning from your channel!
Wonderful. More mplus please
When testing the fully constrained model, why didn't you constrain residual variances, intercepts, and covariances as well?
I was just keeping it simple by testing a single hypothesis (i.e., that the model's regression weights were invariant). However, if you would like to test all these differences at the same time, I believe the literature supports that approach.
Thank you so much for this video! Can you explain why you chose the right-tailed chi-square model fit value instead of the value reported lower down in the output? I'm having an issue with the lower chi-square value keeping my df the same between the constrained and unconstrained model, but not in the right-tailed chi-square value. Any help would be appreciated!
It's just a one-tailed vs two-tailed test. I used one-tailed because I like to be extra sensitive to false negatives when testing moderation. I also didn't know about the p-value for the model test at the bottom :) I tried checking it right now, but my Mplus is not working properly...
Hello, great video. I'm running a path analysis and am checking whether the model fits across different demographic control variables in the same way. Would it appropriate to test each in turn whilst controlling for the others? For example, I have gender, caring responsibility, and ethnicity - would it be appropriate when running the multigroup for gender that I still include caring responsibility and ethnicity as controls (as per the overall general model) or should I take those out so it focuses on the core hypothesized paths? The other question I have is can multigroup analysis be conducted when there are more than 2 groups or would you have to run separate ones, for example if you wanted to look at uk versus usa versus france, would you look at uk vs usa, uk vs france, usa vs france, or could you combine as one overall multigroup analysis?
Good questions:
1. It is usually best practice to include all variables so that the model is as complete as possible and so that we can have greater confidence in the observed effects.
2. You should test them in pairs as you've noted, or you can also do USA vs other, France vs other, UK vs other
Hi, thank you for this wonderful video. Can you test wether indirect effects are different across groups and how would you do that? Same way? I am not sure if indirect effects need to be constraint since the individuals paths are already being constraint. Thank you again!
Just model indirect while conducting MGA. Here is a video about modeling indirect: ruclips.net/video/MkigzqJFNu0/видео.html
Dear James, do you have a reference for the recommended way to report these results in a manuscript?
Here are a few: statwiki.gaskination.com/index.php?title=References#Moderation_and_Multigroup
@@Gaskination Thanks so much for sharing! I appreciate the help!
Thank you so much for this really helpful video! I was wondering how we can incorporate demographic covariates into the model (age, sex, race)
If they are not categorical/nominal, then just include them as you would any other independent variable. If they are categorical, then you'll need to break them into binary dummy variables. With the three you chose, age and sex (if using binary) can be included like normal. Race would need to be split into many binary variables (e.g., European, Japanese, Chinese, Indian, Mexican each have their own binary variable to indicate whether the value is true for that row).
@@Gaskination Thanks so much! It sounds like I should give the covariates their own line, right?
@@hannahloso445 Yes, that makes sense.
Thank you for your videos! I'm having a hard time interpreting a difference I've found. I compared the fully constrained model fit to a model where one of my paths are unconstrained (I'm testing for multigroup invariance by racial group). There was a significant difference (as indicated by the p-val of the difference in chi-square test); however, the path is non-significant for both groups. How do I interpret the difference given the non-significance of the relationships between the variables for both groups? Thanks in advance!
I would report it this way. Say that although the path is significantly different across groups, it is not significant for either group independently. So, any conclusions about the difference should be taken with caution.
@@Gaskination Thank you for your response (and the speed in which you did so)! Your videos have been incredibly helpful!
Thank you for this educational video! This is really helpful! Now I have a question about an analysis that contains three groups. If I unconstrain one path and I find the chi-square different from that of the fully constrained model, what should I do next to know which two groups or if it is all three groups that make this difference?
You would then need to compare these groups pairwise. So, group 1 to group 2, and then 1:3, then 2:3.
@@Gaskination Thank you for your reply! So please let me assume I have three countries (country A, country B, and country C). When I do one of the three pairwise comparisons, in the syntax do I type GROUPING IS COUNTRY (0=A 1=B), and then I compare the constrained model to the model which has the path I am interested in unconstrained? Is it okay not to put 2=C in the syntax? Will Mplus know what I try to do?
@@mln853 Yes, I believe that should work. If Country C isn't defined, then it just won't use it.
@@Gaskination Thank you for your suggestion!
Thanks for the explanation -- when I'm running a grouping syntax, I am not getting any model fit in the output. Do you have any idea why that'd be the case?
Thanks in advance!!!!
Some of the model fit won't show up if you don't asked for standardized output. You might also not get model fit if there are zero degrees of freedom.
Thanks! and the only way to change dF is to put some constraints in ?
Hi James! i was wondering where can I find this data set to practice?
It's linked from the syntax examples doc here: statwiki.gaskination.com/index.php?title=Mplus
If the fully constrained model IS significantly different from a free model, should I then proceed from the free model (and start to constrain path by path) or should I proceed from the fully constrained model (and release constraints path by path)?
I've seen both approaches taken. If you start with the constrained model, and unconstrain one path at a time, essentially this says that the model is assumed to be invariant (constrained to be equal) except for the path that is unconstrained. When the whole model is constrained, if the chi-square difference test is significant, then we have not observed invariance. So, we check each path. If we observe a non-significant chi-square difference test after unconstraining a single path, then we know that this path was the cause of non-invariance when the whole model was fully constrained.
If we do it the other way, constraining only one path at a time, rather than unconstraining, then we are not making any assumptions about invariance except regarding the individual path. This means that even if we do observe a significant chi-square difference test (i.e., the path is not invariant across groups), we'll still need to test the full model for invariance while unconstraining the specific path; otherwise we cannot claim invariance due to that path alone.
Hope this makes sense.
@@Gaskination Yes it makes sense. Thank you for the explanation! My follow-up question is: if I go from the constrained model and release a path at a time, what do I do with the intercepts and residual variances? Do i release some of those correspondingly or do these stay constrained across groups when just 1 path is unconstrained. (i'm using MGA for moderated mediation)
@@yumengf I would recommend having all or none (of the intercepts and residual variances) constrained for path by path analysis. The assumptions in these cases are as stated before. In your case, I would recommend a fully constrained model (including intercepts and residual variances).
Hi,
For individual paths, shouldn't we constrain them one at a time and compare that model with only one constrained path to the unconstrained model?
As shown later in the video, the appropriate approach is to constrain the whole model except the one path to be tested.
@@Gaskination In my understanding, we would like to see if constraining one specific path to be equal diminishes the unconstrained model fit significantly (i.e., the chi-squared difference becomes significant). If it does, then that specific path is not invariant (different between groups); otherwise, it is invariant (equal between groups).
I can see in your AMOS tutorial that you used this procedure (constraining paths one at a time). Is this any different in Mplus? ruclips.net/video/w5ikoIgTIc0/видео.html
@@benjaminghasemi9843 it is true that I have used both approaches. the concepts are the same, but the assumptions are different. The more conventional assumptions recommend constraining the whole model except the path under scrutiny. However, the simpler to understand model is the other way around.
Thanks for your useful video. If there is a mutiple mediating model, how to compare the indirect effect between two groups?
Essentially you'll want to subtract the two indirect effects from each other during a bootstrap and then form confidence intervals around that difference to see if it is significantly different from zero (i.e., does zero fall within the interval).
Thanks for this helpful video. Just a question: I have a mediation model which is identified. How can I compare it with the constrained model made equal by gender? I just have the chisq for the unconstrained model which is zero..
Good question. In this case, when DF=0, you can just make a rough comparison of the regression weights. You can also test whether those regression weights are different by using a difference of slopes test: www.danielsoper.com/statcalc/calculator.aspx?id=103
@@Gaskination Thanks Doctor Gaskin. Very interesting. However, I am also interested in comparing the six indirect effects I have in the model, and see if they change in significance between males and females,. Is it sufficient to inspect the significance of each one separately for each group? And say in the paper that I adopted the subgroup approach because I started with a just-identified model? Thanks in advance
@@Paolo-tu2ft If in Mplus, then you can label the paths and then set a new variable to be equal to the subtraction of one indirect path from another. This should produce a confidence interval and p-value for the difference. However, I can’t double check my advice right now, as I’m on my iPad and not near a device that can run mplus. Best of luck to you.