Validity during CFA made easy
HTML-код
- Опубликовано: 21 авг 2024
- In this video I demonstrate a new excel tool I've developed for quickly creating a factor correlation matrix and validity table using the output from AMOS. The tool is available from the homepage of my wiki: statwiki.gaskin...
Here's a fun pet project I've been working on: udreamed.com/. It is a dream analytics app. Here is the RUclips channel where we post a new video almost three times per week: ruclips.net/channel/UCiujxblFduQz8V4xHjMzyzQ
Also available on iOS: apps.apple.com/us/app/udreamed/id1054428074
And Android: play.google.com/store/apps/details?id=com.unconsciouscognitioninc.unconsciouscognition&hl=en
Check it out! Thanks!
It is on the homepage of my wiki: statwiki. kolobkreations. com
It will probably not work for Lisrel since Excel is expecting a specific format as in the AMOS output.
Excellent question! Usually numbers are red for one of two reason. 1. the factor has a low loading (or average loadings below 0.70), or 2. one of the factor correlations is greater than the average loading. To fix the first issue, see if you can drop a low loading. To fix the second issue, you might want to see how individual indicators correlate across factors. If you find two from separate factors that are highly correlated, see if you can remove one of them. Hope this helps!
My Excel workbook will do it for you. It is available on the homepage of my wiki: statwiki. kolobkreations. com. If you need to do it for a model with less than three latent factors, then email me to get my old Excel workbook: james. gaskin@ byu. edu
It is on the homepage of my wiki: statwiki. kolobkreations. com
YOU ARE MY HERO!!!! thank you so much for doing this, i am currently writing my masters thesis and thanks to your tool I am saving hours of work!!
Hello! How can I attain spurred analyz of excel for this correlation table ? Please help me for this question you only paste on excel then excel have click to reset button. How can I make this table 😥😥😥
it is on the homepage of my wiki: statwiki. kolobkreations. com
Discriminant validity is usually tested between constructs, not models. But if you wanted to run a chi-square test between an unconstrained and a constrained model, watch this video for instructions: "Measurement Model Invariance" In this video I show how to do a chi-square difference test around 3:20
@ryanonfire
The excel sheet is available on my wiki: statwiki. kolobkreations. com
Thank you. I spent the entire date calculating these indicators and just found your excel tool!
Amazing excel program Thanks a lot Dr. Gaskin for providing all this resource free of cost.I appreciate your work!!
Of course! I lived in Tokyo for three years. I was in Yokosuka on the military base. The excel tool is on my wiki. See my channel homepage for a direct link. statwiki. kolobkreations. com
Have you met the caveats and assumptions listed? People send me their data all the time saying there is an error that they just can't figure out, but then I find out they've not met the four caveats and assumptions I list right there below the button on the spreadsheet. If you have met these, and there is still a problem, feel free to email it to me to troubleshoot: james. gaskin@ byu. edu
Thank you so much James! Finding your videos is the best thing happened to me this week!
You're a genius! Really thankful to be able to discover your videos and stat tools
Hello! How can I attain spurred analyz of excel for this correlation table ? Please help me for this question you only paste on excel then excel have click to reset button. How can I make this table 😥😥😥
Low AVE is due to low loadings. The AVE is essentially the average of the squared loadings for each factor. So, if you have a really low loading on one of the items, you might consider removing it (if possible) in order to bump up your AVE.
It is on my wiki on the homepage. statwiki. kolobkreations. com
I have almost a similar model in my PhD i mean it's actually huge. As i have 2 big blocs of determinants ( cultural and sociological) . May I ask please if it's a good idea to test each bloc of determinants apart?
Thank you for the sharing in the video🙏
Best practice is to do a CFA with all latent factors simultaneously, as this will help identify any discriminant validity concerns.
Dear Professor James Gaskin!
Thank you so much for your tool.
Oh, you have a single factor that has eight items. I understand now. You can use my old stats tool and old validity video. email me for the file: james. gaskin@ byu. edu
yes. it will create the validity measures and the matrix with the highest order factors for any set of factors. So, if you have several first order factors as well as some 2nd order factors, all factors will appear in the matrix except the lower order indicator factors of the higher order factors.
MSV is the maximum shared variance squared (so if there are a bunch of covariance arrows going between factor 1 and the other factors, you square the highest one shared with factor 1 to get the MSV for factor 1). ASV is the Average shared variance squared (so just average all the covariance values for factor 1 to get the ASV). Hope this helps.
Hi James, a book that I read also mentions testing Nomological validity. Did you have any video about how to test Nomological validity? thank you
Nomological validity refers to the theoretical completeness of the model. As such, it is difficult to test statistically. However, there are some indirect approaches, such as tests for endogeneity and percent of variance explained (R2) in the dependent variable. A better way to "test" it is to make sure a good literature review is completed before designing the study. This will ensure all appropriate and potentially confounding variables are identified and controlled for.
Awesome, clear videos! And amazing excel tool too. Just saved me hours. Thank you so much, James! You're a boss.
1. Depends on the complexity of the model. If you will not be sacrificing statistical power, then split it 30%/70% for the EFA/CFA. This is a common practice that demonstrates additional rigor. You can then also combine both subsamples back together for the structural model.
2. Second order models are theory-driven. If you have a construct that is really composed of two distinct dimensions (both captured latently), then you may want to use a 2nd order factor.
This tool is godsend!!!! Saves so much time - Thank you very much James - really fantastic stuff!
I would recommend doing it in two steps as you suggest. Evaluate the first order first to ensure they are indeed distinct and reliable constructs, then evaluate the full model with the 2nd order construct in place. I can't remember if my video on 2nd order factors in AMOS covers CFA, but I think it does. Hope this helps.
Incredible tool, I just want to thank you for creating it!
If you google "SPSS Syntax aronlindberg", click the first link and then go to "insert correlation matrix" you can use that syntax to import a correlation matrix to SPSS, save it as a .sav file which can then be opened in AMOS.
Wow, just in time for my dissertation data analysis. You are awesome! Thank you!
Yes, if you have low AVE, then remove the lowest loading item. If you have high MSV/ASV, then identify the Factor that has the strongest relationship with your problematic factor. Then do a bivariate correlations analysis on their items to see which ones are most related. Remove one of them. Or you could just look at the EFA to see if an item from one factor is loading strongly on another factor. If so, then delete it. This is probably the easiest way.
You can try running the EFA separate and then combine them for the CFA to see if you can achieve discriminant validity, but likely you will run into trouble. You cannot run the CFA separately for DVs because of this issue of tautology. I don't know if others have published papers that say you can, but from a purely practical standpoint, you cannot/should not.
@mrahimAustralia
It is on the home page of my wiki. The line containing the link looks like this: "You may find this set of excel tools useful/necessary for many of the analyses you will learn about in this wiki: Stats Tools Package". RUclips won't let me type or paste a URL in a comment (for spam issues I think).
Composite reliability is a specific measure for establishing construct reliability. Cronbach's alpha is another specific measure for construct reliability.
+Viccolino Margharetti, Look at the caveats and assumptions listed under the button. These are critical. I'm in the process of making a new one, but in the meantime, you'll need to conform to these caveats and assumptions.
Most likely this is because you are not meeting the Caveats and Assumptions:
1. Your latent variable names do not end in numbers (bad: F1, Factor12). It is okay to have oberved variables named whatever you want.
2. Your error/residual names (if any) do end with numbers (good: e1, res12)
3. Your variable names are not any of the following: AVE, ASV, CR, MSV
4. You have more than 2 latent variables.
The reason is because items from one indicator are too highly correlated with items from another factor. Did you run an EFA to see what the factor structure should be? If not, that is what you should do first. This will also tip you off to know which items are causing the problem. You might be required to drop one or two items, or create a new factor you hadn't expected, or combine factors that are too similar. Do the EFA. That will help you figure it out.
This is definitely a convergent validity issue. Addressing standardized residual covariances is a model fit issue. As for the AVEs, you can look for items that have the lowest loading and drop those (if possible), or you can try to do the EFA again to see which items are cross-loading. Fidget with those until you get a good pattern matrix in the EFA. Then the CFA should go much smoother.
I am still enjoying your video!!!
you can use the old stats tools package for that, and the old validity video shows how to use it. email me separately for the file james. gaskin@ byu. edu
This means that the factor has some issues, but they could be fixed possibly by removing the lowest loading item. You should aim for an average loading of about 0.700.
Dude, you are my savior. Thank you so much for the tool!
Great program, very easy to use and understand. Thank you so much for creating it.
I'm having to learn SEM on my own and am using MPlus. I'm seriously considering finding AMOS because it is so much more user friendly. The coding is relatively easy in MPlus but the output is very difficult to interpret. Thanks for putting these videos together!
You have the option to do either. I would just run the whole model together for the CFA. Then, if you run into problems, run them separately for each model.
The most likely scenario is that you have not met the assumptions listed. If after reading those, and complying, it still doesn't work, feel free to email it to me separately using the email listed on my wiki or youtube channel.
The excel is extremely useful. Thanks James!
MSV stands for Maximum Shared Squared Variance. So, it is just the square of the highest covariance for that factor. Hair et al 2010 talk about it indirectly.
oh, I finally understood the calculation. thank u so much James.
U just saved my life
As an addendum to my response below, I should have also meantioned that the covariances that you are using should be the standardized ones (which is to say, the correlations table).
Dear Mr Gaskin, Thank you very much for you tutos that are very helpful...Phd student all over the world owe you a lot!!! I used your excel stats tools Package to proceed to the discriminant validity... There are no validity concerns but it shows a MaxR(h) index.. and I not understand what is it about. Thank you in advance for your answer..
Same problem as Karine GG. And, if MSV should be < AVE, why there is not issue with OC and OCC (minute 1:29)? Many thanks and congratulations for these videos!
No need to include MaxR. No one really uses it. No one uses MSV either... Most just use CR and AVE.
That could be due to a few different things. Have you complied with the four caveats and assumptions? If not, then do that and see if it works. If so, then email me separately to see what the issue is. My email is listed on the channel's homepage.
I would test for discriminant validity during the confirmatory factor analysis. Also, I would use composites created by AMOS rather than averages. See my video on Imputing Composite Variables in AMOS.
I must not understand. Discriminant validity of what? Typically we test the discriminant validity of latent factors. But we do this with measures like MSV, correlations, and the square root of the AVE. I do not understand why you would use a chi-square difference test for testing discriminant validity of latent factors. If you are testing between models, then yes, chi-square difference test would work. Just use the chi-square and df for each model and input into my spreadsheet. Hope this helps.
I always run them together. We expect the DVs to be strongly related to the IVs, but if they are not substantially different enough to load separately in an EFA, then we may have a problem with tautology (predicting Y with Y).
Neither of those are issues. You are welcome to send me an excel workbook with your data in it (prior to running the macro). One other issue might be the Excel version you are running. I used Excel 2010, and I know it also works with 2007. I'm not sure about 2008, 2010, or 2013.
The equations are very simple. MSV is Maximum Shared Squared Variance. So, it is just the maximum correlation (squared covariance) with any other factor. ASV is average shared squared variance. So, it is just he average of all correlations with other variables. I'm not sure of the reference, but the math is simple.
Hi James, Is there a work around to check validity if I have two latent variables?
Thanks.
I'm not sure I understand the question, but I'll guess. You could create composite variables (I have a video for this) and then use those composite variables as indicators in a cluster analysis.
The AVE is average variance extracted. The average of a single loading is itself. But this measure is also squared and takes into account some error. However, for a single item, the loading is 1 and the error is zero. So, no. There is not AVE for a single item measure.
Means you probably did not meet the assumptions and caveats of listed under the button. Or your tables were not formatted the way AMOS outputs them.
Most likely this is due to either not having the output formatted exactly as it is in the AMOS output window, or not following the caveats and assumptions listed under the button.
Thank you for your video clips. This one was especially helpful.
Sum of Squared loadings divided by (sum of squared loadings + error). These will be slightly different from the cronbach's alpha, but only slightly.
wow you are amazing james
Most likely due to having non-latent factors (single item factors) included in the CFA. If a factor is not latent, it does not belong in a factor analysis.
Thank you! Yes, I have met all the assumptions. I'll try to get some help from a friend today and if it still doesn't work, I'll email you.
Thank you very very very much!
Kesia
I don't know if there is one. You can do Cronbach's alpha instead of CR. It is found in Analyze, Scale, Reliability Analysis.
Thank you! I really appreciated it.
Caloy Carlo, You should do all you can to bump the CR over 0.700. Usually this is done by dropping the lowest loading item. You can also drop outliers, as these will cause increased error and reduced CR. If you cannot do it, then just list it as a limitation, but say how the discriminant and convergent validity is otherwise fine.
Thanks so much James.
if I am dropping the lowest loading item so it effecting the CFI value and decreasing it from .950 to.931 ? is it ok ?
0.931 is fine enough, as long as other model fit measures are still okay (including SRMR and RMSEA).
James Gaskin Thankyou so much for your reply. I am wonder to see your video when I had the problem of not having Pattern matrix builder plugin and for this matters your video was highly helpful .
Please tell me One more thing, now my all results are fine but convergent validity of 2 variables out of 10 is less than 0.05 as my values are (.468 and for other variable .491) and i can not reduce any item now, so with these AVE values can I go further ?
If the CR is fine, then AVE just below the threshold is fine. See Malhotra and Dash referenced here: statwiki.kolobkreations.com/index.php?title=References
Possibly days :) because you can quickly test alternative models (which previously took hours for complex models).
Amazing tool, thanks so much for building this! Can you calculate CR/AVE if you only have 2x latent variables? I have 6 items I want to test in latent variables of 3 each, and while your tool does a great job with 3x latent variables it doesn't seem to work with 2x. Thanks for any advice!
I use an estimated error value (this is a weakness of my current tool). I plan on adapting the tool to take into account the actual error values in the future. However, the differences in CR are exceptionally slight when comparing the estimated error with the actual error. I'm not sure about the CIs for CR. I think you'll have to do it by hand.
@vahidaryadoust
Factor correlations should be less than the square root of the AVE on the diagonal. As long as they are below it, then discriminant validity is acceptable. However, correlations should be less than 0.9, and ideally less than 0.7. The factor correlations say nothing about convergent validity.
thank you so much for this -- this is going to save me hours !!!
Thanks heaps James! You have been really helpful!
Thank you so much for this tool, it saved hours and hours of my life!
Question though - how do you get the error variance from just the standardized regressions table? I somehow thought you were supposed to copy the variances table...?
Great question. This tool estimates the error, but is close enough to be fine. In my AMOS plugin (which I haven't polished or published), I use the error directly. I need to finish that plugin and publish it...
Simply amazing!
Fantastic tool! Thank you!
hi James and thanks for this amazing tool.
@jalparee1
Low AVE indicates low loadings. Ideally, the loadings should average above .7.
Low CR indicates poor reliability. This is also due to low loadings.
High MSV and ASV is due to correlations between latent factors being too high. Fix this by combining factors that correlate too highly (if this is in line with your theory). You may be able to create a 2nd order construct.
Most of these can also be fixed by improving model fit. See my "Model fit during CFA" video.
Hi James, thanks for all the great stuff you are providing at youtube. I just came up with an idea, that might help a lot of researches. It would be great if there would be an AMOS Plugin which gives the HTMT criterion and Fornell-Larcker criterion to assess discriminant validity.
thanks for the idea! I've added it to my list of plugins.
Thanks for uploading this video. Could you advise what range of factor correlation would be considered optimal to provide the evidence of convergent and discriminant validity? Thanks very much in advance.
This is very odd. I have never heard of this problem before. My recommendation is to either download a new version of the workbook from my wiki, or to email me directly to resolve the issue.
Thank you so much for sharing your great tool! Its so useful :)
This is a measurement error. Sometimes this happens if you have only two indicators on a single factor. You will get reliability of 1.00 if there is only one indicator. More likely the situation is that you are mistaking C.R. in the regression weights table in AMOS. This is the Critical Ratio (or z-score) which should be greater than 1.96 for significant values at the 0.05 level. This is not composite reliability.
Hi James, I have a sample size of 170 and given this size items with factor loading greater than 0.45 is acceptable according to statwiki. I was able to obtain very good model fit, but when I calculated the AVE value, it is only 0.35, far below 0.5 to pass convergent validity. This is rather confusing because with a large sample size it is difficult to obtain high factor loading (which is why statwiki accepts greater than 0.45), therefore it is also hard to obtain an AVE value above 0.5. So does it mean that I have a convergent validity problem despite the fact that my model fit is great? Many thanks.
+eric chen Correct. While the lower factor loading is acceptable for the EFA, the CFA will still require a strong loading for convergent validity. You might be able to still get away with it if your composite reliability score is greater than 0.700 Malhotra and Dash 2008 and 2011 say that you can get away with a bad AVE if you have a good CR.
+James Gaskin Hi, Can pls Pdf version of this book Dash 2008 and 2011?
+sadaf khan I can't, but here is the reference:
Maholtra, N. K., and Dash, S. Marketing Research: An Applied Orientation, 6th Ed. Pearson Education, 2010.
how can we improve the value of CR? as it is below 0.70? i am waiting for your response .THANKS in advance
Here is a video for this exactly: ruclips.net/video/xVl6Fg2A9GA/видео.html
دمت گرم.
It's cool!
I include all latent constructs that are not control variables.
You are amazing. thank you
(Ok... potentially stupid question) Here's the scenario. I got a very complex model integrating four theories together (e.g. Theory 1: A and B affects C; Theory 2: D affects B and C; Theory 3: E and F affects C, Theory 4: G affects B and C). Now, I am thinking of doing five SEM tests: one for each theory and the fifth is for the overall integrated model. Does this mean that I have to do five CFA tests as well (one for each theory and then the fifth for the overall model)?
Thank you!
Bless You! Thank you for creating the most valuable resource on RUclips!! Do you have a Patreon?
haha! You're kind. I don't have a Patreon account. This is just a service I do for the community so you guys don't have to suffer as I suffered to figure it out. I'm happy to do it. Thanks!
Many thanks, James!!!
You will have to look at an EFA with only the two constructs that are overlapping. Then see which items are crossloading the strongest. Perhaps delete those items one at a time and try again.
If you're late to the party like I am and you haven't had your coffee yet. You're looking for the Excel Stattools file on the sidebar of the homepage.
James, one more question for you. On your statwiki page, you show both AVE>MSV and square root of AVE > inter construct correlations as ways to test for discriminant validity. Aren't these essentially saying the same thing? What am I missing?
roughly, yes. The most common test is the latter one.
Dear Gaskin, thank you for sharing this. I am a PhD student in Tokyo University, and I wonder if it is possible for you to share this new excel tool with us in Tokyo. Thanks.
Hi James, I am trying to test a model with only 2 latent variables which goes against caveat 4 :-( does this mean I am unable to do a validity check? PS your tools and youtube tutorials are awesome!!!A big THANKYOU!
+Belinda Flannery You can use my old tool for this. Just email me and I'll send it: james.gaskin@byu.edu
Dear James
Thanks a lot for helping us. We owe you a lot.
I have a small doubt. AMOS and SPSS show starkly different correlations between any two variables. SPSS bivariate correlation is usually quite low as compared to correlation that we get in measurement model incorporating those two variables. Is this common or you think i am doing some mistake? Also, if this is common, then which ones should I report? Thanks
These would be different because your bivariate correlations in spss are using composite or averaged variables (I assume); whereas the correlations in AMOS are using latent factors from your measurement model.