Thank you so much .. I wasted about half an hour looking at the chi-square distribution without the Bollen -Stine checked, obviously my chi square was not falling within the range, now I know .. your video helped.
Hi Joel- thanks for the informative video! Question- is this bootstrapping required for the final CFA model and final SEM model? or is it just for the final SEM model? (currently writing a master's thesis and it's greatly appreciated) :)
@@joelcollier9387 Thank you Joel! Final question- the p value for the Bollen-Stine bootstrap is non-significant (0.642), but it states that the model fit better in 1788 sample, and worse in 3212 samples. Can it still be claimed that the model fit is adequate with the bootstrap samples?
Thank you for the video, what time is the best to do the bootstrapping, my model is not fitting so I am working on my modification indices, I have reach a point were I dont think I can improve more my fitting indicators with the MI. Should I transform non-normal variables to (near) normality before running the model?
My best guess is if you are having model fit issues...you most likely have measurement issues with your latent constructs. You might want to find the troublesome items and see if it warrants deletion if it is not contributing to the validity of the construct.
@@joelcollier9387 thank you for your response, I will look at my latent constructs. My data does not follow a normal distribution, some literature suggests to transform the non-normal variables before uploading them to Amos, if necessary (Niels.J , 2013)., would this approach be far different from bootstrapping?
Dear Joel, thank you for this informative video. In the video you state that if the Bollen-Stine test is significant then you probably already have model fit issues. I'm testing a LPA model, my data is not multivariate normal the fit indices suggest excellent model fit but Bollen-Stine = 0.000. I'm not sure if I should try to respecify the model or just report the contradictory results and the implication of doing the latter. Please help.
If you are looking for clarity on why the differences occur, here is a nice quote from Arbuckle: "The Bollen-Stine bootstrap process involves the transformation of the data to a data set for which the null hypothesis (that the default model fits the data) is true. This requires the use of the covariance matrix to suitably transform the data from which the bootstrap samples are drawn. Each bootstrap sample is sampled from this transformed data and a chi-square is computed for the fit of that bootstrapped data to the model. These chi-squares are not printed, but they are compared internally to the chi-square that was computed for the observed data fit to the model (which is printed in the ""Notes for Model"" section of the output). The proportion of times that the model 'fit worse or failed to fit', i.e. the number of times that the model chi-square for the bootstrapped sample exceeded the chi-square for the observed data, is the Bollen-Stine bootstrap p value.". If your sample is very large it might be impossible to find nonsignificance. I don't know that respecifying the model will always help that.
Hello dear Collier. First of all, thanks for the very important information. My Bollen-Stine Bootstrap analysis result is as follows. What exactly does this mean? My p value is unfortunately significant (p
The Bollen Stine Estimate tries to estimate model fit across all the bootstrap samples. If you are getting a significant bootstrap estimate it means you have a problematic model fit. One caveat with this is the Bollen and Stine Estimate only looks at Chi-square values which we know can be problematic with a large sample size. It is always advisable to assess model fit with goodness of fit (CFI, IFI) along with badness of fit estimates (RMSEA) to assess model fit. If you are having problem with model fit, it could be a number of issues. Try to look at the measurement validity of your items. If that does not work look at the modification indices to see if something jumps out.
@@fatihcelik7537 You would have to reduce it a lot to have a big enough impact based on that p-value listed. Not sure that would be feasible. You might be better off justifying the model with the other fit indices
This very informative. Thankyou so much Sir. I have got a question. Is multivariate normality an assumption of ML estimation method only as some Authors have quoted. Second, Can we use a different estimation method (like ULS ), along with bootstrapping when multivariate normality is not met. Please respond.
When you are bootstrapping the data with 5K samples, you are going to normalize the data. Your data may be non-normal (skewed or kurtosis) but through the bootstrapping process the shear number of samples will start to normalize the data. Saying that, if you want to use ULS you are more than welcome to do so and it will not adversely influence the bootstrapping process. I don't know that you are going to see a huge difference between ML bootstrapping and ULS
Hello! Thank you for this video. It is more than helpful! I got one question. When I use bootstrapping, I get the error message that I can't perform bootstrapping with missing data. So I turn off "estimate means" in the estimation tap in the Analysis properties. But then I get the error message that I must turn it on. Do I have to calculate the means in spss and then put it somewhere into Amos? I would appreciate your help if possible :)
Yes, the best way to handle this is to impute the missing data in SPSS before you bring the data into AMOS. You can impute the series mean or do a linear interpolation (preferred method). If you need step by step instructions on how to do that, my book (chapter 2) goes over how to do this in detail.
Thank you for this video, really helpful. I just have a couple of questions if you're able to help please? I am looking to run CFA on 5 different models reported in the literature with my sample of n=485 (substantial multivariate kurtosis - 3 cases in particular show high Mahalanobis distance values). Q1) Would I run the Bollen Stine bootstrapping procedure for all 5 models or assess which one is the best fit first using other fit indices first, and then just apply bootstrapping to that one model? and Q2) Is it worth testing the models excluding those 3 cases, and if so, do I do that instead of, or in conjunction with, bootstrapping (or does it just depend on the impact of removing those 3 cases)? Many thanks for your help :-)
I am always in the mindset if you have responses with a very high Mahalanbois distance to drop those respondents and then run the analysis again. I like to "clean" the data before even analyzing it. It should help with model fit was well with the outliers removed. As for the 5 different models, I think I would find the best one from a fit perspective and also which one makes the most justification from a theory perspective as well. After settling on one, then run the Bollen and Stine test. Hope that helps.
Hi it's great video! thank you so much. Just one question, when all, but RMSEA, model fit indices fall under the acceptable threshold, and when non-normality is present (skewness and Kurtosis both around +-4 to 6), can I still conclude that my specified model show a good fit because Bollen-Stine bootstrap test did not reject "a null hypothesis that a model is correct"? If yes, is there any good reference I can read through? Thanks in advance.
I don't have a specific reference for your particular results. I can say that if your fit indices are acceptable outside of RMSEA this should not keep you from moving forward. RMSEA is only one of multiple tests to assess model fit. If +4 to 6 are values for kurtosis, that is an acceptable range. That is a high value for skew. If you ran a bootstrap and the Bollen-stine was not significant, then you can say that you have account for the non-normality of the data. Thus, through the bootstrap of 5000 samples the data is converging to a normal distribution. Hope this helps.
Hello Mr. Collier, thank you so much for your helpful videos. I'm a little confused about the model fit with bootstrapping. Do I need to declare both model fit values (7:18) and the Bollen-Stine value (9:11) when I use bootstrapping? Or do the model fit values become irrelevant due to the p value of Bollen-Stine?
@@sergengunesdogan9254 I wish I had a good example but none really come to mind. I usually present the model fit estimates in a table and then on a separate line present the bollen stine bootstrap test.
Hey thanks for ur help.. I hv one more query about process that unlike amos it only take one x variable not more. What if i need to see many parameters direct and indirect effect on the one variable.? Can i use it in this case?
If your sample is small, then you probably do not want to use AMOS. Any sample value under 200 starts to have unstable parameter estimates. If your sample is small, then you probably want to assess normality through SPSS or SAS. Hope that helps.
@@bhavnajaiswal3816 Typically small sample sizes are a problem in AMOS even in path analysis. If the sample is small, you may be better off using the Andrew Hayes Macros PROCESS
@@bhavnajaiswal3816 Search in Google Andrew Haye's Macro. His website will give you instructions on how to install it in SPSS or SAS. There are few youtube videos that show how to install it as well.
Thank you so much .. I wasted about half an hour looking at the chi-square distribution without the Bollen -Stine checked, obviously my chi square was not falling within the range, now I know .. your video helped.
Hi Joel- thanks for the informative video! Question- is this bootstrapping required for the final CFA model and final SEM model? or is it just for the final SEM model? (currently writing a master's thesis and it's greatly appreciated) :)
You only need to run the bootstrap for the structural model. The CFA should be fine without the bootstrapping. Best of luck with the thesis.
@@joelcollier9387 Thank you Joel! Final question- the p value for the Bollen-Stine bootstrap is non-significant (0.642), but it states that the model fit better in 1788 sample, and worse in 3212 samples. Can it still be claimed that the model fit is adequate with the bootstrap samples?
Thank you for the video, what time is the best to do the bootstrapping, my model is not fitting so I am working on my modification indices, I have reach a point were I dont think I can improve more my fitting indicators with the MI. Should I transform non-normal variables to (near) normality before running the model?
My best guess is if you are having model fit issues...you most likely have measurement issues with your latent constructs. You might want to find the troublesome items and see if it warrants deletion if it is not contributing to the validity of the construct.
@@joelcollier9387 thank you for your response, I will look at my latent constructs. My data does not follow a normal distribution, some literature suggests to transform the non-normal variables before uploading them to Amos, if necessary (Niels.J , 2013)., would this approach be far different from bootstrapping?
Dear Joel, thank you for this informative video. In the video you state that if the Bollen-Stine test is significant then you probably already have model fit issues. I'm testing a LPA model, my data is not multivariate normal the fit indices suggest excellent model fit but Bollen-Stine = 0.000. I'm not sure if I should try to respecify the model or just report the contradictory results and the implication of doing the latter. Please help.
If you are looking for clarity on why the differences occur, here is a nice quote from Arbuckle: "The Bollen-Stine bootstrap process involves the transformation of the data to a data set for which the null hypothesis (that the default model fits the data) is true. This requires the use of the covariance matrix to suitably transform the data from which the bootstrap samples are drawn. Each bootstrap sample is sampled from this transformed data and a chi-square is computed for the fit of that bootstrapped data to the model. These chi-squares are not printed, but they are compared internally to the chi-square that was computed for the observed data fit to the model (which is printed in the ""Notes for Model"" section of the output). The proportion of times that the model 'fit worse or failed to fit', i.e. the number of times that the model chi-square for the bootstrapped sample exceeded the chi-square for the observed data, is the Bollen-Stine bootstrap p value.". If your sample is very large it might be impossible to find nonsignificance. I don't know that respecifying the model will always help that.
Hello dear Collier.
First of all, thanks for the very important information. My Bollen-Stine Bootstrap analysis result is as follows. What exactly does this mean? My p value is unfortunately significant (p
The Bollen Stine Estimate tries to estimate model fit across all the bootstrap samples. If you are getting a significant bootstrap estimate it means you have a problematic model fit. One caveat with this is the Bollen and Stine Estimate only looks at Chi-square values which we know can be problematic with a large sample size. It is always advisable to assess model fit with goodness of fit (CFI, IFI) along with badness of fit estimates (RMSEA) to assess model fit. If you are having problem with model fit, it could be a number of issues. Try to look at the measurement validity of your items. If that does not work look at the modification indices to see if something jumps out.
@@joelcollier9387 Could reducing the sample size be one of these suggestions?
@@fatihcelik7537 You would have to reduce it a lot to have a big enough impact based on that p-value listed. Not sure that would be feasible. You might be better off justifying the model with the other fit indices
This very informative. Thankyou so much Sir. I have got a question. Is multivariate normality an assumption of ML estimation method only as some Authors have quoted. Second, Can we use a different estimation method (like ULS ), along with bootstrapping when multivariate normality is not met. Please respond.
When you are bootstrapping the data with 5K samples, you are going to normalize the data. Your data may be non-normal (skewed or kurtosis) but through the bootstrapping process the shear number of samples will start to normalize the data. Saying that, if you want to use ULS you are more than welcome to do so and it will not adversely influence the bootstrapping process. I don't know that you are going to see a huge difference between ML bootstrapping and ULS
@@joelcollier9387 Thankyou so much sir. This helped me for sure. Thankyou once again
Hello! Thank you for this video. It is more than helpful! I got one question. When I use bootstrapping, I get the error message that I can't perform bootstrapping with missing data.
So I turn off "estimate means" in the estimation tap in the Analysis properties. But then I get the error message that I must turn it on.
Do I have to calculate the means in spss and then put it somewhere into Amos? I would appreciate your help if possible :)
Yes, the best way to handle this is to impute the missing data in SPSS before you bring the data into AMOS. You can impute the series mean or do a linear interpolation (preferred method). If you need step by step instructions on how to do that, my book (chapter 2) goes over how to do this in detail.
Thank you for this video, really helpful. I just have a couple of questions if you're able to help please? I am looking to run CFA on 5 different models reported in the literature with my sample of n=485 (substantial multivariate kurtosis - 3 cases in particular show high Mahalanobis distance values). Q1) Would I run the Bollen Stine bootstrapping procedure for all 5 models or assess which one is the best fit first using other fit indices first, and then just apply bootstrapping to that one model? and Q2) Is it worth testing the models excluding those 3 cases, and if so, do I do that instead of, or in conjunction with, bootstrapping (or does it just depend on the impact of removing those 3 cases)? Many thanks for your help :-)
I am always in the mindset if you have responses with a very high Mahalanbois distance to drop those respondents and then run the analysis again. I like to "clean" the data before even analyzing it. It should help with model fit was well with the outliers removed. As for the 5 different models, I think I would find the best one from a fit perspective and also which one makes the most justification from a theory perspective as well. After settling on one, then run the Bollen and Stine test. Hope that helps.
@@joelcollier9387 thanks so much!
Hi it's great video! thank you so much.
Just one question, when all, but RMSEA, model fit indices fall under the acceptable threshold, and when non-normality is present (skewness and Kurtosis both around +-4 to 6), can I still conclude that my specified model show a good fit because Bollen-Stine bootstrap test did not reject "a null hypothesis that a model is correct"?
If yes, is there any good reference I can read through?
Thanks in advance.
I don't have a specific reference for your particular results. I can say that if your fit indices are acceptable outside of RMSEA this should not keep you from moving forward. RMSEA is only one of multiple tests to assess model fit. If +4 to 6 are values for kurtosis, that is an acceptable range. That is a high value for skew. If you ran a bootstrap and the Bollen-stine was not significant, then you can say that you have account for the non-normality of the data. Thus, through the bootstrap of 5000 samples the data is converging to a normal distribution. Hope this helps.
@@joelcollier9387 Thank you so much for your reply!
I will buy your book! :D
Hello Mr. Collier, thank you so much for your helpful videos. I'm a little confused about the model fit with bootstrapping. Do I need to declare both model fit values (7:18) and the Bollen-Stine value (9:11) when I use bootstrapping? Or do the model fit values become irrelevant due to the p value of Bollen-Stine?
So I present both. The initial model fit statistics along with the Bollen-Stine results. It provides the most justification that way.
Thank you. Do you have a paper I can refer to when I present both of them?
@@sergengunesdogan9254 I wish I had a good example but none really come to mind. I usually present the model fit estimates in a table and then on a separate line present the bollen stine bootstrap test.
Hey thanks for ur help..
I hv one more query about process that unlike amos it only take one x variable not more. What if i need to see many parameters direct and indirect effect on the one variable.? Can i use it in this case?
If you have one IV, you can test multiple mediating variables and subsequent indirect effects in PROCESS.
@@joelcollier9387 thank you
What to do when sample size is small?
If your sample is small, then you probably do not want to use AMOS. Any sample value under 200 starts to have unstable parameter estimates. If your sample is small, then you probably want to assess normality through SPSS or SAS. Hope that helps.
@@joelcollier9387 but after normalization can i rely on path analysis for small sample size
@@bhavnajaiswal3816 Typically small sample sizes are a problem in AMOS even in path analysis. If the sample is small, you may be better off using the Andrew Hayes Macros PROCESS
@@joelcollier9387 where i can get it?
@@bhavnajaiswal3816 Search in Google Andrew Haye's Macro. His website will give you instructions on how to install it in SPSS or SAS. There are few youtube videos that show how to install it as well.