Effect size of Pairwise Comparisons using emmeans

A short video on generating effect size statistics to complement pairwise comparisons results from emmeans() in RStudio.
Script used in the video can be downloaded from: github.com/sea....

[@ehuntley]

This was fantastic, and an excellent extension to cover the gaps not clearly explained or spelled out in the emmeans vignettes. Thank you for putting this together!

[@lukasdiedrich2671]

Great! Exactly what i was looking for! Thank you very much!

[@alyssawang144]

Thank you soooooo much for sharing this.

[@larissacury7714]

Thank you! What about the warning about the interaction? What can (or should) we do about it?

[@LaceyMathsStatsConsultancy]

That warning is just informing the user that there are interaction effects. Typically, the interaction effects are your primary outcome, with the main effects being your secondary outcome.
The warning is not an error but just a form of signposting in case the user is not aware of interaction effect(s) being present. The effect sizes from the main effect are still accurate.

[@larissacury7714]

@@LaceyMathsStatsConsultancy so when I'm performing emmeans(model,~ B|C) %>% pairs(adjust="Tukey"), am I accouting for that? I don't get the warning (being the model: Y ~ B * C + (1|ID))

[@LaceyMathsStatsConsultancy]

@@larissacury7714 Yes, as the script you are using is outputting the estimated marginal means for the interaction effect.

[@larissacury7714]

@@LaceyMathsStatsConsultancy thank you!

[@Fly_Jorge]

Why do the emmeans standard error values look the same from the emmeans output but they arent similar when looking at the effect size?

[@LaceyMathsStatsConsultancy]

The output that complements the effect size are the 95% CI limits, which are different to the standard error values. The CI is the mean +- margin of error. When the margin of error is the standard error divided by sqrt(n).

[@larissacury7714]

Would you go over why why 'eta' or the 'partial eta' is the aproppriate effect size estimate here? Which estimate is eff_size actually calculating for the pairwise comparisons?

[@LaceyMathsStatsConsultancy]

eff_size outputs partial eta-squared. There are some nice points, with references to support, on eta-squared and partial eta-squared, and when to use on: stats.stackexchange.com/questions/15958/how-to-interpret-and-report-eta-squared-partial-eta-squared-in-statistically.

[@larissacury7714]

@@LaceyMathsStatsConsultancy ​ thanks! great post, indeed! I've just opened a post on CV asking about that. My case is very similar to yours (youtube won't let me paste my CV link here). I have a mod1

[@larissacury7714]

so are the partial n² suitable in this case?

[@LaceyMathsStatsConsultancy]

@Larissa Cury Partial eta-squared is what I'd use in the scenario you've outlined, clearly outlining it's classification in the Methods section of a report.

[@larissacury7714]

@@LaceyMathsStatsConsultancy thank you! what do you mean by cleary outlining its classification?

[@larissacury7714]

Thanks, do you have any tips on how to report these results?

[@LaceyMathsStatsConsultancy]

The last few lines of script give an example of how to report the effect size statistics with corresponding p-values, em means and CIs. All that would be left is to write the results to Excel and then you are good to go.