So if I understood it correctly, if you are doing pixel wise permutation testing, suppose for pixel (0,0). You have an experiment with two conditions with 5 trials each. For you extract the pixel (0,0) for each of the trial. And then reassign, the conditions for the extracted pixels and construct the empirical distribution from that? And later compare against the actual pixel? Is that right? What about the last method, it wasn't clear. Thank you. Big fan of the course, going to get the Udemy subscription
Hi Mike, thanks for taking the time to make these educational videos, very helpful. I have two questions, first, if I want to compare two conditions (power spectrum) from single channel data do I still need to use cluster or pixel based correction? second, was there any reason that you did not use power db or normalized data to the pre-stimulus condition for permutation testing? if not can I use? thanks so much
in summary, 1. test values --> gaussian; then z-score; then p-value. 2. count; then what about the two-tail test, it's already 0-centered with the middle using this count strategy I suppose
For a two-tailed test, then you simply consider each tail separately and 1/2 the p-value threshold on each side. That goes for the z-score method and the count method. Just be mindful that some non-parametric statistics have only one tail (e.g., cluster size has only a positive tail to consider).
Hey, Mike. Thank you for your video😃. I learned a lot. I still have some questions. Question 1: When we compare the ERP difference between two conditions, what is the lowest level of data put into the permutation test? Trial or subject? Do we need to average all trials of one subject before the permutation test? Question 2: Another question about the further statistics, if we get the significant condition difference in FCz is 489-674ms, Cz is 382-552ms, Pz is 234-499ms, we want to conduct ANOVA: group (treatment, control) * condition (A, B) * electrodes (FCz, Cz, Pz), how could we define the significant time interval of each electrode because they have unique significant time interval? And permutation test only helps us detect the difference between different conditions, how could we conduct ANOVA in the permutation test?
You mean to calculate an empirical bootstrapped confidence interval around an average of Z values? Yes, that's definitely possible, although it's a different technique from permutation-testing.
@@mikexcohen1 Thank you for reply. Yes; that is what I meant . Just thinking of an alternative way to estimate the true value of Z. Thank you for an excellent video. I look forward to watching the rest of your videos.
Thank you Mike, I hav a question about making figures of within subject permutation. If you have 20 subject and you finish all the within subject permutation test. How will you demonstrate your results on a paper? Will you represent all of the 20 result maps one by one? Thank you!
If you are doing individual-based statistics, then that could be informative. But for N=20, it might be better to go for group-level analyses instead of single-subject analyses.
@@mikexcohen1 Thank you very much, that's a very inspiring answer. If I understand correctly, I have n=4, it is also inappropriate to show the averaged within-subject data across each subject on paper. ex image(mean(cat(3, map_result1 to map_result4),3)); I better demonstrate them one by one? Thank you.
I wouldn't say it's inappropriate to average over 4 subjects, just that the statistics wouldn't work out. But in general, I'm a big fan of showing a lot of data and embracing (rather than ignoring) individual differences. For N=100, it's not really feasible to show all the maps, but for N=4 yeah definitely show them all :)
So if I understood it correctly, if you are doing pixel wise permutation testing, suppose for pixel (0,0). You have an experiment with two conditions with 5 trials each. For you extract the pixel (0,0) for each of the trial. And then reassign, the conditions for the extracted pixels and construct the empirical distribution from that? And later compare against the actual pixel? Is that right? What about the last method, it wasn't clear. Thank you. Big fan of the course, going to get the Udemy subscription
Hi Mike, thanks for taking the time to make these educational videos, very helpful. I have two questions, first, if I want to compare two conditions (power spectrum) from single channel data do I still need to use cluster or pixel based correction? second, was there any reason that you did not use power db or normalized data to the pre-stimulus condition for permutation testing? if not can I use? thanks so much
in summary, 1. test values --> gaussian; then z-score; then p-value. 2. count; then what about the two-tail test, it's already 0-centered with the middle using this count strategy I suppose
For a two-tailed test, then you simply consider each tail separately and 1/2 the p-value threshold on each side. That goes for the z-score method and the count method. Just be mindful that some non-parametric statistics have only one tail (e.g., cluster size has only a positive tail to consider).
Hey, Mike. Thank you for your video😃. I learned a lot. I still have some questions.
Question 1: When we compare the ERP difference between two conditions, what is the lowest level of data put into the permutation test? Trial or subject? Do we need to average all trials of one subject before the permutation test?
Question 2: Another question about the further statistics, if we get the significant condition difference in FCz is 489-674ms, Cz is 382-552ms, Pz is 234-499ms, we want to conduct ANOVA: group (treatment, control) * condition (A, B) * electrodes (FCz, Cz, Pz), how could we define the significant time interval of each electrode because they have unique significant time interval? And permutation test only helps us detect the difference between different conditions, how could we conduct ANOVA in the permutation test?
Could you boostrap the central tendency for the collection of Z values? I think that could be useful to get a confidence interval on the Z value
You mean to calculate an empirical bootstrapped confidence interval around an average of Z values? Yes, that's definitely possible, although it's a different technique from permutation-testing.
@@mikexcohen1
Thank you for reply.
Yes; that is what I meant .
Just thinking of an alternative way to estimate the true value of Z.
Thank you for an excellent video.
I look forward to watching the rest of your videos.
Great explanation 👌
Fantastic explanation, clear, concise and intuitive, thank you!
Glad it was helpful!
Thank you Mike, I hav a question about making figures of within subject permutation. If you have 20 subject and you finish all the within subject permutation test.
How will you demonstrate your results on a paper? Will you represent all of the 20 result maps one by one? Thank you!
If you are doing individual-based statistics, then that could be informative. But for N=20, it might be better to go for group-level analyses instead of single-subject analyses.
@@mikexcohen1 Thank you very much, that's a very inspiring answer. If I understand correctly, I have n=4, it is also inappropriate to show the averaged within-subject data across each subject on paper. ex image(mean(cat(3, map_result1 to map_result4),3));
I better demonstrate them one by one?
Thank you.
I wouldn't say it's inappropriate to average over 4 subjects, just that the statistics wouldn't work out.
But in general, I'm a big fan of showing a lot of data and embracing (rather than ignoring) individual differences. For N=100, it's not really feasible to show all the maps, but for N=4 yeah definitely show them all :)