Dr. Buchanan, you rock. I work for a medical center as a business analyst and am in charge of delivering data-driven recommendations and reports. I am constantly refining metrics and running sanity checks against data. Time and time again I find myself watching one of your videos to refresh statistical concepts I have long forgotten or to learn entirely new ones. Thank you so much for putting your wonderful lectures online.
Hey Dr Buchanan, this has been instrumental in my understanding of IRT: thank you so much for explaining and making all the materials publicly available! I wish all of academia would follow your example!
These are amazing! I can't find any lectures on multidimensional IRT though and i have been searching for them all over youtube. I thought this would finally help me modelling multidimensional constructs but this is also unidimensional
@@StatisticsofDOOM I have heard that when I have a multidimensional construct, i can do IRT on each sub construct. However i am not sure how can i find an 'overall score'. I will have ability estimates for each constructs but how do i make a total score out of these ability estimates? My guess is to use these ability estimates as score for factor analysis and then find a factor score . Am i right in my thinking?
@@Dupamine AH, for that you might consider mirt ... I'm not sure how to estimate it from each one separately and have that be accurate, rather I think you'd probably have to combine them to get the higher order value.
Thanks for sharing, it's great to see the `mirt` package being used in classroom settings. The only thing that I think needs correcting is the interpretation of the Expected Total Score plot. This actually has little to do with the Fisher information, and instead is simply what the expected total score is on the test given the underlying theta value. So someone with a theta=0 might be expected to only get around 25 on the test (more or less). Also, if you are interested in the factor scores for each unique response pattern then you can pass `full.scores = FALSE` to the function. Cheers.
No problem :). If your interested, there's also a Github wiki associated with the package with lots of exercises and examples (github.com/philchalmers/mirt/wiki). Feel free to cherry pick from them in your class if you see something you like. Cheers.
I don't think the person fit explanation is correct. For ltm the person fit does not give the ability estimates for each student or unique response vector, these are goodness of fit measures. The ability scores for the students can be found by doing this: LSAT_factor_scores
Yep! Just did it - I am slowly updating the website, links, and everything. Here's the current location: statisticsofdoom.com/page/structural-equation-modeling/ - the files are linked on that page on github and soon the viewing order will be available too.
I have faced problem when i was trying to load the ltm package and tried the solution that you gave in the video. gfortran was successfully downloaded but I got an error when I tried the following command : command I used: sudo tar fvxz gfortran-4.8.2-drawin13.tar.bz2 -C / Error appeared : tar: Error opening archive: Failed to open 'gfortran-4.8.2-drawin13.tar.bz2'
@@StatisticsofDOOM Dear Dr. Buchanan Thanks for your feedback! I have some unknown things. Can you help me? 1. Is the value of column z1 in this video the ability of candidates? 2. When using factor.scores (..) command, the result shows 30 lines corresponding to the ability of 30 candidates. I want to use the abiltity of 1000 candidates, what will I do? Thank you so much!
@@StatisticsofDOOM I mean I get 1000 responses but when using factor.scores () the result is only 30 lines. I want to display the theta value of all responses.
Would you say that if you have a total test information function which measures- say - bellow average then if you plot the distribution of participant scores, theta values would be seriously skewed? When I use mirt I also check CFI TLI RMSEA. I noticed that you also check discriminability the a's good thing to do. I prefer to check the ICC. The trace lines I was not familiar, very nice to mention them here.
I hadn't seen a lot with people using traditional SEM fit indices for IRT, but I don't know why you couldn't to look at fit. ICC is also great. I would assume that if I had a TIF showing me that the information was best measured at less than average thetas, then the data is probably also skewed that direction.
Hey! Thanks for your helpful lessons! I have one question and sorry when you explaned it and didn't get it. What does the "a" value actually mean? Is it the item difficulty?
B is the difficulty parameter, while A is the discrimination parameter. So, B measures who is likely to get the question right (so B = 0 is that an average person will get it right or the in the case of poly where you might expect the person to answer that answer - low B values indicate a person is getting a low score and picking that response option). A is the slope or how well it discriminates between people below and above that B point. So A = 0 means the item is really bad and can't tell the difference between people who answer the question and higher A values means it's really good at telling you who got it right or wrong (or who is likely to pick that response option).
Please, I need help. How to get an individual score for each person using 3PL model? I'm just web programmer. I need to run R script that can calculate person score using IRT with 3PL model on exam result dataset. Really appreciate your help.
@@StatisticsofDOOM , sorry, what I meant was raw score. After following the example in this video, I have managed to get the z1 value using factor.scores. Now, I need a raw score of the z1 value. So, how do you get a raw score? Thank you.
@@IfanIqbal Ah. Usually you are interested in theta, but I would think you could easily convert back to whatever scale you wanted using the traditional z score formula (Z = (X - mu)/sigma). You would have to know what you expected mu (population mean) and sigma (population standard deviation) to be.
@@StatisticsofDOOM Thank you, your response is very helpful to me. I have succeeded in generating theta from the fscores function, using a mirt package with item type 2PL, then converting that value to raw score (X = mu + (Z * sigma)), with mean and standard deviation calculated from the normal score population of each participant .
Hi, I was wondering if you can help with an IRT-related question. I am looking for ways to get an IRT-based score for each item for each individual (e.g., an irt-weighted score of question 1, 2, 3,... for examinee A, B, C...). I know we can get examinee's overall proficiency from the person fit in the mirt package but I need something analogous to item scores. Would you let me know if you know an R or any other software package that provides IRT-based "item" scores for each individual with polytomous data? Many many thanks in advance!!
The lecture video covers the differences a bit better. I do think that the GRM can be harder to fit if your items are not ordered or at each point a highest threshold, so the GPCM tends to work better. They can be equal if the items behave well.
@@StatisticsofDOOM Thank you so much for getting back to me. Can I just ask what you mean 'at each point a highest threshold'? If the scale is a likert scale where all options are not being used but it is not theoretically sound to collapse the scale, then the GPCM would be the better model?
@@StatisticsofDOOM Thank you so much. I listened again to the lecture which gave the perfect explanation of the differences between GRM and GPCM. It was exactly what I was looking for. Your lectures and examples have been of so much help. Thank you again.
Hello!, thank you so much for the videos! i have a question, how do you weight the item score after you analyze the data using 2PL IRT. you said that every item have different qualities, so, if i have an item that have better A scores and have higher difficuty than the rest of the items in the test. should i give that item more weight than the rest of the items? if i should, how can it be done? pardon my english. cheers :)
You could create a vector of the loadings and multiply it by the dataframe with the answers - if you google matrix multiplication r, you can find a link for the practical r wordpress blog that covers this topic.
Many likes many likes many likes - youtube allows me to put only one like :( I am trying to find how to get information from a 4 factor model and from a bifactor model 3 + 1 general factor and plot it. I know this is 4 dimensional space but I wonder............ This is how I ended up here again in your channel.
Like you want to plot the loadings and factors? I've recently discovered the FactoMineR package - don't know if it will do 4D space though: cran.r-project.org/web/packages/FactoMineR/index.html
![Item Response Theory: The Math Behind Standardized Tests [Lecture]](http://i.ytimg.com/vi/n4W4Gwwkbqg/mqdefault.jpg)
Dr. Buchanan, you rock. I work for a medical center as a business analyst and am in charge of delivering data-driven recommendations and reports. I am constantly refining metrics and running sanity checks against data. Time and time again I find myself watching one of your videos to refresh statistical concepts I have long forgotten or to learn entirely new ones. Thank you so much for putting your wonderful lectures online.
Thank you - I appreciate the feedback!
17:15 How awesome is hearing her reacting to numbers in such a phonetic and passionate way!
as Someone about to take Comprehensive Exams that include IRT, this is a GODSEND.
Thank you and best of luck!
Your IRT Lecture and Theory videos are amazing! Thank you so much for making such a complex topic accessible and easier to understand.
Thanks for the kind words!
Hey Dr Buchanan, this has been instrumental in my understanding of IRT: thank you so much for explaining and making all the materials publicly available! I wish all of academia would follow your example!
Thanks for the kind words!
Thank you very much, professor! I am learning IRT and R programming which is not that easy. Your videos and the website are really helping me!
Thanks for the kind words!
Thanks so much - this was so helpful as an R beginner :)
Glad it helped!
These are amazing! I can't find any lectures on multidimensional IRT though and i have been searching for them all over youtube. I thought this would finally help me modelling multidimensional constructs but this is also unidimensional
I haven't done much multi-IRT, but I highly recommend the mirt package and it's help guide!
@@StatisticsofDOOM I have heard that when I have a multidimensional construct, i can do IRT on each sub construct. However i am not sure how can i find an 'overall score'. I will have ability estimates for each constructs but how do i make a total score out of these ability estimates? My guess is to use these ability estimates as score for factor analysis and then find a factor score . Am i right in my thinking?
@@Dupamine AH, for that you might consider mirt ... I'm not sure how to estimate it from each one separately and have that be accurate, rather I think you'd probably have to combine them to get the higher order value.
Thanks for sharing, it's great to see the `mirt` package being used in classroom settings. The only thing that I think needs correcting is the interpretation of the Expected Total Score plot. This actually has little to do with the Fisher information, and instead is simply what the expected total score is on the test given the underlying theta value. So someone with a theta=0 might be expected to only get around 25 on the test (more or less).
Also, if you are interested in the factor scores for each unique response pattern then you can pass `full.scores = FALSE` to the function. Cheers.
No problem :). If your interested, there's also a Github wiki associated with the package with lots of exercises and examples (github.com/philchalmers/mirt/wiki). Feel free to cherry pick from them in your class if you see something you like. Cheers.
I don't think the person fit explanation is correct. For ltm the person fit does not give the ability estimates for each student or unique response vector, these are goodness of fit measures. The ability scores for the students can be found by doing this: LSAT_factor_scores
Excellent clarification - thanks! I need to update these videos, and will make this point clearer.
Hey, i loved this explanation! Awsome work!
Thank you for sharing this!!
Thank You very much. It is really helpful. Best Regards..
Very, very helpful, thanks for the great tutorial!
Statistics of DOOM could you refresh the very useful lecture material link above? The page at statstools.com says the page is not found!
Yep! Just did it - I am slowly updating the website, links, and everything. Here's the current location: statisticsofdoom.com/page/structural-equation-modeling/ - the files are linked on that page on github and soon the viewing order will be available too.
I have faced problem when i was trying to load the ltm package and tried the solution that you gave in the video. gfortran was successfully downloaded but I got an error when I tried the following command :
command I used:
sudo tar fvxz gfortran-4.8.2-drawin13.tar.bz2 -C /
Error appeared :
tar: Error opening archive: Failed to open 'gfortran-4.8.2-drawin13.tar.bz2'
Thank you, this was very helpful!
Dear Dr. Buchanan, could you please help me to show the results of the ability estimation of all candidates? Thank you so much!
I'm not sure I understand your question? Are you looking for an estimation of the person scores?
@@StatisticsofDOOM
Dear Dr. Buchanan
Thanks for your feedback!
I have some unknown things. Can you help me?
1. Is the value of column z1 in this video the ability of candidates?
2. When using factor.scores (..) command, the result shows 30 lines corresponding to the ability of 30 candidates. I want to use the abiltity of 1000 candidates, what will I do?
Thank you so much!
@@nguyenvancanh2653 I am not sure I can really answer this question - factor.scores only shows 30 but you have 1000 lines or? z1 would be theta yes.
@@StatisticsofDOOM I mean I get 1000 responses but when using factor.scores () the result is only 30 lines. I want to display the theta value of all responses.
@@nguyenvancanh2653 Apologies - my brain was not connecting the dots - try using personfit instead: rdrr.io/cran/mirt/man/personfit.html
Would you say that if you have a total test information function which measures- say - bellow average then if you plot the distribution of participant scores, theta values would be seriously skewed? When I use mirt I also check CFI TLI RMSEA. I noticed that you also check discriminability the a's good thing to do. I prefer to check the ICC. The trace lines I was not familiar, very nice to mention them here.
I hadn't seen a lot with people using traditional SEM fit indices for IRT, but I don't know why you couldn't to look at fit. ICC is also great.
I would assume that if I had a TIF showing me that the information was best measured at less than average thetas, then the data is probably also skewed that direction.
Hey! Thanks for your helpful lessons! I have one question and sorry when you explaned it and didn't get it. What does the "a" value actually mean? Is it the item difficulty?
B is the difficulty parameter, while A is the discrimination parameter. So, B measures who is likely to get the question right (so B = 0 is that an average person will get it right or the in the case of poly where you might expect the person to answer that answer - low B values indicate a person is getting a low score and picking that response option). A is the slope or how well it discriminates between people below and above that B point. So A = 0 means the item is really bad and can't tell the difference between people who answer the question and higher A values means it's really good at telling you who got it right or wrong (or who is likely to pick that response option).
Great tutorial I found it very very very useful. If it's not too much to ask, can you make similar one using TAM package.
Haven't use that package, but I can check it out!
Please, I need help. How to get an individual score for each person using 3PL model? I'm just web programmer. I need to run R script that can calculate person score using IRT with 3PL model on exam result dataset. Really appreciate your help.
Using R? I'd tell you to check out fscores: rdrr.io/cran/mirt/man/fscores.html
@@StatisticsofDOOM , sorry, what I meant was raw score. After following the example in this video, I have managed to get the z1 value using factor.scores. Now, I need a raw score of the z1 value. So, how do you get a raw score? Thank you.
@@IfanIqbal Ah. Usually you are interested in theta, but I would think you could easily convert back to whatever scale you wanted using the traditional z score formula (Z = (X - mu)/sigma). You would have to know what you expected mu (population mean) and sigma (population standard deviation) to be.
@@StatisticsofDOOM Thank you, your response is very helpful to me. I have succeeded in generating theta from the fscores function, using a mirt package with item type 2PL, then converting that value to raw score (X = mu + (Z * sigma)), with mean and standard deviation calculated from the normal score population of each participant .
Hi, I was wondering if you can help with an IRT-related question. I am looking for ways to get an IRT-based score for each item for each individual (e.g., an irt-weighted score of question 1, 2, 3,... for examinee A, B, C...). I know we can get examinee's overall proficiency from the person fit in the mirt package but I need something analogous to item scores. Would you let me know if you know an R or any other software package that provides IRT-based "item" scores for each individual with polytomous data? Many many thanks in advance!!
I would think you could multiply the weight of each item by the answer each person put to get that type of score? That's my best guess.
@@StatisticsofDOOM Thank you!
Can I ask whether you think the GPCM is better to use for scale development than the GRM? What are the fundamental differences between the two models?
The lecture video covers the differences a bit better. I do think that the GRM can be harder to fit if your items are not ordered or at each point a highest threshold, so the GPCM tends to work better. They can be equal if the items behave well.
@@StatisticsofDOOM Thank you so much for getting back to me. Can I just ask what you mean 'at each point a highest threshold'? If the scale is a likert scale where all options are not being used but it is not theoretically sound to collapse the scale, then the GPCM would be the better model?
@@louisekatz4310 i.e., where each choice has a peak of being the most likely answer.
@@StatisticsofDOOM Thank you so much. I listened again to the lecture which gave the perfect explanation of the differences between GRM and GPCM. It was exactly what I was looking for. Your lectures and examples have been of so much help. Thank you again.
Hello!, thank you so much for the videos!
i have a question, how do you weight the item score after you analyze the data using 2PL IRT. you said that every item have different qualities, so, if i have an item that have better A scores and have higher difficuty than the rest of the items in the test. should i give that item more weight than the rest of the items? if i should, how can it be done? pardon my english.
cheers :)
One way might be to use the loading weight times the person's score - I don't know that it helps much when items are scored zero and one though.
Wow, okaay. so, how can i calculate the loading using R?
You could create a vector of the loadings and multiply it by the dataframe with the answers - if you google matrix multiplication r, you can find a link for the practical r wordpress blog that covers this topic.
Many likes many likes many likes - youtube allows me to put only one like :( I am trying to find how to get information from a 4 factor model and from a bifactor model 3 + 1 general factor and plot it. I know this is 4 dimensional space but I wonder............ This is how I ended up here again in your channel.
Like you want to plot the loadings and factors? I've recently discovered the FactoMineR package - don't know if it will do 4D space though: cran.r-project.org/web/packages/FactoMineR/index.html