That classic NBME actually sounds oddly fun. It's not your thing, but a video filming someone attempt one of those old NBME exams would be awesome content.
I was a little concerned that this might make me less angry about the score ratcheting (err, "setting") process, but nope, it's exactly what I suspected. Giving them access to real performance data is so transparently outcome motivated. All they've done is get craftier about how they keep the fail rate high. Of course they still "supervise" the process to make sure these experts don't go out of bounds.
Hey! Loved the video. Small correction: when you give the example that a 700 in a 500 mean 100 SD exam is 95th percentile, you are forgetting that you should only account for one tail of the curve: a 700 is 97.5th percentile, and a 300 is a 2.5th percentile
If failures decrease, profits decrease. USMLE retakes and all the NBME practice exams are more money. Inaccurate predictions provided on the practice NBME exams like the CBSSA and CCSSA will increase confidence in weaker performers, they will take the test and fail. The guarantee of profit is padded through multiple measures. I would love to see a profit chart laid over the Mean/Minimum pass chart. Are there complementary troughs to the peaks you pointed out?
Agreed , nbme is a monopoly with a "non profit" disguise that takes advantage of medical students because , well there is no other way , how do you justify IMG exam fees , its almost double of what a US MD grad has to pay, the playing field is not equal and its near extorsion.
Very insightful video!! Please do share how we went from the comprehensive, multi-day style to what we have now. Thanks for the effort you put into your videos!
It would be interesting to also look at whether the difficulty of the questions have increased over time. Comparing older prep material with new feels like there's a noticable difference.
I agree with most of this except that they’re missing badly with their finger on the scale. State legislators granting autonomy and prescribing authority to naturopaths, PAs and now non-licensed IMGs suggests the policy winds favor individuals deciding for themselves who is qualified and who is not. Not that anyone really knows how to do that, but their trust in the free market is absolute.
44:46 You mean the decision was made in these years? If overall performance is improving every year, the nadir points should actually represent the years the minimum passing scores were increased? Sorry if I’m confusing.
Yes, the red arrows highlight a year when the decision was made to raise the passing score - so the difference between the average and the passing score drops a little bit immediately thereafter. What’s interesting to me the slope of each of those segments before the decision gets made. The steeper the slope, the more rapidly they had to step in and increase the passing standard.
You didn’t mention that a certain % of questions are “test” questions that the nbme puts in with the real exams to see how they perform before making them real scored questions for the future. How do “test” questions factor in? Wouldn’t they be ways for nbme to see how well they perform and throw out the bad ones before they are official?
You know, for a while, I’ve had it in my head that I might do a whole video on USMLE experimental items… but now I’m gonna officially add it to the list. Stay tuned.
How much do you estimate the USMLE cheating scandal affected the national mean score/average? Do continuously rising student averages indicate better and better students, easier and easier questions, or more and more unscrupulous activity over time due to mounted pressure?
The typical mean that the USMLE reports is for first-time test-takers from MD-granting medical schools in the United States. So far, all of the examinees whose scores have been invalidated have been IMGs - so they aren’t included in those figures. I actually have a whole video on what’s behind #scorecreep: A Brief Update on USMLE Score Creep ruclips.net/video/1gKKAZ5aO8E/видео.html
That classic NBME actually sounds oddly fun. It's not your thing, but a video filming someone attempt one of those old NBME exams would be awesome content.
Your brain explains things in a way that my brain likes.
Yes please make the video about old medical school practices and exams
That OG exam is insane.
Wake up, babe. New sheriff dropped
Its literally 4am here and im on it hahahaha
i'm up babe
I would love to see a video explaining the evolution of the exams. As always, you put out the highest quality content on the platform! Amazing!
cannot wait to watch this when i have a free hour. i LOVE your content (not even pre med or anything, but you are so engaging)
Awesome video as always. You can’t get this sort of analysis anywhere else. Love the part on the older exam content, it sounds so wack.
If that test was administered now people would fail it. I think you should cover how we went from such a comprehensive test to mcq test
I’m gonna add it to the list.
I was a little concerned that this might make me less angry about the score ratcheting (err, "setting") process, but nope, it's exactly what I suspected. Giving them access to real performance data is so transparently outcome motivated. All they've done is get craftier about how they keep the fail rate high. Of course they still "supervise" the process to make sure these experts don't go out of bounds.
Medical school professors like to get the “right” answer, too.
Yes please , make that video of how the medical licensing exam has changed
It’s going on the list.
wojak faces? you're speaking my language, sheriff.
I really felt that nothing else captured the essence of the minimally-competent examinee quite as well.
Hey! Loved the video. Small correction: when you give the example that a 700 in a 500 mean 100 SD exam is 95th percentile, you are forgetting that you should only account for one tail of the curve: a 700 is 97.5th percentile, and a 300 is a 2.5th percentile
You are right - thank you!
I was worried about keeping my steak dinner. Masterpiece, as always.
Don’t you worry about that. (And thank you for watching!)
If failures decrease, profits decrease. USMLE retakes and all the NBME practice exams are more money. Inaccurate predictions provided on the practice NBME exams like the CBSSA and CCSSA will increase confidence in weaker performers, they will take the test and fail. The guarantee of profit is padded through multiple measures. I would love to see a profit chart laid over the Mean/Minimum pass chart. Are there complementary troughs to the peaks you pointed out?
❤🔥truth
Agreed , nbme is a monopoly with a "non profit" disguise that takes advantage of medical students because , well there is no other way , how do you justify IMG exam fees , its almost double of what a US MD grad has to pay, the playing field is not equal and its near extorsion.
Very insightful video!! Please do share how we went from the comprehensive, multi-day style to what we have now. Thanks for the effort you put into your videos!
It would be interesting to also look at whether the difficulty of the questions have increased over time. Comparing older prep material with new feels like there's a noticable difference.
I agree with most of this except that they’re missing badly with their finger on the scale.
State legislators granting autonomy and prescribing authority to naturopaths, PAs and now non-licensed IMGs suggests the policy winds favor individuals deciding for themselves who is qualified and who is not.
Not that anyone really knows how to do that, but their trust in the free market is absolute.
not only the min. passing score is increasing, also the questions are getting harder.. so, a double whammy!
based
Not gonna lie - as an elderly millennial, I had to Google this. But thank you.
The graph with the memes 😭 lmao
44:46 You mean the decision was made in these years? If overall performance is improving every year, the nadir points should actually represent the years the minimum passing scores were increased? Sorry if I’m confusing.
Yes, the red arrows highlight a year when the decision was made to raise the passing score - so the difference between the average and the passing score drops a little bit immediately thereafter. What’s interesting to me the slope of each of those segments before the decision gets made. The steeper the slope, the more rapidly they had to step in and increase the passing standard.
Love this video!!
You didn’t mention that a certain % of questions are “test” questions that the nbme puts in with the real exams to see how they perform before making them real scored questions for the future. How do “test” questions factor in? Wouldn’t they be ways for nbme to see how well they perform and throw out the bad ones before they are official?
You know, for a while, I’ve had it in my head that I might do a whole video on USMLE experimental items… but now I’m gonna officially add it to the list. Stay tuned.
@@sheriffofsodium please
history lesson another time would be good!
Please do a history video!
Sheriff don't miss
How much do you estimate the USMLE cheating scandal affected the national mean score/average? Do continuously rising student averages indicate better and better students, easier and easier questions, or more and more unscrupulous activity over time due to mounted pressure?
The typical mean that the USMLE reports is for first-time test-takers from MD-granting medical schools in the United States. So far, all of the examinees whose scores have been invalidated have been IMGs - so they aren’t included in those figures.
I actually have a whole video on what’s behind #scorecreep:
A Brief Update on USMLE Score Creep
ruclips.net/video/1gKKAZ5aO8E/видео.html
20:30 OMG
20:20 thank me later
Bilateral cellulitis :D
these graphics are funny