My son and I watched this entire video together. You did a great job teaching. My son understood this, I made sure to ask him questions to ensure he surly did. He's nearly 13 now and loves data-science. Great lecture and you have an excellent voice for teaching. Thanks again for this video.
The best explanation of Metropolis-Hastings algorithm. Thank you for this, no university course nor research paper will provide this much of understandable clarification.
Really nicely done - a very clear and concise explanation. I got a better sense of MCMC and M-H from these 8 minutes than several hours of reading on the topic beforehand. Thank you.
Note for revision. Why "Markov Chain"? Draw a theta (which will be accepted or rejected based on Hastings algo), then drawn another theta based on the new theta. The new theta is dependent on the old theta, so will be each successive theta, hence Markov Chain.
My question is why? Why do we generate a new distribution from an already existing distribution? Why is each new generated value based on the previous value? Coin tosses are independent, are they not? Why would we do this to distributions of independent probabilities?
You wouldn’t do this for simple situations where it’s possible to sample independently. This is just a simplified example. MCMC algorithms are designed for use in multidimensional models where it isn’t feasible or efficient to calculate p(y), and thus a proper absolute posterior probability of theta. In such situations, when p(y) is intractable, dependent MCMC sampling is needed because it allows for the exploration of posterior parameter space through the use of relative frequencies of posterior samples rather than absolute probabilities
It depends on the parameter type. If it’s an unconstrained parameter, then a symmetrical is fine, usually the normal curve is used. If it’s a constrained parameter (e.g. sigma), then the Hastings variation of the Metropolis algorithm is used because it allows for non symmetric proposal distributions. The key to the algorithm and how it works to map out the posterior is the acceptance ratio, which can be affected by the size of the steps proposed (sigma in the normal curve proposal distribution). Auto-tuning allows the proposal distribution to be adjusted periodically to make sure the acceptance ratio is within the target threshold.
thanx for a beautiful video that has made me form an image of what mcmc is all about. God bless you. do kalman filters demo ,eg, for time dpseries predictions,using stata.
All this is excellent for physical, electrical / electronic systems, and the like. However, I have seen people try to use it for human based systems (like predicting the stock market, yeah, right). For that, it would serve best to know if a person likes Shakespeare, Frost, or limericks.
03:35 A new theta is drawn from the Proposal Distribution, which has a normal distribution in this example. How to decide/select the proposal distribution function?
Thanks! This explains MCMC perfectly. However it does not seem to explain why Markov chain is required? Why not just use Monte Carlo alone for drawing from the distribution?
It seems, usage of Beta(1,1,0.286), Beta(1,1,0.380), Binomial (10,4,0.286) and Binomial(10,4,0.380) at the time frame 4:16 has no meaning: the first two are constants and equal to each other, and the latter two describe PDF to express outside [0,1] region, as shown in the plot.
Sir..so MCMC is not seprarete algorithm but uses Meterpolis hasting algorithm.. i thought MCMC was a seperate procedure that helps us to simulate data to help us identify posterior distribution. Also i want to learn more.. how can i do so?
This is how MH accepts candidate/proposed values where the posterior ratio of proposed/current is less than 1. When it’s less than 1, a uniform distribution is used to draw a random number. If that random number is below the acceptance ratio, we accept the candidate value. It’s just a random number generator to make sure we accept our values at that probability
Can someone please explain to me how step one is calculated (posterior theta new / posterior theta old) because if you are sampling from a binomial distribution the output would be 0 or 1. and sampling from beta (1,1) is just sampling from a uniform distribution. for example what does beta(1,1,0.088) and binomial(10,4,0.88) each equal individially? Thanks in advance
With the first example in the video: R code: prior1=dbeta(shape1=1,shape2=1,x=0.517) likelihood1=dbinom(x=4,size=10,prob=0.517) prior2=dbeta(shape1=1,shape2=1,x=0.380) likelihood2=dbinom(x=4,size=10,prob=0.380) r=(prior2*likelihood2)/(prior1*likelihood1) r 1.31 Each part equals: prior1=1 prior2=1 likelihood1=0.19 likelihood2=0.25
Wait so if we use MCMC to walk around the parameter space why do we even need a prior distribution? Is it only used to initialize the starting value of the MCMC?
Thank you for this great tutorial! I have a question though: What happens if the mcmc sampler gives theta>1. You won't be able to calculate the likelihood or the prior. Do you discard this theta in this case? Shouldn't one use a uniform distribution u(0,1) to make sure we are getting a theta in [0, 1]?
You’ll need to read up on the distinction between Metropolis and Metropolis Hastings. The MH was created for this very reason, to address proposal distributions for constrained parameters, like sigma. The regular Metropolis algorithm only works for non-constrained parameters where this isn’t an issue. Basically, MH adjusts for this by allowing non symmetric proposal distributions in that instance. Beyond the scope of this video, I’d recommend watching Ben Lamberts series on Bayesian Statisitcs. He has a video specific to MH and this question
The target distribution is just the posterior, which is proportional to the prior * likelihood. A beta prior is a conjugate prior for the beta binomial posterior (target) distribution. Not all priors are conjugate priors. Regarding the proposal distribution, a normal distribution is often used b/c it is a symmetric distribution, but it’s not specifically mandated to be a normal
![Natanael Cano - Amor Eterno [Official Video]](http://i.ytimg.com/vi/EO4prCs6WpM/mqdefault.jpg)
This is by far the best explanation of Metropolis-Hastings algorithm I was able to find on the web. Thanks a lot.
AGREE
Brilliantly simple explanation of the Metropolis-Hasting Algorithm. Thanks.
best MCMC explanation on RUclips.
The rotated density plot clarified so many things. Thanks!
My son and I watched this entire video together. You did a great job teaching. My son understood this, I made sure to ask him questions to ensure he surly did. He's nearly 13 now and loves data-science.
Great lecture and you have an excellent voice for teaching. Thanks again for this video.
The best explanation of Metropolis-Hastings algorithm. Thank you for this, no university course nor research paper will provide this much of understandable clarification.
easily the best mcmc explanation
The best explanation, available online, on this topic by far. The visualization of the algorithm is superb! Very much appreciated.
The perspective of visualization & presenter's clarity made this the best explanation
Very clear explanation with plots. This is very helpful.
As others have already said, this is hands down the best explanation of MCMC I've seen on RUclips!
The best explanation of MCMC ,,, thank you x 1000
Simply the best illustration I have ever seen on the web - THANK YOU SO MUCH!
One of the best MCMC explanations on RUclips and I'm just learning for fun with no formal math/stats background.
Really nicely done - a very clear and concise explanation. I got a better sense of MCMC and M-H from these 8 minutes than several hours of reading on the topic beforehand. Thank you.
very good. The best explanation so far.
The best video on MCMC! Hands down.
Thank you. I've been looking for an intuitive explanation of MCMC for years!
By far the most clear explanation I've seen. Thanks a lot!!!!!
Most intuitive explanation of the MCMC-MH
I agree with the previous comments! Best explanation on RUclips by far
Thank you so much. This was life changing
Top comment says it all, really clicked for me when you broke down into three stages. Thank you
Thank you so much for this. All the other explainations I found had me very confused
Crisp n Clear!
Thanks for sharing your knowledge.
I have been trying to understand bayesian MCMC for the last couple of days, and this helped my understanding along greatly. Thanks!
Very clean explanation! Much better than other videos online
If you want to know properly then watch at .75x speed
Very nice explanation
Great video!
I think I finally understood what this algorithm actually does.
Thank you.
Best MCMC explanation on RUclips. Thank you !
Videoo that explains mcmc nice and clear. Thank you
Hands Down the best
I love that visaualization !
Great overview, thanks.
Finally I understand MCMC WOooOw😍😍😍😍😍
Brilliantly clear explanation! Thanks a lot! Really Great Job!
very practical and insightful. best explanation of MH / MCMC
Excelent explanation! Best I've found so far!
Excellent.!!!
wow. man ! i have no words ...just perfection.I appreciate the speed.
Great explanation. Awesome job.
Thank you so much. I finally understood MCMC!!!!
Simply the best MH video.
Great Explanation
Note for revision.
Why "Markov Chain"?
Draw a theta (which will be accepted or rejected based on Hastings algo), then drawn another theta based on the new theta. The new theta is dependent on the old theta, so will be each successive theta, hence Markov Chain.
This is a godsend. Thank you.
Excellent explanation. Thank you!
That was the heck of a good video!! Thank you!!
Thank you! Excellent video!
Very good tutorial useful for Stata's users
My question is why?
Why do we generate a new distribution from an already existing distribution?
Why is each new generated value based on the previous value? Coin tosses are independent, are they not? Why would we do this to distributions of independent probabilities?
You wouldn’t do this for simple situations where it’s possible to sample independently. This is just a simplified example. MCMC algorithms are designed for use in multidimensional models where it isn’t feasible or efficient to calculate p(y), and thus a proper absolute posterior probability of theta. In such situations, when p(y) is intractable, dependent MCMC sampling is needed because it allows for the exploration of posterior parameter space through the use of relative frequencies of posterior samples rather than absolute probabilities
Very clear expression
Very helpful. Thanks.
concise and precise
1:59 - it's very important to mention Law of Large numbers (LLN) at this point !!
Why?
Really helpful! Thank you. :)
Amazing
How is the proposal distribution chosen ? And Why isn't that a limitation of the MH algorithm ?
It depends on the parameter type. If it’s an unconstrained parameter, then a symmetrical is fine, usually the normal curve is used. If it’s a constrained parameter (e.g. sigma), then the Hastings variation of the Metropolis algorithm is used because it allows for non symmetric proposal distributions. The key to the algorithm and how it works to map out the posterior is the acceptance ratio, which can be affected by the size of the steps proposed (sigma in the normal curve proposal distribution). Auto-tuning allows the proposal distribution to be adjusted periodically to make sure the acceptance ratio is within the target threshold.
thanx for a beautiful video that has made me form an image of what mcmc is all about.
God bless you.
do kalman filters demo ,eg, for time dpseries predictions,using stata.
Simply the best!
I just love how unapologetically Windows this is.
Very good. Thanks
All this is excellent for physical, electrical / electronic systems, and the like. However, I have seen people try to use it for human based systems (like predicting the stock market, yeah, right). For that, it would serve best to know if a person likes Shakespeare, Frost, or limericks.
I would like to know which programa did tou use to make the trace anda histograma plot simulation. I found It Very useful for classes
Awesome!
Thanks for the nice lecture.
zbdbnfbnxvb c
congratulations
Best ever
beautiful
Under every video I've watched, someone comments that "this is the best video on MH algorithm!!" but I still don't understand it 😭
03:35 A new theta is drawn from the Proposal Distribution, which has a normal distribution in this example. How to decide/select the proposal distribution function?
1:13 Monte Carlo
2:04 Markov Chain
How would it work if theta consist of multiple variables?
Thanks! This explains MCMC perfectly. However it does not seem to explain why Markov chain is required? Why not just use Monte Carlo alone for drawing from the distribution?
The Random Walk is a Markov Chain, problem they using to model the problem not to solve the problem
Well explained
excellent
It seems, usage of Beta(1,1,0.286), Beta(1,1,0.380), Binomial (10,4,0.286) and Binomial(10,4,0.380) at the time frame 4:16 has no meaning: the first two are constants and equal to each other, and the latter two describe PDF to express outside [0,1] region, as shown in the plot.
what is the proposal distribution, which generating theta for each time?
It’s a normal curve in this example, centered around the theta value sampled in the previous iteration
Thanks
Can you PLEASEEE add subtitles in your video? Somehow for these particular video series, automatic closed caption is not available either :(
There is a mistake at 4:38. I.e., .247 > 0.039, not less.
Sir..so MCMC is not seprarete algorithm but uses Meterpolis hasting algorithm.. i thought MCMC was a seperate procedure that helps us to simulate data to help us identify posterior distribution.
Also i want to learn more.. how can i do so?
Contact us at tech-support@stata.com for assistance.
thanks for great video
why you using uniform distribution to generate random variable and compare the result with it ?
This is how MH accepts candidate/proposed values where the posterior ratio of proposed/current is less than 1. When it’s less than 1, a uniform distribution is used to draw a random number. If that random number is below the acceptance ratio, we accept the candidate value. It’s just a random number generator to make sure we accept our values at that probability
Can someone please explain to me how step one is calculated (posterior theta new / posterior theta old) because if you are sampling from a binomial distribution the output would be 0 or 1. and sampling from beta (1,1) is just sampling from a uniform distribution. for example what does beta(1,1,0.088) and binomial(10,4,0.88) each equal individially? Thanks in advance
With the first example in the video:
R code:
prior1=dbeta(shape1=1,shape2=1,x=0.517)
likelihood1=dbinom(x=4,size=10,prob=0.517)
prior2=dbeta(shape1=1,shape2=1,x=0.380)
likelihood2=dbinom(x=4,size=10,prob=0.380)
r=(prior2*likelihood2)/(prior1*likelihood1)
r
1.31
Each part equals:
prior1=1
prior2=1
likelihood1=0.19
likelihood2=0.25
Wait so if we use MCMC to walk around the parameter space why do we even need a prior distribution? Is it only used to initialize the starting value of the MCMC?
This is great but the typo = should say "proportional" on your definition of posterior
Thank you for this great tutorial! I have a question though: What happens if the mcmc sampler gives theta>1. You won't be able to calculate the likelihood or the prior. Do you discard this theta in this case? Shouldn't one use a uniform distribution u(0,1) to make sure we are getting a theta in [0, 1]?
You’ll need to read up on the distinction between Metropolis and Metropolis Hastings. The MH was created for this very reason, to address proposal distributions for constrained parameters, like sigma. The regular Metropolis algorithm only works for non-constrained parameters where this isn’t an issue. Basically, MH adjusts for this by allowing non symmetric proposal distributions in that instance. Beyond the scope of this video, I’d recommend watching Ben Lamberts series on Bayesian Statisitcs. He has a video specific to MH and this question
Please explain this for me...why do you use Beta(1,1) as prior distribution which is flat, why not use Beta(30, 30) you used in previous video?
3:14 I think we accept theta new when u > acceptance probability
how to calculate odd ratio in bayesian ordered logistic plz tell me
why do we do step 3 at 4:00?
The best
what's the program tht he's using? looks so much simpler than rjags that i'm using right now
I am not able to link the binomial likelihood and beta prior with the proposal and the target distributions. Do they correspond to each other?
The target distribution is just the posterior, which is proportional to the prior * likelihood. A beta prior is a conjugate prior for the beta binomial posterior (target) distribution. Not all priors are conjugate priors. Regarding the proposal distribution, a normal distribution is often used b/c it is a symmetric distribution, but it’s not specifically mandated to be a normal
how do you make animation
Can anyone explain how MCMC works as if I'm reaaaaaaaaally stupid (which I am) pleaaaaaaaaaase?
great explanation, but I had to set the speed 0.75
Nice... But I guess people at the level of MCMC Bayesian inference should be comfortable with R lol
English subtitles, please