I would pay so much to have you as my teacher, that's not only the best video i've ever seen on deep leanring, but probably the most appealing way anyone ever taught me CS !
20 дней назад+1
In ML topics most of the videos go to the beef of the math or the code implementation and do not care about a whole picture understanding. Your video fills that gap so nicely! Watching this video is a strong first step to understanding the topic deeply! very well done!
Thanks, this video have many explanations that are missing from other tutorials on VAE. Like the part from 22:45 onwards. I saw a lot of other videos that didn't explain how the p and q functions were related to the encoder and decoder. (every other tutorial felt like they started talking about VAE, and then suddenly changed subject to talk about some distribution functions for no obvious reason).
Thank you for creating such a nice explanation of VAE. When are you bringing the practical implementation video of VAE? please make a video for this as well.
🎯 Key points for quick navigation: 00:13 *understand fundamental ideas* 00:41 *explain autoencoder concept* 02:36 *issues with traditional autoencoders* 03:02 *introduce variational autoencoder* 04:13 *sampling from latent space* 25:02 *Model and prior choice.* 25:17 *Element-wise noise multiplication.* 25:32 *Learning log variance.* 26:00 *Maximizing ELBO for space learning.* 26:15 *Derivation of loss function.* Made with HARPA AI
Great video on VAE Umar, but I keep with a question: At 11:49, the KL divergence is written as "D_{KL}({numerator}||{denominator}". While in 11:34 you wrote KL divergence as "D_{KL}{denominator}||{numerator}". Why is that? I guess it is not because of the "\minus" sign
great video ( am about half way through). I think at minute 13 there is a misstatement (I think). in general if g(x) is greater than h(x), if we find the max of h it doesn't mean we have located the max for g
Variational methods is a class of methods ive always struggled to understand. I think your explanation addresses some key questions but I thunk you jumped into the math too quickly. Questions im still wondering about: - what are the "variational" parameters? They are represented by phi, but what do they do/mean? - why is having the noise source necessary? What happens if i set epsilon to be 0 all the time?
Both VAE and AE map the input over a latent space, the difference lies on the **structure** of this latent space. The AE latent space is not "well-organized" as well as the VAE's latent space.
At the very end of the video Umar mentions that in "the next video" he will explain the coding of the VAE along with some examples. I could not find that video. Was it made? What is its title? Thanks 🙂
Thanks for sharing . In the chicken and egg example, will p(x, z) be trackable? if x, z is unrelated, and z is a prior distribution, so p(x, z) can be writen in a formalized way?
Latent variable is of low dimension compare to input which is of high dimension…so this low dimension latent variable contains features which are robust, meaning these robust features survive the encoding process coz encoding process removes redundant features….imagine a collection had images of cat and a bird image distribution, what an encoder can do in such a process is to outline a bird or cat by its outline without going into details of colours and texture….these outlines is more than enough to distinguish a bird from a cat without going into high dimensions of texture and colors
@@quonxinquonyi8570 that doesnt answer the question. Latent space in autoencoders dont capture semantic meaning , but when we enforce regularization on latent space and learn a distribution thats when it learns some manifold
@@prateekpatel6082 learning distribution means that you could generate from that distribution or in other words sample from such distribution…but since the “ sample generating distribution “ can be too hard to learn, so we go for reparametrization technique to learn the a standard normal distribution so that we can optimize
“ learning the manifold “ doesn’t make sense in the context of variational auto encoder….coz to learn the manifold, we try to approach the “score function” ….score function means the original input distribution….there we have to noised and denoised in order to get some sense of generating distribution….but the problem still holds in form of denominator of density of density function….so we use log of derivative of distribution to cancel out that constant denominator….then use the high school level first order derivative method to learn the noise by using the perturbed density function….
1:21 - The file name of the original file zebra dot jpg should be replaced by zebra dot ppm (a truely uncompressed format). Then the zebra dot zip in the latent space be replaced by zebra dot jpg. And the result by zebra.ppm. Then the analogy would be way better to cover the lossy nature of the encoder but an enormous compression potential. It would be a comparison to the wavelet compression performed by JPEG, which is never lossless essentially.
After following your content for a while, I come to believe that if there is a concept, you will explain it better than anybody else here.
It's the clearest explanation about VAE that I have ever seen.
If you're up to the challenge, watch my other video on how to code Stable Diffusion from scratch, which also uses the VAE
I would pay so much to have you as my teacher, that's not only the best video i've ever seen on deep leanring, but probably the most appealing way anyone ever taught me CS !
In ML topics most of the videos go to the beef of the math or the code implementation and do not care about a whole picture understanding. Your video fills that gap so nicely! Watching this video is a strong first step to understanding the topic deeply! very well done!
@Umar Jamil, you have explained VAE so much better compared to others. You are one of my favorite RUclips teachers.
I love this so much, this channel lands in my top 3 ML channels ever
PLATO MENTIONED PLATO MENTIONED I LOVE YOU THAT'S THE BEST VIDEO I'VE EVER SEEN !!!
fing masterpiece thank you man, you outdid stanford and other papers with this explanation.
This is the best explanation on the internet!
The peps starting from 06:40 are the gem. Totally agree.
Getting philosophical w/ the Cave Allegory. I love it. Great stuff.
It was the best video I found to explain VAE. Thank you so much!
this is the best video on the Internet
You solved my confusion since long! Thank you !
so clear! so on point! love the way you teach!
Great explanation. Clean!!! Reminds me of school where our physics teacher taught everything practical and it felt so simple. subs+1👍
Incredible explanation. Thanks for making this video. It's extremely helpful!
You are a great teacher.
Thanks, this video have many explanations that are missing from other tutorials on VAE.
Like the part from 22:45 onwards. I saw a lot of other videos that didn't explain how the p and q functions were related to the encoder and decoder.
(every other tutorial felt like they started talking about VAE, and then suddenly changed subject to talk about some distribution functions for no obvious reason).
Glad you liked it!
Simply amazing. Thank you so much for explaining so beautifully. :)
Phenomenal teaching.
Thank you so much for this bro
Thank you for creating such a nice explanation of VAE. When are you bringing the practical implementation video of VAE? please make a video for this as well.
Link to the slides: github.com/hkproj/vae-from-scratch-notes
thx for the video, this is awesome!
🎯 Key points for quick navigation:
00:13 *understand fundamental ideas*
00:41 *explain autoencoder concept*
02:36 *issues with traditional autoencoders*
03:02 *introduce variational autoencoder*
04:13 *sampling from latent space*
25:02 *Model and prior choice.*
25:17 *Element-wise noise multiplication.*
25:32 *Learning log variance.*
26:00 *Maximizing ELBO for space learning.*
26:15 *Derivation of loss function.*
Made with HARPA AI
Great Explanation!!
Wow thank you very informative
This actually very good
Great video on VAE Umar, but I keep with a question: At 11:49, the KL divergence is written as "D_{KL}({numerator}||{denominator}". While in 11:34 you wrote KL divergence as "D_{KL}{denominator}||{numerator}". Why is that? I guess it is not because of the "\minus" sign
Appreciate it! it helped me a lot
great video ( am about half way through). I think at minute 13 there is a misstatement (I think). in general if g(x) is greater than h(x), if we find the max of h it doesn't mean we have located the max for g
Variational methods is a class of methods ive always struggled to understand. I think your explanation addresses some key questions but I thunk you jumped into the math too quickly.
Questions im still wondering about:
- what are the "variational" parameters? They are represented by phi, but what do they do/mean?
- why is having the noise source necessary? What happens if i set epsilon to be 0 all the time?
A normalizing flow video would complement this nicely
Would you please give the url for normalizing flows
Both VAE and AE map the input over a latent space, the difference lies on the **structure** of this latent space. The AE latent space is not "well-organized" as well as the VAE's latent space.
You are awesome; cheers
thank you very much
Hey, thank you for the great video. Curious if there is any plan to have a session for code for VAE? Many thanks!
Banger again
thanks UMAR!
At the very end of the video Umar mentions that in "the next video" he will explain the coding of the VAE along with some examples. I could not find that video. Was it made? What is its title? Thanks 🙂
There's a video on coding stable diffusion, which includes the VAE
Thanks a lot
Thanks for sharing . In the chicken and egg example, will p(x, z) be trackable? if x, z is unrelated, and z is a prior distribution, so p(x, z) can be writen in a formalized way?
What is the meaning of Latent Space? How is it different from the code that AutoEncoder have ?
You rock!
Don't AEs also learn a latent space representation? I'm not sure this is the largest difference with VAEs. The sampling is, however.
They do, but nobody forces the latent space of AEs to be a gaussian or any other known distribution.
Sad that you have not released video "Hot to code the VAE"(
Thanks!
Can you do more explanations with coding walk through that video you did on transformer with the coding helped me understand it a lot
Hi Oio! I am working on a full coding tutorial to make your own Stable Diffusion from scratch. Stay tuned!
@@umarjamilai i hope to see it soon, sir
14:41 you dont maximiye log p(x), that is a fixed quantity.
Hey can you do a video on SWin transformer next??
why does learning distribution via a latent variable capture semantic meaning. ? can you please elaborate a bit on that
Latent variable is of low dimension compare to input which is of high dimension…so this low dimension latent variable contains features which are robust, meaning these robust features survive the encoding process coz encoding process removes redundant features….imagine a collection had images of cat and a bird image distribution, what an encoder can do in such a process is to outline a bird or cat by its outline without going into details of colours and texture….these outlines is more than enough to distinguish a bird from a cat without going into high dimensions of texture and colors
@@quonxinquonyi8570 that doesnt answer the question. Latent space in autoencoders dont capture semantic meaning , but when we enforce regularization on latent space and learn a distribution thats when it learns some manifold
@@prateekpatel6082 learning distribution means that you could generate from that distribution or in other words sample from such distribution…but since the “ sample generating distribution “ can be too hard to learn, so we go for reparametrization technique to learn the a standard normal distribution so that we can optimize
I wasn’t talking about auto encoder,I was talking about variational auto encoder…
“ learning the manifold “ doesn’t make sense in the context of variational auto encoder….coz to learn the manifold, we try to approach the “score function” ….score function means the original input distribution….there we have to noised and denoised in order to get some sense of generating distribution….but the problem still holds in form of denominator of density of density function….so we use log of derivative of distribution to cancel out that constant denominator….then use the high school level first order derivative method to learn the noise by using the perturbed density function….
The Cave Allegory was overkill lol
I'm more of a philosopher than an engineer 🧘🏽
VAEs are awesomeeeeeee they make ai art generators work
I lost you at 16:00
fr me too
Missing a lot of details and whys.
1:21 - The file name of the original file zebra dot jpg should be replaced by zebra dot ppm (a truely uncompressed format). Then the zebra dot zip in the latent space be replaced by zebra dot jpg. And the result by zebra.ppm. Then the analogy would be way better to cover the lossy nature of the encoder but an enormous compression potential. It would be a comparison to the wavelet compression performed by JPEG, which is never lossless essentially.