The Problem of Local Optima (C2W3L10)

Take the Deep Learning Specialization: bit.ly/39xFIXq
Check out all our courses: www.deeplearni...
Subscribe to The Batch, our weekly newsletter: www.deeplearni...
Follow us:
Twitter: / deeplearningai_
Facebook: / deeplearninghq
Linkedin: / deeplearningai

[@MrHics]

I just came here for the dragon drawing. Now I know it's a horse-saddle, and now I want to learn AI

[@ubermensch5472]

I can't get over that horse drawing! There is something about it being used in this situation - a deep learning course - that makes it so cute and funny.

[@tbru92]

lovely drawing :)
thanks for your awesome courses!! :)

[@ishraqkhan8222]

Oh my God, he actually drew the horse

[@taeheeko2793]

Thanks for very intuitive lesson! Now I understand why analysis on saddle points becomes interesting more than that of convergence.

[@gorgolyt]

What's happening at the back end of that horse? 😂 My neural network classifies that with high probability as a dinosaur.

[@wifi-YT]

At 4:24, Andrew says algorithms like momentum and RMS prop can lessen problem of plateaus slowing down learning. Isn’t it really only RMS prop (or the RMS prop component of Adam), not momentum, that helps solve the plateau problem? Momentum only solves problem of gradient swinging back and forth (overshooting) between positive and negative, which is not a plateau situation.
On further reflection, maybe momentum can help a bit with plateau problem, as well, in some situations. Where one is not overshooting, but moving very slowly down plateau with declining gradients, momentum’s averaging current small gradient with prior larger gradients yields faster move down plateau. So momentum helps a bit, though not nearly as much as RMS prop, which specifically boosts effective learning rate where the gradients are small.
Am I getting this right?

[@wifi-YT]

@ez sorry, I only know about the Coursera Deep Learning specialization. There’s also on RUclips videos taught by “Mandy”. ruclips.net/video/qFJeN9V1ZsI/видео.html

[@forrowelt]

Great video! Do you have a scientific source of the plateau problem? That would be immensely valuable for my PhD (about learning in sports). 🙏🙏

[@kristenm6]

At 1:46, why is the chance of the 20,000 plotted weights curving downward 2^-20000?

[@blubblubber9460]

for each of the 20000 directions, it can either be downward or upward (with probability 1/2 or 2^-1 respectively). Chance that it is downward for all 20000 directions (when it's really a local minima and not a saddle point) is then 2^-20000, that's basic probability calculation. It's like tossing a coin 20000 times and getting 20000 times head, which is of course extremely unlikely.

[@KompakterOperator]

2:36 epic dinosaur rider

[@fruitbaskets7984]

Woah, very nice to know!

[@deletedaccount2580]

is drawing is horse or dragon ?One of the great professor Andrew Ng

[@EranM]

2:30 that's a dinosaur!

[@kil210bom4]

i steady this problem before.
but now i facing this problem in reality project.
it's difficult problem. but i will beat him 💪

[@brunofilipeaguiar]

How many takes Andrew Ng took to draw the horse without laughing?

[@sandipansarkar9211]

nice explanation .need to watch again

[@ermano5586]

I am watching it again

[@danmo7072]

the dude on horse is facing backwards.though

[@bmclaughlin01]

Surely no one watching needs you to draw a horse to explain a saddle point.