What are Optimizers in Deep Learning?

If you are new to deep learning, you might have stumbled upon optimizers.
In this video, we'll explain what they are, how they fit into the deep learning architecture and look at 3 example optimizers.
Table of Content
- Introduction: 0:00
- Definition of Optimizers: 0:54
- Backpropagation and Gradient Descent: 2:59
- 3 Example Optimizers: 4:14
- Suggestions for Optimizers: 5:37
- Conclusion: 5:57
Visualization video during the introduction from Diego check him out:
📌 • Gradient Descent with ...
Great review article on gradient descent:
📌 arxiv.org/pdf/1609.04747
Great blog post about gradient descent algorithms:
📌 www.ruder.io/optimizing-gradi...
Another great blog post about more modern gradient descent algorithms:
📌 johnchenresearch.github.io/de...
----
Join the newsletter for weekly AI content: yacinemahdid.com
Join the Discord for general discussion: / discord
----
Follow Me Online Here:
Twitter: / yacineaxya
GitHub: github.com/yacineMahdid
LinkedIn: / yacinemahdid
___
Have a great week! 👋

[@tomholroyd7519]

The saddle suggests minimax. This sort of optimizer is used in GANs

[@machinelearningexplained]

Thanks for the insight!