NN - 20 - Learning Rate Decay (with PyTorch code)

Поделиться
HTML-код
  • Опубликовано: 18 ноя 2024
  • In this video we are going to look into Learning Rate decay, what is it, and what is the intuition behind it.
    NN Playlist: bit.ly/3PvvYSF
    Become a member and get full access to this online course:
    meerkatstatist...
    ** 🎉 Special RUclips 60% Discount on Yearly Plan - valid for the 1st 100 subscribers; Voucher code: First100 🎉 **
    "NN with Python" Course Outline:
    Intro
    Administration
    Intro - Long
    Notebook - Intro to Python
    Notebook - Intro to PyTorch
    Comparison to other methods
    Linear Regression vs. Neural Network
    Logistic Regression vs. Neural Network
    GLM vs. Neural Network
    Expressivity / Capacity
    Hidden Layers: 0 vs. 1 vs. 2+
    Training
    Backpropagation - Part 1
    Backpropagation - Part 2
    Implement a NN in NumPy
    Notebook - Implementation redo: Classes instead of Functions (NumPy)
    Classification - Softmax and Cross Entropy - Theory
    Classification - Softmax and Cross Entropy - Derivatives
    Notebook - Implementing Classification (NumPy)
    Autodiff
    Automatic Differentiation
    Forward vs. Reverse mode
    Symmetries in Weight Space
    Tanh & Permutation Symmetries
    Notebook - Tanh, Permutation, ReLU symmetries
    Generalization
    Generalization and the Bias-Variance Trade-Off
    Generalization Code
    L2 Regularization / Weight Decay
    DropOut regularization
    Notebook - DropOut (PyTorch)
    Notebook - DropOut (NumPy)
    Notebook - Early Stopping
    Improved Training
    Weight Initialization - Part 1: What NOT to do
    Notebook - Weight Initialization 1
    Weight Initialization - Part 2: What to do
    Notebook - Weight Initialization 2
    Notebook - TensorBoard
    Learning Rate Decay
    Notebook - Input Normalization
    Batch Normalization - Part 1: Theory
    Batch Normalization - Part 2: Derivatives
    Notebook - BatchNorm (PyTorch)
    Notebook - BatchNorm (NumPy)
    Activation Functions
    Classical Activations
    ReLU Variants
    Optimizers
    SGD Variants: Momentum, NAG, AdaGrad, RMSprop, AdaDelta, Adam, AdaMax, Nadam - Part 1: Theory
    SGD Variants: Momentum, NAG, AdaGrad, RMSprop, AdaDelta, Adam, AdaMax, Nadam - Part 2: Code
    Auto Encoders
    Variational Auto Encoders
    If you’re looking for statistical consultation, work on interesting projects, or training workshop, visit my website meerkatstatist... or contact me directly at david@meerkatstatistics.com
    ~~~~~ SUPPORT ~~~~~
    Paypal me: paypal.me/Meer...
    ~~~~~~~~~~~~~~~~~
    Intro/Outro Music: Dreamer - by Johny Grimes
    • Johny Grimes - Dreamer

Комментарии •