- Видео 4
- Просмотров 118 607
Mikael Laine
Добавлен 26 сен 2013
Neural Network Backpropagation Example With Activation Function
The simplest possible back propagation example done with the sigmoid activation function.
Some brief comments on how gradients are calculated in actual implementations.
Edit: there is a slight omission/error in the da/dw expression, as pointed out by Laurie Linnett. The video has da/dw = a(1-a), but it should be ia(1-a), because the argument to a is the function (iw), whose derivative (with respect to w) is i.
Some brief comments on how gradients are calculated in actual implementations.
Edit: there is a slight omission/error in the da/dw expression, as pointed out by Laurie Linnett. The video has da/dw = a(1-a), but it should be ia(1-a), because the argument to a is the function (iw), whose derivative (with respect to w) is i.
Просмотров: 36 514
Видео
Another Simple Neural Network Backpropagated
Просмотров 6 тыс.6 лет назад
The second simplest neural networks explained mathematically: how to make sense of the gradients and optimization in detail for a very simple neural network. Please refer to the first video if you haven't seen it and want more: ruclips.net/video/8d6jf7s6_Qs/видео.html This is part 2/2 of the "Simple Neural Networks Backpropagated Series"
Implementation of the Absolutely Simplest Neural Network
Просмотров 2,4 тыс.6 лет назад
This is provided for completeness' sake: the code for the neural network in the previous video. Code here: github.com/mikaellaine/random/blob/master/simplest.py Install Tensorflow as described here: www.tensorflow.org/install/
0:03 / 9:21The Absolutely Simplest Neural Network Backpropagation Example
Просмотров 74 тыс.6 лет назад
Easy explanation for how backpropagation is done. Topics covered: - gradient descent - exploding gradients - learning rate - backpropagation - cost functions - optimization steps
Amazing explanation!! Any chance you can add in a bias and show how that works too?
very good explanation
amazing explanation
Thanks for the video!!! at 11:09, da/dw = a*(1-a), hmmm.. as it is e^-(iw), so should it be i*a*(1-a)? or in other words, w' = w - r*i*(da/dw)*(dC/da). Thanks again
I've been trying to find a way to learn back propagation without understanding deep math. Probably not a good idea lol. But my purpose is to implement a neural network with back propagation by hand. The comment at 13:00 really helped elucidate the path forward for me. I think I still don't have a deep understanding of the math, but at least I now have a path for in-code implementation. Thanks for making this video.
Fantastic video! I wish you were still making videos on the subject of AI with this teaching method.
great video!
What about when we have like activation function like relu. Or etc ?
The best explanation.
Wow, thank you
I don't know anything about this subject but I was understanding it until the rate of change function. Probably a stupid question but why is there a 2 in the rate of change function, as in 2(a-y). Is this 2 * (1.2 - 05)? Why the 2? I can't really see the reference to the y = x^2 but that's probably just me not understanding the basics. Maybe somebody can explain for a dummy like me. Wait maybe I understand my mistake, the result should be 0.4 right? So its actually 2(a-1) because otherwise multiplication goes first and you end up with 1.4?
The derivative of x^2 (x squared) is 2x. The cost function C is the square of the difference between actual and desired output i.e. (a-y)^2. Its derivative (slope) with respect to a is 2(a-y). We don't use the actual cost to make the adjustment, but the slope of the cost. That always points 'downhill' to zero cost.
Any help dealing with multi-neuron layers?, the formulas in 11:19 look different for multi-neuron layers
Check my channel for another example with multiple layers.
Good job! Thank you very much!
You are a life saver!! Thank you for breaking the whole process down in such an understandable way!!
Can anyone explain how to plot for 2(a-y),c=(a-y)2. i=1.5
Hi, a = i * w 1.5. * 2(a -y) = 4.5 * w - 1.5 What happened to the y?
y is given = the target value, here = 0.5. => 1.5*2(1.2-0.5) = 2.1 which equal to 4.5*0.8-1.5
on 2:40, Mikael mentioned "...and the error therefore, is 0.5" i think he meant "and the *desired output*, therefore is 0.5"? slight erratum perhaps?
because otherwise, the cost (C) is 0.49, not 0.5
What software did you use to write the stuff.. good lecture
There are some videos which you wish that it never end. This video in among top of these.