yes I agree but for simplicity sake he should have done a 0 to 1, 0 being black 1 being white and 0.5 being grey, because almost everyone follows that pattern, and for new learners its a bit harder for them to switch from thinking about -1 to 1 to 0 to 1
I watched your videos 3 years ago. It'salmost nostalgic. You may not see this. But you're one of the reasons I kept moving through with Machine Learning
I've been struggling to wrap my head around this topic for a few days, now. You went through everything very slowly and thoroughly and I'm now ready to dive into more complex lessons on this. Thank you so much, Brandon!
"Connections are weighted, MEANING".... Holy fuck.....after viewing numerous videos from youtube, online courses and google talks.... (often with comments below saying "thanks for the clear explanation")....This is the FIRST person i have EVER seen that has actually explained what the purpose of weights are....
Tremor244 I am by no means an expert but weighting is what makes the network "learn" to be correct. By changing the weights it changes the output of each neuron and eventually the output of the network. If you tune the weights enough you will eventually get an output that is what it is supposed to be. i hope this helps
I don't work with neural networks but with other types of machine learning. But weighting is more or less the same in all these fields of mathematics. You want a system that, provided with an input (an image, for instance), achieves its classification as the output. For instance you have a scenery (input) and you want to know if it's from vacations at the mountains or at the beach (a classification, ie. the output). So you pass the image trough a set of filters: (1) does the image have umbrellas? (2) does it have clouds? (3) is there a lot of blue? (4) is there a lot of brown?, etc. If the image passes a specific combination of filters, there is a greater probability that the image is of a specific type (for instance, if the image (1) have umbrellas, (3) is blueish and isn't (4) brownish, it's more likely to be from the BEACH). But how much more likely? That's when the WEIGHTING comes into play. Through machine learning we want to calculate some coefficients (weights) that state a sort of likelihood of an image to pass a filter, given its type (for instance, if it has umbrellas there's a probability of 0.9 out of 1 (90%) that it is from the beach and not from a mountain, but if there's a lot of blue maybe only 0.6 of those images are from the beach, and so the WEIGHT IS LIGHTER. That means that, if the image passes a filter of COLOR BLUE it is likely to be from a BEACH, but if it passes a filter of UMBRELLAS it is EVEN MORE LIKELY). Weights, then, are a parameter of RELEVANCE of each of the selected filters to achieve the correct classification. So we make the machine learn from LOTS (thousands, perhaps) of images that we KNOW are from the beach or the mountain. One image from the beach has umbrellas, so the classification through the filters was correct and then the WEIGHT for the umbrellas is increased. But if there is an image of the mountains with umbrellas and the program says it's from the beach, the weight goes down for the umbrellas. When we did this with a lot of images, the weights are FINE TUNED to classify correctly most of the time (if the filters are any good... if we chose wrong filters from the beginning, then there's a chance the dictionary won't get any better even fed with lots of images. That could also happen if the training images are biased: ie. if they don't represent the real set of images that we want to classify). I hope this works better for you!
If you have had a course on linear regression, you will recognize weights are equivalent to parameters. They are just "free variables" you adjust in order to match inputs with outputs. In one-dimensional linear regression, the parameters are the slope and offset of a line, you adjust them so that the distance between the line and your points (your training examples) is the least. Neural networks use the same idea as statistical regression. The main difference is that neural networks use a lot of weights (parameters), and for this reason you have to care about overfitting. This in general does not happen in linear regression, since the models are way more parsimonious (use only a few parameters). The use of a lot of weights is also the reason why neural networks are good general approximators, the large amount of weights give them high flexibility. They are like bazookas, while statistical regression is more like a small gun. The point is that most of the times you need only a small gun. However, people like to apply neural networks to problems where linear regression would do a good job since NN are "sexier".
I already knew how neural networks work, but next time someone asks me, I'll consider showing him or her this video! Your explanation is visualized really nicely.
Great video. Here are my notes: 7:54: The edges going into the bottom right node should be white instead of black. This small error repeats throughout the video. 10:47: You fixed the color error. 11:15: Man, this video feels good. 21:41: Man, this video feels really good. An extension for the interested: Sometimes we calculate the error of a network not by comparing its output to labels immediately, but by first putting its output through a function, and comparing that new output to something we consider to be the truth. That function could be another neural network. For example, in real-time style transfer (Johnson et al.), the network we train takes an image and transforms it into another image; we then take that generated image and analyze it with another neural network, comparing the new output with something we consider to be the truth. The point of the second neural network is to assess the error in the generated image in a deeper way than just calculating errors pixel by pixel with respect to an image we consider to be the truth. The authors of the real-time style transfer paper call this higher-level error "perceptual loss", as opposed to "per-pixel loss". I know this was outside the scope of this video, but it was helpful to me to write it, and I hope it will help someone who reads it.
I had the idea of doing perceptual loss before I even knew the term for it, seems like it would work better for warp transforms and the like versus level transforms.
Alternatively, the edges are correct but the corresponding picture should be flipped. Regardless, the final step (output perceptron at the bottom indicating horizontal) works with either the white white edges or the black black edges scenario.
Brilliant! I was involved 50 years ago in a very early AI project and was exposed to simple neural nets back then. Of course, having no need for neural nets, I forgot most of what I ever knew about them during the interval. And, wow, has the field expanded since then. You have given a very clear and accessible explanation of deep networks and their workings. Will happily subscribe and hope to find further edification on Reinforcement Learning from you. THANK YOU.
I hate how many people try to explain complicated concepts that require math without using math. Because you used clear mathematical notation, you made this much easier to understand. Thank you so much.
As a chemist who just recently took Physical Chemistry, back propagation makes SOOO much more sense to me when you put it into a calculus description, rather than a qualitative one as I've been seeing elsewhere. So THANK YOU!
I've watched many videos on RUclips but non of the videos explained the concepts as intuitively as you did. Thought I have to watch it again as I've failed to grasp some concepts, I am sure that it will be clear as I watch more.
This simple lecture truly makes all viewers fully understand the logic behind neural networks. I strongly recommend this video clip to my colleagues participated in data driven industry. Thanks.
Clean, concise, informative, astonishingly helpful, you have my deepest gratitude. I've never seen anyone explain backprop as well as you just did, great job!
I have been trying develop a DNN for a week. I have seen almost a 100 videos, forums, blogs. Of all those this is the only one with calculus that made complete sense to me. You sir are the real MVP
this is probably the best breakdown ive came across, very dense, you've left no spaces in between your explanations! Thanks for the great lesson! Onward to a calculus class!
30 years ago, I studied computer science. we were into pattern-recognition and stuff, and I was always interested in learning machines, but couldn't get the underlying principle. now, I got it. that was simply brilliant. thanks a lot.
I have seen many many videos regarding NN but this is by far the best; Brandon understands the relationship between the NN and the backbone of the NN, which is the underlining math. He clearly presented them in a very intuitive way. Hats off for you sir. Keep up the good job.
I always had trouble intuitvely understanding how a derivate works and how practically its calculation is reflected in simple terms. Little did I know starting this video, that I'll finally understand it. Thank you! I'm relieved and feel less stupid now.
Fantastic description of what these networks do. I've gone thru a few of these explainers and all they demonstrated was the person knew their subject, they just couldn't teach it. They talk in jargon, that quickly loses those unfamiliar. IOW they're not teaching, they're having a 'conversation' with those who are already versed and have background in the field. Einstein is to have said, 'If you can't explain it simply, then you don't understand it yourself' Thanks, again. I walk away feeling like I actually learned something. You Can Teach.
That was incredible...watched 7 videos so far and every day my brain understands a bit more...I recently learning Houdini VEX code which is 3D graphics programming, and that took 1 year of watching a whole bunch of stuff and not getting it...until I did...so I know I will grasp this soon....Im sticking to these simple examples for now, until I can code it from scratch in Python
My kind of teacher! Subscribed! Nice voice, nice face, nice tempo, nice amount of information, nice visuals. You'd almost start to believe this video was produced with the concepts you've talked about. And my mind was just blown. I realized that we could make a lot more types of virtual neurons and in that way outclass our own brains (at even a fraction of the informational capacity) with a multitude of task-specific sub-brains forming a higher brain that may or may not develop personality.
Thank you Brandon for taking the time to explain the logic behind neural networks. You have given me enough information to take the next steps towards building one of my own... and thank you RUclips algo for bringing this video to my attention.
Great video. A single clear, concrete example is more useful than 100 articles full of abstract equations and brushed-over details. Speaking as someone who's read 100 articles full of abstract equations and brushed-over details.
@Brandon Rohrer To each neuron 1. First a number is assigned 2. Then a weight is assigned 3. Then it is squashed using sigmoid and then the values are summed up! Right ? My question is whether each weighted neuron is squashed or the sum of weighted neurons is squashed , why is sqaushing done?
The function shown at 4:39 and 20:33 are both referred to as being the same function, but the graph on 4:39 is actually 2*(o(a)-0.5) , here o(a) is the sigmoid function.
Holy thank you!! ive watched like 50+ ich tutorials on neural networks but all of em explained things poorly or too fast. But you went through everything slowly and actually explained all the info clearly!!
Yours was a very easy explanation of deep neural networks. Each step in the process was broken down into bite-sized chunks, making it very clear what's going on inside a deep neural network. Thanks so much!
I totally want to implement this now. Can't be that hard - won't be the best approach, just trying to see how close it can "naturally" get to the ideal solution you displayed and how many iterations of training it takes!
Give it a go! Although I would love to get a sticker for my laptop that says "Can't be that hard." I'll put it right above the one that says "What could possibly go wrong?"
I understand every topic. From computer enginerring to calculus and AI basics, but put them all together and working to make a digital functioning brain is magic
This makes sense and all, but how do neural networks make sense in terms of prediction modeling for say a regression problem? Not image classification. Like housing prices for example?
so I ahve a quick questions, In the network you said there was negative weights(black) and positive weights(wight) so when your making the network do you pre-define which weights should be positive and negative or do they work themselves out when you train the network?
They work themselves out during training. If you'd like to go deep on the topic, there's a course on it with detailed tutorials and code: e2eml.school/321
This explanation of back propagation was exactly what I needed. This is very clear and I now have higher confidence in my ability to create my own ANN from scratch.
Great video, just one question. At roughly 7:56, where you show the shaded squares, in the bottom right it is a bit confusing. The far bottom right one is taking negative input values, which, going off what you said, means it should have black on the top and white on the bottom.
Thank you so much! I have been trying to create a neural network of my own for simple tasks, and I haven't been able to learn how until now. Thank you!
I haven't made one of my own yet, sorry for the confusion. The difference is that now, I understand how one works, so I can start structuring my information.
Hi Brandon, thank you for this video! What I am still missing in my understanding is: 1) What does the "inside" look like of a trained CNN, what makes it trained or where do the "weights", if that is correct term get stored that make it a trained CNN 2) You've covered how the neural net does the actual identification but not the math/mechanics of how it gets trained 3) same question but a different way of asking, how is the gradient descent implemented, ie, is it during the training phase? 4) is part of the training that we have to feed the NN batches of images that are let's say cats vs. dogs. That means we are essentially labeling the data correct? 5) so once the labeled data has trained the NN, what does that mean in terms of the state the NN is in to be ready to predict. Is the "learned" part represented in the "filters" and if not how are the filters determined? Lastly, I see images of features that the NN has identified at different stages, how are those features rendered some of which are pretty good pictures of faces. Is that part of the training process? Thank you.
Makai Post in the beginning you assign them random numbers, because you have to start somewhere. Later you change them to get the optimal values with various methods.
thank you for explanation! I have a question: why should weights be in range -1;1 or 0;1 ? why can't we take -2;2 or -3;3 and so on? anyway we use some activation function, which will turn output to 0;1 or -1;1
Than you for the video. So the basic building blocks of a NN are weighted connectors , sigmoid function , and the linear rectifier according to this video. Do most people just put the sigmoid and rectifier in series after each receptor? If not how does one figure when to use sigmoid function or when to use the linear rectified unit ?
Thanks for your excellent explanation! At 4:05 it seems you really use tanh, not sigmoid/the logistic function, since sigmoid goes from 0.0 to 1.0, but your squashing function goes from -1.0 to 1.0. Especially, if you take your definition at 20:26, it confirms you are using a sigmoid/the logistic function going from 0.0 to 1.0, which does not work out with your example network. It's a minor detail, but I tripped upon it when I tried to actually compute your example step by step. Maybe worth pointing out with a note in the video? Thanks again!
I have a question, around 4:00 just before you explain the sigmoid function, the added weighted value was -1.075, but then one frame after it became positive 1.075, was there something significant I missed?
Good catch Anna. Here's what I wrote to Michael: Good catch Michael. Yes, this is a straight up error and oversimplification on my part. The error is that it should be -1.075, not positive 1.075. The oversimplification is that the logistic function (that I show the equation for) varies between 0 and 1, not -1 and 1. You are spot on.
To figure out the error to weight correspondence function would you first iterate the neural network (by changing weight values and testing the resultant error) a few times the "numerically expensive" way and figure out the error function for each neuron? More generally; how would you go about finding the error function for each neuron? Is the function always parabolic?
I think that in 7:51, the most-right and lowest neuron should be "max activated" when the upper row of pixels is black and the lower row of pixels is white (i.e., the negative of what is displayed in the video, see lowest most-right square of pixels and respective neuron)
Watched it many times but still confused. (1) Why does the 4 pixel start with shaded grey but then the example (from 09:38) use a black and white? (2) At 03:01, how come the weights change from 1.0s to -0.2, 0.0, 0.8, -0.5? Where are these values from?
I have one question. In the chaining example, you assumed 'e' as output. But while describing, you interpreted 'e' as error. Why is that? And also, is it possible to know the error function? Thanks in advance.
Hi, great tutorial but I would like to make a clarification: at 7:50 a minute the fourth drawing at the bottom black-white and black, black of the second layer is wrong. The black and black connections would reverse the design, the connection lines should be white-white. Then the sigmoid function should not be 0 to 1? Looks like a TanH,-1 a +1 to me. Correct me if I'm wrong. Thank you ric
wow awesome i never find a video like this with the simple example and clarity of neural network and its a though topic to explain but you make it easy... thanks
![BLACK BAG - Official Trailer [HD] - Only in Theaters March 14](http://i.ytimg.com/vi/Du0Xp8WX_7I/mqdefault.jpg)
This should be recommended as the 1st video to watch when it comes to learn neural networks
Yes
yes I agree but for simplicity sake he should have done a 0 to 1, 0 being black 1 being white and 0.5 being grey, because almost everyone follows that pattern, and for new learners its a bit harder for them to switch from thinking about -1 to 1 to 0 to 1
totally agree. i understood the principle right away and now we have our own nn in operation
I watched your videos 3 years ago. It'salmost nostalgic. You may not see this. But you're one of the reasons I kept moving through with Machine Learning
I legit cried a little bit. Thank you for this.
I've been struggling to wrap my head around this topic for a few days, now. You went through everything very slowly and thoroughly and I'm now ready to dive into more complex lessons on this. Thank you so much, Brandon!
I'm very happy to hear it :)
"Connections are weighted, MEANING".... Holy fuck.....after viewing numerous videos from youtube, online courses and google talks.... (often with comments below saying "thanks for the clear explanation")....This is the FIRST person i have EVER seen that has actually explained what the purpose of weights are....
I feel the same, even though I still can't completely understand how weighting works :/
Tremor244 I am by no means an expert but weighting is what makes the network "learn" to be correct. By changing the weights it changes the output of each neuron and eventually the output of the network. If you tune the weights enough you will eventually get an output that is what it is supposed to be. i hope this helps
I don't work with neural networks but with other types of machine learning. But weighting is more or less the same in all these fields of mathematics.
You want a system that, provided with an input (an image, for instance), achieves its classification as the output. For instance you have a scenery (input) and you want to know if it's from vacations at the mountains or at the beach (a classification, ie. the output).
So you pass the image trough a set of filters: (1) does the image have umbrellas? (2) does it have clouds? (3) is there a lot of blue? (4) is there a lot of brown?, etc.
If the image passes a specific combination of filters, there is a greater probability that the image is of a specific type (for instance, if the image (1) have umbrellas, (3) is blueish and isn't (4) brownish, it's more likely to be from the BEACH). But how much more likely?
That's when the WEIGHTING comes into play. Through machine learning we want to calculate some coefficients (weights) that state a sort of likelihood of an image to pass a filter, given its type (for instance, if it has umbrellas there's a probability of 0.9 out of 1 (90%) that it is from the beach and not from a mountain, but if there's a lot of blue maybe only 0.6 of those images are from the beach, and so the WEIGHT IS LIGHTER. That means that, if the image passes a filter of COLOR BLUE it is likely to be from a BEACH, but if it passes a filter of UMBRELLAS it is EVEN MORE LIKELY). Weights, then, are a parameter of RELEVANCE of each of the selected filters to achieve the correct classification.
So we make the machine learn from LOTS (thousands, perhaps) of images that we KNOW are from the beach or the mountain. One image from the beach has umbrellas, so the classification through the filters was correct and then the WEIGHT for the umbrellas is increased. But if there is an image of the mountains with umbrellas and the program says it's from the beach, the weight goes down for the umbrellas. When we did this with a lot of images, the weights are FINE TUNED to classify correctly most of the time (if the filters are any good... if we chose wrong filters from the beginning, then there's a chance the dictionary won't get any better even fed with lots of images. That could also happen if the training images are biased: ie. if they don't represent the real set of images that we want to classify).
I hope this works better for you!
If you have had a course on linear regression, you will recognize weights are equivalent to parameters. They are just "free variables" you adjust in order to match inputs with outputs. In one-dimensional linear regression, the parameters are the slope and offset of a line, you adjust them so that the distance between the line and your points (your training examples) is the least. Neural networks use the same idea as statistical regression. The main difference is that neural networks use a lot of weights (parameters), and for this reason you have to care about overfitting. This in general does not happen in linear regression, since the models are way more parsimonious (use only a few parameters). The use of a lot of weights is also the reason why neural networks are good general approximators, the large amount of weights give them high flexibility. They are like bazookas, while statistical regression is more like a small gun. The point is that most of the times you need only a small gun. However, people like to apply neural networks to problems where linear regression would do a good job since NN are "sexier".
+Esteban Lucas Solari I want to let you know that I love you for everything you just wrote.
I've watched many videos and read many blogs and articles, you are literally the best explainer at making these intuitive to understand
I agree.Penny dropped for me today with this Video.
Thank you so much Brandon
3blue1brown
I already knew how neural networks work, but next time someone asks me, I'll consider showing him or her this video! Your explanation is visualized really nicely.
Great video. Here are my notes:
7:54: The edges going into the bottom right node should be white instead of black. This small error repeats throughout the video.
10:47: You fixed the color error.
11:15: Man, this video feels good.
21:41: Man, this video feels really good.
An extension for the interested:
Sometimes we calculate the error of a network not by comparing its output to labels immediately, but by first putting its output through a function, and comparing that new output to something we consider to be the truth. That function could be another neural network. For example, in real-time style transfer (Johnson et al.), the network we train takes an image and transforms it into another image; we then take that generated image and analyze it with another neural network, comparing the new output with something we consider to be the truth. The point of the second neural network is to assess the error in the generated image in a deeper way than just calculating errors pixel by pixel with respect to an image we consider to be the truth. The authors of the real-time style transfer paper call this higher-level error "perceptual loss", as opposed to "per-pixel loss".
I know this was outside the scope of this video, but it was helpful to me to write it, and I hope it will help someone who reads it.
Good to know that I'm not the only one who caught the logical mistakes.
9:14 Bottom second squash should be vertically inverted, if I'm not mistaken.
I had the idea of doing perceptual loss before I even knew the term for it, seems like it would work better for warp transforms and the like versus level transforms.
Alternatively, the edges are correct but the corresponding picture should be flipped.
Regardless, the final step (output perceptron at the bottom indicating horizontal) works with either the white white edges or the black black edges scenario.
thanks :)
YES - thanks , I saw that tiny error too 🙂
Brilliant! I was involved 50 years ago in a very early AI project and was exposed to simple neural nets back then. Of course, having no need for neural nets, I forgot most of what I ever knew about them during the interval. And, wow, has the field expanded since then. You have given a very clear and accessible explanation of deep networks and their workings. Will happily subscribe and hope to find further edification on Reinforcement Learning from you. THANK YOU.
I hate how many people try to explain complicated concepts that require math without using math. Because you used clear mathematical notation, you made this much easier to understand. Thank you so much.
A physics guy coming into coding, this video completely clarified the topic. Your treatment of this topic is perfect!
Don't worry people I asked this guy if he was a physicist
@ thanks bro
Finally! You are the only one on the internet who explained this properly!
As a chemist who just recently took Physical Chemistry, back propagation makes SOOO much more sense to me when you put it into a calculus description, rather than a qualitative one as I've been seeing elsewhere. So THANK YOU!
I've watched many videos on RUclips but non of the videos explained the concepts as intuitively as you did. Thought I have to watch it again as I've failed to grasp some concepts, I am sure that it will be clear as I watch more.
Thanks Mukul!
This simple lecture truly makes all viewers fully understand the logic behind neural networks. I strongly recommend this video clip to my colleagues participated in data driven industry. Thanks.
Clean, concise, informative, astonishingly helpful, you have my deepest gratitude.
I've never seen anyone explain backprop as well as you just did, great job!
I have been trying develop a DNN for a week. I have seen almost a 100 videos, forums, blogs. Of all those this is the only one with calculus that made complete sense to me. You sir are the real MVP
this is probably the best breakdown ive came across, very dense, you've left no spaces in between your explanations! Thanks for the great lesson! Onward to a calculus class!
30 years ago, I studied computer science. we were into pattern-recognition and stuff, and I was always interested in learning machines, but couldn't get the underlying principle. now, I got it. that was simply brilliant. thanks a lot.
Clean, concise, informative, astonishingly helpful, you have my deepest gratitude.
You are most welcome
I have seen many many videos regarding NN but this is by far the best; Brandon understands the relationship between the NN and the backbone of the NN, which is the underlining math. He clearly presented them in a very intuitive way. Hats off for you sir. Keep up the good job.
I always had trouble intuitvely understanding how a derivate works and how practically its calculation is reflected in simple terms. Little did I know starting this video, that I'll finally understand it. Thank you! I'm relieved and feel less stupid now.
I'm really happy to hear that Bowbert. Thank you for the note.
i was amazed by the way you talk, and explain very slowly as well you remain slow until the end and you dont rush things. bravo
Possibly one of the best explanations about NN out there... Congratulations!
Fantastic description of what these networks do. I've gone thru a few of these explainers and all they demonstrated was the person knew their subject, they just couldn't teach it. They talk in jargon, that quickly loses those unfamiliar. IOW they're not teaching, they're having a 'conversation' with those who are already versed and have background in the field.
Einstein is to have said, 'If you can't explain it simply, then you don't understand it yourself'
Thanks, again. I walk away feeling like I actually learned something. You Can Teach.
Hey thanks! I really appreciate this. It's the highest compliment.
It's the first time I get to understand how neural networks work. Thank you.
That was incredible...watched 7 videos so far and every day my brain understands a bit more...I recently learning Houdini VEX code which is 3D graphics programming, and that took 1 year of watching a whole bunch of stuff and not getting it...until I did...so I know I will grasp this soon....Im sticking to these simple examples for now, until I can code it from scratch in Python
My kind of teacher! Subscribed! Nice voice, nice face, nice tempo, nice amount of information, nice visuals. You'd almost start to believe this video was produced with the concepts you've talked about.
And my mind was just blown. I realized that we could make a lot more types of virtual neurons and in that way outclass our own brains (at even a fraction of the informational capacity) with a multitude of task-specific sub-brains forming a higher brain that may or may not develop personality.
The best and easily understandable neural net video I have seen
Easiest most elaborate explanation I have found on that matter
Thank you Brandon for taking the time to explain the logic behind neural networks. You have given me enough information to take the next steps towards building one of my own... and thank you RUclips algo for bringing this video to my attention.
Wow, this is a very simple explanation, and it helped me understand the concept of neural networks. Thank you.
Great video. A single clear, concrete example is more useful than 100 articles full of abstract equations and brushed-over details. Speaking as someone who's read 100 articles full of abstract equations and brushed-over details.
This is the most intuitive explanation of neutral nets that I've seen, keep up the great content!
Very very rare way to explain a neural network in such a great depth. Loved the way you explained it ❤
This is a really good video! For me as a complete beginner this really help me understand the basics of neural networks, thanks!
Artem Kovera lovely book , just downloaded. for the lazy people : amzn.to/2ntC9Zm
Who else paused the video at 15:10, went and did a semester of calculus, then came back and finished watching?
Nobody
:)
Muhammed Sahli's mother
SirNate I still remember most of my calculus and have notes from prior study 😜
lol
I almost never post comments, but this one deserve it :D
Truly great :D
I've never seen anyone explain backprop as well as you just did, great job!
I never understood backprop properly until this video...this was the light bulb
@Brandon Rohrer To each neuron 1. First a number is assigned 2. Then a weight is assigned 3. Then it is squashed using sigmoid and then the values are summed up! Right ? My question is whether each weighted neuron is squashed or the sum of weighted neurons is squashed , why is sqaushing done?
Good question Richa, and it will take a bit longer to answer. There's a deeper dive into this material at e2eml.school/312
Awesome description of what neural networks are!
At 7:48, the bottom right receptive field should be Black black white white, not White white black black :)
Congratulations on the explanation!
Yes, there is a mistake.
It is true! Good catch both of you.
I'm in love with the way you explain things! So professional yet simple and easy to follow. Keep it up!
Holy shit! Now I... I actually get it!
Thank you!
Clean, concise, informative, astonishingly helpful, you have my deepest gratitude.
OMG it's even harder then I expected! Thank you very much for the thorough and thoughtful explanation!
it goes without saying that I became a subscriber as well ;)
Really great explanation Brandon. Also, I greatly appreciate that you share your slides as well and that too in raw (PPT) format. Great work.
Very slowly and clearly gets to the point, nice and accessible video on the subject.
HELP, is it right that I'm getting values higher than 1 on the output?!?
The best Back Propagation explanation on you tube. Thank you sir.
The function shown at 4:39 and 20:33 are both referred to as being the same function, but the graph on 4:39 is actually
2*(o(a)-0.5) , here o(a) is the sigmoid function.
Holy thank you!! ive watched like 50+ ich tutorials on neural networks but all of em explained things poorly or too fast. But you went through everything slowly and actually explained all the info clearly!!
Thank you so much! I'm happy to hear how helpful it was, and it means a lot that you would send me a note saying so.
literally THE best explanation i found on the internet.
Yours was a very easy explanation of deep neural networks. Each step in the process was broken down into bite-sized chunks, making it very clear what's going on inside a deep neural network. Thanks so much!
one of the best teacher you cleared all my doubts for neural networks thanks sir let me click an ad for you
I totally want to implement this now. Can't be that hard - won't be the best approach, just trying to see how close it can "naturally" get to the ideal solution you displayed and how many iterations of training it takes!
Give it a go! Although I would love to get a sticker for my laptop that says "Can't be that hard." I'll put it right above the one that says "What could possibly go wrong?"
I've been trying to implement the full network as seen (@ 25:30) in python but hit a roadblock. Have you done it?
I understand every topic. From computer enginerring to calculus and AI basics, but put them all together and working to make a digital functioning brain is magic
You explanation is so far the best. I am sure you are the best teacher so far. Thanks for uploading such a informative video.
Math, baby! It’s cool
Best video explaining neural networks i found so far. Thank you a lot!
This is one of the best explanation of neural network I have seen
This makes sense and all, but how do neural networks make sense in terms of prediction modeling for say a regression problem? Not image classification. Like housing prices for example?
Superb!! The best explanation of DL that I have come across after completing the Andrew NG's Stanford ML course. I am a follower now.
I've seen a lot of videos and this is the most clear explanation. Exceptional graphics too.
so I ahve a quick questions, In the network you said there was negative weights(black) and positive weights(wight) so when your making the network do you pre-define which weights should be positive and negative or do they work themselves out when you train the network?
They work themselves out during training. If you'd like to go deep on the topic, there's a course on it with detailed tutorials and code: e2eml.school/321
@@BrandonRohrer thank you very much, this has cleared alot of stuff up
at 4:32 this isn't a sigmoid function .sigmoid squashes the value to between 0 - 1
Best explanation I have seen so far man. Congratulations!
This explanation of back propagation was exactly what I needed. This is very clear and I now have higher confidence in my ability to create my own ANN from scratch.
Same here. My vision is clear
I just enjoy numbers. Anything to do with them is a fantastic thing.
Very detailed and clear explanation. Thank you for sharing! :)
You explained How Neural Networks in very simple and easy to understand manner. thanks for sharing!
the best explanation for Neural Networks ever !
This is gold if your looking to learn neural networks!! Well done
Great video, just one question. At roughly 7:56, where you show the shaded squares, in the bottom right it is a bit confusing. The far bottom right one is taking negative input values, which, going off what you said, means it should have black on the top and white on the bottom.
9:03. Shouldn't the bottom most square in the 3rd column be different?
It has 2 negative weights as its input neurons, so shoudln't they be flipped? 🤨
The *Best* Video for beginners!
Thanks!
Thank you so much! I have been trying to create a neural network of my own for simple tasks, and I haven't been able to learn how until now. Thank you!
Gavin Haynes please teach me how to start. im a beginner of this field. thank you
I haven't made one of my own yet, sorry for the confusion. The difference is that now, I understand how one works, so I can start structuring my information.
After wasting too much time finally I found the right place , excellent explanation 👏👏👏💯
The internet needed a video like this one. Thanks a lot!
Is there some type of playlist for learning about neural networks?
Give this a try: ruclips.net/video/ILsA4nyG7I0/видео.html
I've a question, how do we determine the number of hidden layers and nodes for a layer in a neural network ?
That's a big question. Here's a course I wrote to help answer it: e2eml.school/314
@@BrandonRohrer Thanks, I'll check it out !
by far the best video about ANN i've watched, thank you so much, really clear
Hi Brandon, thank you for this video!
What I am still missing in my understanding is:
1) What does the "inside" look like of a trained CNN, what makes it trained or where do the "weights", if that is correct term get stored that make it a trained CNN
2) You've covered how the neural net does the actual identification but not the math/mechanics of how it gets trained
3) same question but a different way of asking, how is the gradient descent implemented, ie, is it during the training phase?
4) is part of the training that we have to feed the NN batches of images that are let's say cats vs. dogs. That means we are essentially labeling the data correct?
5) so once the labeled data has trained the NN, what does that mean in terms of the state the NN is in to be ready to predict.
Is the "learned" part represented in the "filters" and if not how are the filters determined?
Lastly, I see images of features that the NN has identified at different stages, how are those features rendered some of which are pretty good pictures of faces. Is that part of the training process?
Thank you.
--------
Additional question is how does feature engineering play into training a CNN?
Thank you.
Thank you so much, really helped me understand several things that were hard to understand during class.
I'm glad to hear it!
+Brandon Rohrer How do you determinate the weights? [2:36]
I would also very much like to know this.
Makai Post in the beginning you assign them random numbers, because you have to start somewhere. Later you change them to get the optimal values with various methods.
REMEMBER...every tutorial has its retarded parts.....poor explanation.. I listened to that part like 5 times ...and it still sound retarded...ugghhhh
I like how much effort you put into making these ideas accessible
18:40 thats quite the same way of expressing error/weight since you can rlly just do cancelling
thank you for explanation! I have a question: why should weights be in range -1;1 or 0;1 ? why can't we take -2;2 or -3;3 and so on? anyway we use some activation function, which will turn output to 0;1 or -1;1
Than you for the video. So the basic building blocks of a NN are weighted connectors , sigmoid function , and the linear rectifier according to this video. Do most people just put the sigmoid and rectifier in series after each receptor? If not how does one figure when to use sigmoid function or when to use the linear rectified unit ?
Thanks for your excellent explanation! At 4:05 it seems you really use tanh, not sigmoid/the logistic function, since sigmoid goes from 0.0 to 1.0, but your squashing function goes from -1.0 to 1.0.
Especially, if you take your definition at 20:26, it confirms you are using a sigmoid/the logistic function going from 0.0 to 1.0, which does not work out with your example network.
It's a minor detail, but I tripped upon it when I tried to actually compute your example step by step. Maybe worth pointing out with a note in the video? Thanks again!
phew, thanks for confirming. I thought I was going crazy and that my previous notes on activation were wrong. Yes, this is tanh and not sigmoid
One of the best videos out there
I have a question, around 4:00 just before you explain the sigmoid function, the added weighted value was -1.075, but then one frame after it became positive 1.075, was there something significant I missed?
I thought no one noticed this and posted the question myself. He should post the squashing function that he's using.
Good catch Anna. Here's what I wrote to Michael: Good catch Michael. Yes, this is a straight up error and oversimplification on my part. The error is that it should be -1.075, not positive 1.075. The oversimplification is that the logistic function (that I show the equation for) varies between 0 and 1, not -1 and 1. You are spot on.
So the function actually squashes it between 0 and 1 rather than -1 to 1?
To figure out the error to weight correspondence function would you first iterate the neural network (by changing weight values and testing the resultant error) a few times the "numerically expensive" way and figure out the error function for each neuron? More generally; how would you go about finding the error function for each neuron? Is the function always parabolic?
I think that in 7:51, the most-right and lowest neuron should be "max activated" when the upper row of pixels is black and the lower row of pixels is white (i.e., the negative of what is displayed in the video, see lowest most-right square of pixels and respective neuron)
Watched it many times but still confused. (1) Why does the 4 pixel start with shaded grey but then the example (from 09:38) use a black and white? (2) At 03:01, how come the weights change from 1.0s to -0.2, 0.0, 0.8, -0.5? Where are these values from?
I have one question. In the chaining example, you assumed 'e' as output. But while describing, you interpreted 'e' as error. Why is that? And also, is it possible to know the error function?
Thanks in advance.
Arnt the axes for the sigmoid function at 4:12 a little messed up? the sigmoid function cant be negative. it's between 0 and 1
Excellently explained !
Automatic subtitle feature enabled would have been nice.
Thanks Tobias! Subtitles for English (and a dozen other languages!) are enabled. I hope they work for you now.
@@BrandonRohrer yes, thx a lot.
Could you make a video about PPO algorythm?
It's on my (very long) list of things I'd like to do.
Hi, great tutorial but I would like to make a clarification: at 7:50 a minute the fourth drawing at the bottom black-white and black, black of the second layer is wrong.
The black and black connections would reverse the design, the connection lines should be white-white.
Then the sigmoid function should not be 0 to 1?
Looks like a TanH,-1 a +1 to me.
Correct me if I'm wrong.
Thank you
ric
wow awesome i never find a video like this with the simple example and clarity of neural network and its a though topic to explain but you make it easy... thanks
Damn, This is perfect. I had kept backing away from all this until now. I finally get it. You really helped me Brandon, thank you.
You're the man at explaining these things. Thank you so much.
This is the best explanation I have ever seen for neural networks. thanks very much!
You should check out 3blue1brown's series on neural networks, its absolutely beautiful