alright, first of all, amazing video's! but i would like to point out that when running this program, in addition to the import numpy as np you corrected in the video yourself, there are a couple of other problems :) on line 7: return 1/(1+np,exp(-x)), the comma after np has to be changed to a dot. on line 10: X = np.array([[0..,1]], the last square bracket has to be removed because it would make for invalid syntax, and the dots should be comma's. on line 28: for j in xrange(60000):, the xrange has to be changed to just range, xrange is not recognized. i guess that's about it, after running it now it works like a charm! thanks again! here are the same lines after the corrections: line 7: return 1/(1+np.exp(-x)) line 10: X = np.array([[0,0,1], line 28: for j in range(60000):
xrange is python2 syntax, which is the same as range in python3. In python2, it's better to use xrange, because it creates a generator instead of range which creates a giant list taking up a ton of memory.
Skwisgaar Skwigelf: Pfft. This is dildos, doesn't he knows there's no such things as religion? Nathan Explosion: You mean you don't believe in God. There IS such thing as religion. Skwisgaar Skwigelf: Well, then proves it! Show me, uh, miracles that religion exists! Nathan Explosion: Well, you know, um... there's the Bible right there. [he points at it] Skwisgaar Skwigelf: Welllll... maybe I reevaluates... my life then.
Then label the video as "in 11 lines", instead of "4 minutes". and take more time in explaining. The video is awesome, but some of us are not fast learners. We need some explaining as well. :)
I feel like the internet should be the safe space wherein I dont have to learn at a snails pace to avoid that social distaste that comes along with self-identity tied to rat race by those whose wits leave them in disgrace?
This is amazing! You managed to squeeze all that info into a 3 min video, but I had to watch it like 5 times, so my mind can keep up with your voice. lol
ye im in grade 11 and in my second year of computer science in high school and we don't do anything interesting and the class moves so slow so your vids are my main source to learn some advanced topics
What do you think about the MOOC "Learning from data" ? I understand the math and theory but there's not a whole lot of application (yet) I'm halfway through.
I do the same. If you ignore music videos I watch nearly all videos on RUclips twice the speed. I would say about three times the speed would be the limit for me in videos with speaking in it but RUclips doesn't offer that.
Also I always make sure I have a spare few ram sticks lying around ready to plug into that DIMM slot on the back of my head for those high intensity moments.
1. import numpy as np 2. deriv=False # should be equal sign in nonlin function. deriv variable's default value. 3. same function 1/(1+np.exp(-x)) # not comma.
I get that your channel is branded by these quick videos, but you should consider doing a follow up "explanatory" video; I think a lot of people would find it helpful
+Siraj Raval Yes I totally agree with this comment and I think many other people will too once this channel scales. I'm a high schooler trying to learn computer science on my own and this stuff is obviously way above my head, but I still find it incredibly enjoyable. However, a more in-depth, elucidative video would be great for people like me to really let the concepts sink in. Your content is great though!
Have you ever experienced frustration in learning something new to you, then you come across a video which gives you that "Ah ha, it all clicks!" moment? This video did that for me, thanks!
would be pretty fucking helpful to mention what the objective of the network is. now it's just "layer layer layer prediction layer layer error DONE next video".
I didn't understand much of the code you shared, but copied it into the IDE and it ran after a little troubleshooting. Now I get to play with it and figure out exactly what each line does. Thank you for putting out a short but working program to explain it. Love your videos!
Here's the code: import numpy as np # sigmoid function def nonlin(x,deriv=False): if(deriv==True): return x*(1-x) return 1/(1+np.exp(-x)) # input dataset X = np.array([ [0,0,1], [0,1,1], [1,0,1], [1,1,1] ]) # output dataset y = np.array([[0,0,1,1]]).T # seed random numbers to make calculation # deterministic (just a good practice) np.random.seed(1) # initialize weights randomly with mean 0 syn0 = 2*np.random.random((3,1)) - 1 for iter in xrange(10000): # forward propagation l0 = X l1 = nonlin(np.dot(l0,syn0)) # how much did we miss? l1_error = y - l1 # multiply how much we missed by the # slope of the sigmoid at the values in l1 l1_delta = l1_error * nonlin(l1,True) # update weights syn0 += np.dot(l0.T,l1_delta) print "Output After Training:" print l1
Couple of suggestions: 1. maybe slow down a bit? A Neural Net built in 6 minutes might be easier to communicate than one built in 4. 2. Also please comment your code more. Your comments are very sparse/high level and provide little to no detail about how backprop is implemented in your NN. Thx!
+Sirajology I feel bad because I'm a senior in my cybersecurity degree and I just took data mining and learned about neural networks but still couldn't keep up at all lol. some High level math you got going on there :c the fact that im still learning Python doesn't help either.
I would also recommend naming your variables better. Why is l0 l0 and not layer0? Why are you using j as your loop counter rather than training_step, which would make your comment just prior to the loop superfluous? X and y are similar, though I imagine I know exactly where they came from. Since you are displaying the code being written in a sped-up fashion, however, making the variable names more readable rather than making them idiomatic in the way neural networks are typically taught by math PhDs would, I think, benefit the audience. Your editor features intelligent autocompletion, I'd recommend taking advantage of it. Also, if the code presented in the video was up on Github or made similarly available in textual format, I could have (and would have) gone through and made these changes myself to show you how much more readable the script could have been... perhaps post the code from your videos on Github and link it in the description?
+Dustin Rodriguez I think that's all pedantic nit-pickery. l0 is perfectly fine in this context for layer 0 for two reasons: 1) It's demonstration code 2) L is the actual symbol used in the scientific field... L0 means layer 0 - people in this field know this. j is perfectly fine for a single loop, what are you talking about man?! Post the code on git hub for you to clean it up?! Do you honestly have nothing better to do :D :D +Sirajology thank you for the concise vid! This is a good starting point.
To answer your question about the superfluous comment: Each *stage* has a title. If we follow your suggestion & rename 'j' to 'training_step' and remove the comment, we a: increase the variable name 13-fold to describe a LOOP variable, and b: we remove the comment from this single stage, making it inconsistent with the rest of your code. Far out... I hope I never have the displeasure of working on a development team with you Dustin :D
I remember watching this back in late May, knowing nothing of what you're going through. After messing around for a short while in python, I have a vague sense of the blur in front of me
I don't know who is this video for - the audience that doesn't understand nn will learn nothing, guys who already understand nn won't benefit from video either. Please slow down or split the topic into smaller chunks.
The title itself is the reason why he got so many views. Guys who genuinely explain the stuff get way fewer views. Check out "giant_neural_network." The guy makes really top notch videos but gets few views. And here we have Siraj, who copied code from iamtrask.github.io/2015/07/12/basic-python-network/ and didn't explain shit.
Wow, it really looks like the code is straight out copied... Hm. Also, thanks for suggesting giant_neural_network! Seems like exactly the channel I've been looking for.
excellent 4 minutes of compressed of information. thank you. i think the picture illustrating the net has different properties than the implementation.
1) Need to add open and closing parentheses to lines 57, 58, 69, 70, 73, and 74. 2) Need to delete the "x" from "xrange" so you just have "range(number_of_training_iterations)" on line 30. That should do 'er. ;)
Hey, isn't your derivative of the sigmoid incorrect ? you have there x*(1-x) but it is supposed to be exp(x)/((exp(x)-1)**2) I applied it and there error rates were lower by about 0.01
The source code is correct, but tricky. The derivative of the sigmoid function is f_sigm'(x)=f_sigm(x) * (1 - f_sigm(x)). When calling it, the number passed is not x, but f_sigm(x). This is much faster, because f_sigm(x) is already calculated and the formula is much simpler. Nice trick, but took some headache to find out what is going on.
"When calling it, the number passed is not x, but f_sigm(x)." facepalm... This was confusing me as well. The original article should at least make this clear, I, of course, don't expect Siraj to :p
Not the best intro to Neural Networks. If you wanted to do it even faster just import TensorFlow or another library and be done with it. Now since you actually wanted to explain the basics behind an NN, 3 minutes won't cut it for the people getting started. As an example, you referred to the Sigmoid function as the function that maps every real number to a number between 0 and 1. Well, I can define an infinite number of functions that satisfy this criterion. The sigmoid is just an example. The Heaviside function works too. You should've thus referred to it as the activation function, and maybe explain why it needs to be differentiable, i.e., to allow for the implementation of a gradient descent backpropagation while training the net. Deep Learning isn't about writing fast code but rather understand what you're doing. Cheers
First, I like that he kept his promise (too many videos say under 4 minutes and it's like 12 minutes long). Second, the speed is actually great for learning efficiently, in that you can just watch it all the way through, and pause rewind where you need to. This makes it really accessible to learners at many different levels so people can easily take it at their own pace. Third, just plain ole good explanation. Bravo. *round of applause*
ERRORS: on line 1: import numpy as numpy, should be import numpy as np on line 7: return 1/(1+np,exp(-x)), the comma after np has to be changed to a dot. on line 10: X = np.array([[0..,1]], the last square bracket has to be removed because it would make for invalid syntax, and the dots should be comma's. on line 28: for j in xrange(60000):, the xrange has to be changed to just range, xrange is not recognized. on line 21: Should not be random.random, should be random.rand() Good video otherwise, I know this is an older vid, and I'm sure your better at cheking for errors! (Or maybe your testing us! Oooo) Keep up the good work!
I get errors such as np not defined, nonlin not defined and xrange not defined. I suppose it's because of python3, but the thing is while I found np and xrange for python3 I couldn't find nonlin. I'd appreciate if you replied with an answer to this :) Edit: Never mind I found out nonlin was function we created not numpy library function, I just misspelled it. Edit 2 : Still doesn't work, problem with np (now numpy). Edit 3: Looked at source code on github, everything works :D Edit 4: Except it really doesn't work, I mean there's no errors, but prediction is around 0.5 when it should be 0 or 1 and error rate doesn't go down. Edit 5: I was missing weight updating for synapses, now it works just great.
Thanks, Siraj. Very engaging. I've upload an annotated, debugged version of the code to github.com/stmorgan/pythonNNexample This includes .py, html and a Jupyter Notebook version.
Imagine making a tutorial on something you don't know yourself. First of all you imported numpy as numpy, not only the "as numpy" part does literally nothing for you, but afterwards you use np instead of numpy, so you would have to have written "import numpy as np" Second the derivative of the sigmoid function is not x*(1-x), it's sigmoid(x)*(1-sigmoid(x)), so that also wouldn't work out. The exact code literally crashes, he literally ran an another script instead of the one he was showing...
This guy is the Usain Bolt of Science. Help! I've just been hit by a bolt of knowledge and my brain is fried. Dude, you're magnificent. Thank your parents for me; they've created a masterpiece. OK, let's try to use an A.I. joke to help us pick up women: Why was the neuron seen storming out of the office? Because it had just been fired. Might send that to Ning Zhang later - my favorite AI geek.
Help me understand what is going on ... I'm going to start learning python but I think I need to learn algebra and calculus too. I really want to do this as a career but I don't know where to start. :(
for this specific "application", yes, I think you need algebra and calculus, since there are concepts like derivatives and matrices. But, if you want to just start learning Python, you don't _absolutely_ need calculus or algebra. Maybe you can start learning the basics of Python first, and then when you have the appropriate mathematical background do more :)
Honestly these are pretty basic calculus concepts. Yes it's called calculus, but most of the actual calculations you would do are still based on algebra. It's the conceptual stuff that would be considered higher level.
Thanks for the help guys. I'm currently learning Python and linear algebra. I already finished a fast tracked calculus course - I can't believe I did that. I guess I'm not as dumb as I thought I was. Either that or things just are easier when you are more interested in them.
Hello i am a neuroscientist but new to neural networking. But i am fascinated to learn networking thing Oops i dont know programming also. So please suggest some lessons,videos,books etc. to starting this wonderful subjects
Why no one suggest languages like JavaScript or Ruby, they are more easier and simpler than a language like C++ to start learning programming I highly endorse Python as well
If you want to get something up and running as fast as possible, choose Ruby, JavaScript, or Python. If you want the maximum functionality possible, choose Java or C++
Hey I got this code working tonight, after all the many modulus where installed, I was self tough on python & and I am relearning it at 39, I think that once more of the modulus are install on the ubuntu 17.04 system the less problems I will have and the code will just run, Keep up the Fun Videos
ah neh. you talk too fast. whats the point of spitting so fast and no one can grab what you talk. for those already know what you talking, they wont even be watching you spit.
Initially the video seems very complex. The speed up the already fast video to 2x (atleast). Then watch the coding part on loop. After watching it 4 times I understood the code perfectly. It saves time. repetitions matter. Siraj, when can we expect new videos?
FYI, you don't have to wrap in brackets with if statements, or any control statement in Python for that matter. So `if(deriv==True)` can become `if deriv==True` which I think is easier to read.
There is a "input layer" which just estimulate a neuron of the next layer, this could be he sensors for example. That second neuron when is estimulated it directly estimulate another layer (this is the layer which changes over time). And the next layer just outputs data like for example jump or press a button. Is this how it works?
I actually have neural networks subject on the 6th semester of computer scientist bachelor (so in the last) and what you said in this 3 minutes was equal to the past month of lectures (roughly 6 hours total time) and made it witha clearer code and better explanation even if it wasn't on my native language. You also made me way more enthusiastic about the topic as i see it's not impossible. Looking forward to study more from your videwos :D
Just want to recommend that you should have a series or playlist of this learning algorithms and explain it deeply Theoretically and Mathematically speaking! Especially Neural Networks, people may think that its just plain simple but it has a deep background behind it and I know that you know it. Anyway, I love your videos keep doing great more things about Machine Learning and looking forward to your future videos.
I think the sigmoid is not really used anymore. You just use something like max(0,a) where a=Wx+b. If what you get is greater than 0 neuron activates. The problem with sigmoid is everything get's just too close to 1 or 0 if it's large.
Thank you so much for this! I love this style. It tells me everything I need to know and I can learn those things at my own pace by watching other videos about it.
There's a great book out there that does an awesome job at explaining every single line of python code for deep learning and artificial intelligence. It's called "Artificial Intelligence and Deep Learning with Python Every Line of Code Explained "
wait how do I get the pyton version like yours mine looks like the one that you get when downloading windows and the one they use in the matrix with the green letters and black background I forgot what it's called?
if you watch it at 2x speed you can build your network in 2 minutes!!!!!
thats my man
lol
I didn't get it at 0,75 speed how it would look when it will bombard you with so much information xD
If I watch it at 1/8 speed I might be able to understand .01% of it
LoL best comment Ever😂😂😂
alright, first of all, amazing video's! but i would like to point out that when running this program, in addition to the import numpy as np you corrected in the video yourself, there are a couple of other problems :)
on line 7: return 1/(1+np,exp(-x)), the comma after np has to be changed to a dot.
on line 10: X = np.array([[0..,1]], the last square bracket has to be removed because it would make for invalid syntax, and the dots should be comma's.
on line 28: for j in xrange(60000):, the xrange has to be changed to just range, xrange is not recognized.
i guess that's about it, after running it now it works like a charm!
thanks again!
here are the same lines after the corrections:
line 7: return 1/(1+np.exp(-x))
line 10: X = np.array([[0,0,1],
line 28: for j in range(60000):
thank you for this! This comment needs more upvotes, great corrections
+Sirajology Thanks, that means a lot to me. I just hope you keep adding more video's! they are all very informative.
xrange is python2 syntax, which is the same as range in python3. In python2, it's better to use xrange, because it creates a generator instead of range which creates a giant list taking up a ton of memory.
You can mark the comment to stay at the top
One more thing.
line 3 correction: def nonlin(x, deriv=False)
*walks away from computer and starts reconsidering life choices*
very good
This seriously made me LOLLLLLLL.
😂
Skwisgaar Skwigelf: Pfft. This is dildos, doesn't he knows there's no such things as religion?
Nathan Explosion: You mean you don't believe in God. There IS such thing as religion.
Skwisgaar Skwigelf: Well, then proves it! Show me, uh, miracles that religion exists!
Nathan Explosion: Well, you know, um... there's the Bible right there.
[he points at it]
Skwisgaar Skwigelf: Welllll... maybe I reevaluates... my life then.
Cristian Soare Lol
the people saying "you should slow down" are not aware that they probably wouldn't have clicked on a video titled "build a neural net in 24 minutes"
I would. In fact, I'd prefer a 10-hour version.
But I wouldn't. High schoolers don't have 10 hours on hand lol.
Then label the video as "in 11 lines", instead of "4 minutes". and take more time in explaining.
The video is awesome, but some of us are not fast learners. We need some explaining as well. :)
I feel like the internet should be the safe space wherein I dont have to learn at a snails pace to avoid that social distaste that comes along with self-identity tied to rat race by those whose wits leave them in disgrace?
Nand Fednu do you identify as a PJW?(programming justice warrior)
watch for 3 minutes, understand for 3 hours
hahah
I'm trying to recreate this in Java... Figuring out how to multiply arrays is the hardest part. I don't have NumPy doing it for me.
@Juubes recreating this in C# and I feel your pain
good job :D I ended up using a math library to do the job
Synetic -Have you been able to recreate this in C# yet? If so, please post the code. Thanks !!!!
This is amazing! You managed to squeeze all that info into a 3 min video, but I had to watch it like 5 times, so my mind can keep up with your voice. lol
Thanks! Yeah I tried, i'll definitely make more of em
check this blog to read more about the code
iamtrask.github.io/2015/07/12/basic-python-network/
completes python class
realizes knows nothing about python
dude keep watching my vids, i want to be the go to source for learning python for ML
ML?
Machine learning
ye im in grade 11 and in my second year of computer science in high school and we don't do anything interesting and the class moves so slow so your vids are my main source to learn some advanced topics
What do you think about the MOOC "Learning from data" ? I understand the math and theory but there's not a whole lot of application (yet) I'm halfway through.
I see many people complaining, but this is not the speed youtubers deserve. It is the speed youtubers need.
Always watch at 2x speed to learn 2x faster. Time is precious!
so precious
I do the same. If you ignore music videos I watch nearly all videos on RUclips twice the speed. I would say about three times the speed would be the limit for me in videos with speaking in it but RUclips doesn't offer that.
More fast more close to God. Wait till your nose got bleeding. Then you'll learn how to fly
Here's a trick I learned from a friend playing poker online: open several browser windows. Learn in parallel x10 faster !
Also I always make sure I have a spare few ram sticks lying around ready to plug into that DIMM slot on the back of my head for those high intensity moments.
1. import numpy as np
2. deriv=False # should be equal sign in nonlin function. deriv variable's default value.
3. same function 1/(1+np.exp(-x)) # not comma.
awesome, was just reading the comments to fix these errors!
Same
yeah he made a ton of mistakes... apparently when he ran the code after he was running a pre-written file instead of the one he was showing us
thanks....i was searching for corrections
Thanks ! I couldn't figure out what was up with error 3
I get that your channel is branded by these quick videos, but you should consider doing a follow up "explanatory" video; I think a lot of people would find it helpful
thanks Troy. I want the live streams to be that, but they aren't really connected directly to the quick videos. I can do that more often
please do more of those explanatory vids,coz these ones are just too fast my man
+Siraj Raval Yes I totally agree with this comment and I think many other people will too once this channel scales. I'm a high schooler trying to learn computer science on my own and this stuff is obviously way above my head, but I still find it incredibly enjoyable. However, a more in-depth, elucidative video would be great for people like me to really let the concepts sink in. Your content is great though!
Have you ever experienced frustration in learning something new to you, then you come across a video which gives you that "Ah ha, it all clicks!" moment? This video did that for me, thanks!
would be pretty fucking helpful to mention what the objective of the network is. now it's just "layer layer layer prediction layer layer error DONE next video".
good point thanks
To answer the implied question: The objective is to get your prediction to equal the correct output - [0,1,1,0]
Matthew Harris thanks man
I didn't understand much of the code you shared, but copied it into the IDE and it ran after a little troubleshooting. Now I get to play with it and figure out exactly what each line does. Thank you for putting out a short but working program to explain it. Love your videos!
1:17
the first line of code should be
import numpy as np
and at line 8 is
return 1/(1+np.exp(-x))
at line 12
X = np.array([0, 0, 1],
I have watched so many tutorials, but never fully understood the logic, but this example is done so simple and brilliant, that i finally got it.
Here's the code:
import numpy as np
# sigmoid function
def nonlin(x,deriv=False):
if(deriv==True):
return x*(1-x)
return 1/(1+np.exp(-x))
# input dataset
X = np.array([ [0,0,1],
[0,1,1],
[1,0,1],
[1,1,1] ])
# output dataset
y = np.array([[0,0,1,1]]).T
# seed random numbers to make calculation
# deterministic (just a good practice)
np.random.seed(1)
# initialize weights randomly with mean 0
syn0 = 2*np.random.random((3,1)) - 1
for iter in xrange(10000):
# forward propagation
l0 = X
l1 = nonlin(np.dot(l0,syn0))
# how much did we miss?
l1_error = y - l1
# multiply how much we missed by the
# slope of the sigmoid at the values in l1
l1_delta = l1_error * nonlin(l1,True)
# update weights
syn0 += np.dot(l0.T,l1_delta)
print "Output After Training:"
print l1
Every frame has new content. No bullshit. That's awesome
This video helped me understand today's dark art
Glad I could help!
bro import numpy as numpy?
Lol
lolllllllll
bro that's not the worst part.
I mean, how many syntax errors can one make? Turns out, 5.
He is importing numpy as alias numpy ..nothing wrong if we don't use the alias np
Hahah
Wow, okay, so I see all the steps you did, but have absolutely no idea why you did any of it.
Thank you for posting the suggested articles in your description. Those are helping out soooo much for understanding neural networks!!!!
Couple of suggestions: 1. maybe slow down a bit? A Neural Net built in 6 minutes might be easier to communicate than one built in 4. 2. Also please comment your code more. Your comments are very sparse/high level and provide little to no detail about how backprop is implemented in your NN. Thx!
great feedback, will do thanks
+Sirajology I feel bad because I'm a senior in my cybersecurity degree and I just took data mining and learned about neural networks but still couldn't keep up at all lol. some High level math you got going on there :c the fact that im still learning Python doesn't help either.
I would also recommend naming your variables better. Why is l0 l0 and not layer0? Why are you using j as your loop counter rather than training_step, which would make your comment just prior to the loop superfluous? X and y are similar, though I imagine I know exactly where they came from. Since you are displaying the code being written in a sped-up fashion, however, making the variable names more readable rather than making them idiomatic in the way neural networks are typically taught by math PhDs would, I think, benefit the audience. Your editor features intelligent autocompletion, I'd recommend taking advantage of it.
Also, if the code presented in the video was up on Github or made similarly available in textual format, I could have (and would have) gone through and made these changes myself to show you how much more readable the script could have been... perhaps post the code from your videos on Github and link it in the description?
+Dustin Rodriguez I think that's all pedantic nit-pickery. l0 is perfectly fine in this context for layer 0 for two reasons:
1) It's demonstration code
2) L is the actual symbol used in the scientific field... L0 means layer 0 - people in this field know this.
j is perfectly fine for a single loop, what are you talking about man?! Post the code on git hub for you to clean it up?! Do you honestly have nothing better to do :D :D
+Sirajology thank you for the concise vid! This is a good starting point.
To answer your question about the superfluous comment:
Each *stage* has a title. If we follow your suggestion & rename 'j' to 'training_step' and remove the comment, we a: increase the variable name 13-fold to describe a LOOP variable, and b: we remove the comment from this single stage, making it inconsistent with the rest of your code.
Far out... I hope I never have the displeasure of working on a development team with you Dustin :D
I remember watching this back in late May, knowing nothing of what you're going through. After messing around for a short while in python, I have a vague sense of the blur in front of me
I don't know who is this video for - the audience that doesn't understand nn will learn nothing, guys who already understand nn won't benefit from video either. Please slow down or split the topic into smaller chunks.
The title itself is the reason why he got so many views. Guys who genuinely explain the stuff get way fewer views. Check out "giant_neural_network." The guy makes really top notch videos but gets few views. And here we have Siraj, who copied code from iamtrask.github.io/2015/07/12/basic-python-network/ and didn't explain shit.
Wow, it really looks like the code is straight out copied... Hm.
Also, thanks for suggesting giant_neural_network! Seems like exactly the channel I've been looking for.
@@nahiyanalamgir7614 yeah giant neural network is the best
excellent 4 minutes of compressed of information. thank you. i think the picture illustrating the net has different properties than the implementation.
thanks!!
i hope i'm not the only one who has set the "video speed" to 0.5 :P one subscribed from italy
lol ciao!
0.75 sounds like normal talking^
I did it too lol
0.75
MatteoProgrammer ITALIAAAAA
Thank you Siraj Raval, I'm really grateful of your work, I'm catching up on your videos; from South Africa.
He even playgiarizes this from udeemy course. Shameless scaammer
1) Need to add open and closing parentheses to lines 57, 58, 69, 70, 73, and 74.
2) Need to delete the "x" from "xrange" so you just have "range(number_of_training_iterations)" on line 30.
That should do 'er. ;)
Those who didn't understand, seemingly similar code can be found here with explanation.
iamtrask.github.io/2015/07/12/basic-python-network/
Thanks man, awesome.
Instructions not clear, created doomsday robot that wants to kill everyone
few mistakes in the code. using periods as commas and vice versa, using np when you imported numpy as numpy
This video is faked .....The code doesn't work and he ran a different file than what he typed.
Hey, isn't your derivative of the sigmoid incorrect ? you have there x*(1-x) but it is supposed to be exp(x)/((exp(x)-1)**2) I applied it and there error rates were lower by about 0.01
The source code is correct, but tricky.
The derivative of the sigmoid function is f_sigm'(x)=f_sigm(x) * (1 - f_sigm(x)).
When calling it, the number passed is not x, but f_sigm(x).
This is much faster, because f_sigm(x) is already calculated and the formula is much simpler.
Nice trick, but took some headache to find out what is going on.
"When calling it, the number passed is not x, but f_sigm(x)." facepalm... This was confusing me as well. The original article should at least make this clear, I, of course, don't expect Siraj to :p
Nice breakdown Siraj.
Not the best intro to Neural Networks. If you wanted to do it even faster just import TensorFlow or another library and be done with it. Now since you actually wanted to explain the basics behind an NN, 3 minutes won't cut it for the people getting started. As an example, you referred to the Sigmoid function as the function that maps every real number to a number between 0 and 1. Well, I can define an infinite number of functions that satisfy this criterion. The sigmoid is just an example. The Heaviside function works too. You should've thus referred to it as the activation function, and maybe explain why it needs to be differentiable, i.e., to allow for the implementation of a gradient descent backpropagation while training the net. Deep Learning isn't about writing fast code but rather understand what you're doing. Cheers
your activation function have some syntax error, how it can even compile/interpret by interpreter in one go?? ==> line number 8
Did u started like this? And understood forward propagation, back propagation, gradient descent..... In just 4 mins?
+Siraj Raval please make a series of how to program something application specific. youre the fuckin man bro
will come soon thx Mike
"A refrigerator filled with lots of foods and drinks"
You are a rapper in a scientist body. This is the most information received in under 4 mj .
thanks! u get me
First, I like that he kept his promise (too many videos say under 4 minutes and it's like 12 minutes long). Second, the speed is actually great for learning efficiently, in that you can just watch it all the way through, and pause rewind where you need to. This makes it really accessible to learners at many different levels so people can easily take it at their own pace. Third, just plain ole good explanation. Bravo. *round of applause*
ERRORS:
on line 1: import numpy as numpy, should be import numpy as np
on line 7: return 1/(1+np,exp(-x)), the comma after np has to be changed to a dot.
on line 10: X = np.array([[0..,1]], the last square bracket has to be removed because it would make for invalid syntax, and the dots should be comma's.
on line 28: for j in xrange(60000):, the xrange has to be changed to just range, xrange is not recognized.
on line 21: Should not be random.random, should be random.rand()
Good video otherwise, I know this is an older vid, and I'm sure your better at cheking for errors! (Or maybe your testing us! Oooo) Keep up the good work!
I get errors such as np not defined, nonlin not defined and xrange not defined. I suppose it's because of python3, but the thing is while I found np and xrange for python3 I couldn't find nonlin. I'd appreciate if you replied with an answer to this :)
Edit: Never mind I found out nonlin was function we created not numpy library function, I just misspelled it.
Edit 2 : Still doesn't work, problem with np (now numpy).
Edit 3: Looked at source code on github, everything works :D
Edit 4: Except it really doesn't work, I mean there's no errors, but prediction is around 0.5 when it should be 0 or 1 and error rate doesn't go down.
Edit 5: I was missing weight updating for synapses, now it works just great.
awesome
That moment that you didn't code with Python in years and you updated from 2.7 to 3.7 and you're stuck in a print function ( *FOR AN HOUR* )
He imported numpy as numpy and then used np 😂
I did this in MATLAB because I don't know python. Glad it worked.
working code here (and you may have to install numpy in terminal with the command 'pip install numpy' and i also updated it after that):
import numpy as np
def nonlin(x,deriv=False):
if(deriv==True):
return x*(1-x)
return 1/(1+np.exp(-x))
#input data
X = np.array([[0,0,1],
[0,1,1],
[1,0,1],
[1,1,1]])
#output data
y = np.array([[0],
[1],
[1],
[0]])
np.random.seed(1)
#synapses
syn0 = 2*np.random.random((3,4)) - 1
syn1 = 2*np.random.random((4,1)) - 1
#training step
for j in xrange(60000):
l0 = X
l1 = nonlin(np.dot(l0, syn0))
l2 = nonlin(np.dot(l1, syn1))
l2_error = y - l2
if(j % 10000) == 0:
print ("Error:" + str(np.mean(np.abs(l2_error))) )
l2_delta = l2_error*nonlin(l2, deriv=True)
l1_error = l2_delta.dot(syn1.T)
l1_delta = l1_error * nonlin(l1, deriv=True)
#update weights
syn1 += l1.T.dot(l2_delta)
syn0 += l0.T.dot(l1_delta)
print ("Output after training")
print (l2)
thanks for posting this :)
Thanks, Siraj. Very engaging.
I've upload an annotated, debugged version of the code to github.com/stmorgan/pythonNNexample
This includes .py, html and a Jupyter Notebook version.
scott you da man. added it to the description thank you
No problem. Thanks for the nod!
0:45 "import numpy as numpy" really? so much faster now that you got it down to the same number of letters lol
damn that's just pretty smart
why am saying something to a 3-year-old comment
PYTHON 3 code:
import numpy as np
def nonlin(x,deriv=False):
if(deriv == True):
return x*(1-x)
return 1/(1+np.exp(-x))
#input data
X = np.array([[0,0,1],
[0,1,1],
[1,0,1],
[1,1,1]])
#output data
y=np.array([[0],
[1],
[1],
[0]])
np.random.seed(1)
#synapses
syn0 = 2*np.random.random((3,4)) - 1
syn1 = 2*np.random.random((4,1)) - 1
#training step
for j in range(60000):
l0 = X
l1 = nonlin(np.dot(l0,syn0))
l2 = nonlin(np.dot(l1,syn1))
l2_error = y -l2
if(j % 10000) == 0:
print("Error:" + str(np.mean(np.abs(l2_error))))
l2_delta = l2_error*nonlin(l2, deriv=True)
l1_error = l2_delta.dot(syn1.T)
l1_delta = l1_error * nonlin(l1, deriv=True)
#update weights
syn1 += l1.T.dot(l2_delta)
syn0 += l0.T.dot(l1_delta)
print("Output after training")
print(l2)
ohh so...potato?
No potato
potato
Very cool. I like how you home brewed this.
This is to difficult for me to understand. Do you have simpler videos to start with?
i'll work on it
Imagine making a tutorial on something you don't know yourself.
First of all you imported numpy as numpy, not only the "as numpy" part does literally nothing for you, but afterwards you use np instead of numpy, so you would have to have written "import numpy as np"
Second the derivative of the sigmoid function is not x*(1-x), it's sigmoid(x)*(1-sigmoid(x)), so that also wouldn't work out.
The exact code literally crashes, he literally ran an another script instead of the one he was showing...
Incredibly well explained in such a short amount of time! thank you!
Thanks Bryan!
This guy is the Usain Bolt of Science. Help! I've just been hit by a bolt of knowledge and my brain is fried.
Dude, you're magnificent. Thank your parents for me; they've created a masterpiece.
OK, let's try to use an A.I. joke to help us pick up women:
Why was the neuron seen storming out of the office? Because it had just been fired.
Might send that to Ning Zhang later - my favorite AI geek.
i just told my dad someone said to say thank you. he appreciated it. lol thanks for watching. love the joke!
wait, this is python 2.X, not python 3.X, can you update this for python 3.X?
xrange->range
pastebin.com/3HLCGFwC
here you go, completely for python 3, and without all the syntax errors.
@@RiedlerMusics OMG thanks!
Amazing! It's so simple to build a (simple) neural network, actually. Thanks!
anytime!
Help me understand what is going on ... I'm going to start learning python but I think I need to learn algebra and calculus too. I really want to do this as a career but I don't know where to start. :(
for this specific "application", yes, I think you need algebra and calculus, since there are concepts like derivatives and matrices. But, if you want to just start learning Python, you don't _absolutely_ need calculus or algebra. Maybe you can start learning the basics of Python first, and then when you have the appropriate mathematical background do more :)
Honestly these are pretty basic calculus concepts. Yes it's called calculus, but most of the actual calculations you would do are still based on algebra. It's the conceptual stuff that would be considered higher level.
Thanks for the help guys. I'm currently learning Python and linear algebra. I already finished a fast tracked calculus course - I can't believe I did that. I guess I'm not as dumb as I thought I was. Either that or things just are easier when you are more interested in them.
That's the beauty of applied learning. Knowledge that actually be applied to something useful.
how did you learn linear algebra? I think I need to learn that to so i can understand the video.
Wow You Really Did Steal People Work Word For Word :/
I'm Happy Your Entire Job Is Over :)
He crazy?
just passionate
Copying code from iamtrask.github.io/2015/07/12/basic-python-network/ and vaguely explaining anything.
Lol Eyan good catch. Too many ghosts on youtube and besides that not even credit is given
instructions unclear, i now have *FoOT fUnGeS*
Hello
i am a neuroscientist but new to neural networking. But i am fascinated to learn networking thing
Oops i dont know programming also. So please suggest some lessons,videos,books etc. to starting this wonderful subjects
either learn python or c++
Thanks macha.
Why no one suggest languages like JavaScript or Ruby, they are more easier and simpler than a language like C++ to start learning programming
I highly endorse Python as well
If you want to get something up and running as fast as possible, choose Ruby, JavaScript, or Python. If you want the maximum functionality possible, choose Java or C++
Even though I know a bit of Matlab and Python, I still use R for this sort of thing lol.
Hey I got this code working tonight, after all the many modulus where installed, I was self tough on python & and I am relearning it at 39, I think that once more of the modulus are install on the ubuntu 17.04 system the less problems I will have and the code will just run, Keep up the Fun Videos
ah neh. you talk too fast. whats the point of spitting so fast and no one can grab what you talk. for those already know what you talking, they wont even be watching you spit.
Initially the video seems very complex. The speed up the already fast video to 2x (atleast). Then watch the coding part on loop.
After watching it 4 times I understood the code perfectly. It saves time. repetitions matter.
Siraj, when can we expect new videos?
Don't follow this dude.. he doesn't understand what he's talking about
@@io1921 haha
FYI, you don't have to wrap in brackets with if statements, or any control statement in Python for that matter. So `if(deriv==True)` can become `if deriv==True` which I think is easier to read.
Or even 'if deriv:', since deriv is a boolean. It's much cleaner, and a little faster.
alexthi94 Good point.
good point
You explain for 4 minutes. I understand for 4 Years
# awesome but,
import numpy as np
derive = "True" # always True
np.exp( -x ) # instead np,exp( -x )
#then will work
*Corrected code:*
import numpy as np
def nonlin(x, deriv=False):
if deriv:
return x * (1 - x)
return 1 / (1 + np.exp(-x))
# input data
x = np.array([[0, 0, 1], [0, 1, 1], [1, 0, 1], [1, 1, 1]])
# output data
y = np.array([[0], [1], [1], [0]])
np.random.seed(1)
# synapses
syn0 = 2 * np.random.random((3, 4)) - 1
syn1 = 2 * np.random.random((4, 1)) - 1
# training step
for j in range(60000): # use range in Python 3
l0 = x
l1 = nonlin(np.dot(l0, syn0))
l2 = nonlin(np.dot(l1, syn1))
l2_error = y - l2
if (j % 10000) == 0:
print("Error:" + str(np.mean(np.abs(l2_error)))) # use parentheses with print in Python 3
l2_delta = l2_error * nonlin(l2, deriv=True)
l1_error = l2_delta.dot(syn1.T)
l1_delta = l1_error * nonlin(l1, deriv=True) # corrected variable name typo
# update weights
syn1 += l1.T.dot(l2_delta)
syn0 += l0.T.dot(l1_delta) # corrected variable name typo
print("Output after training") # use parentheses with print in Python 3
print(l2)
4mins but video is 3:27.....
Clickbait: Am I a joke to you?
X = np.array([0,0,1]) instead of X = np.array([0.0,1])
Ok this showed up in my recommendation! I found the crazy Siraj!
Stole this code too.
There is a "input layer" which just estimulate a neuron of the next layer, this could be he sensors for example.
That second neuron when is estimulated it directly estimulate another layer (this is the layer which changes over time). And the next layer just outputs data like for example jump or press a button. Is this how it works?
I actually have neural networks subject on the 6th semester of computer scientist bachelor (so in the last) and what you said in this 3 minutes was equal to the past month of lectures (roughly 6 hours total time) and made it witha clearer code and better explanation even if it wasn't on my native language. You also made me way more enthusiastic about the topic as i see it's not impossible. Looking forward to study more from your videwos :D
Just want to recommend that you should have a series or playlist of this learning algorithms and explain it deeply Theoretically and Mathematically speaking! Especially Neural Networks, people may think that its just plain simple but it has a deep background behind it and I know that you know it. Anyway, I love your videos keep doing great more things about Machine Learning and looking forward to your future videos.
Thanks a lot for the advice and the compliment!
your explanation sounds like .. "for 4 mins dogs fighting for a chicken piece "
I think the sigmoid is not really used anymore. You just use something like max(0,a) where a=Wx+b. If what you get is greater than 0 neuron activates. The problem with sigmoid is everything get's just too close to 1 or 0 if it's large.
first off have anyone tried to run this code ??????
Yep as I imagined....
No one comented typying proposital errors
@Alessandro B i tried running this code on pycharm it does not work bcoz o a type error
Quick question: how many hidden layers does our neural network have?
But the derivative of the sigmoid function is sigmoid(x)*(1 - sigmoid(x)) right? using this gave a even better/faster result.
Thank you so much for this! I love this style. It tells me everything I need to know and I can learn those things at my own pace by watching other videos about it.
Yo, the code you wrote in the video is not the code you are running. WTH?
Don't follow this dude.. he doesn't understand what he's talking about
Thank you so much Siraj!! You are the man!
Can you explain it to me like I'm not a genius? hahaha
Great content dude. :)
yes a new version of this is coming very soon thanks
^_^
There's a great book out there that does an awesome job at explaining every single line of python code for deep learning and artificial intelligence. It's called "Artificial Intelligence and Deep Learning with Python Every Line of Code Explained "
I program you to say that
Some intuition tells me this is a great introductory video for AI,
but consciously, I didn't understand a word of it.
...and subscribed +1
love the whole format, thanks for showing how this is done
thanks Chris, more to come like this always open to suggestions
I subscribed and I am wondering why everyone is in a hurry. Nobody can slow down any more.
Great work!
import numpy as numpy just why ?
Hello man
Your videos are more confusing then informative.
Anyways thanks for a litttttttttle information your videos give
wait how do I get the pyton version like yours mine looks like the one that you get when downloading windows and the one they use in the matrix with the green letters and black background I forgot what it's called?
Hey, siraj, the github that you share is not the same as the code you made.
Thanks for amazing videos. Greetings
Can you give me full neural network script in word format
So in 4 minutes huh,learn how to build it in 3:27 and build it in the rest 33 seconds🤔,jk nice vid tho