AI Learns to play Snake!
HTML-код
- Опубликовано: 21 сен 2019
- Using a Genetic Algorithm and Neural Network, a population of snakes are able to gradually learn to play the game with no prior knowledge!
Link to code: github.com/Chrispresso/SnakeAI
Music: / ashamaluevmusic
If you listen closely, you can hear Code Bullet wailing and having an aneurism off in the distance.
@Dario Roman How and in what way does your shilling have ANY relevance to either this video or my comment on the suffering of a specific programmer? I'll give you a hint: it has none.
Also a trick: Piracy is not only wrong, it's unlawful. You shouldn't be shilling that shit site, or any others like it for that matter.
@Kaiser Felipe Another account, another report. Keep them coming, I enjoy shitting in your cheerios when you think it's a good idea to try to advertise in my comments, especially with a piracy site. Quit being a basement dweller and and actually buy shit. You might find you might help the economy a little.
@@chaoseclipse0121 hahahah its cute that you think your two reports will actually bring his chanel down , the kid is not only learning and applying knowlege but also make the effort to teach others . While u practice how to make multiple gmail accounts to report ppl he is probably learning more ai stuff. I would suggest u learn how to code fand create a neural network to detect plagarism . Probs ask @
Chrispresso to teach u how to make one , cause the extend of knowledge is very limited
@@iam_devansh Riiiiiight, all this coming from someone who can't be bothered to spend an extra minute or two to use proper grammar and spelling, on top of praising a moron who uses a bot to advertise a PIRACY website and then having the gall to tell me to learn something from criminal scumbags while lacking the willpower to type properly on the internet. I wouldn't be surprised in the lightest if you're the same person who owns the other two accounts, then turning around and giving yourself praise while thinking you're telling people off who challenge or insult you, because there's nothing praiseworthy of your typing skills and your praise of criminals.
You should sit down and seriously consider your life choices. I again reiterate that piracy is not only wrong, it's unlawful. It will come back to bite you in the ass one of these days, and no VPN will protect you when you come under investigation as VPN's are required by law to turn over data and logs when it comes to criminal investigations. The fact you're logged into an account and making posts leaves your IP linked to your account removes whatever "safety" you think you have.
@@iam_devansh I cant see who the OP is talking to. Seems his posts got deleted. Maybe he did that himself, or maybe you're wrong and his account got deleted.
Just thought youd like to know.
Starting the ai in a small area was definitely the key to its success. I have seen a few of these snake ai’s and they always struggle with avoiding their own tail. By putting the ai in a small area, you’re forcing it to deal with the lack of space early on and it overcomes the problem much better.
good observation
This is the first Snake AI simulation that actually finishes the game
It is because it is the first one to use multiple hidden layers.
Probably because the person who made spent a lot longer actually training it rather than just a a couple hours or a day
There's no time or step constraints.
@@luigisgl2639 the problem is though, it's not very fast. It just goes in a pre-defined pattern which is safe, but terribly inefficient
@@williamluong7743 yes. Couldn't a dumb algo be written that goes top to one row from the bottom, up and down moving right by one colum each time then taking the bottom row back to the left most colum and starting again? That would always win wouldn't it? It would also always win on the same squarebottom left one row up.
I especially appreaciate the part where you explained the input nodes. Most of the videos are missing that part.
I just now read your blog and implementation...great work man!
I have never felt so outsmarted by a snake in my life,
Also maybe give the snakes a reward for being quicker to each apple to encourage a speedier strategy for larger grids
It needs to be scaled to the current length though, since more efficient pathing is OK when short, but longer snakes actually need to zigzag to "store" that length so it won't unalive itself
You are the first one one that i seen making a Snake AI that is not failing or using tricks to finish the game bravo
Absolutely incredible. Amazing video, my congratulations!
It would be really interesting to take the genetic makeup that succeeded in winning the game and training it with a fitness function that prioritizes speed/effeciency and then try again on the large grids.
Awesome vid.
This video was (or more will be, dont have the time today) so helpfull improving my NEAT. I only had a Highscore of about 4-5 in Generation 3300
I thank you really much
nice job man! it was harder than I thought making the representation. Really clever solution getting the direction account so that the snake knows that when the head goes to where the tail is currently it will not crash into itself :)
Thanks :) Glad you appreciated it!
It was beautiful to watch! Well done
And...It is fantastic! congrats!
this is THE most underrated video and channel on RUclips.
Glad you enjoy it :) I'm working on more as well, and if you have ideas for AI's or games you want to see, I'm open to suggestions.
Sure! How about a Deep Learning algorithm to play Tetris?
@@idr7789 I will definitely look into this to see what I can come up with!
there are even more
You are the best, Gratz Chrispresso
Awesome video!
Thank you for your insights 😊
i Love these AI Channels they're always so Fun to watch
Glad you enjoy them!
Thanks for your video! That's so good, congratulations
Thank you! Great explaination!
Great job man!
Great demo. Would it be possible to do "brain surgeory" on one of last snakes in order to see how changeing vision changes output and maybe make an interpretation of nodes in the hidden layers?
Nice work!
Back in the 80s there was a multiplayer version of Snake for the Commodore 64 which was a type-in game in RUN Magazine, although the setup was that each snake would have a rule set for which direction to move given the surrounding 8 cells, and the point of the game was to program your snake with a rule set that made it outlast all the other snakes. Being the kind of nerdy kid I was I decided to see what happened if I randomly combined the rule sets of the two best-performing snakes, basically coming up with genetic algorithms on my own, and it got pretty good... then I let it also use the human player’s rule sets, and it got scary good.
So anyway I appreciate this video.
well done fluffy. keep it up
Unexpected awesome sauce!
Great work!
Thanks!
what a great work. well done
Thank you!
Amazing!! I am speechless!!
AI is so cool, it's amazing how many applications it has and how easily accessible it is for people.
Yep! I'm hoping to continue to show how and where it can be used. I know AI is a hot term these days, but I think understanding it and showing it in games makes it a lot more relatable for people.
Great video
Wow, man. You and your brilliant snake did well. make ai on more complex games!
Glad you enjoyed :) Definitely working on some, just need to find some more time. I'm also open to suggestions if you have any games in mind!
😏
Nice! I wonder how they would behave if there were two or more snakes competing for the same apples.
It would definitely make it so the more aggressive snake would get some of the early apples. I'm not sure how it would change the population once the snakes got to a larger size. I might have to test it out!
Chrispresso I would guess that some snakes will try to kill its opponent(s) to be able to get more apples.
@@xhargh well, trying to kill its opponent means that generation has failed. Similarly, the 'victim' generation would learn that dying by attack is a bad idea. I reckon the game would tend toward a scenario whereby both snakes survive for a maximum time by unintentionally collaborating to maximise each one's own chance of getting an apple.
@@Takumi-San might be so, it could also depend on whether it is one kind of snake or different kinds of snakes competing.
I think the snakes would learn to cooperate
I wish I know how you came up with the formula & the training layers. The only two parts that were not explained. Thank you very much. It feel like watching a baby walk and go to college then graduates. I felt very connected to the snake.
This is so cool!
Nice video. Thanks for sharing your code. I'm working on my own snake game.
Of course! Glad you liked it. Any project I post will always contain source, just FYI :)
This was amazing.
Amazing!
Me : nothing scares me, but those blue red lines, those scares me
I just love these AI videos!! But may i request something? I notice a lot of people usually remake the games, and then trains an AI to it. And here it appeared as if you gave the snake eyes, but when humans play, they look at the whole map. So then, could you try sometime to make an learning AI to an original game, that reads the screen as a human but hopefully doesn't run too slow because of the image it has to process?
Either way im definitelly looking forward to more videos in the future!
Yep! This is actually my plan. I know I'm currently slow at making videos but hopefully I pick up the speed this year. I've got a lot planned, so fingers crossed. I just wanted to start the AI intro slow by first showing how to use a neural network + genetic algorithm on something like a game I made replicating snake. Then I made it more advanced by doing the same thing with car evolution and then having it play a game that I didn't change (super mario bros). Recently I made it learn grid world by just using pixels from the screen. So next up is combining some of these techniques ;) My plan was to add AI ideas slowly so that people can always just watch the "AI Learns" series in order to see how to add different components, or to skip around and find an AI solution they might like.
@@Chrispresso could you please try and give TIME a high weight in the fitness of a snake. It will be nice to see what new strategies they come up with to win the game quickly.
it would be slither.io then
It would require a CNN/RNN combination, and it wouldn't be trainable by genetic algorithm anymore.
Its awesome man
Love this video OMG best A.I. snake explanation
Glad you liked it!
amazing!
Nice job!!!!
the best ai learning video I have watched so far, nice job!
darn those 2 dislikes :V
Glad you liked it!
Awesome! I think you should put a metric to decrease unnecessary movements, it will increase snake's efficiency.
It was simply amazing
What a great video!! :O :')
Dude, this is just straight up awesome. I need to do more of this stuff on my channel. I don't get exactly how the binary sensing works though.
Glad you like it! Binary sensing is just a simple yes or no (1 or 0) fed into the network. Either it sees a particular object or it doesn't.
@@Chrispresso Thanks for answering my question. So, the binary sensing that you set up doesn't give any distance information? If so, its scary impressive that you were able to get that to work with the binary sensing.
@@scottbigbrain3944 That's correct. I trained with both types to see what would happen and that's what is shown in the graphs. I was impressed that something as simple as binary sensing was able to solve it.
This stuff is so amazing. It learned - and then taught me - that the best algorithm is to stack itself in neat columns with patience to find the apple. Actually by the end it has learned to just scan vertically until it happens to find the apple and then is biased to return to an average scanning position at the top left of the domain.
It would be interesting to introduce a decaying apple that requires finding before disappearing because the snake needs to "eat". I'd guess that the most interesting behaviors could be developed by specifying the snake find the apple in X moves to stay alive + grow and if >X moves grows but loses energy and needs to find the next apple sooner or else die.
Nevertheless, incredible.
The problem with something like that is that at a certain point the lowest possible time to get to an apple would be greater than the rate at which the apples decay so it would get stuck around a certain score
@@ultraeon there could some intermediate window where can be true, depending on the setup. Interesting observation.
At high lengths, the AI has more opportunity to "snake" which increases time required to get the apple, on average. There's a maximum length snake that is playable assuming perfect reaction time. I wonder how fast time-stressed vs. lazy populations would converge to the maximally efficient solution at the boundary of playability. If you notice, some of the mid-gens in this video get lazy and make less efficient and more numerous moves (shorter length requiring sooner changes in direction) until converging on the ultimate snake which is necessarily the most efficient use of 2D (3 point max. direction choices) moves (linear paralleling with minimized lengths of direction changes).
Thank you. The great one. Some questions: (1) Is there any reasoning behind the NN architecture (number of layers, number of neurons in the layer) choice? (2) What if we change all directions from absolute (U,D,LR) to relative to current head direction; wouldn't it help? (3) Why do you need crossovers; you anyway do mutation, what's the difference either to mutate the best or mix of two best?
Sick it actually won!
Bravo!
The end was like watching a thrilling movie?
Amazing! I'm curious if the best snake can always finish the game? Is there an apple sequence fails it?
super cool !!!
What in the..... Your AI figured out the best way to beat snake (aka Hamiltonian cycle) in just a few hundred generations. That's miraculous lol
Nice one
best snake ai, no cap
Thanks!
wow Congrats !
Surreal!
It's incredible how the names of all the components have changed till the 90ies yet they do exactly the same as they did back then....example: one-hot-encoded -> winner-takes-all (WTA)
Well its been a year since i saw this and im happy to be your 999th subscriber
Awesome! Thanks for the support :D
The PERFECT ONE I was looking for...😃😃😃😲😲😲😲😲
Great awesome.. Could train AI to play slide puzzle with 3x3, 4x4,.. I love to see that evolution on playing... 😍😍😍
Wonderful! Hope I can do things like these but using SNN
Amazing, the AI discovered that the brute force approach is in fact the best approach. That kind of generalization is pretty crazy.
I agree! I think it's pretty amazing that an AI was able to learn to "play it safe" to win
Interesting, the latest iteration on the huge grid appears somewhat close to an optimal strategy.
I can say this snake has smart moves
Could you next run a simulation using the best-performing snake to prune the number of neurons needed? Would love to see what the minimum number of neurons is to accomplish this game, since simple driving games only need 1 hidden layer of 3 (!) neurons.
Simple driving games might not even need any hidden node, since they basically use linear PIDs
This is much more complex, especially at longer length, and without making directions relative to head
That's the pattern I use for playing snake as well, but since there are no walls in the snake I play, you can zigzag one point and instead of 3 movements narrow it down to 2 movements and just go through the ceiling and come out the floor.
This Ai should had a time limit/reward to force it into risking it for the reward or playing it safe, OR playing risky early on and safe later when space got cramped.
Impressive!
Thank you :D
Underrated
Thanks :D
That was the pattern I used back in 1999 to play snake till maxed out (managed that 3 times)
It would probably be harder to apply that strategy to odd grid sizes maybe
That was great. Even i want to build something like this, can you please tell which frameworks you are using for NN and genetic algorithms
Thanks! I actually created my own simple frameworks for them. You can check out the code on my GitHub if you're curious how it works: github.com/Chrispresso/SnakeAI
But what about efficiency?
The snake AI : we don't do that here!!
Yeah, not only was it totally inefficient but I could code a routine to do it this way in about 20 seconds.
Fascinating! Didn't that bigger board bear the same issue Code-Bullet ran into?
what issue
Hi, great job!
I have a question: how did you come up to this fitness function?
I'm actually kinda curious how the later generations would handle non-rectangular arenas or obstacles.
I would have to adjust the code a little bit, but it could handle small obstacles if I did. The main issue is if I add something quite large and keep the code as is, the snake will see an obstacle and treat it as a wall. So it would never know the difference between obstacles and walls. For something more sophisticated I could use deep Q-Learning to allow it to learn obstacle avoidance!
@@Chrispresso that would be so awesome.
i wonder if its possible for it to learn that once the snake takes up half of the map, it should stay as close to its body as possible(otherwise it spends a increasing amount of time navigating) maybe something like "the closer it is to its body, the more likely it is to stay close to itself" which would work like "as the map fills up with its body, the weight of an input increases, once the map is 50% filled, the input is at 50% and then becomes active, triggering a change in the model. This would take forever to train normally, but if you start the snake in a map that is 49% full it wouldnt take long before the new part triggers so it could be tested and mutated quickly. it would also need to be able to keep functioning or it would die before it got to that 50%.
Is there any python libary I could use for these custom weights and biases?
Wow!
Subbed
OI, primeiramente parabéns isso foi fenomenal. Gosto de ver AI aprendendo e se aprimorando, muito bom o vídeo. Eu não entendo muito de programação, mas gostaria de ver esse seu algoritmo rodando aqui no meu PC, se vc puder fazer um passo a passo de como instalar os programas e fazer o algoritmo rodar eu agradeço. Já sou inscrito e vou passar a acompanhar o seu blog, abraço.
The first thing I see here is that the snake ended up following a hamilton cycle. It could be faster, too. Very interesting
I've been working on learning ml for a few months but mainly with libraries like tensorflow. I was wondering where you learned to make NNs from scratch?
There a lot of resources out there. If you're interested in learning the math behind them, you might want to look at www.deeplearning.ai/deep-learning-specialization/. It's a good balance of math and higher level libraries.
how did you make the sensor beams for vision?
I wish the snakes were able to further develop themselves to be more efficient on using time
Very Good Video! Helps a lot !!
But i dont get it with the Hot encoded Variables
So they is always one active but when is which active?
Glad you enjoy it! One-hot encoded variables make it so you can give equal weight to different things. Since the inputs are multiplied by weights, using a one-hot encoded variable allows the AI to learn which input matters without giving bias toward one over the other.
This is very impressive. Improvements that I would like would be that the neural network can 'explain' or 'formulate' somehow how to play snake. Ofcourse not in a conscious way and not in a normal human language. Just that it would give us the basic concepts of its success. I believe we could achieve this by letting us be inspired again by the human brain. Those connections are not meaningless calculations. Every connection has a small meaning and together they form a bigger meaning. We already know the meaning of the inputs of our artificial neural networks, and we know the meaning of the outputs. Let search for the meaning of what's in between them.
This is definitely possible and usually involves a heatmap to see what is "hot" and "cold" for the AI. That concept can help a lot when working with other types of networks too such as CNN's.
Snake evolves into Ouroboros.
Why would’ve give the whole grid in the input layer instead of only 8 directions? What different result could we have then?
That is very cool. There seems to be some initial "weirdness" going on early in the learning process with search patterns. In the beginning was it just trying to find the envelope of the playing area, or did it already know that? If so, why was it just going semi randomly across from side to side without (apparently, by its actions) even acknowledging the apple's location? I guess what I'm asking is does it have to randomly "hit" an apple once before it even "knows" there is reward in that, or is it instructed from the beginning that getting each apple is a large part of the goal? Enjoyed it, even though I'm not at all sure I really understood what was going on.
Glad you enjoyed! The snakes have no idea how large the space is that they are in. In the beginning they are just randomly initialized and they don't know what an apple is. If the weights of the Neural Network result in them getting an apple, they are rewarded, but not until the end of their play. So they don't have an immediate feedback saying "oh, that thing I just grabbed was good". I plan to do more videos similar to this, so I'll try to improve how I explain things in the future!
@@Chrispresso Got it, makes sense now. "They" really did get the hang of it, didn't they? ;-) I'll watch for your further videos. I remember sitting beside a couple of really early neural net guys decades ago talking about training the algos using ground truthing and sims, didn't have a clue what they were talking about then, either. Guess it's a good thing I was a HW guy...couldn't run "your stuff" without "our stuff", though, could you?
8:30 Looks like its starting to play Snake as if it was a printer lol. Especially 3D one :P
Distracted by great music ;__;
The computer is playing with itself.... give it some privacy! oO
It basically set a hamiltonian cycle for itself
Yep! I was pretty impressed it figured it out.
I wonder what is the "thinking" part in a NN?
well Done :)
could you explain fastly how did you creat de fitness function?
I wanted to reward early exploration (grabbing an apple or two), reward snakes for each additional apple they get and penalize snakes that take a long time to get apples. From there I tried to express my wants mathematically and came up with what you saw. Hope that helps!
Anyone who reads this, don't just watch a video and forget about it, give the creator some support for once
Yaaaaaaaaaa, I agree