Neural Network Learns to Play Snake
HTML-код
- Опубликовано: 3 ноя 2018
- In this project I built a neural network and trained it to play Snake using a genetic algorithm.
Thanks for watching! Subscribe if you enjoyed and Share if you know anyone who would be interested!
GitHub Repo: github.com/greerviau/SnakeAI
Twitter: / greerviau
Support me on Patreon: / greerviau
Thanks to Josh Cominelli for the music!
Soundcloud: / josh-cominelli 
Наука
You guys think that the snake died because of the lack of left turns, but in reality the snake evolved to the point where it got consciousness and understood that life dedicated to running in circles is not worth living.
No One ur joke is so dark, it darken my life
WoW
almost cut myself on all that edge
No One agreed
That's deep
For anyone who wonders why it seems to prefer right turns, I believe that is because it started at the top-left, going towards the right. There was no way for it to turn left. So with 2000 snakes per generation, a LOT of those learned that left is death. Since right worked every time, it simply had no reason to learn that turning left after leaving the wall would be safe.
I believe that is also why you got those wiggly motions. That's it trying to turn left, but then immediately turning right again, so its profile won't go any further to the right than the starting position.
It might be interesting to see what happens if the starting position is randomised
What about the middle ?
@@trex70 middle and just go down, snake will choose Left or right way by red points
About what I thought, but does this mean it is unable to get significantly better scores, because it will suicide inevitably by coiling up instead of folding itself?
@@badkingjohn5235 The most likely scenario is that it will discover that it can fold itself in that direction, which makes it survive for some time longer.
It will therefore take days, possibly even weeks, to simulate it to the point where it learns to fold in both directions, or fold and twist around.
ME: "Hello World">>20 errors found.
Funny: Funny! Funny ... ?
@@NoctumusTV what?
@@NoctumusTV Can you pls explain again. Thanks in advance.
@@NoctumusTV oh ok
@@marls3518 Explain what again?
2:26, that’s literally just a dog
literally
literally
Literally
literally
yllaretiL
My favorite part is every time you think the AI finally has it down, then runs into a wall for no reason
Every time
that's what God said, when watching humanity
there is a reason, the human element is a fuck up disgrace in this case
Maybe time for revolution
@@user84074 Then God killed the program
Same as humans)
Left turns: *Am I a joke to you?*
I observed the same thing; is that a design flaw?
It's not an ambiturner.
Neural networks is a lazy algorithm and will take the shortest route to achieve its goal. My guess is that the input of the distance from the left wall from the snake plays a significant importance to its decision making.
You can use dropout which will force other nodes to train that never gets a chance when the whole network gets trained
@@pocketrocket27 God damn you Ivan - you beat me to it. Damn you to hell :D
but it took left turns
I think another one reason why this doesn’t get higher because in input it gets -
1. Distance to food
2. Distance to wall
3. Distance to tail
Wait but what about its whole body ??
So that’s why snake trap around it’s own body.
Just a guess though 🤔
thought the same, but could the lenght just be another input neuron ?
We could probabaly include the previous outputs as an input like an LSTM or GRU
Perfectly correct. If you feed it the location of the whole body in terms of a matrix it will evolve to the point where its better than any human playing the game
Nice observation
Yup, using a CNN would be a good approach for this problem, I think. Use a different weights for the head, tail and the location of food.
Generation 30: *dies*
Me: YOU WERE THE CHOSEN ONE
I love that the reason it failed is because that's the one flaw of the technique it's honed from the start
Lock in. You hit on a successful strategy which gets you all the way to the point where it is no longer successful, but by then you can't do anything else. A typical failure mode these kinds of systems, from corporations to civilizations.
Benjamin Feddersen
While you’re correct that adaptation is a very necessary skill. Michaels point was more about BAD HABITS than an inability to adapt to new circumstance.
Bebolife A bad habit can inhibit adaption.
@@benjaminfeddersen7937 Dude this shit is deep as fuck.. It's the epistemological concept of paradigm. Any paradigm in order get surpassed need first to collapse on its own rules, unable to explain or resolve newer problems
There are no bad habits. Just bad outcomes.
I have deeply learned that in the end, nothing is left.
You're right
Lmao
This is hilarious
But don't massacre your clan in future
@@re_claimer_ how about you go watch shippuden? clearly you dont know shit
Humans: *computers will take over the world and destroy us all*
Computer: *hehe line go zoom*
not funny didn’t laugh
My disappointment is immeasurable and my day is ruined
snake go brrr
@@mesq999 And this is why no one likes you at school
@@nahimafing Just because he has an opinion slightly different than your opinion, it means no one likes him? You are a fucking asshole
"What is my purpose ?"
"You pass butter...."
This neural network is incredibly inefficient. Right from the beginning, it learned to not turn left by any means. This video is perfect as a demonstration that neural networks can easily get stuck on a very wrong local optimum.
bit of an analogous for humanity, don't you think?
It's not inefficient - it has an energy cost of 0, there's nothing to constrain it's time. If there was an energy cost (negative reward function) for turning, it would optimize its routes.
@@manzell good point!
@@superpantman not really, as humans use a mixture on neural networks and symbol manipulation. That’s why AI (recently overly focused on association and deep learning) are not progressing as fast as hoped.
@@manzell And perhaps adding in more genetic variances from generation to generation to allow novel ideas to die or take hold.
congratulations for the big work you've done, not only at the algorithmic part, but the visual part which i can see it's a huge effort to present us your job.
I can't imagine how happy would be the first guy who developed these algorithms.... ❤️❤️
Yes
Yes
yes
Yes
We
6:07 love how the snake eating the food is perfectly synced up to the songs snare until around 6:22
lmfaoo thats pretty cool
Lol neato
The snake evolved into being able to understand the music
actually also at the start of gen 30 (from around 5:00 onwards) it’s synced up in some ways
😂😂😂
I think it probably would have learned better if you had started off with a lower number of moves left (maybe like 60?) so that it doesn't have so much security to take its time.
That is definitely possible
@@GreerViau Also, If you want them to evolve how to avoid hitting themselves better try making the map small so that they encounter tat problem sooner
@@arthurfacredyn That's especially true if the improvement yield was already capped, with a lot of room still available around.
You could also add a small negative score for each frame, so that it prefers to die rather than do nothing, but it might get stuck in a local optimum of immediately killing itself.
@@arthurfacredyn Or making the snake longer right at the beginning so it can develop strategies for avoiding his body
The mind of the Snake in the first few generations, spinning to infinity a pixel away from the food
"FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD"
sciencesyfy this actually made me laugh and not just breathe fast out of my nose, gg
I laughed at this
It's so fascinating to look at a neural network learn and it be visualized, it's like a mini brain in a computer learning and reacting to their surroundings, telling a machine that only follows orders to figure it out themself
Thank you so much for making this educational video! Well done! We are so grateful
I would like to see this but also with an adversarial neural network placing the next food piece.
or two snakes, each racing for the food
@@JohnSmith-xf6nb I feel like allowing it to change size would result in it shrinking the board as small as it can to reduce the number of points available.
@@JohnSmith-xf6nb I think that might go to far the other way, because a bigger board would mean less points per apple. Maybe if the board is smaller than whatever the "standard" is, then the points awarded increases in proportion to the number of points lost?
If you're trying to add a new thing for the adversarial network to do to try and mess up the main one maybe it could also spawn "bad" apples that either kill the snake or remove points. I think that would be interesting because then the snake couldn't always just navigate directly to the apple, it might need to avoid something it it's way and the adversary could try to place them in choke points and such.
Could you make a video where you explain your code an how you determined fitness and the mutation and crossing over procedure?
I looked at the code a little and while I don't know the language, most of it is rather simple.
The weights are stored using a self-written matrix class, which is a 2-dimensional array with a few methods to do matrix stuff and for mutating and crossover.
Mutating adds some random gaussian noise to every weight. You can look that up in the github repo in the file Matrix.pde
The crossover method selects a random coordinate inside the matrix. Anything above or left of that coordinate uses the values from partner A, anything below or to the right of that coordinate uses the values from partner B.
The fitness is just the length of all the snakes in a generation added up.
I learned this kind of stuff in university and this project goes against a lot of what I learned. For a practical application, these functions would be pretty bad and most importantly, very slow. But the whole thing still works very well, so well in fact that without knowledge of the subject, most people wouldn't be able to tell it apart from a more professional approach.
It shows that machine learning isn't hard on its own, but the tools that are used nowadays are pretty complex.
@@nottheengineer4957 in which program or app can I do these kind of stuff?
@@mauriciomontalvo5885 Well u can use any programming language i presume, though some are better than others for these kind of things. If u want to hard code it yourself i would use something fast, but you won't likely achieve great performance unless u really know how to optimize the hell out of it. What you can do is use NEAT or tensorflow for example in python. Combined with pygame you could do all kinds of things like this. NEAT is extremely easy to use, to the point that you barely have to understand what is going on.
@@nottheengineer4957 Where to learn about more professional ways and tools they use? Just for curiosity and learning purposes (obviously without getting into uni, too old and too broke now for that).
Hi Kant☺️👋
No one:
RUclips when my lil brother uses Wi-Fi 1:22
he must be downloading extra ram
@@Faisalamin01 no he was downloading graphics card 😂😂😁
It takes a few generations for any significant progress to be made
What I think is most fascinating about this project is that the neural network never learned the dimensions of the game board and kept returning to the start
The song works so well with this video. I am feeling so calm right now lol.
Theyd get further with more information. You forgot a key piece. Direction of "motion" of its tail. While not immediately obvious in game its something human players take full advantage of when they get stuck on inner loops
becominghuman.ai/designing-ai-solving-snake-with-evolution-f3dd6a9da867
Do you think it would perform better if the input to the network was the grid array containing all the information about the game state. eg a 50x50 array of numbers 0, for empty, 1 for snake body and 2 for food. Or is it better to explicitly tell it the distance from the food, is it unlikely to work it out itself?
@@dananderson8459 using convolutional layers instead of fully connected layers probably yes, otherwise probably only with a significantly larger network
what if there was one value for head position, one for head direction, one for food position and a vector for the entire body
I'm a noob but I think it could do very well with this
if the network also used some recurrent design (such as an LSTM) it could possibly compute motion and have better planning abilities
More intense than any latest action movie fight scenes :D Respect!
This made me genuinely happy, thnx for posting
Nice visualization combination of the neural network firing and its effect.
Let's say I also want to create such visualisation, how should I do it?
Heres to where youtube recommendations lead me to today during quarantine :D
Wow! Really cool! Thank you
Amazing work
This is what happens when you don’t consider the “time-to-solution” in your fitness algorithm!
That's exactly what I was thinking, along with the fact that the player usually is not the snake, so there should be a couple of input neurons more with the position x-y of the food
@@yurimrt Yes and the cartesian distance to the food sqrt((Xsnake-Xfruit)²+(Ysnake-Yfruit)²)
@@uwu_senpai yeah but that works only for whne the snake itself is not blocking the path, there needs to be a priority set that it just need to find the shortest path to is next objective, like going out of the block by the snake which can be obtained by looking if the snake is on the x way and the y way to the food and if it is look for the shortest path possible for that not to happen.
@@uwu_senpai Euclidian? Sounds like a bad idea as you cannot reach the fruit in less than |xsnake-xfruit| + |ysnake-yfruit| ticks
@@mirabilis but you know if there the snake doesn't block that path, it's the fastest way possible, there is no faster way, it's just math.
This is so amazing. Next topic to learn - Genetic Algorithms
thanks for showing us
The music is so nice!
A quick suggestion: don't constrain the neural net so much. Give it the entire 38 by 38 grid with three possible values for each location (off, snake, apple) and train using those inputs. It can even be considered a vision problem at that point, and modern ML libraries can solve it with a convolutional neural net pretty effectively.
Wouldn't that be far, far more computationally intensive? Genuinely asking
@@Caffeine_Addict_2020 not really, considering modern hardware can comfortably run CNNs on proper images, 32x32 grid of pixels is nothing
@@Caffeine_Addict_2020 relative to this model? Yeah. But it still wouldn't run slow on modern hardware by any means
Anyone else deeply in love with the first song ? It’s so calm and nostalgic
Nice work bro
really enjoyed this
It would be interesting to see colors for the hidden layer nodes as well, colored for their activation level
Plus a gradient for the weights instead of just blue/red
2:31 me when i play tetris and i know im gonna lose
Beautiful story. Never give up!
amazing progress
People: omg ai is going to dominate the entire world
Meanwhile, the AI: gonna go get max scoring in snake
This comment hasn’t aged well :))
You should nickname your snake Derek Zoolander because it appears to struggle to turn left
it doesnt
@@user-kx5es4kr4x it really does
Underrated comment
AWESOME man this was so awesome !!!
What an interesting excercise.
At 2:20 starts feeling like I'm watching a movie about a guy who was weak at the beginning but he starts training more and more despite his failures and finally he comes to success
Thank you, your code help me understand well about the AI. I am a newbie :
Great example thanx!
And it was delight to watch
In such simple networks, the encoding of inputs can make all the difference. Representing distance in some sort of a grey or logarithmic code may be worth a try to speed things up :)
this is the coolest thing i have seen in my entire 18 years of existence
Great work
This is the first video I saw of you!
Samir, you are breaking the snake.
Samir, you are not listening !
What the deuce !!
Who are you ?
@@stewiegriffin6503 that's what I am supposed to ask. Who are you! And why do we look same
Shut up, dont tell me who to drive
Looks like Stewie has been messing with the time machine again.
Awesome! Now make it two AI-players: Your snake vs AI that places the food with the opposite target: Reward if the snake dies. That would be an interesting experiment :)
This is stunning.
Nice work fella
The playful ones are especially cute OML XD
5:48 it's starts eating the red dot on the beat
6:17 begins the killer moves to the beat 😂😂
this video helps me to understand the nn better.thank you
great video thank you
You might have gotten better results had you let it start from the middle or from different places every time ^^
Great video! I glad more people are taking interest in neural networks
I'd have said that the problem might be that he is selecting only the best out of the 2000 snakes. That leads to a strategy which is only a local maximum. That's also the reason he doesn't get better results by training further. It's hard to get out of that when you don't allow the chance of exploring other strategies which are not locally the best. His population was too small and the mutation rate too low to fix this issue. You'd probably get better results by selecting a small group of snakes with equally distributed fitness.
I mean, this then adds an "RNG" variable, which you really don't want, no? A snake may have better fitness because it got a lucky placement, and you don't want to breed for luck because that will be completely random
*Me:* * _sees the thumbnails_ *
*"Wait, That's illegal!"*
I love how it likes to return to top left before making next manoeuvre, shows the training
Wow awesome stuff man!
The strategy backfired if it becomes long enough, a rule telling it should calculate its length vs the size of the field before making a move should be applied
1st generation : I'm hungry
30th generation : solved the hunger problem
100th generation : discover the network
500th generation : taking over the network
1000th generation : human extension.
*extinction
@@vibinv8905
let it go man 😂😂
@@adomustafa1777 The OCD just took over :D
@@vibinv8905 Butt in your moment of "OCD" did you notice the choice ?
You see even though it might not feel like it (regardless if you have this so called OCD or not) there is always a moment where you have the choice. The thing that told you that you wanted to correct him is simply an impulse and you have complete control over your impulses. It no longer works to say oh blame it on my OCD because YOU chose to listen to it. Whatever reason you have for making the decision, it always comes down to you.
A habit is just a choice you keep making.
@@PsychoBackflip thanks for the pep talk.
This is so amazing.
Super vidéo ! Parfait pour un pic nic entre potes pour se relaxer. MERCI
RUclips Algorithm: Dis looks guud, lemme recommend it to everyone
The limiting factor is the input vector IMO. If the snake operates only on relative distances then no matter what - it'll end up encircling itself and getting stuck.
Wow! Great video! ❤️ Neural networks rules! 👍🤓
Awesome trained snake :D
I think using some form of DeepRL coupled with CompVision could yield great results for Snake. Of course, for such a simple game you could skip the CV component, but I feel it'd be more fun that way. Also to avoid biasing, you could perhaps pick a random spot as a starting point.
2:39 - The snake has evolved into a dog.
Holy shit! This is amazing
i just love that part when the 30th is synced with the music as is turning on walls
the snake consistantley modes clockwise.
Woah, great visualization
Can you name-drop some of the tools used to create this?
ㅇㄷ
There aren’t any “tools”. You should start by learning about neural networks and deep learning, and try out a few simple networks to learn how to program them. Once you’ve got a grasp of neural network programming you can pretty much adapt them for any problem, and expand the hidden layers and neurona where necessary.
@@26dimensions70 Thanks for replying!
I am in fact a computer science graduate, and I also have a degree in Industrial Design.
My question was about the visualization tools you used to produce the video, I'm fascinated by the animations and would love to learn how to produce similar demos of my own ML research :)
You're welcome to take a look at one of my projects (an artificial intelligence constructing objects from a 'LEGO' like building block I designed) where I used Python's mplot3d to create a set of images I converted to animated gifs to visualize the algorithm.
www.razkarl.com/projects/kawaz
@@razkarl Take a look at OpenAI gym. It is a virtual environment used for reinforcement learning
That's really entertaining video
Fascinating
zoolander bot only turns left
At last he did it!
You mean right. It only turns right. But on the bright side, it is RIDICULOUSLY good looking.
And eat only from top to bottom.
Hello, amazing video, thoroughly enjoyable. I'm very interested in starting to program stuff like this, can you point me what direction I should go to start learning to write programs like these? I already have myself familiar with both neural networks and a few optimization algorithms but this program seems to be a mix of both of them., since it doesn't really have any training data and relies on generations and random behavior to train the neural network.
nice opening track. really enjoyed it.
That is satisfying to watch
Pleas make explaination for control kamikaze drone next!
Terminator: ....SnakeNet begins to learn at a geometric rate. It becomes self-aware at 2:14 AM, Eastern time, August 29th. In a panic, they try to pull the plug.
At the end it felt like the snake was synced up with the music and dancing along that was pretty groovy
btw, awesome job mate!!
Я всё ждал, что нейронная сеть будет управлять змейкой по оптимальному и короткому пути, в том числе по диагонали! 😃
В конечном итоге я дождался другого, когда нейронная сеть будет проигрывать из-за столкновения змеи об саму себя. 😅
на самом деле это оптимальный вариант движения по кругу, т.к. змейка может быть ограничена только размером карты, движения по диагонали уменьшает свободную площадь от 10 до 50%.
привет от диванных РУвойск
__
вот вот_ чето автор логику игры не допилил _ когда змейка заходит во внутринний круг то конец сразу __ хотя может лишние проверки.. а нужна была производительсть .. хотя... хотя...
It`s cool to see how this is going :D
But sad to say that I`ve played way more rounds and never get a better score than 7 :(
😂
Great video
It really loves that top left corner
- what is my purpose?
- your purpose is to play snake.
- oh. Oh my god...
Yeah, welcome to the club pal.
The 30x speed felt like I was playing osu for some reason...
*DADADADADADADADA INTENSIFIES*
Fantastic!
Very cool!