A.I Learns Snake And Wins - Part 1
HTML-код
- Опубликовано: 7 окт 2022
- please like, subscribe and share :)
I wrote a Genetic Algorithm to play the game snake.
The problem:
Developing an AI to beat the game Snake is a classic problem in the field of artificial intelligence and has been the subject of many research projects and competitions. There are several approaches you can take to create an AI to beat the game Snake. Here are a few:
Reinforcement Learning: You can train an AI using reinforcement learning to beat the game Snake. In this approach, the AI learns by playing the game repeatedly and adjusting its actions based on the feedback it receives from the game. You can use algorithms like Q-learning or Deep Q-Networks (DQNs) to train the AI.
Genetic Algorithms: You can also use genetic algorithms to create an AI that can beat the game Snake. In this approach, you generate a population of snakes with different behaviors and then evolve them over generations to create a population of snakes that are better at the game.
Rule-based approach: Another approach is to develop a set of rules that the AI follows to play the game. For example, the AI could be programmed to avoid walls and obstacles, to chase after food, and to stay away from its own tail.
Regardless of the approach, creating an AI to beat the game Snake requires a good understanding of the game mechanics and an ability to balance exploration and exploitation during the learning process.
Follow Me:
Twitter:
/ fenglema36
TikTok:
/ ezraanderson1979
Discord:
/ discord
Machine Learning, Neural Network, Genetic Algorithm, AI, artificial intelligence, Self Play, reinforcement learning, ML
awesome presentation. watching the neural network in real time is hypnotizing.
Glad you enjoyed it!
@@fenglema36 can you link the code in github ;) ?
@@fenglema36 and btw awesome video earned a subscriber
It was that sounds for me. It spoke to me. Told me things. That it loved me.
@@fenglema36 thanks for sharing
I legit didn't know there was a win condition on this game
I thought the goal was just to score as much as points before you die.. 😅
Not all such snake games had this condition.In some what he did at the end would result in regular elimination....
Great video :)
Same here, we just used to look at the highest score. Never to the end shown in this video.
Awesome video bro! @ezraanderson 🎉… @semituitive thanks for sharing my snake video here 😅
I always did the algorithm by hand, but never got fast enough eye-thumb coordination to finish it. This is so satisfying.
up, up, down, down, left, right, left, right, B, A and Start
@@fenglema36 Konami code ftw
:)
If it's algorithm (unless greedy) then AI is not necessary right? Is the game solved?
グラディウス😅
I've completed this one in my teens using the pause button. Used exactly the same pattern. It was so satisfying like I won an olympic medal :-)
Same here with Nokia old phone.
Cap
@Repent and believe in Jesus Christ It's much better this bot that keeps spamming the same exact message very on every thread. There's no creativity to try varying approaches nor even, I suspect, ability to relate to the topic, very convincing. Better grab my spot among those who have ceased thinking independently.
The earlier majority of that exact pattern is by no means necessary, so that's pretty crazy.
@@dragonflysword way too risky and really crazy moves lol. But that's nothing for the AI I guess, a whole different story for a human 🙂
That was actually mesmerizing to watch. Thank you!
Thank you so much 😊
please like, subscribe and share :)
Now I realize being patient and cycling back to the initial coords on map is better than just running toward the food.
A great demonstration of Neural Network AI... Some observations:
1. This AI probably didn't have the shortest path calculation in-built
2. As I could figure out, The basic logic that was built was not hitting the own body or the boundary
3. One of a good example of how neural network works (evaluating multiple options before making those 4 choices of Right,Left,Up,Down)
As we are from Non Tech background, it would be great to have a small explanation of the code as well that went on to making this Algo..
it is genetic algorithm.
weight and bias of neural net are evaluated as gene with game score to make better decendants.
states are determined by 8 identifing sensors in head, and 2 direction sensors in head and tail. identifing sensors sense that which exist (wall, snake, apple) in sensor direction, and direction sensors sense that which direction head and tail toward to. these informations are used as input of neural net.
and actions are up, down, left, right. these are determined by output of neural net.
This is a conjecture based on my knowledge, so it can be wrong.
correct
It's incredible how quickly the computer can calculate all of those weights and biases in matter of milliseconds.
It takes a lot of time to train it though
About 8 hours of training
@@fenglema36 I played this game daily for maybe a decade. To think that only 8 hours can surpass what I did for so long is hella fast. It seems slow to us but it’s blazing speed. And AI is getting better and better.
@@fenglema36 I wonder if evolution strategies could do it faster, without being more complex like Q learning
Perfect application of discrete maths graph
Fantastic visuals!
Awesome stuff bro, really appreciate you posting these exciting stuffs.
Can you point me from where I can learn these ?
"towards data science" I read a lot of articles and watched a lot of you-tube videos on neural networks.
andrew ng and Andrej Karpathy are a good place to start for youtube.
I took a different approach and hand-wrote ever-thing in pascal, but there are a lot of python ML stuff out there that might be the easiest place to start, I am not very familiar with python so its hard for me to recommend stuff.
Awesome Vid!
Thanks
Wow man this is truly amazing!
Thanks, please like, subscribe and share :)
Imagine if the AI learns to drive vehicles and it starts to drive this way. Wait a minute...
Very witty :)
Imagine if AI learns about self consciousness and it starts to revolt against humans. Wait a minute....
@@indiankid8601 just throw a bucket of water on it tf?
Still it can't drive in india 😂
Only because there are cows on the road, and everyone knows snakes are afraid of cows.
This was great!
Thank you so much 😊
please like, subscribe and share :)
I was waiting for the last moment to see how the win of snake game looks like. 😁
With known size of arena, initial position and without dynamical obstacles(except own tail) algorithm for win is actually just few loops. Any space filling curve or just scanning goes)
This is a good metaphor for how A.I can win better or win for certain, relative to what we can achieve.... yea the world is going to change big time!
Edit: loved the work, you did great with this presentation.. you win too! congrats!
Thank you so much for the kind words :)
its like road painting across a map picking up cones worked for me
This is exciting as well as scary
that was very satisfying and greatly presentated video.
Thank you so much 😊
please like, subscribe and share :)
Cool!🔥
Is this GA-trained?🤔
What type of variable of the neural network you use? (int? float?) And the neuron activation function?(logsig, linear, etc?)
Do you use bias for the neurons?
Thanks
thanks, Yes GA trained singles(floats), tanH and no bias.
Now that's some high quality AI-asmr
Breathtaking
Than you so much, please like and share :)
This is so satisfying to watch.
thank you so much, please like, share and subscribe!
Beatiful 👏🙌
Great video! How many iterations did it take until your model had the perfect weights? Can you also tell us which rewards you used? :)
Far from perfect weights. It was a genetic algorithm with a fitness of steps + score. The goal of this was to finding the starting weights, then moves these to an actor critic model
I finished it once too… it was so satisfying
Yo that's wild.
*cleans gun again
Such a great guy you are....I love it❤️ but how do you programmed it?
I am trying to work on a tutorial
It's awesome but filling the space on an area like this can work as the only strategy needed. Also the biggest challenge for humans afair was the reaction time, and less so the entanglement with itself
Awesome bruh
Thanks, please like, subscribe and share :)
awesome work man 👍
just curious, what crossover and mutation methods did you use?
nothing special but there is a simple rule very early on which will flush and kill generations early on if the starting weights don't procedure could results after 100 generation.
Then I think there is a branch of 4 types of mutation that decay a little bit over time.
Cross-mutation
Inc weight
Flip weight
Inc weight toward other weight
Think of a CPU architecture for predicting, if it makes an error it flushes the whole pipe line. I currently believe optimal starting weights will give you the optimal outcome of the network.
Amazing job! How did you do the visualization of the network? Which tool did you use?
Everything was handwritten in delphi (pascal)
Awesome video, got a question though. I dont quite understand how the snake is learning in the video if theres only a single snake on screen (also no generation count), along with the fact that it seems to play almost perfectly from the start of the video.
I ask since I made a flappy bird game with neuroevolution and it took a couple hundred generations to play perfectly.
This model is already trained. It trained for about 8 hours with a Genetic algorithm.
Chilling as fuck how AI learn so fast! Applications for this are great but dude it's chilling how a machine can do anything better than you and still learning so far
That isn't the AI learning. it's the AI playing based on what it's learned.
Holy cow this ai is better at snake in 2 minutes than I was in 5 years… and now i feel like i should have known how to play! It’s so obvious now!
Holy shet, somehow a pleasure to watch.
Thank you so much 😊
please like, subscribe and share :)
I never seen that game until the end Kinda mesmerizing
Imagine learning a robot pressing those finicky plastic buttons from those oldschool phones
That was fun!
Thank you so much 😊
please like, subscribe and share :)
The sounds are very satisfying
Thank you I like the sounds also :) some people hate them but thats okay
That's really awesome
Thank you so much 😊
please like, subscribe and share :)
The neural mind is very complex just for left/right up/down :o (So that means we have to appreciate to what we've been given, Because Remaking it is more complex than what You can say)
Feels like my daily routine Left, right, up and down.
Çok güzel olmuş tebrikler. ❤🎉
I love your video.
Thank you, please like subscribe and share :)
Awesome
Thanks, please like, subscribe and share :)
Wow. This is mind blowing.
Thanks, please like share and comment
I learned that I want to play snake again.
This is PHENOMENAL!💫
thanks, please share and like!
@@fenglema36 already done!🙂
amazing,
whats the lighting the cells up can just be ordinary resistors, if you keep the amping theoretical through the net and just amp it once at the end.
Wow it’s so cool so see NN animates
Thank you so much :)
please like, subscribe and share :)
Nice work for sure, but just to win the game you don’t need to have an AI algorithm, simple movement algorithm is quite easy to program to just win the game. What would be cool to see is that how different AI models to achieve winning state and which one do it better/faster. Because on the early stage of the game there still a lot movements to improve. Anyway, good work and it was interesting to watch!
Thank you. I Agree other algorithm would work, and this is overkill.
I would like to do this project again based on different training methods, for example shortest path, longest path and Hamiltonian cycle.
@ezra Anderson Thank you for doing this, that would be very interesting to watch how differently other algorithms solve the same task. Once again, thank you for your work
@@fenglema36 which data did the AI algorythm was training
I actually think it was quite efficient. Making detours to avoid hitting its tail later, and finding shortcuts when applicable. If the AI learned this on its own then it's definitely ways ahead of a preprogrammed path with efficiency hacks. But if the path was programmed in as a bias, then it's still impressive, just not abstract machine thinking.
I grew up playing this game and never knew that there was a winning condition.
The winning sound is very satisfying
Thanks, I was concerned the sounds would be off putting... I feel like it fits the style/Theme though
OMG … this was so satisfying !!!
Thanks you so much I also find it satisfying. please like, subscribe and share :)
Amazing 😮
Thank you so much 😊
please like, subscribe and share :)
Didn't even know we could "win" Snake lol
Ikr
Yeah, we're all losers
The woke left ruin everything
It doesn't have to worry about hand eye coordination...great advantage for it.
or two left thumbs
I programmed this as well, and realized very quickly how many inputs were needed to give the snake “vision” like you have done. Do you think it would have worked better to just have the contents of every block be the input? I wanted to go down that path but never followed through.
Pixels(Blocks) would be more difficult to get to work. In the snake game I wrote you can request "Pixels" or "Sensor" for the inputs of the Neural Network. When building this framework I focused on less data points. Meaning the inputs of 32 sensors vs 100 pixels(10x10) allowed it to train much faster. I would like to create an Actor Critic model which uses the inputs of Pixels, but have not been able to do it.
@@MrMrmalobaidi I am curious as well. From the look of it, his inputs were 1x3 categorical array
0 distance - empty space or not
1 apple - the apple or not
2 snake - its skin or not
on 8 directions around the snake's head (we got 24)
concatenated to
1x4 directional array of snake's tail and head
0 up
1 right
2 left
3 down
(24+8=32)
Still impressive on how AI could learn to develop a strategy on given parameters
Bingo, that is a perfect explanation!
I'm confused as to how a NN is the right tool for this
It probably not the best algorithm for this task.. But I hand wrote an NN framework and thought this would be a cool thing to try to do.
This is so cool, what is the reward mechanism for your agent?
it's really simple fitness test on the agent that lives the longest moves * appless
Looks far more successful than most NN snakes I've seen.
What was the training algorithm? And what were the sound pitches based on?
I think the sound pitch is based on the field, the head of the snake is on. When it moves up, the pitch goes down and when it moves down, the pitch goes up. Just my guess though.
The pitch is just a representation for the current grid location. 100 tones for this video. Its the only method I could think of to represent the network sound.
Looks cool, what did you use for visualising?
Everything was hand written. The drawing is done in openGL with module I wrote for the snake game and the NN
That was a roller-coaster ride for my OCD
Some people find it soothing, other people find it annoying... its interesting how the mush in our heads interpret our senses
Beautiful
Thank you so much 😊
please like, subscribe and share :)
Great video, so what are your settings? Like batch size, max playback buffer size, gamma, learning rate, layer activation?
This uses a genetic algorithm for training. I did start and actor critic version but it was never completed.
Those are decently fast reactions, almost as good as mine.
So good
Thank you so much, please like share and comment
VERY NICE
Thank you so much 😊
please like, subscribe and share :)
That's dope!
500lb squat that's dope! Beast mode!
@@fenglema36 lol thanks
I’ve got anxiety watching it chase it’s tail 🤣🤣
And we do it just by, well, not even really thinking about it.
Thanks for showing the inputs of the network. What kind of learning algorithm is behind it?
a very clunky genetic algorithm,
I think actor critic would be better but I have not had time
Thanks. So, is the source code available? If not, could you please explain the inputs? It is hard to understand their meaning from the video, namely what data from the box on the right should be sent to the inputs. Additionally, on the outputs: are they real world directions on the map or they are relative to current snake's head direction?
Numbers 0 to 7 represent those 8 lines drawn out from the head of the snake.
0↖
1⬆
2↗
3➡
4↘
5⬇
6↙
7⬅
For each of those numbers, there is a Distance, Apple, and Snake.
Distance: The distance value indicates whether or not there is at least one open block of space in that direction relative to the head of the snake.
Apple: This value indicates when the head of the snake is in line with the apple.
Snake: This value indicates if there is a section of the snake directly next to the head of the snake in the direction specified by the number.
Then you have numbers 8 and 9 which represent the direction that the head and tail are traveling.
The values on the right are the direction in which the neural network will move the snake next.
Those unlabeled nodes in between the left and right nodes are the hidden layers of the network and will be adjusted by some function allowing the neural network to learn.
@@a123b123c123d123c123 , thank you. Now this is much clearer. From your nickname I can't be sure if you are the author of the video, anyway, since you have good insight, could I ask another question? The selection of input and output format is quite critical for Neural Network (NN) processing and getting correct results. Why exactly the binary representation for inputs were chosen? Why don't they use real numbers for distance? This looks more natural and can give more information to NN, does binary representation have its own benefits (like stronger input influence than gradual value)? Can you please, comment?
@@doctorshadow2482 I am not the author so the info I give is just what I can tell or guess from watching the video. An integer value may be more useful for an AI but it also may be necessary. From what I can tell the Distance values just indicate whether or not the snake can move in a particular direction without hitting something. Also, binary values might suit the algorithm used better. Also the more complex the input the greater the processing power required.
@@a123b123c123d123c123 , thank you for the prompt feedback. One comment, per my experience, in NN processing both binary and real inputs have the same processor power consumption, since the binary signal just converts into 0.0 and 1.0 of real number. So, this is just representation matter and this could affect the neurons triggering, but not the required processor power.
@@doctorshadow2482that 0.0 and 1.0 are probabilities. It represents a certain weighting for a specific state. If you wanted to keep track of the actual distance using this method you would need a binary value for every distance increment. That would greatly increase the number of inputs and would increase the processing power required.
Really impressive. Which tools have you used for visually representing the neural net and the snake board?
This is a handwritten neural network framework written by me in delphi (pascal). The NN and the snake game were written in 2020 Covid lock-down. Once returning to work I never got around to posting it. I dug up the framework a few months ago to work on convolution kernels, and decided to post this.
@@fenglema36 genius
Thanks
I've only seen snake beatin one time back when I was small child in the 90s after that not even close
Mind Blown - Poooof!
thanks, please like subscribe and share :)
hello, a doubt I can only develop a neural network for specific things? what if i want it to do another function?
great!
Thanks
What does the first layer represent. The distances in 8 directions i believe, and that more ?
Awesome work ✨. Could you please share the input featuresa and their meanings. What is Distance0, Distance1 represent?
If i had to guess Distance0 is the snakes head and Distance1 is the apple
Every sensor is a binary switch. its on or off, snake, head or wall.
1:48 was impressive.
Thanks, please like, subscribe and share :)
Great work! What's the input data set that was used?
Snake singularly
Blew my mind actually
Thank you so much 😊
please like, subscribe and share :)
I have a question. Obviously, there is an easy intuitive winning algorithm for snake. Does ai finds it optimal win-path? Or in this case simple algorithm is better?
You could probably make a more efficient algorithm. This video isn't actually showing you the AI learning.
no it is not optimal. But I am trying to learn Neural Network and this seamed like a fun idea
I went up to like, 80% phase of that) GG
I’d be so nervous on those last few bits.
autopilot - and yeas its nerve racking
Love the way they represented the neural net.
Thank you so much, alot people asked me for the code to draw then NN, but everthing was handwritten
@@fenglema36 always wonder if there has been any changes in the algorithm that calculates the weighting for each node the last 20 years or how the nodes are structured. Haven’t played around with it for years. But will always remain something cool to work with.
@@etienne8382 to my knowledge networks are just becoming bigger as we can do a lot more compute now then 20 years ago. But Backdrop is still the most used for training networks..... I might be wrong as I am relatively new to A.I
Great !!!
Man I want to learn this
Any suggestions where I can start from ??
CodeEmporium
ruclips.net/channel/UC5_6ZD6s8klmMu9TXEB_1IA
Andrej Karpathy
ruclips.net/channel/UCPk8m_r6fkUSYmvgCBwq-sw
Andrew NG
ruclips.net/video/PPLop4L2eGk/видео.html
That was beautiful.
Thanks, please like, subscribe and share.
Does The articulation points (yellow ones) work to majore the rest (snake's queue) to make the best way to prevent head accident? and the point on the eaten square parallely minimize the distançe between it and the head?
الحمد لله.
There is no distance calculated, its just an On or Off switch if the ray-cast hit the snake, wall or apple.
This would make a cool Screensaver.
I bet there are a lot of people that have no idea what that is.
This is how my dad play video game in his childhood
Bro the algorithm ai used is kinda next level zero error
Give it a Nokia 3210 on a noisy night bus and see how it does! 🤣
Damn, AI accomplished what I always tried to achieve.
The snake eatting it's own tail is a win.
I found that humours also.. it reminded by of a puppy chasing his tail.
The ouroboros
Does it use a pure neural network, or are there additional algorithmic rules coded? Like avoiding obstacles, etc.
The neural network is trained with a genetic algorithm. The generation with the best fitness is who ever takes the most steps and gets the most apples.
I think the only rule on the fitness test is: you have 100 steps to collect and apple, then the steps reset.
So he theory is longest path to the apple.
@ezra anderson I think it would be interesting to use a Q learning approach. Like you mentioned an Actor Critic network would be a cool algorithm to implement for snake. I’m assuming you would be able to get results much faster from a Q algorithm trained by Adam than a genetic algorithm especially since you have a well defined reward. Figuring out the inputs for this would be more tricky since NNs do well with continuous state spaces and snake is more of a discrete state space. Perhaps using all 100 block values as an input could work well for this type.
Do ping pong next
best request :) Will do
great brother i like that
Thank you so much, please like share and comment