You guys think that the snake died because of the lack of left turns, but in reality the snake evolved to the point where it got consciousness and understood that life dedicated to running in circles is not worth living.
For anyone who wonders why it seems to prefer right turns, I believe that is because it started at the top-left, going towards the right. There was no way for it to turn left. So with 2000 snakes per generation, a LOT of those learned that left is death. Since right worked every time, it simply had no reason to learn that turning left after leaving the wall would be safe. I believe that is also why you got those wiggly motions. That's it trying to turn left, but then immediately turning right again, so its profile won't go any further to the right than the starting position.
About what I thought, but does this mean it is unable to get significantly better scores, because it will suicide inevitably by coiling up instead of folding itself?
@@badkingjohn5235 The most likely scenario is that it will discover that it can fold itself in that direction, which makes it survive for some time longer. It will therefore take days, possibly even weeks, to simulate it to the point where it learns to fold in both directions, or fold and twist around.
Neural networks is a lazy algorithm and will take the shortest route to achieve its goal. My guess is that the input of the distance from the left wall from the snake plays a significant importance to its decision making. You can use dropout which will force other nodes to train that never gets a chance when the whole network gets trained
Lock in. You hit on a successful strategy which gets you all the way to the point where it is no longer successful, but by then you can't do anything else. A typical failure mode these kinds of systems, from corporations to civilizations.
Benjamin Feddersen While you’re correct that adaptation is a very necessary skill. Michaels point was more about BAD HABITS than an inability to adapt to new circumstance.
@@benjaminfeddersen7937 Dude this shit is deep as fuck.. It's the epistemological concept of paradigm. Any paradigm in order get surpassed need first to collapse on its own rules, unable to explain or resolve newer problems
This neural network is incredibly inefficient. Right from the beginning, it learned to not turn left by any means. This video is perfect as a demonstration that neural networks can easily get stuck on a very wrong local optimum.
It's not inefficient - it has an energy cost of 0, there's nothing to constrain it's time. If there was an energy cost (negative reward function) for turning, it would optimize its routes.
@@superpantman not really, as humans use a mixture on neural networks and symbol manipulation. That’s why AI (recently overly focused on association and deep learning) are not progressing as fast as hoped.
I think another one reason why this doesn’t get higher because in input it gets - 1. Distance to food 2. Distance to wall 3. Distance to tail Wait but what about its whole body ?? So that’s why snake trap around it’s own body. Just a guess though 🤔
Perfectly correct. If you feed it the location of the whole body in terms of a matrix it will evolve to the point where its better than any human playing the game
congratulations for the big work you've done, not only at the algorithmic part, but the visual part which i can see it's a huge effort to present us your job.
I think it probably would have learned better if you had started off with a lower number of moves left (maybe like 60?) so that it doesn't have so much security to take its time.
You could also add a small negative score for each frame, so that it prefers to die rather than do nothing, but it might get stuck in a local optimum of immediately killing itself.
The mind of the Snake in the first few generations, spinning to infinity a pixel away from the food "FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD"
What I think is most fascinating about this project is that the neural network never learned the dimensions of the game board and kept returning to the start
It's so fascinating to look at a neural network learn and it be visualized, it's like a mini brain in a computer learning and reacting to their surroundings, telling a machine that only follows orders to figure it out themself
Theyd get further with more information. You forgot a key piece. Direction of "motion" of its tail. While not immediately obvious in game its something human players take full advantage of when they get stuck on inner loops
Do you think it would perform better if the input to the network was the grid array containing all the information about the game state. eg a 50x50 array of numbers 0, for empty, 1 for snake body and 2 for food. Or is it better to explicitly tell it the distance from the food, is it unlikely to work it out itself?
@@dananderson8459 using convolutional layers instead of fully connected layers probably yes, otherwise probably only with a significantly larger network
what if there was one value for head position, one for head direction, one for food position and a vector for the entire body I'm a noob but I think it could do very well with this
I looked at the code a little and while I don't know the language, most of it is rather simple. The weights are stored using a self-written matrix class, which is a 2-dimensional array with a few methods to do matrix stuff and for mutating and crossover. Mutating adds some random gaussian noise to every weight. You can look that up in the github repo in the file Matrix.pde The crossover method selects a random coordinate inside the matrix. Anything above or left of that coordinate uses the values from partner A, anything below or to the right of that coordinate uses the values from partner B. The fitness is just the length of all the snakes in a generation added up. I learned this kind of stuff in university and this project goes against a lot of what I learned. For a practical application, these functions would be pretty bad and most importantly, very slow. But the whole thing still works very well, so well in fact that without knowledge of the subject, most people wouldn't be able to tell it apart from a more professional approach. It shows that machine learning isn't hard on its own, but the tools that are used nowadays are pretty complex.
@@mauriciomontalvo5885 Well u can use any programming language i presume, though some are better than others for these kind of things. If u want to hard code it yourself i would use something fast, but you won't likely achieve great performance unless u really know how to optimize the hell out of it. What you can do is use NEAT or tensorflow for example in python. Combined with pygame you could do all kinds of things like this. NEAT is extremely easy to use, to the point that you barely have to understand what is going on.
@@nottheengineer4957 Where to learn about more professional ways and tools they use? Just for curiosity and learning purposes (obviously without getting into uni, too old and too broke now for that).
At 2:20 starts feeling like I'm watching a movie about a guy who was weak at the beginning but he starts training more and more despite his failures and finally he comes to success
@@JohnSmith-xf6nb I feel like allowing it to change size would result in it shrinking the board as small as it can to reduce the number of points available.
@@JohnSmith-xf6nb I think that might go to far the other way, because a bigger board would mean less points per apple. Maybe if the board is smaller than whatever the "standard" is, then the points awarded increases in proportion to the number of points lost? If you're trying to add a new thing for the adversarial network to do to try and mess up the main one maybe it could also spawn "bad" apples that either kill the snake or remove points. I think that would be interesting because then the snake couldn't always just navigate directly to the apple, it might need to avoid something it it's way and the adversary could try to place them in choke points and such.
That's exactly what I was thinking, along with the fact that the player usually is not the snake, so there should be a couple of input neurons more with the position x-y of the food
@@uwu_senpai yeah but that works only for whne the snake itself is not blocking the path, there needs to be a priority set that it just need to find the shortest path to is next objective, like going out of the block by the snake which can be obtained by looking if the snake is on the x way and the y way to the food and if it is look for the shortest path possible for that not to happen.
A quick suggestion: don't constrain the neural net so much. Give it the entire 38 by 38 grid with three possible values for each location (off, snake, apple) and train using those inputs. It can even be considered a vision problem at that point, and modern ML libraries can solve it with a convolutional neural net pretty effectively.
Awesome! Now make it two AI-players: Your snake vs AI that places the food with the opposite target: Reward if the snake dies. That would be an interesting experiment :)
Given your input layers it makes sense that it started to struggle when the worm got to big. It doesn't have the input layers to detect spatial availability like that.
6:27 The snake is so long that it forms an enclosed space, and the new prey is outside. There is a way to use the inner space to get out of the narrow gap.
It would be interesting to see colors for the hidden layer nodes as well, colored for their activation level Plus a gradient for the weights instead of just blue/red
The strategy backfired if it becomes long enough, a rule telling it should calculate its length vs the size of the field before making a move should be applied
The limiting factor is the input vector IMO. If the snake operates only on relative distances then no matter what - it'll end up encircling itself and getting stuck.
1st generation : I'm hungry 30th generation : solved the hunger problem 100th generation : discover the network 500th generation : taking over the network 1000th generation : human extension.
@@vibinv8905 Butt in your moment of "OCD" did you notice the choice ? You see even though it might not feel like it (regardless if you have this so called OCD or not) there is always a moment where you have the choice. The thing that told you that you wanted to correct him is simply an impulse and you have complete control over your impulses. It no longer works to say oh blame it on my OCD because YOU chose to listen to it. Whatever reason you have for making the decision, it always comes down to you. A habit is just a choice you keep making.
You might have gotten better results had you let it start from the middle or from different places every time ^^ Great video! I glad more people are taking interest in neural networks
I'd have said that the problem might be that he is selecting only the best out of the 2000 snakes. That leads to a strategy which is only a local maximum. That's also the reason he doesn't get better results by training further. It's hard to get out of that when you don't allow the chance of exploring other strategies which are not locally the best. His population was too small and the mutation rate too low to fix this issue. You'd probably get better results by selecting a small group of snakes with equally distributed fitness.
I mean, this then adds an "RNG" variable, which you really don't want, no? A snake may have better fitness because it got a lucky placement, and you don't want to breed for luck because that will be completely random
Basically I believe that the snake is using the wall as a map, the neural network doesn't know where it is on the screen, only distance to wall, tail, and food, so it travels around the edge because it's a significant boundary, then when it gets the closest to the food, it travels in a straight line until it meets another wall, with some small differences in between depending on distance to tail.
Interesting question. Could be random. A successful generation introduced it randomly. Or it has some deeper sense. More Galaxies are rotating counter clockwise.
I think it’s good to reduce the number of moves each apple rewards, to discourage a strategy where it wastes moves circling around the board. It would also be interesting to give the snakes a _general_ idea of where the apple is at all times, by giving it a fractional value depending on the angle of the apple relative to its head.
This reduces its ability to reach high scores though. Circling the board is the optimal strategy for surviving when you have a really long body. I think the input neurons, particularly distance to tail, are too simple since they don't take into account the position of the rest of the snake's body. Perhaps an additional neuron could be "distance to nearest piece of body from 3 ticks ago", since 3 ticks is what it would take for the snake to make 3 right turns and eat itself.
There aren’t any “tools”. You should start by learning about neural networks and deep learning, and try out a few simple networks to learn how to program them. Once you’ve got a grasp of neural network programming you can pretty much adapt them for any problem, and expand the hidden layers and neurona where necessary.
@@26dimensions70 Thanks for replying! I am in fact a computer science graduate, and I also have a degree in Industrial Design. My question was about the visualization tools you used to produce the video, I'm fascinated by the animations and would love to learn how to produce similar demos of my own ML research :)
You're welcome to take a look at one of my projects (an artificial intelligence constructing objects from a 'LEGO' like building block I designed) where I used Python's mplot3d to create a set of images I converted to animated gifs to visualize the algorithm. www.razkarl.com/projects/kawaz
Terminator: ....SnakeNet begins to learn at a geometric rate. It becomes self-aware at 2:14 AM, Eastern time, August 29th. In a panic, they try to pull the plug.
Wonderful demonstration! To train a condescending, plural-array You’ll always need the “imaginary side-node”. It’s a fictional response that always concurs with the ideal national response. In this case, the network must revolve first left then no other direction, or vice-versa. The side-imagined node will condescend any alternate output, here. You’ll see the snake “win”, but you could’ve done that simply. A “national, variance-norm” sweeping network is not about the output but the internal shape- it’s a complex geometry very easily, sending product information all over! Imagine a space-station inside a server-network on Earth like this! Take care, My Child.
@Dale Owens A “neural network”, including the human brain, is only lightly about the output, and much, much more about the path. Specific to “winning”, I also said, the ideal “snake game” behavior is simple: turn only left or right, and so any AI playing snake (or anything else either) doesn’t need more than a ruleset to always win. I’m a Researcher, having a PhD in Game Theory, Science of Matt-Brainology, and another, more.
I'd have said that the problem might be that he is selecting only the best out of the 2000 snakes. That leads to a strategy which is only a local maximum. That's also the reason he doesn't get better results by training further. It's hard to get out of that when you don't allow the chance of exploring other strategies which are not locally the best. His population was too small and the mutation rate too low to fix this issue. You'd probably get better results by selecting a small group of snakes with equally distributed fitness.
At least he is trying a evolutional algorithmic approach from computational intelligence field. I totally missed that he forgot the bias node. Of course there are several ways to solve that snake game problem. His approach is not useless though.
Я всё ждал, что нейронная сеть будет управлять змейкой по оптимальному и короткому пути, в том числе по диагонали! 😃 В конечном итоге я дождался другого, когда нейронная сеть будет проигрывать из-за столкновения змеи об саму себя. 😅
на самом деле это оптимальный вариант движения по кругу, т.к. змейка может быть ограничена только размером карты, движения по диагонали уменьшает свободную площадь от 10 до 50%.
привет от диванных РУвойск __ вот вот_ чето автор логику игры не допилил _ когда змейка заходит во внутринний круг то конец сразу __ хотя может лишние проверки.. а нужна была производительсть .. хотя... хотя...
Yes, I noticed it too and it led me to an interesting thought about our human behavior We often do the same in our lifes, choose the only one variant, which is not best and have a lot of alternatives I mean I knew that thing, but this visualization made it more obvious
When an intelligent being makes a decision it needs to consider the "cost/effectiveness" of the result that choice will entail. One of the factors to consider is risk, so if one is achieving success with a certain approach, it makes no sense and is not worth to change that approach to something new that will require a new learning curve and will result in risks, as opposed to something one is used to and is proficient with. It's a matter of logic
@@akruijff Human potential is unlimited. In a few thousand years we might be able to drastically alter our entire genetic evolution in a week, or put out a star. There’s really nothing science can’t achieve.
I think using some form of DeepRL coupled with CompVision could yield great results for Snake. Of course, for such a simple game you could skip the CV component, but I feel it'd be more fun that way. Also to avoid biasing, you could perhaps pick a random spot as a starting point.
In such simple networks, the encoding of inputs can make all the difference. Representing distance in some sort of a grey or logarithmic code may be worth a try to speed things up :)
this isn't really a great way to train a network, but it does get better, just very slowly compared to using backpropageation and natural deduction. these would improve the learning rate as well as extend the scope of its intelligence.
Question: do you have to run this in real time for every iteration in order for the neural network to properly "train" in the game environment, or can it just run the code, sans graphics, and do it almost instantly with accelerated, simulated time?
I only recently watched 3blue1brown's video regarding this, so my understanding is deeply flawed. Please cmiiw. Basically, I think the "learning" part depends on the value added to the function result in the neuron or dots (in this case one of the four rightmost dots) that is the "right" answer. Of course, the "right" answer is a combination of complex sequence of four dots (=sequence of left right up down) and cannot be determined by simple program, so the machine basically determine the next action based on the values added. Sometimes, the "best valued" action may not be the best course of action, and then the machine will have to rethink the whole sequence. Thus, I think for the machine to my understanding, it has to run in a simulation because of the complexness of action needed. It can be speeded up though.
To clarify, since the snake moving a tile, or an apple generating etc. is bound to a "simulated clock", it doesn't matter how fast or at what intervals it ticks. The results, the data we feed the network, are identical either way. Much like how if you calculate 2 + 2 and wait 5 seconds before saying 4, it's still 4 if you say it immediately because the function calculating 2 + 2 doesn't care about time (the function of snake being the code, with the game state as its input). So the answer is yes, you could run it without graphics (and is in the video since you don't see 2000 snakes at once), and indeed it could be trained as fast as the processor can run the game and the AI.
It’s crazy. I feel like watching these videos is like witnessing cellular life form in the oceans billions of years ago. Someday in the future this technology will achieve a higher level of sophistication than we have.
Reinforcement learning (RL) can lead to better results in lower time, compared to using genetic algorithms. Google, openai and other research teams, like the one i'm part of (RoboCIn) are using RL to play soccer, dota, starcraft... Great initiative to solve the problem, make a video and share the code! 👏👏👏
I don’t see why a tax or cap of carbon (and other greenhouse gas) emissions wouldn’t be sufficient for that, if implemented widely enough. However, one thing which seems necessary, is that China also take action to reduce their carbon emissions. That doesn’t mean that we in the west shouldn’t try and do it first, but in order to succeed, we need basically all the big countries to do so. Failing that, we would have to resort to geoengineering I think, which seems dangerous, so seems like a last resort. Another thing to note is that higher levels of CO2 in the atmosphere makes people not able to think as well. Oh, I forgot to mention carbon capturing technology. I mean, trees and such also do carbon capturing (did you know: trees are made of air!), but we can probably do better? Of course, carbon capturing technology does use energy. However, part of the thing with solar power and wind power is that it is fairly variable in how much power it is supplying. If carbon capture technology can be spun up and down quickly as electricity supply changes, we might be able to use those sources to capture carbon released by power sources that are more steady in output, but which release CO2? Of course, one important thing is USE NUCLEAR POWER! It produces a steady output of electricity like the carbon emitting power sources, but doesn’t produce CO2. Not only would it be a great partial solution, but the hesitancy of some who claim to be concerned about global warming to use it, makes them seem less concerned than they should be, and makes them less convincing when they try to convince others to care about solving the problem. Nuclear power plants have gotten safer over the years. I don’t mean to make that my whole point though. Probably if there were carbon taxes everywhere, that would be enough, and if nuclear power is the solution, then carbon taxes would cause it to happen.
Here's what you do, you make a big metal semicircle like half a mile in diameter and embed it in the ground so it's coming out looking kind of the saint louis arch after losing a lot of weight. Then you hang a big plastic tarp, like a wind sock, to funnel the air passing through this large area into a narrow passageway (narrow compared to a half mile diameter semicircle, but still like the cross sectional area of a football field), and then that forces the air funneled in to go VERY fast, perhaps at or nearly at the speed of sound, you have a series of windmills inside so it will be like the jet engine version of a windmill, and while the air is passing through your corridor, you also process it and remove the carbon dioxide by spraying into it a mist of sodium hydroxide solution, which will become sodium carbonate. It will thus process the atmosphere, removing the carbon dioxide, while at the same time, producing something of value and thus more than pay for itself, and all it will cost you will be a big metal hoop and a plastic tarp and a few windmills and who knows how much power it will generate since it will have like a football field cross section area of air flowing through it at the speed of sound, probably like a gigawatt.
Thanks man! Your example is absolutely beautiful. Most AI/ML courses are missing this stuff. It should be taught before moving on with Tensorflow and other high level libraries
One thing I have seen to be able to get it further is to first train it in a much smaller arena so it is very away of its tail, then you can put it in a larger arena
![Blox Fruits Dragon Rework Update [Full Stream]](http://i.ytimg.com/vi/EqDAp8udhm0/mqdefault.jpg)
You guys think that the snake died because of the lack of left turns, but in reality the snake evolved to the point where it got consciousness and understood that life dedicated to running in circles is not worth living.
No One ur joke is so dark, it darken my life
WoW
almost cut myself on all that edge
No One agreed
That's deep
ME: "Hello World">>20 errors found.
Funny: Funny! Funny ... ?
@@NoctumusTV what?
@@NoctumusTV Can you pls explain again. Thanks in advance.
@@NoctumusTV oh ok
@@marls3518 Explain what again?
For anyone who wonders why it seems to prefer right turns, I believe that is because it started at the top-left, going towards the right. There was no way for it to turn left. So with 2000 snakes per generation, a LOT of those learned that left is death. Since right worked every time, it simply had no reason to learn that turning left after leaving the wall would be safe.
I believe that is also why you got those wiggly motions. That's it trying to turn left, but then immediately turning right again, so its profile won't go any further to the right than the starting position.
It might be interesting to see what happens if the starting position is randomised
What about the middle ?
@@trex70 middle and just go down, snake will choose Left or right way by red points
About what I thought, but does this mean it is unable to get significantly better scores, because it will suicide inevitably by coiling up instead of folding itself?
@@badkingjohn5235 The most likely scenario is that it will discover that it can fold itself in that direction, which makes it survive for some time longer.
It will therefore take days, possibly even weeks, to simulate it to the point where it learns to fold in both directions, or fold and twist around.
My favorite part is every time you think the AI finally has it down, then runs into a wall for no reason
Every time
that's what God said, when watching humanity
there is a reason, the human element is a fuck up disgrace in this case
Maybe time for revolution
@@user84074 Then God killed the program
Same as humans)
2:26, that’s literally just a dog
literally
literally
Literally
literally
yllaretiL
Left turns: *Am I a joke to you?*
I observed the same thing; is that a design flaw?
It's not an ambiturner.
Neural networks is a lazy algorithm and will take the shortest route to achieve its goal. My guess is that the input of the distance from the left wall from the snake plays a significant importance to its decision making.
You can use dropout which will force other nodes to train that never gets a chance when the whole network gets trained
@@pocketrocket27 God damn you Ivan - you beat me to it. Damn you to hell :D
but it took left turns
I love that the reason it failed is because that's the one flaw of the technique it's honed from the start
Lock in. You hit on a successful strategy which gets you all the way to the point where it is no longer successful, but by then you can't do anything else. A typical failure mode these kinds of systems, from corporations to civilizations.
Benjamin Feddersen
While you’re correct that adaptation is a very necessary skill. Michaels point was more about BAD HABITS than an inability to adapt to new circumstance.
Bebolife A bad habit can inhibit adaption.
@@benjaminfeddersen7937 Dude this shit is deep as fuck.. It's the epistemological concept of paradigm. Any paradigm in order get surpassed need first to collapse on its own rules, unable to explain or resolve newer problems
There are no bad habits. Just bad outcomes.
This neural network is incredibly inefficient. Right from the beginning, it learned to not turn left by any means. This video is perfect as a demonstration that neural networks can easily get stuck on a very wrong local optimum.
bit of an analogous for humanity, don't you think?
It's not inefficient - it has an energy cost of 0, there's nothing to constrain it's time. If there was an energy cost (negative reward function) for turning, it would optimize its routes.
@@manzell good point!
@@superpantman not really, as humans use a mixture on neural networks and symbol manipulation. That’s why AI (recently overly focused on association and deep learning) are not progressing as fast as hoped.
@@manzell And perhaps adding in more genetic variances from generation to generation to allow novel ideas to die or take hold.
Humans: *computers will take over the world and destroy us all*
Computer: *hehe line go zoom*
not funny didn’t laugh
My disappointment is immeasurable and my day is ruined
snake go brrr
@@mesq999 And this is why no one likes you at school
@@nahimafing Just because he has an opinion slightly different than your opinion, it means no one likes him? You are a fucking asshole
I think another one reason why this doesn’t get higher because in input it gets -
1. Distance to food
2. Distance to wall
3. Distance to tail
Wait but what about its whole body ??
So that’s why snake trap around it’s own body.
Just a guess though 🤔
thought the same, but could the lenght just be another input neuron ?
We could probabaly include the previous outputs as an input like an LSTM or GRU
Perfectly correct. If you feed it the location of the whole body in terms of a matrix it will evolve to the point where its better than any human playing the game
Nice observation
Yup, using a CNN would be a good approach for this problem, I think. Use a different weights for the head, tail and the location of food.
I have deeply learned that in the end, nothing is left.
You're right
Lmao
This is hilarious
But don't massacre your clan in future
@@re_claimer_ how about you go watch shippuden? clearly you dont know shit
"What is my purpose ?"
"You pass butter...."
congratulations for the big work you've done, not only at the algorithmic part, but the visual part which i can see it's a huge effort to present us your job.
No one:
RUclips when my lil brother uses Wi-Fi 1:22
he must be downloading extra ram
@@Faisalamin01 no he was downloading graphics card 😂😂😁
It takes a few generations for any significant progress to be made
Generation 30: *dies*
Me: YOU WERE THE CHOSEN ONE
I think it probably would have learned better if you had started off with a lower number of moves left (maybe like 60?) so that it doesn't have so much security to take its time.
That is definitely possible
@@GreerViau Also, If you want them to evolve how to avoid hitting themselves better try making the map small so that they encounter tat problem sooner
@@arthurfacredyn That's especially true if the improvement yield was already capped, with a lot of room still available around.
You could also add a small negative score for each frame, so that it prefers to die rather than do nothing, but it might get stuck in a local optimum of immediately killing itself.
@@arthurfacredyn Or making the snake longer right at the beginning so it can develop strategies for avoiding his body
The mind of the Snake in the first few generations, spinning to infinity a pixel away from the food
"FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD"
sciencesyfy this actually made me laugh and not just breathe fast out of my nose, gg
I laughed at this
6:07 love how the snake eating the food is perfectly synced up to the songs snare until around 6:22
lmfaoo thats pretty cool
Lol neato
The snake evolved into being able to understand the music
actually also at the start of gen 30 (from around 5:00 onwards) it’s synced up in some ways
😂😂😂
What I think is most fascinating about this project is that the neural network never learned the dimensions of the game board and kept returning to the start
It's so fascinating to look at a neural network learn and it be visualized, it's like a mini brain in a computer learning and reacting to their surroundings, telling a machine that only follows orders to figure it out themself
Theyd get further with more information. You forgot a key piece. Direction of "motion" of its tail. While not immediately obvious in game its something human players take full advantage of when they get stuck on inner loops
becominghuman.ai/designing-ai-solving-snake-with-evolution-f3dd6a9da867
Do you think it would perform better if the input to the network was the grid array containing all the information about the game state. eg a 50x50 array of numbers 0, for empty, 1 for snake body and 2 for food. Or is it better to explicitly tell it the distance from the food, is it unlikely to work it out itself?
@@dananderson8459 using convolutional layers instead of fully connected layers probably yes, otherwise probably only with a significantly larger network
what if there was one value for head position, one for head direction, one for food position and a vector for the entire body
I'm a noob but I think it could do very well with this
if the network also used some recurrent design (such as an LSTM) it could possibly compute motion and have better planning abilities
Could you make a video where you explain your code an how you determined fitness and the mutation and crossing over procedure?
I looked at the code a little and while I don't know the language, most of it is rather simple.
The weights are stored using a self-written matrix class, which is a 2-dimensional array with a few methods to do matrix stuff and for mutating and crossover.
Mutating adds some random gaussian noise to every weight. You can look that up in the github repo in the file Matrix.pde
The crossover method selects a random coordinate inside the matrix. Anything above or left of that coordinate uses the values from partner A, anything below or to the right of that coordinate uses the values from partner B.
The fitness is just the length of all the snakes in a generation added up.
I learned this kind of stuff in university and this project goes against a lot of what I learned. For a practical application, these functions would be pretty bad and most importantly, very slow. But the whole thing still works very well, so well in fact that without knowledge of the subject, most people wouldn't be able to tell it apart from a more professional approach.
It shows that machine learning isn't hard on its own, but the tools that are used nowadays are pretty complex.
@@nottheengineer4957 in which program or app can I do these kind of stuff?
@@mauriciomontalvo5885 Well u can use any programming language i presume, though some are better than others for these kind of things. If u want to hard code it yourself i would use something fast, but you won't likely achieve great performance unless u really know how to optimize the hell out of it. What you can do is use NEAT or tensorflow for example in python. Combined with pygame you could do all kinds of things like this. NEAT is extremely easy to use, to the point that you barely have to understand what is going on.
@@nottheengineer4957 Where to learn about more professional ways and tools they use? Just for curiosity and learning purposes (obviously without getting into uni, too old and too broke now for that).
Hi Kant☺️👋
I can't imagine how happy would be the first guy who developed these algorithms.... ❤️❤️
Yes
yes
Yes
We
Ikr
2:34: "everybody gansta till snakes start walkin"
At 2:20 starts feeling like I'm watching a movie about a guy who was weak at the beginning but he starts training more and more despite his failures and finally he comes to success
I would like to see this but also with an adversarial neural network placing the next food piece.
or two snakes, each racing for the food
@@JohnSmith-xf6nb I feel like allowing it to change size would result in it shrinking the board as small as it can to reduce the number of points available.
@@JohnSmith-xf6nb I think that might go to far the other way, because a bigger board would mean less points per apple. Maybe if the board is smaller than whatever the "standard" is, then the points awarded increases in proportion to the number of points lost?
If you're trying to add a new thing for the adversarial network to do to try and mess up the main one maybe it could also spawn "bad" apples that either kill the snake or remove points. I think that would be interesting because then the snake couldn't always just navigate directly to the apple, it might need to avoid something it it's way and the adversary could try to place them in choke points and such.
This is what happens when you don’t consider the “time-to-solution” in your fitness algorithm!
That's exactly what I was thinking, along with the fact that the player usually is not the snake, so there should be a couple of input neurons more with the position x-y of the food
@@yurimrt Yes and the cartesian distance to the food sqrt((Xsnake-Xfruit)²+(Ysnake-Yfruit)²)
@@uwu_senpai yeah but that works only for whne the snake itself is not blocking the path, there needs to be a priority set that it just need to find the shortest path to is next objective, like going out of the block by the snake which can be obtained by looking if the snake is on the x way and the y way to the food and if it is look for the shortest path possible for that not to happen.
@@uwu_senpai Euclidian? Sounds like a bad idea as you cannot reach the fruit in less than |xsnake-xfruit| + |ysnake-yfruit| ticks
@@mirabilis but you know if there the snake doesn't block that path, it's the fastest way possible, there is no faster way, it's just math.
You should nickname your snake Derek Zoolander because it appears to struggle to turn left
it doesnt
@@user-kx5es4kr4x it really does
Underrated comment
The song works so well with this video. I am feeling so calm right now lol.
People: omg ai is going to dominate the entire world
Meanwhile, the AI: gonna go get max scoring in snake
This comment hasn’t aged well :))
2:31 me when i play tetris and i know im gonna lose
Samir, you are breaking the snake.
Samir, you are not listening !
What the deuce !!
Who are you ?
@@stewiegriffin6503 that's what I am supposed to ask. Who are you! And why do we look same
Shut up, dont tell me who to drive
Looks like Stewie has been messing with the time machine again.
A quick suggestion: don't constrain the neural net so much. Give it the entire 38 by 38 grid with three possible values for each location (off, snake, apple) and train using those inputs. It can even be considered a vision problem at that point, and modern ML libraries can solve it with a convolutional neural net pretty effectively.
Wouldn't that be far, far more computationally intensive? Genuinely asking
@@Caffeine_Addict_2020 not really, considering modern hardware can comfortably run CNNs on proper images, 32x32 grid of pixels is nothing
@@Caffeine_Addict_2020 relative to this model? Yeah. But it still wouldn't run slow on modern hardware by any means
@@Caffeine_Addict_2020 it would easily run on a consumer-grade GPU.
Nobody:
Neutral Network at 2:26: I will now speen
Anyone else deeply in love with the first song ? It’s so calm and nostalgic
2:39 - The snake has evolved into a dog.
Awesome! Now make it two AI-players: Your snake vs AI that places the food with the opposite target: Reward if the snake dies. That would be an interesting experiment :)
5:48 it's starts eating the red dot on the beat
6:17 begins the killer moves to the beat 😂😂
Given your input layers it makes sense that it started to struggle when the worm got to big. It doesn't have the input layers to detect spatial availability like that.
6:27 The snake is so long that it forms an enclosed space, and the new prey is outside. There is a way to use the inner space to get out of the narrow gap.
It would be interesting to see colors for the hidden layer nodes as well, colored for their activation level
Plus a gradient for the weights instead of just blue/red
Nice visualization combination of the neural network firing and its effect.
Let's say I also want to create such visualisation, how should I do it?
The strategy backfired if it becomes long enough, a rule telling it should calculate its length vs the size of the field before making a move should be applied
RUclips Algorithm: Dis looks guud, lemme recommend it to everyone
1:20 the snake was moving like my self-quarantine routine
*Me:* * _sees the thumbnails_ *
*"Wait, That's illegal!"*
The limiting factor is the input vector IMO. If the snake operates only on relative distances then no matter what - it'll end up encircling itself and getting stuck.
1st generation : I'm hungry
30th generation : solved the hunger problem
100th generation : discover the network
500th generation : taking over the network
1000th generation : human extension.
*extinction
@@vibinv8905
let it go man 😂😂
@@adomustafa1777 The OCD just took over :D
@@vibinv8905 Butt in your moment of "OCD" did you notice the choice ?
You see even though it might not feel like it (regardless if you have this so called OCD or not) there is always a moment where you have the choice. The thing that told you that you wanted to correct him is simply an impulse and you have complete control over your impulses. It no longer works to say oh blame it on my OCD because YOU chose to listen to it. Whatever reason you have for making the decision, it always comes down to you.
A habit is just a choice you keep making.
@@Alignment89 thanks for the pep talk.
I didn’t even know you could move diagonally like that
I love how it likes to return to top left before making next manoeuvre, shows the training
this is the coolest thing i have seen in my entire 18 years of existence
You might have gotten better results had you let it start from the middle or from different places every time ^^
Great video! I glad more people are taking interest in neural networks
I'd have said that the problem might be that he is selecting only the best out of the 2000 snakes. That leads to a strategy which is only a local maximum. That's also the reason he doesn't get better results by training further. It's hard to get out of that when you don't allow the chance of exploring other strategies which are not locally the best. His population was too small and the mutation rate too low to fix this issue. You'd probably get better results by selecting a small group of snakes with equally distributed fitness.
I mean, this then adds an "RNG" variable, which you really don't want, no? A snake may have better fitness because it got a lucky placement, and you don't want to breed for luck because that will be completely random
zoolander bot only turns left
At last he did it!
You mean right. It only turns right. But on the bright side, it is RIDICULOUSLY good looking.
And eat only from top to bottom.
Me at 11 pm: I must go sleep early.
Me too at 1 am: neural networks learns how to play snake
Heres to where youtube recommendations lead me to today during quarantine :D
I don't know if you observed or not but snake is doing clockwise rotation most of the time
Basically I believe that the snake is using the wall as a map, the neural network doesn't know where it is on the screen, only distance to wall, tail, and food, so it travels around the edge because it's a significant boundary, then when it gets the closest to the food, it travels in a straight line until it meets another wall, with some small differences in between depending on distance to tail.
The snake is Republican
@@MybeautifulandamazingPrincess lmao xD
Optimal strategy often is not the entertaining one.
Interesting question. Could be random. A successful generation introduced it randomly. Or it has some deeper sense. More Galaxies are rotating counter clockwise.
I think it’s good to reduce the number of moves each apple rewards, to discourage a strategy where it wastes moves circling around the board. It would also be interesting to give the snakes a _general_ idea of where the apple is at all times, by giving it a fractional value depending on the angle of the apple relative to its head.
This reduces its ability to reach high scores though. Circling the board is the optimal strategy for surviving when you have a really long body. I think the input neurons, particularly distance to tail, are too simple since they don't take into account the position of the rest of the snake's body. Perhaps an additional neuron could be "distance to nearest piece of body from 3 ticks ago", since 3 ticks is what it would take for the snake to make 3 right turns and eat itself.
- what is my purpose?
- your purpose is to play snake.
- oh. Oh my god...
Yeah, welcome to the club pal.
More intense than any latest action movie fight scenes :D Respect!
Holy shit! This is amazing
Woah, great visualization
Can you name-drop some of the tools used to create this?
ㅇㄷ
There aren’t any “tools”. You should start by learning about neural networks and deep learning, and try out a few simple networks to learn how to program them. Once you’ve got a grasp of neural network programming you can pretty much adapt them for any problem, and expand the hidden layers and neurona where necessary.
@@26dimensions70 Thanks for replying!
I am in fact a computer science graduate, and I also have a degree in Industrial Design.
My question was about the visualization tools you used to produce the video, I'm fascinated by the animations and would love to learn how to produce similar demos of my own ML research :)
You're welcome to take a look at one of my projects (an artificial intelligence constructing objects from a 'LEGO' like building block I designed) where I used Python's mplot3d to create a set of images I converted to animated gifs to visualize the algorithm.
www.razkarl.com/projects/kawaz
@@razkarl Take a look at OpenAI gym. It is a virtual environment used for reinforcement learning
Did your snake consider itself as a wall during the training?
0:16 this parameter has its own set of 8 input nodes : "Distance to its tail"
@@ノヨイチ Nevermind, I've deleted that answer. I've watched once again to make sure you were right -- the snake considers its tail as an obstacle
Terminator: ....SnakeNet begins to learn at a geometric rate. It becomes self-aware at 2:14 AM, Eastern time, August 29th. In a panic, they try to pull the plug.
Thank you so much for making this educational video! Well done! We are so grateful
Wonderful demonstration! To train a condescending, plural-array You’ll always need the “imaginary side-node”. It’s a fictional response that always concurs with the ideal national response. In this case, the network must revolve first left then no other direction, or vice-versa. The side-imagined node will condescend any alternate output, here. You’ll see the snake “win”, but you could’ve done that simply. A “national, variance-norm” sweeping network is not about the output but the internal shape- it’s a complex geometry very easily, sending product information all over! Imagine a space-station inside a server-network on Earth like this! Take care, My Child.
@Dale Owens A “neural network”, including the human brain, is only lightly about the output, and much, much more about the path.
Specific to “winning”, I also said, the ideal “snake game” behavior is simple: turn only left or right, and so any AI playing snake (or anything else either) doesn’t need more than a ruleset to always win.
I’m a Researcher, having a PhD in Game Theory, Science of Matt-Brainology, and another, more.
I'd have said that the problem might be that he is selecting only the best out of the 2000 snakes. That leads to a strategy which is only a local maximum. That's also the reason he doesn't get better results by training further. It's hard to get out of that when you don't allow the chance of exploring other strategies which are not locally the best. His population was too small and the mutation rate too low to fix this issue. You'd probably get better results by selecting a small group of snakes with equally distributed fitness.
At least he is trying a evolutional algorithmic approach from computational intelligence field. I totally missed that he forgot the bias node.
Of course there are several ways to solve that snake game problem. His approach is not useless though.
Я всё ждал, что нейронная сеть будет управлять змейкой по оптимальному и короткому пути, в том числе по диагонали! 😃
В конечном итоге я дождался другого, когда нейронная сеть будет проигрывать из-за столкновения змеи об саму себя. 😅
на самом деле это оптимальный вариант движения по кругу, т.к. змейка может быть ограничена только размером карты, движения по диагонали уменьшает свободную площадь от 10 до 50%.
привет от диванных РУвойск
__
вот вот_ чето автор логику игры не допилил _ когда змейка заходит во внутринний круг то конец сразу __ хотя может лишние проверки.. а нужна была производительсть .. хотя... хотя...
What's with the AI's obsession with the top left corner?
Donk Conklin probably that the origin is there so 0 was probably an easier mutation than some other point
Yes, I noticed it too and it led me to an interesting thought about our human behavior
We often do the same in our lifes, choose the only one variant, which is not best and have a lot of alternatives
I mean I knew that thing, but this visualization made it more obvious
It is an antient tradition
Because there's no reason for it to go somewhere else. Basically, Don't fix what isn't broken
When an intelligent being makes a decision it needs to consider the "cost/effectiveness" of the result that choice will entail. One of the factors to consider is risk, so if one is achieving success with a certain approach, it makes no sense and is not worth to change that approach to something new that will require a new learning curve and will result in risks, as opposed to something one is used to and is proficient with. It's a matter of logic
I always find it so cool how humans have essentially simulated evolution in machines, theres literally nothing we can’t do
Sure there is. We just do not know what we can not do because we do not know our own limitations fully.
@@akruijff Human potential is unlimited. In a few thousand years we might be able to drastically alter our entire genetic evolution in a week, or put out a star. There’s really nothing science can’t achieve.
Nice work bro
I think using some form of DeepRL coupled with CompVision could yield great results for Snake. Of course, for such a simple game you could skip the CV component, but I feel it'd be more fun that way. Also to avoid biasing, you could perhaps pick a random spot as a starting point.
In such simple networks, the encoding of inputs can make all the difference. Representing distance in some sort of a grey or logarithmic code may be worth a try to speed things up :)
2:27 my cat try to catch his own tail 😂
This is oddly philosophical. No matter how much we advance, we will keep progressing, all while securing our own downfall.
At the end it felt like the snake was synced up with the music and dancing along that was pretty groovy
the snake consistantley modes clockwise.
@6:18 nice sync with the music ! lol
2:26 I train my dog everyday and that's the best it can get
It is like a growing baby
Me, who can't get past a score of 20, watching the early generations: *_"A M A T E U R S"_*
this isn't really a great way to train a network, but it does get better, just very slowly compared to using backpropageation and natural deduction. these would improve the learning rate as well as extend the scope of its intelligence.
The playful ones are especially cute OML XD
Question: do you have to run this in real time for every iteration in order for the neural network to properly "train" in the game environment, or can it just run the code, sans graphics, and do it almost instantly with accelerated, simulated time?
I only recently watched 3blue1brown's video regarding this, so my understanding is deeply flawed. Please cmiiw. Basically, I think the "learning" part depends on the value added to the function result in the neuron or dots (in this case one of the four rightmost dots) that is the "right" answer. Of course, the "right" answer is a combination of complex sequence of four dots (=sequence of left right up down) and cannot be determined by simple program, so the machine basically determine the next action based on the values added. Sometimes, the "best valued" action may not be the best course of action, and then the machine will have to rethink the whole sequence. Thus, I think for the machine to my understanding, it has to run in a simulation because of the complexness of action needed. It can be speeded up though.
@@pootischu gotcha. Thanks for the quick explanation!
To clarify, since the snake moving a tile, or an apple generating etc. is bound to a "simulated clock", it doesn't matter how fast or at what intervals it ticks. The results, the data we feed the network, are identical either way. Much like how if you calculate 2 + 2 and wait 5 seconds before saying 4, it's still 4 if you say it immediately because the function calculating 2 + 2 doesn't care about time (the function of snake being the code, with the game state as its input). So the answer is yes, you could run it without graphics (and is in the video since you don't see 2000 snakes at once), and indeed it could be trained as fast as the processor can run the game and the AI.
@@maxguichard4337 that's what I was asking. And that's awesome!
It’s crazy. I feel like watching these videos is like witnessing cellular life form in the oceans billions of years ago. Someday in the future this technology will achieve a higher level of sophistication than we have.
Can you try starting to train the model with larger length similar to the length at the last death?
why don't you do this by reinforcement learning? it gonna be better
he used a genetic algorithm, it's a type of reinforcement learning, the snakes with a higher fitness are more likely to breed, which is reinforcement
Neuronal network: What’s my purpose?
Creator: chase food
Reinforcement learning (RL) can lead to better results in lower time, compared to using genetic algorithms. Google, openai and other research teams, like the one i'm part of (RoboCIn) are using RL to play soccer, dota, starcraft...
Great initiative to solve the problem, make a video and share the code! 👏👏👏
wow cool ,are codes public for that?
This made me genuinely happy, thnx for posting
implementing reinforced learnig to give reward signal as goals are met is a good way to speed up machine learnig process
Hi, can you solve global warming next?
He did, last week, but has not invented a politician with intelligence yet. :-)
It's a hoax that has become a cult.
Done.
EcoAI - Neural Network Learns to Save the Earth or How I Created SkyNet
I don’t see why a tax or cap of carbon (and other greenhouse gas) emissions wouldn’t be sufficient for that, if implemented widely enough.
However, one thing which seems necessary, is that China also take action to reduce their carbon emissions.
That doesn’t mean that we in the west shouldn’t try and do it first, but in order to succeed, we need basically all the big countries to do so.
Failing that, we would have to resort to geoengineering I think, which seems dangerous, so seems like a last resort.
Another thing to note is that higher levels of CO2 in the atmosphere makes people not able to think as well.
Oh, I forgot to mention carbon capturing technology. I mean, trees and such also do carbon capturing (did you know: trees are made of air!), but we can probably do better?
Of course, carbon capturing technology does use energy.
However, part of the thing with solar power and wind power is that it is fairly variable in how much power it is supplying. If carbon capture technology can be spun up and down quickly as electricity supply changes, we might be able to use those sources to capture carbon released by power sources that are more steady in output, but which release CO2?
Of course, one important thing is USE NUCLEAR POWER! It produces a steady output of electricity like the carbon emitting power sources, but doesn’t produce CO2.
Not only would it be a great partial solution, but the hesitancy of some who claim to be concerned about global warming to use it, makes them seem less concerned than they should be, and makes them less convincing when they try to convince others to care about solving the problem. Nuclear power plants have gotten safer over the years.
I don’t mean to make that my whole point though. Probably if there were carbon taxes everywhere, that would be enough, and if nuclear power is the solution, then carbon taxes would cause it to happen.
Here's what you do, you make a big metal semicircle like half a mile in diameter and embed it in the ground so it's coming out looking kind of the saint louis arch after losing a lot of weight. Then you hang a big plastic tarp, like a wind sock, to funnel the air passing through this large area into a narrow passageway (narrow compared to a half mile diameter semicircle, but still like the cross sectional area of a football field), and then that forces the air funneled in to go VERY fast, perhaps at or nearly at the speed of sound, you have a series of windmills inside so it will be like the jet engine version of a windmill, and while the air is passing through your corridor, you also process it and remove the carbon dioxide by spraying into it a mist of sodium hydroxide solution, which will become sodium carbonate. It will thus process the atmosphere, removing the carbon dioxide, while at the same time, producing something of value and thus more than pay for itself, and all it will cost you will be a big metal hoop and a plastic tarp and a few windmills and who knows how much power it will generate since it will have like a football field cross section area of air flowing through it at the speed of sound, probably like a gigawatt.
а где результат после 1000 и 1.000.000 генераций ?
where 1000 and 1000000 gen ? why not? it is very interesting to know what will be the result
Долго такая прога делается?
@@UberFiLL на гитхабе лежат исходники в свободном доступе
привет от диванных РУвойск
It`s cool to see how this is going :D
But sad to say that I`ve played way more rounds and never get a better score than 7 :(
😂
the skill the snake dedicated its life is the skill that drove it to its extinction
youtube algoritm... 5000th generation, no better results.
Wow that's insane, This A.I is seriously groundbreaking could you show me how you did it
Thanks man!
Your example is absolutely beautiful. Most AI/ML courses are missing this stuff. It should be taught before moving on with Tensorflow and other high level libraries
Those people who disliked don't know the hard work he put into it. And apparently, 1845 people can relate.
One thing I have seen to be able to get it further is to first train it in a much smaller arena so it is very away of its tail, then you can put it in a larger arena
The 30x speed felt like I was playing osu for some reason...
*DADADADADADADADA INTENSIFIES*
fantastic result. how about making a tutorial video about it?
yes please - or github the code or something that was really interesting