Hi Finn, I tried to use NEAT to play snake as well, and I got much better results by giving the distance to the apple in the left, forward-left, forward, forward-right and right directions. I then have the same directions as features, but with the distance to the nearest obstacle (wall or snake). Manages to average about 50 points per game, although I do play on a 25x25 grid, so there is of course also space for getting more points without becoming too large and suiciding
@@finneggers6612 Kill 10%, 10% link mutation, 2% node mutation, 2% weight shift, 2% random weight and 0.5% toggle link. Not sure if these are the most optimal parameters, but I was able to get pretty good results with it. As a quick note, I made sure to scale all the inputs between 0 and 1, where 1 is equal to the diagonal of the playing field
@@afroeuropean5195 Did you use my NEAT implementation? You can check with print_species() how many species you have. Your settings look good. Do you mean 90% survivors or 10% survivors?
Severus Snape I just thought about it for a while and you should definitely use a logarithmic scale. Like 1 if the obstacle is 1 block away 0.5 if the obstacle is 2 blocks away 0.25.... This should highly increase the accuracy
I got it working, the missing files are not needed. I also implemented a thread pool so rateClient can be parallelized. As for the game it self, I changed the inputs to 8 [0-6] are the number of free spaces in a direction (n, nw, w, sw, s, se, e, ne) plus [7] which is still the direction towards the food. A bit better results, than with just 4 inputs, and slower to evolve time. My best results I got was 109, starting with a gnome where all outputs are connected to all inputs and only weights and shift waits were allowed. Again, thank you for taking the time to post this.
Yo Finn, even tho you haven't uploaded for 9 months I still rewatch your tutorials since they perfectly fit my expectations and theres nothing better out there. I would really like to see more.
Hey, that makes me happy to hear! I am working on some other projects myself very frequently and I do not find the time to work on my videos. I thought about doing some mechanical topology explanations but decided that it might be too complicated and too big of a project for YT to explain. I also almost did the qlearning tutorials yet I did not find the time to actually make the videos and just ended up writing the scripts... I am really sorry for that but I can only encourage you to write your own little games and if you have questions, you can always ask me on discord : Luecx#0540
i have worked on this again with a c++ project and better NEAT implementation and found abetter way. maybe of interest for you. instead of just doing that, create a space of like 7x7 around the snakes head. the impotrant part is to rotate that data bsaed on the direction of the snake. basically pretend that not the snake is changing directions but its always going up while the board rotates. this way the output is either left, forward or right and is relative to the pov of the snake. i got a prefect game with that.
Hi Finn! I loved your videos, I used your tutorial to write a c++ implementation. This may be too late but I would like to know if you tried to solve problems like XOR or pole balancing using your neat. I tried XOR with mine following the paragraph in the paper but it didn't really worked, don't know if I messed up something or your implentation misses something from the original one.
I havent tried the pole balancing problem. XOR has worked sometimes. I guess there is a lot of finetuning to the final values. Try increasing the population size maybe
I have also started some while ago porting that entire thing to C++. Its not yet done but potentially a good starting point to compare. github.com/Luecx/NEAT_CPP
@@finneggers6612 Thanks for the quick response! xor does work with a big population, however I don't get any good result if I use values similar to the ones in the paper (150 individuals and they get a result in under 100 generations). Maybe you are right and playing with all the variables has a big impact. Also they start from input and output layer fully connected so this for sure saves some time. I'll try to tweak some paramethes and see how it goes. Thanks also for the github, I'll have a look as soon as possible
BTW I forgot to mention that I made some performance optimizations (not sure why I was reminded of this 9 months later) Some of the nodes were not connected to the output layer, but the Calculator would still do the calculations when feeding forward. Since the nodes were not connected to the output layer, there is no reason to perform the calculations for these nodes. I excluded them from the Calculator if the links to the output layer were disabled
there are other performance issues which you can improve. its not very obvious but you can do these: A)instead of using 2d/3d-arrays, use a 1d array and compute the index. this will in fact be faster. B) when doing A), memory alignment can be very important. this means that you can play around with the indices. this will have a huge impact on performance. I am sure you could get a 200% speedup from A) and B)
I also did this in a recursive way by using a flag that tells if the node has already called calculate() or not. So the getOutput() method in the node is just if(!flag) calculate() and then return output. In this way you just calculate what you need and you don't need to order the hiddden nodes since everything works by itself
I wanted to change the killing algorithm a little bit so the best of all clients would actually survive no matter what. but yet i couldn't figure out how to modify the library files... or is there a better Solution?
geile katze ! Well the problem with that is that you ignore the idea of speciating. The core idea is to have genomes which are worse than others but to keep diversity.
geile katze ! But well if you want to adjust it, look at my github repo. Just search Luecx/AiLibrary and you will find the full NEAT code there somewhere under genetic algorithms
Hi Finn, do you have any suggestions as to how would one go about learning Machine Learning and how to implement it in java. I've been looking at some books and such, but wanted a second opinion before I go shell out some money towards something. Thanks!
I can only tell you what I did. I started when I was young with fully connected neural networks. Actually my very first series is about this topic (and my first implementation). But for neural networks it’s important to understand these topics: - automatic differentiation (especially backpropagation) - learning the difference between supervised/unsupervised - understanding what eta (learning rate does) - understand vanishing gradients - learning different activation functions and error functions including when to use them. Once you are done with this you can maybe look into unsupervised learning methods like genetic algorithms An interesting topic is also AI algorithms that do not require learning like AlphaBeta/MiniMax for chess or any other type of board game. Once you managed to understand genetic algorithms and standard neural networks, you are good to go for any other stuff like convolutions (which are tough) or you can combine genetic algorithms with alphaBeta to let your chess engine learn I am currently working on a chess engine myself :) A good starting point is definitely Wikipedia for the math and basically any “fully connected neural network” webpage
Great video man, just wanted to ask if you could explain the fitness function you used cause I'm trying to implement this Snake AI in python, giving a 'maximum view distance' of one block in every direction as inputs but without great results and I think this is happening probably because of the different fitness function that I used.
well the best fitness function is probably simply the size of the snake. As we deal with unsupervised training, this should not be an issue. It has been some time since i coded that but I think I also added a rule which makes the snake lose if it doesnt eat the apple. Also You probably want to encode the angle of the snake towards the apple so it could theoretically know where to go.
Also you could use a grain for the distance to the wall. For example use 1 if theere is a wall right in front, 1/2 if 1 block away, 1/4 if 2 blocks away, 1/8 .... . So the AI will also see borders from further away
@@finneggers6612 Logarithmic scale I see thanks, I was thinking of implementing a fitness function based on distance travelled and size and also to change the inputs because the behaviour the AI is producing is too simple and does not really do a lot also how much time your AI has been trained to reach the results shown in the video?
@@bruxio76 I sadly dont really remember but I know it wasnt that long. I think it played 10k self-play games or so. Just try to not overly complexify it. If you need help, you can also add me on Discord: Luecx #0540. I usually answer there a bit quicker :)
Hi finn, I followed your playlist of the neat algorithm and cloned the code from your git. Now the problem is I don't really know how to implement it in my code... What i'm trying to achieve is having all clients on the board at the same time and see how they go. To be more specific i'm trying to make an automated snake battle royale where all clients (or maybe each specie's representative) compete with eachother and assign their score on various factors (rank, size, kills)... I have already coded the game but now I can't understand how to get inputs from each snake to each clients, make them chose a direction and set their score. Could you please explain to me what methods should I call for each phase? Thanks.
Okay so basically the Neat object has a list of clients generated. You will need to access this list and each snake in your game will get a client object. Something like this I guess: public class Snake{ //this is your class private Client client; ... public Snake(Client client){ this.client = client; } } Somewhere in your code you will have something like this: Neat neat = new Neat(inputSize,outputSize,clientAmount) for(int i = 0; i < clientAmount; i++){ snakes[i] = new Snake(neat.getClient(i)); } You can get the client by calling getClient(index) on the neat object. Once you have the client, you can call calculate(inputs) where inputs is a double array and it outputs a double array aswell. The input/output size is determined in the neat object.
You can download the source files from the linked github repo. You will also need the libraries in the github repo. You should create a new project in intellij and copy the folders into the project. after linking the libraries, you should be good to go
@Finn Eggers Hi im ######, ## years old and very interested in neural networks and Neat. So i wanted to ask you if there is an easy way to contact you something like discord or so. Because i also wanna study it and when i have questions i can ask you and this would REEEAAAALLLYYY help me. Pls contact me ;) Btw. are you german?
Hi Finn, I tried to use NEAT to play snake as well, and I got much better results by giving the distance to the apple in the left, forward-left, forward, forward-right and right directions.
I then have the same directions as features, but with the distance to the nearest obstacle (wall or snake).
Manages to average about 50 points per game, although I do play on a 25x25 grid, so there is of course also space for getting more points without becoming too large and suiciding
Oh that is interesting! Thank you for your comment! I will try that aswell. What learning parameters did you use?
@@finneggers6612 Kill 10%, 10% link mutation, 2% node mutation, 2% weight shift, 2% random weight and 0.5% toggle link.
Not sure if these are the most optimal parameters, but I was able to get pretty good results with it.
As a quick note, I made sure to scale all the inputs between 0 and 1, where 1 is equal to the diagonal of the playing field
@@afroeuropean5195 Did you use my NEAT implementation?
You can check with print_species() how many species you have. Your settings look good.
Do you mean 90% survivors or 10% survivors?
Severus Snape I just thought about it for a while and you should definitely use a logarithmic scale. Like
1 if the obstacle is 1 block away
0.5 if the obstacle is 2 blocks away
0.25....
This should highly increase the accuracy
@@finneggers6612 I wrote an implementation in C# based on your java implementation.
I mean 90% survivors
Yay, you're back! Thank you so much for continuing these videos, they really help
I got it working, the missing files are not needed. I also implemented a thread pool so rateClient can be parallelized. As for the game it self, I changed the inputs to 8 [0-6] are the number of free spaces in a direction (n, nw, w, sw, s, se, e, ne) plus [7] which is still the direction towards the food. A bit better results, than with just 4 inputs, and slower to evolve time. My best results I got was 109, starting with a gnome where all outputs are connected to all inputs and only weights and shift waits were allowed.
Again, thank you for taking the time to post this.
Yo Finn, even tho you haven't uploaded for 9 months I still rewatch your tutorials since they perfectly fit my expectations and theres nothing better out there. I would really like to see more.
Hey, that makes me happy to hear! I am working on some other projects myself very frequently and I do not find the time to work on my videos.
I thought about doing some mechanical topology explanations but decided that it might be too complicated and too big of a project for YT to explain.
I also almost did the qlearning tutorials yet I did not find the time to actually make the videos and just ended up writing the scripts...
I am really sorry for that but I can only encourage you to write your own little games and if you have questions, you can always ask me on discord : Luecx#0540
Hi, I do not know if you are still alive, but judging by your SOF, still how =)
Why did you decide not to use your NEAT implementation?
Wow, great idea with inputs to the neural network. I was looking for an info on how to do it!
i have worked on this again with a c++ project and better NEAT implementation and found abetter way. maybe of interest for you. instead of just doing that, create a space of like 7x7 around the snakes head. the impotrant part is to rotate that data bsaed on the direction of the snake. basically pretend that not the snake is changing directions but its always going up while the board rotates. this way the output is either left, forward or right and is relative to the pov of the snake. i got a prefect game with that.
Hi Finn! I loved your videos, I used your tutorial to write a c++ implementation. This may be too late but I would like to know if you tried to solve problems like XOR or pole balancing using your neat. I tried XOR with mine following the paragraph in the paper but it didn't really worked, don't know if I messed up something or your implentation misses something from the original one.
I havent tried the pole balancing problem. XOR has worked sometimes. I guess there is a lot of finetuning to the final values. Try increasing the population size maybe
I have also started some while ago porting that entire thing to C++. Its not yet done but potentially a good starting point to compare.
github.com/Luecx/NEAT_CPP
@@finneggers6612 Thanks for the quick response! xor does work with a big population, however I don't get any good result if I use values similar to the ones in the paper (150 individuals and they get a result in under 100 generations). Maybe you are right and playing with all the variables has a big impact. Also they start from input and output layer fully connected so this for sure saves some time. I'll try to tweak some paramethes and see how it goes.
Thanks also for the github, I'll have a look as soon as possible
Exactly what I ve been trying to learn, thanx man!
BTW I forgot to mention that I made some performance optimizations (not sure why I was reminded of this 9 months later)
Some of the nodes were not connected to the output layer, but the Calculator would still do the calculations when feeding forward.
Since the nodes were not connected to the output layer, there is no reason to perform the calculations for these nodes.
I excluded them from the Calculator if the links to the output layer were disabled
there are other performance issues which you can improve. its not very obvious but you can do these:
A)instead of using 2d/3d-arrays, use a 1d array and compute the index. this will in fact be faster.
B) when doing A), memory alignment can be very important. this means that you can play around with the indices. this will have a huge impact on performance.
I am sure you could get a 200% speedup from A) and B)
I also did this in a recursive way by using a flag that tells if the node has already called calculate() or not. So the getOutput() method in the node is just if(!flag) calculate() and then return output. In this way you just calculate what you need and you don't need to order the hiddden nodes since everything works by itself
I wanted to change the killing algorithm a little bit so the best of all clients would actually survive no matter what. but yet i couldn't figure out how to modify the library files...
or is there a better Solution?
geile katze ! Well the problem with that is that you ignore the idea of speciating. The core idea is to have genomes which are worse than others but to keep diversity.
geile katze ! But well if you want to adjust it, look at my github repo. Just search Luecx/AiLibrary and you will find the full NEAT code there somewhere under genetic algorithms
geile katze ! github.com/Luecx/AILibrary/tree/master/src/genetic_algorithm/neat
Hi Finn, do you have any suggestions as to how would one go about learning Machine Learning and how to implement it in java. I've been looking at some books and such, but wanted a second opinion before I go shell out some money towards something. Thanks!
I can only tell you what I did.
I started when I was young with fully connected neural networks. Actually my very first series is about this topic (and my first implementation).
But for neural networks it’s important to understand these topics:
- automatic differentiation (especially backpropagation)
- learning the difference between supervised/unsupervised
- understanding what eta (learning rate does)
- understand vanishing gradients
- learning different activation functions and error functions including when to use them.
Once you are done with this you can maybe look into unsupervised learning methods like genetic algorithms
An interesting topic is also AI algorithms that do not require learning like AlphaBeta/MiniMax for chess or any other type of board game.
Once you managed to understand genetic algorithms and standard neural networks, you are good to go for any other stuff like convolutions (which are tough) or you can combine genetic algorithms with alphaBeta to let your chess engine learn
I am currently working on a chess engine myself :)
A good starting point is definitely Wikipedia for the math and basically any “fully connected neural network” webpage
Hey Finn, How can I get in touch with you about an open source project that I think you might be interested
You can add me on discord: Luecx@0540
Hello Finn, im trying to implements pacman with NEAT and i rlly struggle with it . if u can help me i will very appreciate it and please let me know
Great video man, just wanted to ask if you could explain the fitness function you used cause I'm trying to implement this Snake AI in python, giving a 'maximum view distance' of one block in every direction as inputs but without great results and I think this is happening probably because of the different fitness function that I used.
well the best fitness function is probably simply the size of the snake. As we deal with unsupervised training, this should not be an issue. It has been some time since i coded that but I think I also added a rule which makes the snake lose if it doesnt eat the apple. Also You probably want to encode the angle of the snake towards the apple so it could theoretically know where to go.
Also you could use a grain for the distance to the wall. For example use 1 if theere is a wall right in front, 1/2 if 1 block away, 1/4 if 2 blocks away, 1/8 .... . So the AI will also see borders from further away
@@finneggers6612 Logarithmic scale I see thanks, I was thinking of implementing a fitness function based on distance travelled and size and also to change the inputs because the behaviour the AI is producing is too simple and does not really do a lot also how much time your AI has been trained to reach the results shown in the video?
@@bruxio76 I sadly dont really remember but I know it wasnt that long. I think it played 10k self-play games or so.
Just try to not overly complexify it.
If you need help, you can also add me on Discord: Luecx #0540. I usually answer there a bit quicker :)
@@finneggers6612 thanks man don't you have a discord server?
I'm trying to code a NEAT ai for snake as well, however I use python instead of java :/
Same here! So, did you make it? Ive got some problems - my snake just chaotic moves around desk xD
Hi finn, I followed your playlist of the neat algorithm and cloned the code from your git.
Now the problem is I don't really know how to implement it in my code...
What i'm trying to achieve is having all clients on the board at the same time and see how they go.
To be more specific i'm trying to make an automated snake battle royale where all clients (or maybe each specie's representative) compete with eachother and assign their score on various factors (rank, size, kills)...
I have already coded the game but now I can't understand how to get inputs from each snake to each clients, make them chose a direction and set their score.
Could you please explain to me what methods should I call for each phase? Thanks.
Okay so basically the Neat object has a list of clients generated. You will need to access this list and each snake in your game will get a client object.
Something like this I guess:
public class Snake{ //this is your class
private Client client;
...
public Snake(Client client){
this.client = client;
}
}
Somewhere in your code you will have something like this:
Neat neat = new Neat(inputSize,outputSize,clientAmount)
for(int i = 0; i < clientAmount; i++){
snakes[i] = new Snake(neat.getClient(i));
}
You can get the client by calling getClient(index) on the neat object.
Once you have the client, you can call calculate(inputs) where inputs is a double array and it outputs a double array aswell.
The input/output size is determined in the neat object.
I am interested in that project! Once its finished, feel free to show it to me. If you dont mind, I could also show this on the channel here
@@finneggers6612 thanks man, there's no problems for me if you want to show it. let's just hope it works😅
Thanks for sharing, well done
How can I implement this in Intelj?
You can download the source files from the linked github repo. You will also need the libraries in the github repo.
You should create a new project in intellij and copy the folders into the project. after linking the libraries, you should be good to go
Q Learning and Reinforcement Learning!! I am really looking forward to it.
How about recurrent neural networks?
Thats what I am working on rn
@@finneggers6612 Okay.
What about evolution strategies or enforced subpopulations?
Simple recurrence? Or gated like JANET?
@Finn Eggers Hi im ######, ## years old and very interested in neural networks and Neat. So i wanted to ask you if there is an easy way to contact you something like discord or so. Because i also wanna study it and when i have questions i can ask you and this would REEEAAAALLLYYY help me.
Pls contact me ;)
Btw. are you german?
Yes I am german. You can contact me via Mail because I like my private things to remain private.
@@finneggers6612 I can understand that ^^ where do i find your Mail?
@@manju_kura you can find it on my YT-channel. i can also give that one to you:
finn.eggers@gmx.de
I used NEAT to evolve my COC
Guys got nothing on me
I suck at chess, and checkers