Thanks for your work sir, it definitely helped me through my implementation project of NEAT. I think reproduction part of the original essay could have been clearer.
I'm confused by the number of offspring calculation at 5:29. Do we round or truncate to get the number for decimal results? For example, species 1's calculation is 5*(6/6.9) = 4.348 but you say it should be 5? Then species 3 is 2*5/6.9 = 1.449 and you say 1? Thanks so much for the series by the way!
EDIT: This is not correct, see edit at bottom. Allowed offspring: Avg F / Global Avg * N = (Tot F / N) / Global Avg * N = Tot F / Global Avg. Avg FA / Global Avg * N = (Tot F / N^2) / Global Avg * N = (Tot F / N) / Global Avg = Avg F / Global Avg Not sure why you're complicating things so much. Overall a great series that has been to much help as I've been building my own NEAT system. Thanks! Edit: After a long break from working on my own implementation of NEAT and recently coming back to it I've redone a lot of the work and realized that there's an issue here. It's the global average adjusted fitness, not just the global average as claimed in the video. Thus you can't simplify as I did in my comment. Instead it simplifies to: Avg Fs' / sum(Avg Fs) * Np where Fs' refers to a specific species fitness (not adjusted), Fs any species fitness (not adjusted) and Np is the global population size. So in short, take the average fitness (not adjusted) of the species you want to calculate the number of members for, divide it on the sum of all species' average fitnesses (not adjusted) and mutliply by the total population size.
First off I want to say great videos! They are most definitely very helpful! Second, I am hoping you can help clear some things up for me. I'm not sure I am understanding exactly how Speciation is performed. Currently, I have it set so that at the end of each generation it creates species from scratch, then shares the fitness and multiplies based on that. However, doing it that way does not allow for a drop-off or a way to kill a species. I watched your Speciation video as well that shows you kill off a species if it doesn't improve over 15 generations. Do you do this by keeping all the species from generation to generation, and recording its best score, and if that score does not improve then you kill it off? I also notice that since the threshold changes species fluctuate above or below the target. Do these numbers drop as you kill species off? Or do these species still exist in some way, just not used? I have tried to find more in-depth information on speciation, but it seems everyone repeats the exact same stuff. This is what I get from watching the video For each generation g: Sort the population into species For each species s: calculate the average adjusted fitness for s calculate the global average adjusted fitness for all species For each species s: offspringCount = s.averageAdjustedFitness / globalAverageAdjustedFitness * s.size List newPopulation += s.createOffspring(offspringCount) repeat By doing this new species are created every generation and I'm pretty sure that's not how it's suppoesd to go. I'm just trying to find clarification on this.
So there's a species target (preferred number of species produced each generation) and a compatibility threshold. If the CT is too high then all members get swept into species 1, too low and each pop. member is assigned to its own species.. So the CT will fluctuate which allows each generation to try and produce the desired number of species for your population (i.e. species target).. At the end of each generation each pop. member has its species id zeroed and speciation occurs i.e. a pop member is selected at random to be the Species 1 champion and all other pop members are compared to it.. If that comparison is below the CT then it also gets assigned to Species 1.. Once complete, an unassigned pop. member gets randomly selected to be the species 2 champion and again, all unassigned pop members get compared to it to see if they should belong to species 2.. This repeats until all pop members have been assigned to a species.. It then works out how many kids each species is allowed as per your pseudocode.. I have a separate array which tracks how many generations have passed since a species average fitness has improved.. If it goes over 15 gens then the allowed kids for that species gets set to zero.. I then copy the population into a temp array.. and reinitialize the main population array with the identical basic networks from gen 0.. Crossover then occurs from the population in the temp array and each species produces its allowed number of kids.... and these get put into the main population array.. If for example species 3 was supposed to produce 10 kids, but this got set to zero, once crossover is complete the main population array will have 10 basic networks in it.. Then its onto mutation (weight, nodes, connections) and off it goes again Hope that helps !
@@neatai6702 I see mine is mostly the same. The part I get confused on is if you zero out the species each generation (which I also do), how do you know if a species is going stagnant since you are potenially changing which species is which each generation. Mine get to a stagnant point and just stop progressing. Starting with some fresh networks would probably solve this for me so that part is very helpful. I'm just not able to picture how each species is tracked for 15 generations. For example, say you have 10 in species 1, and 10 in species 2, you mate them each generates 10 new ones. Now the next generation comes, each member has no species yet. So you do the species check. Say a member that would fit in the privious species 2 is now set to the species 1 champion randomly. How can you add a counter to that seperate array saying it hasn't progressed when it's not even the same species as it was the last generation? Also thanks for the response. Your videos are by far the best I have seen on the subject so far! Love them!
Hi Steve.. your comments got me thinking, so thanks for that. You're absolutely correct.. For the my latest video I made the change so species members stay in their species unless a mutation forces them out. I no longer zero them and start from scratch on each generation.. I can't find the reference to zero them, and it just makes more intuitive sense to leave them where they are.. This had the nice side effect of letting me track the genealogy of the nets through the generations.. On more than one occasion a mutation would result in a single net starting its own species which would then go on to dominate..
@@neatai6702 I had thought about doing that as well. However on my quest to better understand exactly how speciation is greatly improving genetic algorithms i discovered its a combination of many small tweaks that play a fairly big role. Even though NEAT is a massive improvement over standard GA with fixed NNs I have been pondering for quite some time and have realized that NEAT doesn't really search a lot better than standard GAs, it mostly just gets to the destination a bit faster. I realized this when testing simple GA vs NEAT on the classic snake game. I had about 50 snakes running at the same time. NEAT and SGA (Simple Genetic Algorithm). They would both reach similar solutions (play styles) with NEAT having a small advantage with recurrent connections, and would get to the solutions faster. The main issue i observed is exploration of solutions. I noticed after resetting many times sometimes the snakes would move in a square like fashion, other times they would move in a stepping manner, and all other snakes would start to follow the best performing snakes. Thats when I realized both are just greedily searching for the nearest local optima. Making it incredibly difficult to get out of it once in. That being said I have been doing some deep thinking on solutions to this problem and am slowly iterating over ideas. If I come up with a solid system I'll let you know if you would be interested in trying it out?
I am so confused at 2:45-3:00. why is in_id_16 9 and 12 not part of d, and why is 11,19,29 not part of e? are these the only one's enabled? maybe highlight that or...?
Awesome video! This really helped me understand how speciation is done. Speciation is the main hurdle in developing my own implementation of NEAT.
Glad it was helpful! Crossover is next; but agreed, speciation is the tough part..
Could k-medoids be easier?
This was a very clear explanation of speciation, thank you!
I've been trying to understand NEAT for YEARS!
Thank you, sir and all your work. ^^
great ! hope it helps..
Wow... that was a great vid. I overcomplicated my solution by a tone. Can't wait for crossover. Thank You so much!
You're very welcome!
Thanks for your work sir, it definitely helped me through my implementation project of NEAT. I think reproduction part of the original essay could have been clearer.
I'm confused by the number of offspring calculation at 5:29. Do we round or truncate to get the number for decimal results? For example, species 1's calculation is 5*(6/6.9) = 4.348 but you say it should be 5? Then species 3 is 2*5/6.9 = 1.449 and you say 1?
Thanks so much for the series by the way!
EDIT: This is not correct, see edit at bottom.
Allowed offspring:
Avg F / Global Avg * N = (Tot F / N) / Global Avg * N = Tot F / Global Avg.
Avg FA / Global Avg * N = (Tot F / N^2) / Global Avg * N = (Tot F / N) / Global Avg = Avg F / Global Avg
Not sure why you're complicating things so much.
Overall a great series that has been to much help as I've been building my own NEAT system. Thanks!
Edit: After a long break from working on my own implementation of NEAT and recently coming back to it I've redone a lot of the work and realized that there's an issue here.
It's the global average adjusted fitness, not just the global average as claimed in the video. Thus you can't simplify as I did in my comment.
Instead it simplifies to: Avg Fs' / sum(Avg Fs) * Np
where Fs' refers to a specific species fitness (not adjusted), Fs any species fitness (not adjusted) and Np is the global population size.
So in short, take the average fitness (not adjusted) of the species you want to calculate the number of members for, divide it on the sum of all species' average fitnesses (not adjusted) and mutliply by the total population size.
You have saved me so much time, thank you!
I wonder if cross-species mating would be beneficial or if it would defeat the purpose of speciation. (Maybe allow it, but make it rare??)
its allowed, but very rare (.001%) .. I've never switched it on
can you leave a link to the reference paper
Cool video.
Thanks for the visit
How do you calculate compatibility threshold depending on the target?
First off I want to say great videos! They are most definitely very helpful! Second, I am hoping you can help clear some things up for me.
I'm not sure I am understanding exactly how Speciation is performed. Currently, I have it set so that at the end of each generation it creates species from scratch, then shares the fitness and multiplies based on that. However, doing it that way does not allow for a drop-off or a way to kill a species.
I watched your Speciation video as well that shows you kill off a species if it doesn't improve over 15 generations. Do you do this by keeping all the species from generation to generation, and recording its best score, and if that score does not improve then you kill it off? I also notice that since the threshold changes species fluctuate above or below the target. Do these numbers drop as you kill species off? Or do these species still exist in some way, just not used?
I have tried to find more in-depth information on speciation, but it seems everyone repeats the exact same stuff.
This is what I get from watching the video
For each generation g:
Sort the population into species
For each species s:
calculate the average adjusted fitness for s
calculate the global average adjusted fitness for all species
For each species s:
offspringCount = s.averageAdjustedFitness / globalAverageAdjustedFitness * s.size
List newPopulation += s.createOffspring(offspringCount)
repeat
By doing this new species are created every generation and I'm pretty sure that's not how it's suppoesd to go. I'm just trying to find clarification on this.
So there's a species target (preferred number of species produced each generation) and a compatibility threshold. If the CT is too high then all members get swept into species 1, too low and each pop. member is assigned to its own species.. So the CT will fluctuate which allows each generation to try and produce the desired number of species for your population (i.e. species target)..
At the end of each generation each pop. member has its species id zeroed and speciation occurs i.e. a pop member is selected at random to be the Species 1 champion and all other pop members are compared to it.. If that comparison is below the CT then it also gets assigned to Species 1.. Once complete, an unassigned pop. member gets randomly selected to be the species 2 champion and again, all unassigned pop members get compared to it to see if they should belong to species 2.. This repeats until all pop members have been assigned to a species..
It then works out how many kids each species is allowed as per your pseudocode..
I have a separate array which tracks how many generations have passed since a species average fitness has improved.. If it goes over 15 gens then the allowed kids for that species gets set to zero..
I then copy the population into a temp array.. and reinitialize the main population array with the identical basic networks from gen 0..
Crossover then occurs from the population in the temp array and each species produces its allowed number of kids.... and these get put into the main population array..
If for example species 3 was supposed to produce 10 kids, but this got set to zero, once crossover is complete the main population array will have 10 basic networks in it..
Then its onto mutation (weight, nodes, connections) and off it goes again
Hope that helps !
@@neatai6702 I see mine is mostly the same. The part I get confused on is if you zero out the species each generation (which I also do), how do you know if a species is going stagnant since you are potenially changing which species is which each generation.
Mine get to a stagnant point and just stop progressing. Starting with some fresh networks would probably solve this for me so that part is very helpful.
I'm just not able to picture how each species is tracked for 15 generations. For example, say you have 10 in species 1, and 10 in species 2, you mate them each generates 10 new ones. Now the next generation comes, each member has no species yet. So you do the species check. Say a member that would fit in the privious species 2 is now set to the species 1 champion randomly. How can you add a counter to that seperate array saying it hasn't progressed when it's not even the same species as it was the last generation?
Also thanks for the response. Your videos are by far the best I have seen on the subject so far! Love them!
Hi Steve.. your comments got me thinking, so thanks for that. You're absolutely correct.. For the my latest video I made the change so species members stay in their species unless a mutation forces them out. I no longer zero them and start from scratch on each generation.. I can't find the reference to zero them, and it just makes more intuitive sense to leave them where they are..
This had the nice side effect of letting me track the genealogy of the nets through the generations.. On more than one occasion a mutation would result in a single net starting its own species which would then go on to dominate..
@@neatai6702 I had thought about doing that as well. However on my quest to better understand exactly how speciation is greatly improving genetic algorithms i discovered its a combination of many small tweaks that play a fairly big role.
Even though NEAT is a massive improvement over standard GA with fixed NNs I have been pondering for quite some time and have realized that NEAT doesn't really search a lot better than standard GAs, it mostly just gets to the destination a bit faster.
I realized this when testing simple GA vs NEAT on the classic snake game. I had about 50 snakes running at the same time. NEAT and SGA (Simple Genetic Algorithm). They would both reach similar solutions (play styles) with NEAT having a small advantage with recurrent connections, and would get to the solutions faster. The main issue i observed is exploration of solutions. I noticed after resetting many times sometimes the snakes would move in a square like fashion, other times they would move in a stepping manner, and all other snakes would start to follow the best performing snakes. Thats when I realized both are just greedily searching for the nearest local optima. Making it incredibly difficult to get out of it once in.
That being said I have been doing some deep thinking on solutions to this problem and am slowly iterating over ideas.
If I come up with a solid system I'll let you know if you would be interested in trying it out?
Great Video. Can you explain when a species is going extinct?
Yes, I'll cover that in the next video, due out on Sunday..
I am so confused at 2:45-3:00. why is in_id_16 9 and 12 not part of d, and why is 11,19,29 not part of e? are these the only one's enabled? maybe highlight that or...?