A.I. learns to play | Neural Network + Genetic Algorithm
HTML-код
- Опубликовано: 2 окт 2024
- This is how I created an AI that learns to play the game and beats it!
#neuroevolution #geneticalgorithm #artificialintelligence
Watch the full evolution and relax -
• A.I. beats the Game (f...
For more videos please subscribe -
bit.ly/normaliz...
Support me if you can ❤️
www.paypal.com...
www.buymeacoff...
Article on this -
towardsdatasci...
Source code -
github.com/Suj...
The game -
suji04.github....
The coding train's playlist on genetic algorithm
• 9: Genetic Algorithms ...
Facebook -
/ nerdywits
Instagram -
/ normalizednerd
Twitter -
/ normalized_nerd
----------------------------------------------------------------------------------------
The music used is under a Creative Commons license
Artist: incompetech.com/
Almost in F - Tranquillity by Kevin incompetech.com...
Great simulation. Neural network coupled with genetic algorithm is very interesting. Thanks for sharing!
Thanks! Just went through your channel. Really nice video topics :D
@@NormalizedNerd Many thanks
I really loved this. I will surely learn from your video and implement this on my own.
Thanks! I've uploaded the code on github (link in description). You can check that out to get started.
Vro amazing keep making such ai I love to learn this from you 😊
Great to hear that! :D
Hey, it was awesome!! I believe that some more inputs such as next bar height, speed of the bars or the frame, etc might help in gaining the maturity stage much sooner. It is just my opinion. Also, more hidden layers may help those creatures store much info (in form of weights oviously) in their individual brains, thus they transfers more genetic info to their offsprings.
Baki game ta mojja chilo...
Thanks! Yes more information will definitely help.
Always left one individual unchanged from th previous generation, in case of all mutations caused worse performance.
Preserving the champion(s) to the next gen helps deal with bad mutations
The collision is elastic: it's his choice😂
lol...should have used "choice" instead of "wish" XD
Hey did you try with elitism option which keeps the best score ball into the next generetion ?
Nice suggestion...I didn't use elitism here.
I thought about that too. Not close to being able to do this but I remember code bullet doing it and it made sure there were no backwards steps in the evolution
Great job! How did you select the number of hidden layers?
Well I knew that the problem I'm trying to solve is pretty simple so I don't need to build a complex architecture. Another thing I had to keep in mind is if I increase the complexity then it would become computation intensive. So I decided to have 1 hidden layer.
@@NormalizedNerd many thanks! Actually I asked the wrong question, sorry: how did you select the number of 'nodes' in the hidden layer? Was it by trial-and-error, or was there any kind of rule of thumb?
@@Antonio-lt1sp Hi, don't know if this will help but usually there is a bit of a rule of thumb. My rule of thumb is it is roughly around the number of inputs + 2 and this usually works. Hope this helps!
Just some remarks:
1. If you do not use mutation, it does not mean that the children will be identical. That's an unlikely scenario given reasonable crossovers and different parents. Mutation is there to add change after the crossover has been applied, since there is a level of "determinism" hidden in the crossover operators, for example convex crossover, which will restrict the children in the convex hull of the parents.
2. No, the next generation will not be worse because of mutation. Just no. When the population of children and parents has been grouped, the survival operator is applied to select the parents for next generation and discard the others. In your case, it seems you are not using the elitist approach, i.e. you are not enforcing the survival of the fittest solution. Mutation never changes the state of an existing solution, it is only used during the phase of creating the solution, with a certain probability.
Other than that, great video.
Can you really call it genetic algorithm when you skip recombination? Aren't we more in the territory of evolution strategy, then?
That's a debatable thing!
This is super cool. At the beginning when all the balls roll right into the rectangle, how do balls which bounce higher but land on the same rectangle have a higher fitness? If the fitness function is just horizontal distance
Is it Reinforcement learning?
Its showing score so it clicked my mind..
I appreciate the thinking process. It is true that the aim of RL and GA is the same (to maximize reward/fitness score). But the techniques are completely different. For example, in RL we have no concept of population, generations, selection etc. And one more thing, GA is largely based on heuristics where as RL involves strong mathematical ground.
@@NormalizedNerd thanks for the rply, it's been a good help...
May I connect to you on any social platform?
@ANKIT RAJ you can follow me on twitter @normalized_nerd
Some would classify GA and evolution strategies(ES) as _generational_ RL, and PPO, A3C as _ontological_ RL.
The best thing is that the two can be combined in interesting ways :)
Nice video. What is the parent selection method used in this? Picking a random parent based on probabilities which in turn depend on fitness? Also in a particular generation, do you include the best agent so far (which might not have come from the previous generation, but from, say, 10 generations before)?
@Sankarsan Bhattacheryya Oh a lot of questions! (and I like this)
1. Yes, parents are picked based on their fitness score. You can think like this...First we are calculating the probabilities based on the fitness score then according to the probabilities the parents are chosen.
2. I'm always picking the parents only from the current generation. Remember when we are selecting a parent we are changing its weights before creating its offspring. We are expecting that next generation will always contain better agents so we are keeping track of earlier generations.
So I guess there is no backpropagation happening. You base the weights of the next generation NN based on the weights of the best performing individuals from your previous population using GA. Did I understand it right? Awesome work!
You have mentioned a great point! Yes, no back prop (gradient based learning) is happening here. The weights of the best performing NNs are transferred to the next generation. While transferring, we change weights slightly (aka mutate) in the hope that modified weights will work even better.
You might be able to use crossover if you use node mutation or enforced subpopulations(ESP)
Will try them in future...
@@NormalizedNerd amazing! Those algorithms need more recognition
Well-done & concise explanations on your videos' topics btw
Your fitness function is not good imo, from my understanding you are basically scoring with proportion to time
Hello, great video, can you share the code that is behind this plz ?
I was wondering why it didn't learn how to play in like 5 generations until 6:41 - only one hidden layer :(
Haha...that too with only 8 neurons!
@@NormalizedNerd were you normalizing the inputs? Which activation function was used?
if that is how human evolution work, we just saw Nikola Tesla
9:26 Elon musk spotted
This video made my day..🤩
Glad to hear that!
You should keep the originals of the previous generation also
Your game seems a lot like flappy bird.
Ok, so why is this better than back propagation? If you used back propagation every "generation" would be correcting itself towards the correct answer rather than just moving the weights randomly.
This is not a genetic algorithm. If you are not using a crossover operator, its not a genetic algorithm, point. Too many mistakes about GA to be honest. This is basically a population based algorithm with just some stochastic movement.
this is a very good explanation of the genetic algorithm.
Thanks!!
the great thing about ai is that it moves at simple rules but it leads to a complicated result
Yes!
What does the bar at the bottom center do?
I added that to control the speed of simulation. It's useful for debugging purpose.
I tried it on trading. My subjects are still didnt understand they have to sell to earn money. How can you solve it on stochastic environment without gradiant-free and no reward system ? They are still like buy,buy,buy,buy,buy ...
I guess the problem is with fitness function.
@@NormalizedNerd fitness is basically profit from end of the trading session
@@rshsrhserhserh1268 Ok that should work. Btw when you agents are supposed to sell? I mean did you use a sigmoid output (buy if closer to 10, sell if closer to 0)?
@@NormalizedNerd after predicting values (buy,sell,hold) like (0,75, 0,30, 0.15) using argmax function so it's selecting buy action (0.75). After buying with that condition next prediction is so similiar to the old one like (0.74, 0.29, 0.14). So it's predicting next best action is "buy" again even though the input state is refreshed with new stock price and portfolio status (after buying it's 1000 etc.). I used every one of the activation functions they all are the same.
@@rshsrhserhserh1268 If you are using the softmax at the last layer then the probability values should add upto 1. In your example it's not.
Best RUclipsr...
❤❤
This is freaking me out
Can we add the judgmental ability to the balls to determine how high the wall is. It seems it just jumps on prediction not from the information coming from the surroundings
You can see that I'm taking wall height and location as an input to the neural network so it must be taking making the decision based on its surroundings too.
Nice energy 👌♥️🖤
Thanks :D
nice video, loved the game and explanation.
Can there be a video on how to make the game and write the AI as well as how to write the AI if you don't have access to the source code of the game
Making a game and writing the AI may be covered in future videos. If we don't have access to the source code of the game then we can take screenshots continuously and use CNNs to extract features out of it.
but where is code?
Please see the video description. There's a link to my github.
Wow this was awesome 😍😍
Thanks!! Keep supporting.
Always !!!!