Training a Neural Network to operate drones using Genetic Algorithm
HTML-код
- Опубликовано: 1 дек 2020
- After my first try with flappy I wanted to see how would a genetic algorithm handle more complex situations.
Github github.com/johnBuffer/AutoDrone
Music used
freepd.com/music/Limit%2070.mp3
freepd.com/music/Rulers%20of%...
freepd.com/music/Lurking%20Sl... 
Наука
Good idea, and I like your smoke!
Thanks! I think smoke is where I spent the most time :D
@@PezzzasWork Why do we get hung up on those small sidequests?
@@mendelovitch it’s an easy way to procrastinate the main problem
How or where you stimulate this in unity or special software
I love how they seem to move so organically even though it seems like a relatively simple model. I bet there's some really interesting optimisation problems and extra restrictions you could throw at this.
Also thanks for uploading the demo and source code, very fun to play around with!
I think that the need to center themselves perfectly with the sphere is what makes them not become speed machines. Because when they reach the target they always gotta somehow "dock". And that requires their inertia to be 0 when they reach that point so they have to slow down. If somehow this was changed by making the drones to just need to touch the point at any part and maybe making the orb bigger I would certainly expect that there would be more speedy manoeuvres to just arrive at the target and pass through it. Perhaps even in an elliptical patrolling. Would be certainly interesting to see.
im currently working on the same thing but with more inputs;
I will try ours too;
@@00swinter21 Don't forget to post the result on your RUclips channel !
Exactly my thoughts. It looks like the target requires pixel perfect precision to count as a success. Careful approach is the only way when the targeting criteria are so unnecessarily strict.
@@Wock__ I believe you are right. On one of their videos, there is an actual clock face that counts down on top of the target, like a circular loading bar.
The goal is to dock, not to touch the target. Changing the goals to achieve a better outcome does not mean that your model improved. Making them just have to touch the target so they could go really fast does not mean that they are suddenly better. Your thinking is flawed.
5:28, gen 900: Ok, you guys are too good and I'm tired now. Bye!!!
true
"I have to go now, my planet needs me"
I'd love to see a game where your enemies are all neural network trained AI, and the higher the difficulty, the more trained AI variant you will have to face
Give it 10 years
imagine if the AI is being trained while you play. The better you play the less hard the ai is, but if you slow down the difficulty increases
@@ChunkyWaterisReal it's already possible now lol
@@marfitrblx AI has been shit since the 64 hush yourself.
Or even the player being an AI - I can totally see a 2D game with your cursor being the target point, and the more you play/the more enemies you defeat/etc. the smarter your character gets
Would be really interesting to add fuel consumption to the mix and watch them optimize their fuel economy
and give them more fuel for every target they reach as more reward for doing that
Very cool stuff, well done!
Now make them go through an obstacle course 😁
I am working on it ;)
a combination of the ants finding the optimal path and then the drones following that? :)
@@PezzzasWork where's the video 🗿
This channel is a gem
Great work. I think many people would appreciate seeing background of the work.
love this channel. what separates this guy from others is his consistent ability to make his sims look cool.
This is one of the coolest projects I've ever seen. Would be awesome to extend to add walls and an environment! Great work.
Props to Gen 300 and 400 for beings underdogs and yet surviving for so long
This is one of the coolest implementations i've seen. Nj!
The end of play lineup was a cute touch. Nice work!
Dude I love it when they get sooo roofless! So fun to watch!
You are amazing. Thank you for sharing your fascinating work.
Very nice! Please make more such content, with neural network and drones! :)
Thanks for the video! It's really inspiring.
Impressive Stuff! Had my hands on GAs too for my Bachelor Thesis but with a 6 DOF 3D acting robotic arm. Kinda addicting when you dive deep down in ML :)!
OMG This is so cool, your video actually change my attitude toward neural network from hate to love.
very nice! I'd love to see the same tests, but with added random disturbances like wind gusts from the side, to see how well they can adapt to that!
I wrote my autopilot cargo drone for space engineers and still i am impressed by the work
Very nice result!
5:56 The drone in the left down corner synchronized with the beat in the music. Perfection.
Pls make more vids like this I love them
That end result with the live-tracking is so good! I wonder how viable it is to train simple neural networks like this for game enemy AI
Depends on the game, but on games with a clear goal, it is fairly trivial and will quickly surpass humans.
@@originalbillyspeed1 i guess for different difficulty levels game designer can use agents (enemies) from different generations, for example "easy" = generation 400, medium = generation 500, hard=generation 1000.
@@AB-bp9fi I don't think that would work for most applications. When you want to make enemy AI easier or harder, you always have to think of it in relation to the player - for instance, in a stealth game, harder AI could mean it detects you faster - which pushes the player to improve and be more careful. That won't happen if you just made the enemies drunk (which is basically what would happen if you pick bad neural networks) - it just adds randomness which can be annoying to deal with. Maybe it could work better in things like racing games though.
I'm now imagining a game cloud coordinating through the internet. The AI uses background CPU while the game is running to simulate and evolve against itself, spits its best results against the player to see how they fare, and takes those results as more data to go back to the cloud with to keep working. The bots will start laughably bad at first, but they'll learn how players act, and make players devise new tactics... You might even get good teammate and wingman AI out of it if you put those AIs on the player's side.
@@commenturthegreat2915 What about training AI to match the certain level of intelligence? Like if AI detects a player too fast, then it failed the test.
I like these projects !
Which parameters give the drone positive or negative feedback?
Is flying time a positive or a negative parameter? An acceleration to the target?
Getting some strong Factorio vibes at 4:57
1:58 that faint Vader "noooooo" put me on the floor for some reason
Hello,
very interesting work !
Did you think about testing scenarios with obstacles ?
It would be also interesting to compare the last trajectories and controls with optimal control algorithms solutions.
Cheers.
Totally amazing!!!
That was really cool.
Love that video
So good! :o very impressive ✨✨✨
Great video! How long have you been training them? Greetings from Uruguay!
Wonderful!
7:25 the music moves to your left and right ear as the drone in the top right moves it's power to it's left and right thruster.
Wow that's amazing and looks amazing, how did you cross the two neural networks?
Nice work!
What mutation/crossover did you use?
Really cool project!!!
I was wondering what fitness function you used?
somewhat smaller models and policy gradient following might have increased convergence speed. MLPs are differentiable, so you could just backpropagate through them, sampling distance to the target at every frame and accumulating rewards over the trajectory for an unbiased estimate of a policy’s optimality. you could even use a decay term to incentivize the robots to move faster by downweighting rewards acquired later in the trajectory: distance to the target is ideally the same in the end, but according to the gradient of this reward function, faster would be better.
the only thing left would be running the simulations in parallel or faster than real-time by simply not fully rendering the state of the environment at every training step
What about creating new variables? Like saving fuel or energy consumption, or giving priorities like speed over energy/fuel consumption
It would be great to have a remake of this one
I am actually working on a follow up :)
@@PezzzasWork noice! I will certainly watch it
The target tracking would be cool for a background
Amazing 😮😮😮
I like how it learned to turn off its thrusters to arrest upward motion and to speed up descent.
It's really nice 👍
400 was such a trooper
great Awesome!!👌👌😀. where did you learn to do this?
im more impressed by the smoke, great project though!
7:24 Loved how the Gen-400's legs synced with the music...
Btw, How do we decide the size of the hidden layers? Is there some rule or formula for the best size approximation?
xDDD the "ok..." almost kills me
The memes are fun on this vid
Amazing
Very cool
Other than giving us almost 20 seconds to read 6 words at 4:39 this was very enjoyable to watch :p
Fantastic. Did you published it? Why did you choose GA for learning instead of traditional NN methods, e.g. stochastic gradient descent?
Achei muito interessante o seu canal, obrigado mano
I tryed the mouse controlled vesion what you uploaded on github. And i saw that it's easy to confuse the A.I. in that way to lose controll and fall off the map. I think if you crate a small Trainer A.I. for the target control what best interest to confuse the drone and make it fall off the map, it can train the drone to not fall off no matter how the target moves.
Yes I did a more robust version that I can upload as well
gen 400 is like that one kid in your class that cant stand still when waiting in a queue
That drone that got yeeted at 5:30 had me dieing 😂
its really beautiful.... can you please suggest how do I learn all this. What I learn in what seuqence ??
You should make a game out of this, it looks very funny!!
It would be interesting to see which learning algorithm would produce better results, this genetic algorithm or back propagation or something similar.
you can't achieve that comparison unless you use super humains to play that game and train the models... since it's not the case, the genetic algorithm will always reach far more better results at some point, it's about time.
Wowwww I'm amazed
With what enforcement do you achieved it (what is input/output), which rewards?
This is cool.
how do you tune the weight and bias using GA,? do you intercept the backward process with GA?
Adding a fuel allowance would probably add a more varied result, possibly get those burn hard drones quicker. Also maybe increase your destination bubble a fraction ? This increase the prize rate and hopefully the drones would tighten up the homecoming naturally like the ants do for food routes
I have been working in ML for quite some time now but haven't learned anything about GA and simulation yet .Can you please point to any resource or tutorial that can help me learn to build something like this?
Hi Pezzza, I really liked the video and the way you trained it. Can you tell me how can I learn to code to train a model like this ?? I really want to learn how to do this level of coding. pls reply
ask chat gpt.
it knows a lot about it
i used unity for the physics and did my recreation there and it was even better then the original
Beginning of the video: LOL!! those squeaks as they fall are really funny
End of the video: let's run to buy some food cans before they come for me!!!
good job :)
I'm kind of upset that you didn't publish the thing at the end on itch. Its so satisfying to see the drone follow your mouse and I want to play around with it. Great video!
You can download the control demo here github.com/johnBuffer/AutoDrone/releases/tag/v1
@@PezzzasWork thank you! :)
Hi! i am new, in 3rd year of undergraduation. can u guide me how have you got this far? and what tools and softwares are u using?
I would love for you to make an eco system like the bibites using those drones
man this is so cool. a bit off topic but how are you rendering the thruster particles and smoke?
The smoke is just made out of static sprites and the thruster particles are baked into the flame's texture
This video felt like it's 30 minutes because I somehow kept falling asleep every ten seconds or so.
And it's not boring and no I am not high, idk I guess I just got tired or something
Can u make a tutorial how to choose the best inputs depend on sample ?
This would be a great premise for a game where the character tracks the mouse so instead of controlling the character you're directing it and it gets better as you play through AI learning
I love this! I'm gonna implement it right now in Python. What genetic algorithm were you using? I'm planning on using Neat
how did you get this environment in Python? I want to test policy gradient RL algorithms
@@CE-ov7of not sure if you still need this question answering however i'll give it my shot. My guess is hes implementing the basic algorithm of the envirment in python using pygame and and numpy. Then for the AI my second guess is he'll be using NEAT Python library or custom AI/NN algorithm for the agent and training. That's my guess however if you want any question just reply and i'll do my best to help. Python isn't my strongest language however but i'll try my best.
Hey @@j_owatson , unfortunately this is not something I have time/interest for anymore.
But I really appreciate your willingness to help! This is what makes the software/tech community great!
i have no idea about how you did it ..but it seems like something fun to learn
Machine learning is extremely fun and addictive :)
@@PezzzasWork can confirm
How are you training the NN and what does your genome represent?
oooh idea. Space Invaders: Drones Addition. Different levels use different generations of drones as enemies.
if you had an body orientation/angle input they would have been able to recover from a spin out or even fly upsidedown
Is there a windows version for this dwonload? or am i being dumb lmaooo. but anyway, this is *E P I C*, i loveee the design and everythingggg ahhh ahaha, keep this up :) great work :D
Wooow drone is very cool
Am I right, that the final code controlling the drone is really slim and efficient code? It's just some products and sums, and an activation curve, right? Seems pretty amazing for how organic the movement appears
yes exactly. thats the good part about this.
thats epic
How i can use this or download it? Because is EPIC
Imagine spending hours and hours trying to get to something and then when you finally get there you just have to go to another one
give a consolation prize to generation 300!
It deserves it all
Have you ever tried using a neural network on a hardware platform?
it's cool to see your using dropout, so it learns better
where do i star to do games and machine learning like this?
Why don't you upload a Gen 5500 drone to github? I tryed it but i only founded the Gen 3100 one. I really want to try that one. :D
Would it be possible to have the drones compete? For example, by simulating the entire population of drones at once, and only rewarding the first drone to reach a target.
That's exactly what genetic algorithms (GA) means
After a few tweaks, I have a feeling this could have real-world use.
1:05: this one looks like Los Angeles Battle drones
Would be interesting to have a drone sumo where they can collide and try to shove each other out of a ring.
What activation functions did you use for the hidden and output layers of the neural network?
Looking at his code I think he used sigmoid