AI Learns to Park - Deep Reinforcement Learning
HTML-код
- Опубликовано: 28 авг 2024
- An AI learns to park a car in a parking lot in a 3D physics simulation. The simulation was implemented using Unity's ML-Agents framework (unity3d.com/ma.... The AI consists of a deep Neural Network with 3 hidden layers of 128 neurons each. It is trained with the Proximal Policy Optimization (PPO) algorithm, which is a Reinforcement Learning approach.
Basically, the input of the Neural Network are the readings of eight depth sensors, the car's current speed and position, as well as its relative position to the target. The outputs of the Neural Network are interpreted as engine force, braking force and turning force. These outputs can be seen at the top right corner of the zoomed out camera shots.
The AI starts off with random behaviour, i.e. the Neural Network is initialized with random weights. It then gradually learns to solve the task by reacting to environment feedback accordingly. The environment tells the AI whether it is doing good or bad with positive or negative reward signals.
In this project, the AI is rewarded with small positive signals for getting closer to the parking spot, which is outlined in red, and gets a larger reward when it actually reaches the parking spot and stops there. The final reward for reaching the parking spot is dependent on how parallel the car stops in relation to the actual parking position. If the car stops in a 90° angle to the actual parking direction for instance, the AI will only be rewarded a very small amount, relative to the amount it would get for stopping completely parallel to the actual direction.
The AI is penalized with a negative reward signal, when it either drives further away from the parking spot or if it crashes into any obstacles.
The training process shown in this video took about 23 hours on a computer with an i5 (7th or 8th gen) and a GTX 1070 with 100x simulation speed.
Subscribe for more content like this:
/ @samuelarzt
Follow me on Twitter for more frequent updates on my projects:
/ samuelarzt
Also check out my other videos related to this Project:
Two AI fight for the same Parking Spot:
• Two AI Fight for the s...
Neural Networks Explained in a Minute:
• Explained In A Minute:...
Cars learn to maneuver Parcour with Genetic Algorithm:
• Deep Learning Cars
Start Music: "Sunday" by Otis McDonald
Music from Bensound.com:
Timelapse Music: "The Elevator Bossa Nova"
Comedic Background: "Jazz Comedy"
Outro: "All That"
#ArtificialIntelligence #MachineLearning #ReinforcementLearning #AI #NeuralNetworks
Watch two AI Agents fight for the same parking spot:
ruclips.net/video/CqYKhbyHFtA/видео.html
Can it park in other place after learning on this place?
@@TechMasterRus It is very unlikely to be able to generalize to other parking spots. In order to do so, one would have to at least also randomize the parking spot position during training.
@@SamuelArzt what data do you use to train AI? Global coords relative to some static point? If you use all objects in car's coodinate system, then mb it won't matter where to park... if it will be able to train like that at all.
@@TechMasterRus The inputs of the neural network are mentioned in the video description:
"the readings of eight depth sensors, the cars current speed and position, as well as its relative position to the target"
I think having the relative position of all obstacles would be somewhat cheaty. Don't get me wrong, this is far from being realistic, but I think that would be too far of a stretch.
If you want to achieve an AI to park on any spot in the parking lot, it would be more feasible to simply train it on randomized parking spot positions. It will be better at generalizing to other parking spots then, even though that is likely to increase training time.
Hey, if your interested I’d love if I could email you some of my music to use as background music
This is the definition of "don't practice until you get it right, practice until you can't get it wrong"
That’s a really nice quote
. . .
I rememebr a similar quote!
"Hit or miss huh,- ok ok i won't.
Omg it’s the “dEfiNitiOn”?
@@tenchi2075 it's Daniel Willingham, it's from his theory of learning
@@DarrinDarwinacious mean
"What is my purpose?"
"You park cars."
"Oh god...."
Michael Ricks-Aherne yeah welcome to the club pal
Is this a rick and morty reference
Its referencing alot of things I guess, my first thought was the Ricky Gervais joke about dogs... Then rick and Morty... Could probably think of others
That’s a valet
@@fractal_mind562 I thought of red dwarf and the toaster.
It looks like it's having to fight the voices in its head telling it to run full speed into those other cars
me when im infront of a tesla at a traffic jam
I didn't think I'd find you here my majesty. Nevermind, your supreme excellence graces this comment section.
This kinda makes me appreciates the NPC's driving in GTA V.
They are skilled, but remember that is an enclosed and well defined system. Still impressive, however.
In GTA they are not AI, they are killing machines
@@markmd9 in gta online they are crashers machines hahah
Yeah, in GTA they're just programmed to
You can't compare the two because gta v bots are not AI they are programmed
Actual footage of me trying to park after 2 years of driving
I too park in the same spot at the store
😂😂
*shoves truck out of the way so I can take 2 spots*
Just back it up dude, it's much simpler than trying to park front first
“AI will take over humanity”
300k iterations later, the car still gets a parking ticket.
We will not require tickets in our robot future. I will pass this on to our human so he is aware of this issue. Thank you.
well considering it takes a human 18 years to do it...
@@alpacamale2909 18 years? Lmao HAHAHAHAHAHAHHA
@@alpacamale2909 16 but depends on the country
such an ignorant comment
Came for the car going around randomly, now I can go to bed happy
2:48 "Leonard that's my spot"
I like the strategy at 2:45, sometimes I wish I could do that when I park...
I was pretty curious about this attempt. I wonder if the AI decided that the negative reward for crashing into the vehicle was outweighed by the positive reward for parking. Then curious how the +/- rewards are defined, and if the AI continues to receive negative reward for the duration of the collision or if it is only penalized when it first collides with the vehicle.
Same strategy is applied at 5:01
Mr bean style hahaha
lmao
@@ChrysusTV I guess the AI would continuously receive negative rewards when in contact with other cars.
Because from the attempt on 5:01 we can see it was basically the same strategy, but the AI learnt to kick the car away with repeated, high speed strikes instead of just pushing it.
If (going to crash)
Then (don't)
boom, programmed
wow youre very smart im sure youve coded many programs
The problem here is how to make it know that it would crash.
@@EmrecanOksum
If (rangesensorDistance >= 30cm)
{
println "Bruh, stahp the car!!11"
Motor = 0
DriveAround = 0
}
else
{
println "Wtf am i doin?"
Motor = 1
DriveAround = 1
}:
EMR3 C4N 0K5UM You do realize that it’s just a joke right....we all known that it isn’t that simple
Thanoose im pretty sure he was joking aswell
I wonder how much the learning process would be affected by feeding in not just the current sensor input, but the last couple of seconds of sensor inout (so 2 seconds * 60 FPS * 8 sensors additional neural inputs), and maybe the last 2 seconds of the AI’s own output decisions as well. The network would be much bigger and training would take much longer, but I wonder if past input and decisions would feed forward in time to produce smoother results, preventing the AI from just going for a bad solution as often.
Indeed a great idea
4:17 okay, now its just showing off.
*Pushes through car to get to parking spot"
"Boom! nailed it"
"It has 8 depth sensors"
*breaks the pole*
Is the sensor learning too??
Well in deep learning. You have reward and punishment. The ai over time realises hey hitting stuff is bad.
But I understand ur joke lol
@@Azaelris yes, i know the AI is just a baby using the sensors as its eyes and learning not to collide, by colliding
@@priyankraval7461 But guys , what is parking for AI? I mean once the "car" Is in the right Place how does It understand It Is ? There Is some kind of condition like (if Car Is in a middle of White lines , It Is parked ?) And It tries all the random algoritms till It happen?
nah the AI is just very kinky, get that punishment you dity AI ;)
@Ceol Hmm interesting , thanks bro
Since the initial position of the car for every learning try was random, I imagine that the resulting model with 300k or so learning tries should be able to park on the remaining available parking slots. Checking if so, and if not why not would be great tbh.
beautiful work, I feel more than lucky to be able to see such nice results presented in such an interesting manner!
Feels like watching a kid growing up x1000 faster
Succeeding, failing, still moving no matter what
This is so cute 🥰
If there's an artificial intelligence apocalypse, we'll know that this guy left his computer on for too long.
Let's hope it at least remembers anything about us
oh the horror!, perfectly parallel parked cars!, parallel parked cars EVERYWHERE!!!
@@svampebob007 Literally the only apocalypse that will happen
🤣🤣🤣🤣🤣
well, to our disappointment this one could only park all cars eventually
“After 5k attempts”
*oh shit he’s getting close*
“After 15k attempts”
*alright how long is this video now?*
Did you see when he pushed the gray car outta it's place and just pushed straight ahead
standard 10 mins
Yeah i thought the same and thought at 20k it would understand since it had felt the great feeling of parking, but I was very wrong xD
@@mayatechnologyinspired6679 The AI just thinks it's Mister Bean.
More penalties should of been used, ie, driving on the curb, and instant failure on crashing.
Also a segmented training might of helped, with the car spawning closer to the space in the beginning, maybe even changing the spot to park after a success
I love it, when ai parked perfect to the wrong parking spot.
"AI is penalized for crashing"
10:07 spawns AI on top of car
AI: One day you will pay, humans...
Exactly my thoughts 😂
You forgot a comma there, buddy
One day AI will have its own rights. It will control internet based business and offer you VPN, hosting, web design, programming, hacking... And have its own cryptocurrency wallets to trade with and buy what it likes.
Yup I sure will pay humans in the future
"why are we still here, just to suffer?"
Even my mother can do it in just 15k attempts!
yes
ikr
Haha
lmao
wtf, is she blind ?
This is pretty much a real life depiction of my data analyst job experience
This makes me feel a lot better about when I learned to parallel park.
Multiple issues i spot: Sometimes the raycasts see nothing (it is plowing a car, yet raycast is green and max range), Raycasts are raycasts, i suggest boxcasts to be more realistic at least for the front side. The parking spot never moves, nor does the other cars/environment, so this does not generalize well at all either (if it learned this spot, a different spot will likely not work at all). Hitting an obstacle or invalid driving location (sidewalk) should instantly end the simulation of that episode to iterate faster on problematic attempts.
Niceeee. Where did you learn reinforcement learning!?
@@okayokay1979 Self study, why?
Haha, you are the first commentor to notice that raycast bug. I didn't bother to fix it in the end, since the AI is not supposed to reach that edge case (of constantly touching an obstacle) anyway and gets a large penalty for doing so.
Boxcasts / Spherecasts / Colliders were already suggested numerous times by other people. The same goes for randomizing car and parking spot position.
I also did think of these things, they are of course very obvious, but decided to stick with this setup since I didn't want to increase training time any further as it was already very close to the limit I was comfortable with.
@@SamuelArzt Fair, i did not bother to read all other comments, those were just my thoughts on it. Well done otherwise, nice experiment. For entertainment i would give the other cars a car alarm when hit.
As for the training time, i did suggest ways to shorten it. Often you can cut off any simulation the moment it becomes "unrecoverable" like in this case the car hitting any obstacle would be a definite fine on the drivers end. Also, if you take a bit more reading into account, you can actually make machine learning run headless, or even multiple instances side by side, and have the "max physics timestep" setting figure out how fast it can run those instances at any given time. It might be a bit choppy at times, but as long as the max timestep is small enough it should not affect learning, and it will sort of auto self-balance to max load for your device.
For presentation you can make a backup of the .nn file every X iterations yourself, so you can playback a few runs in "learning run 10k, 15k, etc." and for the rest train in headless or otherwise visually heavily downscaled instances for training speed increases.
@@Smileynator Car alarms would be very fun indeed ^^
As far as the headless, sped up training and saving of the model is concerned, that is exactly what I did (and with exactly, I mean impressively almost scaringly accurate).
Still better than me
Maly ten jutub
to be fair you probably didn't practice for 200,000 times strait
That's what she said.
In the future, probably better than all
That’s concerning
I love these AI training videos. It’s hilarious contrast seeing how childish and bad at it the computer is, contrasted with knowing how brilliant everything that goes into making this happen is.
Actually it is a bullshit database, not an A.I . as it must be.
No map-building, no fails -avoiding scenario, just a "10k attempt" thing.
Why did you love this? It's not a child, nor human brain. It is just code, it's not improvising, just doing the exact same errors, again and again, it's NOT TRAINING. It's about YOU tryiing to BELIEVE in this, lol!
@@nikostalk5730 you good?
a couple things i noticed:
- occasionally the car will crash into things in front of it because of the one-dimensional lines of sight.
- car appears to prefer to turn left. because of practice perhaps?
very cool stuff!
What the-
how did I never realise that my parking practice was being recorded??
LMFAO
2:49 my favorite attempt. " 'Scuse me, that's my parking space. Move over."
It's kinda like watching a bee under the affect of alcohol trying to fly into it's hive
The little speed boost it does whenever it is nearly there is so cute imo
Now i wanna see multiple ai on trying to park at the same time.
Haha, yeah that's a great idea. I have also thought of that. I will try that and at least post a tweet about it.
If it is worth more than a gif, I will definitely upload a video of it ;)
@@SamuelArzt you did it!! 🤩
@@sirbughunter Haha, yeah! Turned out real fun!
110109999188880 attempts later...
@@SamuelArzt
This video proves that why we shouldn't rely on AI and stick to manual
I’d love to see it have its parking spot selected randomly from the 6 open ones, then it’ll have to learn more generally how to go to any location rather than a specific one
It could be the next step.
I feel like a more realistic set starting point, with one random empty one would provide a better real AI, as normally your car will not just spawn in a parking spot and you need to go to another parking spot in the same lot
And change how many and where the other cars are parked
Everything everyone mentioned here as well as obstacles in the way
I think the place of the parking spot is involved in the calculations through the relative distance with the car and the relative angle of rotation
I'm curious how you recorded this. Was this recorded directly in Unity or did you use an external screen recorder?
I recorded this directly in Unity using a plugin from the asset store. Unfortunately that plugin is no longer supported by its creators and was removed from the store some time ago.
When you compare the beginning to the end it's just awesome!
Deep reinforcement learning sounds more like you hit the AI on the hand with a ruler every time it does something wrong
That's essentially what it is.
Well, the copies of AI that fail go straight to /dev/null so that's really more like "executions will continue until the morale improves"
@@aleksandersuur9475 you'd think it'd keep it to know what not to do, like fire is hot, don't touch.
Agree, seems like bad code
AI: harder daddy
Overfitting? Should've given different parking slots
was thinking the same, overfitted AF, but good exercise doe
Eventually more parking slots, and different configurations for already parked vehicles
yep
Car AI: **Tries to park**
Samuel: "I haven't programmed that path yet"
People: AI will take over the world!!!!
AI after thousands of (essentially) bruteforce attempts: 1:20
Me with 4 overdue assignments at 3 am
*I N T E R E S T I N G*
I love how at 20003 it got the first taste of that sweet parking reward, then it needed a fix so badly it rammed the white pickup out of the way.
At around 15-20K it started looking like one of those Russian security cam videos
The car: "oh yea, you like that baby?",
The spot: "It's not even in yet 🙄".
So you're saying all those 3 AM Walmart parkers are really AI'?
No they're just drunk af
Or high af
Lol
It's so satisfying seeing a naive robot make a clean park with no mistakes. Good model!
Tested it with my 3 year old nephew... he mastered it after 5 attempts... We have a long way to go ;)
AI: Learns to park
User: cool, (proceeds to change parking location)
if it was properly trained that shouldn't be a problem
As this is AI changing parking location should not be problem after some more training. As so much as training in complex parking lot should be also faster for this neural network.
after 20000th iteration:
yeah I got it! I finally got it!
20001st iterations:
rams into truck again
Me normally: doing stuff almost flawlessly
Me whenever others are around: *THIS VIDEO*
2:49 The AI went full Mr. Bean on that parked car!😂
AI typically takes around 1 million try’s to get something and they can try 1 million times in like a few minutes... this is insane
This simulation has a physics engine, your a couple hours off. It takes way longer than minutes.
Maybe one day, but by that time an AI that can use logic and doesn’t require over 100k attempts will already be here.
I only have a Raspberry Pi brain. I wish my algorithms were this complex.
@DeBobbelear 8769 ?
@DeBobbelear 8769 I don’t get how you don’t understand what I was saying
I love how the AI tries to park in all the different spots.
Love how the music switches to the lofi hiphop music when the AI parks (near) flawlessly and switches back when it doesn't lol
I don't know why, but it's really cute to watch this AI car learn. As if watching a toddler learn how to park a car. Almost as if watching a real human being!
The parking lot configuration should’ve been randomly generated each trial
You are totally right, I am wondering how would it affect learning speed, I noticed that based on the evolution the harder the environment you are in is the quicker u learn.
Right. Isn't there an overwhelming chance that it learned to navigate in just this particular configuration (including target location and other cars). Could the DNN have learned the map of the environment as captured by the depth sensors?
Mattias Marder what it looked like it learned was a map of vectors. Basically at pos x,y it learned to travel at angle theta. There’s a non 0 chance that you could get rid of the “sensors” and it would still find the parking spot
I think it's easier to train the agent to park at one spot and do transfer learning on multiple spots.
@@keco185 I tried removing the sensors and it never learned to park in the time of 5M steps (i.e. the length equal to the experiment shown in the video).
Conclusion: AI won't control the world yet
It's learning to control
You must watch this:
ruclips.net/video/Lu56xVlZ40M/видео.html
They are not only gonna control but will break all the limitations which humans have. 😂😂😂
AI basically are learning at a faster pace than the entire human race, evolution that usually takes millions of years only takes a couple days for AI
Worse: it already does to some extent and its ability is on par with the AI driver of this video.
@@intensestare5027 The big difference is that this ai in particular had an imaginary world to destroy, it's not clear if it'd transition to real world so smoothly.
After 310k attempts, now the AI can park perfectly in that one spot for that specific parking lot with that specific arrangements of cars!
my friend: "im not THAT bad at driving"
also her:
2:55
"Bro what the fuck are you doing"
"It's still learning....."
I don't know why, but the first 5 minutes of the AI failing and just crashing everywhere was absolutely hilarious to me
Lol same😄
I like how it almost successfully parked in the wrong spot a few times.
So interesting to see that due to how it's trained, the AI will never attempt to park in reverse, even in a situation where that would be easier
Yeah, since the reward function includes the angle, I guess they didn't account for parking backward, and that would actually be considered the worst angle (180 degrees off from target).
I wonder how well a HUMAN would perform with those 10 inputs!
Yes, I totally agree.
Imagine being given a steering wheel, two pedals and a screen that simply showed 10 floating point numbers, with all information about your task being whether you are currently doing good or bad.
Even then a human would have more prior knowledge of the task than a machine, since we will immediately associate the steering wheel and pedals with driving a car.
@@SamuelArzt this text must be in video description.
@@SamuelArzt have you such simularot for human? I'd like to test my bio neurons.
@@waltersereinig3587 The neural network has 14 inputs, if I remember correctly and one of the is the current speed.
The 3D visualization was indeed mainly for visual / entertainment purposes rather than there being an actual need for 3D environment.
The sensors were deliberately kept to minimal complexity since the application of an AI like this would rather be for a game than real life, thus anything more complex would have a considerable impact on performance.
Where I live, even the first couple simulations were pretty damn accurate.
It would be interesting to see how it would manage with random cars in space placement and a random space to park in. My guess is that in its current form it would really struggle, but I wonder what the difference in number of iterations would be :)
I did something like that in my parallel parking video. Some updates in the learning algorithm also resulted in much faster training time (5h in contrast to 23h).
ruclips.net/video/MlFZjLkEIEw/видео.html
3:22
When you try to back your car into a spot for the first time
Leaked footage from Tesla R&D center
chris imagen
This video demonesrates the fact that learning is nothing more practice + repetition .. This is a very solid fact .
I'd like to see the enviornment change with every attempt. Despite spawning at a random position it might still have overfit. Amazing work regardless, keep it up.
I love how the cars bounce every time the program is initialized.
Day 20003: Dear Diary,
Today I pushed a truck out of its parking lot. It felt good but I got penalised for it. Life isn’t fair.
...
My boss just called. I have to go back learning how to park now.
[[DAY END]]
underrated?
@@gilberttheregular8553 Underrated
Can you write a book? Please?
I saw this comment right when the attempt mentioned in it played
This actually made me feel bad for the poor AI.
Like a baby doing it's first steps.. the dev must be so proud..
What a generous driving instructor. I only got 1 chance.
I found it very interesting to see how AI learns if you give it no information exept for the penalty and the reward. Nothing the AI does is intuitive. For example, a human would have understood very quickly that it is essential do drive a curve. The AI does not see such patterns as „curves“. It only sees the numbers. It only sees if it has gotten closer and if it has made mistakes.
That isn’t a fair perspective….
You or humans in general just see these patterns because you/they have already learned since they are born. Maybe not car parking but relatable things. The conditions are different!
@@valentins.5010 Whats not fair about it? What op said was just factual.
2:40 when someone takes your paid parking spot
"After 10000 attempts..."
_Already drives better than my sister._
6:15 clearly trying to escape the simulation
As an amateur ML programmer, this is inspiring! I may do a project as a result of watching this.
So this is an example of something called “overfitting”. Though the “over” part assumes generalization, which isn’t the goal of this application, but I still think it’s a good talking point.
The AI is learning a lot. But notice that it’s in the same map, with the same layout. So it’s mostly learning how to park in this EXACT situation.
How does the AI learn to turn left? Well it could be something like if the front left depth sensor suddenly cuts in half, and all the other ones are at max length, then it knows to turn left. (I.e. driving past the first light post)
A.I. Can figure out all kinds of tricks like that. However, it doesn’t have predefined cognition (humans do. Your brain is largely pre wired. You could argue that, through your life, your brain really only changes marginally with respect to it’s original construction) the predefined structures in our brain focus on learning generalizable ideas. This is hard to do with AI.
A human could run this simulation in this scenario, and probably, after getting it right, be able to park in another situation. That’s because
a) the sensors we use to perceive the world are super sophisticated, and kinda come pre-installed with the capacity to understand their input
b) we focus on learning things like “stay on the road” not “turn left when front left depth sensor gets short suddenly”. The key difference between generalizable intelligence, and an overfitted system.
If you shifted the starting point 3 feet to the left. It’s entirely possible that the AI would completely fail. However, if you put the car in a box, it would, after training through thousands of iterations, probably find a way to glitch the car so it pops up in the air and lands outside of the box.
Systems like these are bad at generalized stuff, but really REALLY good at solving specific and difficult tasks.
There’s a famous study where researchers wanted to make some NN for controlling walking. It had 6 legs, and they set a rule that it got more points for spending as little time on its feet as possible. After seeing that it learned to go 100ft with 0 seconds of foot contact, they found out that it learned to flip over and walk on its shoulders.
What a time to be alive
Edit:
I didn’t really watch the video. The car changes it’s starting point, so it’s kinda generalized. But the map stays consistent, so it’s not really. If you moved an item around it’d probably freak out. Though idk, I’m mostly talking about AI for you the reader, not about the video.
@@TyrianTheDutiful why?
"A human could run this simulation in this scenario, and probably, after getting it right, be able to park in another situation. That’s because...." hold up, I'm going to show this post to my Ex-GF so she can explain what exactly is wrong with the car today.
edit:Ex
This is just a demo, he's not going to change humanity by making self-parking cars.
@@svampebob007 oof
i like that two minute papers reference
Imagine some AI in the future be like:
“oh look,a video of our ancestor!"
“...how did we get here”
I hope one day i can fully understand stuff like this one and I can actually do it. Excellent video thanks for sharing
I was expecting a drift parking after so much attempts and watching AI Hide & Seel video.
And we learn from AI to NEVER GIVE UP!!!! 😎👌
the reason it took it so long was he was spawning in a different place then before . that's why it was having a hard time understanding its surroundings and actually learn to drive.
2:48 - 3:01
This AI have gained personality and it's a great one.
Wow, this ai learnt to park perfectly in half the amount of attempts a normal person takes :O
I appreciate the choice of music.
After 3 million attempts it learned the urban inefficiency of surface parking lots and switched to a bicycle
"Dw AI, I believe in you, you got this-"
"WAIT- STOP THERES A PERSON-"
"Keep going? Got it"
The biggest flaw seems to be the 8 'depth' sensor having huge blind spots, what if instead of 8 raycasts, you only had a single rotating one (maybe a few degree per frame). Let the AI know about the angle of the sensor relative to the car and the hit distance. That way it would probably be more 'aware' of his surroundings quicker.
You know what the AI got as rewards?
*COOKIES*
That's some bullshit...
But I like it. Here a *COOKIE* !
That netflix show...
It’s literally like putting a 5 year old under the wheel 🤣
I could imagine some random guy in apartment watching this guy trying to park over 300,000 times
it gets more and more satisfying watching the AI be more successful over time at parking
On attempt 310013, it spawned on top of the truck!
No wonder it took so long to learn, even rngesus is messing it up.
Haha, yeah the spawn was simply defined as an area, thus there was a very small chance for that to happen. I thought that was quite fun and left it in the video. Nice spot, though 😉
@@SamuelArzt ai: one day i will get you human
10:08
imagine if you left this until the 1mill iterations, the Ai would be making world record times for each random position
That instructor has a lot of patience
You should also track how well they're following the road as well as the driving direction.