In the paper, you claim that your approach learns from scratch to navigate using monocular images solely in the "real-world". So, did you train in a simulation and then evaluate the trained model in the real world or train it directly in the real world (it may be a real collision happen to the car to get more experience) ?
what about use many linear neural net to works as seprarte networks but are targeted the same travel path by let each network read the same input data but compete for their output by forwarding into a output network that will select the optimum path ahed of time before the actual try? would not this speed up the overall learning performace?
Hi, It was a great stuff. Anyway, a question about how you trained the agent. Is there any terminal state? For example when the agent hit the wall or flipped and how about the reward? Thank you
Good question. Yes, any type of collision is a terminal state. After a collision, the car performs a hard-coded backup procedure and then continues learning. The hard-coded backup is not necessarily needed, and we are going to remove it soon. When evaluating the (N-step) Q-learning prior methods, the reward was the speed of the car, or 0 if a collision occurred. We tried adding a negative reward for collisions, but this actually hurt the performance of Q-learning.
Thank you for the reply. Can I contact you directly in case I have any further question? I am doing a Deep RL for robotic too, but a lot simple than yours.
The simulator in this release uses Bullet (for physics simulation) and Panda3d for graphics rendering. However, adding an interface to a new environment should hopefully be straightforward.
Fake! You can clearly hear the mouse you've trained to push it along! Nice work :)
dont know if you are sarcastic. Im guessing a mouse wouldn't push this fast.
@greg kahn i will try to implement this project, any sort of help will be appreciated
I enjoyed this.
In the paper, you claim that your approach learns from scratch to navigate using monocular images solely in the "real-world". So, did you train in a simulation and then evaluate the trained model in the real world or train it directly in the real world (it may be a real collision happen to the car to get more experience) ?
what about use many linear neural net to works as seprarte networks but are targeted the same travel path by let each network read the same input data but compete for their output by forwarding into a output network that will select the optimum path ahed of time before the actual try? would not this speed up the overall learning performace?
Hi, It was a great stuff. Anyway, a question about how you trained the agent.
Is there any terminal state? For example when the agent hit the wall or flipped and how about the reward?
Thank you
Good question. Yes, any type of collision is a terminal state. After a collision, the car performs a hard-coded backup procedure and then continues learning. The hard-coded backup is not necessarily needed, and we are going to remove it soon.
When evaluating the (N-step) Q-learning prior methods, the reward was the speed of the car, or 0 if a collision occurred. We tried adding a negative reward for collisions, but this actually hurt the performance of Q-learning.
Thank you for the reply. Can I contact you directly in case I have any further question? I am doing a Deep RL for robotic too, but a lot simple than yours.
this is bloody awesome!
good job
Can it simulate in gym-gazebo?
The simulator in this release uses Bullet (for physics simulation) and Panda3d for graphics rendering. However, adding an interface to a new environment should hopefully be straightforward.
Do you have a paper published ? Thanks!
arxiv.org/abs/1709.10489
so uc berkley isnt all bad😂 jk but this is siiick
what about the code bro?