Combining Learning-Based Locomotion with Model-Based Manipulation for Legged Mobile Manipulators

Deep reinforcement learning produces robust locomotion policies for legged robots over challenging terrains. To date, few studies have leveraged model-based methods to combine these locomotion skills with the precise control of manipulators. Here, we incorporate external dynamics plans into learning-based locomotion policies for mobile manipulation. We train the base policy by applying a random wrench sequence on the robot base in simulation and add the noisified wrench sequence prediction to the policy observations. The policy then learns to counteract the partially-known future disturbance. The random wrench sequences are replaced with the wrench prediction generated with the dynamics plans from model predictive control to enable deployment. We show zero-shot adaptation for manipulators unseen during training. On the hardware, we demonstrate stable locomotion of legged robots with the prediction of the external wrench.
Link to the publication: ieeexplore.ieee.org/abstract/...

[@stevenhong7099]

awesome work!

[@claudiagiacomazzi5204]

very interesting work! many compliments :)

[@RonLWilson]

Looks very promising.

[@manastiwari693]

In India, we don't have this kind of research going on. Seeing what Dr Hutter has been achieving in robotics, inspires me to study at ETH, under his guidance and bring some positive impact in my country.

[@user-ds9yp8ef8g]

😀

[@DgurlSunshine]

TARDED