Hybrid Control for Reinforcement Learning---the Half-Cheetah Benchmark

Our IJRR paper (with Allison Pinosky, Ian Abraham, Alex Broad, and Brenna Argall) on hybrid control for reinforcement learning is out (available with open access).
journals.sagep...
This video is a comparison between techniques, using the half-cheetah benchmark. Instead of using RL as only a data-driven approach for control, we use optimal control to improve RL by optimally switching between model-based and model-free approaches, deriving an analytical formula for control that depends on both.
We use soft-actor-critic (SAC) as our model-free approach and NN-MPPI as the model-based approach. In comparisons, both do well on simple problems (e.g., swing-up problems) but perform worse on the half-cheetah and hopper problems. We trained with small networks (2-layers) and training steps (50k), but model-based and model-free methods can successfully learn these problems with larger networks and many more training steps.
Code is available at github.com/Mur....