Model-Based Transfer Learning for Contextual Reinforcement Learning

This research introduces Model-Based Transfer Learning (MBTL), a novel framework for improving the efficiency and robustness of deep reinforcement learning (RL) in contextual Markov Decision Processes (CMDPs). MBTL strategically selects training tasks to maximize generalization performance across a range of tasks by modeling both the performance set point using Gaussian processes and the generalization gap as a function of contextual similarity. The method uses Bayesian optimization to guide task selection, achieving theoretically sublinear regret and experimentally demonstrating up to a 50x improvement in sample efficiency compared to traditional training methods. The effectiveness of MBTL is validated across various continuous control and urban traffic benchmarks. Further analysis shows the method's insensitivity to the underlying RL algorithm and hyperparameters.
arxiv.org/pdf/...