Learning to Plan Optimistically: Uncertainty-Guided Deep Exploration via Latent Model Ensembles

Add code
Oct 27, 2020
Tim Seyde, Wilko Schwarting, Sertac Karaman, Daniela Rus

Figure 1 for Learning to Plan Optimistically: Uncertainty-Guided Deep Exploration via Latent Model Ensembles
Figure 2 for Learning to Plan Optimistically: Uncertainty-Guided Deep Exploration via Latent Model Ensembles
Figure 3 for Learning to Plan Optimistically: Uncertainty-Guided Deep Exploration via Latent Model Ensembles
Figure 4 for Learning to Plan Optimistically: Uncertainty-Guided Deep Exploration via Latent Model Ensembles

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon

Learning complex behaviors through interaction requires coordinated long-term planning. Random exploration and novelty search lack task-centric guidance and waste effort on non-informative interactions. Instead, decision making should target samples with the potential to optimize performance far into the future, while only reducing uncertainty where conducive to this objective. This paper presents latent optimistic value exploration (LOVE), a strategy that enables deep exploration through optimism in the face of uncertain long-term rewards. We combine finite horizon rollouts from a latent model with value function estimates to predict infinite horizon returns and recover associated uncertainty through ensembling. Policy training then proceeds on an upper confidence bound (UCB) objective to identify and select the interactions most promising to improve long-term performance. We apply LOVE to visual control tasks in continuous state-action spaces and demonstrate improved sample complexity on a selection of benchmarking tasks.

View paper onarxiv icon

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon