Alert button

On the Practical Consistency of Meta-Reinforcement Learning Algorithms

Add code
Bookmark button
Alert button
Dec 01, 2021
Zheng Xiong, Luisa Zintgraf, Jacob Beck, Risto Vuorio, Shimon Whiteson

Figure 1 for On the Practical Consistency of Meta-Reinforcement Learning Algorithms
Figure 2 for On the Practical Consistency of Meta-Reinforcement Learning Algorithms
Figure 3 for On the Practical Consistency of Meta-Reinforcement Learning Algorithms
Figure 4 for On the Practical Consistency of Meta-Reinforcement Learning Algorithms

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon

Consistency is the theoretical property of a meta learning algorithm that ensures that, under certain assumptions, it can adapt to any task at test time. An open question is whether and how theoretical consistency translates into practice, in comparison to inconsistent algorithms. In this paper, we empirically investigate this question on a set of representative meta-RL algorithms. We find that theoretically consistent algorithms can indeed usually adapt to out-of-distribution (OOD) tasks, while inconsistent ones cannot, although they can still fail in practice for reasons like poor exploration. We further find that theoretically inconsistent algorithms can be made consistent by continuing to update all agent components on the OOD tasks, and adapt as well or better than originally consistent ones. We conclude that theoretical consistency is indeed a desirable property, and inconsistent meta-RL algorithms can easily be made consistent to enjoy the same benefits.

View paper onarxiv iconopen_review iconOpenReview

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon