Alert button

A Geometric Perspective on Optimal Representations for Reinforcement Learning

Add code
Bookmark button
Alert button
Jan 31, 2019
Marc G. Bellemare, Will Dabney, Robert Dadashi, Adrien Ali Taiga, Pablo Samuel Castro, Nicolas Le Roux, Dale Schuurmans, Tor Lattimore, Clare Lyle

Figure 1 for A Geometric Perspective on Optimal Representations for Reinforcement Learning
Figure 2 for A Geometric Perspective on Optimal Representations for Reinforcement Learning
Figure 3 for A Geometric Perspective on Optimal Representations for Reinforcement Learning
Figure 4 for A Geometric Perspective on Optimal Representations for Reinforcement Learning

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon

This paper proposes a new approach to representation learning based on geometric properties of the space of value functions. We study a two-part approximation of the value function: a nonlinear map from states to vectors, or representation, followed by a linear map from vectors to values. Our formulation considers adapting the representation to minimize the (linear) approximation of the value function of all stationary policies for a given environment. We show that this optimization reduces to making accurate predictions regarding a special class of value functions which we call adversarial value functions (AVFs). We argue that these AVFs make excellent auxiliary tasks, and use them to construct a loss which can be efficiently minimized to find a near-optimal representation for reinforcement learning. We highlight characteristics of the method in a series of experiments on the four-room domain.

View paper onarxiv icon

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon