Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees: Paper and Code

Get our free extension to see links to code for papers anywhere online!Free add-on: code for papers everywhere!Free add-on: See code for papers anywhere!

Add to Chrome

Add to Firefox

Add to Edge

Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees

Jul 16, 2018
Guiliang Liu, Oliver Schulte, Wang Zhu, Qingcan Li

Figure 1 for Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees

Figure 2 for Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees

Figure 3 for Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees

Figure 4 for Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees

Share this with someone who'll enjoy it:

Deep Reinforcement Learning (DRL) has achieved impressive success in many applications. A key component of many DRL models is a neural network representing a Q function, to estimate the expected cumulative reward following a state-action pair. The Q function neural network contains a lot of implicit knowledge about the RL problems, but often remains unexamined and uninterpreted. To our knowledge, this work develops the first mimic learning framework for Q functions in DRL. We introduce Linear Model U-trees (LMUTs) to approximate neural network predictions. An LMUT is learned using a novel on-line algorithm that is well-suited for an active play setting, where the mimic learner observes an ongoing interaction between the neural net and the environment. Empirical evaluation shows that an LMUT mimics a Q function substantially better than five baseline methods. The transparent tree structure of an LMUT facilitates understanding the network's learned knowledge by analyzing feature influence, extracting rules, and highlighting the super-pixels in image inputs.

* This paper is accepted by ECML-PKDD 2018

View paper on

Share this with someone who'll enjoy it: