Alert button
Picture for Hyunjae Woo

Hyunjae Woo

Alert button

Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization

Add code
Bookmark button
Alert button
May 25, 2022
Sungryull Sohn, Hyunjae Woo, Jongwook Choi, lyubing qiang, Izzeddin Gur, Aleksandra Faust, Honglak Lee

Figure 1 for Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization
Figure 2 for Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization
Figure 3 for Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization
Figure 4 for Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization

We tackle real-world problems with complex structures beyond the pixel-based game or simulator. We formulate it as a few-shot reinforcement learning problem where a task is characterized by a subtask graph that defines a set of subtasks and their dependencies that are unknown to the agent. Different from the previous meta-rl methods trying to directly infer the unstructured task embedding, our multi-task subtask graph inferencer (MTSGI) first infers the common high-level task structure in terms of the subtask graph from the training tasks, and use it as a prior to improve the task inference in testing. Our experiment results on 2D grid-world and complex web navigation domains show that the proposed method can learn and leverage the common underlying structure of the tasks for faster adaptation to the unseen tasks than various existing algorithms such as meta reinforcement learning, hierarchical reinforcement learning, and other heuristic agents.

* Accepted to UAI 2022 as an oral presentation 
Viaarxiv icon
Github IconAccess Paper or Ask Questions
Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon

Meta Reinforcement Learning with Autonomous Inference of Subtask Dependencies

Add code
Bookmark button
Alert button
Jan 01, 2020
Sungryull Sohn, Hyunjae Woo, Jongwook Choi, Honglak Lee

Figure 1 for Meta Reinforcement Learning with Autonomous Inference of Subtask Dependencies
Figure 2 for Meta Reinforcement Learning with Autonomous Inference of Subtask Dependencies
Figure 3 for Meta Reinforcement Learning with Autonomous Inference of Subtask Dependencies
Figure 4 for Meta Reinforcement Learning with Autonomous Inference of Subtask Dependencies

We propose and address a novel few-shot RL problem, where a task is characterized by a subtask graph which describes a set of subtasks and their dependencies that are unknown to the agent. The agent needs to quickly adapt to the task over few episodes during adaptation phase to maximize the return in the test phase. Instead of directly learning a meta-policy, we develop a Meta-learner with Subtask Graph Inference(MSGI), which infers the latent parameter of the task by interacting with the environment and maximizes the return given the latent parameter. To facilitate learning, we adopt an intrinsic reward inspired by upper confidence bound (UCB) that encourages efficient exploration. Our experiment results on two grid-world domains and StarCraft II environments show that the proposed method is able to accurately infer the latent task parameter, and to adapt more efficiently than existing meta RL and hierarchical RL methods.

* In ICLR 2020 
Viaarxiv icon
Github IconAccess Paper or Ask Questions
Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon