The following paper is accepted to the IEEE Internationcal Conference on Robotics and Automation 2022 (ICRA 2022):  

Learning Multi-Task Transferable Rewards via Variational Inverse Reinforcement Learning 

Se-Wook Yoo, Seung-Woo Seo

Many robotic tasks are composed of a lot of temporally correlated sub-tasks in a highly complex environment. It is important to discover situational intentions and proper actions by deliberating on temporal abstractions to solve problems effectively. To understand the intention separated from changing task dynamics, we extend an empowerment-based regularization technique to situations with multiple tasks based on the framework of a generative adversarial network. We consider the problem of learning a reward and policy from expert examples under multitask environments with unknown dynamics. In this study, we define situational empowerment as the maximum of mutual information representing how an action conditioned on both a certain state and sub-task affects the future. Our proposed method derives the variational lower bound of the situational mutual information to optimize it. We simultaneously learn the transferable multi-task reward function and policy by adding an induced term to the objective function. By doing so, the multi-task reward function helps to learn robust policy for environmental change. We validate our approach on various high-dimensional complex control tasks. We demonstrate the advantages of multi-task learning and multi-task transfer learning on the robustness of both randomness and changing task dynamics. Finally, we prove that our method has significantly better performance and data efficiency than existing imitation learning methods on various benchmarks.