Abstract
Learning shared structures across changing environments enables an agent to efficiently retain
obtained knowledge and transfer it between environments.
A skill is a promising concept to represent shared structures. Several recent works
proposed unsupervised skill discovery algorithms that can discover useful skills without a reward
function. However, they focused on discovering skills in stationary environments or assumed that
a skill being trained is fixed within an episode, which is insufficient to learn and represent shared
structures. In this paper, we introduce a new unsupervised skill discovery algorithm that discovers
a set of skills that can represent shared structures across changing environments. Our algorithm
trains incremental skills and encourages a new skill to expand state coverage obtained with compositions
of previously learned skills. We also introduce a skill evaluation process to prevent our
skills from containing redundant skills, a common issue in previous work. Our experimental
results show that our algorithm acquires skills that represent shared structures across changing
maze navigation and locomotion environments. Furthermore, we demonstrate that our skills are
more useful than baselines on downstream tasks.