Deep reinforcement learning has been used to solve complex tasks in various fields, particularly in robotics control. However, agents trained using deep reinforcement learning have a problem of taking overconfident actions, even when the input state is far from the learned state distribution. This restricts deep reinforcement learning from being applied to real-world environments as overconfident actions in unlearned situations can result in catastrophic events; such as the collision of an autonomous vehicle. To address this, the agents should know ``what they do not know'' and choose an action by considering not only the state but also its uncertainty. In this study, we propose a novel uncertainty-conditioned policy (UNICON) inspired by the human behavior of changing policies according to uncertainty, e.g., slowing a car on a narrow road that has never been visited before. Our experimental results demonstrate that the proposed method is robust to unfamiliar scenarios that are not seen during training.