Topic: multi-state structured environment

Exploration is an essential part of learning, and the question of how to achieve efficient exploration has been extensively studied in the field of Reinforcement Learning (RL). Recent works have demonstrated -- both in theory and in practice -- the importance and effectiveness of sophisticated methods of exploration which are sensitive to the long-term exploratory consequences of actions, and to the global structures of the environment. This is analogous to the standard concept of value in RL. How "good" an action is depends not only on the immediate reward gained by taking it, but also on the