Topic: Markov decision process

The free-energy principle and active inference have received a significant attention in the fields of neuroscience and machine learning. However, it remains to be established whether active inference is an apt explanation for any given neural network that actively exchanges with its environment. To address this issue, we show that a class of canonical neural networks of rate coding models implicitly performs variational Bayesian inference under a well-known form of partially observed Markov decision process model (Isomura, Shimazaki, Friston, Commun Biol, 2022). Based on the proposed theory, w

We trained five rats to perform a complex auditory-guided task in a large environment (diameter 160 cm) with twelve nose-poke ports. To obtain rewards, rats had to position themselves at specific locations indicated by sounds. Despite the nontrivial task, rats reached high success rates within two 70-minute sessions. We modeled the task as a Markov Decision Process. Observed rat trajectories resembled the model's optimal policies. However, while optimal policies were deterministic, observed behavior was non-deterministic. We introduced non-deterministic, information-limited policies that reali

When an agent is trained on a complex episodic task, different task parts may be learned at different rates. How do we determine which part of the task challenged the agent the most? Since reward is provided usually only at the end of each trial, it cannot be used to infer within-trial learning trends. Behavioral features such as speed or trial duration capture trends in the agent's decision-making, but do not necessarily indicate that the agent is getting better at the task. We address this issue by modeling the task as a Markov Decision Process (MDP). The Q values of the optimal policy measu