Mechanisms of prediction in linear networks

Predicting the future state of the environment is a crucial task for neural systems, which
must compensate for significant delays in both sensation and action. Despite this fundamental importance, we
lack a rigorous theoretical understanding of the underlying mechanisms of prediction. While several
phenomenological and biophysical mechanisms have been proposed to model experimental data, they are
typically nonlinear and difficult to analyze. On the other hand, prediction has been exhaustively studied in
classical linear filtering theory, but this engineering approach does not explicitly address the question of
mechanism. Here, we start with the simplest possible operational definition of prediction and derive necessary
and sufficient conditions for a linear network to be predictive. We identify feedback and feedforward inhibition
as the primary mechanisms of prediction, and calculate analytically how the temporal horizon for prediction
depends on connection weights and time constants for small networks. We then show how to generalize from
this first order description to higher order, and prove that the minimum size of the network increases with the
complexity of the desired prediction. While the theory is developed in the absence of noise, we demonstrate
analytically that sensitivity to noise grows rapidly with the prediction horizon and order, limiting the practical
utility of overly ambitious predictors. Finally, we discuss how interacting populations can be made predictive,
which is key since strong lateral interactions are associated with a slowing of dynamics. The linear approach
developed here provides a basis for understanding nonlinear systems through linearization and conjugacy
arguments, and thus unifies some of the disparate nonlinear models in the field. We hope it will aid in the
dissection and interpretation of neural circuit motifs involved in prediction, and conjecture that explicitly
incorporating mechanisms of prediction in artificial neural networks may facilitate their training and
performance on prediction tasks.