Topic: variational lower bound

Information theory provides a powerful theoretical framework for a quantitative understanding of the structure and function of neural circuits. Despite this, the application of these ideas to neuroscience and machine learning has been hampered by the fact that the computation and maximization of mutual information is only analytically tractable for simple coding problems, under the assumption of small network sizes and low dimensional stimuli. Using tools from the theory of spin-glasses we present a novel approach to estimating the entropy of large neural populations in response to arbitrary s