ePoster, COSYNE 2025
Implicit generative models using Kernel Similarity Matching
- Montreal, Canada
- Machine Learning, Computational Neuroscience and
Abstract
Understanding how the brain encodes stimuli is a fundamental problem in computational neuroscience. This has inspired the development of artificial neural networks that learn representations by incorporating brain-like learning abilities. Recent work has focused on learning representations (latents) by capturing similarity between inputs and latents, but these methods have typically been used to learn downstream features from the input and have not been studied in the context of a generative paradigm, where one can map the representations back to the input space, incorporating not only bottom-up interactions but also learning features in a top-down manner. In this work, we investigate a Kernel Similarity Matching (KSM) framework for generative modeling. Using a sparse coding objective as our generative framework for learning representations, we show that representation learning in this context is equivalent to maximizing similarity between the input and a latent kernel leading to an implicit generative map. Additionally, by imposing a metabolic-limit-inspired prior on the latent space, we demonstrate the framework’s ability to learn latents that characterize the input manifold’s geometry, potentially suggesting a method that could offer insights as to how task representations can be encoded in the brain. To solve the KSM objective, we propose a novel Alternate Direction Method of Multipliers (ADMM) based algorithm to calculate the gradients of the optimization variables. Finally, we discuss how the KSM representation learning problem can lead to a biologically plausible architecture that integrates similarity matching (bottom-up interactions) with predictive coding (top-down interactions).
Topics
- computational neuroscience, search World Wide
- artificial neural networks, search World Wide
- Kernel Similarity Matching, search World Wide
- generative modeling, search World Wide
- sparse coding, search World Wide
- representation learning, search World Wide
- latent space, search World Wide
- metabolic-limit-inspired prior, search World Wide
- predictive coding, search World Wide