Understanding how the brain encodes stimuli is a fundamental problem in computational neuroscience. This has inspired the development of artificial neural networks that learn representations by incorporating brain-like learning abilities. Recent work has focused on learning representations (latents) by capturing similarity between inputs and latents, but these methods have typically been used to learn downstream features from the input and have not been studied in the context of a generative paradigm, where one can map the representations back to the input space, incorporating not only bottom-up interactions but also learning features in a top-down manner. In this work, we investigate a Kernel Similarity Matching (KSM) framework for generative modeling. Using a sparse coding objective as our generative framework for learning representations, we show that representation learning in this context is equivalent to maximizing similarity between the input and a latent kernel leading to an implicit generative map. Additionally, by imposing a metabolic-limit-inspired prior on the latent space, we demonstrate the framework’s ability to learn latents that characterize the input manifold’s geometry, potentially suggesting a method that could offer insights as to how task representations can be encoded in the brain. To solve the KSM objective, we propose a novel Alternate Direction Method of Multipliers (ADMM) based algorithm to calculate the gradients of the optimization variables. Finally, we discuss how the KSM representation learning problem can lead to a biologically plausible architecture that integrates similarity matching (bottom-up interactions) with predictive coding (top-down interactions).