Understanding the trade-off between plasticity and stability–how neural networks form new memories while avoiding runaway potentiation or fast overwriting of memories–is a fundamental problem in neuroscience. Previous theoretical studies that relied on non-plausible plasticity rules did not reproduce realistic synaptic weight distributions or neural activity statistics and predicted fast forgetting times. We revisit this problem by investigating the implications of a multiplicative calcium-based plasticity rule in a recurrent network. This rule was recently fitted to experiments done in physiological conditions, and is qualitatively different from classical STDP. We analytically approximate the full distribution of synaptic weight modifications as a function of pre- and postsynaptic firing rates, and temporal correlations. Using mean-field analysis and spiking network simulations, we show that the multiplicative plasticity rule, without fine-tuning, gives a stable, unimodal synaptic weight distribution characterized by a large fraction of strong synapses, as seen in experiments. The strong synapses within this distribution remain stable over long times but do not cause run-away dynamics in recurrent networks, consistent with previous electrophysiological and behavioral studies. Our results provide a mechanistic understanding of how stable learning and memory emerge on the behavioral level from an STDP rule measured in physiological conditions. Furthermore, the plasticity rule we investigate is mathematically equivalent to other learning rules which rely on the statistics of coincidences, so we expect that our formalism will be useful to study other learning processes.