Recent evidence suggests the brain performs sampling-based Bayesian inference to solve various cognitive tasks [1–6]. Neural sampling requires neural circuits to exhibit statistically stable variability to draw posterior samples. While ad-hoc frameworks simply inject randomness into spike emission, it remains unknown how such variability for sampling is generated in biological neural circuits. Here we propose that recurrent spiking circuits can internally generate statistically stable variability for posterior sampling, eliminating the need for an external random source. Our proposed circuit model, based on the classical excitation (E) and inhibition (I) balanced spiking network, features recurrent connections between E neurons comprising two components: a strong, unstructured wiring that exists in all balanced networks, along with a weak, structured wiring depending on the tuning similarity between E neurons. The strong, random connections induce an asyncrhonous balanced state, internally generating Poissonian spiking variability to drive sampling. Concurrently, the weak structured E-to-E connections shape the statistics of the sampled posterior. Through this model, we demonstrate how optimal sampling arises from the interplay between weak structured connections and strong random ones. We illustrate that a neural circuit of coupled neuron populations with fixed connections can sample multivariate stimulus posteriors with varying uncertainties, utilizing Langevin sampling dynamics. Our study provides novel mechanistic insight into how neural circuits internally generate variability to accomplish Bayesian sampling.