Adaptive Generalized Elliptical Slice Sampling

A central goal in Bayesian computation is designing MCMC algorithms that apply broadly, need no manual tuning and scale efficiently with dimension. Elliptical slice sampling meets many of these needs: it is tuning-free, does not require a differentiable target, adapts its step size to local geometry automatically and, through generalized elliptical slice sampling, applies to a broad class of target distributions. Under favourable conditions its mixing time can scale logarithmically with dimension, making it more scalable than Hamiltonian Monte Carlo and an optimally tuned adaptive random walk. That advantage is fragile, however: efficiency degrades rapidly when the target distribution and the distribution generating the ellipse-defining auxiliary variables are mismatched, which explains the poor efficiency often seen in high-dimensional problems. The talk introduces adaptive generalized elliptical slice sampling (AGESS), which gradually adapts the distribution of the auxiliary variables so that the sampler self-corrects from a slow-mixing to a fast-mixing regime through online learning. Under mild regularity conditions AGESS is proven ergodic without requiring differentiability of the target and while allowing heavy-tailed targets. The algorithm is robust and efficient for multimodal targets, strong inter-parameter dependence, regions of high curvature and high dimension, as demonstrated on applications including deep Gaussian process surrogate modelling and high-dimensional sparse regression.