Random Embeddings, Matrix-valued Kernels and Deep Learning

The success of deep neural networks raises questions about the scalability of kernel methods and the roles of large datasets, depth and training algorithms. This talk examines techniques that make kernel learning practical for large datasets in both scalar and multivariate prediction. The methods combine randomized data embeddings, Quasi-Monte Carlo acceleration, distributed convex optimization and input-output kernel learning. Experiments on speech-recognition and computer-vision datasets compare randomized kernel methods with deep neural networks and report essentially matching performance. The talk explores how randomized kernel constructions can resemble neural-network architectures, and how invariant kernel learning and matrix-valued kernels might support deeper architectures. It discusses research results and connections between these approaches.