Infrastructure for AI at Scale - With Benny Chen (Fireworks AI)

Fireworks AI co-founder Benny Chen explains the infrastructure required to serve machine-learning models at scale, from GPU procurement and kernels to runtimes and request routing. He discusses power constraints, hardware utilization, mixture-of-experts models, quantization, reinforcement-learning efficiency and speculative decoding. The conversation also covers distributed training across data centres, sparse autoencoders, the constraints on open-source interpretability research, and the relationship between open models and compute availability. Hosted by Ravid Shwartz Ziv and Allen Roush; the full conversation is available on YouTube.