Scaling Agentic Inference Across Heterogeneous Compute

Zain Asgar examines running agent inference across heterogeneous hardware. The discussion covers splitting workloads between GPUs and CPUs, compilation for different targets, automated kernel optimisation and hardware-aware scheduling. It also explores networking constraints and trade-offs among numerical precision, application accuracy, performance and cost.