Model weights

The first part characterizes how continuous diffusion models generate beyond their training examples using the empirical distribution itself, clarifying their inductive bias without a population-limit assumption. The second introduces reward-guided remasking for discrete generation without changing model weights. The proposed sampler corrects earlier choices and comes with robustness guarantees under noisy process verification, contrasting its quadratic complexity with potentially exponential error accumulation in common selection heuristics.

…explores representations of model weights, transferring knowledge across architectures and tasks, and the possibility of specializing models with less computation by reusing what existing networks have learned.

Jacob Buckman examines long-context transformer architectures and the balance between model weights and recurrent state. Topics include windowed, grouped-query and latent attention, Power Retention, efficient GPU kernels and a coding model using recurrence. The conversation also discusses measuring how much useful information models extract from context.