Skip to contentSkip to filters

We use cookies for analytics.

The Parallelism Tradeoff: Understanding Transformer Expressivity Through Circuit Complexity

Will Merrill

New York University

Recording

45 min

Abstract

The speaker uses circuit complexity to characterize transformers operating at logarithmic arithmetic precision. A simulation by uniform, constant-depth threshold circuits gives conditional limits on solving linear equalities and recognizing certain grammars. The talk asks whether the parallelism that enables large-scale training also imposes computational limitations.

Topics