Topic: Globally Gated Deep Linear Networks

Gating represents multiplicative interactions in neural systems, and are well known motifs of biological neural circuits. Recently neural networks with gating have attracted increased attention. Gated Linear Networks (GLNs) have been studied for their interesting capabilities such as biologically plausible learning and reduced forgetting in sequential learning. The network has also been shown to mimic some aspects of cerebellar architecture and plasticity. However, there has been virtually no theoretical understanding of the capabilities and limitations of GLNs. To close this gap, we introduce