Topic: sleep phase

Weight sharing among neurons is widely used in deep learning: convolutional networks are an obvious example, but in addition transformers need it for matrix-matrix multiplications. Without weight sharing, deep networks perform badly on hard tasks. This is potentially problematic for the brain, since weight sharing is biologically implausible - a fact that is ignored in deep learning models of brain activity, especially of visual processing, which are becoming widely used. Recently it was shown that partial weight sharing can be implemented with a "sleep phase", in which plasticity is anti-Hebb