Topic: threshold-linear activation function

Memories are thought to be stored in synapses between neurons, but these connections in the hippocampus show complete turnover within a fortnight[1]. Dynamics have also been seen at the representational level, as the cells responding to a particular position change across days even when the animal isn’t overtly learning or forgetting[2]. While some researchers consider representational dynamics to be an inevitable consequence of ongoing learning, others suggest that drift may be beneficial. For instance, theoretical work by Kappel et al.,[3] suggests that if initial learning leads the system t