Topic: rectified-superlinear activation function

Recurrent neural networks have been trained to perform a wide variety of experimental tasks in order to understand the underlying dynamics employed by the brain to solve those tasks. However, some of the most commonly trained network architectures have not been biologically realistic, typically limiting their analogy with the brain to the level of abstract dynamical motifs. For example, they had noiseless dynamics or no separation of excitatory and inhibitory cells. The stochastic stabilized supralinear network (SSN) incorporates these features, as well as a rectified-superlinear activation f