Topic: feed-forward networks

Trained artificial neural networks have become essential models in neuroscience. However, the robustness of these models to -- seemingly arbitrary -- design or initialization choices is currently debated. Some studies reported universality, others variability between solutions found by training. One choice, the magnitude of output weights, has recently received a lot of attention in machine learning, as it induces two very different classes of solutions in feed-forward networks. How this parameter affects solutions in recurrent neural networks (RNNs) trained on neuroscience tasks is not well

Behaviourally relevant sensory input is routinely transmitted and processed in biological networks, despite perpetual perceptually unrelated external, and spontaneously generated internal activity. Such activity has traditionally been understood as noise. In feed-forward networks, such noise can pose a challenge, because disambiguating stimulus-evoked activity and noise in deeper layers becomes increasingly difficult, for example in pattern classification tasks. On the other hand, the beneficial effects of noise for stochastic regularization and preventing co-adaptation were recognized in deep