Apply neural networks to biological sequence analysis, including sequence classification, functional-feature discovery, source tracking and sequence generation. Hands-on notebooks introduce sequence encoding and language-model concepts, progressing from convolutional networks and LSTMs to attention and Transformers. Exercises use TensorFlow and Keras to connect model training and performance evaluation with the biological interpretation of genomic data. 2027-01-25–2027-01-27, 14:00–20:00 daily, Europe/Berlin (CET, UTC+01:00). Online course; fee €480. Participants should be familiar with Pytho
ePoster · Neuroscience
Do LSTMs see gender? Probing the ability of LSTMs to learn abstract syntactic structure
Priyanka Sukumaran · Neuromatch 5
Wed, Sep 28, 2022
LSTMs trained on next word prediction can accurately perform linguistic tasks that require tracking long-distance syntactic dependencies. Notably, model accuracy approaches human performance on subject-verb number agreement tasks including cases with interfering attractors (Gulordava et al., 2018). However, we do not have a mechanistic understanding of how LSTMs track syntactic structures to perform such linguistic tasks. Do LSTM language models learn abstract grammatical rules like humans, or do they rely on simple heuristics and patterns? Here, we test this using long-distance gender agreeme