LSTMs

…from convolutional networks and LSTMs to attention and Transformers. Exercises use TensorFlow and Keras to connect model training and performance evaluation with the biological interpretation of genomic data. 2027-01-25–2027-01-27, 14:00–20:00 daily, Europe/Berlin (CET, UTC+01:00). Online course; fee €480. Participants should be familiar with Pytho

LSTMs trained on next word prediction can accurately perform linguistic tasks that require tracking long-distance syntactic dependencies. Notably, model accuracy approaches human performance on subject-verb number agreement tasks including cases with interfering attractors (Gulordava et al., 2018). However, we do not have a mechanistic understanding of how LSTMs track syntactic structures to perform such linguistic tasks. Do LSTM language models learn abstract grammatical rules like humans, or do they rely on simple heuristics and patterns? Here, we test this using long-distance gender agreeme