SEQUENCE MODIFICATION TRAINING IN SONGBIRDS WITH POSITIVE REINFORCEMENT LEARNING

Songbirds, such as Bengalese finches (Lonchura striata domestica) produce syntactically organized learned motor sequences. Their song is composed of individual acoustic elements called syllables. Variability in the syllable sequence can be found at branch points, where one syllable can be followed probabilistically by multiple syllable types and in repeat phrases, where the same syllable type is repeated a variable number of times. Adult songbirds can learn to modify specific aspects of their song through reinforcement learning: transition probabilities between syllables can be decreased when a syllable is targeted with aversive auditory feedback. Here, we show that positive reinforcement using food reward can increase the frequency of rare transitions in song. Because the food reward signal is slow compared to the timescale of song, we used an auditory secondary reinforcer which predicted later food reward. Five male Bengalese finches learned to associate the primary reinforcer (food delivery by automated feeder) with the secondary reinforcer (short click sound). These birds participated in several song trainings, where specific target syllables were paired with the click during singing. After the song ended, we granted access to the feeder for a time proportional to the number of target syllables in the previous song. In both branch points and repeat phrases, all birds increased the frequency of the targeted syllables in the experiment. Our results demonstrate that sequence modification training is possible using positive reinforcement, which can be utilized to dramatically increase the likelihood of rare transitions in song.