Reinforcement-trained recurrent networks reproduce human beat-synchronization dynamics

Most adult humans can move their bodies in time with rhythmic music, but how this capacity develops is poorly understood. Moving to music is often pleasurable, suggesting that a system of intrinsic reward guides this developmental process. Here, we ask what reward systems can scaffold the robust development of human-like synchronization. We explore this question in a recurrent neural network model whose input is a series of metronomic cues delivered at a range of tempi, and whose output is a series of “finger taps” executed with a short delay. The network is trained through reinforcement learning under four different schemes that incentivize tap/cue synchrony in distinct ways. We find that the most successful of these schemes rewards early taps more than late ones and additionally incentivizes tempo matching. The trained agent exhibits tapping error correction that captures key qualitative and quantitative features of human error correction data, including an asymmetry in which lateness is corrected more than earliness. When timing jitter is introduced into training or testing, the agent shows a human-like tendency to tap ahead of the cues (a “negative mean asynchrony”) that closely resembles human data in its scaling with jitter and amount of training. The agent also maintains accurate inter-tap timing during a tapping continuation phase, demonstrating human-like internalization of tempo, despite the fact that this condition was not included in its training. Neural activity exhibits patterns observed in monkeys trained to synchronize, as well as signs of subjective rhythmitization—the human tendency to perceive metronome clicks in groups of two. These results suggest that intrinsic reinforcement for early action and imitation may be important for the development of audiomotor synchronization, and provide a model system in which to explore synchronization learning. We train a recurrent neural network to synchronize its output with a metronome across a range of tempos using reinforcement learning. The most effective reinforcement scheme also reproduces key features of human synchronization behavior.