The MABe22 Benchmarks for Representation Learning of Multi-Agent Behavior

To scalably study multi-agent behavior, advances in unsupervised and self-supervised learning have enabled many different behavioral representations to be learned from trajectory data. However, such representation learning approaches are generally evaluated on specific datasets and tasks, and it is difficult to compare methods quantitatively to measure progress on representations for behavior analysis.

We aim to address this by introducing a large-scale, multi-agent trajectory dataset from real-world behavioral neuroscience experiments that covers a range of behavior analysis tasks. Our dataset consists of common model organisms (mice and flies) in a variety of settings (different strains, lengths of interaction, optogenetic and thermogenetic stimulation), with a subset consisting of expert-annotated behavior labels.