Awesome Multi-Agent Learning

Multi-Agent Learning is a very exciting research area, which has strong connections with single-agent RL, multi-agent systems, game theory, evolutionary computation, communication framework and adaptation.

This repo contains a set of research papers as well as related information. The papers are sorted by time. Any suggestions and pull requests are welcome.

Overview

• Tutorials
• Research Papers
  • RL in multi-agent
  • Imitation Learning and Inverse RL
  • Offline Learning
  • Communication
  • Adaptation
  • Multi-Agent Influence
  • Others
• Environment

Tutorials

• SJTU Multi-Agent Reinforcement Learning Tutorial [website]
  • J. Wang, W. Zhang at SJTU 2018
• Multiagent Learning: Foundations and Recent Trends [website]
  • S. Albrecht, P. Stone, IJCAI 2017
• COMP310: Multi Agent System [website]
  • T. Payne, 2017-2018
• CompSci 285: Multi-Agent Systems [website]
  • D. Parkes, 2013
• CS 224M : Multi Agent Systems [website]
  • Y. Shoham, 2013-14
• Videos for "An Introduction to Multiagent Systems (Second Edition)" [website]
  • M. Wooldridge, John Wiley & Sons, 2009

Papers

RL in multi-agent

• Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition [paper]
  • B. Liu, Q. Liu, P. Stone, A. Garg, Y. Zhu, A. Anandkumar, ICML 2021
• Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learning [paper]
  • S. Iqbal, C. A. Schroeder de Witt, B. Peng, W. Böhmer, S. Whiteson, F. Sha, ICML 2021
• Deep Implicit Coordination Graphs for Multi-Agent Reinforcement Learning [paper]
  • S. Li, J. K. Gupta, P. Morales, R. Allen, M. J. Kochenderfer, AAMAS 2021
• Multi-Agent Game Abstraction via Graph Attention Neural Network [paper]
  • Y. Liu, W. Wang, Y. Hu, J. Hao, X. Chen, Y. Gao, AAAI 2020
• Actor-Attention-Critic for Multi-Agent Reinforcement Learning [paper] [code]
  • S. Iqbal, F. Sha, ICML 2019
• QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning [paper]
  • K. Son, D. Kim, W. J. Kang, D. E. Hostallero, Y. Yi, ICML 2019
• QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning [arxiv]
  • T. Rashid, M. Samvelyan, C. Witt, ICML 2018
• Emergent Complexity via Multi-agent Competition [website][Code]
  • T. Bansal, J. Pachocki, S. Sidor, ICLR 2018
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments [paper][code1][code2]
  • R. Lowe, Y. Wu, A. Tamar, NeurIPS 2017

Competition Scenario

• Emergent complexity through multi-agent competition [paper]
  • T. Bansal, J. Pachocki, S. Sidor, I. Sutskever, I. Mordatch, ICLR 2018

Imitation Learning and Inverse RL

• Multi-Agent Adversarial Inverse Reinforcement Learning [paper]
  • L. Yu, J. Song, S. Ermon, ICML 2019
• Multi-Agent Generative Adversarial Imitation Learning [paper]
  • J. Song, H. Ren, D. Sadigh, S. Ermon, NeurIPS 2018

Offline Learning

• Offline Pre-trained Multi-Agent Decision Transformer [paper]
  • L. Meng, M. Wen, Y. Yang, C. Le, X. Li, W. Zhang, Y. Wen, H. Zhang, J. Wang, B. Xu

Communication

• Multi-Agent Graph-Attention Communication and Teaming [paper] [code]
  • Y. Niu, R. Paleja, M. Gombolay, AAMAS 2021
• TarMAC: Targeted Multi-Agent Communication [paper]
  • A. Das, T. Gervet, J. Romoff, D. Batra, D. Parikh, M. Rabbat, J. Pineau, ICML 2019
• Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks [paper]
  • A. Singh, T. Jain, S. Sukhbaatar, ICLR 2019
• Learning Attentional Communication for Multi-Agent Cooperation [paper]
  • J. Jiang, Z. Lu, NeurIPS 2018
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning [paper]
• Learning Multiagent Communication with Backpropagation [paper]
  • S. Sukhbaatar, A. Szlam, R. Fergus, NeurIPS 2016

Adaptation

• Meta-CPR: Generalize to Unseen Large Number of Agents with Communication Pattern Recognition Module [paper]
  • W. C. Tseng, W. Wei, D. C. Juan, M. Sun, pre-print
• A Policy Gradient Algorithm for Learning to Learn in Multiagent Reinforcement Learning [paper]
  • D. Kim, M. Liu, M. Riemer, C. Sun, M. Abdulhai, G. Habibi, S. Lopez-Cot, G. Tesauro, J. P. How, ICML 2021
• Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning [paper]
  • Q. Long, Z. Zhou, A. Gupta, F. Fang, Y. Wu, X. Wang, ICLR 2020
• From Few to More: Largescale Dynamic Multiagent Curriculum Learning [paper]
  • W. Wang, T. Yang, Y. Liu, J. Hao, X. Hao, Y. Hu, Y. Chen, C. Fan, Y. Gao, AAAI 2020
• DiCE: The Infinitely Differentiable Monte Carlo Estimator [paper]
  • J. Foerster, G. Farquhar, M. Al-Shedivat, T. Rocktäschel, E. P. Xing, S. Whiteson, ICML 2018
• Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments [paper]
  • M. Al-Shedivat, T. Bansal, Y. Burda, I. Sutskever, I. Mordatch, P. Abbeel, ICLR 2018

Multi-Agent Influence

• Learning Latent Representations to Influence Multi-Agent Interaction [website] [paper]
  • A. Xie, D. P. Losey, R. Tolsma, C. Finn, D. Sadigh, CoRL 2020
• Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning [paper]
  • N. Jaques, A. Lazaridou, E. Hughes, C. Gulcehre, P. A. Ortega, DJ Strouse, J. Z. Leibo, N. de Freitas, ICML 2019

Application

• Distributed Heuristic Multi-Agent Path Finding with Communication [paper]
  • Z. Ma, Y. Luo, H. Ma, ICRA 2021

Others

• Adaptable Agent Populations Using a Generative Model of Policies [paper]
  • K. Derek, P. Isola, NeurIPS 2021
• Emergent Tool Use From Multi-Agent Autocurricula [paper]
  • B. Baker, I. Kanitscheider, T. Markov, Y. Wu, G. Powell, B. McGrew, I. Mordatch, ICLR 2020

Environment

Neural MMO	SMAC	Grid-World	Unity MLAgent
Hanabi Learning Environment	MAgent	multiagent-particle-envs	multiagent-competition
Multiagent emergence