Authors : Rajiv Ranganath; Thrisha R.; Vismaya M.; Akshitha Katkeri

Volume/Issue : Volume 11 - 2026, Issue 9 - September

Google Scholar : https://tinyurl.com/3k3235s3

DOI : https://doi.org/10.38124/ijisrt/26sep517

Note : A published paper may take 4-5 working days from the publication date to appear in PlumX Metrics, Semantic Scholar, and ResearchGate.

Abstract : When several self-directed learners share one environment, each must improve its own behavior while the others are also changing theirs. This is the setting studied by multi-agent reinforcement learning (MARL), and the agents involved may be working toward one goal, pulling in opposite directions, or some combination of the two. In this review we first lay out the mathematical objects used to describe such interaction, namely Markov games and their decentralized, partially observed variants, and then group the algorithmic literature into three lineages: agents that learn in isolation, methods that factor a team value into per-agent pieces (VDN, QMIX and their descendants), and actor-critic schemes that train against a critic with global knowledge (MADDPG, COMA, MAPPO). We discuss the obstacles that distinguish MARL from its singleagent counterpart, namely a moving-target learning problem, the difficulty of dividing a shared reward among team members, limited local views, and growth of the joint action space, and explain why training with global information but acting on local information has become the standard design. Uses of MARL in real-time strategy games, robot teams, automated vehicles, wireless resource sharing, and the orchestration of language-model agents are reviewed alongside the test suites the community relies on. We close by pointing to unresolved questions concerning scale, cooperation with unfamiliar partners, and deployment safety.

Keywords : Multi-Agent Reinforcement Learning; Cooperative Learning; Value Decomposition; Centralized Training Decentralized Execution; Policy Gradient Methods.

References :

1. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed. Cambridge, MA, USA: MIT Press, 2018.
2. L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent reinforcement learning,” IEEE Trans. Syst., Man, Cybern. C, vol. 38, no. 2, pp. 156–172, 2008.
3. A. Oroojlooy and D. Hajinezhad, “A review of cooperative multi-agent deep reinforcement learning,” Appl. Intell., vol. 53, no. 11, pp. 13677–13722, 2023.
4. P. Sunehag et al., “Value-decomposition networks for cooperative multi-agent learning based on team reward,” in Proc. Int. Conf. Autonomous Agents Multiagent Syst., 2018.
5. T. Rashid, M. Samvelyan, C. Schroeder de Witt, G. Farquhar, J. Foerster, and S. Whiteson, “QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning,” in Proc. Int. Conf. Mach. Learn., vol. 80, 2018, pp. 4295–4304.
6. R. Lowe, Y. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environments,” in Adv. Neural Inf. Process. Syst., vol. 30, 2017.
7. J. Foerster, G. Farquhar, T. Afouras, N. Nardelli, and S. Whiteson, “Counterfactual multi-agent policy gradients,” in Proc. AAAI Conf. Artif. Intell., vol. 32, no. 1, 2018.
8. C. Yu, A. Velu, E. Vinitsky, J. Gao, Y. Wang, A. Bayen, and Y. Wu, “The surprising effectiveness of PPO in cooperative multi-agent games,” in Adv. Neural Inf. Process. Syst., vol. 35, 2022.
9. M. L. Littman, “Markov games as a framework for multi-agent reinforcement learning,” in Proc. Int. Conf. Mach. Learn., 1994, pp. 157–163.
10. D. S. Bernstein, R. Givan, N. Immerman, and S. Zilberstein, “The complexity of decentralized control of Markov decision processes,” Math. Oper. Res., vol. 27, no. 4, pp. 819–840, 2002.
11. M. Tan, “Multi-agent reinforcement learning: Independent versus cooperative agents,” in Proc. Int. Conf. Mach. Learn., 1993, pp. 330–337.
12. M. Samvelyan et al., “The StarCraft multi-agent challenge,” arXiv preprint arXiv:1902.04043, 2019.
13. K. Son, D. Kim, W. J. Kang, D. E. Hostallero, and Y. Yi, “QTRAN: Learning to factorize with transformation for cooperative multi-agent reinforcement learning,” in Proc. Int. Conf. Mach. Learn., 2019, pp. 5887–5896.
14. Y. Yang, J. Hao, B. Liao, K. Shao, G. Chen, W. Liu, and H. Tang, “Qatten: A general framework for cooperative multiagent reinforcement learning,” arXiv preprint arXiv:2002.03939, 2020.
15. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017.
16. N. Bard et al., “The Hanabi challenge: A new frontier for AI research,” Artif. Intell., vol. 280, Art. no. 103216, 2020.
17. J. Foerster, I. A. Assael, N. de Freitas, and S. Whiteson, “Learning to communicate with deep multi-agent reinforcement learning,” in Adv. Neural Inf. Process. Syst., vol. 29, 2016.
18. S. Sukhbaatar, A. Szlam, and R. Fergus, “Learning multiagent communication with backpropagation,” in Adv. Neural Inf. Process. Syst., vol. 29, 2016.
19. P. Hernandez-Leal, B. Kartal, and M. E. Taylor, “A survey and critique of multiagent deep reinforcement learning,” Auton. Agents Multi-Agent Syst., vol. 33, no. 6, pp. 750–797, 2019.
20. G. Papoudakis, F. Christianos, L. Schafer, and S. V. Albrecht, “Benchmarking multi-agent deep reinforcement learning algorithms in cooperative tasks,” in Proc. Adv. Neural Inf. Process. Syst. Datasets and Benchmarks Track, 2021.
21. R. Mirsky, I. Carlucho, A. Rahman, E. Fosong, W. Macke, M. Sridharan, P. Stone, and S. V. Albrecht, “A survey of ad hoc teamwork research,” in Proc. Eur. Conf. Multi-Agent Syst., 2022.
22. J. Treutlein, M. Dennis, C. Oesterheld, and J. Foerster, “A new formalism, method and open issues for zero-shot coordination,” in Proc. Int. Conf. Mach. Learn., 2021, pp. 10413–10423.
23. O. Vinyals et al., “Grandmaster level in StarCraft II using multi-agent reinforcement learning,” Nature, vol. 575, no. 7782, pp. 350–354, 2019.
24. C. Berner et al., “Dota 2 with large scale deep reinforcement learning,” arXiv preprint arXiv:1912.06680, 2019.
25. Z. Ning and L. Xie, “A survey on multi-agent reinforcement learning and its application,” J. Autom. Intell., vol. 3, no. 2, pp. 73–91, 2024.
26. H. Ma et al., “Coevolving with the other you: Fine-tuning LLM with sequential cooperative multi-agent reinforcement learning,” in Adv. Neural Inf. Process. Syst., vol. 37, 2024.