Authors : Mohanad Ibrahim Elassadi

Volume/Issue : Volume 11 - 2026, Issue 9 - September

Google Scholar : https://tinyurl.com/5n84xm5s

DOI : https://doi.org/10.38124/ijisrt/26sep1187

Note : A published paper may take 4-5 working days from the publication date to appear in PlumX Metrics, Semantic Scholar, and ResearchGate.

Abstract : As more companies embrace the use of autonomous mobile robots to organize their smart warehouses, one of the major issues facing them is the coordination of these fleets under very restrictive throughputs. Task allocation and path planning go hand in hand: if tasks are assigned without considering congestion, then each task can result in a conflict or a deadlock. Conversely, if path planning takes place totally reactively, then the total travel effort and energy lead to wasteful results. This research introduces TwinSync-HRL, a framework that involves the coupling of reinforcement learning for task allocation in multi-robot settings with classical search-based path planning. The allocation is based on a learned Q-learning policy to manage the assignment of pickup-and-delivery tasks to robots while the actual path planning takes place using A* with space-time reservation. TwinSync uses a digital twin of the overall warehouse in order to facilitate synchronization and to quantify the conditions under which this synchronization takes place. The results show that the learned Q-learning allocator is superior to both the greedy and auction-based methods in the sense that it clears all 20 tasks per episode without conflict and at the lowest possible distance (483 ± 5.3 cells traveled). However, it does have a slower rate of task clearing (under mean conditions of 100.6 ± 6.5 steps as compared to 84.4 to 86.5 for the other two methods). When the total synchronization latency time increases from 0 to 15 steps, the number of replans per episode goes from 17.4 to 37.9 and the total conflicts in execution triples.

Keywords : Reinforcement Learning; Multi-Robot Task Allocation; Multi-Agent Path Finding; Digital Twin; Smart Warehouse; Autonomous Mobile Robots.

References :

1. R. de Koster, T. Le-Duc, and K. J. Roodbergen, “Design and control of warehouse order picking: A literature review,” European Journal of Operational Research, vol. 182, no. 2, pp. 481–501, 2007, doi: 10.1016/j.ejor.2006.07.009.
2. B. P. Gerkey and M. J. Matarić, “A formal analysis and taxonomy of task allocation in multi-robot systems,” The International Journal of Robotics Research, vol. 23, no. 9, pp. 939–954, Sep. 2004, doi: 10.1177/0278364904045564.
3. R. Lowe, Y. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environments,” in Advances in Neural Information Processing Systems 30, 2017.
4. T. Rashid, M. Samvelyan, C. Schroeder de Witt, G. Farquhar, J. Foerster, and S. Whiteson, “QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning,” in Proc. 35th Int. Conf. Mach. Learn., PMLR 80, 2018, pp. 4295–4304.
5. C. Yu, A. Velu, E. Vinitsky, J. Gao, Y. Wang, A. Bayen, and Y. Wu, “The surprising effectiveness of PPO in cooperative multi-agent games,” in Advances in Neural Information Processing Systems, Datasets and Benchmarks Track, 2022. [Online]. Available: https://arxiv.org/abs/2103.01955
6. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv:1707.06347, 2017, doi: 10.48550/arXiv.1707.06347.
7. G. Papoudakis, F. Christianos, L. Schäfer, and S. V. Albrecht, “Benchmarking multi-agent deep reinforcement learning algorithms in cooperative tasks,” in Proc. 35th Conf. Neural Inf. Process. Syst. (NeurIPS), Track on Datasets and Benchmarks, 2021. [Online]. Available: https://arxiv.org/abs/2006.07869
8. J. Zhao and C. Zhang, “Order allocation strategy optimization in a goods-to-person robotic mobile fulfillment system with multiple picking stations,” Applied Sciences, vol. 15, no. 16, Art. no. 9173, 2025, doi: 10.3390/app15169173.
9. T. Zuo, H. Liu, S. Yang, W. Wang, Y. Peng, and R. Wang, “A reinforcement learning method for automated guided vehicle dispatching and path planning considering charging and path conflicts at an automated container terminal,” Journal of Marine Science and Engineering, vol. 14, no. 1, Art. no. 55, 2025, doi: 10.3390/jmse14010055.
10. F. Tao, H. Zhang, A. Liu, and A. Y. C. Nee, “Digital twin in industry: State-of-the-art,” IEEE Transactions on Industrial Informatics, vol. 15, no. 4, pp. 2405–2415, Apr. 2019, doi: 10.1109/TII.2018.2873186.
11. M. Russo, C. Santoro, F. F. Santoro, and A. Tudisco, “Advancing robotic systems with distributed multi-agent digital twins: A scalable and adaptive framework,” SN Computer Science, vol. 6, no. 8, Art. no. 975, 2025, doi: 10.1007/s42979-025-04525-w.
12. S. Werbińska-Wojciechowska, R. Giel, and O. Stryhunivska, “A digital twin-driven sensing and fuzzy decision framework for safety monitoring of autonomous mobile robot systems in intralogistics,” Sensors, vol. 26, no. 16, Art. no. 5284, 2026, doi: 10.3390/s26165284.
13. S. Farrell, C. Li, H. Yu, R. Yoshimitsu, S. Gao, and H. I. Christensen, “Safe human robot navigation in warehouse scenario,” arXiv:2503.21141, 2025, doi: 10.48550/arXiv.2503.21141.
14. M. Otte, M. J. Kuhlman, and D. Sofge, “Auctions for multi-robot task allocation in communication limited environments,” Autonomous Robots, vol. 44, nos. 3–4, pp. 547–584, 2020, doi: 10.1007/s10514-019-09828-5.
15. J. Rodriguez, S. Koenig, W. Dong, and Q. Lu, “Grouping auction-consensus algorithm for decentralized task allocation in multi-robot systems,” arXiv:2608.15884, 2026, doi: 10.48550/arXiv.2608.15884.
16. P. E. Hart, N. J. Nilsson, and B. Raphael, “A formal basis for the heuristic determination of minimum cost paths,” IEEE Transactions on Systems Science and Cybernetics, vol. 4, no. 2, pp. 100–107, Jul. 1968, doi: 10.1109/TSSC.1968.300136.
17. S. Koenig and M. Likhachev, “Fast replanning for navigation in unknown terrain,” IEEE Transactions on Robotics, vol. 21, no. 3, pp. 354–363, Jun. 2005, doi: 10.1109/TRO.2004.838026.
18. S. Karaman and E. Frazzoli, “Sampling-based algorithms for optimal motion planning,” The International Journal of Robotics Research, vol. 30, no. 7, pp. 846–894, Jun. 2011, doi: 10.1177/0278364911406761.
19. G. Sharon, R. Stern, A. Felner, and N. R. Sturtevant, “Conflict-based search for optimal multi-agent pathfinding,” Artificial Intelligence, vol. 219, pp. 40–66, 2015, doi: 10.1016/j.artint.2014.10.006.
20. H. He, X. Wu, S. Zhao, and Z. Ren, “Conflict-based search for multi-agent path finding with elevators,” arXiv:2602.20512, 2026, doi: 10.48550/arXiv.2602.20512.
21. G. Sartoretti, J. Kerr, Y. Shi, G. Wagner, T. K. Satish Kumar, S. Koenig, and H. Choset, “PRIMAL: Pathfinding via reinforcement and imitation multi-agent learning,” IEEE Robotics and Automation Letters, vol. 4, no. 3, pp. 2378–2385, Jul. 2019.