Reinforcement Learning-Based Variable Horizon Model Predictive Control of Multirobot Systems in Dynamic Environments

Abstract

Multirobot systems (MRS) consist of multiple autonomous robots that collaborate to perform tasks more efficiently than single-robot systems. These systems enhance flexibility, enabling applications in areas such as environmental monitoring, search and rescue, and agricultural automation while addressing challenges related to coordination, communication, and task assignment. Model predictive control (MPC) stands out as a promising controller for multirobot control due to its preview capability and effective constraint handling. However, MPC's performance heavily relies on the chosen length of the prediction horizon. Extending the prediction horizon significantly raises computation costs, making its tuning time-consuming and task-specific. To address this challenge, we introduce a framework utilizing a Collective Reinforcement Learning strategy to generate the prediction horizon dynamically based on the states of the robots. We propose that the prediction horizon of any robot in MRS depends on the states of all the robots. Additionally, we propose a versatile on-demand collision avoidance (VODCA) strategy to enable on-the-fly collision avoidance for multiple robots operating under varying prediction horizons. This approach establishes a better tradeoff between performance and computation costs, allowing for adaptable prediction horizons for each robot at every time-step. Numerical studies are performed to investigate the scalability of the proposed framework, the stiffness of the learned reinforcement learning (RL) policy, and the comparison with the fixed horizon and existing variable horizon MPC methods. The framework is also implemented on multiple TurtleBot3 Waffle Pi for various multirobot tasks.