Deep Reinforcement Learning for Helicopter Assembly Workshop Scheduling Considering the Workers' Operational Proficiency

Abstract

Abstract. As a critical stage in helicopter manufacturing, the assembly process relies on effective scheduling to ensure both efficiency and quality. Traditional, experience-based scheduling methods are often inadequate for complex shop-floor environments, especially given the heterogeneity of worker skills and complex technological constraints. Therefore, this article proposes a deep reinforcement learning approach based on proximal policy optimization with an attention mechanism (PPO-AM) to solve the helicopter assembly workshop scheduling problem (HASP) with consideration for worker skill proficiency. First, an assembly time prediction model was established that integrates workers' skill levels, task criticality, and the dynamic evolution of proficiency. This model provides a precise foundation for task-time estimation in assembly workshop scheduling. Based on this foundation, a shop scheduling model incorporating multiple process constraints was constructed. Furthermore, the state space, action space, and reward function were dynamically adjusted according to production progress, thereby formulating the problem as a Markov decision process (MDP). Within this framework, a PPO-AM method incorporating a self-attention mechanism was proposed, and an assembly shop scheduling agent was developed based on this method to enable flexible and efficient worker allocation in practical scenarios. The PPO-AM method leverages the self-attention mechanism to assess the importance of state features, enabling the agent to adaptively focus on critical information, thereby enhancing its state awareness and policy generalization capability. Experiments were conducted on assembly shop cases of various scales as well as in real-world engineering scenarios, and the results demonstrated that PPO-AM exhibits superior performance and practical value in complex assembly scheduling tasks.