Estimation of Control Intervals for Reinforcement Learning-Based Manufacturing-Line Control

Abstract

Abstract. Generative AI is reshaping manufacturing by generating designs, diagnosing bottlenecks, and optimizing complex systems, which includes controlling manufacturing lines. Efficient control of manufacturing lines requires coordinated decisions in routing, worker allocation, and scheduling. Although model-based approaches can yield optimal or near-optimal plans, they struggle to scale more complex systems. With advances in discrete-event simulation and AI-driven decision-making, generative decision models, such as reinforcement learning (RL), have emerged as alternatives for optimizing manufacturing systems. We show that a crucial design choice is the policy control interval. Because parts take time to propagate through stations and buffers in manufacturing systems, choosing this interval without analysis can result in suboptimal behavior and lower throughput. We introduce a layout-aware method for selecting the control interval in policy training. From the line layout, we estimate the distribution of end-to-end unit transit times and derive a control interval based on the distribution. Our results demonstrate that policy performance versus control interval is nonmonotonic with an interior optimum. Aligning the control interval with the dynamics of the manufacturing process yields gains up to 2.7× improvement in return and increased training stability relative to one-tick baselines, without modifying environments or algorithms. These results indicate that the control interval is a crucial factor the agent learning process. By aligning the control interval of policies with the way material flows, our method provides a plug-and-play procedure for a more robust RL in manufacturing control.