Safe Reinforcement Learning-Based Balance Control for Multi-Cylinder Hydraulic Press

Abstract

Considering the load uncertainty and unmodeled dynamics in multicylinder hydraulic systems, this paper proposes a balance control algorithm based on safe reinforcement learning to release the restrictions of classical model-based control methods that depend on fixed gain. In this paper, the hydraulic press is controlled by a trained agent that directly maps the system states to control commands in an end-to-end manner. By introducing an action modifier into the algorithm, the system states are kept within security constraints from the beginning of training, making safe exploration possible. Furthermore, a normalized exponential reward function has been proposed. Compared with a quadratic reward function, the precision is greatly improved under the same training steps. The experiment shows that our algorithm can achieve high precision and fast balance for multicylinder hydraulic presses while being highly robust. To the best of our knowledge, this research is the first to attempt the application of a reinforcement learning algorithm to multi-execution units of hydraulic systems.