Operation Optimization Strategy for Continuous Extracorporeal Blood Circulation Devices Based on Digital Twins and Reinforcement Learning

Abstract

Abstract. Traditional fixed strategies struggle to balance efficacy and safety due to the nonlinear coupling of control variables like pump and flow rates, and poor personalized control from patient variability. This paper constructs a digital twin-reinforcement learning (RL) soft actor-critic (SAC) framework for continuous extracorporeal blood circulation devices. The twin layer employs a physical-data hybrid model with a residual network for online error correction and Ensemble Kalman Filtering for real-time parameter assimilation. The control layer uses a constrained SAC algorithm, integrating a Lagrange cost, action change rate constraints, and a safety projection operator. Training involves offline pretraining followed by online refinement on the digital twin, with prioritized experience replay and domain randomization. Systematic validation includes simulation, benchtop, and real-world testing. Results show the framework achieves average steady-state errors of 0.37%, 0.45%, and 0.73% in low/medium/high-viscosity patient groups. The comprehensive assessment reports oxygenation efficiency of 95 ± 2 mL O2/min, response time of 1.2 ± 0.1 s, and average severity of 1.8, improving personalized regulation accuracy, real-time response, and operational safety.