A Multistep Reinforcement Learning–Based Economic Adaptive Cruise Control Strategy for Electric Vehicles in a Car-Following Scenario

Abstract

To simultaneously achieve vehicle safety, riding comfort, longer battery life, and energy economy of electric vehicles (EVs) in a car-following scenario, this paper proposed an economic adaptive cruise control (Eco-ACC) strategy based on the multistep tree backup of the double deep Q network (MTDDQN) algorithm. First, the paper presented the longitudinal dynamics model, the energy storage system model, and the control objectives for EVs, which will be used to develop the MTDDQN-based Eco-ACC strategy. Next, the Eco-ACC strategy is provided by designing the state variables, control variables, and unity functions within the MTDDQN framework in order to achieve the aforementioned control objectives. Then, to facilitate the understanding of the Eco-ACC strategy, the algorithm of the MTDDQN-based Eco-ACC strategy is given. Simulation results conducted under different driving cycles demonstrate the effectiveness of the MTDDQN-based Eco-ACC strategy for EVs. Comparisons with a benchmark algorithm reveal that the proposed strategy effectively reduces battery capacity loss and decreases vehicle energy consumption.