Modeling and Reinforcement Learning Control of an Autonomous Vehicle to Get Unstuck From a Ditch

Abstract

Autonomous vehicle control approaches are rapidly being developed for everyday street-driving scenarios. This article considers autonomous vehicle control in a less common, albeit important, situation “a vehicle stuck in a ditch.” In this scenario, a solution is typically obtained by either using a tow-truck or by humans rocking the vehicle to build momentum and push the vehicle out. However, it would be much more safe and convenient if a vehicle was able to exit the ditch autonomously without human intervention. In exploration of this idea, this article derives the governing equations for a vehicle moving along an arbitrary ditch profile with torques applied to front and rear wheels and the consideration of four regions of wheel-slip. A reward function was designed to minimize wheel-slip, and the model was used to train control agents using Probabilistic Inference for Learning COntrol (PILCO) and deep deterministic policy gradient (DDPG) reinforcement learning (RL) algorithms. Both rear-wheel-drive (RWD) and all-wheel-drive (AWD) results were compared, showing the capability of the agents to achieve escape from a ditch while minimizing wheel-slip for several ditch profiles. The policy results from applying RL to this problem intuitively increased the momentum of the vehicle and applied “braking” to the wheels when slip was detected so as to achieve a safe exit from the ditch. The conclusions show a pathway to apply aspects of this article to specific vehicles.