Adaptive Frequency Green Light Optimal Speed Advisory Based on Deep Reinforcement Learning

Abstract

The Green Light Optimal Speed Advisory (GLOSA) system suggests speeds to vehicles to assist them pass intersections during green intervals. In application, drivers can ultimately decide whether to change speed based on their driving experience. However, with the gradual popularization of autonomous driving, drivers are gradually leaving the driving area. Therefore, the central algorithms in on-board systems need to be trained more intelligently for autonomous decision-making. Specifically, we found that the frequency of decision making can significantly affect the performance of the GLOSA system, but this issue has not been discussed in previous research. In this paper, we propose an adaptive frequency GLOSA (AF-GLOSA) model based on deep reinforcement learning (DRL) algorithm. Different from traditional models, this model can extract effective features from raw data and learn decision-making experience through constant interaction with simulated environments. By using parameterized action spaces, we divided the GLOSA task into two parts: frequency control and speed consultation. The frequency control module helps filter out unnecessary operations, and the speed consultation module provides acceleration suggestions based on the results of the upper model. In addition, we have designed a novel reward function to balance fuel consumption and travel efficiency. Finally, the AF-GLOSA model was evaluated in both single intersection and multi-intersection scenarios in SUMO. The results indicate that the model can effectively reduce fuel consumption and carbon dioxide emissions in both cases. In term of the number of stops, the single-intersection outperforms the state-of-the-art method and the multi-intersection approaches the state-of-the-art method. The final results also demonstrate the necessity of considering decision-making frequency.