Spatial Layout Optimization Approach for Highway Variable Speed Limit Zones Based on Improved Q-Learning Algorithm

Abstract

Variable speed limit (VSL) is a commonly used traffic control method to optimize the mainstream of highways. In previous studies, the spatial layout design of VSL zones (including quantity, length, and position) was adaptively determined based on the study content, overlooking the impact of the VSL zone spatial layout on the VSL strategy applicability. This study proposes a framework based on an improved Q-learning algorithm to optimize the spatial layout of VSL zones, with the reward function of the algorithm considering traffic utilization. First, the spatial density of the VSL zones is divided into three levels, and the Q-learning algorithm is used to control the speed limit value of each VSL zone. The reward function is updated by positively varying traffic utilization in the acceleration and bottleneck zones. Subsequently, the effects of different spatial layouts of VSL zones on the total time spent (TTS), time loss (TL), and number of stops (NS) of traffic flows are evaluated in SUMO microsimulations of a high-traffic demand scenario for a highway two-lane ramp merge area. The tendency effect of different levels of speed limits on traffic flow control under a fixed VSL zone length is further analyzed. The results showed that: (1) The QL-VSL has optimal and similar TTS reduction (5.47%, 4.90%, and 5.15%) when the total length of the speed limit zone is near 500–600 m; and (2) compared with Level I, Level II shows better performance in terms of TL (reduced by 9.29% TL), while Level III demonstrates better performance in stabilizing traffic flow (reduced by 15.17% NS). The results of this study can provide suggestions for highway managers to optimize the spatial deployment of VSL control zones according to different traffic demands.