Three-Dimensional CNN-Based Model for Fine-Grained Pedestrian Crossing Behavior Recognition in Automated Vehicles

Abstract

Revealing pedestrian behaviors and intentions is conducive to ensure the secure interaction between automated vehicles and pedestrians in urban roads. Most previous research on pedestrian intention has lacked consideration of fine-grained behavioral characteristics associated with intention. This paper establishes a classification framework for identifying the specific intentions and behaviors of pedestrian crossings, which is concerned with head posture and gestures. A novel pedestrian fine-grained crossing behavior recognition model based on three-dimensional (3D) convolutional neural network (CNN) is proposed. It has the improved attention mechanism modules and cascaded residual networks, which can pull out channel and spatiotemporal features and make feature granularity better. Then, a new large-scale pedestrian behavior video data set based on the proposed classification framework is captured by a real-car test. The model and data set proposed in this study are experimentally verified and compared with the Resnet-3D and spatiotemporal graph convolution network (ST-GCN) model and a publicly available pedestrian detection data set. The results showed that the proposed method can effectively detect the fine-grained behavior of pedestrians and understand their intentions accordingly. The findings can be indicative to the safe interaction between autonomous vehicles and pedestrians and therefore improve the traffic efficiency of vehicle.