Vision-Based Excavator Detection and Tracking Using Hybrid Kinematic Shapes and Key Nodes

Abstract

Enhancing workplace safety continues to be a major task in the construction industry. Approximately 75% of struck-by fatalities are caused by inappropriate spatial-temporal relationships between construction workers and heavy equipment. Construction safety can be improved if the location and movement of heavy equipment are tracked in real time. However, detecting and tracking heavy equipment with kinematic joints and changing poses, such as excavators, is still a challenge for vision-based sensing methods. This study proposes to detect and track excavators using stereo cameras based on hybrid kinematic shape and key node features. Specifically, templates of excavator components are synthesized for detection following kinematic constraints of each component. Thereafter, a fast directional chamfer matching algorithm is used to detect the excavator components, and the detected components are articulated at the key nodes. Finally, the three-dimensional positions of the key nodes are tracked through triangulation to depict the excavator movements. Results from field experiments demonstrated that concatenating the detected components following a matching order enhances the detection performance. It is also found that the stereo triangulation enables efficient tracking of excavator movements by targeting at the key nodes.