Framework for Location Data Fusion and Pose Estimation of Excavators Using Stereo Vision

Abstract

The application of computer vision (CV) in construction projects has been investigated for many years, resulting in several advanced algorithms and methods. However, there is still a need to advance the current methods for improving the productivity of operations and safety on job sites. The excavator is one of the highly used pieces of equipment on construction sites that needs to be monitored to evaluate both safety and productivity. Knowing the productivity of excavators helps to plan the excavation process more accurately. A long queue of trucks waiting for the excavator(s) means paying more money while the trucks are not being loaded. Moreover, excavators have a higher risk of accidents due to their articulated shape compared to other excavation-related equipment. On the other hand, monitoring an object with four degrees of freedom using sensory data is a very difficult task. Therefore, this research investigates the opportunities to fuse CV-based methods and real-time location systems (RTLSs) and apply stereo vision methods to formulate a comprehensive framework for estimating the three-dimensional (3D) poses of excavators as some of the most widely used equipment on construction sites. Instead of using specialized tools, such as off-the-shelf stereo cameras or markers, this study evaluates the applicability of using the surveillance cameras on construction sites as stereo cameras. Moreover, RTLS data and two or more cameras’ data are fused by synchronizing the time and coordinate systems of the cameras and RTLS to investigate the potential of enhancing the accuracy of the pose estimation system and reducing the computational load. Finally, the performance of the proposed framework is evaluated by integrating the results of the excavator parts’ detection, the backgrounds’ subtraction, and the two-dimensional (2D) skeletons’ extraction of the parts from each camera’s view.