Recognizing Diverse Construction Activities in Site Images via Relevance Networks of Construction-Related Objects Detected by Convolutional Neural Networks

Abstract

Timely and overall knowledge of the states and resource allocation of diverse activities on construction sites is critical to resource leveling, progress tracking, and productivity analysis. Despite its importance, this task is still performed manually. Previous studies have taken a significant step forward in introducing computer vision technologies, although they have been oriented toward limited classes of objects or limited types of activities. Furthermore, they especially focus on single activity recognition, where an image contains only the execution of an activity by one or a few objects. This paper introduces a two-step method for recognizing diverse construction activities in still site images. It detects 22 classes of construction-related objects using convolutional neural networks. With objects detected, semantic relevance representing the likelihood of the cooperation or coexistence between two objects in a construction activity, spatial relevance representing the two-dimensional pixel proximity in the image coordinates, and activity patterns are defined to recognize 17 types of construction activities. The advantage of the proposed method is its potential to recognize diverse concurrent construction activities in a fully automatic way. Therefore, it is possible to save managers’ valuable time in manual data collection and concentrate their attention on solving problems that necessarily demand their expertise.