Content-Based Image Retrieval for Construction Site Images: Leveraging Deep Learning–Based Object Detection

Abstract

Visual data comprising images and videos has become an integral aspect of construction management, potentially supplanting traditional paper-based site documentation. With the vast amount of image data generated in construction projects, an efficient retrieval system that not only enhances visual data documentation but also promotes reutilization is needed. Existing label-based image retrieval methods for construction images require manual labeling and ignore visual information. Moreover, other content-based methods that consider visual properties of construction images are limited to utilizing simple visual features of the entire image. This poses a challenge when attempting to retrieve target images from the same construction site or those involving similar construction activities, particularly considering that construction images often share similar visual properties. This research introduces a content-based image retrieval method that employs object detection to identify significant subregions within construction images and convolutional neural networks to extract refined visual features of these subregions. By simply inputting a query image, the proposed method can accurately retrieve target construction images of interest. The proposed method was validated through experiments designed to retrieve target images in both same-site and same-activity retrieval scenarios. The proposed method achieved the best mean average precision (86.4%). This technology could contribute to construction data management and decision-making processes by providing an efficient information retrieval system.