Construction Photo Localization in 3D Reality Models for Vision-Based Automated Daily Project Monitoring

Abstract

Recent research has focused on visualizing and analyzing massive visual data (images/videos) captured at construction sites to improve coordination, communication, and planning. One key approach is to generate reality models (point clouds) from ordered visual data using three-dimensional (3D) reconstruction pipelines and then compare them with the as-planned four-dimensional (4D) building information model (BIM). However, ordered photo collection requires a strict capture plan and happens only after a specific time interval. Additionally, the reality models demand a considerable amount of processing time. As a result, the construction project status is often unreported between the intervals. The random photos captured daily by construction practitioners from different parts of the projects are helpful in filling this void. Localizing them in the 3D reality model helps to extract valuable information on time for effective construction monitoring. This study develops a system that localizes random photos in reality models using computer vision and deep learning–based approaches. Specifically, given a set of photos and a point cloud pair, a deep learning network for a six-degrees-of-freedom (6DOF) camera pose regression is trained to take any random photo within the point cloud region and estimate its position and orientation. The network performance is enhanced through data augmentation by another generative adversarial network: the pix2pix generative adversarial network (GAN). Finally, the poses are refined through traditional vision methods such as Perspective-n-Point (PnP) pose computation with random sample consensus (RANSAC). The proposed method was evaluated on a construction site. The system could localize random images captured during construction engineers’ daily work with as low as 0.04 m position error and 0.70° orientation error. In the end, this paper indicates further applications of construction image localization in the context of progress, quality, and safety monitoring.