Automated Bridge Inspection Image Retrieval Based on Deep Similarity Learning and GPS

Abstract

The inspection of highway bridge structures in the US is a task critical to the national transportation system. Inspection images contain abundant visual information that can be exploited to streamline bridge assessment and management tasks. However, historical inspection images often go unused in subsequent assessments because they tend to be disorganized and unlabeled. Further, due to the lack of global positioning system (GPS) metadata and visual ambiguity, it is often difficult for other inspectors to identify the location on the bridge where past images were taken. Although many approaches are being considered toward fully automated or semiautomated methods for bridge inspection, there are research opportunities to develop practical tools for inspectors to make use of those images already in a database. In this study, a deep learning–based image similarity technique is developed and combined with image geolocation data to localize and retrieve historical inspection images based on a current query image. A Siamese convolutional neural network (SCNN) is trained and validated on a gathered data set of over 1,000 real world bridge deck images collected by the Indiana Department of Transportation. A composite similarity (CS) metric is created for effective image ranking, and the overall method is validated on a subset of eight bridge’s images. The results show promise for implementation into existing databases and for other similar structural inspections, showing up to an 11-fold improvement in successful image retrieval when compared with random image selection.