Deep Learning–Based Inspection Data Mining and Derived Information Fusion for Enhanced Bridge Deterioration Assessment

Abstract

Inspection data are usually utilized to assess bridge situations for directing further maintenance and preservation. However, due to the complexity of inspection data, mining and fusing valuable information to assess bridge situations remains challenging. To address these issues, a novel inspection data analysis framework was proposed in this study. The framework integrated a gated recursive unit (GRU) model, a semantic segmentation (Seg) model, and a Yolo V4 object detector to analyze both time-series data and images. Seg and Yolo were used to detect defective pixels, which were then evaluated using refined fuzzy inference systems (RFISs) to determine the deterioration grade. The GRU and RFIS models were employed used to infer the probability of bridge deterioration grades. These probabilities were then fused by the novel fusion technique to determine the final deterioration grade. A verification showed GRU, Seg, and Yolo detectors to have 0.9299, 0.9580, and 0.7967 accuracy values for analyzing time-series data and images, respectively. RFISs also performed well in determining concrete and steel deterioration grades with R-values of 0.9968 and 0.9962. Compared with Dempster–Shafer and its two variants, the proposed fusion technique improved the accuracy rates by 11.65%, 2.19%, and 3.38%, respectively. Prototype models also demonstrated abilities to clearly understand deterioration grades and the spatial relationship of defects. Overall, the proposed method could sufficiently mine inspection data and more reasonably assess bridge situations. The practical application of this study lies in the fact that it presents a framework for thoroughly mining bridge inspection data, including time-series data and member surface images, to improve deterioration assessments. Combining the gated recurrent unit, you only look once (Yolo) V4 detector, convolutional semantic segmentation (Seg) model, refined fuzzy inference systems, and a novel information fusion technique, the framework provides a powerful solution for mining and integrating information to determine a reasonable deterioration grade, outperforming Dempster–Shafer and its variants. In addition, this study includes 3D prototype models of real bridges to showcase the deterioration situations of bridge components and help understand defect spatial relationships. In practice, once the inspection records are obtained, the programming code can automatically process them to determine the final deterioration grade and visualize the results in 3D mode. This is of great significance in ensuring the longevity, safety, and functionality of a bridge, because the inspection records are difficult to be processed manually over the long operation and maintenance period.