C2F-RMD: Automated Road Manhole Detection and Condition Assessment

Abstract

Road manhole detection and damage assessment are crucial for ensuring the safety and efficiency of transportation systems. Traditional methods, reliant on costly and inaccessible three-dimensional (3D) cameras, pose challenges, especially in resource-limited settings. This study introduces C2F-RMD, a groundbreaking deep learning (DL)–based algorithm that revolutionizes road manhole detection and damage assessment using only two-dimensional (2D) images. C2F-RMD adopts a two-stage approach. In the first stage, the coarse-to-fine (C2F) detection technique, coupled with scale-adaptive region-based convolutional neural network (R-CNN), accurately detects and classifies road manholes. Achieving an impressive F1 score of 0.96 and an intersection over union of 0.95 across eight classes, the C2F model provides robust results. The second stage, road manhole damage (RMD) index, employs self-crack segmentation and an elevation prediction model. The self-crack segmentation, trained without labeled data, attains remarkable accuracy rates: 0.821 for precision, 0.805 for recall, and 0.813 for F1 score. The innovative elevation prediction model forecasts manhole surroundings’ elevation maps using solely 2D image input, with a regression score (R2) of 0.77 and a mean absolute error (MAE) of 4.35 mm. Notably, this method was successfully applied to an 802-km road network in Seoul City, encompassing various road types, including urban, principal, and supplementary roads, as well as expressways. It accurately detected and classified eight types of manholes with an accuracy rate of 0.98. Additionally, the method achieved accuracy rates of 0.80 for crack segmentation, 0.88 for crack segmentation grading, and 0.83 for elevation difference grading in manhole condition evaluation, demonstrating its adaptability in detecting, classifying, and evaluating manholes across diverse road types. This promising approach has the potential to replace traditional manual visual assessments of road manholes. The C2F-RMD approach is a cutting-edge solution for identifying and assessing road manhole conditions, which has significant implications for transportation infrastructure management. This algorithm uses advanced DL techniques to accurately detect and categorize manholes in standard 2D images, eliminating the need for costly 3D cameras. This innovation streamlines the assessment process, saving time and resources, especially in resource-limited environments. Furthermore, the self-crack segmentation model and elevation prediction capabilities provide valuable insights into manhole deterioration, enabling proactive maintenance and improved safety measures. This method has demonstrated success across 802 km of Seoul’s roads in South Korea, improving accuracy in manhole detection (0.98), crack segmentation surrounding manholes (0.88), and grading elevation differences between manhole covers and pavement surface (0.83). These accomplishments highlight its adaptability to a variety of road networks. In conclusion, the C2F-RMD approach is an effective and efficient solution for automating the detection and assessment of manhole conditions, with the potential to revolutionize infrastructure management practices worldwide.