Comparison of Asphalt Pavement Crack Segmentation Based on Different Fusion Methods of RGB Images and Thermal Images

Abstract

Pavement maintenance has become a critical priority in recent years. There has been a growing focus in research on advancing image-based pavement crack monitoring tools that utilize deep learning models to automate the detection of damage in civil infrastructure. Accurate automatic damage detection using deep learning models requires a comprehensive and extensive data source that can effectively capture anomalies in the photos. Nevertheless, these tools primarily rely on RGB/thermal images, which perform effectively in optimal lighting conditions but may experience diminished performance in challenging environments. For example, these RGB-based methods often struggle in challenging scenarios with low contrast, cluttered backgrounds, poor lighting, fog, or smoke obstruction inherent limitations of thermal images, such as edge blurring, low contrast, and local unevenness, can hinder the accuracy and robustness of some pavement crack detection methods. To improve crack detection accuracy, this research proposes a method based on RGB and thermal image fusion strategies such as early, intermediate, and late fusion. The comparative analysis demonstrated that the intermediate RGB-thermal fusion technique exhibited the highest performance, achieving F1 scores and mean intersection over union (MIoU) of 96.26% and 93.00%, followed by early fusion (F1: 95.36%, MIoU: 91.45%). The three fusion methods showed notably enhanced performance compared to segmentation models based on a single type, with the early and intermediate fusion methods demonstrating greater stability. The RGB-thermal fusions not only achieved a higher detection rate for damage but also excelled in distinguishing between different types of damage. It is evident that the integration of multimodal RGB-thermal fusion technologies significantly enhances the accuracy of asphalt pavement crack segmentation.