Multiclass Transportation Safety Hardware Asset Detection and Segmentation Based on Mask-RCNN with RoI Attention and IoMA-Merging

Abstract

Transportation assets, including retaining walls, noise barriers, rumble strips, guardrails, guardrail anchors, and central cable barriers, are important roadside safety hardware and geotechnical structures. They need to be inventoried accurately to support asset management and ensure roadway safety. There are detection methods for some of these transportation assets, which however lack flexibility and multiclass abilities. Moreover, the potential of neural networks has not been fully utilized although some detection methods have used deep learning. This paper for the first time proposes a multiclass transportation asset detection and pixel-wise segmentation model on two-dimensional images, based on region-based convolutional neural network (Mask-RCNN) with a feature pyramid network (FPN). The scale diversity and intensive continual appearance of transportation assets are identified as the main challenges, tending to result in numerous false-positive detections. A methodology with self-attention mechanisms based on the generic region of interest extractor (GRoIE) model along with the intersection over the minimum area merging (IoMA-Merging) postprocessing algorithm was then proposed. The evaluation outcomes demonstrated that our proposed methodology, including GRoIE-global context (GRoIE-GC) with IoMA-Merging achieves the best performance with a significant improvement over baseline. The precision increased by 10.0% on detection and 10.7% on segmentation. This proposed methodology will consequently improve the accuracy of asset inventory. This section serves as an illustration on how we could build a unified transportation inventory as a practical application based on the discrete detection results from our methodology. More details about the methodology can be found in the following sections. For simplicity, the demo in this section is only done with detected bounding boxes of a single object. After detection and segmentation, a tracking algorithm can play an important role in associating detection on the same object across different frames. In this way, we could recognize the distinct objects in the video frames with the help of a tracking algorithm. We also find it feasible to estimate the geographical and geometric information of the object of interest. With high-accuracy and high-frequency global positioning system (GPS) information recorded by our sensing vehicle, it is practical to locate the object and estimate the width in the physical world. Height information is also accessible with either camera calibration or combination of image and light detection and ranging (LiDAR) data.