Large-Scale Loop Detector Troubleshooting Using Clustering and Association Rule Mining

Abstract

The archived data from traffic sensors are used in a wide range of traffic management applications. However, missing or invalid data are becoming an important concern. This study proposes a systematic approach to identify and characterize data error patterns to facilitate large-scale loop detector troubleshooting. Data were collected from loop detectors in Phoenix. A set of quality control criteria was applied on daily 20-s data to find the error percentage for each loop detector. A fuzzy c-means clustering method was implemented on the data quality check results and preliminary clusters were identified. To discover the most frequent rules within the clusters, an association rule mining method was applied to the clusters’ data subsets. Loop detector stations with different error patterns were visited in the field to verify the clustering and association rule mining results, identify potential causes, and recommend appropriate solutions. The analysis identified four key patterns, indicating that the proposed approach successfully found the relationships in the data errors. The findings of this study help traffic engineers to more easily diagnose and troubleshoot large-scale loop detector errors.