Automated Anomaly Detection and Localization in Sewer Inspection Videos Using Proportional Data Modeling and Deep Learning–Based Text Recognition

Abstract

In recent decades, closed circuit television (CCTV) has been the most used tool for visually inspecting the internal condition of pipelines. However, CCTV inspection requires long videos to be observed and analyzed by certified inspectors, which is time-consuming, labor-intensive, and error-prone. This paper proposes a novel approach for automated anomaly detection and localization in sewer CCTV inspection videos. The developed algorithms employ three-dimensional (3D) Scale Invariant Feature Transform (SIFT) to extract spatio-temporal features in sewer CCTV videos. Anomaly detection is performed using a one-class support vector machine (OC-SVM) trained by frames without defects to model states considered normal and to classify outliers to this model as anomalous frames. Then, the identified anomalous frames are located by recognizing included text information in them using an end-to-end text recognition approach. The proposed localization approach is divided into two main steps: text detection using maximally stable extremal regions (MSER) algorithm and text recognition using a deep convolutional neural network (CNN). Extracting and localizing the suspicious frames out of these videos for further analysis can reduce the time and cost of detection because thousands of normal frames would be detached in the inspection process. The proposed model performance showed acceptable viability, because the testing accuracy was 92.3% in anomaly detection and 86.6% for frame localization in sewer inspection video frames.