Structural Material Condition Assessment through Human-in-the-Loop Incremental Semisupervised Learning from Hyperspectral Images

Abstract

Engineering materials in constructed systems in service exhibit complex patterns, including structural damage, environmental artifacts, and artificial anomalies. In recent years, machine vision methods have been extensively studied, most of which train models using regular grey or color images in the visible bands and label at pixel levels with a large volume of data. The authors propose using hyperspectral imaging (HSI) for structural material condition assessment in this work. Compared with visible images, the research challenge is that HSI pixels with high-dimensional spectral profiles are beyond human perceptive capabilities with hidden discriminative power. Learning from labeled and unlabeled data is one direct approach to unlocking this power. A deep neural network-enabled spatial-spectral feature extraction and a semisupervised learning architecture were developed in this work. A human-in-the-loop (HITL) framework was comparatively studied with three incremental training-data configuration schemes. The paper concludes with the following empirical findings: (1) fully supervised learning determines the baseline of the detection performance; (2) an extensive range of ratio values exists between the unlabeled and the labeled data for incremental semisupervised learning, and a 1∶1 ratio can be taken as a conservative and operational ratio; and (3) with parametric semisupervised learning with equal labeled and unlabeled data participation, the proposed HITL operational workflow can be implemented as a practical approach for HSI-based structural material and damage detection.