Importance of Testing with Independent Subjects and Contexts for Machine-Learning Models to Monitor Construction Workers’ Psychophysiological Responses

Abstract

Because workers’ abnormal psychophysiological responses (e.g., high levels of stress and fatigue) are directly or indirectly linked to disorders and accidents at construction sites, monitoring workers’ abnormal psychophysiological responses during ongoing work enables preventive interventions, thereby improving their health and safety. As such, wearable biosensors (e.g., wristbands) have been extensively applied with machine-learning models in construction fields as a means of continuous and less-invasive psychophysiological monitoring. However, there is a significant knowledge gap in how to validate machine-learning models that monitor human responses from biosignals. Specifically, despite the importance of generalizability across different people and contexts for psychophysiological monitoring tasks, current validation methods do not ensure different subjects and contexts between training and testing data sets, and thus overestimate the generalization performance of models. To address this issue, the authors propose a new independent subject and context testing method, leave-one-subject-and-context-out cross validation (LOSCOCV), which ensures that training and testing data sets are collected from different subjects and contexts. The proposed LOSCOCV method’s generalizability estimation performance was compared with current validation methods through conducting a test wherein machine-learning models were developed to detect construction workers’ stress levels from biosignals collected during their ongoing work. The proposed LOSCOCV method showed statistically lower errors in estimating machine-learning models’ generalizability than other benchmarks. The results indicate that LOSCOCV is more valid than current validation methods in assessing models’ generalizability for tasks that monitor human responses from biosignals. Accurately tracking generalization performance is fundamental to efforts toward advancing the generalizability of models. This study therefore significantly contributes to the field’s use of biosensors and machine learning to monitor construction workers’ psychophysiological responses—ultimately advancing their health and safety.