LSTM-CNN Architecture for Construction Activity Recognition Using Optimal Positioning of Wearables

Abstract

Enhancing construction worker performance, safety, and project management through automated activity classification is a promising endeavor. By extracting activity-level information, this technology provides valuable insights for informed decision-making, facilitating project schedule adjustments, efficient resource management, and improved construction site control. Previous studies in this domain focused on basic activities, neglecting optimal sensor placement and no regard for worker comfort. This paper extends beyond existing research, encompassing a broader range of complex construction activities and surpassing current methods. Utilizing unobtrusive wearables like a smartwatch and smartphone, the study determines optimal sensor positions (dominant/nondominant wrist, dominant/nondominant leg pocket). Notably, it introduces a novel deep neural network structure, merging long short-term memory (LSTM) and convolutional layers, offering an innovative solution for automated activity classification tasks in the construction industry. This model extracts activity features automatically reducing the need for manual feature engineering and performs classification with few model parameters indicating efficiency in terms of computational resources and memory requirements making the model more suitable for real-time applications and deployment on resource-constrained devices. By leveraging the strengths of both convolutional layers and LSTM, this approach offers a powerful and efficient solution for activity classification tasks. An experimental study was carried out to recognize four different activities: manual excavation, rebar stirrups, cement plastering, and bar binding. These were performed by four subjects (three males and one female) for 30 s each with different positions of smartwatch and smartphone producing 24,080 data points. Results indicate the optimal positioning of wearables to be smartwatch on dominant hand and smartphone on opposite leg pocket because of a balanced and effective coverage of the relevant movements and contextual information yielding 98.18% accuracy, 98.20% precision, 98.17% recall, and F1 score of 98.17%. The integration of LSTM and convolutional neural network (CNN) architecture and strategic sensor placement on wearables holds promise for revolutionizing construction project management. By optimizing sensor placement, we can improve the accuracy of activity recognition, allowing for better tracking of worker productivity and process optimization, ultimately leading to increased efficiency and productivity on construction sites. Additionally, strategically placing sensors on wearables enables real-time monitoring of worker movements, facilitating the early identification of safety hazards and ergonomic issues, thereby enhancing worker safety and reducing workplace injuries. Furthermore, integrating wearable technology with smart construction technologies enables real-time data exchange and analysis, promoting collaboration among stakeholders and enhancing progress monitoring and control. Leveraging data from strategically positioned wearables allows for comprehensive visualization of construction processes, aiding in the detection of project bottlenecks and streamlining project management strategies. Lastly, analyzing work allocation data from optimally positioned wearables improves project bidding accuracy, minimizing cost estimation errors and increasing profit margins. These findings highlight the transformative potential of LSTM-CNN architecture and optimal wearable placement in construction project management.