Design and Usability Evaluation of an Annotated Video–Based Learning Environment for Construction Engineering Education

Abstract

Advancement in video technology has made it possible for instructors to provide students with applied knowledge of construction practice. While videos can stimulate students’ interest in the construction domain by providing opportunities to observe real-life construction work, videos can sometimes contain extraneous information that may distract learners from essential learning contents. Computer vision techniques can be utilized to detect and direct learners’ focus to important learning concepts in videos. This study investigated the design and usability evaluation of an annotated video-based learning environment designed to direct learners’ attention to significant learning contents. Faster R-CNN with VGG16 backbone was trained with 21,595 images to detect practice concepts within videos. A Visual Translational Embedding Network was trained with the object detector and 8,004 images to predict interactions between subjects and objects of the practice concepts. The object detection model could detect all instances of subjects and objects, making the model sufficient for interaction detection. Usability evaluation was conducted using questionnaires, verbal feedback, and eye-tracking data. Results of the usability evaluation revealed that cues, such as bounding boxes, texts, and color highlights, drew learners’ attention to the practice concepts. However, students allocated more attention to the signaled images than the texts. The study contributes to dual-coding theory and the cognitive theory of multimedia learning through the use of cues to select, organize, and direct learners’ attention to noteworthy information within videos. This study also provides insights into the key features of cues that can facilitate learning of construction practice concepts with videos.