LaserDex: Improvising Spatial Tasks Using Deictic Gestures and Laser Pointing for Human–Robot Collaboration in Construction

Abstract

In the context of the unstructured and fast-changing construction environment, the adaptability of robots to human improvisation is crucial. However, existing construction robots are limited in understanding spatial goals communicated spontaneously in the workplace and require substantial human-user training. Incorporating deictic gestures into human–robot interfaces holds promise for enhancing the intuitive operation of construction robots and for on-site collaboration. However, applying deictic gestures in precision-demanding construction tasks presents challenges because in a large-scale work environment, humans have limited accuracy in pointing at objects at a distance. This study introduces LaserDex, a novel interface for distant spatial tasking based on a global-to-local identification approach, in which the human first guides the robot toward the general area of the task using a deictic gesture and then indicates the precise area by dynamically using a laser pointer. Our user study with 11 participants demonstrated that LaserDex can achieve an intersection over union (IoU) of 0.830 when outlining a rectangular drywall opening (compared with an IoU of 0.514 for the baseline, a handheld controller), and an 11.4-mm distance error of the center of the estimated rectangle compared with the center of the targeted rectangle. The findings of this study underscore the potential of LaserDex to seamlessly combine intuitive human–robot interaction with precise robot operations.