Real-time process-level digital twin for collaborative human-robot constructionwork

Widespread use of autonomous robots in on-site construction has been limited because it is impractical to preprogram robots to perform quasi-repetitive tasks due to the relatively loose work tolerances and deviations of as-built work from the project design. Robotization of field construction work must thus be conceived as a collaborative human-robot endeavor capable of planning and improvising during the performance of construction tasks. Although humans can control robot motion through teleoperation, it is often impractical due to the range of a robot's motion and associated safety issues arising from heavy or large construction materials. An intuitive and safe bi-directional interface is thus needed to enable construction robots to seamlessly interact with and partner with human co-workers. This paper proposes a framework that allows human-robot interaction and collaboration within a real-time, process-level, immersive virtual reality (VR) digital twin that is created by combining the as-designed BIM model and the evolving as-built workspace geometry obtained from on-site sensors. Humans can use the digital twin to remotely demonstrate a task plan to the robot. The robot understands the communicated objectives and plans its motion to complete the task, which is communicated back through the system for human evaluation and approval before the robot executes the task. A case study involving imperfect rough carpentry (i.e., stud framing) and a 6DOF KUKA drywall-installing robot arm is conducted to demonstrate and evaluate the digital twin system.