Human Language-Instructed Robotic Excavation based on Behavior Trees

Abstract: Collaborative construction robots have emerged as a promising alternative to relieve construction workers from both physically and cognitively demanding tasks, contributing to a safer and more productive construction industry. However, communicating with robots is not a trivial task as human workers and robots speak different languages. From the human-centered perspective, allowing human workers to communicate with robots using natural language is desirable because it minimizes additional cognitive load to human workers. Existing studies, however, have been focusing on converting language instructions into sequential actions, leading to a rigid task plan and inability to handle complex situations and unstructured working environments. To address this critical limitation, this paper explores the use of behavior tree (BT), an alternative architecture for describing and controlling complex tasks like excavation. A behavior tree is a hierarchical tree structure that specifies the switching between the agent’s actions (i.e., execution nodes) via control flow nodes. Its modular nature allows the BT of excavation to be generated through linking reusable actions based on the human task descriptions. The resulting BT structure enables the robot to alter its behavior by selecting different tree branches in response to changing working conditions, thus improving its adaptability to dynamic construction environment and its capability of error-handling. In addition, the BT eases the human understanding of robot behavior for debugging and correcting robot behavior. A corresponding framework is proposed for enabling humans to guide a robotic excavator using goal-oriented language instructions. The framework consists of four modules: interpretation and reasoning, knowledge management, structural analysis and parsing, and BT generation. The interpretation and reasoning module decomposes instructions into structured executable intents. The knowledge management module organizes the knowledge for instruction reasoning, including the robot capable skills and its current working environment. Structure analysis and parsing module further grounds the intents and extracts associated parameters, while BT generation module maps the extracted elements with predefined BT nodes, building and refining the BTs of desired tasks. A case illustration is performed to demonstrate the viability of the proposed framework with executable demos. The findings are expected to facilitate efficient and transparent human-robot cooperation in earthmoving construction from a human friendly perspective.

Keywords: Human-robot collaboration, Human-robot communication, Natural language understanding, Robot task planning and control, Human-centered design, Behavior tree

DOI: 10.54941/ahfe1005651

Cite this paper:

​