Enhancing Communication Transparency and Teaming in Human-Autonomy Systems: Integrating Large Language Models for On-Site Construction Operations

Abstract: The integration of autonomous systems in the construction industry is rapidly advancing, driven by the need to enhance efficiency, safety, and productivity on-site. With autonomous agents evolving from mere tools to collaborative teammates, there is a critical need to understand the dynamics of human-autonomy teaming (HAT) in complex construction environments. This research addresses the gap in knowledge surrounding the communication and collaboration between human workers and autonomous agents, specifically within the context of tower crane operations, where effective teamwork is crucial for safety and task success. Despite the growing implementation of autonomous systems, research on human-autonomy teaming in construction remains limited, particularly regarding the interdependencies of key attributes like team composition, communication transparency, interaction dynamics, trust, and overall team performance. Current studies on autonomous crane operations predominantly focus on the technical aspects, such as smart sensing, perception, and motion control, with little attention given to the team-based human factors involved in these operations. The lack of comprehensive studies on how various human psychological and behavioral factors interact with autonomous systems during real-time operations presents a significant challenge, which this research aims to address. This paper proposes and evaluates a novel VR-based system architecture designed to enhance understanding of the impact of communication transparency on human-autonomy teamwork in construction settings. In addition, the integration of a Large Language Model (LLM)-enabled autonomous agent adds a critical layer to the HAT dynamic by facilitating real-time communication and decision-making. The LLM processes inputs such as sensor data, crane motion sequences, and operator commands to provide actionable insights and hazard warnings, further improving transparency and reducing the cognitive load on the operator. The agent’s ability to generate predictive models also assists with obstacle detection and motion planning, enhancing the overall safety and efficiency of crane operations. Organized into a three-layered HAT framework—(1) Team Goals (Bottom Layer), (2) Team Synchronization (Middle Layer), and (3) Team Outcome (Top Layer)—the VR environment simulates a construction job site where a semi-autonomous tower crane operates alongside a human worker. The system is designed to assess how varying levels of communication transparency, based on the situational awareness transparency model, affect the cognitive states, trust levels, and task performance of human operators during dynamic role allocation. This research contributes to both academic and industrial communities by offering a foundational architecture for studying human-autonomy teaming in construction, while the VR-based system serves as a powerful tool for future experimental research. For industry practitioners, the study provides insights into designing more effective and transparent communication systems that enhance safety and efficiency in autonomous crane operations, ultimately contributing to safer and more reliable construction practices.

Keywords: Human-autonomy teaming, tower crane, transparency, on-site construction, large-language model

DOI: 10.54941/ahfe1005749

Cite this paper:

​