High-Fidelity Simulation and Digital Twin Framework for Real-Time Adaptive Human-Robot Collaboration

Abstract

Effective real-time adaptation in human–robot collaboration requires significant testing and validation to avoid compromising human safety and improve task efficiency. Developing and validating such systems in real-world industrial environments is resource-intensive. This paper presents a high-fidelity simulation and visualisation platform for shared robotic workspaces that supports both synthetic data generation and digital twin operation in Unity 3D. The platform models and visualises human body movement and gaze behaviour, enabling the representation of body movement and visual attention during collaborative manufacturing tasks. Using procedurally generated scenarios, the system supports controlled and repeatable experimentation for the development and evaluation of real-time adaptation strategies. When integrated with real sensor and eye-tracking data, the platform operates as a digital twin, allowing real-time mirroring, monitoring, and analysis of human–robot interactions under operational conditions. By enabling both offline simulation and online system integration, the platform facilitates rapid prototyping and systematic assessment of real-time adaptation approaches, helping bridge the gap between theoretical methods and practical deployment in intelligent human–robot collaborative systems.

Keywords: Human-robot collaboration, Simulation, Digital twin

DOI: 10.54941/ahfe1007153