Evaluation of different HMIs to improve operator hazard perception and user experience in teleoperation of automated shuttles

Abstract

Autonomous vehicle technologies are rapidly evolving. In challenging situations that exceed the vehicle's autonomous capabilities, the teleoperation of vehicles enables human operators to remotely control and maneuver a vehicle. Operating remotely is not a trivial task, especially due to the lack of feedback caused by physical decoupling from the vehicle, the reduced quality of information of the vehicle environment, and the latency caused by the data transfer. We aim to explore if a human operator can be supported in performing the task by enhancing visual cues, acting as a proactive alert system using Augmented Reality (AR) overlays. We studied three types of hazards (behavioural precursor, environmental precursor, and sudden hazard appearance) and three Human-Machine-Interfaces (no support, bounding boxes, and bounding boxes with warning sign) in 3x3 within-subjects design. We analysed the perceived criticality of the hazard, and perceived difficulty when performing the task. The study was conducted with N = 37 participants using 15 short, real-world videos. Perceived criticality ratings did not significantly differ between various HMI concepts (p = .610). However, there was a significant difference in the evaluation of perceived difficulty between no support and bounding box (p = .001) and no support and bounding box with warning (p = .001). Participants perceived the scenario as easier when displaying bounding boxes for hazards with sudden appearances (p = .004) and behavioural predictions (p = .021), and the inclusion of warning signs for sudden hazards (p = .002) further decreased the difficulty. These results can be incorporated to refine the HMI design of teleoperation cockpits and to facilitate safety and operator experience in teleoperation. Future research should investigate if the timing or data representation has effect on the operator experience and performance.

Keywords: Teleoperation, Autonomous Driving, Autonomous Shuttle, Hazard perception, Augmented reality, User experience

DOI: 10.54941/ahfe1005206