Facilitating Collaboration Between Humans and Unmanned Heavy Vehicles using Verbal Interaction and Augmented Reality

Abstract: The development of unmanned self-driving vehicles appears to have great potential to increase safety and transport productivity. In industry, these vehicles are expected to be adopted first in confined contexts (hubs), such as mines. However, there is a need to better understand how to design interaction models to handle complex situations when professional operators interact with such vehicles in collaborative tasks. Research has suggested that verbal communication may facilitate interaction between automated vehicles and humans. Augmented reality (AR) has also been investigated to support operators in various work domains. Overall, research has indicated that these emerging technologies can improve operators’ productivity and safety.ObjectivesThe main study aim was to investigate the potential of verbal interaction and AR to support the user experience and facilitate collaboration between operators and unmanned self-driving heavy vehicles in hub environments. The second aim was to gain better insights into the effects of more social and natural verbal human–machine interaction. The study focused on loading scenarios in two hub types: logistic centers and underground mines.MethodsThe study had 32 participants: 16 forklift operators and 16 rock-loading operators. Two variants of concepts were designed for each hub type. The natural voice interaction (NVI) variant was designed to mimic natural and social verbal interaction between two human operators. In the basic voice interaction (BVI) variant, verbal interaction was kept to a minimum, and the operator controlled the vehicle with short verbal commands. AR was used in both variants to provide visual guidance, for instance how to distribute the load in the vehicle. The concepts were implemented in animated movies in which the operator in the movie collaborated with the unmanned vehicle to prepare and conduct a loading task. The study was conducted using online interviews in which the participants watched the movies. Subjective opinions related to acceptance, work efficiency, and other aspects of the user experience were assessed using rating scales and open-ended questions.Results and discussionOverall, the concepts generated high scores of user expereince and work efficiency in both user contexts. For instance, on a scale of 1 (inefficient) to 7 (efficient), forklift operators’ mean scores were 6.4 for the NVI and 6.4 for the BVI. The results support the use of verbal interaction and AR to facilitate collaboration between human operators and vehicles. The proposed concepts also provide promising examples of interaction models for further investigation and implementation. Interestingly, most forklift operators stated that they trusted the vehicle more in the NVI because the interaction felt, for instance, more human-like and safe. These results indicate the potential to include more natural dialogues when developing future verbal interfaces for human–vehicle collaboration. However, no difference in trust between concept variants was found among the rock loaders. Also, more rock loaders preferred the BVI, and several subjects highlighted that the extensive use of verbal interaction could become a disturbance. These results support the importance of adapting interaction design in future implementations to the needs of specific users to gain high acceptance and productivity.

Keywords: Interaction design, artificial intelligence, verbal interaction, augmented reality, autonomous vehicles

DOI: 10.54941/ahfe1002310

Cite this paper:

​