The Effects of Multi-modal Takeover Request on Distracted Drivers’ Takeover Performance and Perception

Abstract

Visual and cognitive distractions caused by non-driving related tasks (NDRT) engagement and over-trust were exposed and considered a serious driving safety hazard for the automated driving system (ADS). However, the typical HMI design of takeover request (TOR) in the market (displaying a symbol in the digital cluster with an auditory warning) can be easily unnoticed, especially in auditory conflicts scenarios, and should be optimized urgently. This study introduced four multi-modality combinations of TOR. It examined the differences in drivers’ reaction time, reaction quality, and subjective perception in situations where the drivers were experiencing visual and cognitive distractions. Eighteen drivers participated in the driving-simulator experiment. Results showed that the symbol and speech combination was difficult to notice, yet it was the easiest to understand. Vibration and speech combination could effectively alert the distracted drivers within the shortest time, but there was negative experience feedback from the drivers. Ambient light and speech combination could guide drivers but was inconvenient in time-critical situations. Vibration, light, and speech combination has shown a better performance and user experience in general. Our results provide multi-modal TOR design implications that improve takeover performances and user experiences.

Keywords: Automated driving, Multi-modality, Cognitive load, Takeover

DOI: 10.54941/ahfe1002430