Machine learning agent to recommend the best modality for takeover during conditionally automated driving

Abstract: >ContextConditionally autonomous vehicles are getting more and more visibility worldwide, but they bring many challenges. In particular, the transition of responsibility for the driving task from the car to the driver can be a dangerous situation, especially if the driver is absorbed in a non-driving-related task. In these cases, the car issues a takeover request (TOR), using different available modalities (such as auditory or visual). Research showed that the choice of modalities affects the takeover quality and should be adapted depending on the situation. >ContributionIn this study, we propose a smart agent recommending to the vehicle the best choice of modality for a TOR. This agent considers the driver physiological state based on the last 90 seconds prior a TOR, the external environment and three possible multimodal TOR. >Methodology>>ExperimentA 50 minutes driving session took place on a fixed-base driving simulator, where 15 drivers’ physiological signals (EDA, ECG and respiration) were recorded. There were 9 takeover situations during the driving session, caused by a fixed obstacle with a time-to-collision of around 7 seconds. The drivers had to perform three different non-driving-related tasks under two different weather conditions.The possible TOR modalities were combinations of haptic (vibrating seat), auditory (short beep sound) and visual (logo on the dashboard). Combinations tested were haptic-visual, auditory-visual, and haptic-auditory-visual. The takeover quality metrics were the reaction time between the TOR and the takeover, and the maximum steering wheel angle attained during the takeover process. >>Machine LearningTakeover quality metrics were aggregated to create a unique label to predict. Features were processed from raw physiological signals, the external environment and feature-selection techniques were applied to keep only the most relevant ones. After outlier suppression and data processing, 80 TOR were kept for the Machine Learning models training. Data Augmentation methods, such as SMOGN and Random Noise were implemented and tested to boost the training dataset.KNeighbors Regressor, Support Vector Regressor, Random Forest Regressor and Neural Networks were trained using a grid search approach and cross validation. >>AgentThe agent predicts the takeover quality using the ML model, with every possible set of modalities. It then chooses the modalities providing the best theoretical takeover quality. The agent was evaluated to avoid the following trivial situations: 1.The agent always chooses the same modalities, meaning that the training was biased toward one set of modalities.2.The agent chooses a random modality because the choice has no impact on predicted performance, meaning that the agent was not able to capture the modalities impact on takeover quality.A novel metric representing modality impact on takeover quality was defined and used to optimize agent behavior. >ResultsThe agent was able to predict takeover quality using a Support Vector Regressor with a MSE of 0.0685, beating the baseline (mean) by 39.65%. It was also able to suggest the 3 different set of modalities in the final phase, with a distribution of (0.625, 0.25 and 0.125). Modality impact on takeover quality had a mean score of 4.95% (std: 2.7%).

Keywords: AI, Takeover requests, Machine Learning, HMI

DOI: 10.54941/ahfe100845

Cite this paper:

​