Generalized model for driver activity recognition in automated vehicles using pressure sensor array

Abstract: One of the key aspects for an efficient cooperation between human driver and automated vehicle lies in the accurate interpretation of the driver state by the automated system. Flawless driver monitoring and consequently successful driver-vehicle interaction can increase safety of the traffic in the future when automated agents are one of the involved road users. Driver activity recognition is an important component of driver monitoring, as drivers in automated vehicles drivers are allowed to engage in driving irrelevant activities. Detecting these activities during driver monitoring can improve assessment of automated system about driver’s readiness to react in critical driving situation. However, the confined space hinders the in-vehicle activity detection by sensors such as cameras, which require a complete overview of the driver’s body movements within the frames. On the other hand, utilizing other sensors such as accelerometers, placed on the driver’s body is obtrusive and undesirable in the driving context. In this contribution, two pressure sensor mats are used as sensors that are placed on the seat and the back of the driver seat. This type of sensor is non-intrusive and can be easily applied in vehicles. To gather the necessary data for training the models, an experiment is conducted using a static driving simulator whose cockpit layout is comparable to that of a real vehicle. The experiment is executed with eight sparsely selected participants based on the fractional factorial criteria. During the designed scenario, several activities are expected from the participants, either directly through the given instructions or indirectly through the arranged driving situation. A total of 20 activities are selected for the classification task based on the result of a previously carried out survey on the non-driving related activities that are most demanded by drivers in an automated vehicle. To model the driver activity, three neural networks from the RNN family are chosen, namely LSTM, stacked LSTM, and CNN-LSTM. Since the data obtained from the activities are time series, the criterion for selecting the networks is their capability to handle the temporal aspect of the data. Another emphasis in training the networks is to create a generalized model that can deal with the data from all drivers, rather than creating an individualized model for each driver.The results show that the pressure distribution from seat and back of drivers provides valuable information about the current activity of the driver. As expected, individual models achieve higher accuracy than generalized models built on data from all drivers. However, all generalized algorithms are able to recognize the selected activities with more than 70% accuracy. The generated models can be employed at lower automation levels to estimate the engagement of drivers in driving task, as well as at higher automation levels to predict readiness of drivers for potential takeover situations. In addition, accurate estimation of driver state helps the automated system to increase the comfort and improve driver state. Fusion of the seat pressure distribution and data from other unobtrusive in-vehicle sensors, in the next step, can further increase the accuracy of the models.

Keywords: Driving simulator, Driver state, NDRT, Time series classification

DOI: 10.54941/ahfe1002733

Cite this paper:

​