A Novel Method of Subjective Sound and Audio Playback User Experience Evaluation in Multitasking Context: Two Case Studies

Abstract: Today's telecommunication devices are rarely used by comfortably seated users who are not performing any other task in parallel. Typical communication scenarios include walking, driving a car, watching TV, working on a PC, etc., during a conversation. However, audio User Experience has traditionally been evaluated in ideal laboratory conditions. The authors of this contribution have prepared a new set of methods for subjective audio and sound quality testing with a parallel task which have been approved as ETSI TR 103 503. The contribution summarizes the most widely-used speech and audio quality of experience testing methods, discusses some of their disadvantages, and introduces a new testing methodology with a parallel task. One possible way to bring subjective tests closer to reality while keeping them in a controlled laboratory environment is to introduce a parallel task. The tests can be then based on standard recommendations but the test subjects are asked to perform a secondary physically and/or mentally challenging task when listening to and evaluating the samples. The parallel task technique is designed to distract subjects from their primary task in a similar way that various activities are distracting in real-life situations.Physically oriented parallel tasks are based on various physical activities (e.g., cycling, running, etc.). A disadvantage of this type of parallel task is that not all subjects are in the same physical condition. Mentally oriented parallel task is based on mental activities (math, memorizing tasks, etc.). Similarly, as in the previous case, subjects with better mathematical skills can deal with the secondary task much more easily than less skilled subjects. Hybrid parallel tasks are designed to include both physical parts and mental parts so that the subjects experience a similar workload irrespective of their individual abilities during the testing process. Therefore, with the exception of specific (professionally oriented) situations, hybrid tasks can be considered more appropriate for general-purpose testing than for physically-only or mentally-only oriented tasks. This contribution also presents two experiments performed in the context of parallel task scenarios and highlight sthe identified differences in human perception in these scenarios. The first presented experiment shows the results of subjective quality testing performed on audio data using the car audio system. The quality of the stimuli was subjectively assessed while driving a (simulated) car as the parallel task. The results show decreased sensitivity of subjects for samples with significantly compromised quality while keeping excellent sensitivity for higher quality samples. The second presented experiment was based on ITU-T P.835. The principle of this standard is to make subjects listen to the same sample 3 times: first to assess the speech quality, then to assess the noise annoyance, and finally to assess the overall sample quality. During the parallel task testing, an additional psycho-motor paralle task (aimed shooting towards randomly moving target on a professional laser shooting simulator) has been introduced. Also in this experiment, a non-linear subjective results change after the introduction of a parallel task has been identified.

Keywords: Qoe, Parallel Task, Psychomotor Task, Subjective Testing

DOI: 10.54941/ahfe100890

Cite this paper:

​