Timbre estimation of compound tones from an auditory cortex by deep learning using fMRI: Sound pressure levels detection of specific frequency

Abstract

Brain decoding have been widely treated in the neuroscience. However, compared to research in the visual cortex, progress in the field of auditory cortex has not been made. Therefore, the purpose of this study is to establish a technique to estimate sounds heard by human using deep learning from brain images captured by fMRI. The sounds we hear in usual have a unique timbre. Timbre is determined by the combination of sound pressure levels at the overtone, which is the natural multiple of the fundamental frequency, in a compound tone. Before, this research group decoded the pitch of pure tones, which are waves of a single frequency. As a result, the discrimination of two tones in increasing degrees and the detection of a specific pitch in triad were realized. Next phase of this research is to decode a sound pressure level at a specific frequency. By combining these methods, we believe it is possible to decode timbre by detecting a sound pressure level of specific overtone. In a previous report, we examined whether the brain activity of listening to pure tones at two different sound pressure levels at specific frequency can be discriminated by deep learning binary classification. The result was a discrimination rate of 70.84% with relative levels of 0 [dB] and -20 [dB] when 90 [dB] was used as a reference. This result indicates that the difference in brain activation intensity by sound pressure level could be handled as classification problem by deep learning. Therefore, the purpose of this paper is to detect the sound pressure level of pure tones using deep learning for application to timbre decoding. Specifically, we attempt to detect specific sound pressure level among the three tones of 0 [dB], -10 [dB], and -20 [dB] when 90 [dB] based on an absolute level of 90[dB]

Keywords: Decode, Sound pressure level, Auditory cortex, Deep learning, fMRI

DOI: 10.54941/ahfe1005705