Multi-Sensory Integration and Emotional Responses: The Impact of Materials on Perception and Emotion

Abstract: With the rapid advancement of virtual reality (VR) technology, multi-sensory integration has become a significant area of focus in the emotional design of virtual environments. Materials play an important role in shaping both perception and emotion, while tactile stimuli are particularly influential in cognitive and emotional responses. Additionally, research indicates that there is an integration between vision, hearing, and touch. Studies have demonstrated that combining various sensory inputs, particularly tactile, visual, and auditory stimuli, can enhance cross-sensory integration. While research has examined the individual effects of tactile, visual, and auditory stimuli on perception, less attention has been paid to how these senses integrate and combine, especially within virtual reality. The study explores the impact of materials on cognition and emotion, as well as the integration between tactile, visual, and auditory inputs. An experiment was conducted in the virtual reality world "The Library of Solitude by the Sea," with thirty-two participants interacting with three materials. The research focused on the integration of tactile, visual, and auditory stimuli by stimulating all three senses simultaneously. Emotional responses were measured through questionnaires and skin conductance responses (GSR). The findings revealed that different materials influenced participants’ emotions and perceptions, with the multi-sensory group reporting greater levels of pleasure and disgust than the single-sensory group. This suggests that various materials evoke different emotional responses and that the combination of tactile, visual, and auditory stimuli leads to a more intense emotional response. The use of multiple sensory stimuli is essential for enhancing immersion and emotional consistency in virtual environments. The study fills a gap in the literature on how multi-sensory integration influences emotional responses. It demonstrates that combining tactile, visual, and auditory stimuli produces a stronger emotional response than the use of a single sensory input. The study also highlights how different materials affect emotional responses. The results indicate that multi-sensory integration enhances emotional engagement and depth. This research offers valuable insights into the emotional design of virtual environments, particularly in the development of immersive experiences through multi-sensory integration. Furthermore, it provides a foundation for future research on multi-sensory design and contributes to the continued evolution of emotionally immersive virtual worlds.

Keywords: Virtual Reality, Multi-sensory Integration, Emotional Design

DOI: 10.54941/ahfe1006024

Cite this paper:

​