EmoMasque: A Virtual Jewelry Emotional Regulation System Based on Facial Emotion Recognition and Multisensory Feedback

Abstract

In the current era of human-centered design, emotional design has become a crucial paradigm for enhancing user experience. A common issue in emotional design today is the singularity of emotional regulation methods, as users are unable to directly and accurately perceive their emotions or actively participate in the process of transforming their emotional states. This paper proposes EmoMasque, an interactive virtual jewelry system designed to address this gap. The system transforms abstract emotions into perceivable and adjustable visual forms, guiding users in improving their emotions autonomously. EmoMasque integrates three core modules: a real-time facial emotion recognition module based on Plutchik’s Wheel of Emotions, which captures eight basic emotions and maps them to a dynamically color-changing virtual mask; a multisensory interaction module, allowing users to select specific facial areas such as the forehead, eyes, nose, mouth, and ears to generate corresponding visual stimuli for targeted emotional regulation; and an AI-driven dialogue agent that offers personalized healing suggestions based on sensory therapy theories. the system visualizes emotional data, assisting users in actively participating in emotional health management. The system also incorporates a long-term user follow-up mechanism and an AI iterative optimization mechanism to enhance the accuracy and adaptability of the suggestions. Preliminary user tests have shown that this system significantly improves users' ability to perceive emotions and their positive emotional experiences. This research provides a practical and evidence-based framework for the intersection of emotional design and pleasurable design.

Keywords: Emotional Design, Pleasurable Design, Facial Emotion Recognition, Virtual Jewelry, Multisensory Regulation, Emotional Transcription

DOI: 10.54941/ahfe1007351