Affective and Pleasurable Design

AHFE International

Accelerating Open Access Science in Human Factors Engineering and Human-Centered Computing

Affective and Pleasurable Design

book-cover

Editors: Shuichi Fukuda

Topics: Affective and Pleasurable Design

Publication Date: 2024

ISBN: 978-1-958651-99-5

DOI: 10.54941/ahfe1004669

Articles

Evaluation of the Effect of the Candle Holder Pattern on Concentration and Mental Stress

In this study, we investigated the possibility of combining the concepts of candles and fractals to further enhance their relaxation and concentration effects. In the experiment, 30 participants performed the numeric addition task, followed by a rest task and then an addition task again. The three conditions of the resting task included a patterned and pattern-less candle holders and quiet resting condition without light (Control). Biometric measurements taken during the experiment, questionnaires administered before and after the experiment, and performance on the numerical addition task were used as evaluation indices. The results suggested that candle gazing significantly improved the concentration of the participants. In addition, the combination of candles and fractals caused more relaxation than the combination of candles alone.

Masamitsu Nagayoshi, Keisuke Kisu, Keiichi Watanuki, Isao Inoue

Conference Proceedings

Factor of Discomfort During Jumping Extended in Virtual Space

In conventional games, enhancing a character’s physical abilities leads to increased entertainment and exhilaration. However, in virtual reality (VR) games, the movements of the in-game characters and players are synchronized, enhancing the sense of physical presence and immersion. Therefore, it is believed that the entertainment value of VR games can be enhanced by extending the characters’ physical abilities. This study focuses on the impact of enhancing a character’s jumping ability in a VR game, which is a fundamental human movement. Our findings indicate that significantly increasing the jump height diminishes the sense of motor subjectivity. This is attributed to the incongruence between visual input and the anticipated sensation based on previous experience. We then investigated the senses that exhibited the largest discrepancies. We found that the discrepancies were particularly large in terms of speed and height, with a higher sense of levitation correlating to increased entertainment value. Future work will aim to propose a trajectory that can minimize the discrepancy in sensations of speed and height and enhance the sensation of floating.

Takuto Adachi, Keiichi Watanuki, Kazunori Kaede, Yusuke Osawa

Conference Proceedings

Design of 3D point cloud dataset of indoor spaces for feature extraction using Autoencoder with PointNet

In this study, a novel method was developed to automatically construct a virtual space with a high degree of freedom of expression. The constructed virtual space was designed to reflect the spatial shape of the real space and the arrangement of objects. First, the global shape of the interior space was used to design a dataset for extracting the spatial features of the real space by three-dimensional (3D) scanning of the real space and using a PointNet-based autoencoder. The dataset consisted of the point cloud data of a rectangular 3D object that was a simple imitation of a room in real space and focused on two items, namely the number of input points and the number of data points. The results of the autoencoder restoration indicate that spatial feature extraction can be performed when the number of data is 5000 or more.

Takahiro Miki, Yusuke Osawa, Keiichi Watanuki

Conference Proceedings

Generating Paintings Eliciting Specific Emotions Using Machine Learning for Application in Painting Therapy

This study aims to label paintings based on biometric information and generate paintings that elicit specific emotions using machine learning. To create the dataset, we conduct experiments with eight participants using multi physiological measurement sensors. We focus on the arousal axis of emotion and use the skin conductance response as a measure of arousal. The results suggest that machine learning may be effective in generating paintings that elicit emotions because features related to arousal, such as brightness and color, can be appropriately learned.

Keisuke Kisu, Kozawa Motohiro, Keiichi Watanuki

Conference Proceedings

Estimation of Worker Stress Considering Differences in Listening Tempo

In recent years, a stressful society wherein various stresses accumulate only by living has become a problem. This accumulation adversely affects our physical and mental health. In particular, psychosocial stress is considered to be related to the onset and pathology of lifestyle-related diseases such as diabetes and myocardial infarction. Recently, there has been a growing interest in the problems of increased daily mental stress and decreased productivity. Many studies have been conducted with the aim of alleviating mental stress and improving productivity. In particular, it is acknowledged that listening to music during work improves concentration and facilitates work. In addition, as the term "music therapy" indicates, it has been reported that listening to music can relieve mental stress. To further verify this effect, research is being conducted actively using various types of music. Furthermore, analysis is being performed based on a number of classification methods, such as classification based on genre, by instrument, and presence or absence of lyrics. However, the music used as representative music for these classifications also varies, and consistent observations have not been obtained. In addition, individual differences significantly impact the effectiveness of music listening. This problem has prevented the acquisition of consistent observations. Solving the above two problems, (complexity of classification and individual differences, will be very) is highly important for gaining new knowledge regarding music therapy and, beyond that, in relieving daily stress and improving productivity. Machine learning classification is effective in addressing the complexity of music. The structure of music has rules such as musical scales, chord structures, and chord progressions based on music theory. It is considered feasible to analyze the structure using methods such as machine learning. In addition, incorporating biometric information into the parameters is considered effective for solving the problem of individual differences. A previous study of emotion induction using music reported that the accuracy of the linear regression model was exceeded by incorporating EEG information into the parameters of the generative model. Therefore, in this study, we utilize "machine learning" (which is good at classification) and personalization by "biometric feedback" to construct a personalized music generation model for alleviating mental stress and improving productivity. The target tasks are VDT tasks. The demand for these has been increasing recently with the introduction of work-at-home, IoT, and DX, etc. VDT tasks are tasks performed on a PC or other devices on a display Although these enable working without time or place constraints, these are considered to be a cause of the progression of a stressful society because individuals cannot leave their work. Listening to music generated by the model constructed in this study is likely to alleviate daily mental stress and improve productivity.

Ryo Suetake, Keiichi Watanuki, Kazunori Kaede, Yusuke Osawa

Conference Proceedings

Effects of Dual Tasking with Visual Display Terminal Work and Advertisement Presentation on Workers' Interest and Emotions

In recent years, the use of visual-display terminals (VDTs) such as PCs and cell phones, for work, is proliferating. To perform efficiently, workers must be able to concentrate on and become immersed in their work. Most VDT work is performed while connected to the Internet and unintended distractions such as Internet advertisements (that appear on the display during work) can interfere with concentration. Distracting advertisements are generally considered disturbing and socially problematic. Previous studies have reported that visually disturbing stimuli presented in the peripheral-visual field reduce the ability to concentrate on tasks that are performed using central vision and have suggested that Internet advertisements may adversely affect work efficiency and accuracy. In-feed advertisements that blend into the timeline of articles and SNS, and recommendation-widget advertisements at the bottom of certain articles have been attracting attention recently, and their market is rapidly expanding. These native advertisements deliver value in line with the interests of workers in a natural flow. They promote consumption and fulfil their fundamental role as advertisements. In this context, we plan to investigate appropriate advertisement-presentation conditions, for inclusion in the worker's field of vision, in which the advertisements are recognizable, and worker concentration is unaffected. We aim to consider advertisements that benefit both advertisers and workers. Our objective is to investigate the nature of the interest and emotions generated by the advertisements in the workers. A cognitive memory task will be created, and during its execution the system will place simulated advertisements on the monitor for effect analysis. Task results, subjective evaluation, and eye-gaze information will be used to investigate advertising-element effects (such as color and movement) on worker interest and emotion and the corresponding impact on work concentration. Therefore, this study will provide an in-depth understanding of advertisement impact on worker interests and emotions while examining advertisement presentation conditions that are beneficial to both advertisers and workers. The study will explore the benefits that Internet-advertising optimization can bring to individuals and will contribute to the development of more effective and acceptable advertising strategies.

Kanta Onimaru, Yusuke Osawa, Keiichi Watanuki, Kazunori Kaede

Conference Proceedings

Estimating Work Efficiency Using Biological Information During Computational Work with Cognitive Load

In recent years, long working hours have become a problem in Japan. One of the measures to solve this problem is to improve work efficiency, and it is essential to establish an evaluation method for this purpose. It is also said that work efficiency decreases due to an increase in cognitive load caused by long working hours. Cognitive load refers to the amount of information processed by the brain's working memory, and it is difficult to process new information or complete a task when cognitive load is high.In previous studies on cognitive load and work efficiency, biometric information is often used as a method for evaluating cognitive load, and its effectiveness has been suggested. However, at the present stage, quantitative estimation of work efficiency under cognitive load has not been studied using biometric information. In this study, we measured biometric data during a continuous addition task with the aim of estimating work efficiency under cognitive load using biometric data. As evaluation indices, we used biological information such as heart rate variability and changes in the relative concentrations of oxygenated hemoglobin (Oxy-Hb) and deoxygenated hemoglobin (Deoxy-Hb) in the prefrontal cortex. Machine learning methods were used to estimate work efficiency, and Convolutional Neural Networks (CNNs) were used to create the machine learning models. CNNs are used in a wide range of fields as machine learning methods and are also widely used in research in the fields of cognitive load and biological information.The experiment was conducted five times on four male subjects. The continuous addition task was performed on a VDT screen for 20 minutes, and the number of correct answers in 30 seconds was used as the task efficiency. Electrocardiograms and cerebral blood flow in the prefrontal cortex were measured during the experiment.Four models were created for everyone to estimate work efficiency every 30 seconds using Oxy-Hb and Deoxy-Hb in the electrocardiogram and prefrontal cerebral blood flow. The biometric data were time-series data, and CNN was used to extract time-series features. To examine the effects of electrocardiograms and cerebral blood flow on learning, we created a model with electrocardiograms alone as explanatory variables, a model with Oxy-Hb and Deoxy-Hb in cerebral blood flow as explanatory variables, and a model with electrocardiograms and Oxy-Hb and Deoxy-Hb in cerebral blood flow as explanatory variables. We also used PFI to evaluate the importance of prefrontal cortex regions on learning.The training results showed that the ECG-only model was not sufficient for estimation, while Oxy-Hb and Deoxy-Hb in the cerebral blood flow provided highly accurate estimation. The PFI values suggested that the dorsomedial prefrontal cortex and the left dorsolateral prefrontal cortex were relatively important.

Kosuke Sato, Keiichi Watanuki, Kazunori Kaede

Conference Proceedings

Development of High-Precision Emotion Estimation Method using Speech Sound Information with Environmental Noise Reduction and Low Sampling Rate

Current research on emotion estimation demonstrates its feasibility at a reduced sampling rate of 6 kHz, thus moving away from traditional methods that depend on higher sampling rates; however, low sampling rates have not been adequately investigated. In addition, noise factors have been limited to electronic sounds rather than environmental. Therefore, this study explores the development of a high-precision emotion estimation method using spoken speech data, focusing on scenarios with environmental noise and low sampling rates. To suppress noise, the proposed method extracts feature quantities for emotion classification using band-pass filters and stacked autoencoders. However, the construction of a high-precision emotion estimation model with these feature quantities required further investigation. Thus, emotion estimation was investigated using a one-dimensional convolutional neural network. The results showed an emotion estimation accuracy of 94.7%, indicating successful noise control. Future work will build on this research to develop emotion estimation methods using spoken speech data that can be employed even in noisy environments.

Kanji Okazaki, Keiichi Watanuki

Conference Proceedings

Feature Selection and Estimation of Route and Gait During Walking with Route and Speed Changes by Surface Electromyograph Using Transformer

Recently, with the increase in the number of elderly people requiring nursing care, walking assist devices using an inverted pendulum model and electrical stimulation have been researched and developed to improve the walking ability of elderly people. However, most of these assistive devices are designed only for straight walking on a flat road and do not support unsteady walking, such as turning and acceleration/deceleration. It has been reported that unsteady walking accounts for 40% of our daily life. Hence, it is important to provide training and support for unsteady walking. The purpose of this study is to develop a walking training and support system for gait that includes unsteady walking. Therefore, we built a model that predicts the gait and route of the next walking cycle by extracting muscle movements from surface electromyography (sEMG) data using machine learning. We used transformer for machine learning because of its powerful expressive capabilities and its success in various fields, such as natural language processing, computer vision, and speech processing. Transformer can also evaluate sEMG features in learning by visualizing attention. sEMG and joint angles were recorded while walking along the walking routes set up in this study, and these data were divided into data sets for each cycle to predict the next walking motion, which were then used for training. sEMG was recorded using a wireless electromyograph and the joint angles were recorded using optical motion capture. A Butterworth bandpass filter was applied to the recorded sEMG to remove noise in the range of 20–450 Hz. Then, sEMG at the maximum voluntary contraction was normalized to the reference. The recorded joint angle data were normalized with respect to the range of motion of the joint. The training model utilizes the encoder block of transformer, which consists of a multi-head attention and feed forward network. The multi-head attention can extract different expressions by setting up multiple attentions in parallel. The learning parameters were Adam for the optimization algorithm, a learning rate of 0.001, a batch size of 1, and the number of epochs was set to the value that optimizes the loss, with the maximum value set to be 200. Therefore, the mean absolute error of the joint angles was 2.04°, which was sufficient to estimate the gait in the next cycle. The visualization of attention confirmed the extraction of different representations, indicating that the learning process was successful.

Naoki Nozawa, Yusuke Osawa, Keiichi Watanuki

Conference Proceedings

AI-Driven Music Generation and Emotion Conversion

With the integration of Generalized Adversarial Networks (GANs), Artificial Intelligence Generated Content (AIGC) overcomes algorithmic limitations, significantly enhancing generation quality and diversifying generation types. This advancement profoundly impacts AI music generation, fostering emotionally warm compositions capable of forging empathetic connections with audiences. AI interprets input prompts to generate music imbued with semantic emotions. This study aims to assess the accuracy of AI music generation in conveying semantic emotions, and its impact on empathetic audience connections. ninety audios were generated across three music-generated software (Google musicLM, Stable Audio, and MusicGen), using four emotion prompts (Energetic, Distressed, Sluggish, and Peaceful) based on the Dimensional Emotion Model, and two generated forms (text-to-music and music-to-music). Emotional judgment experiment involving 26 subjects were conducted, comparing their valance and arousal judgments of the audios. Through Multi-way variance analysis, the AI-music-generated software had a significant main effect on the accuracy of conversion. Due to the diversity of generated forms of MusicGen, it has a lower accuracy of conversion compared to Google musicLM and Stable Audio. There was a significant interaction effect of generated forms and emotion prompts on the accuracy of conversion. The differences in accuracy between emotion prompts in the form of text-to-music were statistically significant, except for the differences between the accuracy of Distressed and Peaceful. Compared with the generated form of text-to-music, the form of music-to-music showed statistically significant emotional conversion ability for low arousal. The diversity of AI software input elements (i.e., text or music) may affect the effectiveness of emotional expression in music generation. The ability of different software to convey different emotions according to different prompts was unsteady in the form of text-to-music. This study advance computer music co-composition and improvisation abilities, facilitating AI music applications in fields such as medical rehabilitation, education, psychological healing, and virtual reality experiences.

Xinwei Gao, Deng Kai Chen, Zhiming Gou, Lin Ma, Ruisi Liu, Di Zhao, Jaap Ham

Conference Proceedings

Intelligent Chair Product Design Based on Embodied Emotion Theory

This study aims to explore how embodied emotion theory can be applied to the design of smart seats to enhance users' physical comfort and emotional experience. Firstly, the article introduces the basic concepts and development history of embodied emotion theory. Then, it analyzes the current application of embodied emotion theory in product design and summarizes the relevant design principles. Subsequently, a product design scheme for a smart seat is proposed, which integrates sensor technology, data analysis, and feedback mechanisms. This scheme aims to adjust the user's emotional state by identifying their emotional status and setting the seat's posture accordingly. The effectiveness of this seat design in improving user emotion and comfort has been verified through experiments. Finally, the paper discusses the application prospects and potential value of smart seats in the field of human-computer interaction.

Xiaoqin Cao, Huiqiang Xing, Aoyue Dong

Conference Proceedings

Analysis of the emotional design and application of educational parent-child interactive products

In the realm of product design, when functional value reaches a certain plateau and stops increasing, there is a shift towards seeking intrinsic value. Users place greater emphasis on the experience of using a product. When designers conceptualize a product, they incorporate emotional and creative elements, paying attention to the emotional needs of the users. Emotional design is a "user-centered" approach, which includes the analysis and practical application of users' emotional factors. Through design, it enhances the spiritual needs that a product can fulfill and values the emotional feedback users receive after using the product.By employing qualitative analysis methods, this study examines the "quality" aspects of educational parent-child interactive products and the attributes of emotional design, comparing the purchasing tendencies of the target audience. Furthermore, we analyze three dimensions of design to provide systematic theoretical support for the design of parent-child interactive products. The results indicate that during the design process of parent-child interactive products, emotional design can enhance the user experience and encourage positive emotional responses from users.An in-depth exploration of emotional design in educational parent-child interactive products reveals its significant influence during the promotion of such products. The design revolves around the emotional needs of users, with the overall design process centered on improving user experience and focusing on the emotional value of the product.

Changfeng Jiang, Rui Xu, Hongjun Yu, Danmeng Xu, Ziqiong Yang

Conference Proceedings

Understanding Emotional Needs across Stroke Rehabilitation Phases for Affective Design

Despite significant physical and psychological challenges, stroke patients often face overlooked emotional needs in rehabilitation. Such emotional needs can be satisfied by the innovative design of rehabilitation equipment, but the specific patient needs across recovery phases (acute, stable, and maintenance) remain unclear for targeted emotional design. This study aims to identify pain points and emotional needs in each recovery phase through observations and interviews with 31 participants and 60 hours of data. Acute-phase patients struggled with both communication challenges and psychological barriers from disability and identity shifts. Stable-phase patients experienced loneliness, self-blame, and boredom during intense therapy. Maintenance-phase patients, while physically improving, felt hospital fatigue, desired social interactions and entertainment, and worried about reintegration. Based on these findings, we also discuss emotional design implications and examples tailored for each stage.

Yixuan Lyu, Yue Chen

Conference Proceedings

Development of an AI Literacy Scale Using Multiple-Choice Questions

With continuously emerging and developing artificial intelligence (AI) technologies, we now have more opportunities to interact with AI agents, to use AI applications to assist our jobs, and to assess the solutions provided by AI. This ability to properly identify, use, evaluate, and collaborate with AI-related products is referred to as AI literacy (Long & Magerko, 2020; Wang et al., 2022). The objective of the current study is to develop an instrument to measure general users’ AI literacy by replacing subjective self-report questions with objective, multiple-choice questions. 12 questions were derived from four dimensions of AI literacy (i.e., awareness, evaluation, ethics, and future AI), and a total of 230 validated responses were collected through the online survey. After deleting an unqualified item, the explorative factor analysis revealed a 3-factor structure of the remaining 11 items in the AI literacy scale: interacting with AI products, understanding AI’s capabilities, and understanding AI’s limitations. Each sub-scale is of acceptable reliability and validity. Furthermore, we examined the relationships between AI literacy and actual use of AI products, digital literacy, the attitude towards AI agents, and individual characteristics such as gender and education. The results suggested that a higher level of overall AI literacy was associated with better digital literacy, and a more positive attitude towards AI agents. The ability to interact with AI literacy, however, was correlated with more negative feelings about AI agents. Gender and differences in education were shown to have a significant impact on AI literacy. This exploratory study contributes to developing a more objective measurement of general users’ AI literacy and provides some insights on users’ attitudes towards AI.

Xinyun Li, Qin Gao, Pei- Luen Patrick Rau

Conference Proceedings

Review and appraisal of approaches to assess comfort of wearable devices

In recent years, the market for lightweight wearable devices for eye-, ear-, and wrist-worn has grown rapidly. For these lightweight wearable devices, comfort directly impacts consumer adoption. However, most of the existing literature on the comfort of wearable devices have focused on wearable computers that are large in size and weight, and there is still a lack of comprehensive insights for approaches to assess the comfort of lightweight wearable devices. The present study reviewed existing research on the comfort of lightweight wearable devices, discussed the characteristics and limitations of current comfort assessment approaches, and provided feasible directions for foreseeable more extensive comfort assessment research.

Xuan Wang, Zilin Jiang, Qin Gao

Conference Proceedings

The application of interaction design to human emotion and pleasure based on interactive concepts

With the continuous progress and development of technology people's demand for interactive experience is increasing, the interactive concept has become an important trend in today's design field. It refers to an approach to designing and developing products, services or systems that emphasises the interaction between the user and the product, system or environment. It considers the user not only as a user of the product, but also as an active participant in the design process. It mainly includes interaction methods, feedback mechanisms, personalisation and other aspects to reflect the interactive concept. Emotional design is a design concept belonging to a deeper level of interaction design, which is the integration of design concepts with emotional aspects with interaction design to meet the emotional needs of users. Storylines, symbols and metaphors are used to trigger an emotional response from the user. Interaction design aims to improve user satisfaction and experience by designing interfaces that are easy to use, intuitive and enjoyable. It is not just about meeting the functional needs of the user, but also about its impact on human emotion and pleasure. And interaction design is the practice of defining and constructing interactive experiences is a new design approach based on the development of new media technology and user experience derived from the aspects of design, psychology, computer science and other disciplines, in order to create an experience that meets the needs of the user, the pursuit of fun and novelty.The aim of this paper is whether the specific application and methods of interaction design on human emotion and pleasure under the interactive concept enhances and improves the user's emotional experience. It is hoped that through this study, the use of interaction design on human emotion and pleasure under the concept of interaction can be enriched to enhance the value of user experience and product value. Moreover, on the basis of the interactive concept, the close integration of interactive design and human emotion will make the public more aware of the importance and necessity of interactive design for human emotion and pleasure.This paper explores the cases of human-computer interaction, interactive games, interactive advertisements, and interactive videos through specific case studies, functional analysis, and case studies, which can be combined with cases in different fields to analyse the importance of interactive design on human emotions. The purpose of this study is to investigate the impact of interactive design on human emotion and pleasure based on the concept of interactivity. Better interaction design can enhance users' emotional experience and pleasure, help users deal with negative emotions, increase users' satisfaction with products, enhance users' engagement and the value of products or services, and promote innovation of products or services. An in-depth study of the use of interactive design on human emotion and pleasure, and maintaining products through interactive concepts and emotional design is important for improving product functionality and enhancing user experience.

Fu Siyu, Rui Xu, Danmeng Xu, Yan Yue

Conference Proceedings

Sit back, relax, and sniff some lavender!: Investigating the Impact of Scent on Passengers in Air Taxis

Air taxi services (ATS) have emerged as a potential solution to tackle congestion and pollution issues associated with traditional taxis. However, the perception of passengers on innovative experiences remains under-studied. Extensive literature has established that scents possess a strong influence on human emotions and can contribute to relaxation. To bridge this research gap, we adopted the Wizard-of-Oz Study method, widely employed in human factors research, to investigate (1) the scent preferences of individuals regarding ATS and (2) the influence of scents on passengers in air taxis. We show that there was a statistically significant difference between male and female responses to lavender(p<0.001) and peppermint(p=0.001) flavours in liking. However, males and females responded statistically significantly differently in terms of relaxation only to peppermint scent(p=0.042*), and not to lavender(p=0.063). Our findings have the possibility to provide a guide to the development of experience design on the ATS.

Shengdi Xiao, Jingjing Li, Dmitrijs Dmitrenko

Conference Proceedings

Enhancing User Immersion in Virtual Reality by Integrating Collective Emotions through Audio-Visual Analysis

In the rapidly evolving field of virtual reality (VR), deep user immersion remains a major challenge for researchers and developers alike. Effectively integrating emotional cues into VR environments to enhance the user experience could be a key issue. This study introduces a positive advancement by presenting an innovative solution that combines audio and video analysis to detect and integrate collective emotion while watching 360° events. Our approach sets itself apart from previous work, which focused either on the visual or auditory aspect, by embracing a holistic perspective that more accurately mirrors the complexity of the human experience. We developed a machine-learning architecture that utilizes advanced models. Existing datasets have been enriched in a balanced way and are used to train various models, including a face extraction model, and emotion classification models based on spectrograms and audio features. Predictions from these analyses are merged to generate a value representative of crowd emotion valence using 360° videos as input. All models and the final architecture are assessed using accuracy, F1-score, precision and recall metrics. This proposed architecture enabled the creation of a nuanced representation of collective emotion, which is then used to generate targeted visual, auditory, and haptic stimuli. These stimuli are designed to enhance user engagement and immersion in VR by adding a layer of emotional interaction. The latter is assessed through user experience while watching 360° penalty event. The sound model architecture, which employs Random Forest and XGBoost models that feed into a meta-learner, achieves an accuracy of 98.71% on test set. Meanwhile, the model for classifying human facial emotions, tackling a challenging 7-class classification problem, achieves an accuracy of 56.15%. While this shows promising potential, incorporating additional visual elements such as object detection and scene analysis could further enrich the understanding of collective emotions and enhance the robustness of our model. The results of our study indicate that integrating these stimuli based on collective emotion recognition significantly increases user immersion. Tests with 10 participants demonstrate a notably pronounced improvement when haptic feedback is involved, highlighting the tactile dimension as an especially powerful channel for conveying and amplifying emotions in VR environments. This research has demonstrated that the integration of audio and visual analyses can significantly enhance the performance and robustness of crowd emotion detection models in VR environments. By synthesizing these two input modalities, we have been able to provide a more comprehensive understanding of collective emotions, which in turn has positively impacted the user’s immersion. These findings underscore the potential for more sophisticated and emotionally aware VR systems, suggesting that similar approaches could be beneficial in advancing the field and enriching the user experience across various applications.

Sam Corpataux, Marine Capallera, Omar Abou Khaled, Elena Mugellini

Conference Proceedings

Effect of forest bathing for mindwandering

After COVID-19, people are interested in healthcare, so the market for health tourism is increasing. Forest bathing is one of the attractive activities during a trip. To provide tourists with information on the healing effects of forest bathing, we are researching to clarify the questions of "What exactly are the effects of forest bathing, and what causes the healing effects? and "How can we optimally present tourist information on the healing effects? By building a generic model of technology for providing information on forest bathing through experiments in Oku-Nikko, we aim to spread our research results to the health-oriented society. In recent years, research on the default mode network, the idle state of the brain, has been progressing. It has been reported that during meditation and other tasks with low cognitive load, a state of mind wandering occurs, in which attention is directed not to events in the external environment but to one's internal self, and one thinks about various things in a hazy way. It has also been reported that the default mode network, which consists of multiple memory areas in the brain, is activated during mind wandering, and it has been pointed out that this activation may connect memories in various brain regions under conscious awareness, facilitating creativity. In forest bathing, people are relaxed with low cognitive load, which is thought to facilitate the activation of the brain's default mode network, and thus, benefits such as increased creativity can be expected. In this presentation, we will describe the results of our investigation into the relationship between forest bathing and the default mode network, and in particular, whether forest bathing activates the default mode network.

Atsushi Ito, Yuko Hiramatsu, Kazutaka Ueda, Madoka Hasegawa, Yasunari Harada

Conference Proceedings

Creativity: The Writer as Transmitter and the Recipient as Reconstructor -The Dual Structure and the Imagination that Bridges the Gap between Them

This study examines creation by information recipients by the linguistic landscape survey in Nikko, a World Heritage Site in Japan. Linguistic landscape refers to the language signs along public roads. We can see traditional Japanese ways there. Old Japanese sometimes used a part of the ancient poems as a common culture in their poem, and from there, they created a double emotion of their own compositions and the ancient poems. This is a proactive and creative method in which the recipient adds a new vision to the one created by the creator/sender of the information. It is said Japanese language has a strong recipient-responsibility tendency, in which the recipient should take the initiative in grasping the meaning. A world-famous example is the stone garden at Ryoanji Temple in Kyoto. The stones and sand remind the viewers of large natural bodies of water and mountains. The creator assumed that those who see it will be actively involved and imagine. This tradition remains in Nikko roadside. For example, "Akebono(dawn)" is written on the board, as a name of a confectionery. When Japanese people look at the word "Akebono”, they remember the beginning of the Pillow Book, "Spring is Dawn (is best)”. Strawberry sweet reminds them of the beautiful dawn of spring. This is a double structure and is one of the manifestations in the Japanese culture of "Mitate". The pedestrians who walk along the road, are reminded of their cultural backgrounds, and there they pause to think and savor the double structure. It is the creation by the recipient that bridges the gap between the two. The uniqueness is not created by the creator, but rather the shared one is used to expand the imagination and share the senses. Considering tourism information application, it is not enough for such a creation. Applications provide as much detailed information as possible. Such information may be useful before the trip. However, when tourists stand in front of some stores, do they need detailed information about inside by smartphone? What is needed is for tourists to stop in front of the store. It is important that they are actively interested. Rather, it is also effective to leave the information incomplete (Zeigarnik effect). Novelty is required in tourism.However, in traditional Japanese cultural areas such as Nikko, where foreign tourists are increasing, the recipients do not have the same cultural background as senders of information. A major challenge is how to have a creative space for those who do not make common knowledge. There are three possibilities.Standardization is underway in the global. There is a possibility of new "Mitate" based on the commonality of the standardized parts.Foreign tourists will know the Japanese way of seeing things as repeaters.The "Mitate" will not be established. However, each visitor will create something new from what he/she has obtained. (Horizon of Expectation)Any or more of these are possible. Creation is in the hands of both the creator and the recipient.

Yuko Hiramatsu

Conference Proceedings

Brand associations with traffic patterns in store layout planning

In a brand store, different traffic patterns emerge as the layout plan guides customers toward what they want to buy. This study aims to clarify the types and definitions of traffic patterns, to understand the preferences of spatial designers for traffic patterns, and to explore the relationship between traffic patterns and brand associations. Focus group interviews with experts with spatial design backgrounds were conducted to obtain textual information, and the data analysis method of grounded theory was applied for content analysis. The study found that: (1) there are three types of store traffic patterns: open, sequential, and path-based traffic design; (2) space designers prefer to consider three categories in the design of store traffic patterns: layout flow, feedback and decorative atmosphere; and (3) To create a brand association, product characteristics, price information, and store lighting are regarded as the three most important properties.

Jiacheng Wang, Regina W Y Wang

Conference Proceedings

Embodied Cognition in Customer Experience (CX)

In this paper, we will focus on ACT (behavioral/physical experience) as a specific customer experience among the five strategic experiential modules (SEM) proposed by Schmidt. Like RELATE (relational experience), ACT (behavioral/physical experience) is said to be difficult to understand among the five strategic experiential modules (SEM), and it is difficult to understand whether it is possible to create it in the first place.Therefore, the authors redefined ACT (behavioral/physical experience) from the idea of embodied cognition, and then proposed ways to create that ACT (behavioral/physical experience) and realize differentiation. demonstrated the methodology.(1) ACT (behavioral/physical experience) by SchmittBernd H. Schmitt defines ACT (behavioral/physical experience) as `the experience that appeals to physical experience, lifestyle, and interaction with others.' Schmitt's definition of the behavioral/physical experience makes it difficult to understand the concept of `physical experience,' and that `interaction with others' is not related to the behavioral/physical experience. Therefore, it is easy to confuse it with RELATE (relational experience). This difficulty in understanding the concept of behavioral/physical experience is thought to be connected to the difficulty in its application.However, as mentioned in the previous chapter, according to Schmitt, ACT (behavioral/physical experience) is derived from the concept of Embodied Cognition (Schmitt, 1999, p.258). Therefore, in this paper, we reorganize the content of ACT (behavioral/physical experience) by reconfirming the concept of embodied cognition.(2) Redefining ACT (behavioral/physical experience)Schmitt developed the concept of embodied cognition into ACT (behavioral/physical experience). Embodied cognition is based on the concept of ecological psychology as described in the previous section. In ecological psychology, `body movement is determined by the interaction between the body and the environment,' and the characteristics of the environment in this interaction are called affordances.Therefore, we reorganized ACT (behavioral/physical experience) from this concept of embodied cognition and redefined it as follows. Physiological and psychological activities that occur along with the actions of consumers themselves during consumption activitiesBased on the concept of embodied cognition, which involves perceiving the environment through physical movement, both "behavior" and "physiological and psychological activities associated with behavior" can be said to be ACT (behavioral/physical experience). For example, physical movements such as running, sitting, and grasping, as well as the actions themselves such as studying, driving a car, and going to the park, as well as the feelings and emotions that come with these movements and actions, such as feeling good, refreshing, and having fun. Both are included in ACT (behavioral/physical experience).This definition of ACT (behavioral/physical experience) is an original proposal by the authors.(3) Interaction between ACT (behavioral/physical experience) and productsFrom the perspective of embodied cognition and ecological psychology, behavioral/physical experience is generated through interaction with the environment. Considering the process of consumer experience of acquiring, using, and disposing of products and services, the environment that is most affected is the product and service. In other words, the consumer's behavioral/physical experience is generated through the interaction between the product/service and the consumer. Movements and actions occur depending on how a product or service is used, and emotions such as pleasure and pleasure are also generated in the process of using the product or service.For example, it is the movement/behavior and feeling of use related to a product, such as the pleasantness of a moderate response when closing a car door or the comfort of opening a foldable cell phone, and this is behavioral/physical experience. The design of the product has a lot to do with this operation and usability. Nintendo's home game consoles Wii and Switch Sports are designed with new game controls such as `shaking,' `turning,' and `aiming' using the remote control.Donald A. Norman, called the design of products that convey the joy and utility of use in this way as behavioral design, and pointed out its importance (Norman, 2004). Furthermore, affordances are pointed out as an important element on the product side that influences and determines the usability (Norman, 1990). In the field of product design, Fukasawa et al. also refer to affordances and point out the importance of design that takes human behavior into consideration, using the expression `design that corresponds to behavior' (Fukasawa, Sasaki, and Goto, 2004, p.86).

Shinya Nagasawa

Conference Proceedings

Designing an Application to Encourage Appreciation of Body’s ‘Hyoh-Joh’ in Movement: Supporting Embodied Knowledge Learning from Phenomenological View

Athletes and dancers, as practitioners of bodily movement, are particularly engaged in the learning of embodied knowledge, which is a flow aiming at somatic meaning-making. In the learning of embodied knowledge, 'intentionality’ such as perception and thought play an important role; however, in visual intentionality, especially, there tends to be a lack of 'soma'. To activate visual intentionality 'through and with the soma’, the author hypothesizes that it is beneficial to attempt to see the ‘Hyoh-Joh’ of the body in movement. ‘Hyoh-Joh’, as a phenomenological concept, corresponds to Ausdruck in German and Expression in English, and refers to a living Gestalt that embodies ‘emotional values’ and ‘action affordance values’ in material things. a In this study, we are designing an application that encourages the appreciation of the ‘Hyoh-Joh’ of the moving body for practitioners of body movement. This app features 'moving figures' drawn in three-dimensional space on the screen, with each joint point of body movement as a vertex. Users can 'edit and appreciate' these figures artistically on their own. Moreover, they can introspect and describe their appreciation experience, name the created figures as a ‘Hyoh-Joh’ in their own way, and save them in a database along with the figures. This specification aims to encourage practitioners to perceive ‘Hyoh-Joh’ of body and support the learning of embodied knowledge. Through the designing and practicing with this app, this study explores ‘Hyoh-Joh’ of body in movement, a crucial aspect of emobodied knowledge.

Takahito Horiuchi

Conference Proceedings

Analysis of the impact of playing area size on ball retention time and number of touches in soccer and ice hockey possessions: A case study

Does a change in training organization affect to play? In soccer, coaches often require players to pass a ball quickly. This speed includes the speed of the ball. In many cases, players must shorten their time on the ball and switch to the next play with as few touches as possible. How can such quick passing training be achieved? Coaches point to quick passage using language, but the critical factor in whether a player can execute it is considered to be the organization of the training: for example, the practice environment, such as the size of the playing area.The size is a factor in the organization of training to elicit appropriate play. For example, whether the playing area players want to familiarize is 5 or 10 meters is extremely different. Thus, providing the playing area which is different from the size of the entire soccer field may encourage players to a specific practice. One of the types of passing training described above involves players who do not have a ball and taking the ball away from the other players of the team who do. This training is called possession and considered to be widely used by modern soccer teams in various forms. How could the play be changed? Research areas, such as motor learning, have shown that humans and animals unconsciously shift their movement from a specific pattern to another by changing parameters. For example, a horse change the manner in which it moves its legs from walking to running. As the walking speed increased, the foot changed to running at a certain speed. Such changes in movement are not consciously caused by the humans or animals, but can be regarded as bodily knowledge. By this physical knowledge, coaches may induce the plays they want their players to execute by altering the practice environment.In this study, we examined how ball or puck in ice hockey retention time and the number of touches changes if the size of the playing area changes during 3-on-1 possession training in soccer and ice hockey.If the distance between each player is sufficient, it is expected that the possession players will have more time to hold and touch a ball because the defending players who do not control the ball will have time to apply pressure. However, as the distance shortened, the time it takes for the defending players to use pressure decreases; therefore, it is expected that possession players will have less time and fewer touches on the ball. This is a case study of gradual changes in training organizations. However, it is unclear that whether players can intentionally implement quick passing and adaptively play. We want to examine the training organization necessary to enable the metacognition of physical knowledge and adaptive changes in play.

Masayuki Yamada, Yuta Ogai, Sayaka Tohyama

Conference Proceedings

Developing AI Video Analysis Systems to Explore Human Behavior in Infant and Ethnographic Footage

The advancement of Information and Communication Technology (ICT) has enabled the storage of large volumes of video data. In recent years, research has focused on technologies for extracting this video data in formats suitable for specific purposes. For instance, it is possible to derive insights about developmental processes from daily life video data or to extract specific segments of ethnographic footage for artistic expression.Scholars stress the importance of analyzing daily videos of infants to understand their developmental processes, but watching all infants' daily videos would require an enormous amount of human effort. Therefore, we considered employing AI technology, which has advanced rapidly in recent years. This presentation will present two examples of our AI-based video analysis to demonstrate the possibilities and challenges involved.In our first study, focused on an infant, we examined the possibility of using object-detection, action-recognition, and caption-generation AI to detect infant movements for the purpose of developmental research and monitoring. The object-detection AI, YOLOv8, extracted images surrounding the area detected as a human in infant videos. The caption-generation AIs CATR and BLIP were then used on each image to evaluate whether they could detect infants and provide information about their behavior. SlowFast, an action-recognition AI, was also used to detect infant behavior in the videos. On the basis of the results of these studies of individual AIs, we will discuss the potential of combining them. Ethnographic video data presents similar challenges.Another example of using AI is the experiential video installation "Diverse and Universal Camera," a media artwork that uses AI to analyze ethnographic footage. This project employed SlowFast and YOLOv8 to develop a system that automatically labels the actions of people and objects in videos and enables efficient video retrieval for exhibitions of ethnographic footage archives.In these examples, AI tools process amounts of video data that are too large to be managed by humans, extracting parts of the video that merit human attention to provide a better understanding of human behavior. One challenge is that installing multiple cameras in a household to capture everyday situations often necessitates reducing the video resolution due to storage and network bandwidth constraints. Moreover, because of the need to cover wide areas, an infant frequently appears small in the video, and other objects commonly appear in it, such as family members and the infant's bedding and toys. Ethnographic footage is also difficult to analyze, as it is typically old, in black-and-white, and in low resolution. Furthermore, each segment of footage, shot with different themes, emphasizes varying subjects or objects.We are investigating methods of using AI to address these problems, such as cropping individual image segments, classification of video by generated captions, and diffusion in the time direction. We believe that these methods can be applied to other video types, significantly enhancing the potential for research that analyses everyday human behavior.

Yuta Ogai, Yuto Ono, Yasushi Noguchi, Sayaka Tohyama, Hideaki Kondo, Masayuki Yamada

Conference Proceedings

Awareness and Empathy in Self-Body Control for Manufacturing Skill Education

In this presentation, we will discuss the value of skills in the field of manufacturing and introduce a novel approach to effectively acquire skills by focusing on bodily movements. In recent years, the field of manufacturing has rapidly embraced digital transformation (DX). In particular, sensor and computer technologies have significantly advanced over the past decade. This has enabled the real-time acquisition of various physical information related to manufacturing, leading to a substantial increase in the parameters of models. Consequently, the precision of automatic control and robotic technology has greatly improved, facilitating the automation of complex manufacturing processes.However, the importance of human skills has become more pronounced amidst this digital revolution. Although DX ensures certain levels of precision and quality in production, digital information heavily relies on resolution and sampling frequency. Moreover, the number of parameters is finite. Many manufacturing processes involve environments that are not as stable as factory settings. In the pursuit of differentiation through high-precision, high-quality products, human skills are indispensable.Despite ongoing research on human skills in the context of DX, most prior studies have focused on skill evaluation, often framed as the "Expert-Novice Problem.” Evaluation methods assess the quality of end products and use motion capture or image analysis techniques to detect differences in skill movements across various stages. Subsequently, learners are expected to imitate the skill movements of experts, with post-evaluation feedback highlighting product defects and low-skill-level movements.Although imitation learning is effective to a certain extent, it presents limitations in achieving a mastery level of proficiency. To precisely control tools for manufacturing objects, learners must understand how to control their own bodies. Bodily control involves perceiving sensory information related to muscle activity timing, with the balance between body parts and tools collectively referred to as somatosensory information. Learning how to consciously utilize this somatosensory information while processing states during movements is crucial for skill improvement.Because somatosensory information is difficult to verbalize and exhibits an unclear relationship with skills, it has often been regarded as tacit knowledge. We therefore propose a skill information structuring method to clarify somatosensory information, presenting a support approach that encourages learners to be conscious of somatosensory information. This method facilitates mutual understanding between instructors and learners by sharing their respective mental images, enabling empathetic training. Our learning approach is expected to be effective not only for manufacturing, but other skills that involve bodily movements.

Yoshifusa Matsuura

Conference Proceedings

Increasing Importance of Body Intelligence-The Age of Feeling

Changes in our world is occurring rapidly, with yesaterday's changes being smooth and predictable, while today's changes are sharpt and unpredictable, Therefore,, it is essential to be context-aware at all times, as immediate decision-making is crucial. Sensation and feeling , rather than reason, play a significant role in the process, with germplasm being a necessary component.

Shuichi Fukuda

Conference Proceedings