Accessibility, Assistive Technology and Digital Environments

AHFE International

Accelerating Open Access Science in Human Factors Engineering and Human-Centered Computing

Accessibility, Assistive Technology and Digital Environments

book-cover

Editors: Matteo Zallio

Topics: Human Factors and Assistive Technology

Publication Date: 2024

ISBN: 978-1-958651-97-1

DOI: 10.54941/ahfe1004607

Articles

Symbolism in Extended Reality

For centuries, humans used symbols to describe scenes, objects, and events. Some of the symbols have evolved into pictorial languages such as petroglyphs and Chinese. The rapidly growing Extended Reality (XR) technologies enable users to experience immersive photorealistic views with head-mounted displays, 3D projection screens, or holographic display panels. The purpose of Extended Reality (XR) is to make invisible to be visible to the user. In this study, we explore how to superimpose symbolic feedback to the user in forms of geometric shapes, trajectory traces, spectra image, semantic network, and multimodal alerts, including audio and tactile signals. We zoom into the laparoscopic surgery train as a case study for applications. Our preliminary experiment results show that the symbolism approach does not significantly increase the distraction of attention or computational load. We also found that Extended Reality, especially, multimodal alert would be a bridge between manual operations and autonomous operations.

Yang Cai

Conference Proceedings

Video Generation Method Unconsciously Gaze-Guiding for a Passenger on Autonomous Vehicle with Controlling Color and Resolution

While automobiles are an indispensable means of transportation in our daily lives, many people suffer from motion sickness. Various symptoms such as nausea, fatigue, and vertigo develop when people become motion sickness. There are several theories on the cause of motion sickness, including the sensory confusion theory, which states that the brain becomes confused due to discrepancies in information from sensory organs such as vision, vestibular sensation, and somatosensory perception, and the eye movement theory, which states that motion sickness is caused by optokinetic nystagmus. According to these theories, gaze movements are deeply related to the cause of motion sickness. Moreover, existing research has shown that motion sickness can be alleviated by looking at a particular object. Therefore, it is considered that motion sickness can be reduced by providing gaze guidance that encourages fixation. However, explicit gaze guidance that gives visual stimuli such as flashing lights may greatly disturb the user experience while riding in a car, so it is preferable to provide unconscious gaze guidance in which the user does not feel the intention of gaze guidance. This research aims to realize unconscious gaze guidance methods to reduce motion sickness. We proposed three types of unconsciously gaze-guided video generation methods.　The first method is a resolution-controlled gaze guidance method. This method reduces the resolution by applying a smoothing process to the unguided areas and guides the gaze to the unprocessed high-resolution areas. The second method is the color-control method. This method is based on the idea that regions with high saliency in a visual saliency map are preferentially gazed at. This method guides the gaze by changing the hue and brightness of the image so that the saliency of the area to be guided becomes higher in the image. The third method is a combination of resolution-control method and color-control method. By combining these two methods, we hypothesized that we can complement the problems of both methods and achieve a stronger gaze guidance effect. We expected that unconscious gaze guidance is possible, since neither of these methods provides explicit visual stimuli. Experimental evaluations to verify the effectiveness of each method are conducted. In the experiment, we measured the gaze position while watching the generated video using an HMD with a gaze measurement function. We compared the success rate of gaze guidance between movies generated by the proposed method and movies with explicit stimuli that were shown to be effective in reducing motion sickness in existing studies. The gaze guidance rate of the movie generated with the resolution-control method was 53%, that of the movie generated with the color-control method was 34%, and that of the movie generated with the combination of the two methods was 70%. These results show that the combination of the two methods has the closest gaze guidance effect to explicit stimuli. We also conducted a questionnaire survey of the intensity with which participants felt they were being gaze guided while watching the movie. All participants felt the strongest intention of guidance with the explicit stimuli, whereas all participants felt less the intention with the proposed methods than with the explicit stimuli. Furthermore, the color-control was almost imperceptible to the intention. These findings indicate that there is a correlation between the strength of the intention of the guidance and the guidance effect. As the result, it can be considered that the guidance effect and naturalness are controlled by adjusting the degree of changes.

Yamato Miyajima, Chun Xie, Itaru Kitahara

Conference Proceedings

Measuring Remote Collaboration Supported by Extended Reality: A Multidimensional Evaluation Approach

Remote collaboration supported by eXtended Reality (XR) is a paradigm that transforms how geographically dispersed teams collaborate on shared tasks by leveraging immersive technologies that bridge physical distances. This approach goes beyond traditional communication tools, by creating shared, three-dimensional spaces that enhance the depth and effectiveness of collaboration. Regardless, it also introduces new challenges, like evaluating how collaboration occurs during such contexts, which literature shows is not straightforward. This work delves into evaluation in the realm of remote XR collaboration by proposing that a multidimensional approach is used, considering the level of information understanding, communication effectiveness, or social presence, among others. By analyzing how evaluation could be performed, we intend to offer insights to steer research and development in this dynamic field. These assessments can contribute not only to refining XR-system development but also deepen comprehension of how teams collaborate over time.

Bernardo Marques, Samuel Silva, Betriz Sousa Santos

Conference Proceedings

The Role of Collaborative Virtual Reality Engagement in Stroke Survivors' Rehabilitation

Stroke stands out as a significant contributor to disability and mortality rates among individuals worldwide. Despite their important role, conventional rehabilitation methods often struggle to maintain motivation due to their repetitive nature, leading to potential social and mental health repercussions. This work advocates for the integration of Virtual Reality (VR) serious games in a collaborative, multi-user setting, informed by a Human-Centered Design (HCD) methodology with stroke survivors and healthcare professionals from a rehabilitation center. Through immersive, task-specific exercises in dynamic virtual environments, we strongly believe that stroke survivors can engage in shared experiences, fostering motivation and social interaction. We report our initial efforts toward the creation of a collaborative VR framework and delve into the next steps of the proposed research line. With this, we hope to transform existing approaches and better complement traditional rehabilitation methods for improving the overall quality of life for stroke survivors.

Bernardo Marques, Sérgio Oliveira, Hugo Ferreira, Paula Amorim, Paulo Dias, Betriz Sousa Santos

Conference Proceedings

Aligning digital twins and metaverse with the UN SDGs and applying them to understand human behaviour in smart and virtual cities.

Smart Cities have a strong environmental and social impact. Their digital operations eliminate emissions and accessibility challenges and contribute to the development of sustainable communities, economic growth, responsible production and consumption, climate actions, and more. However, since smart cities remain a vision without any such city being fully built yet, it is important to understand and test human behaviour, performance and well-being when functioning in digital environments. Metaverse technology can provide such information and operate as a test environment in future adaptations of smart technologies in the physical world. This paper explores the impact of metaverse in the built environment and the smart cities, highlights the technologies involved in such a digital ecosystem, the user’s behaviour in smart cities, and their contribution to the UN Sustainable Development Goals (SDGs). Furthermore, this work presents an initial process framework for the identification of the smart cities' functional characteristics and minimum technological applications that can relate them to specific SDGs.

Evangelos Markopoulos, Panagiotis Markopoulos, Akash Nandi, Kui Zhao, Mascha Samkova, Tong Wu, Jussi Kantola

Conference Proceedings

Adaptive Human-Machine Interfaces and Inclusivity in the Automotive Field: A Review

With advances in automated driving, the importance of the vehicle interior continues to grow. However, the growing number of functions leads to a large amount of information on automation or infotainment systems. To create an individual customer experience and not to overwhelm the driver with the growing amount of information, an adaptive human-machine interface (HMI) can be a solution. (Amditis et al., 2011) With the help of adaptive HMI, it is possible to display contextual and user-specific information and to adapt the interaction with the vehicle. Inclusion is a form of adaptation focusing the needs of the respective users. Thereby, adaptive HMI addresses, among other things, the enhancement of driving safety and empathy in the vehicle. (Rittger et al., 2022) This paper provides an overview of the current state of the art with respect to adaptive and inclusive HMI in the automotive context and facilitates the identification of open research questions. A systematic literature search was conducted to facilitate this scoping review.

Lena Wirtz, Emirhan Sever, Lutz Eckstein

Conference Proceedings

Exploring the Relationship Between Acceptance and Usage Intention of Smart Mobility Applications and the Mobility of Older Adults: A Systematic Review

In recent decades, rapid advancements in science and technology have raised awareness of the potential benefits that smart technology may bring to transportation. It is increasingly recognized that the integration of new technical services, grounded in information technology, can enhance the independence and autonomy of older individuals in their transportation choices, thus supporting active aging. Smart mobility applications, as an innovative solution, offer an integrated digital platform that simplifies travel by merging various services and providing real-time hazard detection, thereby enhancing travel flexibility, convenience, and safety. However, there is a clear gap in research focusing on applications specifically designed for older users. This paper aims to identify the critical factors that influence older users' acceptance and intention to use smart mobility applications, as determined through a systematic literature review. The findings indicate that while older adults generally hold positive attitudes towards smart mobility apps and demonstrate a willingness to learn, this does not significantly alter their travel habits. The adoption of smart mobility applications by older users is shaped by four principal factors: personal characteristics, travel habits, technological literacy, and age-related psychological aspects. Urban older adults, particularly those with higher education, are more inclined to utilize these apps, in contrast to their counterparts in suburban or rural areas who may lack awareness of such technologies. The diversity in older adults’ preferences is crucial in determining their willingness to engage with smart mobility apps. Additionally, the promotion of these apps, coupled with government support for digital platforms, plays a significant role in influencing older users' intentions to use them. This study offers essential insights for stakeholders in city-building, including urban and transport policymakers and programmers, to better develop age-friendly smart mobility applications tailored for an aging society.

Chengyuan An

Conference Proceedings

User Study: Generative AI-based scientific writing assistant for students with visual impairments

This paper investigates the potential of AI-based writing assistants to help students with visual impairments in writing scientific texts. In a user study, ten visually impaired students tested a prototype for a multimodal Writing Assistant application, built on GPT-4, which specifically aids in the scientific writing process by providing templates, answering questions about the text, and providing automated editing and formatting. Their interactions with the prototype were compared with their usual writing process to identify factors that facilitate the writing task of visually impaired users. Results indicate a unified tool was most beneficial to improve the writing process. Leaving the user in control of options and informed on changes was also identified as good practice. University support and guidance appear as conditional to the adoption of such tools.

Alireza Darvishy, Stefano Anzolut, Oriane Pierrès, Felix Schmitt-koopmann

Conference Proceedings

About ride comfort due to differences in running speed of manual attendant-controlled wheelchairs

One of the problems with wheelchairs is that the vibrations generated during driving can cause users to experience motion sickness, discomfort, and annoyance. Wheelchair tire pressure is considered to be one of the factors that affects the ride comfort of a wheelchair. However, it is not yet known how much influence the tire pressure of a wheelchair has. One of the reasons for this is that it is not possible to accurately measure tire pressure with the English valve used in wheelchairs. Therefore, we created a tire pressure indicator to accurately measure the tire pressure of wheelchairs. The purpose of this study is to use the tire pressure display to evaluate the impact of changes in tire pressure on assistive wheelchairs on vibration. In addition, in this experiment, we created a device to push the assistive wheelchair using an electric wheelchair in order to make the assistive wheelchair move at a constant speed, aiming to improve the reliability of the data. In this paper, we examined the effects of different running speeds on ride comfort.In this study, we use a triaxial accelerometer to measure the vibration of the tire pressure of a manual attendant-controlled wheelchair in four patterns: 80kPa, 160kPa, 240kPa, and 320kPa. This measurement is performed at three different running speeds: low speed, medium speed, and high speed. FFT analysis is performed on the acceleration in three directions measured by the triaxial accelerometer, and the maximum power spectrum, frequency and integral value at the maximum power spectrum are determined. From these values, evaluate the trends caused by changes in tire air pressure. Verify the relationship with tire pressure. Riding comfort was evaluated using the SD method, and two types of evaluation were performed: stationary and running. The evaluation items consisted of three items, sitting comfort, sense of security, and sitting comfort of the buttocks, for both static and running conditions, and four items, including the strength of shaking while running. Responses were obtained on a five-point scale from 1 to 5. In this study, we verified the results of vibration measurements and sensory evaluations, and verified ride comfort due to differences in driving speed and tire air pressure.

Shoichiro Fujisawa, Shunsuke Jutori, Koutaro Ishimoto, Yitong Wang, Kaito Arashi, Jyunji Kawata, Jiro Morimoto, Yoshio Kaji, Mineo Higuchi, Toru Yamamoto, Masayuki Booka

Conference Proceedings

The Ethics of Immersion: A Scoping Review of VR and AR Technologies

Immersive technologies such as augmented reality (AR), virtual reality (VR), mixed reality (MR), and extended reality (XR) are rapidly evolving, presenting significant opportunities for enhancing user experiences. Nevertheless, their widespread adoption introduces substantial ethical challenges that necessitate thorough exploration. This research conducts a comprehensive investigation into the ethical considerations associated with these technologies, employing a methodical literature review and synthesis approach. Utilizing the PRISMA-ScR framework alongside the KJ method, this study analyzes 34 pertinent articles, uncovering 189 unique ethical concepts that are subsequently organized into 15 coherent categories. The analysis illuminates critical ethical concerns, including the protection of user rights within virtual spaces, the dichotomy between real and virtual worlds, the imperative for inclusive and equitable access, the necessity to ensure user safety and well-being, and the call for the establishment of robust ethical governance frameworks. The derived findings propose a future research agenda that accentuates the importance of navigating ethical dichotomies, enhancing inclusivity, advocating for user-centric development practices, instituting strong governance measures, and bolstering ethical awareness. By tackling these identified ethical dimensions, the present study seeks to promote a judicious and ethical advancement of immersive technologies, aiming to enable society to leverage their profound benefits responsibly while adhering to ethical standards and mitigating potential risks.

Matteo Zallio, Taihe Huang, Yutaka Osaki, Shiqi Hong, Xiaomin Chang, Wei Liu, Takumi Ohashi

Conference Proceedings

Exploring user preferences regarding facial interface expressions of service robots across multiple age groups: A case study of the Kebbi Air robot

The integration of artificial intelligence (AI) and remote communication technology has enabled development and application of AI robots beyond industrial use, with such robots being applied for household use and services. In such applications, the facial expressions of robots are crucial to information exchange between humans and machines. The ability of robots to subtly change their facial expressions and respond sensitively to the emotional states of humans is a key focus in the development of service robots.This study analyses the Kebbi Air robot, exploring the preferences of users across various age groups (young people, middle-aged adults, and older adults) regarding the facial interface design styles applied to service robots (flat vs. realistic design). It also analyses how different design styles affect user recognition of robot facial expressions portraying emotions. This study developed a set of recommendations pertaining to facial expression styles for the Kebbi Air robot. The study comprised 2 phases. In the first phase, 21 older participants from Zuozhen, Tainan, Taiwan, were recruited to participate in a questionnaire survey and interview to enable assessment of their preferences regarding facial interface designs for robots. In the second phase, the survey plus interview format was repeated to compare the age-stratified data collected from four groups of participants stratified by age (i.e., older adults, middle-aged adults, prime adults and young people).The results indicate that regardless of design style, the younger participants were generally more accurate in recognising robot facial expressions than the other participants were. Furthermore, they demonstrated a higher level of emotional recognition for expressions portrayed in the realistic style, and they expressed a greater willingness to interact with robot interfaces. On the basis of this study’s findings, qualitative suggestions were proposed for various age groups; these suggestions encompassed style recommendations for robot facial expressions (e.g., eyebrows, eyes, mouth, and auxiliary symbols). Through its empirical exploration, this study provides valuable insights and recommendations for designing robot-friendly interfaces for multiple age groups.

Chao Yang Yang, Chun-ting Wu, Yi Chi Fu, Hsu-hsu Lai

Conference Proceedings

The Impact of Digital Twins Technology in Maritime Fleet and Safety Management

The integration of advanced technologies in areas such as safety, navigation, maintenance, security, supply chain management, and environmental impact can enable shipping maritime companies to enhance operational efficiency, reduce costs, mitigate risks, and drive innovation in a highly competitive industry. This paper introduces the application of Digital Twins in the shipping and maritime sector, highlights its contributions to safety and fleet management with live ship monitoring on all operations and crew, and extends such applications with the integration of Virtual Reality and Artificial intelligence for immediate decision making, risk avoidance and corrective actions. Its concept is based on and follows the developments and improvements of SOLAS (Safety Of Life At Sea), the ISM (International Safety Management) and ISPS Codes (International Ship and Port Facility Security), and STCW (Standards of Training, Certification and Watchkeeping). Furthermore, this work analyzes user requirements from different actors in the shipping and maritime industry (officers, ship-agents, ship owners and ship managers, insurance companies, charterers, etc.) and creates a high-level operations framework to guide the design and implementation of successful maritime digital twin projects. Lastly, it highlights the relationship between the UN Sustainable Development Goals and the digital twin adaptation strategy in the maritime and shipping sector. The paper concludes with the impact and contributions of such intelligent and interactive Digital Twins technologies in the maritime and shipping sectors from operational and environmental sustainability perspectives.

Evangelos Markopoulos, Panagiotis Markopoulos, Akash Nandi, John Faraclas, Chris Leontopoulos, Stavros Meidanis, Dimitris Monioudis, Grigoris Filahtos

Conference Proceedings

Exploring the Willingness of Knowledge Workers to Use the Metaverse Office: A Qualitative Study

The metaverse, with its potential to provide immersive work environments, may emerge as an effective solution for telecommuting. However, the adoption and promotion of the metaverse office are slowed down due to the public's limited acceptance of this new technology. This study employed interviews (N = 8) and questionnaires (N = 12) to explore the willingness of knowledge workers to use the metaverse office and identify the main reasons influencing this decision. The findings indicate that while most participants hold a favourable view of the metaverse office concept, they concurrently acknowledge a significant disparity between its envisioned blueprint and its practical implementation. The reasons affecting the willingness to utilize the metaverse office primarily fall into three categories: the current development level of metaverse technology, the completeness of functions in the metaverse office platform, and the characteristics of the users. These insights provide guidance for the future advancement direction of the metaverse office.

Yuzhen Wang, Ruifeng Yu

Conference Proceedings

A human-computer interaction design solution for women's home nodule detection devices

Women exhibit a higher susceptibility to nodules compared to men, attributed to factors such as genetics, physiology, and the dual pressures of managing both home and work. A nodule, defined as a diseased mass in human tissue or organs with a diameter of less than 3 cm, necessitates thorough follow-up testing for early detection and cancer treatment. Notably, breast and thyroid nodules, predominantly located in superficial tissues, can be initially screened through palpation. Breast and thyroid cancers, the most prevalent solid malignant tumors among women, continue to show an increasing incidence, posing a significant threat to women's physical and mental well-being. Despite this, recent studies indicate that only a minority of women can afford the time and financial resources for regular medical checkups, primarily due to economic pressures and the apprehension and fear associated with healthcare .Experts recommend regular monthly self-examinations for women as a proactive measure to reduce the risk of cancer, improve treatment outcomes, and enhance personal health awareness. Commonly employed methods to assist women in refining their self-examination skills and overall experience include graphic awareness education, simulators, and mobile apps. However, due to the often small or deeply hidden nature of nodules, tactile detection by touch can be challenging for the average individual. Some experts are exploring detection technologies and home-based devices, such as wearable devices integrated with bras [9]. Yet, as the home health detection devices, a human-centered approach is essential, necessitating careful consideration of human-computer interaction design to optimize the user experience.Following user research and analysis, this study has developed a compact home nodule detection device to facilitate early detection. In consideration of ergonomics and users' familiarity with smart products, the self-test device is designed to be compact and easy to operate, catering to non-professional healthcare users in a home setting. Paired with a mobile app, the device offers testing guidance, feedback on results, and additional features such as symptom recording, reminders, and health education. This comprehensive approach enhances the device's functionality and user-friendliness. In comparison to previous auxiliary detection methods, this device provides a more comfortable human-computer interaction experience, significantly boosting users' willingness and confidence in self-testing. Overall, this solution addresses current challenges in nodule self-inspection,encouraging people to pay more attention to their health status.Main research contents:1.Establishing Design Foundations: conduct competitive analysis and user research to define the product design framework.2.Integrating Human Factors: utilize human factors engineering analysis to shape the product's appearance and style. Develop a design prototype based on these considerations.3.User Review: engage users in the evaluation of the design prototype to validate design feasibility.Main research methods:1.Competitive Product Analysis: conduct a thorough analysis of existing nodule detection products in the market, identifying strengths and weaknesses. Summarize findings to guide the design process of this study.2.Semi-Structured Interviews: engage in semi-structured interviews with users to explore the timing, specific scenarios, and preferences related to self-tests. Understand user demands regarding product appearance, functions, and interactions.3.Usability Testing: implement usability testing by inviting users to assess the accessibility, comfort, reliability, functional integrity, and overall satisfaction of the product. Use feedback to refine and enhance the product design in the future.

Kaiqiao Zheng, Jing Luo

Conference Proceedings

Leveraging Human Data for Avatar Action Recognition in Virtual Environments

In the dynamic metaverse landscape, avatars serve as digital embodiments, crucial in representing user interactions. This research investigates the dynamics of avatars within the metaverse, with a specific focus on their movements and actions in virtual environments. Real-world pose estimation techniques, notably OpenPose, are applied to establish a correlation between human and avatar movements. A Multilayer Perceptron model, trained on the NTU RGB+D dataset, proves highly effective in accurately classifying avatar actions, achieving an impressive accuracy rate exceeding 95%. The study explores the impact of different avatar types on action recognition, revealing notable performance differences. Notably, a half-body avatar with only the upper part demonstrates comparable performance to a full-body avatar, while the absence of head joints adversely affects accuracy. Additionally, the research assesses the generalizability of human data for avatar recognition, highlighting its superiority over models trained exclusively on avatar data. The findings underscore the adaptability of these techniques to diverse avatar configurations and offer promising advancements in action recognition within the ever-evolving metaverse. This research contributes valuable insights into effectively integrating real-world action recognition techniques to understand avatar behaviors in virtual spaces comprehensively.

Somaya Eltanbouly, Osama Halabi

Conference Proceedings

Unraveling Interaction Challenges for Deaf and Hard-of-Hearing Users: An Exploration of Digital Content and Interfaces Accessibility

Efficient engagement with ubiquitous digital content and interfaces presupposes a profound level of reading comprehension, a potential impediment for users within the deaf and hard-of-hearing community. Existing literature demonstrates a correlation between diminished reading comprehension and deafness [1, 3], accentuating the formidable barriers faced by this demographic in accessing and understanding digital content and navigating an increasingly digital world. Despite decades of efforts to enhance digital accessibility for all users, limited research has focused on the specific challenges confronting deaf and hard-of-hearing individuals, particularly those not adept with technology [2].This study aims to address this gap by investigating the daily interaction challenges faced by deaf and hard-of-hearing users with digital content and interfaces. Conducting a focus group with four deaf participants, we chose this method to facilitate rich, in-depth discussions in American Sign Language (ASL), a mode of communication familiar to the participants. We selected participants with educational levels from high school to those who had attained an Associate's degree seeking to get a range of possible challenges and investigating if their level of education might impact the challenges they face.Qualitative analysis of the focus group's dialogue unveiled insights into the technological struggles experienced by participants, coupled with their expressed needs for technological support. Three salient themes emerged: 1. Difficulties comprehending textual information across various tasks (email, online search, reading news, text messages, etc.), as unanimously reported by all participants; 2. Challenges associated with writing text and text entry, universally confirmed by all participants; and 3. Challenges in content and interface comprehension, influencing the installation, navigation, and utilization of various technologies, universally acknowledged by all participants.In this paper, we discuss the implications of these findings for researchers, technology developers, and digital content providers alike. We further explore avenues for future research along with potential technological enhancements aimed at mitigating the challenges confronting the deaf and hard-of-hearing population. The insights derived from this study are positioned to guide digital content developers, policymakers, and educators in effecting essential changes to amplify the accessibility of digital content and interfaces for deaf and hard-of-hearing users, both within the United States and on a global scale. References[1]M. Marschark and P. C. Hauser, Deaf cognition: foundations and outcomes. Oxford ; New York: Oxford University Press, 2008.[2]M. Maiorana-Basas and C. M. Pagliaro, “Technology Use Among Adults Who Are Deaf and Hard of Hearing: A National Survey,” Journal of Deaf Studies and Deaf Education, vol. 19, no. 3, pp. 400–410, Mar. 2014, doi: https://doi.org/10.1093/deafed/enu005.[3]E. Toofaninejad, E. Zaraii Zavaraki, S. Dawson, O. Poquet, and P. Sharifi Daramadi, “Social media use for deaf and hard of hearing students in educational settings: a systematic review of literature,” Deafness & Education International, vol. 19, no. 3–4, pp. 144–161, Oct. 2017, doi: https://doi.org/10.1080/14643154.2017.1411874.

Olarinde Farayola, Dastyni Loksa, Jinjuan Feng

Conference Proceedings

Investigation of Potential fNIRS-based Biomarkers in Multi-Domain Virtual Reality Tasks for MCI Assessment

Alzheimer's disease (AD) is a progressive neurodegenerative condition and is currently the fourth leading cause of death in advanced nations. The primary cause of AD is the deterioration of neurons in areas of the brain crucial for memory, typically presenting symptoms like loss of memory and a decline in cognitive abilities. Mild Cognitive Impairment (MCI) represents a transitional phase between normal cognitive health and AD. Recent studies have shown that within five years, 32% of individuals diagnosed with MCI experience a progression to Alzheimer's disease. Hence, the early detection and treatment of MCI are vital in decreasing the likelihood of developing AD. Traditionally, MCI assessment has relied on neuropsychological tests such as the Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA). Nevertheless, these methods have limitations, including inducing participant anxiety and fatigue, cultural biases, and the need for skilled administrators. This has shifted towards more innovative assessment methods, particularly Virtual Reality (VR) technology. VR's engaging and multisensory environment offers the potential for more effective MCI assessment. Various VR tasks, such as the Virtual Supermarket Task (VST) and VR adaptations of the Morris Water Maze and Trail Making Test, have shown promise in delivering insightful performance metrics. However, existing research has primarily focused on VR task performance evaluation, often overlooking the corresponding brain activation patterns these tasks stimulate. Compared with the task performance, the stimulated brain patterns could more directly reflect the cognitive function changes resulting from MCI. Whether these VR tasks can induce distinguishable changes in functional near-infrared spectroscopy (fNIRS) data between MCI and healthy individuals and which fNIRS parameters could be useful for MCI assessment is still unknown. To address this research gap, we investigated human brain activity across MIC and healthy individuals in multi-domain VR tasks. First, we selected a VR drumming task which engages multiple cognitive domains, including motor skills, rhythm, and spatial-temporal orientation. Second, we extracted some potential MCI indicators, such as functional connectivity from fNIRS data to analyse brain activity across MIC and healthy individuals in the VR task. Lastly, we examined the statistically significant parameters and discussed the underlying brain activity patterns and their potential for MCI assessment. Our findings revealed that specific brain activity and functional connectivity parameters indicated significant differences between healthy and MCI groups, suggesting the potential value of these parameters as biomarkers for VR-based MCI assessment. This study introduced the potential fNIRS parameters for MCI assessment and discussed their implications and underlying reasons. In conclusion, our study lays a promising foundation for developing and refining VR-based MCI assessments. We anticipate our findings will lead to more effective VR task designs and promote widespread MCI screening in larger populations, ultimately aiding early detection and intervention in individuals at risk of dementia. Future research should address the identified limitations and explore further enhancements in MCI and related condition assessments.

Yanjie Zhang, Fan Li, Su Han, Donglin Li

Conference Proceedings

Investigating the Use of Blockchain Technology in Fashion and Textile Industry

Nowadays, with the advancement of technology, software designed to provide perfect security for information in every field is emerging. Blockchain has also become a crucial component in such software, safeguarding information across various sectors. Blockchain technology represents one of the rapidly evolving methods for organizing and storing data online. Data is stored in blocks, and when enough blocks are connected, they form a chain, creating a blockchain. Additionally, the use of blockchain technology enhances the security of data on the network. This technology finds applications in various sectors, including media and telecom, energy and utilities, finance, healthcare, and basic industrial sectors. Notably, the fashion industry is also among the sectors benefiting from blockchain, primarily due to its transparency, a key factor given the increasing importance of sustainability.However, transparency is just one aspect of blockchain technology. It encompasses other essential features such as enhanced security, instant monitoring, increased efficiency and speed, and automation. Today, major players in the fashion industry, including LVMH, Nike, Zara, and textile-producing countries, employ blockchain technology. Blockchain facilitates the examination and recording of many issues, including supply chain management, product preparation, and fair labor practices. Luxury fashion companies, in particular, use blockchain to record product certificates, preventing the production of counterfeit models and ensuring the authenticity of their products.In this context, this study aims to reveal the perspectives of textile and fashion companies regarding blockchain, a topic with limited academic sources. As a case study, the research focuses on investigating the application of blockchain in the Turkish textile and fashion design industries. The research involves interviews with Turkish fashion designers and managers from textile companies, aiming to understand how and why businesses might potentially integrate blockchain internally. Although most interviewees stated that their companies currently lack active blockchain applications, they expressed their intention and progress toward incorporating blockchain in the near future.While blockchain is used worldwide, it is observed that Turkish fashion designers and the textile sector employ it for specific applications, and there is limited available information about Turkey. This study aims to bridge the knowledge gap by providing insights into how and why blockchain can be utilized in the Turkish fashion and textile sector, drawing from the data gathered through extensive research and interviews.

Simge Güner, Arzu Vuruskan

Conference Proceedings

Revitalizing Urban Village Environments through Digital Twin Technology: A Case Study of Nantou Ancient Town

The rapid advancement of the digital age has significantly expanded the potential applications of digital twin technology. However, urban villages, characterized by low socio-economic status and limited access to digital twin technology. Additionally, Prior studies often overlook structural factors like inadequate planning and infrastructure, hindering efforts to improve urban village living conditions.Consequently, this study aims to investigate the implementation of digital twin technology to enhance the environment of urban villages and address issues such as outdated infrastructure and inadequate living conditions.The purpose of this study is to create a virtual model of an urban village (Nantou Ancient Town) using digital twin technology (Unreal Engine). The model will use 3D geospatial and data visualization to visually present the living conditions of urban villages, including buildings, population density, water supply and green space. The aim is to provide practical insights for enhancing the overall environment and improving living conditions in urban villages through targeted infrastructure improvements and environmental upgrades.Therefore, this research is of significant importance in improving the environment of urban villages and providing more opportunities for accessing digital technology. Furthermore, it highlights the potential of utilizing digital technology to enhance living conditions in urban villages, based on the premise of promoting balanced development of basic infrastructure in urban villages, and it also strengthens community inclusivity and sustainable development in urban villages.In conclusion, this research has promising prospects in advancing the field of digital improvement of living environments by exploring the widespread application of digital twin technology like the Unreal Engine, especially in areas where digital technologies are lacking, and addressing existing research gaps. By uncovering the potential of these technologies, this study has the possibility to enhance the well-being of urban village residents through the utilization of technology, drive innovation, and shape the future of digital living environments.

Yangyang Pan, Yuanli Yu, Izzy Yi Jian, Yu Liu, Sky Tiantian Lo, Shuyi Cheng

Conference Proceedings

A comparative analysis of AR controller tracking technologies

Augmented Reality (AR) and Virtual Reality (VR) controllers utilize a combination of different tracking technologies to determine spatial orientation and position. These tracking technologies include IR/LED tracking, IMU (inertial measurement unit) tracking, and VIO (visual-inertial odometry) tracking. The IR/LED system utilizes infrared sensors and LED constellations for precision tracking, the IMU system relies on accelerometers for responsive mobility, and the VIO system combines cameras and inertial measurements for a balanced solution in dynamic environments. These technologies play a critical role in enabling seamless interactions within virtual environments by providing real-time data on user movement.The most sophisticated AR/VR controllers utilize a combination of these three technologies. While utilizing all three technologies together maximizes controller tracking` performance, it also increases production costs while introducing constraints to the form and design of the controller. For controllers used with AR systems intended for enterprise use, it is hypothesized that controllers that feature all three tracking technologies may be over-engineered as enterprise use does not typically involve the dynamic movements required for consumer use such as gaming.This study presents findings from a series of user studies that evaluates the strengths and differences among these tracking technologies when performing different types of controller interactions. The user studies focused on quantitative user performance differences between controllers with different combinations of tracking technologies enabled as well as differences in user perception for performance and comfort. Among the key findings that will be presented are a breakdown of the types of interactions that rely on VIO tracking and how user performance and perception is affected by not having VIO tracking enabled.

Mohammad Jeelani, Jose Muniz, Shaun Marsh

Conference Proceedings

Combining AR and ball-shape input interface to control remotely a robot-arm

Robot arm technology has revolutionized the manufacturing industry, offering worker safety and productivity benefits. However, issues like unintended actions and difficulty in intuitive operation remain. This study proposes a remote-control system for robot arms that combines augmented reality (AR) and a multi-sensor interactive smart ball (PALL0) from Ai2Ai Corp. The AR application allows operators to synchronize the control of a virtual and physical robot arm using their hand and the PALL0 held in the hand.The AR application operates the robot arm by matching the position of its head with the operator's hand and the posture of the head with the acceleration and angular velocity of the PALL0. A button on the PALL0 controls the opening and closing of the gripper, while the physical robot arm also operates. The start and end of the operation, as well as the return of the robot arm to its initial posture, can be performed using buttons in the AR application.Future research should compare the method of controlling robot arms using PALL0 with other methods, such as those that use the movement of the body's upper or lower extremities, and invest resources in building the robot's digital twin to improve system portability.

Harutaka Abe, Víctor Blanco Bataller, Henrik Terävä, Mika Luimula, Guillaume Lopez

Conference Proceedings

Mitigating Orbital Debris via Responsible System Design

The escalating space debris problem threatens satellite operations and space sustainability. This study proposes an intelligent debris removal spacecraft integrating reconfigurable surfaces, optimization algorithms, and ethical design principles. The architecture adapts to diverse debris shapes, ensuring precision targeting and capture efficacy. The development integrates sustainability criteria for responsible manufacturing and end-of-life deorbiting. Quantitative trajectory analysis and case studies of successful projects inform refined workflows. With machine learning enabling responsive debris characterization and risk probabilities alongside stakeholder participation driving conscientious design choices, the system pioneers a contingency-based approach to space preservation. The research underscores technical ingenuity balanced with environmental foresight as instrumental for the principled advancement of aerospace engineering for shared posterity.

Janne Heilala, Kalea Texeira

Conference Proceedings

Evaluation of Helicopter Performance Indicators for use in the Development of Digital Twin based on Physiological Sensor Data from the Aviator

Ongoing work within the military, as well as in the civilian sector, continues to pursue the goal of operator state monitoring (OSM) through physiological metrics. The overarching goal of this effort is to provide a near real-time objective assessment of an operator’s state that is predictive of performance degradation. It is anticipated that such an endeavor would result in a human digital twin system (HDTS) (Miller & Spatz, 2022), whereby the physiological data collected from the human operator would be used to model and predict operator states under different flight conditions, subsequently used to predict future performance. Creation of such a system will require large quantities of data that will need to be accumulated from a variety of flight conditions (i.e., weather) and types of maneuvers (e.g., hover, takeoff). Although significant progress has been made to-date, a key component remains undefined. We have yet to define what aspects of performance are critical to be predicted within such a system. To maximize the utility of OSM within an operational setting, detecting when performance is likely to degrade, or has begun to degrade, is essential for the system to make use of this information, ultimately engaging some form of adaptive automation – through an HDTS. While a plethora of work has been completed to-date to determine the performance parameters necessary for implementing various cues to the aviator or in some cases, automation (e.g., automatic ground collision avoidance system), much of this work has been done with a relatively narrow scope. Ongoing work, through literature review, is aimed at evaluating performance parameters to determine what aspects of aviator performance have been demonstrated to indicate adequate performance across different maneuvers. Alongside this effort, we have recently evaluated the effects of various cues (auditory and tactile) within a UH-60 full-motion simulator on different performance outcomes (Feltman et al., 2024). While manipulating these cues, we evaluated the physiological data and performance outcomes of 16 Army aviators. From this study, we measured approach performance using a parameter derived from one of the visual cues (vertical speed indicator). The performance outcome was the amount of time participants spent within the target range of this visual cue, which indicated adequate performance for the task. Using this as an outcome measure, we demonstrated that auditory and tactile cueing combinations had different effects on this measure, F(3, 297) = 3.76, p = 0.011. The physiological data collected concurred with the findings of performance data, where those with better performance on this outcome measure showed increased physiological arousal (increased heart rate, F[3, 118] = 8.02, p < 0.0001, decreased heart rate variability, F[3, 118] = 9.39, p < 0.0001). Taken together, the result of this study provides an initial step toward defining flight performance outcomes that correspond with changes in physiological parameters of the aviator. Ongoing work continues to evaluate various types of performance measures that may be used independently or in combination with others (to create an index) for the purposes of OSM/HDTS development.

Kathryn Feltman, Amanda Kelley

Conference Proceedings

A comprehensive digital twin for assessing feasibility and ergonomics of high-tension cable assembly operations

The focus of this study is on a comprehensive digital twin illustrating the assembly of a high-tension cable in an electric vehicle. This involves the integration of two software programs, IPS Cable Simulation and IPS IMMA (Intelligently Moving Manikin in Assembly), to simulate the cable assembly and analyze ergonomic factors. The study uses the vehicle geometry and task sequence of Stellantis on a hypothetical use case of an extreme assembly line with the vehicle raised and the operator performing the activity underbody. The simulation of the cable assembly analyses extension-compression and torsion behaviors, ensuring compliance with acceptable limits. Anthropometric variations among operators are considered using IPS IMMA's module to create a family of manikins representing the global Stellantis workforce. Ergonomic analysis with the EAWS tool identifies risk factors for operators of different anthropometries. Additionally, a comparison is made on task time between the MTM-UAS methodology and IPS simulation's biomechanically modelled time. Results indicate a successful integration of IPS Cable Simulation and IPS IMMA, creating a digital twin that accurately simulates the operator's task with the high-voltage cable. The study emphasizes the importance of biomechanical models in understanding issues related to reachability, incongruous postures, and their impact on task time, contrasting with predetermined time approaches. In the current landscape of electrified vehicles, where there is a noticeable increase in the dimension and number of the cables, and of autonomous vehicles, where packaging challenges arise from redundancy requirements and the growing reliance on virtual verification, comprehensive and realistic dynamic simulations will become increasingly vital.

Manuela Vargas Gonzalez, Maria Pia Cavatorta, Valerio Cibrario, Lidia Ghibaudo, Ludovico Rossi

Conference Proceedings

Haptic Perception with Artificial Tissues

Haptic perception is critical in Minimally Invasive Surgeries (MIS) such as laparoscopic and robotic procedures in which the field of view is limited and the haptic force feedback is distorted or not available. Alternative haptic feedback is rendered by visual elements such as forced tissue deformation or using physical haptic interfaces with augmented reality. These approaches normally need additional training, add-on devices, and maintenance. In this study, we investigate an affordable method for creating a multimodal training simulator that integrates augmented reality (AR), extended reality (XR), and realistic artificial tissues from available human CT data. For the physical artificial tissues our objectives are threefold: first, to objectively measure tissue or organ hardness using a durometer in Shore Units (SU); second, to efficiently produce tissues and organs based on reference SU values and CT data; and third, to create specialized tissues. Additionally, we aim to arrange organs and tissues according to CT data, exemplified by forming the Calot Triangle for cholecystectomy surgery training. Finally, experienced surgeons tested the artificial tissues and organs inside the realistic cavity for basic surgical operation and provided professional feedback.

Yang Cai, Talia Perez

Conference Proceedings

Validation of a 3D body scanner system with automated circumference measurement calculation while considering the influence of different 3D data types

Anthropometry is crucial in ergonomic designing to ensure accommodation of diverse body sizes and shapes. 3D body scanning systems are now widely employed in anthropometric surveys for collecting data, requiring validation according to ISO 20685-1 for ISO 7250-1 measurements. The aim of this study was to assess the validity of the 3D body scanning system, Vitus Bodyscan, in combination with Anthroscan, for automatically calculated circumference measurements specified in ISO 7250-1, while considering the influence of different 3D data types. Overall, 9 circumference measurements were collected from 44 adult subjects. For the manual measurement, ISO 7250-1 guidelines were considered. For the 3D scanning, each participant was scanned once in a standing posture. Thereafter, proprietary Anthroscan algorithms were used to generate a watertight mesh from the initial 3D point cloud and computed the circumference measurements automatically. With the exception of calf circumference, scan-derived measurements exceeded the acceptable error margins specified in ISO 20685-1 and were therefore not comparable to manual measurements due to several possible factors. Practitioners need to consider these variations when using the scan-derived values. Looking at the differences between scan-derived measurements based on point clouds or watertight meshes, the deviations were negligible for most circumference measurements, except for arm circumference with better results based on point clouds. For simplicity, the authors recommend the use of point clouds for all circumference measurements studied. However, the recommendations are based on the anthropometric measurements and the 3D body scanning system used in this study and cannot be generalized.

Alexander Ackermann, Thomas Jaitner, Sascha Wischniewski

Conference Proceedings

Fuzzy Ergonomic Expert System for Assembly Line Design Problem

In the era of Industry 5.0, prioritizing ergonomics in manufacturing systems is crucial. Assembly Line Balancing Problems (ALBPs) are integral to efficient manufacturing, optimizing lines to eliminate bottlenecks and enhance productivity. Recent developments emphasize Ergonomic ALBPs (Ergo-ALBPs) and the integration of Human Factors and Ergonomics (HFE) to address ergonomic risks. A research gap exists in applying ergonomic considerations in the design phase known as Ergo-ALDPs as corrective ergonomic interventions cost significantly more than preventive measures taken during the design phase. This study presents a novel approach, the fuzzy Ergo-ALDP, which extends the Ergo-ALBP to handle imprecise task times and ergonomic risks in the design phase.It introduces a fuzzy ergonomic expert system, utilizing fuzzy logic and Digital Human Modeling (DHM) to simulate worker interactions in assembly line optimization. The proposed fuzzy Ergo-ALDP addresses this gap with a constructive heuristic integrated with fuzzy logic, emphasizing feasibility. Our research introduces a unique fuzzy ergonomic assessment method to evaluate task, workstation, and assembly line ergonomics using an expert system. We validate this approach using one synthesized numerical instance. This research contributes to assembly line optimization, aligning with Industry 5.0's human-centric vision. The comprehensive fuzzy ergonomic assessment model bridges gaps and optimizes ALDPs under uncertainty, promising improvements in productivity, worker satisfaction, and operational efficiency. By addressing the intersection of ergonomics, uncertainty, and assembly line optimization, this paper significantly contributes to advancing the field and promoting a safer and more efficient manufacturing environment.

Elham Ghorbani, Samira Keivanpour, Firdaous Sekkay, Daniel Imbeau

Conference Proceedings