Human Interaction and Emerging Technologies (IHIET-AI 2024): Artificial Intelligence and Future Applications

AHFE International

Accelerating Open Access Science in Human Factors Engineering and Human-Centered Computing

Human Interaction and Emerging Technologies (IHIET-AI 2024): Artificial Intelligence and Future Applications

book-cover

Editors: Tareq Ahram, Redha Taiar

Topics: Artificial Intelligence & Computing, Human Systems Interaction

Publication Date: 2024

ISBN: 978-1-958651-96-4

DOI: 10.54941/ahfe1004549

Articles

Acknowledging tacit knowledge: Outlining participatory workshops in a human-centered design process

The energy transition presents complex challenges that require multidisciplinary insights and innovative solutions. However, traditional research methods often overlook valuable tacit knowledge, hindering effective contributions from relevant social groups. To counter this, the inter- and transdisciplinary research project MEnergy – My energy transition, funded by the German Federal Ministry for Economic Affairs and Climate Action, aims to create inclusive environments where the citizens’ perspective and experience is acknowledged, unlocked, and visualized. Based on the Social Construction of Technology (SCOT), the project aims to harness this knowledge, including gaps in knowledge, in analog, digital, and virtual communication formats to inform citizens about the energy transition and its technologies. This is done in three incremental design cycles over three years. This paper focuses on the first design cycle, which identifies participants' existing knowledge, attitudes, and emotions toward the energy transition. This is achieved in two consecutive cocreation workshops with different target groups. The participants' tacit knowledge and ability to imagine positive energy transition narratives is captured in the first workshop. Through storytelling, visualization, and collaborative ideation, participants articulate their tacit knowledge and provide insights based on their experience and expertise. Building on this, the second workshop is designed with modular interactive learning units to capture and increase the participants’ dimensions of knowledge (What can I do?) and willingness to act (What will I do?). Visual tools and collaborative techniques facilitate sharing and representation of tacit knowledge, improving the researchers’ and participants’ understanding of challenges and solutions. The data collected, both material artifacts and observation logs, is analyzed and mapped.The findings suggest an urgent need for low-threshold, local opportunities for citizens to learn about the potential of renewable energy. Furthermore, many citizens struggle to visualize a successful energy transition and have instead internalized narratives of sacrifice and prohibition. Here, scientists and policymakers are challenged not only to develop long-term sustainable solutions but also to communicate them in an accessible way.

Anne Karrenbrock, Laura Brendel, Laura Popplow, Valerie Varney

Conference Proceedings

LIFANA – User-centered design of a personalized meal recommender app for the elderly

As the global population continues to age, the healthcare and technology sectors have witnessed an increased interest in understanding and addressing the unique needs of the elderly population. The health challenges faced by the elderly are multifaceted and often interconnected. Age-related cognitive decline can impair memory and decision-making abilities, while physical frailty, with reduced muscle strength and bone density, raises the risk of falls and fractures, affecting their quality of life. Comprehensive healthcare strategies that focus on prevention, early intervention are crucial to addressing these challenges.The general purpose of the LIFANA nutrition solution was to support healthy nutrition through all phases of aging in elderly people, from active seniors to elderly users and patients in need of daily care. The LIFANA mHealth application provided personalized meal plans that helped users to prevent undernutrition or overweight and followed WHO recommendations regarding caloric intake based on height, weight, gender and age as well as measured physical activity. The weekly meal plans maintained a balance between e.g fish and meat and plant-based dishes, ingredient variety, and macronutrients, and considered individual food restrictions. The LIFANA solution distinguishes itself from conventional food logging applications by providing users with seamless access to personalized meal plans that account for factors such as food restrictions, ingredient variety, macronutrient balance, and dietary preferences. For active users (60+) the solution integrated physical activity level measurement to balance caloric intake using the GoLive wearable clip provided by the Dutch project partner Gociety Solutions. Aimed towards users with limited mobility, the LIFANA prototype also integrated a digital shopping list with grocery home-delivery services during the field trials in the city of Porto. This service was provided by our Portuguese retail business partner MC Sonae to streamline the whole process of planning meals and acquiring food products.The target group for LIFANA differs from the general population in many aspects, i.e., cognitive and physical abilities, experience with smartphones, and eating behaviours. There are also considerable differences in food culture within Europe. For these reasons, we implemented a User-Centered Design (UCD) process with two iterative cycles of three phases: i) understanding the needs of the target group, ii) conceptualizing: specifying use cases and details of the technology, and iii) testing the system for usability and the assumed benefits. To facilitate understanding of the user’s needs, 37 stakeholders (seniors aged 65+, health professionals, and informal caregivers) participated in focus groups and co-creation workshops in The Netherlands (NL), Portugal (PT), and Switzerland (CH). The insights gathered from these sessions informed the creation of 7 personas and related scenarios, offering our developers a better understanding of the target users and their needs, considering factors such as age, gender, education, disabilities, and food preferences. Further, we added examples of typical meals for the age group in NL and PT. In the conceptualization phase, the functional requirements for the LIFANA solution were defined, considering the specific interests of all consortium partners, and a first prototype was implemented based on the personas defined in the first phase. The testing phase started with usability evaluation studies to validate that seniors can solve typical tasks. After successful request of ethical approval from the responsible national authorities, the LIFANA solution was then tested in a first round of short field trials in PT and NL. The feedback from the trials was used in a second iteration of the process to refine the requirements and develop a second prototype, which was then tested again in larger, long-term field trials in PT (53 participants, 14 months duration) and NL (107 participants, 3 months). The trials assessed various aspects including user friendliness and -acceptance, as well as changes of health-related aspects, determining its effectiveness regarding the improvement of markers related to health outcomes, and well-being, as well as potential effects related to budgeting. The results show a neutral effect regarding the participants’ anthropometric measures and blood pressure, while their motivation over time somewhat decreased in Portugal, though more pronounced in the Netherlands. This paper introduces the relevance of dietary applications for the healthcare system and the main objectives of the project LIFANA and its consortium. It provides insights into the results of the focus groups and shows two examples of the personas that were developed from them. Further, the results of the field trials are briefly presented. The paper closes with a discussion of the experienced benefits and challenges of the UCD process.

Christoph Stahl, Krizia Ferrini, Torsten Bohn

Conference Proceedings

User Preferences for Smart Applications for a Self-Powered Tabletop Gestural Interface

Nowadays, technologies and applications have become remarkably widespread and versatile. This versatility lies in their ability to be adapted and controlled by users through different kinds of interfaces. A self-powered tabletop gestural interface presents a significant technological advancement in a sustainable manner, resulting in energy-harvesting technologies that alter the concept of power consumption. However, determining the applications for some gestural interfaces to align with users' requirements and preferences requires further consideration. To this end, we conducted a survey and a semi-structured interview through a user-centred design (UCD) approach with 20 end-users in pairs to increase engagement between participants in selecting a smart application for a self-powered gestural interface. We presented the self-powered gestural interface with eight different applications and domains determined from the literature. Our findings indicated that users may find one application more beneficial than the other based on the quantitative data. In addition, our qualitative investigation focused on the reasons behind the ratings, where we used open coding from participant interviews and conducted a thematic analysis that revealed three high-level themes: touchless gestures, user experience, and other applications. Ultimately, the combination of users' feedback from quantitative and qualitative means led us to capture the selected application by users and build on it in future research.

Nora Almania, Sarah Alhouli, Deepak Sahoo

Conference Proceedings

Conceptual voicebot in the context of a passenger information system in an automated bus

Automated driving is seen as an essential part of the mobility of the future. This does not only affect the private sector but the public one as well. One consequence of this automation, in the public sector, is, that there won’t be any vehicle driver. This will lead to a need for additional systems, taking over further tasks on top of driving, which were previously executed by the vehicle driver too. One of those tasks is to answer the passengers’ questions, e.g. regarding future stops or alternative routes. This is particularly important in the bus sector. In this sector, there is greater passenger uncertainty, because the vehicle is not bound to fixed routes, due to the use of public roads, like in the case of a (suburban) train. This usage can lead to route changes, caused by e.g. road works or traffic jams. Consequently, an automated vehicle needs to be able to answer questions asked by the passengers. In order to address the passenger’s uncertainty, these answers should be easy to understand and personalized/ fitted to the questions asked.In this paper, three different approaches to automatically answer questions in the context of an automated bus are proposed. These approaches are 1.) a rule-based system, 2.) a system based on a large language model (LLM) like GPT-4 or LaMDA, and 3.) a hybrid system of rules and an LLM. The different approaches are being conceptualized and discussed. In the scope of the conceptualization, requirements as well as further challenges are derived. The discussion focuses on the capabilities to answer questions correctly and handle different languages as well as bad language. Additionally, the remaining challenges are further addressed. This includes e.g. handling emergency calls, vandalism reports, and the distribution of responsibilities regarding and the interaction with a passenger emergency management system. Following the discussion, the arguments are used to rate the approaches. Using this rating, the hybrid approach seems to be the most suited one. The reasons for this conclusion are, on the one hand, the capability to restrictions using the rules, including a defined course of action in certain situations, and on the other hand, the LLM’s ability to answer in a natural manner. Lastly, possible extensions for the hybrid approach are discussed.

Benedikt Haas, Eric Sax

Conference Proceedings

Team Effectiveness of Higher Education Students through Project-based and Agile Education for Sustainable Development

Agile methodologies derived from software development are today successfully applied in educational contexts as they provide a robust description of process workflows in team-based learning. It appears that the Agile Scrum model enhances team effectiveness, particularly focusing on teamwork and team interaction, which indicate effective collaboration. Agile methodologies further help learners develop sustainability competences because they reflect the current fast-paced and dynamically changing processes of our world. As Higher Education (HE) students are at the threshold of their careers, it is essential to strengthen soft skills beyond their academic subject. On these grounds, this study attempts to evaluate team effectiveness through the participation of 63 higher education students in an e-course, designed and developed according to the processes of the Scrum model appropriately embedded in Project-Based Learning (PjBL). Students were divided into small teams to provide sustainable smart solutions for their city, contributing to UNESCO's Global Goal 11 for Sustainable Development. Team effectiveness was assessed employing the "Big Five" framework. A questionnaire of 18 questions (Likert scale 1-5) was distributed to the students after the processes of the Agile Scrum. Initial results appear to be encouraging and motivating for authors to conduct further research.

Vasiliki Karampa, Foteini Paraskeva

Conference Proceedings

Fostering Student Creativity: A Curriculum Proposal in the Context of "Innovation and Entrepreneurship" and "Double Reduction" Policies

In today's fast-paced society, innovation is the engine of progress, and education must evolve to foster creativity and practical skills in students. Traditional exam-focused systems hinder creativity while existing maker education inadequately nurtures innate creativity. The "Empathy Design Thinking" plan addresses these gaps by combining research and pedagogical methods, moving away from exams, and implementing diverse teaching strategies. It also focuses on equipping teachers with the right resources and training. To validate its effectiveness, a 12-week pilot will monitor 60 students, comparing their creativity before and after the program using the Williams Creativity Tendency Scale. Another group of 60 students will serve as a control for comparison. With institutional support from Beijing Normal University, the plan aims to not only enhance student creativity but also fine-tune the "Empathy Design Thinking" methodology for broader implementation.

Ruonan Huang, Bowen Li, Jingjing Zhang, Takumi Ohashi, Y Chen, Jing Ma, Zi Wang, Mengyi Zhang, Ruilu Yu, Dan Qiu, Yiyang Liu, Z Zou, Wei Liu

Conference Proceedings

Effects of Digital Art Emotional Expression Therapy on the Rehabilitation of Children with Autism Spectrum Disorder

Evaluating the Effectiveness of Digital Art Emotional Expression Therapy in Combination with Conventional Rehabilitation to Relieve Autism in Children. Methods: Thirty Children with Autism were selected and randomly divided into the Experimental and the Control Groups, with fifteen children in each group. In the Control Group, the children with Autism Spectrum Disorder were given regular Psychological Rehabilitation Training; in the Experimental Group, they were given eight weeks of Digital Art Emotional Expression Therapy in addition to the regular Psychological Rehabilitation Training. In the Experimental Part, the Autism Treatment Assessment Scale and the Childhood Autism Rating Scale were used to score and compare the thirty children to verify their validity. Conclusion: The result showed that Digital Art Emotional Expression Therapy combined with conventional Psychological Rehabilitation Training could enhance the Rehabilitation Effect of children with Autism, alleviate their Autistic Symptoms, and improve their ability to interact with others and their language expression.

Zhenyu Liu, Cheng Yao, Fangtian Ying

Conference Proceedings

Enhancing Worker Efficiency and Reducing Cognitive Workload Through Assistive Assembly: A Proof-of-Concept

Nowadays, fast-paced and competitive industry, assembly tasks demand significant levels of concentration, precision, and cognitive effort. These challenges often result in mental fatigue, errors, and decreased efficiency among workers. The concept of Assistive Assembly offers a promising solution, harmonizing human dexterity with cutting-edge technology. By integrating ergonomics and robotics, Assistive Assembly has the potential to provide invaluable support to assembly line workers, enabling them to achieve peak performance effortlessly. This study presents a proof-of-concept approach for a future Assistive Assembly, simulating this condition with human-human collaboration. This preliminary step aims to support a human-centered design for an assembly workstation comprising a human worker, a collaborative robot, and a video camera system. A total of 25 participants were recruited to perform a simulated window assembly task under two conditions: Assistive and Non-assistive. The NASA Task Load Index (NASA-TLX) and the number of errors committed were measured. The subjects reported significantly lower perceived cognitive workloads in the assistive condition. Related to the number of errors, a significant difference in median test scores between the two conditions was found, meaning a decrease in errors registered in the assistive setting. Although these preliminary results are promising, further development and testing are essential to refine the Assistive Assembly concept within collaborative robotics settings.

André Cardoso, Estela Bicho, Ana Braga, Carla Alves, Luís Louro, Duarte Fernandes, Pedro Arezes, Ana Colim

Conference Proceedings

Designing a Positive Initial Experience with a Companion Pet Robot for Older Adults in Kuwait

Social robots have been increasingly integrated into social environments for older adults, such as care homes. Despite this increase, it is imperative to understand how individuals perceive various robot technologies. The initial experiences could shape users' perceptions and have a lasting impact on attitudes and behaviours towards robot technology adoption. This research investigates the design of a positive initial experience with a companion pet robot for older adults in Kuwait through a user-centred design (UCD) approach. To explore user perception and preferences regarding robots, we conducted a semi-structured interview with eight older adults, during which participants engaged with an off-the-shelf companion pet robot (JoyForAll Cat), took part in two main design challenges, and completed a Godspeed questionnaire to assess their perception. We presented our findings through thematic analysis and identified five high-level themes: user impression, design and functionality, social interaction, shared context, and using desire. We also presented the Godspeed findings, showing that introducing a companion pet robot has caused significant changes in older adults' perceptions. The results imply that designing a visually appealing and interesting initial experience could improve Kuwaiti older adults' acceptance of and relationships with companion pet robots, enabling them to satisfy users' needs for emotional support and companionship, contributing to overall well-being.

Sarah Alhouli, Nora Almania, Deepak Sahoo

Conference Proceedings

APPEND: A Blockchain-Based Model of Digital Product Passport for Furniture Industry

The digital product passport has been introduced as a policy instrument to enable traceability throughout the product life cycle and to support a circular economy. Anyhow, as a relatively new concept, there are a lot of uncertainties throughout the industrial landscape regarding the challenges and opportunities it brings. One such branch of industry is the furniture sector. In this paper, we present a DPP solution for furniture sector based on blockchain technology and inspired by a design thinking mindset. Through a prototype implementation, we highlight the key aspects such as data governance, stakeholder constellation, data interoperability, and data integrity that have significant research potential. Furthermore, we discuss these key concepts of DPP through the lens of eco-design principles in order to promote sustainability, improve energy efficiency, and protect the environment.

Fisnik Dalipi, Arianit Kurti, Mexhid Ferati

Conference Proceedings

Vibration Discomfort of Hand-Arm Vibration influenced by Energy Content, Frequency, Waveform and Exposure Duration

The ergonomics and user satisfaction when using technical systems, such as power tools like hammer drills, are greatly influenced by the perceived vibration. So far, the harmful effects of vibrations, based on frequency weighting equivalent to DIN EN ISO 5349-1, have been focused in the context of hand-arm vibration. However, the sense of discomfort caused by hand-arm vibration has not been considered so far. To address this research gap, a study was conducted using a full factorial experimental design with two levels of factors. The study aimed to explore the influence of energy content, frequency, waveform, and exposure duration on perceived discomfort caused by hand-arm vibration. Six subjects evaluated 16 different vibration patterns on a CP50 scale. To minimize potential confounding variables, the subjects were provided with blindfolds and in-ear hearing protection in the form of noise-cancelling headphones with white noise. Significant correlations were found between vibration discomfort and the factors of energy content, frequency, exposure duration, and waveform. These findings provide a solid foundation for further research on perceived discomfort caused by vibration, and subsequently, for enhancing the ergonomics and user satisfaction of technical systems.

Diana Fotler, Klara Lorenz, Sven Matthiesen, Simon Saurbier

Conference Proceedings

Flight Simulation In Geography Teaching: Experience Reports In Two Scenarios

There is an increasingly innovative range of resources in education, seeking to create a motivating environment for learning. The aim of this work was to present experience reports on the use of a flight simulator in Brazilian and biblical geography classes in an elementary school and a reformed Christian theology church, respectively. Microsoft® Flight Simulator was used for this purpose. Two educational scenarios are presented here: (1) teaching of geographical aspects of Brazil for 10-years old students; and (2) teaching of biblical aspects for 8 to 10-years old children from a reformed Christian theology church. The classroom was prepared to simulate an internal airplane environment. First, in the elementary school scenario, students could learn about Rio de Janeiro, Niterói (school city), the Amazon rainforest, Brasília (Brazilian capital), Pantanal and the southern region of the country. On the other hand, in the church scenario, children were able to have a bigger picture about Egypt, Sinai desert, Dead Sea, Jordan river, the Sea of Galilee, and other important biblical sites in Palestine, providing a rich opportunity to learn the main stories of the Old and New Testaments. Children approved the use of technology to assimilate the content, and further projects are intended to present this application for adults.

Christiano Bittencourt Machado

Conference Proceedings

A cross-cultural contextual study of Wong Kar-wai’s films driven by online reviews

With the development of science and technology, the distance between people in space is getting smaller and smaller, and economic globalization has been a historical inevitability. At the time of accelerating globalization, the film and television industry, as a part of the economy, cannot escape the impact of globalization. Moreover, compared with other industries, film and television works often carry more cultural connotations. How should filmmakers cope with the cultural differences in storytelling, create cross-cultural works, and get the same or different word-of-mouth feedback from different cultural groups? In the current multi-cultural context, Wong Kar-wai can be said to be one of the representative figures. In his films, From "Carmen", "Ashes of Time" and "Days of Being Wild" to "Chungking Express", "In the Mood for Love" and "The Grandmaster", Wong Kar-wai has been widely understood and accepted in different cultural and social contexts. This article start with online comments and study the elements of cross-cultural film creation in the context of globalization according to data mining technology. Research methods and processes: Firstly, data acquisition and pre-processing are carried out. The data source of online comments needs to be determined before crawling the review data. For the research objects selected in this study, the relevant sections of online movie rating websites, online movie platforms and social attribute platforms with large audiences were chosen as data source. For example, metascore has a weighted average of many movie reviews from well-known critics, which is highly reliable. Then, the key step in text mining is word segmentation. It's the process of breaking a sentence into several words. Accurate word segmentation can improve the efficiency and accuracy of subsequent text mining. After the word cloud is established, attribute features of the comments are extracted and classified, and the factors that highly affect the acceptance and empathy of Wong Kar-wai's film works in the cross-cultural context are summarized. Finally, combined with the study of his film scenes and narrative structure, the research focuses on the following factors:1.Theme selection: Explore universal themes, such as love, loneliness, loss and dreams. These themes are cross-cultural, and people face similar issues and emotions across cultures.2.Narrative structure: Such as non-linear narrative structure, which not only increases the complexity of the film, but also provides the audience with space to think and interpret, so that audiences from different cultural backgrounds can interpret the film in different ways.3.How emotions are expressed: Wong Kar-wai's films emphasize the emotions and inner world of characters, which is a universal emotional experience that can touch people's emotions across cultural boundaries.4.Character complexity: Complex and multi-dimensional characters with unique personalities, desires and inner conflicts enable the audience to immerse themselves in understanding the inner journey of these characters, regardless of their cultural background.5.The beauty of cultural elements: Different cultural elements, such as the backgrounds and traditions of China Mainland, Hong Kong, or elsewhere. Although these elements sometimes have a specific impact on the story, they are usually presented in an aesthetic and emotional way that allows the audience to enjoy the film without knowing the cultural details.6.Aesthetic of sight and sound: Films have a distinctive vision and sound style. Create memorable movie atmospheres. This visual and aural allure can transcend the language barrier and touch the hearts of the audience.In a word, the reason why Wong Kar-wai's films can be accepted and understood by people in cross-cultural contexts will be explored in a more specific way along the above directions, and the methods of film creation will be explored under the background of multi-context globalization.

Hanzhang Dong

Conference Proceedings

Evaluating Human – AI Design Interaction From the Perspective of Cognitive Abstraction

Together abstraction and iteration are two foundational cognitive processes used by designers to expand intellectual experimentation, realize innovation, and construct theoretical positions and formulate generic models. Nersessian’s work on conceptual change highlights abstraction and the role of non-specificity in the formation of generic models, understood to be mental abstractions of a system or relationships seen in objects and dynamic processes. (Nersessian, 2005) we analyze the way small groups of designer structure data inputs for AI tools vizcom and Midjourney to rapidly materialize futuristic visualizations. The construct of consistency is used to evaluate the designers creative process utilizing AI tools and examine future applications, scenarios, design strategies, solutions, and futuristic product feasibility. The aim of this study is to identify degrees of interface abstraction in prompt inputs and AI visualization outputs that may indicate the formation of generic models characterized by non-specificity, understood here as abstraction of a design intent. The ubiquity of iteration in design projects also highlights its importance in the design process. A second aspect of this study is the way designers decompose, integrate or revise semantic structures of prompts for Human-AI integrated workflows to improve design visualization outcomes. Human-artificial intelligence (AI) visualization outputs are aligned to prompts and sketches, and examined to identify levels of non-specificity (expression of abstraction) using the domain constructs of spatial structure, social quality, and spatial quality, to indicate the possible present of a generic representational model. Nersessian, Nancy. 2005. Abstraction via generic modeling in concept formation in science, in Martin R Jones, Nancy Cartwright, Idealization XII Correcting the model: Idealization and abstraction in the sciences. Pozanan’ Studies in the Philosophy of the Sciences and Humanities, Vol. 86. Amsterdam Brill.

Deborah A Middleton

Conference Proceedings

USID - Unsupervised Identification of the Driver for Vehicle Comfort Functions

Modern vehicles must meet the rapidly increasing customer requirements for in-vehicle comfort. Moreover, the consumer’s demand for personalized comfort functions keeps growing accordingly. Vehicle comfort functions are designed to ensure comfort with a more pleasant and convenient driving experience, including thermal comfort, seating comfort, noise reduction, and entertainment systems. They are highly customized for individual drivers to enhance their effectiveness. To achieve precise adjustment for each driver of a vehicle, it is imperative to identify the driver accurately.The distinctive signature of each driver’s driving style is embedded within the time series data gathered from various vehicle sensors. Therefore, there is no need for integrating novel driver identification sensors, as the identification task can be accomplished by utilizing existing and standardized vehicle data from the On-Board Diagnostic II system (OBD II).This research paper investigates the viability of precise driver identification based on unsupervised machine learning methods. For this purpose, the USID (Unsupervised Identification of the Driver) concept is introduced. Existing driver identification systems are functional, yet they exhibit limitations in precision. For example, a vehicle could detect the presence of a driver’s smartphone but fail to distinguish whether the individual is the actual driver or a passenger. In contrast to the state-of-the-art, the USID approach proposes a seamless and precise solution for driver identification. Due to its unsupervised characteristic, USID offers significant scalability, as its models don’t rely on predefined classes to distinguish and identify drivers. Our USID approach is inspired by the Knowledge Discovery in Databases (KDD) process and comprises several stages. In the initial phase of our process pipeline, we perform data quality checks, identifying and addressing outliers to avoid any potential distortions in model inferences. Given that time series data represents a flow of data points, we apply moving windows to split the time series sequence into variable-length segments. The choice of window length is intended to capture relevant events for predictive purposes. In order to ensure the relevance of our time sequence comparisons across a diverse range of driving contexts, drivers, and vehicles, we utilize global normalization techniques incorporating physical signal boundaries. Our approach integrates feature selection methods, such as information gain, signal-to-signal correlation analysis, and principal component analysis (PCA), facilitating the extraction of the most informative features while effectively reducing input dimensionality. As a final step in transformation, we propose an optional feature space transformation to align with the selected model’s input space. The recorded real OBD II time series of 24 drivers and their driving cycles are used for the purpose of training the models in USID. The dataset comprises 10 driving cycles from a single vehicle driven by 10 different individuals and an additional 14 driving cycles from 14 vehicles, each driven by distinct drivers. Consequently, this paper also explores the feasibility of accurately distinguishing between different drivers operating various vehicles.Unsupervised machine learning methods used in USID models are K-Means, Autoencoders, Self-Organizing maps, and Density-based spatial clustering (DBSCAN). In the end, the models are tested, and the results are outlined with the comparison of the quality of models for driver identification.

Veljko Vučinić, Luca Seidel, Marco Stang, Eric Sax

Conference Proceedings

Virtual Reality (VR) and Simulated Air Traffic Control Environment (SATCE) in flight training: The Purdue Case study

Adaptive learning capabilities based on Artificial Intelligence (AI) can provide learners with a personalized learning path. It is a capability that customizes the trainee’s learning experience to their identified learning style and preference while providing the quickest route through the pilot training program. To accomplish this, every training program should be integrated to provide context and relevance and improve performance by generating insights on the performance of learners or cohorts and the efficacy of associated content. The aviation industry seeks novel methods for pilot training that are more efficient. Competency-Based Training and Assessment (CBTA) is a method that proposes an assessment process that helps understand how a flight crew may be able to manage both foreseen and unforeseen incidents and uses this data to help the crew achieve a higher level of efficiency.With a centralized data capture process centered on the pilot’s information, a pilot profile can be created to provide personalized training and advanced insight into the pilot’s learning experience. New technologies like Virtual Reality (VR) training combined with biometric data like eye-tracking and facial tracking can be a powerful platform to obtain the required data. Describing the communication competency from a training perspective, an AI – VR training environment (Simulated Air Traffic Control Environment - SATCE) would allow the pilots to improve their communication skills, enable pilots to ask questions with a specifically trained Generative Pre-Trained (GPT) model, and receive a validated answer. The Purdue case study focuses on the cognitive aspects of flight training using emerging technologies. This research aims to improve training effectiveness by incorporating immersive technologies in aviation training. Dynamic real-time visualization, automatic human profile assessment, and training system adaption technologies can potentially improve flight training’s overall efficacy and efficiency. This digitization process includes various immersive virtual technologies and synthetic learning environments. By using these technologies, all persons participating in flight training will obtain a complete insight into the participants’ performance cognitive limitations, ultimately optimizing the training lifecycle.

Dimitrios Ziakkas, Gede Bagus Michael Kim, Dimitra - Eirini Synodinou

Conference Proceedings

Sense of technology self-efficacy and motivation coexisting with algorithmic thinking

Modern warfare presents the state of the art in technological advances. Even the most developed weapons and information systems are implemented the users and final decision makers are still human. There has been extensive discussion about contemporary skillset that is often drawn the components of digital literacy. In that context digital literacy is combination of technical skills to use technology, to interact with technology, and understand technology. It can be seen also as capability to understand different forms of information, algorithmic, computational, and epistemic thinking. Even the digital literacy framework applied is multi-faceted, it operationalises the perspectives of human performance with autonomous or semiautonomous systems. This paper discusses skillset needed to operate and operate with complex information driven systems. The aim is to construct a comprehensive way to assess capabilities among people in service to allocate the most suitable person to a post. The study is based on expert interviews and focus groups session on the topic. The presented competency framework and measures to assess different perspectives. Paper also discusses the practical application of the measurements and how items related to technology self-efficacy and motivation are related to algorithmic thinking.

Jussi Okkonen, Antti Syvänen, Jari Laarni, Jarmo Viteli, Mikael Wahlström, Mia Laine

Conference Proceedings

TauchiGPT_V2: An Offline Agent-based Opensource AI Tool designed to Assist in Academic Research

Recent progress in artificial intelligence, particularly deep learning, has ushered in a new era of autonomously generated content spanning text, audio, and visuals. This means Large Language Models (LLMs) such as ChatGPT, Llama2, Claude, and PaLM 2 are now developed enough to not only fill in the gaps within user-generated content, but also create unique content of their own, using predefined styles, formats, and writing techniques. With selective modelling and fine-tuning relevant training data, LLMs can output original content for a wide range of tasks previously considered solely the domain of human creativity. However, if we look at the area of research and development within academics, this AI renaissance has yet to make a meaningful impact finding in the pedagogical domains. Crafting a tailored R&D instrument, adept at intricate research procedures, previously presented a formidable challenge regarding expertise, time, and fiscal resources. However, the latest development Within this context, Generative Pre-trained Transformers (GPT) and their foundational structures offer a beacon, given their potential to exploit pre-trained Large Language Models (LLMs) for optimizing standard research operations. Our previous work on Autonomous Agents shows that using existing tools and deductive reasoning techniques built on the LangChain model can create a customized tool for academic research. This study builds on the existing work in autonomous agents and open-source LLMs to develop TAUCHI-GPT_V2, a novel adaptation of the academic research assistant. TAUCHI-GPT_V2, conceptualized as an open-source initiative, is built on top of the LangChain architecture employing LLaMA2-13b as the core LLM, ingesting users’ own data and files to provide highly relevant contextual results. In this paper, we discuss how TAUCHI-GPT_V2 uses custom offline localized vectorDB for parsing users’ personal files to output relevant contextual results within a chat interface. We also put the model to the test by having academic researchers utilize the tool within their daily workflow and report its efficacy and reliability in both hallucinations as well as citing relevant information to enhance user workflow for academic research-related tasks.

Ahmed Farooq, Jari Kangas, Mounia Ziat, Roope Raisamo

Conference Proceedings

Integrating Generative AI into the Design Process: A Case Study on Space Engineering

Developing a product that satisfies certain standards follows a methodical approach known as the design process. It involves multiple phases of requirement elicitation, analysis, conceptualization, and evaluation. Generative Artificial Intelligence (AI) tools like ChatGPT have impacted the design process in various engineering fields, but there is limited research on how to integrate AI tools into the design process effectively. In a case study, we used ChatGPT to design an innovative object - a space boot with haptic technology. We explored various prompt methods and found that the process is iterative, requiring multiple adjustments to the prompts. This research paper highlights the design process and its evaluation methods, discusses its limitations, and provides a plan for future improvements.

Mohammad Amin Kuhail, Jose Berengueres, Fatma Taher, Sana Khan

Conference Proceedings

Exploring Metamorphic Testing for Self-learning Functions with User Interactions

Self-learning functions, an evolving field in modern technology, are increasingly being integrated into a multitude of applications. They primarily rely on data-driven learning techniques, such as supervised, unsupervised machine learning and reinforcement learning. In the field of autonomous vehicles, self-learning functions are important for real-time decision-making, as they adapt to dynamic scenarios by collecting extensive data from sensors. Likewise, self-learning functions with user-interaction, a subset of self-learning functions, are asserting their influence in the automotive industry, as they observe driver behavior and recognize user-specific interactions with the system. In addition to data-driven learning, these functions incorporate real-time user interactions, such as activating seat heating or ventilation, and autonomously execute these interactions, enhancing overall comfort of the driver. The growing integration and interaction of self-learning functions underscore the importance of conducting research and refining testing methodologies to ensure their reliability and effectiveness. To meet the growing need for trusted and reliable self-learning features, effective testing methods are essential for validating the accuracy and robustness of self-learning functions with user interaction. In contrast to traditional software, self-learning functions change and adjust themselves based on data and interactions. This causes challenges to predict and verify their intended behavior. Furthermore, each potential user exhibits distinctive individual behavior that differentiates them from other users. As a consequence, attempting to address every potential user interaction with traditional testing methods and predefined test case specifications becomes impractical. Moreover, the behavior of a self-learning function adapts over time to that of the respective user. As a result, the user behavior to be tested evolves, rendering traditional testing through predefined test cases unfeasible. Consequently, adapted testing methods are indispensable to effectively address the test-oracle problem. This paper presents a solution to address the test-oracle problem by leveraging metamorphic testing as a method for validating self-learning functions with user interaction. Metamorphic testing approaches the problem of the test-oracle from a perspective not typically employed by other testing strategies: instead of focusing on individual test cases, metamorphic tests examine the outcomes of multiple test cases within a testing system and their relationships with each other. Metamorphic testing assesses whether the test inputs and outputs fulfill specific metamorphic relationships upon multiple test executions. These metamorphic relationships describe the essential properties of the intended functionality. They transform existing input-output test cases into new follow-up test cases. If the behavior of the self-learning functions deviates from the metamorphic relationship in these original and follow-up test cases, the testing system is considered faulted. The effectiveness of fault detection significantly relies on metamorphic relations. Thus, the analysis of metamorphic relations becomes an essential task and a creative endeavor for the tester. Furthermore, it will be a significant contribution of this publication.The proposed paper offers an analysis of the insights gained from the application of metamorphic testing to a self-learning comfort function. This underscores how effective the testing methodology is in identifying inaccuracies in the self-learning function's interpretation of user behaviors, thus contributing to our understanding of their reliability and adaptability in simulated scenarios. In conclusion, the utilization of metamorphic testing in the context of self-learning functions, with a specific emphasis on user interactions, emerges as a promising and efficient strategy.

Marco Stang, Luca Seidel, Veljko Vučinić, Eric Sax

Conference Proceedings

Artificial Intelligence-Derived Clinical Reports For Multidisciplinary Health Education: A Preliminary Approach

Health education is a pivotal component in promoting overall well-being and preventing various diseases. With the rapid advancement of technology, particularly in the field of artificial intelligence (AI), health education is undergoing a transformation. This project aims at using artificial intelligence to generate clinical cases for multidisciplinary health students on a higher education institution. A large language model (LLM) known as OpenAI's ChatGPT (Generative Pretrained Transformer; OpenAI) is proposed to generate clinical case reports in several medical areas. For a more realistic insight about the reports, Fotor (an online platform for photo editing) was used to provide AI-generated images of each patient. By means of this AI tool, it is possible to generate clinical case reports containing the following information: patient’s main sociodemographic data, main complaint pathology (clinical diagnosis), history of previous illness etc. Further research aims at assessing quality and applicability of these data for health students.

Christiano Bittencourt Machado

Conference Proceedings

Immersive Innovation: Bridging Digital Design and Virtual Realities in Jewellery

The paper presents the results of the workshop "Utopia: Jewelry Beyond the Body," held at the School of Design of the Politecnico di Milano as part of the Master degree program with the aim to develop an innovative design methodology for future creatives. The workshop's main objective was to explore the process of designing a virtual jewelry collection conceived to be worn and experienced in the digital world. The paper describes the workshop's deifferent stages of the methodology, with a specific focus on the use of Artificial Intelligence for the ideation phase, and the creation of a virtual exhibition hosted on Spatial.io, in the final stage.First, the paper addresses the context of the workshop. The jewelry and fashion fields have undergone a profound transformation, with a gradual shift from tangible, physical interactions to the dematerialized domain of the virtual. The pervasive integration of digital technology has affected the entire value chain of these sectors, from design practices to distribution consumptions. Considering this change, the work highlights the need to explore digital manifestations of tangible products and experiment with spaces of digital co-creation. Then the contribution focuses on the methodology implemented in the workshop, integrating digital technologies, such as virtual reality and artificial intelligence.Finally, the document presents the experience's outputs with quantitative and qualitative results. The results provide insights into the effectiveness of the design methodology, highlighting the impact of the research conducted.Furthermore, the experience is evaluated with a focus on the possibilities that can be obtained merging jewellery design and virtual exhibition practices.The "Utopia: Jewelry Beyond the Body" workshop represents an initiative between jewelry design, digital innovation, and academic pedagogy. By describing the workshop's evolving context, methodology, and results, this article contributes to a deeper understanding of the relationship between digital technology, design creativity, and the evolution of the jewelry industry.

Livia Tenuta, Susanna Testa, Beatrice Rossato

Conference Proceedings

Design and implementation of a marker-based AR-enabled tool tracking system for manufacturing manual operation

Although automation and robotics are widely implemented in manufacturing industry nowadays, assembly tasks and rework processes are still carried out manually by human operators because of their complexity and the need for a level of adaptability and flexibility greater than automation solutions can provide. However, manual operations exhibit variability and are subject to human errors, which could lead to unexpected delays or quality issues. Enabling traceability of manual operations is also intrinsically more challenging than for automated processes as it typically relies on the operator consistently providing direct input (e.g. HMI/operator interaction).Research suggests that augmented reality (AR) technology can contribute to enhance human-machine interaction by providing operators with a seamless digital bi-directional interface with physical systems (e.g. product or production systems), thus improving manual operations’ overall effectiveness. Existing research related to the development of AR-based solutions for manufacturing focus essentially on training and maintenance use cases, while there is limited development of applications aiming at supporting in-production operations. In addition, while AR technologies are used to provide information to the operator, the development of capabilities allowing manual process and operators to be monitored during operations, are lacking. The research presented in this paper adopt a holistic approach combining manual operation monitoring and operator feedback capabilities. The DAMPO (Digitally Augmented Manual Process Optimisation) system implements a) computer vision technologies to provide continuous operator and manual operations monitoring capabilities b) content rich and highly interactive user interfaces using screen-based 3D and AR-based information display, and c) near real-time data capture, management and processing pipelines that provide both real-time system/user interaction and collection of historical process data. The use case for this research focuses on specific manual assembly operations of safety critical components of seating systems for the automotive industry, which require precise sequence of operations and full traceability of the assembly process for audit purposes (safety critical operations). The DAMPO system is implemented in the Automation Systems Group's Digital Automation Laboratory at the University of Warwick, WMG department, UK, and is used to support both low TRL level research and direct engagement with industry on developing human centric manufacturing solutions. The DAMPO systems implement both process monitoring capabilities (i.e. data capture) and Electronic Work Instruction functions (information feedback and presentation of the operator) in view of improving process traceability and implementing no-fault forward capabilities. The DAMPO solutions combines a wide range of a) visual computing methods (fiducial, IR marker and 3D object based pose estimation), and b) information display and operator interface technologies (e.g. projected, screen-based and head-mounted AR layer display, HMI and work instruction screens with interactive 3D content, haptic feedback), which can be combined differently depending on the use case, and use case requirements such as hand free operations, complexity of work instructions or operator feedback and interactions, etc. This production-ready solution is tested and evaluated on a replicable production cell using real product and real assembly process.

Vivian Chung, Daniel Vera, Jiayi Zhang

Conference Proceedings

Hybridization possibilities of two approaches: Participatory Design and User-Centered Design

Participatory Design and User-Centered Design are two approaches commonly used when designing solutions. Both approaches involve users, albeit to a different degree across different phases of the design process. The recent shift enforced by COVID-19, brought the need that more activities are expected to be conducted online. In these circumstances, there are adaptation challenges in both approaches, especially for the Participatory Design. Technology mediation in the activities in both approaches brings overhead that makes the changes between them less obvious. Thus, in this paper we bring up for discussion a discourse of hybridization of these approaches as a way forward for online and distributed design processes. Furthermore, we present our initial thoughts regarding the possibilities that this might bring.

Mexhid Ferati, Arianit Kurti

Conference Proceedings

Digital competences in health professionals. Public hospitals case of peru.

In the modern and globalized world, information and communication technologies (ICT) have emerged that make it possible to manipulate information and provide the necessary means to store, retrieve, produce, record, present, transmit and communicate data, facilitating communication among all human beings and has also contributed to the transformation of the social world.At the same time, technological advances have further democratized the use of the Internet. This has led to hyper-connected scenarios, data creators and increasing digitization, as a result, almost all production areas today are caught in a process of change, whose main objective is to adapt to this new digital age.Indeed, in this context, digital capabilities were born, defined as the set of knowledge, skills, behaviors, strategies, strengths, and awareness required in the use of ICTs and the digital environment when carrying out activities, solving problems, reporting, managing information. , with positive, qualified, suitable, analytical, imaginative, fair, weighted by tasks, inactivity, collaboration, learning, socialization, use and empowerment.Given the situation observed, it is necessary to describe the digital skills in health professionals, since it is an essential requirement to deepen knowledge, by using new technologies it enhances their capabilities, thus improving health services, human quality and well-being of people in the field of nursing, also some professionals are teaching, a vital field in providing a comprehensive university education according to the needs, benefit not only this group by improving their work performance in the face of the challenges demanded by the professional world through of the development of digital skills, but also Include the environment, individuals, families, communities in care centers and at the same time train future nursing professiona.The general objective is to describe the digital skills of health professionals. For the collection of information, the following techniques and instruments were used: systematic review, for the bibliographic search, search engines were used (Scopus, PubMed, ScienceDirect, Dialnet, Scielo, ProQuest) finding 10 investigations that were evaluated through the list of Boverieth Astete, the advanced search was made with different search strategies combining descriptors, keywords and booleans. The sample consisted of 10 articles. The results obtained showed 6 relevant categories: creation of digital content, use and security of technology, information and information literacy, communication and collaboration, problem solving and factors related to digital competence. Conclusions: Health professionals have many deficiencies in information skills and information literacy, communication and collaboration, creation of digital content, use of technology and security; however, they show more skills when it comes to solving problems; the ability factors are correlated, and it is not a negative correlation, that is, the higher the ability, the younger they are.In the limitations there are deficient articles clearly related to the studied area. It is suggested to carry out more quantitative, comparative research; as well as qualitative studies.Keywords: digital competence, professionals, health sciences

Danicsa Espino, Madeleine Espino Carrasco, Luis Dávila Valdera, Edson Valdera Benavides, Anny Dávila Valdera, Mayury Espino Carrasco, Royer Vásquez Cachay, Lady Dávila Valdera, Carmen Arbulú Pérez Vargas, Ricardo Diaz Calderon, Cindy Vargas Cabrera

Conference Proceedings

Automatic transcription of the Methods-Time Measurement MTM-1 motions in VR

We concentrate our research on using VR technology for the automatic transcription of basic human motions in the Methods-Time Measurement (MTM) system. It is a predetermined motion time system that consists of a list of predefined basic motions and the mean time values corresponding to those motions. This system is used to analyze manual workplaces. Currently, the MTM analysis is conducted manually. The working process that needs to be analyzed is video captured and further analyzed by dividing it into a sequence of basic MTM motions. There are various MTM systems that differ by their granularity level, such as MTM-1, MTM-2, MTM-UAS, etc.We propose and evaluate an approach of the automatic transcription of the MTM-1 basic motions. Here are the MTM-1 basic motions our algorithm transcribes that are grouped by the involved body parts. Hand motions: Grasp, Release, Position, Disengage, Apply pressure. Arm motions: Reach, Move, Crank, Turn. Body motions: Sit, Bend, Kneel one knee, Kneel both knees. Leg motions: Step, Leg gestures.Methodology: For our research, we use Unity software to create the virtual environment (VE) and interactions within it. Additionally, we use the HTC Vive tracking system and Sensoryx VRfree data glove that enable body- and hand-tracking. Additionally, the MTM-1 system distinguishes between different grasping types, therefore, using hand-tracking technology instead of controllers becomes essential. To visualize the VE, an HTC Vive Pro headset is used. Our automatic transcription algorithm employs four decision trees that run simultaneously, each dedicated to transcribing hand, arm, body, and leg motions in real time.Hand motions occur when objects are grasped or released. Therefore, the trigger for the hand motion decision tree is the beginning or end of touching an object. Hand motions were observed to commonly occur in the following order: Grasp -> Disengage -> Position -> Apply pressure -> Release. The grasp and release motions are always present in a cycle. The motions disengage, position, and apply pressure are optional in a cycle and are performed while holding an object, so between grasping and releasing.Arm motions decision tree is triggered by obtaining the output of the hand motion decision tree. It is transcribed backward, using the recorded information. The body motions category consists of bending, sitting, and kneeling motions. The assumed initial state of the user is the standing position. These motions occur sequentially, so to transcribe a kneeling motion, the threshold values for the sit (T_sit) and bend (T_Bend) motion must also be achieved.The leg motions consist of step motions and leg gestures. A leg gesture is a motion where only the leg and/or foot move, for example, pressing a pedal. The metric chosen to detect these motions is the velocity of the foot. Results: We conducted the user study with 33 participants, which performed a total of 2738 motions. 2670 of them were true positives (TP), and 176 were false positives (FP). 68 of the motions performed by the users were not captured by the algorithm and represent the false negatives (FN). Additionally, we introduced precision and recall metrics.Precision = sum(TP)/(sum(FP)+sum(TP))=0.938; Recall = sum(TP)/(sum(FN)+sum(TP))=0.975Besides overall statistics, we also calculated statistics separately for every basic motion type.

Valentina Gorobets, Roman Billeter, Rolf Adelsberger, Andreas Kunz

Conference Proceedings

A framework for assessing and enhancing social and spatial presence in mediated communication to support remote collaboration

Although extended reality (XR) technologies offer promising new ways to interact and collaborate, adoption of XR in real world applications comes with significant challenges. This is especially true for remote collaboration settings, which are often characterized by heterogeneity. Knowledge is transferred from an expert to a novice, users are located in different physical locations with different systems and often only a limited number of users perform activities in the physical world.In this setting an especial challenge is a lack of guidance on how to measure and improve the quality of mixed reality interaction. Presence, specifically social presence and spatial presence, are two key aspects to measure the quality of experience in XR and computer-mediated communication in general. A multitude of social presence measures is available, but few studies attempt to provide a comprehensive way to measure all underlying factors of presence that affect the quality of experience, much less offer support on how to improve on these factors. Moreover, most measures of presence focus solely on XR applications, while a mix of communication systems is use. In this paper, we aim to address these issues and propose a comprehensive framework for the assessment and enhancement of social and spatial presence based on existing literature, regardless of the actual technology used. The framework provides a breakdown of relevant subscales of spatial and social presence in remote collaboration, easy-to-use means to measure these subscales and offers different degrees of system recommendations based on social cues that have been found to evoke presence on the respective subscales. Lastly, our framework includes practical suggestions for accommodating different types of users within a use case to ensure quality of experience for all users involved. Our framework should enable XR-developers and end-users to optimize their system for their specific application.

Liv Ziegfeld, Maarten Michel, Ivo Stuldreher

Conference Proceedings

Privacy Preserving Personal Assistant with On-Device Diarization and Spoken Dialogue System for Home and beyond

In the age of personal voice assistants, we have witnessed the proliferation of "personal" vocal companions across smartphones, smart speakers, and other smart devices. Yet, the question arises: Are these virtual assistants genuinely "personal"? The answer may surprise you. Most of these digital companions lack the ability to remember past interactions or truly understand who you are. They heavily rely on an internet connection to process your spoken words in remote servers. Even though users provide informed consent for these interactions, concerns linger regarding potential misuse of speech data, like invasive targeted advertising. The advent of high-performance co-processors, such as GPUs and TPUs, in modern smartphones has rendered cloud-based speech processing obsolete, paving the way for local, on-device solutions.Personal assistants for the elderly serve a unique role, requiring functionalities distinct from those catering to digital natives. Notably, they must excel at aiding memory recall during conversations, making them invaluable in scenarios like medical examinations. By documenting and contextualizing exchanges during medical visits through diarization, a personal assistant can empower individuals or caregivers to revisit and understand the details at their convenience. This autonomy necessitates operation without an internet connection, ensuring utmost privacy during such sensitive interactions.The e-ViTA project has successfully developed a versatile conversational application with a rich set of features:• Local use on both Android and iOS smartphones, no internet connection required.• The capability to remember previous interactions.• Speaker recognition for personalized experiences.• Local processing for automatic speech recognition, spoken language understanding, dialogue management, and speech synthesis.• Secure web searches after anonymizing requests.• The ability to handle telephone calls, read emails, SMS, and messages.• Text preparation through voice dictation.• Assistance with daily activities and acting as a companion or butler.• Facilitating inter-lingual communication via integration with TalkMondo, among other functions.Unlike facial recognition, vocal recognition, and speaker differentiation provide a less invasive and cost-effective solution. Being based on the smartphone's microphone, they do not rely on the camera, which would necessitate complex mechanisms to track the speaker's position.This paper highlights the critical importance of speaker diarization, which allows the system to preserve users' conversations while ensuring the highest level of privacy. Additionally, when deployed on embedded devices, this technology can contribute to monitoring the well-being of the elderly, offering vital contextual information enriched by domotics sensors (motion, intrusion, door or window sensors), actimetry sensors from smartphones or smartwatches, or weather station. The data fusion of these different data streams leverages more personalized and optimized assistance and services, through user-adapted dialogues, or the elderly based on his context and activity.In conclusion, the ability of a system to generate personalized dialogue synthesis is pivotal in the realm of personal voice assistants. With secure, local processing and advanced features, such as speaker differentiation and diarization, enriched by sensor data fusion, we can ensure that virtual companions truly cater to the individual needs of users, without compromising their privacy or data security. This marks a significant step towards a more "personal" experience with our digital assistants.

Gérard Chollet, Hugues Sansen, Yannis Tevissen, Jérôme Boudy, Mossaab Hariz, Christophe Lohr, Fathy Yassa

Conference Proceedings

Adaptive Visualization Framework for Human-Centric Data Interaction in Time-Critical Environments

In today's data-driven era, handling information overload in time-sensitive scenarios poses a significant challenge. Visualization is a valuable tool for comprehending vast amounts of data. However, it's crucial to have self-adapting visualizations that are tailored to the user's cognitive level and grow with their expertise. Existing solutions often fall short in this regard.This paper introduces a framework integrating Artificial Intelligence (AI) techniques for context awareness and emotion sensing, offering visualizations that adjust to user requirements. The framework makes use of cross-modal sensors and Machine Learning (ML) algorithms to analyse behavioural signals, usage statistics, and user feedback. This data guides real-time data ingestion techniques and ML-driven mechanisms, ensuring that visualizations adapt while safeguarding data privacy and confidentiality.

Andreas Alexopoulos, Jean Vanderdonckt, Luis Leiva, Ioannis Arapakis, Michalis Vakalellis, Vassilis Prevelakis

Conference Proceedings

Elaborating a framework that is able to structure and evaluate design workflow and composition of Generative AI visualizations.

One aspect of using generative artificial intelligence (AI) in design workflows that is not well understood, is how designers navigate the use of prompts and imagery to control design outputs. To learn more about the ways generative AI can elaborate different innovative design aspects linked to a conceptual genre, we undertake an analysis of designer outputs and link these to a mapping of iterative design workflow, and the composition of prompts and imagery used to inform generative AI visualizations. A speculative design workshop presented young designers with a proposed framework to facilitate and guide them through an iterative design process using generative AI tools (Vizcom and Midjourney). Design outputs included designed objects, atmosphere, spatial configurations able to generate social communication and relations for futuristic residential lighting environment. Prompt writing and sketching were two crucial variables that enable high fidelity design visualization. Traditionally research on prompt writing focuses on variables of descriptive design imagery, point of view, quality of output, ordering, precision, tone and style. (Microsoft,2023) In this study we investigate how designers initiate and continually engage with a generative AI system to achieve consistency of outputs and elaborate design variations. The literature review identifies variables to assess the impact of creative input to structure the generative AI visualizations. Our analysis focus on the cognitive workflows and its ability to integrate a mix of analogical and digital methods( sketching, prompts writing, and digital imagery). The analysis constructs focus on the way the design methods translate different spatial qualities, phenomenal experiences that have potential to facilitate social communication and relations, (proximity, sightline-acoustic comfort). Our insights constructs a design framework to guide prompt composition that inform iterative generative AI outputs. The schematic framework may be used to guide AI generated visualizations according to the qualities of atmosphere linked to the spatial forms and arrangements of interior spaces. Written prompts and imagery are examined in their ability to articulate design principle, spatial qualities, spatial characteristics, spatial forms and social relations.

Gianmarco Longo, Deborah A Middleton, Silvia Albano

Conference Proceedings

A PBL experience: Joining Human-Computer Interaction and Mobile programming

This paper presents a project-based learning approach based on the combination of Human-Computer Interaction (HCI) and Mobile Programming. This approach aims to develop and implement a graphical user interface and respective interaction of an application. The project was implemented in the HCI and Mobile Programming courses located in the 2nd and 3rd years of an Informatics Engineering Degree at a Portuguese Higher Institution. The objectives of this project were to specify, prototype, and implement an interactive system containing the following topics: (i) user and task analysis, (ii) problem-solving, and (iii) Gamified activity in a Problem-Solving Scenario where students must identify user needs and design tasks to address those needs were done. This proposal could be embodied in the form of a theoretical study or practical implementation on a form of connection with hardware that allows an alternative or innovative form of communication/interaction. In the presented study, the students went through four distinct phases: Ideation, User and Task Analysis, Prototyping, and Exploring New Forms of Interaction. In summary, the teacher expressed dissatisfaction with the students' work. Many tasks were completed last minute, and collaboration within groups was lacking. The students showed irresponsibility in their work distribution. The proposals lacked reflection and research on technical aspects, which is expected in engineering. Students demonstrated low curiosity about new technologies and proposed uncreative solutions. Students were also disappointed with their grades, and they preferred more closed and focused problems over open-ended ones.

Álvaro Santos, Anabela Gomes

Conference Proceedings

The Convergent Validity of Computer Operating Systems’ Usability Evaluation by Popular Generative Artificial Intelligence (AI) Robots

This article seeks to examine the convergent validity of (and thus the consistency between) computer operating systems’ (OSs’) usability evaluation by a number of popular generative artificial intelligence (AI) robots. Totally 18 popular OS versions were included in the study, they specifically being the various versions of the three leading OS families of Windows, macOS, and Linux. Usability was evaluated in eight major dimensions, namely, (1) effectiveness, (2) efficiency, (3) learnability, (4) memorability, (5) safety, (6) utility, (7) ergonomics, and (8) accessibility. Experimenting with a handful of generative AI robots, Microsoft’s Copilot, Google’s PaLM, and Meta’s Llama managed to individually accord rating scores to the aforementioned eight dimensions. For each robot of this trio, the minimum, the maximum, the range, and the standard deviation of the rating scores for each of the eight dimensions were computed across the OS versions. The rating score difference for each of the eight dimensions between each pair of these robots was calculated for each OS version. The mean of the absolute value, the minimum, the maximum, the range, and the standard deviation of the differences for each dimension between each robot pair were calculated across the OS versions. A paired sample t-test was then applied to each dimension for the rating score difference between each robot pair over the versions. Finally, Cronbach's coefficient alpha () of the rating scores was computed for each dimension between all the three robots across the versions. These computational outcomes were to affirm whether each robot awarded discrimination in evaluating each dimension across the OS versions, whether each robot vis-à-vis any other robots erratically and/or systematically overrate or underrate any dimension over the OS versions, and whether there was high convergent validity of (and thus consistency between) all the three robots in evaluating each dimension across the OS versions. Among other ancillary results, it was found that the convergent validity of the three robots in evaluating all the eight dimensions was high, and thus such evaluation is trustworthy at least to an extent.

Victor K Y Chan

Conference Proceedings

How can physical studio space integrate and support speculative AI design experimentation and visualization workflows?

Artificial intelligence is anticipated to reduce design project development times, enhance our understanding of user experience, and be instrumental in stimulating radical socio-technological changes for the second half of the 21st century. The aim of this paper is to propose typologies of physical workspaces to improve the use of Artificial Intelligence (AI) in design practice. This paper examines how physical workspaces were used to support designers engaging in variable creative workflow that integrates generative AI. The research critically investigates and analyzes how physical and virtual spaces (studio working spaces and virtual platforms) were used to generate a functional environment in which create speculative phenomenal experiences through AI visualizations (Vizcom and Midjourney). The space configurations, during a series of pre-structured activities, are seen to better support the designers’ productivity and increase social collaboration and interaction through the arrangement of a range of interior spaces.

Silvia Albano, Deborah A Middleton, Gianmarco Longo

Conference Proceedings

Do you know what's going on? - Examining Situation Awareness for Different Communication Concepts of an E-Cargo Bike Autonomous Parking Function

Offering the opportunity to comfortably carry heavy load, e-cargo bikes are an environmentally friendly alternative to motor vehicles. Especially in urban areas, where e-cargo bikes contribute to resolve challenges like noise pollution and limited space, there is an increasing number of sharing-systems that offer users the option of renting powerful e-cargo bikes without having to buy them. In this context, an automation of the e-cargo bike return process might further enhance the convenience and time-efficiency of the rental process. However, for such a highly automated function, a need for communication between the autonomously operating vehicle and the user may arise, for instance for providing feedback about the successful completion of parking. Users moreover need to develop an accurate awareness of the situation to prevent misinterpretations and to ensure a safe interaction with the autonomous function. In this regard, research on automated cars has found external Human-Machine Interfaces (eHMIs) to be a possible solution to communicate relevant information between traffic participants in situations where implicit driving cues from the vehicle’s trajectory are not sufficient. The aim of the current study was to investigate the effects of three different communication concepts for an autonomously parking cargo e-bike in a one-factorial between-design laboratory study. A total of N = 26 participants were randomly assigned to 3 experimental groups and watched videos presenting either (1) a visual communication concept (via light bands on the handlebar), (2) an auditory communication concept (via signal tones), or (3) a concept without additional signals (via movements of the e-cargo bike). The videos demonstrated typical return situations at the rental station, where the e-cargo bike should transmit information to the user via the different signals. According to an adapted version of the Situation Awareness Global Assessment Technique (SAGAT), the videos were freezed at certain points in the parking process. During each freezing, participants were interviewed regarding their Situation Awareness based on the three levels (perception, comprehension, and prediction). For each level of Situation Awareness, participants’ statements were compared with the expectation of the correct interpretation of the parking situation. Results of the qualitative analysis reflected a better understanding of the situation for the communication concepts containing explicit communication cues, particularly concerning the second (comprehension) and third (prediction) level of situation awareness. The visual and auditory communication concepts were correctly interpreted by the participants. For the first level of Situation Awareness including the perception of objects, signals and movements, results did not considerably differ between communication concepts. However, the communication concepts incorporating additional cues (visual, auditory) achieved better results in terms of comprehending the situation (level 2) and predicting future states (level 3) compared to the condition without eHMI. Overall, participants’ answers regarding the visual communication concept was mostly consistent with the expectations. In sum, results imply that eHMIs have the potential to enhance users’ Situation Awareness of the autonomous parking of e-cargo bikes, whereas cues from the vehicle’s movements alone might be not sufficient. The results of this study suggest potential design options for eHMIs for autonomous e-cargo bike parking and identify the need for further research.

Dorothea Liehr, Isabel Kreißig, Jakob Siegmund, Susanne Creutz, Hendrik Meißner, Josef Krems

Conference Proceedings

Dual Particle Filtering of Intelligent Driver Model Parameters for Indirect Detection of Driving Anomalies

Abnormal driving behavior, such as excessive speed and reckless and aggressive driving, is recognized as causing more than 50% of fatal accidents. The detection of abnormal driving behavior has a wide range of applications and is expected to be used not only to directly suppress abnormal driving behavior but also to be factored into the price of automobile insurance premiums. This paper proposes a new approach to abnormal driving detection that requires neither in-car cameras nor physiological sensors. Instead, this approach makes full use of an intelligent driver model (IDM) and its parameters, where vehicle acceleration is assumed to indicate abnormal driving behavior. In an experiment using a driving simulator, subjects were asked to text on a mobile phone to collect data on their driving behavior with and without distractions. Numerical analysis showed that IDM parameters estimated by dual particle filtering could accurately detect driving abnormalities without the use of direct driver monitoring systems such as on-board cameras or sensors.

Hironori Suzuki, Ryuya Seki

Conference Proceedings

Retrospective Temporal Perception with Virtual Products in Local Environments

It was observed that a group of students in a cross-sectional study perceived a rapid passage of time during the exploration of the virtual product used as a tool in the classroom. The characteristics that accompanied this temporal perception included the detection of a resource with a high percentage of spatial interaction as well as integrated by a closed linguistic system, additionally a negative valence in the emotional response in the majority of people as users in relation to the analyzed virtual product. The results indicate that a fast perception of time is not necessarily linked to activities with a positive valence in emotional response, but there are factors related to the object that contribute to this type of perception, and they could also be linked to complex cognitive processes that inhibit certain processes for the development of others.

Lorena Olmos Pineda, Jorge Gil Tejeda

Conference Proceedings

Assessment Type Preferences in Informatics Engineering: before, during, and after the pandemic

A variety of societal sectors were significantly impacted by the COVID-19 pandemic. In the specific situation of higher education, many classrooms switched to remote learning throughout the several lockdowns. Numerous constraints related to this problem are reported in the literature, particularly in classes where students were not proficient in using technology or where there was a heavy emphasis on in-person communication. The problem was worsened in countries with inadequate or insufficient technology resources.Even in the absence of a pandemic or a situation requiring a similar treatment, it is important to comprehend the added value of this experiment and how it can live up to students, families, educators, and institutions’ expectations. With this aim in mind, we analysed, in Portugal, the adoption of the remote/hybrid model during the confinements and its execution using inputs from students enrolled in a technology course (Bachelor in Informatics Engineering). This choice aimed to capture the perception of this specific profile, characterized by strong digital proficiency, and, in most cases, access to technological resources. This allows us to focus the research on the teaching mode itself, rather than potential constraints of its operationalization.We collected information to analyse several aspects, namely the class type preferences, the assessment type preferences and the behavioural and attitudinal aspects, before, during, and after the pandemic. Even though some negative aspects were noted, especially in the social and psychological domain, it was also clear that these students responded favourably to some of the tactics and strategies used at the time, which they deemed suitable for a post-COVID scenario. In this paper, the focus is on the assessment type preferences. For that, we questioned students and analysed the following aspects: the type of assessment students prefer (online or presential); the type of online and presential assessment students prefer (multiple-choice assessments, open-ended questions, filling in blanks, or other modalities); the type of tests students prefer in face-to-face assessment; the adequacy of the time allocated to the duration of the online tests; their difficulty level when compared to face-to-face tests; and the students' opinion on whether their ethical principles could potentially disadvantage them in a context of possible academic fraud. For each aspect, the reasons behind them were analysed.The findings suggest that students faced numerous challenges when transitioning to virtual environments but also reveal their overall satisfaction with the assessment process. They also support the idea that the predominantly face-to-face teaching system currently in use should undergo a reflection. Once it has been tried, remote evaluation processes endure inside the system and must be adapted to the present expectations of the various players. In our study, students paid significant attention to aspects such as time and cost savings, exam duration, the appropriateness of the exam type for each topic, and the potential for fraud.The full paper will be structured as follows: Section 1 will focus on the impact of the pandemic on evaluation processes, section 2 will describe the research methodology adopted, section 3, will describe and analyse the research results, and, finally, section 4, will present the conclusions and discuss future research.

Anabela Gomes, Cristina Chuva

Conference Proceedings

Exploration of Patterns in Emotional Response and Retrospective Cognitive Load during Interaction with a Virtual Product

A cross-sectional study was conducted with 9 students in a classroom, allowing the identification of certain patterns during their interactions with a virtual product used as an educational tool. A correlation was observed among emotional response, cognitive load, and the dominant linguistic system in the virtual product. It was also noted that the prevalent use of a linguistic system can lead to a higher demand for resources for accurate spatial localization, contributing to the generation of high cognitive load. Additionally, these factors were observed to influence the perception of time in these studies. The utilization of self-report methods can offer valuable insights into the user's perspective, enhancing our understanding of how their individual interaction processes are constructed.

Jorge Gil Tejeda, Lorena Olmos Pineda

Conference Proceedings

Digital creativity and problem-solving in higher engineering education: Developing an e-course using the educational affordances of STEAM methodology

STEAM methodology has been proven effective in combining different learning topics in inter-disciplinary teaching approaches, to achieve better learning results. It appeals to the contemporary world’s demands by preparing tomorrow citizens in terms of job efficacy and 21st century skills. Especially, students with high engineering expertise need to develop core skills, including problem-solving, information processing, collaboration and communication skills, and engagement in life-long learning perspectives, embedded in current social and economic circumstances. To do so, inter-disciplinary education which -among others- blends science and humanitarian studies, and creativity development explored in technology-enhanced learning environments, are found to be of great importance. In this direction, we developed a web-based educational framework appealing to engineering students’ needs in higher education. In this study, it is explored how STEAM methodology bridges the desired educational standards to 21st century skills and consecutively it is proved that students’ involvement in a properly designed STEAM problem-solving instructional scenario affects positively their creative thinking skills. Through the experimental design, students elaborated on the factors that structure creativity, such as imagination, self-esteem, experimentation, implementation, and communication throughout their engagement in the creative problem-solving process in a STEAM methodology framework, called “STEAMapT2theGalaxy”.

Foteini Paraskeva, Eleni Neofotistou

Conference Proceedings

Design Methodology for Product Designers in the Context of Domestic 3D-Printing

The study examined the domestic use of desktop 3D printers, recognizing that this market is still in a developmental stage, which will impact the overall products market. The study's goal focused on providing a straightforward understanding of the field to benefit its promotion by tutors and current or future interested parties. The study examined several precedents for focusing the technology on home end-users and reviewed the current situation in the market. The primary research findings include an illustration of the value chain changes that connect design sources and end-users, a taxonomy matrix of product types in the market, and a design methodology aimed at efficiently reflecting the design process and its application. The proposed methodology is structured on the basis of frameworks of design methodologies reviewed as part of the study. It presents an updated structure that includes aspects unique to the field. The discussion chapter focused on summarizing the possibilities inherent to the field concerning different perspectives.

Zuk Turbovich

Conference Proceedings

Human Drivers and Automated Vehicles team up for the better and the worse: a framework for the analysis of future accidents

By and large, the so-called “Human Error” is considered as the main cause of traffic crashes – proportions of “more than 90%” of crashes “caused” by human errors are often mentioned in the press, in the political debate, even in scientific literature. By gradually removing the influence of human factors, the introduction of Automated Vehicles (AV) would then allegedly hugely benefit road safety. However, these analyses leave aside a few hard facts: human factors are often found in combination with other types of factors (e.g. environmental, traffic or vehicular) as generators of road crashes. Indeed, some studies suggest that it is only in about half of the fatal road crashes that human factors are the sole contributors – and can hence be labelled as “sole causers”. Confusions often arise because only the main or the final failure leading to the crash is mentioned by e.g., police forces collecting accident data. Moreover, precise analyses of crash-producing mechanisms highlight human factors relating to failures in the driving functions (e.g., perception failure, lack of anticipation, wrong decision) and human factors explaining these driving failures (e.g., low alertness or attention). AV will remain prone to the former type of failure and will avoid the latter to replace them with entirely new types of failures (e.g., inability to properly identify and classify an obstacle, to assess the intentions of other road users in mixed traffic). All these issues require new analysis models for retrieving the relevant information when collecting accident or safety critical event data and suggesting new road safety measures. Having recently published an entirely new Automated Driver Functional Failures (ADFF) model and an updated taxonomy for Human Functional Failures (HFF) leading to road crashes in the context of shared or delegated driving tasks between human and automated drivers, the authors propose to address, in the present paper, the Explanatory Factors for both types of failures. The analysis of traffic crashes shows that automated drivers didn’t perform as well as human drivers when confronted with conditions impairing their perception system or when driving in a complex environment. Most worrying would be that this complexity could result from failure to identify other road users or obstacles. Finally, although the introduction of AV could suppress some accident-causing human failures, e.g., willful violations of road rules, new features explaining human failures would appear, of which overtrust in AV out of ignorance of the automation’s limits would be an example.

Henri Chajmowicz, Pierre Van Elslande, Laura Bigi, Philippe Lesire, Jean Baptiste Haué, Cyril Chauvel, Stéphane Buffat

Conference Proceedings

Workforce Planning in Aviation: The implementation of Artificial Intelligence in Recruitment

The International Civil Aviation Organization (ICAO) is responsible for international standards and recommended practices (SARPs) to unify aviation globally through regulations and practices with safety, security, efficiency, and green policy of the highest interest. One component of ICAO’s standards is the documentation required to be on board an aircraft, as stated in Article 29, which mandates crew members to carry their “appropriate licenses,” evidence for their competence, and required hiring documents. However, barriers persist to standardizing electronic personnel licensing. ICAO is developing the required technical specifications to implement and verify personnel licenses globally. To support this transformation, IATA (International Air Transport Association) encourages airlines, companies, and aviation organizations to innovate modern solutions, using artificial intelligence and machine learning (AI/ML) to optimize operational goals. A starting point to support this conversion to electronic personnel licensing is during the employee recruitment phase. This research project focuses on implementing a potential pilot recruitment EPL system on digital platforms using visual and graphic design tools with UI/UX considerations for airlines, companies, and aviation organizations. The project followed a SWOT analysis, Safety-Risk Assessment, Benefit-Cost analysis, Sustainability Assessment, and Management of Change, and presented a Functioning digital prototype. The emerging selected technologies are AI/ML following the EASA AI classification Roadmap 2.0 (2023). The application benefit is the offered organizational culture adaptability and Key Performance Indicators selection following a Lean Six Sigma approach. Cybersecurity is granted following multiple layers design - user approach. Finally, the project takes into consideration accessible design features.

Dimitrios Ziakkas, Ibrahim Sarikaya, Anastasios Plioutsias

Conference Proceedings

AI powered Social Communication: a Qualitative Investigation in to Social and Ethical Concerns

Social media has become an integral part of modern day lifestyle of new generation. The social media penetration in day to affairs is to such an extent that, there hardly is any day that goes without having any social media interaction. Although an easy way of distant communication and entertainment are the driving force behind emergence of social media platforms, in recent times it has been used to fulfil the undesired motives. Many instances of fake news, bias, spreading of hatred, deep fakes have been noticed. This paper is an attempt to conduct a conceptual analysis of AI technology alongwith the literary survey of impact of AI on social communication. Based on the qualitative literary survey, the author has proposed some recommendations, some of which includes, inclusion of social media ethics in school curriculum, international norms for use of social media platform etc.

Vikrant Yadav

Conference Proceedings

Using Future Thinking as a steering tool for Generative AI creative output: a case study aiming at rethink lighting in the next future

Contemporary business sustainability relies on its flexibility and adaptability to unforeseen changes in societal and environmental challenges. In that sense, Speculative Design and Future Thinking offer a unique methodology for businesses to plan for the unpredictable. While those theories have been tested by decades of artistic and business applications, the recent introduction of generative AI tools widened the complexity and likelihood of answers that can be thoroughly investigated. At the same time, this enhanced capability to peek into our futures raises several concerns: How can we retain control of generative AI creative output using future thinking as a steering tool? Would that allow designers to proceed beyond strategic choices, addressing specific product development?This paper shares a case study of how future thinking was applied to provide a lighting manufacturer with design visualisations related to the lighting industry's future. HE institution offered this service (hidden for peer review) through an interdepartmental collaboration (Industrial Design and Architecture). Futures Thinking and Speculative Design apply as a methodology for consulting firms on strategic and communication choices (Mc Kinsey & Company, report, 2021). This case study explores the impact of such methodologies on choices related to product, interior, and systems design. Generative AI tools were prompted to visually interpret the forecasted scenarios through narrative discourse and creative works. the questions addressed were: What is the future of the light industry in the context of human emotions, enhanced user experience, well-being, or energy and resources?MethodsA workshop involving undergraduate and postgraduate students of the respective departments was organised in two phases: the first phase was about describing a scenario ten years in the future and conceptualising the design vision; the second phase was addressing the materialisation of light and its execution in a product or service proposal. After selecting the context, students followed methodological steps to envision future products, spaces and interactions following the four identified possible scenarios (Slaughter, 2013). In both phases, students used generative AI tools to visualise scenarios and solutions, Midjourney for the conceptual narrative, and Vizcom was used to generate illustrations of the interior/product lighting ecosystem. The latest tool does not rely solely on prompting but requires the designer to submit a sketch or a draft rendering of the proposed solution.ResultsThe design team of the participating lighting company attended each phase presentation involving products unfolding the relationship with the light in the future. The feedback received from the company appreciates the depth of the research and acknowledges the competitiveness of the identified research directions. The results achieved in the second phase of the workshop will be showcased at the IDI Shanghai yearly exhibition 2023.The format proposed could be easily adapted and implemented in a range of specific design domains, such as transportation and automotive, fashion and accessories, consumer electronics and household appliances, to name a few.

Ivan Parati, Mariia Zolotova

Conference Proceedings

The Role of Artificial Intelligence in the Fatigue Risk Management System

Fatigue in aviation is defined as “a physiological state of reduced mental or physical performance capability resulting from sleep loss, extended wakefulness, circadian phase, and/or workload (mental and/or physical activity) that can impair a person’s alertness and ability to perform safety-related operational duties”. Fatigue compromises human performance and is considered an identified threat to flight safety. Fatigue is inevitable within the aviation operations context. Therefore, fatigue cannot be eliminated; it must be managed. Fatigue Risk Management System (FRMS) has recently been implemented in airline operations. FRMS is “a data-driven means of continuously monitoring and managing fatigue-related safety risks, based upon scientific principles, knowledge, and operational experience that aims to ensure relevant personnel are performing at adequate levels of alertness”. Nevertheless, studies reported prominent levels of fatigue for 68.5% to 93% among professional pilots. This implies that current FRMS strategies need to be strengthened. The ever-growing and constantly changing aviation industry mandates novel complementary approaches to enhance the established FRMS. EASA has already acknowledged the potential benefits of artificial intelligence (AI) in aviation. Implementing AI applications in FRMS could benefit both flight safety and cost efficiency. AI can analyze various sources of data to enhance real-time detection and prediction. Countermeasures could, then, be employed to address pilots’ drowsiness and mental impairments and, thus, prevent fatigue-related incidents. This study provides a holistic human-centric approach regarding AI utilization in FRMS by outlining the potential benefits and challenges. Implementing AI applications in FRMS could benefit both flight safety and cost efficiency. AI can be integrated with existing FRMS and provide a more comprehensive approach to fatigue management. Given AI’s computational capacity, it can analyze vast amounts of various sourced data to enhance real-time detection and prediction. Countermeasures could, then, be employed to address pilots’ drowsiness and mental impairment and, thus, prevent fatigue-related incidents. AI applications are promising in using pilot’s data sourced through wearable devices (e.g., smartwatches, fitness, and eye trackers) for the detection of fatigue in domains such as a) Monitoring/analyzing electroencephalogram (EEG) signal; b) Yawing detection; c) Facial muscle detection; d) Drowsiness detection; e) Pupil detection and monitor pilot fatigue levels in real-time. Additionally, AI could analyze natural language during the pilot’s communication to detect signs of fatigue. Also, AI could integrate and contribute to fatigue flight data. Moreover, AI can contribute to FRMS by identifying high-risk periods for fatigue and developing customized mitigation strategies based on individual characteristics such as circadian rhythms, sleep patterns, and workload. AI can be used to develop predictive models that can forecast the likelihood of fatigue for individual crew members based on numerous factors, such as sleep patterns and workload. These models can inform scheduling decisions and other fatigue risk management strategies. AI algorithms could also optimize crew rostering to avoid last-minute unfit-for-duty reports and prevent additional costs. On the contrary, there are considerable challenges to overcome. The reliability and validity of data used by AI systems are critical to their effectiveness in predicting and managing fatigue. Methods to measure the accuracy and consistency of data must be established and continually monitored. Additionally, there is a risk that AI systems can perpetuate or even amplify existing biases and inequalities based on training data. Regulations and legal frameworks must be established to guide the use of AI in fatigue risk management. EASA is working on a structured AI integration within the aviation industry. Personalized and sensitive data should be collected for the intended purpose. Data governance should ensure privacy and security to protect individuals. Also, some ethical considerations could be raised (e.g., the responsibility for decision-making and accountability in case of errors or incidents. Moreover, it is essential to understand how AI systems can integrate with human factors and not become a source of stress or distraction for pilots. Non-intrusive wearable AI systems’ sensor integration is of utmost importance. Implementing AI systems can be costly, and the return on investment must be carefully considered. Determining the cost-effectiveness of AI systems in fatigue risk management is essential. Lastly, resistance to change by the users could be anticipated. The Purdue research case study connects the ongoing measurement of Fatigue in aviation training (Purdue SATT student pilots) using AI applications with the commercial aviation FRMS

Dimitrios Ziakkas, Julius Keller, Sudip Vhaduri

Conference Proceedings

A Relationship Model between the Perceived Economic Value of Computer Operating Systems and their Usability: All Variables Evaluated by Copilot AI

This article proposes and implements a relationship model between the perceived economic value of computer operating systems (OSs) and their usability where all the model variables are evaluated by the generative artificial intelligence (AI) robot Copilot of Microsoft. In the underlying study, the perceived economic value of a particular OS was estimated by a proxy variable, namely, the OS’s total cost of ownership (TCO) whereas the usability of the OS was gauged by eight usability variables corresponding to the eight major usability dimensions (1) Effectiveness, (2) Efficiency, (3) Learnability, (4) Memorability, (5) Safety, (6) Utility, (7) Ergonomics, and (8) Accessibility. All these variables were evaluated on a scale of 1 (the lowest or worst, as appropriate) to 10 (the highest or best, as appropriate) by the robot Copilot based on global users’ textual comments on the TCO and the above eight dimensions of the OS as appear all around the web. A total of 18 OSs were covered in the modeling, they being the popular versions of the three leading OS families of Microsoft Windows, Apple macOS, and Linux. Multiple regression of the TCO on the eight usability variables was performed, followed by individual simple regression of the TCO on each of the eight usability variables in a bid to avert multicollinearity’s effect. On the one hand, the multiple regression rendered a model with a coefficient of determination R2 = .853 and an F-statistic = 8.290 (df = 7, 10; p = 0.002 < 0.01) and (2) Efficiency being the only usability variable statistically significantly (p < 0.05) but negatively impinging on the TCO. On the other hand, the individual simple regression models revealed that in an individual manner, (2) Efficiency (R2 = 0.562; F = 20.542 (df = 1, 16; p = .000)), (5) Safety (R2 = .547; F = 19.335 (df = 1, 16; p = .000)), (6) Utility (R2 = .392; F = 10.331 (df = 1, 16; p = .005)), and (7) Ergonomics (R2 = .253; F = 5.431 (df = 1, 16; p = .033)) all statistically significantly but negatively impacted the TCO. Deducing from these models, the better an OS in these four usability dimensions, the lower its economic value as perceived by the market was. In other words, the market of OSs did not seem to have priced in usability or seemed to have even priced in them in a direction at odds with intuition and logic. These confounding results may be ascribable to multicollinearity and probably imperfection of the current generative AI technology, on which this study heavily hinged. After all, the multiple regression model predicts and explains the TCO and thus the perceived economic value very well.

Victor K Y Chan

Conference Proceedings

Designing a User-Centred Team Role Testing APP: Revealing Team Dynamics Through Visual Analysis

In today's fast-paced work environment, teamwork is considered to be one of the most effective ways of working. Teams facilitate cross-functional collaboration by assembling members with diverse backgrounds and areas of expertise. Nonetheless, the calibre of teamwork is contingent not solely upon the individual competencies of specific team members but also upon how well the roles and skills of each member are coordinated and balanced within the team. The allocation of roles within a team exerts a profound influence on the team's overall performance, and unfair or ineffective distribution principles can lead to significantly lower team performance in practice. Hence, in practical team collaboration, the equitable allocation of roles within the team is a challenging issue. The primary aim of this study is to design a visual tool to foster the establishment of an appropriately skills-balanced team while facilitating team members' comprehension of team dynamics.Following a comprehensive review of team working and team role allocation practices we proposed Belbin team roles as the theoretical underpinning for the tool’s team role equilibrium processes. We employed a qualitative methodology to gather user requirements, undertaking unstructured interviews with ten individuals experienced in collaborative working. Detailed user requirements were gathered from this data and an interactive prototype was designed using Axure. The prototype was designed to analyse users’ personality types with respect to Belbin team roles and use this information to allow the optimum allocation of members to teams for a proposed project. The tool also allows the make-up of the team to be visualized using a range of graphics which present the outcomes of team assessments and offer developmental recommendations tailored to the specific industry of the team.The tool was evaluated using a combination of the User Experience Questionnaire (UEQ) and semi-structured interviews. Two rounds of evaluation were conducted, using a further 10 individuals; five were non-industry users and five were industry experts. After each iteration the prototype design was refined.The design underscores the significance of product design and iterative processes grounded in user requirements within the realm of mobile application design. Such insights may hold substantial implications for teams across diverse industries and work environments, particularly concerning the equilibrium of team roles and the dynamics within teams in the future.

Xinlei Zhao, Gail Hopkins

Conference Proceedings

ReActIn: Infusing Human Feedback into Intermediate Prompting Steps of Large Language Model

This paper introduces ReActIn, a framework designed to infuse human feedback into the intermediate prompting steps of large language models. The practicality and effectiveness of ReActIn are validated through experiments that apply four established prompting strategies, evaluated both with and without human feedback integration. The proposed architecture's performance is compared against traditional large language models across various tasks using four standard evaluation tests. Our findings reveal that the integration of human feedback has a direct impact on the reasoning, action prompting, and overall decision-making capabilities of the language models. This study underscores the potential of ReActIn to shape a future where sophisticated, context-aware AI systems, empowered by human feedback, can effectively navigate complex real-world scenarios.

Manuel Delaflor, Carlos Toxtli, Claire Gendron, Wangfan Li, Cecilia Delgado Solorzano

Conference Proceedings

Development of a System for Real-time Assessment and Modification of Team-based Exercises

This project seeks to develop a Multimodal Performance Evaluation System (MPES), a system for real-time evaluation of individual and team stress levels suitable for a wide array of military medical training and assessment domains and conditions. This system was designed to identify changes in stress levels during training scenarios and determine whether changes to environmental or task stressors are needed. MPES capitalizes on the predictive abilities of pretest data (cognitive and affective) along with real-time physiological and sociometric data, to make claims about individual team member stress levels during training and aggregating that data into a team stress-level score. Our overarching goal was to improve the quality of training for medical teams, by adapting scenario-related characteristics to maximize stress levels, while avoiding overstressing to the point of performance failure. MPES is designed to be utilized in a variety of training environments of varying complexity, including screen-based, head-mount, immersive VR theatre, and open field exercises. Independent variables include heart rate, skin temperature, vocal characteristics. Moderating variables are working memory capacity and eight affective variables (e.g., extroversion and emotional stability). A Machine Learning approach using TPOT (a Python Automated Machine Learning Tool) was utilized for validation of the MPES analytic engine. Results indicate that the system is sensitive to individual differences and is capable of reliably reporting on perceived stress level.

Alan Liu, Richard Wainess

Conference Proceedings

Investigating AI Model Limitations in Recognizing Faces and Bodies in Ballroom Dance Settings

The advancement in the face and body detection algorithms has sparked an interest in using them to assist physical education and sports training, where AI can analyze students' body postures and movements to offer corrective guidance and prevent injuries. However, ballroom dance settings are uniquely different from traditional settings often used for face and body detection. On the one hand, most traditional face and body detection algorithms detect individuals instead of a collaborative dyad. Moreover, specific dancing postures may pose additional challenges for AI algorithms to detect. In order to unlock the power of AI in enhancing real-time feedback for dancers on their movements, postures, and expressions, there needs to be a thorough understanding of the capabilities of AI in analyzing complex dance sequences and identifying subtle nuances in body language. In this work, we examined four widely adopted body and face detection models on their effectiveness in detecting ballroom dancers. We found that these models shared key limitations. First, they are more likely to detect the man than the woman in the dyad, especially when the woman is curved backwards. Second, they often detect the couple as one person, mixing different body sections. Third, errors are frequent when the dancers do not face the camera or when they are wearing specialized costumes. This project offers suggestions to diversify the training datasets of such algorithms to make them suitable for new settings including ballroom dance.

Lewen Ivy Huang

Conference Proceedings

Human values assessment toward AI-based patient state prediction

Recent advances in artificial intelligence (AI) technology are remarkable, and AI may predict a patient’s future state (prognosis) using large amounts of data in the future. However, patients are not always satisfied with the prediction results. In order for a patient to accept prediction results and change behaviors in life, prediction results should reflect his/her values. AI-based patient state prediction should be implemented in a way that is not only data-driven but also integrated with domain knowledge about human values. This paper reports a case study where we assessed data related to values toward building a domain knowledge for AI-based patient state prediction.We designed a rehabilitation exercise service based on a person-centered principle [1] and intervened with one patient [2]. From the preliminary stage of service design, we attempted to build rapport and motivate her. We expected her to self-disclose through repeated dialogues. While understanding her life backgrounds and values, we defined rehabilitation goals for her independent living.12 rehabilitation exercise sessions were held for three months, and questionnaires and semi-structured interviews were conducted before the first session and after the final session. Even after the experiment ended, we continued to have conversations with her and obtained information about her life after the experiment ended. Measurements taken after the experiment showed no noticeable effect on her physical functions. Regarding her self-disclosure, as the experiment progressed, she revealed more of her true feelings, including negative ones. Before the experiment, she negatively commented about daily activities that she could not do well, but after the experiment, she positively did with focusing on the things she could do, although she was not perfect. We confirmed her active participation in social activities, such as going out wearing a clothe made by sewing, a task that she had viewed negatively before the experiment. Factors that influenced her behavior changes include building rapport through repeated conversations and self-disclosure as an effect of the experimenter’s interest in and empathy for her. Regarding self-disclosure, at the beginning of the experiment, most disclosures were positive-sounding superficial ones, but gradually they changed to deeper disclosures that included negative content.The patient’s value data suggested by the self-disclosure in this study is still insufficient to form a value domain knowledge system. However, we believe that the acquisition of value data based on a patient’s self-disclosure and behavioral changes will help realize AI state prediction that will lead to patient acceptance in the future. Future work will include a detailed analysis of the factors that influenced self-disclosure and the construction of a value assessment framework.[1] Kitwood, T. and Bredin, K. (1992) Towards a theory of dementia care: Personhood and well-being, Ageing and Society, Vol.12, No.3, pp.269-287.[2] Yukihira, T et al. (2023). Toward an online rehabilitation exercise service based on personal independent living goals and risk management. Human Systems Engineering and Design (IHSED 2023): Future Trends and Applications, Vol. 112, pp.187-194.

Masayuki Ihara, Hiroko Tokunaga, Hiroki Murakami, Shinpei Saruwatari, Akihiko Koga, Takashi Yukihira, Shinya Hisano, Kazuki Takeshita, Ryoichi Maeda, Masashige Motoe

Conference Proceedings

Precision and reliability of digital blood pressure monitor when monitoring the blood pressure of patients in critical units of a hospital in Peru

The increasing integration of digital technologies in the healthcare field has raised questions about the accuracy of these devices. Therefore, this study aims to support, through scientific evidence, its use in critical environments. The main objective of the article was to evaluate the accuracy and reliability of digital blood pressure monitors used to monitor the blood pressure of patients in critical health units. To carry out the analysis, a qualitative approach was adopted that involved the review and analysis of a set of ten previously published studies, which were selected from highly reliable academic and medical sources. The choice of a qualitative approach was based on the need to carry out a thorough and detailed examination of the studies available in the scientific literature, with the purpose of identifying patterns, trends and limitations in the use of digital blood pressure monitors in critical healthcare settings.During the review and analysis process of the ten selected studies, special attention was given to several key aspects. Firstly, the methodology used in each of the studies was evaluated, paying particular attention to their methodological quality. Likewise, the measurement techniques used, the sample size, the population studied and the conditions were examined. environments in which the measurements were carried out. The results obtained from this analysis revealed interesting trends regarding the accuracy and reliability of digital blood pressure monitors in blood pressure monitoring. Overall, digital devices demonstrated satisfactory accuracy and reliability in blood pressure measurement. However, it is important to highlight that some limitations and variations in the results are identified, which can be attributed to various factors. One of the factors identified as influencing the precision of the measurements was the obesity of the patients, since in these cases digital blood pressure monitors can provide less precise measurements. Additionally, the position of the arm during measurement can also influence the results, underscoring the importance of standardizing position to obtain consistent and reliable measurements.On the other hand, poor quality cuffs can affect the accuracy of blood pressure measurements. Therefore, the use of high-quality cuffs in clinical practice is strongly recommended. Finally, the environmental conditions under which measurements are made can influence the results. Factors such as temperature and humidity can affect the function of digital blood pressure monitors, highlighting the importance of controlling these conditions for accurate and reliable measurements. In conclusion, digital blood pressure monitors have emerged as valuable tools in monitoring blood pressure in critical health units. However, its successful implementation requires careful evaluation of the circumstances and precise calibration.

Carlos Ojeda, Danicsa Espino, Diego Henrri Ojeda Alberca, Zayra Marivel More Davis, Fátima Giovana Vásquez Vigil, Fadia Zulema Huerta Huerta, Flor Juliana Mercedes Risco Ipanaque, Samame Talledo Belinda Elvira

Conference Proceedings

Recommendations for the use of resilience matrix in healthcare institutions

In recent years, the threat of cyberattacks has been increasing annually, necessitating organizations to prioritize security measures. This urgency is particularly critical in healthcare institutions, given their status as crucial infrastructures and the potential risks cyberattacks pose to human lives. However, limited IT investment in healthcare institutions has resulted in inadequate cybersecurity measures and underinvestment in IT infrastructure. Outdated and unsupported operating systems pose significant vulnerabilities, escalating the risk of security incidents. Despite the challenges imposed by limited IT investment, organizations must strive to develop resilience that enables swift recovery in the event of a security incident, even while accepting a certain level of risk. As it stands now, healthcare institutions that have gotten cyberattacks are hesitant to disclose some of the findings of security incidents involving vulnerabilities that have occurred at healthcare institutions because they reveal vulnerabilities within the industry and suggest the possibility of further cyber-attacks. However, NIST SP800-61, which is also a guideline for computer incident response, states that it is important to prepare for the next security incident by learning lessons from past security incidents. In the first half of this study, security incidents that have occurred in healthcare institutions in the past are discussed and issues are organized using the PPT (People, Process, and Technology) framework used to examine cybersecurity measures. In this study, issues were organized based on two hospital incident cases that occurred in Japan in 2021 and 2022 and for which investigation reports were issued.In terms of People (organizations), the lack of security personnel is positioned as the cause of incidents. However, healthcare institutions prioritize patient care, and for economic reasons, there is also the issue of difficulty in securing not only security personnel but also IT personnel.In terms of Process, the issues were that the information collection system for urgent vulnerabilities and the patch application process were not in place, and the password policy was not well maintained and was weak. In terms of Technology, the vulnerability of the equipment and systems had not been addressed, and the anti-virus system was not in operation. Due to the compatibility of the electronic healthcare record system, it was not possible to update the equipment or run the anti-virus.The second half of the paper will focus on People and Processes in particular, and organize their roles in the incident response phase using the Incident Command System (ICS). In a normal ICS, the five actors are Command, Operations, Planning, Logistics, Finance, and Administration, but since many of the roles played by the legal department could be read from the incident case studies, a resilience matrix was organized to identify the relevant actors. In the future, we will further discuss the practicality of the resilience matrix and the triggers and considerations for shifting to the OODA loop (Observe, Orient, Decide, Act) when responding to an incident by applying the results of previous studies, such as incident response exercises.

Kenta Nakayama, Kenji Watanabe

Conference Proceedings

Expanding the Horizon of Learning Applications: A Study on the Versatility of Prompting Architectures in Large Language Models

This paper presents an exploration of the versatility of prompting architectures in large language models (LLMs), expanding the horizons of their application in learning and language interfaces. By leveraging the expansive capabilities of LLMs, this research probes the potential for creating structured prompts that can simultaneously support multiple use cases, namely paraphrasing, grammatical syntax guidance for introductory sentences, and conducting experiential conversations in a foreign language. In this study, we delve into the specifics of enabling such technology, including the design of the prompting architecture, the deployment process, and the intricacies of applying the same structure across diverse applications. An extensive field experiment incorporating interfaces powered by LLMs using this structured prompt has been conducted to evaluate the model's efficiency in real-world scenarios. Results from the field experiment highlight the promising adaptability of these prompting architectures, revealing remarkable efficiency across the multiple use cases explored. Furthermore, this research uncovers a new dimension of flexibility in the design and deployment of learning applications using LLMs, potentially revolutionizing language learning interfaces by establishing a one-size-fits-all solution. This paper aims to stimulate further research into refining and expanding the potential of LLMs, encouraging the exploration of how artificial intelligence can optimally benefit language learning and related applications.

Cecilia Delgado Solorzano, Carlos Toxtli

Conference Proceedings