Human Factors and Systems Interaction

AHFE International

Accelerating Open Access Science in Human Factors Engineering and Human-Centered Computing

Human Factors and Systems Interaction

book-cover

Editors: Isabel L. Nunes

Topics: Human Systems Interaction

Publication Date: 2024

ISBN: 978-1-964867-30-4

DOI: 10.54941/ahfe1005346

Articles

Ergonomic Adjustment Needs of Transport and Mining Machines: A Preliminary Study of Operators' Attitudes in Serbia

Previous studies suggests that accidents and/or incidents involving heavy machinery are unanticipated and unfortunate occurrences most frequently caused by human error. Therefore, the primary objective of this research was to conduct a preliminary investigation into the views held by operators of transport and mining machinery towards the ergonomic adjustment of their work environment to accommodate their specific needs. Namely, the aim of this paper is to analyze the various factors that contribute to the degradation of working conditions, leading to the emergence of occupational diseases and to the likelihood of accidents and incidents. In this research, the factors that influence the possibility of human error due to unfavorable workplace conditions, ergonomic characteristics of the cabin, the physical condition of the operator, job satisfaction and management commitment were analysed, as those factors are recognized by previous research.The research utilized a questionnaire tool to gather data from individuals operating machinery in the transportation and mining industries. There were 93 operators of transport and mining machines willing to participate in this survey, of which 65 were operators of various mining machines, while 28 operators worked on cranes. All surveyed crane operators have an average height in the range of 165-182 cm and a weight of 70-102 kg. Surveyed operators of mining machines have an average height in the range of 166-190 cm, but there are high deviations from the average weight, i.e. 9.23% of them have a weight of over 110 kg, that is, the weight is in the range of 60-150 kg. The examined machines included cranes with a span from 9 to 25 m and its load capacity from 10 to 160 tons, as well as excavators, bulldozers, drills, dumpers, backhoe loaders, bucket wheel excavators and loaders. The data was subsequently subjected to further analysis using descriptive statistics, cluster and principal component analysis. The study's results have provided evidence to support the initial premise, suggesting that human error is the primary cause of accidents and incidents involving transportation and mining machinery. The main cause factor, as perceived by a considerable majority of mining operators (79.9%) and crane operators (40%) is human error. The results suggest that there are differences in the ergonomic adjustment of workplaces of mining and transport machinery operators in manner that mining industry workplaces are better adjusted then transport industry workplaces.Cluster and principal components analysis led to the following conclusions. The key factors affecting the reduction of the quality of the working conditions of operators on mining machines are related to the seat rotation, absence from work due to poor working conditions (sick leave), nonadjustable armrests or non-existing armrests or armrests which are not set at the right height. The key factors that affect the quality of the crane operator's working conditions, are connected to characteristics of armrests (no armrests, armrests not at the right height, armrests are not adjustable), seat characteristics (seat height is not adjustable, seat is not adjustable horizontally, seat has not lumbar support, seat cannot recline, seat cannot be rotated), controls adjustment issues, the temperature in cabin regulation, visibility issues and to the absence of management commitment.A general recommendation as a measure to improve working conditions for operators of mining and transport machines is the usage of ergonomically adjusted seats with armrests. A proposal to introduce cameras that provide operators with better side and rear visibility could be considered, too, in aim to reduce the burden on the operator during reversing movements. In terms of management commitment, further focus on ergonomics climate is necessary to enable better working conditions for mining and transport machinery operators.Sample size enlargement is proposed as the future research avenue.

Vesna Spasojevic Brkic, Mirjana Misita, Neda Papic, Aleksandar Brkic, Martina Perišić

Conference Proceedings

Model-based Systems Engineering (MBSE) Enterprise Architecture Framework (EAF) with Human System Integration (HSI) – A Smart-City (SC) Case Study

This paper researches and analyzes how Model-based Systems Engineering (MBSE) methods can be applied to the construction industry in developing an Enterprise Architecture Framework (EAF) for long-term success of the Operations and Maintenance (O&M) of emerging Smart-Cities (SC). A properly built EAF complements an organization’s Digital Transformation (DT) effort, enabling the construction of these human-focused urban developments that purport sustainable practices in response to environmental threats heightened by typical city infrastructure. The complexity of an SC is comparable to architecting a System-of-Systems (SoS) with the imperative need to identify and account for emergent properties between systems when making design decisions. From a Systems Engineering (SE) perspective, the re-usability of standardized Architecture Frameworks (AF) and libraries built within an MBSE environment will decrease rework, therefore reducing cost and improving reliability of similar projects in the future. This paper will (1) review existing literature related to MBSE EAF implementation and results within the construction industry with an emphasis on SCs, (2) propose a Reference Architecture (RA) that extends an existing MBSE framework with customizations for SC construction, and (3) architect and evaluate The Line in Saudi Arabia as a case study. This research will lay the foundation for conclusions regarding the efficacy of practicing MBSE in construction as the domain becomes increasingly interconnected to “smarter” applications that require additional considerations, constraints, and technologies.

Sarah Rudder

Conference Proceedings

Model Training Through Synthetic Data Generation: Investigating the Impact on Human Physical Fatigue

Collaborative robots, or cobots, are one of the Industry 4.0 technologies that have and continue to change many industrial procedures. However, amid this technological advancement, the persisting physical strain on human workers remains a significant concern. Even with the advent of cobots aimed at alleviating burdensome tasks, certain physical jobs continue to induce fatigue in human workers. Addressing this challenge necessitates the development of robust solutions that combine technological innovation with human-centric considerations. One critical aspect in mitigating physical fatigue in human workers involves the application of Machine Learning (ML) models. These models heavily depend on data obtained from real-world situations that accurately represent the complexities of physical strain. However, this kind of data is frequently limited and costly to gather using sensors, which hinders the development of an effective ML model. This scarcity underscores the need for alternative approaches, with Synthetic Data Generation (SDG) emerging as a viable solution to this problem. The production of synthetic data offers a new approach to address the lack of relevant data needed to train machine learning algorithms. By employing techniques like Tabular Generative Adversarial Networks (GANs), synthetic datasets can be created, simulating realistic human physical fatigue detection features. Tabular GANs have, for example, been shown to be effective in creating synthetic data that closely resembles the statistical characteristics and patterns of real-world datasets. Furthermore, tabular GANs present a scalable and affordable response to the problem of data scarcity. The research reported here presents a novel approach centred on employing the Tabular GAN methodology to create synthetic datasets encompassing key features pertinent to the detection of human physical fatigue. The results of this study are expected to contribute substantially to creating robust solutions to alleviate physical strain and enhance human workers' overall well-being in industrial settings. The goal is to create datasets that accurately represent the complexities found in real-world scenarios where physical fatigue notably influences human performance. These synthetically generated datasets will serve as the foundation for training specialized ML models designed explicitly for detecting the development of human physical fatigue. The trained ML model will undergo rigorous testing and validation using a substantial repository of authentic real-world data. The model's accuracy and reliability in detecting human physical fatigue will be assessed through this evaluation process. The ultimate objective is to achieve a level of accuracy that demonstrates the model's proficiency in identifying and predicting the onset of physical fatigue in human workers within industrial settings. This research endeavours to bridge the gap between Industry 4.0 innovations and human well-being by leveraging synthetic data generation techniques to enhance the accuracy and efficiency of ML models in detecting human physical fatigue.

Arsalan Lambay, Phillip Morgan, Ying Liu, Ze Ji

Conference Proceedings

Ergonomic support for manual assembly through data-based assistance systems - challenges and solution ideas considering the legal framework conditions

While automation is already well advanced in series production, manual production is still used for small batch sizes and multi-variant production. However, such workplaces also need to be modernized to produce not only economically but also sustainably. Improving ergonomic working conditions poses a challenge for work research: how can data-based assistance systems provide employees with recommendations on favourable work design while considering the tight legal framework regarding the collection of biometric data? A consortium of labour researchers from five colleges and universities, six network partners and more than 30 companies are working on this and other questions in the context of the introduction of AI solutions in the PerspektiveArbeit Lausitz structural change project (PAL). The declared aim is to support the transformation of the Lusatia coal mining region by enabling small and medium-sized enterprises to drive forward digitalization and use simple human-centred AI solutions to make work more people-friendly, thereby increasing their competitiveness and flexibility.The prototype is to be implemented at an electronics service provider with manual PCB production. In collaboration with Mittweida University of Applied Sciences, an application is being developed to improve activity-related ergonomics at the workplace. Visual sensors at and around the workplace will capture images that are compared with data from standards, guidelines, and ergonomics methods in real time to provide a direct message at the workplace in case of negative stress (e.g. forced posture, one-sided strain, gripping space). The video-supported analysis is intended to help identify and correct ergonomically critical movement sequences and avoid them in the long term to maintain the health of employees.Mittweida University of Applied Sciences develops an application to improve ergonomics by using visual sensors to record and evaluate ergonomic factors. The video-supported analysis is intended to identify ergonomically critical movement sequences and avoid them.It is important to not only "get employees on board” in this process from the outset in the sense of "informing" them, but to "involve" them in the implementation of the project to be able to incorporate their expertise into the development and to increase acceptance of the planned project. This is because AI applications that process personal data are regulated by the GDPR and soon also by the European Union's AI Regulation, which is expected for 2024. The Works Constitution Act (§87) must also be observed. It stipulates an economical and dedicated data collection and the avoidance of unnecessary data. What exactly needs to be considered when processing personal data using AI systems and what specific technical solution the assistance system provides to protect sensitive biometric data - this content is conveyed to the employees in awareness workshops

Katharina Müller-eppendorfer, Katrin Meusinger

Conference Proceedings

Excavators’ Cabins Ergonomic Design Influential Factors Modelling: Preliminary Study

Excavator operators encounter demanding work environments with very high risks for discomfort, musculoskeletal disorders, and workplace accidents. In line with that, this study examines the relationship between ergonomic design influential factors using the structural model of excavator cabins design factors, using a sample of 32 excavator operators. Descriptive statistics were performed to describe the operator's age, height, weight, working experience, and excavator's lifespan. After that, the structural equations model was developed to describe the impact of latent variables related to ergonomic design of the cabin. This model was constructed by using 17 questions, which were categorized into 5 groups based on ergonomic design characteristics such as seat, armrests, commands, cabin, and working conditions. The findings indicate that the model exhibits favourable reliability and validity coefficients, a substantial effect size, and a satisfactory model fit. Further research is needed to increase the sample size, despite the preliminary nature of the current research and its satisfactory results.

Martina Perišić, Vesna Spasojevic Brkic, Ivan Mihajlovic, Aleksandar Brkic, Nemanja Janev

Conference Proceedings

Dumper Operator’s Workplace Risks: Preliminary Study

The dumper vehicle is essential for carrying out material handling activities and they are the primary cause of fatal accidents at both construction and mining sites. According to earlier studies, the vast majority of these incidents are caused by the operator’s behaviour. On other side, it is evident that dumper operators are often taking awkward postures and most of them are subjected to high and medium levels of musculoskeletal disorders.Due that fact, the aim of this paper is directed towards preventing accidents and/or incidents in dumper’s operations that originate from the operator’s error and which could be mitigated by the ergonomic adjustment. Specifically, the purpose of this study is to utilize artificial neural networks in order predict the probability of incidents and accidents on the basis of dumper operators’ workplace ergonomic intervention.In this research 40 dumper’s operators participated. Survey started with the 39-item checklist, which is based on prior research and there has been noticed that questions with lowest percent of positive answers are questions related to seat and armrests, so factors such as the seat height adequacy, the seat height adjustability, the seat be horizontal adjustment, the seat back support, the seat lumbar support, armrests availability, armrests height, armrests adjustability, vibrations through the seat, vibrations from the equipment through the floor, vibrations from the equipment through the control devices, the seat attachment to the sub cabin, and possibility to recline and rotate the seat are considered as primary factors in the investigation of the ergonomic adjustment of the dumper operators’ workplace. Poor working conditions originating from organizational factors are also recognized as one of the important focuses of this research. The obtained data were statistically analysed. Surveyed dumper operators came from various age groups and with varied work experience periods in the current study. The research revealed that the average age and height of dumper operator is 36.33 years and 178.35 centimetres, respectively. Moreover, despite the fact that their average weight is 90.33 kg, these body measurements, unlike the others which followed a Gaussian distribution, regardless of the average value indicate that a substantial proportion of operators are overweight and weigh more than 110 kg. In addition, the vast majority of operators in this survey had less than ten years of experience. Regarding the machines, the age of examined dumpers was less than ten years.In order to reduce the likelihood of accidents/incidents and injuries at work resulting from poor seat design, this study uses questions that covers the most significant factors affecting the quality of the working conditions related to the seat and armrests in dumper’s cabin. Furthermore, neural network was trained to predict the likelihood of injuries based on 14 different responses related to the seat and armrest, as indicated by the research's finding that the questions to which respondents provided the worst responses concerned the cabin's seat and armrest. The training dataset consisted of 80% of the obtained data, while 10% was allocated for validation and the remaining 10% was reserved for testing purposes. Used network has one hidden layer with 12 neurons. The neural network underwent training for a total of 16 epochs. The correct classification of 92.5 percent of the data demonstrates a high degree of accuracy in predicting potential accidents/incidents on sites where dumpers operate.Conclusions from the study indicate the need for additional research into an ergonomically optimized workspace for dumper operators in terms of the possibility of resolving identified issues through the further anthropometric surveys and possible application of modern information technologies, such as the installation of sensors, monitoring, and similar solutions.The sample size is a limitation of this study, and additional data collection is ongoing. The proposal for additional research is to collect additional data to increase the reliability of the analysis, and possibly investigate other types of mining machines to determine if similar problems occur with them.

Neda Papic, Mirjana Misita, Vesna Spasojevic Brkic, Martina Perišić, Nemanja Janev

Conference Proceedings

Work design in production: Foundations and recommendations for the implementation of mobile, time-flexible work design in chipping production

The discussion on new work models and the benefits of mobile, time-flexible work often focuses exclusively on employees in administrations. Employees in production and production-related areas often lack the opportunity for mobile and time-flexible work. Companies need to ensure that this imbalance doesn't impact collaboration among employees or influence the organizational culture. This paper presents the results of an empirical survey that provides an overview of the opinions of employees and executives in chipping manufacturing, addressing the research question as to what extent a mobile, time-flexible work design is possible for shop floor employees. Based on the findings, a model is developed on how mobile, time-flexible work can be facilitated in chipping production. For the survey study, an empirical investigation was conducted using a mixed-method design. A quantitative data collection was carried out through an online survey, while qualitative data were collected through guided interviews. 61 employees participated in the online survey, and eleven executives, including CEOs and production managers, were interviewed. Both employees and executives work in companies operating in the chipping production in Germany. The results indicate that, among other factors, the secure automation and digitalization of production processes enable employees in chipping production to perform tasks mobile and time-flexibly while the machine tool produces workpieces. Building on these results, a model for the design of mobile, time-flexible work is developed, emphasizing constraints for its implementation. In the future, the digitalization and automation of production will enable a mobile, time-flexible work design.

Jakob Weber, Marc-André Weber, Sascha Stowasser

Conference Proceedings

Technical and Socio-Technical Success Factors of AI-Based Knowledge Management Projects

The demographic change has a large impact on the labour market and poses a challenge to companies. With many employees going into retirement within the next 10 years, it is not just the workforce itself leaving the firms, but also their experiential knowledge that the workers gained over the years. Much of it is tacit and thus unobtainable through common documentaries of work processes. Keeping it inside of the company is crucial to ensure productivity and educate the upcoming generation of workers in their company. The project “KI_eeper – Know how to keep” has the goal to capture experiential knowledge and provide it to the workers during the production process automatically through an AI-based assistance system. The system is currently under development and requires careful consideration of the users’ needs at the production line. By choosing a participative approach, the employees are directly in touch with the developers and can influence the development of the system significantly. Managing both the available technical capabilities as well as the demands of the employees towards the system at the same time is key to have a successful outcome of the project. This paper shares the essential success factors both on the technical and socio-technical level to secure a seamless integration of an AI-based assistance system into production processes, based on a case study in a German manufacturing company.

Christian Cost Reyes, Nicole Ottersböck, Christian Prange, Adrian Discher, Sven Peters, Holger Dander

Conference Proceedings

Trusting AI: Factors Influencing Willingness of Accountability for AI-Generated Content in the Workplace

In the rapidly evolving landscape of Artificial Intelligence (AI) and business ethics, a critical area of focus has emerged: the willingness of leadership to assume responsibility for AI-generated content in decision-making processes. While the current public discourse predominantly addresses AI’s impact on customer service, potential biases, and job displacement, etc., a less explored yet significant aspect is how AI reshapes tasks and roles within organizations, particularly in decision-making.AI’s capability to analyse vast data sets expeditiously supports both operational and strategic decisions across various sectors. However, this support comes with ambivalent outcomes, ranging from enhanced efficiency to risk of taking decisions with negative business impact based on AI outputs with hidden biases. Such ambiguity can undermine trust in AI, especially when the rationale behind AI-generated recommendations is opaque.The central question of this paper will be the extent to which leaders are prepared to be accountable for decisions made based on AI insights. This includes scenarios where leaders themselves make AI-driven decisions, as well as situations where they are responsible for overseeing and endorsing decisions made by their team members based on artificial intelligence.In understanding the adoption of AI in decision-making, key factors influencing trust and usage of algorithms emerge. Research suggests, for example, that trust extends beyond algorithm accuracy, significantly influenced by social validation such as prior adoption by others, which can reduce cognitive load and improve engagement. Furthermore, cultural and age differences may play a crucial role. Additionally, an expectation of near-perfection performance from automated systems might lead to scepticism, especially when an algorithm falters, impacting ongoing trust and usage. These elements and many more might be vital in evaluating the readiness to assume responsibility for AI-generated decisions in the workplace.This paper aims to identify and categorize those criteria that have a central effect on the willingness to assume responsibility for AI facilitated decisions and AI generated content in companies. Those categories may later serve as a framework to be considered by management when adopting a strategy concerning their policies for AI based decision-making processes.

Ulrike Aumüller, Eike Meyer

Conference Proceedings

Participatory Approaches to Design Work in the Context of Digital Transformation: An Analysis of the Needs of Employees in Public Administrations

Digital transformation is significantly changing work, especially in the public sector. It's increasingly important for employees to be actively involved in designing work processes and making decisions about new or changing ICT. This article focuses on comprehensive analysis of participatory approaches in work design in public administration at local and state level in Germany. It examines the extent to which public administration employees at local and state level are involved in the design and decision-making processes pertaining to ICT implementation in their work area. Additionally, the article underscores the role of employee participation in mitigating the stress associated with workplace technology. The methodological basis of the article is an online survey conducted in 2023 among employees of public administration at local and state level.

Miriam Maibaum, Marc-André Weber, Sascha Stowasser

Conference Proceedings

Hand Vibration Threshold Mapper (HaViThreMa): a Haptic Vibration System

This paper describes the design aspects of a system created to assess vibration perception thresholds (VPT). i.e., the minimum thresholds of mechanical vibration that humans can perceive on their hands. The system can carry out these assessments on various areas of the hand, such as the fingertips and the palm, and assessments are conducted according to pre-established psychophysical protocols. The technical specifications of this system, referred to as Hand Vibration Threshold Mapper (HaViThreMa), are discussed throughout this paper, as well as its advantages and disadvantages when compared with similar systems that have already been used in other studies. The HaViThreMa uses piezoelectric actuators to produce vibration stimulus, and can acquire VPT data from one area while, at the same time, also delivering stimuli to one or more other areas. When carrying out assessments on multiple areas of the hand, this platform allows these areas to be tested in a randomized order, reducing subjects’ expectations regarding the location in which the next stimulus will be actuated, and increasing data consistency. By making use of these capabilities to design and carry out psychophysical studies, the information obtained from the gathered VPTs can help elaborate guidelines on how to better use piezoelectric actuators to design more effective Human-Machine Interfaces (HMIs).

Emanuel Silva, Isabel Lisboa, Tiago Matias, Rui Gomes, Adriano Carvalho, Paulo Cardoso, Nelson Costa

Conference Proceedings

Enhancing Ergonomics in Construction Industry Environments: A Digital Solution with Scalable Event-Driven Architecture

The construction sector remains among the least digitized and automated industries, where human cognitive intervention is still necessary for many tasks. These tasks often entail significant physical exertion, increasing the risk of Musculoskeletal Disorders (MSDs) when workers perform unexpected movements or events. While assessment methods and technologies like wearable devices, bio-signal sensors, and digital tools enable real-time monitoring of ergonomic factors, integrating them simultaneously presents a challenge. This paper describes developing and implementing an event-based architecture to address this complexity. This architecture monitors each integrated system in real time, offering workers immediate feedback on their ergonomic behaviors and adjustments while predicting potential hazards. Furthermore, it facilitates group work by enhancing coordination and communication among team members through real-time sharing of relevant ergonomic data and insights.The architecture consists of three layers that can be scaled according to needs. 1) The physical layer manages all data sources, such as motion capture, electromyography, and external sensor systems. Each system uses the IoT messaging protocol MQTT to send and exchange data. 2) The digital layer consists of several frameworks for stream-producing data and distributing the parallel computing through several microservices, and 3) the human-system interaction layer where the results or outcome of the services go. The information could be displayed to give feedback, warning alerts, or just notifications to the worker through AR glasses, or an external monitoring system could be used for post-ergonomics analysis. Our initial findings highlight integrating a motion capture system into the architecture. This integration empowers the deployment of ergonomic evaluation methods such as RULA, REBA, and the body joint angle range classification to mitigate ergonomic risks during construction tasks effectively.

Enrique Bances, Urs Schneider, Thomas Bauernhansl, Jörg Siegert

Conference Proceedings

The Role of Training Duration in Frequency Discrimination of Electrotactile Feedback

Electrotactile feedback is a promising communication channel in various applications, from healthcare to human-machine interfaces. However, the time needed to train the users remains one of the main challenges. This study examines the impact of training duration on user performance when using frequency modulation to convey information through electrotactile stimulation. We have employed an electrotactile stimulation system that includes a custom-designed 32-pad electrode for the thigh and custom-developed software for psychometric evaluation. Software included two electrode activation regimes, i.e., single electrode pad and distributed stimulation with multiple pads, that were used for training, reinforced learning and testing of the discrimination between four frequency levels. The study involved 34 healthy volunteers subjected to short and long training protocols to evaluate the impact of learning. The results showed that longer training significantly improved the recognition, confirming that training duration is a crucial factor for effective electrotactile feedback based on frequency modulation. The training effects were especially pronounced in more complex task, when stimulation was delivered to a randomly selected pad of the electrode array. These findings provide valuable insights for optimizing training duration in electrotactile applications.

Tanja Boljanic, Milica Baljic, Strahinja Dosen, Matija Strbac

Conference Proceedings

Technology for improved drivers’ safety: Testing a multimodal HMI

Using driving automation technology to shift from reactive to anticipatory HMI could enable drivers to improve safety. To this end, a system helping drivers to anticipate risks was developed. In a previous study, cross-analysis between accident databases, driving instructor expertise and on-road observation led to prioritize seven types of risk. A system based on sensors and prediction algorithms was then developed to recognize and objectify risk levels. The present study was the first user-test of the system. Twelve participants were asked to drive as usual and evaluate timing, relevance, utility, and usefulness of warnings to improve risk awareness. They were also required to report risks that they considered missed by the system. Participants drove in an area involving risks related to infrastructure and traffic configurations: (1) a pedestrian crossing frequently hidden by buses stopped at the station (BUS), (2) an unexpected sharp curve, possibly hiding an obstacle (CURVE), and (3) a pedestrian crossing with reduced visibility on sidewalks and high pedestrian traffic (CROSSING). Informative icons were displayed when approaching respectively CURVE and CROSSING to indicate the type of risk (i.e., permanent risks linked to the infrastructure). They were associated with a soft tone to ensure they were perceived and, thus, evaluated by drivers. A LED bar, activated at the bottom of the windshield, indicated the location of potential hazard in CURVE and BUS (i.e., transitory risks linked to traffic). Due to the high probability of meeting pedestrians in BUS, the LED bar was associated with an urgent sound. The results showed that both LED bar and sound were highly relevant in BUS situation, as drivers recognized that overtaking the bus was a frequent and very dangerous practice. In CURVE, drivers considered that an informative icon and sound were useful or, at least, not annoying since they experienced the severity of the turn. However, the LED bar appeared not very relevant because drivers were already warned by the informative icon and thought that encountering an obstacle on their lane was not certain. They rather considered that the main risks were lane departure or oncoming traffic. In CROSSING, the informative icon was not understood because the presence of a pedestrian crossing seemed obvious, or because the driver was already coping with potential pedestrians. Finally, drivers expected that the system would report pedestrians walking on the road, or close to cross, because they could represent an obvious risk of collision in case of distraction. We conclude that the LED bar is only effective for guiding attention on risk related to the traffic. Informative icon related to infrastructure seems understandable only when risk is experienced by drivers. Reporting collision risk with pedestrian, when possible, is a desired function for improving safety. The study supports changes in multimodal HMI strategy to improve system efficiency, especially to carefully design HMI signals to be associated to perceived risks or, afterwards, to missed risks.

Arnaud Koustanai, Sabine Langlois, Jean-baptiste Haue

Conference Proceedings

Design and Implementation of Gesture Interaction for a Command And Control System Based on Multimodal Data Fusion

With the continuous improvement of people's demand for interactive experience,gesture interaction technology, as a natural and intuitive way of human-computer interaction, has attracted more and more attention and research. However, the research and application of gesture interaction in command and control system is still lacking. Firstly, the characteristics of command and control system and the key technologies of gesture interaction are investigated in this study.Then a series of operating gestures that meet the application scenarios of the command and control system are designed,and a gesture recognition algorithm based on the fusion of image data and skeletal data is proposed.By integrating with the command and control system, the function of controlling the command and control system through gestures is realized. Finally, the feasibility and effect of gesture interaction technology in the application scenarios of command and control system are evaluated through experiments.The experimental results show that the application effect of the technology in the command and control system system is good, which can reach 94.89% under typical tasks, improving the interaction efficiency and interaction experience.

Méng Méng Gāo, Deqiang Fu, Ning Li

Conference Proceedings

Region Selector Usability Test：Dropdown vs. Tabs

When ordering cloud computing products, users need to select the instance location based on their local area, which can effectively reduce network delay and improve network access speed. At present, the cloud products have a wide geographical coverage, with more than 30 regional options. The style of region selector will affect the efficiency of user region selection. The purpose of this study is to compare the usability of the dropdown and the tabs. Firstly, the optimization process is determined based on the international standard process of user experience, and the problem of the current selector is analyzed through the analytic hierarchy process and Dropdown is determined as the comparison scheme. Then through the comparative analysis of page performance indicators, behavior indicators and business indicators, the data results show that the dropdown selector has better user experience and business value. The above findings can guide cloud computing platform designers to improve a region pooling selector design and provide users with a more user-friendly interactive experience. At the same time, the research process and method can be used for reference in interface design to improve work efficiency and effectiveness.

Yuanyuan Liu, Ying Xia

Conference Proceedings

Bimodal Affective Computing Interfaces for Emerging Artificial Intelligence Paradigms

For nearly 30 years, Affective Computing has described a paradigm for human-computer interaction where machines are designed to respond to human affect. With recent acceleration in the development of artificial intelligence, the possibility of conscious machines with their own experiences and emotions is emerging. This is an opportune moment for engineers and designers to speculate and consider how affective computing paradigms might be expanded to also include machine affect and an emotional capacity of AI. We explore these themes through the development of a prototype that introduces human users to machine affect.

Ian Gonsher, Joshua Phelps, Shiyu Yan, Xiaoxi Yang

Conference Proceedings

The Evolution and Evaluation of Pie Menu Interactions in Text Input

Pie menus, also known as radial menus, have been studied in the realm of user interfaces. Various applications of pie menus have been explored, with research focusing on their usage in diverse contexts and their impact on user behaviour. As interactive devices such as touch screen mobiles, virtual reality, mixed reality, and touch-based physical products rapidly permeate into the users’ hands, they are driving the demand for studies in multi-model interactions. The pie menu is a promising UI pattern that offers various selection options within a constrained radius. Mostly, the interaction requires human action like click, swipe or steer gestures, and efforts like visual search, decision time. Despite the efforts by different research groups to derive an effective model on human performance, we observed that the derived models on Pie menu are studied independently, but all models are based on Fitts’ law and its extension in text input methods. We conducted a comprehensive literature review of publications from ACM, IEEE, HFES, and few others focusing on the evolutions of Pie menus and their derived models. Our qualitative study examined three aspects: application of pie menu in the context of text input, touch mobile interfaces, models applied or derived in these studies. Through a mixed-method analysis, we evaluated these aspects and narrowed our focus to 38 papers that specifically address text input methods and human performance. Our findings revealed that the performance of pie menus varied significantly across different use cases. Some studies reported that the northwest side of the menus was harder to access, while others suggested that pie menus were theoretically easy to learn but performed less effectively than square menus. However, in the context of text input, pie menus outperformed conventional keyboards. We propose that future research should aim to develop a standard methodology for testing pie menus that can be generalized across different contexts with minimal modifications. The results of this review will provide valuable insights for researchers and designers working on Pie menu interactions, helping them understand previous research, identify gaps, and determine future directions.

Rupesh Nath, Anirudha Joshi

Conference Proceedings

Error Communication in Manual Assembly through a Projection-Based Assistance System

While automation is advancing, manual labor continues to be relevant due to its flexibility, adaptability, cost considerations, and the unique qualities that humans bring to the workforce. In certain tasks and contexts, human involvement remains essential. One context is manual assembly which is still an essential part of many production systems due to the diversity of product variants and the challenges of producing in batch size 1. Manual assembly often outweigh the high costs of automated solutions. Nevertheless, a high variability of products and high quality standards pose a challenge to employees. Given the shortage of skilled workers, it is crucial to provide support for employees and ensure the quality of manual assembly. This includes assisting unskilled workers in the assembly process. In addition to optimizing training time and available instructions, error detection during the assembly process can serve as a mean to reduce errors and support employees. Error detection can be separated in two aspects: on the one hand the technological solution to recognize process steps and deviations and on the other hand the error communication when errors occur. This study lays the groundwork for the latter by investigating the effect of error communication during the process on assembly quality and employee satisfaction.An on-site manual assembly station was set up and n=12 participants have taken part in the study. Their task was to complete an assembly task based on instructions projected onto the work surface. In the control condition, participants independently navigated through the description of instructed assembly steps using displayed arrows. The experimental condition used a Wizard-of-Oz experimental design in which participants were informed that the assistance system automatically recognizes their process and detects errors. In the case of an error, a correction prompt emerged instead of the next instruction step. Performance measures included the required assembly time and recorded errors. Participants also completed questionnaires on workload (NASA TLX), usability (SUS), control beliefs in dealing with technology, and subjective performance.Data analysis is pending. Comparable measures will show in which regard the two conditions differ, shedding light on the effect of error communication during the assembly on quality and employee satisfaction. In the end, the implications for future research and application will be discussed in the paper.

Antonia Markus, Lea Marleen Daling, Esther Borowski, Ingrid Isenhardt

Conference Proceedings

Human Factors in Technology and Knowledge Transfer: A Qualification Concept of the WIN:A Project for Effective Personal and Medial Transfer Structures

Labour research continuously develops new findings on the design of work. It also aims at translat-ing research results into practical products such as guidelines, checklists or implementation tools. These products can support small and medium-sized enterprises (SMEs) in overcoming current challenges such as the skills shortage, digital transformation or the energy and climate crisis while ensuring good working conditions. But recent scientific contributions show that there is still an ex-isting transfer gap (for an overview, see Borowski et. al. 2023). Research findings and products from the field of ergonomics are often unknown in business practices. Links to „everyday business“ in SMEs and intermediary organisations (e.g. consultants or associations) are not evident or are hindered by difficulties in translating these findings and products into the respective application context.Due to limited time and personnel resources, SMEs often do not have the necessary means to thoroughly search for labour science and ergonomics insights or products and to check their rele-vance and practical suitability. At the same time, intermediaries also have major gaps in their knowledge, even though considering labour research topics to be highly relevant for their work and aspiring to have more customised tools (Cernavin, Joerißen 2022). Thus, how can research findings be processed more effectively and disseminated more successful-ly? What could be suitable concepts for relevant stakeholders (scientists, works councils, SMEs and intermediaries) to enable them to use the (technical resp. medial) products and results from labour research effectively? In order to answer these research questions, there is ongoing investigation in the project WIN:A (Knowledge and Innovation Network for Labour Research). This investigation aims at deducing and suggesting different instruments for effective knowledge transfer, including a qualification concept differentiated by target groups and a software platform – the “Transfer-Plattform – „’Management – Arbeit – Forschung’“ – with a network and topic graph as well as a mixed methods toolbox. In order to manifest the importance of labour science results and findings to SMEs and intermediar-ies, the stakeholders need to be qualified to use the products prepared by WIN:A in their daily work. For this purpose, a training module will be designed that will be integrated into the existing qualification structures of intermediary organisations. Due to specific needs according to the re-spective group of stakeholders, it is necessary to understand their requirements towards those training module as a qualification measure. In order for it to be easily accessible and comprehensi-ble for the target group, their semantics have to be met on the one hand. On the other hand, it is considered useful to provide the possibility for interaction and participation as it is assumed foster learning and knowledge transfer. As a basis for this qualification, a training guide („Qualifizierung Arbeitsforschung“) is being developed together with intermediary organisations that will also be accessible to further stakeholders. Initial insights into the training module will be presented in the paper available. Sources:Borowski, E.; Cernavin, O.; Hees, F.; Joerißen, T. (Hrsg) (2023): Erfolgreicher Transfer in der Arbeits-gestaltung - Wie Dienstleistungen zur präventiven Arbeitsgestaltung und Ergebnisse der Arbeitsfor-schung die Akteure in den Unternehmen wirkungsvoll erreichen. Münster, New York: Waxmann Verlag (erscheint Ende 2023)Cernavin, Joerißen (2022): Arbeitsforschung in der KMU-Praxis - Untersuchungsergebnisse zum Transfer von Kompetenzzentren Arbeitsforschung in der Region, Stiftung Mittelstand Gesellschaft Verantwortung, Schriftenreihe 1 8/2022.

Anja Koonen, Nina Collienne, Theresa Joerißen, Esther Borowski, Ingrid Isenhardt

Conference Proceedings

Virtual training for the maintenance of machine tools

In the course of digital change, systems and processes are increasingly being planned, adapted and tested using computers. Innovative technical aids are needed to guide staff and relieve them physically and mentally in order to cope with flexibly changing work tasks such as the maintenance of machine tools.Virtual reality (VR) offers attractive opportunities for employee training and enables the practical acquisition of knowledge. The use of VR is based on the idea that experiences are sustainable and do not have to be learned in the abstract.The interaction between man and machine plays a decisive role in the operation and maintenance of machines. Efficient interaction is necessary to minimize errors and maximize efficiency. Traditional training methods reach their limits here, as they are often not realistic enough and employees find it difficult to apply what they have learned in practice.A virtual learning environment, which virtualizes a real work area, uses integrated, didactically prepared scenarios to clearly convey methodical knowledge and practical skills to increase operating safety on production systems. The virtual training thus enables employees to practice specific technical sequences of actions and sub-actions or hand movements in a realistic manner and to learn them in a sustainable way for later practical use. The focus is on typical operating actions such as deliberate, sequential actuation or disassembly for the maintenance of add-on parts on machine tools.In order to ensure the desired learning success (error-free, safe execution of the real work task) and employee acceptance, the correct structure of the operating sequences and an exact representation of the graphic interfaces are conducive to learning and acceptance.The advantages of using virtual learning environments were demonstrated using the example of the virtual commissioning of a machine tool. For example, after successfully completing the virtual learning scenario, the test subjects felt much more confident when carrying it out on the real machine.However, it also became clear that VR-based learning environments with complex virtual learning scenarios also require further development. For example, the use of VR hardware (VR glasses and controllers) and software has potential for improvement in terms of user-friendliness. Tests with users showed that handling the VR hardware was problematic at the beginning. The virtual learning scenarios often could not be completed because the virtual, user-induced movement and implementation of operating actions was not possible for the user in the scenario. For example, employees were asked to use the controller to perform various operating actions, such as virtually switching the machine tool on and off, which very often failed. Eight out of ten test subjects were unable to cope with the controls and navigation within the virtual learning environment, which meant that the actual training was not carried out at the user's request. This conflicts with the fact that the prerequisite for the acceptance of a virtual learning scenario is, on the one hand, the existence of a high recognition value between the virtual learning environment and the real environment and, on the other hand, the success of the training must be ensured in every case.Based on the findings described for the virtual commissioning of a machine tool, the article presents the methodical development of a virtual learning scenario that focuses on maintenance tasks for a lathe. This includes the initial development and two stages of improvement.The usability of virtual training in the field of maintenance is to be evaluated and improved using this learning scenario as an example. The article therefore describes the technical tests carried out on the feasibility, improvement and acceptance of the test subjects and discusses questions of usability and learning success.

Robert Eckardt, Leif Goldhahn

Conference Proceedings

Innovative Methods for Robot Programming: Development and Comparison

In considerations of Human-Machine Interaction or Human-Robot Interaction, humans are often regarded as users or employees. However, the programming of the robot also plays a crucial role, as it is a form of Human-Machine Interaction. According to the International Federation of Robotics, over 500,000 industrial robots were newly installed worldwide in 2022. It is predicted that annual installations will continue to rise in the coming years. This necessitates providing access to robot programming for less specialized personnel. In addition to traditional programming methods, there are already some more intuitive concepts in this field. This includes, for example, guiding the robot arm by hand as an alternative to control through buttons or a joystick. Furthermore, classical textual programming is often replaced by a form of Drag & Drop programming.In addition to these commercially available programming variants, novel programming concepts are conceivable, such as the integration of voice control, the use of mobile devices such as phones or tablets, robot control via gestures, or the utilization of products from the field of video games. Moreover, Augmented Reality technology exists, which can support intuitive robot programming.In this article, we present our own developments on the topic of innovative methods of robot programming. Firstly, an innovative approach to robot programming using gesture control via the Kinect camera is introduced. This method allows the operator to precisely control the robot through multiple waypoints and save them using virtual elements projected by a video-projector. Subsequently, the robot can efficiently traverse the saved points, demonstrating significantly faster programming compared to conventional methods in experiments.Furthermore, we present an innovative Mixed Reality approach. By using a Head-Mounted Display, the operator can control the robot in a particularly advanced manner. This includes not only intuitive path planning in both joint and Cartesian space but also the realization of force-controlled movements and the provision of crucial status information from the robot. This holistic approach creates a novel interface between humans and robots, significantly improving efficiency and control in robotics.Next, a joystick-based robot movement with haptic feedback is introduced. This control allows the operator to precisely and intuitively control the robot using a joystick. The haptic feedback enhances the user experience by providing tactile feedback to better understand the robot's movements.The goal of all these methods is to simplify programming, even without in-depth expertise. We compare these methods with each other and with established programming procedures in terms of time efficiency, ergonomics, programming comfort, etc. Additionally, we analyze whether the considered procedures are suitable for individuals without specific expertise.

Mohammad-ehsan Matour, Christian Thormann, Alexander Winkler

Conference Proceedings

The Potential of Haptic Motion Cueing to mitigate Motion Sickness in Highly Automated Passenger Cars

The evolution of road vehicle towards high levels of automation is forecasted along with a higher prevalence of motion sickness on board such transportation systems. Experiencing such situations may precipitate concerns related to comfort, safety and trust, potentially compromising the overall acceptability of these vehicles among users. While prevalent in various modes of transportation, motion sickness is an intricate physiological reaction of the human body, likely to be caused by inconsistent perception of the motion forces, and a lack of postural stability. The use of haptic stimuli as sensory motion cues (“haptic cues”) should be underscored as an effective countermeasure. This approach offers the advantage of seamless integration with other mitigation techniques. Haptic cues also have virtues for helping in the estimation of self-motion in space and anticipation. Through a concise analysis of the prior research, this paper surveys the potential strategies and systems for the effective delivery of haptic cues to alleviate motion sickness in cars while considering the prerequisites associated with passenger comfort. The results show that vibrotactile and arthrokinetic signals can act as force-based haptic cues to mitigate motion sickness in cars. The provision of airflow, concurrently serving as a thermal cue, shows potential for motion sickness mitigation, but the exact underlying mechanisms remain unclear. Studies conducted in cars suggest that haptic stimulations can only be effective when congruent visual cues are provided. The different types of haptic systems are proffered for potential in-car integration, along with their respective operating principles to enhance perceptibility.

William Emond, Mohsen Zare

Conference Proceedings

Concept development and implementation of a trend-based work analysis using digital tools and studies to identify load peaks

As part of the “Perspectives on labour research Lusatia” (PAL) project, five chairs at Zwickau University of Applied Sciences are working with industrial partners to develop low-threshold methods and tools for simplified screening of work characteristics. Traditionally, the assessment of work systems requires extensive specialist knowledge in the areas of occupational safety, ergonomics and operational design. Creating a hazard and stress register as a basis for work system design is therefore time-consuming and most of the times cost-intensive, which leads to a wait-and-see attitude, especially among small and medium-sized companies. However, innovations in the field of work analysis are now enabling the use and integration of smart, digital assistance systems, such as smartwatches or fitness trackers, combined with portable, intelligent environmental measurement technology. This combination allows an autonomous, in-house assessment of the stresses occurring without the need for external specialists and expensive equipment. The protection of personal rights is guaranteed by anonymized and pseudonymized data transmission. The departmental or process-specific evaluation of the collected data using machine learning creates an indicative stress assessment that enables work to be organized in line with all requirements. The resulting rough classification of key areas for action serves to define priorities for action and supports targeted decision-making processes for further measures, in which experts are involved on a selective basis. As a result, companies can carry out a focus-oriented and therefore economically sensible optimization of work design. Of particular importance, however, are the expected positive effects on employees, such as increasing motivation as well as higher job satisfaction.

Fabian Dietrich, Stefanie Liebl, Torsten Merkel, Gabriele Buruck

Conference Proceedings

Who is Responsible for What? Combining HTA and RACI for Modelling Cooperation in Remote Operation Center of Trains in Future Railway Sector

Since 2020, the German national railway company (DB) has been working on the "Digital Rail" project, with aim to create a better environmental balance, higher travel capacities and smoother processes by 2030. The focus here is on the large-scale use of automation. In the future, trains will run autonomously and identify hazards themselves using sensor technology. Autonomous trains with AI will take over tasks from actors, which means that the scope of actor’s tasks will be transformed. Actors, such as train operators will move away from the train to Remote Operation Centers (ROC). This will require new processes for humans and machines and a fundamentally new digital infrastructure, as humans will now act as a fallback level for the autonomous train. This will lead to new organizational and technical structures (monitoring, intervention in autonomy) for control centers. These control centers can be seen as safety-critical systems with multiple actors depending on each other. Therefore, it is important to look at the cooperation between these actors and how they execute tasks. In our case, hierarchical task analysis (HTA) was chosen to identify the tasks of the actors. In order to conduct this HTA, some assumptions were made regarding the scenario and the actors involved. The scenario depicts an autonomous train running on a track section that has an insufficient infrastructure to support autonomous train operation. The actors are organized in a ROC. There is the remote train operator, who has the task of monitoring the autonomous train and intervenes if necessary, and the dispatcher, who handles the disruption at management level. In our context the AI that controls the train is also classified as actor.This information is now being used to create an HTA and to apply the RACI model (Responsible, Accountable, Consulted, and Informed). RACI assigns these responsibility-roles to the individual actors in a so-called responsibility assignment matrix in which all relevant actors are involved.In addition, each responsibility-role gets assigned a quantitative value that reflects the level of responsibility. This makes it possible to identify tasks that have a high concentration of responsibilities by different actors (depending on the sum of the values). The overall model was then validated and adjusted with the help of semi-structured expert interviews. The experts from the rail digitization sector suggested minor adjustments to the model. The response to applying the RACI model to HTA in the rail sector of the future (for stakeholders in the ROC sector) was received positively.This HTA-RACI-ROC model can be used in several ways. On the one hand, the model can be used to detect patterns between new tasks and roles in ROC in order to provide appropriate actions for the organizational structure. On the other hand, it can be used in design methods for control rooms that use HTAs as input. This would require the development of a formalism of the RACI model to the HTA.

Marcel Saager, Jörn Kruse, Anja Naumann

Conference Proceedings

The General Automation Level Allocation (GALA) Framework, or: Why Do We Need Another Level of Automation Framework?

In the realm of sociotechnical systems, Level of Automation (LoA) frameworks are commonly used to determine adequate types of automation support for tasks in which human operators are involved. This paper introduces the General Automation Level Allocation (GALA) framework in response to recognized limitations in existing LoA frameworks. While these frameworks have contributed significantly to the formalization of human-automation interaction for the systems they were designed for, they often struggle when dealing with new sociotechnical systems. Some of the main limitations recognized for existing LoA frameworks include: (1) Lack of versatility in terms of missing levels for some “automated functions”, since they are designed with specific systems in mind; (2) Limited precision in the definition of the categories for assigning LoA to specific functions and complex technologies; (3) Limited support in the identification of outcomes of human-automation Interaction at different LoA (e.g. in terms of emerging behaviors or in terms of safety-related implications); (4) Limitations regarding characterization human cognitive processing in off-nominal or complex conditions; (5) Not fully addressing the dynamic allocation of tasks and responsibilities based on changing conditions and real-time priorities. Because of these limitations, some researchers are not satisfied with existing LoA taxonomies and believe that there is even no need to think deeper about LoA taxonomies as basis for or input to design of complex sociotechnical systems.To address the stated issues, GALA offers a two-dimensional approach aiming at being compatible with other previous LoA frameworks and applicable to the design of future systems. It is designed to analyze and classify the appropriate levels of automation for different information processing stages (e.g. information acquisition, information analysis, decision making, action implementation) involved in a task based upon the results of a hierarchical task analysis. GALA is compatible to established state-of-the art methods (Parasuraman, Sheridan & Wickens, 2000; Save, Feuerberg & Avia, 2012; Kaber, 2018) applied to study specific aspects of human-system collaboration in more depth, such as the coactive design method (Johnson et al., 2014). Finally, plans for GALA validation will be presented aiming to ensure that it provides sufficient applicability to various sociotechnical systems of diverse domains, each with its unique requirements and challenges. Further, an outlook on an alternative more compact version of the framework is provided which addresses the specific needs of dynamic task allocation in real-life situation.

Daniele Ruscio, Adrien Metge, Marvin Schopp, Daniel Dreyer, Benedikt Petermeier

Conference Proceedings

Usability assessment of Extended Reality applications. A review

Extended Reality (a continuum that encompasses Virtual Reality, Mixed Reality, and Augmented Reality) is a recent concept that is gaining traction as new concepts of Cyber-Physical Systems are being researched and developed exploiting and integrating different modes of interaction (e.g., visual, language, audio, haptic). The new interface technologies find application in a host of fields, such as education and training, healthcare, security and defense, engineering and maintenance, and entertainment. Extended Reality usability assessment faces new challenges, considering the potential negative impacts (e.g., sickness, discomfort, and cognitive load) of using an immersive environment, and the need for strategies to avoid or, at least, mitigate such impacts. This paper reviews the state-of-the-art of Usability assessment methods applicable to the Extended Reality spectrum, categorizing them and identifying gaps to be bridged in the future.

Mario Simões-Marques

Conference Proceedings

Are Structured Analytic Techniques (SATs) the Missing Component in Cognitive Warfare? The Future of ISR Military Operations

U.S. Intelligence, Surveillance, and Reconnaissance (ISR) operators collect critical information with respect to our adversaries ground movement patterns, weapon capabilities, and strategic framework to support future military direction and enhance Joint All-Domain Command and Control (JADC2) situational awareness. However, the ability for ISR operators to (1) detect and identify essential elements of information (EEI) within vague or ill-defined content and (2) fuse and disseminate collected information across the JADC2 enterprise is extremely challenging. To combat these issues, ISR tools have been developed to assist and facilitate the comprehension of collected intelligence in an effort to augment and enhance decision-making efficacy. More recently, Structured Analytic Techniques (SATs) has been an area of interest within military applications in an effort to support processing, exploitation, and dissemination (PED) of collected intelligence. SATs comprise of a systematic process that enhances critical thinking and logical reasoning by reducing cognitive biases. Previous research has discovered that implementing SATs when providing vague/ill-defined content has been shown to improve decision-making performance. Therefore, the objective of this study was to add to the body of knowledge by evaluating the effectiveness of Sphinx, an ISR decision-support tool focused on SAT methodology, when providing vague/ill-defined content. Four groups of 10 active-duty military operators (N=40) were randomly assigned to one of two analytic techniques (Sphinx or Control) and provided textual content in either incremental or complete sections with the objective of correctly detecting embedded EEIs resulting in improved performance accuracy. The findings discovered that providing active-duty military operators with Sphinx coupled with an incremental workflow methodology resulted in the highest solution accuracy compared to all other conditions (i.e., 6 of 10 operators identified the correct solution – 60%). Moreover, operators that were provided Sphinx detected significantly more EEIs compared to the Control condition (p<0.01). This discovery provides new evidence that equipping active-duty military operators with Sphinx, an ISR decision-support tool, can enhance the detection of EEIs resulting in improved performance. More specifically, Sphinx enabled the operators to better understand vague content leading to greater detection of EEIs. The processing, exploitation, and dissemination (PED) of EEIs within Sphinx can greatly benefit the JADC2 enterprise by enhancing situational awareness and future military direction and recommendations.

Justin Nelson

Conference Proceedings

Negative Emotional Valence Shades, but Does Not Inhibit, Interaction and Spatial Processes

In the context of developing new inclusive products tailored to the individual needs of users, understanding emotional valence and its influence on human interaction processes is essential. Particularly, when analyzing interaction with specific virtual products, the influence of various factors inherent to both the user and the product in obtaining the experienced emotional valence becomes evident. In a cross-sectional study involving non-equivalent user groups, it was observed that negative emotional valence derived from interaction processes with virtual products did not interfere with the retrospective spatial construction of individuals as users. For this purpose, an experimental group and three control groups were employed to compare behavior regarding spatial processes. The spatial components were derived from the virtual product analyzed in the experimental group.

Lorena Olmos Pineda, Jorge Gil Tejeda

Conference Proceedings

The use of wearable sensors for ergonomic risk assessment of surgical procedures: a literature review

Surgical procedures place significant physical demand on surgeons, frequently requiring long periods of standing, repetitive and/or forceful movements, and sustained awkward postures, which raises the possibility of developing work-related musculoskeletal disorders (WRMSD). In response, several ergonomic risk assessment methods have emerged to identify risk factors in the workplace. A transformational approach involves associating wearable sensors to the ergonomic risk assessment data collection procedures, offering significant advantages over self-reporting and observational methods. Wearable sensors enable the use of a real-time quantitative approach to monitor surgeon’s exposure to risk factors during surgeries.This paper provides a comprehensive literature review on the use of wearable sensors for ergonomic risk assessment of surgeries, highlighting their strengths and limitations. Moreover, it provides an in-depth analysis of the assessments described in the studies. The majority of the reviewed studies were published in the last three years, confirming a growing trend in research on this topic. The wearable sensors, whether used individually or in combination, include inertial sensors to assess exposure to awkward postures or repetitive movements and sEMG sensors to measure muscle activity parameters.The significance of this paper lies in its potential to guide future research directions, inform best practices in ergonomic risk assessment methodologies, and influence the development of targeted interventions to mitigate the exposure to risk factors faced by surgeons.

Catarina Santos, Ana Teresa Videira Gabriel, Cláudia Quaresma, Isabel L. Nunes

Conference Proceedings

Exploring the impact of visualizing key factors in badminton tournaments on audience experience

The current statistical data in badminton event broadcasts primarily present score information for real-time match situations, offering limited insights into key events that influence the direction of the game. Additionally, past research has predominantly focused on providing post-game analysis for professionals, with a limited investigation into audiences' opinions on broadcasted statistical information. However, for both broadcasters and audiences, statistical data can aid in understanding the game content and enhance entertainment value. This study explores the evaluation of badminton event broadcasts by audiences with varying levels of experience and examines whether visualizing key events can enhance audience comprehension and enjoyment.This study consolidates international badminton event broadcast channels, selecting three channels based on event grades for testing. Through surveys, 30 participants were invited to participate in physical experiments, comprising 15 high-experience and 15 low-experience individuals (participants), and viewed simulated broadcast interfaces to assess their evaluations of the content. Two samples displayed scores for all games in badminton matches, while the third sample presented statistics for winning game points and the current game's score. Evaluation methods included: (1) participants are asked to explain and illustrate the game situation after watching the match; (2) participants respond to event-related questions regarding the broadcast interface, and complete the survey that included re-viewing intention, word of mouth, sport involvement, hedonic and utilitarian attitudes scales; (3) semi-structured interviews aimed at understanding participants' evaluations and suggestions for the viewing experience.Results revealed that (1) participants with high experience rated scales of re-viewing intention, word of mouth, sport involvement, hedonic and utilitarian attitudes significantly higher than low-experience participants; (2) 50% of participants accurately recorded scoreboard information, including winning player, final score, and game points. There are no significant differences between high and low-experience participants or different broadcast interfaces; (3) 80% of participants indicated that presenting scores for all games on the broadcast interface helped anticipate the match's overall direction; (4) 60% of participants preferred using the colour to highlight winning game points in the scoreboard for clearer identification of the ongoing game score; (5) 67% of participants believed trajectory information improved understanding the game; (6) 76% of participants suggested adding visual trajectory information to enhance understanding and enjoyment of the game, with 53% recommending providing this information during replays, as offering it during the game might disrupt the viewing experience. This study, by understanding audience evaluations of badminton event broadcast interfaces, provides relevant suggestions for future improvements in visualizing broadcast information to cater to fans with different levels of involvement.

Yuan Chieh Lee, Meng-Cong Zheng, Li-Jen Wang

Conference Proceedings

The Impact of Enhanced Information Presentation in Sports Event Broadcasting on Viewer Experience - A Case Study of Penalty Shootout in Football

The widespread adoption of online media allows sports events worldwide to engage audiences collectively; football is no exception. It plays a significant role in providing entertainment and promoting social interaction. Due to varying levels of audience engagement, it is challenging to collectively enjoy the pleasure of watching the game. Therefore, this study aims to investigate how data visualization can enhance the experience of audiences with different experiences.The study focuses on the penalty shootout interface in football broadcasts, aiming to assess whether the level of sports engagement among viewers influences the perception, satisfaction, and intent to rewatch through visualizations in the interface. Through a survey of participants' football viewing experiences, 36 participants were invited to participate in physical experiments, comprising 18 high-experience and 18 low-experience participants. The control group had no visualization information added, and the experimental group had five different visualizations: penalty shootout data for both sides, tactical board, shot ball speed, player information, and score - added to the contextual simulation videos. Evaluation methods included: (1)Participants were requested to watch scenario simulation videos and record their behavior and verbal expressions. (2) Participants were required to complete three questionnaires - Perceived Quality, Satisfaction, and Re-viewing Intentions. (3) Task responses and semi-structured interviews were conducted to understand participants' perspectives on the visualization information. (4) A five-point Likert scale was used to evaluate the participants' comprehension and preference for the visualization elements. Results indicated that 97% of participants preferred interfaces with data visualization, believing that adding visualization information contributes to understanding the match situation and enhances the enjoyment and immersion during viewing. We found (1) significant differences in perception between high and low-experience viewers, with higher-experience participants having a greater perception of visualization information, (2) Although the level of understanding of visualization varied between the high and low-experienced respondents, the highest level of understanding of score visualization was found in all of them, (3) From the investigation of five visualizations, it was found that both high and low-experienced participants exhibited similar preferences for visualizations. In descending order, the preference ranking includes player information, penalty shootout data for both sides, shot ball speed, tactical board, and score visualization. (4) There was a significant difference in the level of comprehension and preference among the five types of visualized information. This research suggests that when optimizing interfaces in the future, designers should prioritize design improvements based on preference levels and use evaluations as a reference for subsequent enhancements.

Ting Chun Lung, Meng-Cong Zheng, Li-Jen Wang

Conference Proceedings

Enhancing Low Basketball Experience Viewer Broadcast Experience via Data Visualization: The National Basketball Association Case Study

Sports broadcasting companies have been actively advancing various information visualization technologies in recent years. The current technologies and related literature predominantly focus on the professional aspects of sports instead of the general audience. According to statistical data, as much as 49% of the population in the United States are not fans of the National Basketball Association (NBA). Therefore, this study aims to broaden the NBA audience base by exploring the preferences and understanding of individuals with low experience regarding the basketball broadcast interface and the five visualized information elements. The goal is to enhance the viewing experience and economic benefits of NBA broadcasts. This study utilizes three NBA broadcasting platforms, Bally Sports, ESPN, and TNT as experimental samples. Each sample comprises 10 participants. Through an online survey, we selected 30 participants with low basketball experience for the experiment. The assessment steps included: (1) Participants watched broadcast videos and employed the think-aloud protocols. (2) Applied comprehension quizzes to assess participants' understanding. (3) Participants will fill out scales including the Evaluation of Media Entertainment Experience, NASA-TLX, Hedonic and Utilitarian Attitudes, and Re-viewing Intention, reflecting their evaluations and experiences regarding the broadcast visuals. (4) Conduct semi-structured interviews to gather insights and suggestions regarding participants' thoughts on broadcast interfaces and the five visualized informational elements. Results revealed that: In comparing the three sports broadcasting platforms, TNT performed the best on the Evaluation of Media Entertainment Experience scale, Hedonic and Utilitarian Attitudes scale, and Re-viewing Intention scale. ESPN was rated next, followed by Bally Sports. According to the NASA-TLX scale results, Bally Sports scored the highest in workload, followed by ESPN, and TNT scored the lowest. Most participants indicated that TNT's interface is simple and easy to understand, and ESPN's interface presents information clearly. In contrast, Bally Sports' interface has an excess of visual elements, which leads to a suboptimal viewing experience and a higher viewing workload. In the five visualized information elements, participants' preferences for viewing information were ranked as follows: (1) on-court shot clock, (2) lower third-player stats, (3) player position tracking, (4) shot probability, and (5) three-point shot distance. Most participants believed that the on-court shot clock provides excitement and a sense of engagement, while the lower third-player stats allow for a better understanding of each player. The player position tracking was regarded as an essential tool for the participants' understanding of team tactics, player positioning, and basketball movements. Most participants considered shot probability less important and most expressed a reluctance to see information about three-point shot distance, deeming it challenging to understand and lacking significance. This study provides insights into the perspectives and evaluations of NBA broadcast interfaces from individuals with low basketball experience and their preferences and comprehension levels regarding visualized information. The findings can guide future designs of basketball broadcast interfaces.

Wei Chen, Meng-Cong Zheng, Li-Jen Wang

Conference Proceedings

Mixed Reality Handheld Displays for Robot Control: A Comparative Study

Robotic systems for several applications from healthcare to space explorations are being developed to handle different levels of autonomy – from working independently to working in collaboration with or under control by human operators. To ensure optimal human-robot cooperation, appropriate UIs are needed. In this context, applying Mixed Reality handheld displays (MR-HHDs), an ubiquitous tool for virtually augmenting reality, seems promising. As existing MR-HHD-UIs for robot control employ fatigue-prone and view-obstructing touch input, we propose controlling a robot arm via an enhanced MR-HHD-UI based on peripheral touch and device movement. Our detailed, comparative user study on usability and cognitive load demonstrates that the proposed MR-HHD-UI is a powerful tool for complementing the strengths of robots and humans. In our experiment, the MR-HHD-UI outperformed a Gamepad- and Desktop-UI in terms of temporal and cognitive demands and was rated as the preferred UI.

Vera Marie Memmesheimer, Ian Thomas Chuang, Bahram Ravani, Achim Ebert

Conference Proceedings

Preliminary exploration of tripartite social pain games: Expressions of feelings among three players and game mechanism design

This study aimed to explore the social pain issues that school-age children may encounter when facing social challenges such as misunderstandings, exclusion, and bullying. To simulate social pain scenarios, we designed a three-player game mechanism to assist school-age children in building coping abilities through experiential learning and understanding. In the game design phase, we created game mechanisms involving misunderstandings, exclusion, and bullying, using a three-player ball-passing game to create corresponding situational atmospheres that simulated the experience of social pain. To avoid potential negative impacts on children through direct experimentation, we initially used adults as the experimental subjects. They participated in a three-player social pain game experience to gain preliminary insights into reactions in social pain situations. In the experiment, one participant and two informed players familiar with the game rules engaged in a ball-passing game, completing specific tasks to elicit social pain responses. Assessment tools included a social pain perception questionnaire and stress detection through Garmin wristbands to understand the participants’ physiological responses. Through recording and evaluation, we preliminarily explored the participants’ reactions, experiences, and empathy expressions when facing social pain. The results indicated that these three social pain mechanisms successfully elicited social pain experiences in the three-player game, while also stimulating participants’ empathy. This study provided preliminary research references for future applications of social pain games targeting children.

Yu-pin Weng, I-jui Lee

Conference Proceedings

Exploration in the Design and Development of Emotive Teaching Aids

Children with autism spectrum disorder often struggle with emotional communication due to innate emotional disorders. They find it difficult to express their own emotions and also struggle to understand the emotional states of others. These innate emotional barriers force them to rote memorize or mechanically learn specific emotional expressions, hindering effective emotional communication and expression.However, the abstract nature of emotional communication issues makes it challenging for autistic children to effectively express their feelings, leading to negative behaviors such as violence or self-harm due to emotional repression.To address this, we aimed to develop emotive toys, incorporating metacognitive game mechanics, to serve as tools for emotional communication and training for children with autism. For this purpose, we recruited several designers to collaboratively develop these emotive toys through workshops. Empathy mapping was used in these workshops to help the designers understand the emotional communication challenges faced by children with autism and inspire the design of emotive toys.After discussions, the designers proposed three different concepts for emotive toys, from which one was selected for further experimental validation. This selected toy was provided to eight pairs of players for an emotional communication game. The players used the toy's operations and sensory stimuli to communicate and express emotions.The results showed that through this game mechanism, all eight pairs of participants were indeed able to effectively communicate and express emotions, demonstrating that the toy effectively conveyed emotional information.

Chia-hao Lin, I-jui Lee

Conference Proceedings