AHFE Open AccessNeuroergonomics and Cognitive Engineering
Editors: Hasan Ayaz
Topics: Neuroergonomics and Cognitive Engineering
Publication Date: 2022
ISBN: 978-1-958651-18-6
DOI: 10.54941/ahfe1001811
Articles
Application of ExpressDecision2 in User-Centered and Shared-with-Expert Decisions Under Risk and Uncertainty
ExpressDecision2 is a general-purpose web application designed to support the individual in making difficult decisions under uncertainty, which are emotionally driven and typically solved by using rational intuition. This web app is based on the self-regulation model of the thinking process developed within the framework of the systemic-structural activity theory. This paper demonstrates the application of two customized versions of ExpressDecision2: 1. ED2StatinChoice – for making a patient-centered and shared-with-clinician decision about taking statins for cholesterol reduction to prevent a heart attack or stroke. The two primary resources regarding taking statins for cholesterol reduction are The 2018 AHA/ACC Cholesterol Guideline and Mayo Clinic Statin Choice Decision-Aid tool. These and other guidelines and decision aids, as well as information derived from a health professional, provide the patient with essential information regarding the pros and cons of using statins, while also empowering the patient to make the ultimate decision regarding whether they should take statins. Overall, such a problem is both uncertain and difficult for the patient and so requires them to establish both short- and long-term goals, as well as relevant options for selection. ED2StatinChoice is designed specifically to help the patient make the best choice in such a difficult scenario. ED2StatinChoice complements existing decision-support tools, such as the Mayo Clinic Statin Choice Decision Aid. Its method of assistance involves clarifying the goal and various choices with subsequent aggregation of all pros and cons, thus helping make a motivated decision regarding which statin therapy is most preferable. 2. ED2InsuranceChoice – for making a client-centered and potentially shared-with-agent decision about buying an insurance policy in order to reduce financial uncertainty and make accidental loss more manageable. People buy health, car, life, home and other types of insurance to protect themselves from financial loss in the event of illness, car damage, house fire and other accidents, respectively. For example, they make decisions when choosing from among liability, comprehensive and collision insurance types. This decision-making process is guided by tangible statistical factors regarding people’s risks of accidental losses and by non-tangible factors, such as “peace of mind” due to being protected against financial loss in the event of an accident. It is important to note that peace of mind is an essential and decisive factor when selecting an insurance policy. It reflects subjective justification of rate adequacy for the premium: the premium should be reasonable and coverage must be sufficient. Unfortunately, such non-tangible factors as peace of mind from being protected against financial loss in an accident, as well as anxiety from losing money while paying an insurance premium for coverage that doesn’t get used unless you have an accident are not sufficiently reflected in existing models of insurance choice.
Alexander M. Yemelyanov, Rahul Sukumaran, Alina Yemelyanov
Open Access
Article
Conference Proceedings
The Psyche of “Self” in Students’ Systemic and Structural Interaction with Online Teaching-Learning Platforms
This research explores and provides insights on students’ psyche of “self” as characterized by their self-evaluations of their systemic and structural interactions with the online teaching-learning platforms that serves as their virtual classroom since the advent of the COVID-19 pandemic. Having such insights is of significance towards understanding the functional interactivenesses of virtual platforms that serve as online digitized classrooms used for teaching and learning in tertiary academic institutions, and which usage has gained global acceptance since the advent of COVID-19 pandemic. This has resulted in a systemic and structural shift towards virtual education among tertiary institutions, with the requisite restructuring of face-to-face teaching-learning mechanisms into new online delivery systems. Considering the fact that such new online systems, which are digitized educational instruction media, are mostly designed by third parties who are not the direct users, there is a need to provide users, namely teachers and students, the space to share the psyche of their “selves” which could be used to develop a sense of their self-evaluative perspectives of the effectiveness of the current approaches to such instructional design, in terms of the quality and effectiveness of their interactivenesses. As it is posited in the extant literature, self-evaluation is crucial to mental and social well-being due to the influences it exerts on a person’s aspirations, personal goals and interaction with others. Thus, self-evaluation, provides personal insights on the beliefs and evaluations individuals hold about themselves, helping to determine who they are, their capabilities, and future developments. These insights are manifestation of the psyche of “self”, deemed as powerful inner influences that provide individuals internal guiding mechanisms that help steer and nurture them through the dynamics of life, governing their behavior in the process, and defining the character of their individual self-concept and self-esteem, and by extension their self-image and self-perception. With self-concept manifesting individual beliefs and knowledge about personal attributes and qualities, it represents a cognitive schema that organizes abstract and concrete views about the “self”, and controls the processing of self-relevant information. The extraction of such an information, especially from students perspectives, is deemed important to enable the systemic and structural design of quality virtual platforms used as online classrooms and quality interactive teaching-learning activity. In this study, therefore, data was collected from six hundred and eighty-seven graduate students in a Ghanaian university, using a structured questionnaire that enabled the students to process self-relevant information associated with the quality of their systemic and structural interaction with online teaching-learning platform used in teaching them throughout the semester. Guided by Bedny and Karwowski's well-established knowledge that activities of individuals are realized by goal-directed actions, informed either by mental or motor conscious processes, as objects of the cognitive psychology of skills and performances, systemic analysis is conducted and the learning from the students self-evaluation determined. The findings will provide additional insight in the design of virtual platforms serving as online classrooms for teaching-learning.
Mohammed Aminu Sanda
Open Access
Article
Conference Proceedings
Surgeon’s performance: analogy with aircraft pilot’s challenges
There is an obvious analogy between the challenges that an aircraft pilot has to cope with when fulfilling his/her long-term mission or when encountering a short-term abnormal situation, and the challenges that a surgeon faces during his/her "mission", a surgical operation, which is always a highly challenging and sometimes unpredictable effort. Kao and Thomas seem to be the first ones who paid attention to this analogy that could be viewed today as a part of what is identified as human-system-integration/interaction (HSI) field. This extraordinarily broad field includes the role of human performance in various psychological and ergonomics tasks in general and critical aspects of human interactions with an intelligent system in particular: predictive modeling (PM), both computer simulations based and analytical; vehicular engineering, such as aerospace, automotive, railway, and maritime; medical electronics; and, of course, all kind of the human-in-the loop (HITL) and human factor (HF) related activities, attributes and challenges. The outcome of these activities is highly dependent on the mental (cognitive) workload (MWL) and, mostly long-term, human capacity factor (HCF). Probabilistic predictive modeling (PPM) enables evaluating, improving, assuring and ultimately, if possible and appropriate, even specifying the acceptable (adequate and never-zero) probability of failure of a HITL mission or a situation, when the reliability of the equipment/instrumentation, the performance of the human-in-control (the pilot or the surgeon) and the response of the object-of-control (the air or spacecraft, not to mention the patient undergoing surgery), and the interfaces of these and other uncertainties contribute jointly to the importance of the outcome of the undertaking. Systemic-Structural Activity Theory (SSAT) is applicable to the analysis and improvement of the efficiency and reliability of the highly challenging types of human activity. The objective of this paper is to indicate the need for quantifying the role of the HF in making a surgical operation less risky and to indicate that the consideration of the analogy of this effort with the aircraft pilot challenges, which have been addressed and modeled in a number of recent publications, could be helpful. The paper uses a simplified double-exponential-probability-distribution function (DEPDF) to make our point and to "bring down to earth" the more general model for the probability of the human-non-failure (HnF). By predicting this probability and making it adequate for a particular surgical application, one could put various "educated guesses" and "gut feelings" about the instrumentation and human reliability during the fulfillment of the surgical mission on a really "reliable" quantified foundation. Plenty of additional, both analytical and computer simulation-based modeling, as well as experimental and clinical and statistical work should be done to “reduce to practice” the general idea of the need for quantifying, in one way or another, the numerous challenges that a surgeon faces in his/hers never-routine activity, in which the analogy with the pilot's performance might be helpful.
Ephraim Suhir, Inna Bedny
Open Access
Article
Conference Proceedings
Modelling And Simulation With Biofeedback For Operators Of Human-Machine Systems
Introduction. The simulators are designed for human-machine systems operator training process. It is more appropriate to estimate current operator efficiency and the operator professional readiness level for operator training process effectiveness. Operator functional state monitoring during training process is useful for detection of operator efficiency decreasing. The biofeedback based of operator functional state monitoring can increase operator training process effectiveness. The work is concern with different approaches to development of simulators for human-machine systems incorporated with biofeedback based of operator functional state monitoring.Hypothesis. The work targeted to experimental validation of hypothesis that biofeedback can significantly increase operator training process effectiveness.Aim. The aim of the work is to select features are used for operator efficiency estimation, and to formalize the concept of operator task complexity.Methods. The measurement procedure is viewed as computer simulator. Simulation consists of functional tests series step by step from low complexity level to high complexity level. There are reach stimuli represented by geometric figures. Each figure is characterized by color, shape, size and method of appearance on the screen. Simple stimulus leads to simple operator deeds, but complicate stimulus leads to complicate operator deeds. Each complicate stimulus can be represented as sequence of elementary stimuli. The set of permissible figures and permissible appearance methods are the description of simulation structure. Each functional test is to presentation of fixed complexity level stimuli. The basic features are used for operator efficiency estimation are: reaction speed and reaction exactness. These parameters objectively decrease as a result of stimulus complexity level increasing. Accordingly, there are more sophisticated features such as tiredness degree and operator functional limit.Discussion. The work is concern with measurement experiment is used for operator efficiency estimation. There are some features such as: reaction speed, reaction exactness, tiredness degree and operator functional limit. These parameters demonstrate a strong variability. Accordingly, we need investigate influence of figures characteristics to measurement experiment results individually for each operator. So, there is requires implementation additional psychology tests (for each operator) and comparison their results with computer simulation. Different methods of stimuli formation for the solving this problem are consider and investigate. The question is required by particular consideration is: can we represent the sequence of simple stimulus as complicate stimulus. Biofeedback introduction to simulation leads to possibility of selection of individual complexity level for each operator, and detection of individual functional limit for each operator. Furthermore, biofeedback guarantees robust operator efficiency estimation. Also, biofeedback can help operator to increase his concentration, and increase operator training process effectiveness.Conclusions. Human-machine systems operator efficiency estimation is need for increase the level his functional readiness to prevent critical cases in real world. The technique of operator efficiency estimation is proposed and proved in the paper. Stimuli are described and formalized. The preliminary measurement experiment results are represented. The future considerations linked with more complicated simulators, which can simulate some real world tasks. Also, there is an interest in clarification of biofeedback role in increasing of operator training process effectiveness.
Oleg Zhvalevsky, Sergey Roudnitsky
Open Access
Article
Conference Proceedings
Assertiveness in the System of Behavioral Strategies of the Modern Youth
This article is an empirical study of the psychological features of assertive strategies of modern youth. The research methods include the modified test-questionnaire "Studying the assertiveness level", K. Thomas' test of behavioral strategies, and the test to determine the integral empathy in adolescents and young people. The results of an empirical study of the dominant behavioral strategies of Ukrainian modern youth, including early and mature adolescence are presented. The hierarchy of behavioral strategies of boys and girls is empirically determined as assertive, conformal, passive, altruistic, aggressive. Among the assertive strategies the following hierarchy is established: assertiveness as a representation of one's own autonomy, assertiveness as a manifestation of confidence in typical situations, assertiveness as a finding of compromise and as a real assistance not to the detriment of oneself in empathogenic situations. It has been shown that assertive strategies such as cooperation, compromise, and real self-help in empathogenic situations represent the most symmetrical subject-subject interpersonal relationships. It was found that the indicators of behavioral assertive strategies have significant positive age dynamics during adolescence with a simultaneous increase in their egocentrism and asymmetry in interpersonal interaction. The existence of gender differentiation during interactions in terms of assertiveness is demonstrated. It is proved that masculine and feminine assertive behavioral strategies are formed during early and mature adolescence in boys and girls in situations of interpersonal interaction.
Larysa Zhuravlova, Inna Bedny, Vitaliya Luchkiv, Liubov Pomytkina, Iryna Grechukha, Natalia Muzhanova
Open Access
Article
Conference Proceedings
Rationale and Application of Express Algorithms for Mental Health Assessment in Professional Selection and Screening Examinations
The aim of this study was the physiological substantiation of express algo-rithms for assessing cognitive functions according to the data of clinical, psy-chological and neurophysiological studies in conditions of time deficit. The in-tegrative indicator SCL-90-R - "general index of severity" has a high statistical significance (p <0.05) in both healthy subjects and neuropsychiatric patients. The effectiveness of the "Mini-Mult" method in conditions of time deficit is de-termined by the results of the scales of hypochondria, depression, hysteria, par-anoia, psychasthenia, schizoid and hypomania (p <0.05). We also used a block of logical techniques. A similar survey for 15 - 25 min can be used as a stand-ard for assessing neurocognitive status in screening studies of large groups. EEG power indices and spectra in theta, delta and alpha frequency ranges are an effective reflection of mental status.
Sergey Lytaev
Open Access
Article
Conference Proceedings
Evaluation of Real-time Assessment of Human Operator Workload during a Simulated Crisis Situation, Using EEG and PPG
We report on initial evaluation findings regarding a human cognitive state assessment tool that was tested in various operational security operations centers (SOC).
Thomas De Groot, Johan De Heer, Rafal Hrynkiewicz, Manon Tolhuisen, Tije Oortwijn
Open Access
Article
Conference Proceedings
Physiological markers of vigilance variation in a supervisory task
The ability to maintain an appropriate level of vigilance over long periods of time underlies success on a range of tasks. Particularly, staying alert allows to detect infrequent signals and to allocate the right level of cognitive resources to respond to expected or unexpected events. A review of literature shows that some physiological markers can assess this variation of attention in a lot of lab studies. These findings are interesting for human factors in aeronautics as it appears as a way to characterize and quantify the observation during assessment with pilots in a cockpit simulator. The objective of this present study is to integrate a set of physiological metrics in a representative cockpit and to test their robustness in a more ecological environment with the associated constraints. This paper presents the first step of this project with a series of two lab experiments where we tested physiological markers of vigilance proposed in the literature.We first explored these markers in a psychomotor vigilance task (PVT) classically used in the study of vigilance. ECG (Heart Rate Variability or HRV), oculometrics (blink frequency, Percentage of Eye Closure or Perclos, oculomotor pattern) and EEG (alpha rhythm) were collected. The results show an increase in reaction time over time, which indicates a decrease in vigilance. They also confirm the relevance of HRV, Perclos and Alpha rhythm as a metrics of change in vigilance. In contrast, blink frequency did not appear to correlate with vigilance in our task. We then applied the relevant metrics to a second task that combined an alarm detection task and a supervisory task, the objective being to observe the robustness of the device for a task closer to the operational context. Subjective reports and changes in performance appear to reflect a decline in alertness over time. Interestingly, HRV and Perclos also seem sensitive to these changes in vigilance, but not Alpha rhythm (which could be related to the presence of noise in the signal as well as to the small number of participants). The results obtained demonstrate that the combination of ECG and eye-tracking indicators is a promising solution for the investigation of pilot vigilance in a cockpit simulator.
Nicolas Maille, Mick Salomone, Andrea Desantis, Kevin Le Goff, Jean-François Sciabica, Marie-Christine Bressolle, Bruno Berberian
Open Access
Article
Conference Proceedings
Classifying mental workload using EEG data: A machine learning approach
Mental workload is related to the difference between the available mental resource capacity of the operator and the mental resource required by the job. To decide the number of tasks assigned to operator and the difficulty levels of those tasks, it is important to know the operator's mental workload. An overload occurs if the amount of resources required by the task exceeds the available capacity of the person. Mental workload analysis helps to recognize the mental fatigue, evaluate the human performance of different level tasks and adjust cognitive sources for safe and efficient human-machine interactions. Excessive levels of mental workload can lead to errors or delays in information processing. Monitoring brain activity has been verified to be sensitive and consistent reflector of mental workload changes. Classification, regression, clustering, anomaly detection, dimensionality reduction, and reward maximization are common machine learning models. Classification of mental workload has critical importance in the domain of human factors and ergonomics. In recent years, with the need to analyze continuous and large-scale data obtained by physiological methods, the use of machine learning algorithms has become widespread in estimating and classifying mental workload. The objectives of the current study were two-fold: (1) to investigate the relationship among EEG features, task difficulty levels and subjective self-assessment (NASA-TLX) scores and (2) to develop machine learning algorithms for classifying mental workload using EEG features. N-back tasks have been commonly used in the literature. In this study, N-back memory tests were performed at four different difficulty levels. As the number of n increases, so does the difficulty of the task. Four participants performed the tests. Seventy EEG features (5 frequency band power for 14 channels) were selected as independent variables. One output variable reflecting the difficulty level of N-Back memory was classified. The machine learning algorithms used in our study were K-Nearest Neighbor (KNN), Support Vector Machine (SVM), Artificial Neural Network (ANN), Random Forest (RF), Gradient Boosting Machine (GBM), Light Gradient Boosting Machine (LightGBM) and Extreme Gradient Boosting (XGBoost) algorithms. As the task difficulty increased, theta activity in prefrontal and frontal regions increased. Especially frontal theta power, parietal and occipital gamma power were significantly correlated to perceived workload scores obtained via NASA-TLX. Prefrontal beta-high activity had a significant negative relationship with self-assessment workload ratings. Prefrontal and frontal theta, prefrontal beta-high, occipital, parietal and temporal gamma and occipital alpha activities were found to be the most effective parameters. The results obtained for the four classes of classification problem reached the accuracy of 68% with EEG features as input and the Random Forest algorithm. In addition, the results obtained for the two classes of classification problem reached the accuracy of 87% with EEG features as input and the GBM algorithm. The results from the analysis indicate that EEG signals play an important role in the classification of mental workload. Another remarkable result was high classification performance of GBM, LightGBM and XGBoost algorithms that have been developed in the recent past and therefore not frequently used in studies on this subject in the literature.
Şeniz Harputlu Aksu, Erman Çakıt
Open Access
Article
Conference Proceedings
An Innovative Measure of Cognitive Function in the Human-Autonomy Partnership
Understanding how the user will interact with the system is fundamental to ensuring success in achieving a given goal. Therefore adopting a human-centered design approach will assist in integrating the human as a key component of the system during the design process. With the increased use of autonomy across different domains, the role of the human will inevitably change; in that how the user interacts with the system is dependent on the level of delegated authority the system has been assigned. To understand these interactions and the impact this has on the user, it is important to assess how the human interacts with the system. However, as these systems become more complex we must ask whether the measures we currently use are sufficient in allowing us to better understand the underlying cognitive functions involved in human-autonomy interaction. Evaluating this partnership we can not only assess the effectiveness and efficiency of human-autonomy interaction, but also provide guidance for future designs. Novel techniques such as functional Near Infrared Spectroscopy (fNIRS) offer a direct measure of cortical blood flow changes related to brain activity. This paper discusses findings from an experiment that examined human-autonomy interaction in a simulated Autonomous Vehicle (AV) whilst exploring the neural correlates of trust and workload. Participants were asked to complete a series of primary driving scenarios with secondary distraction tasks using both manual and autonomous vehicles. fNIRS was used to assess driver cognition across both conditions. Participants were also confronted with different levels of system transparency to determine whether the level of information presented by the system effected driver trust. Findings suggest that when autonomy was presented then the cognitive activity in the right and left dorsolateral prefrontal cortex (dlPFC) and the left ventrolateral prefrontal cortex (vlPFC) was reduced, whilst secondary task performance improved. These regions are associated with effortful decision-making based on working memory (WM) and reasoning, suggesting that using autonomy helps to reduce cognitive effort by removing the user’s need to make these decisions. During the system transparency scenarios, areas of the right and left vlPFC and left dlPFC showed significantly increased activity when the system provided very little information. These regions have previously been associated with uncertainty of decision making and increased visual processing, suggesting that a lack of information provided by the system meant the driver attempted to process the decisions of the vehicle through monitoring the environment. These findings demonstrate how novel measures of cognitive function could inform the design of future systems and facilitate a more effective human-autonomy partnership.
Samson Palmer, Dale Richards, Graham Shelton-Rayner
Open Access
Article
Conference Proceedings
Thermal Imaging of the Face: Mental Workload Detection in Flight Simulator
Thermography-based physiological measurement is an active re-search topic. Using such contact-free approach can be particularly helpful for detecting pilots’ mental state in operational settings. In particular, thermal infrared imaging of the face is a powerful and non-invasive tool that enables rapid and automatic analysis of changes in regional facial blood flow. This blood flow changes index sympathetic activity and are measured by capturing thermal imprints of particular facial regions such as nose, forehead, or around the eyes. Although several studies suggest a relationship between cognitive workload and facial thermoregulation profile, evidence about this link has not been yet sufficiently investigated and infrared imaging has yet to prove its importance in the cognitive workload detection scenarios. In this work, we investigated how thermal measures can allow continuous assessment of cognitive workload variations of pilots undergoing simulated flight tasks, and compare these measures with the heart rate, a more established marker,. The approach has been evaluated with 20 participants in flight simulator, and cognitive workload was modulated by the difficulty of two landing scenarios. Participants also performed a resting task (also called cool off). Thermal imprints did not varied significantly with landing difficulty. However, we found that the nose tip and nose area were significantly colder during the flight scenario vs the rest period (signal slope). Further analysis is needed to confirm that the thermal measures could identify more fine grained mental workload variations in a flight simulator setting.
Almoctar Hassoumi, Vsevolod Peysakhovich, Arthur Le Coz, Christophe Hurter, Mickael Causse
Open Access
Article
Conference Proceedings
Neuromarketing as a tool for environmental conditioning and sustainable consumption
The impact of human factors on climate change is unequivocal. While consumers are increasingly becoming aware of their environmental footprint, this is not sufficient: contextual factors such as pricing, convenience, and packaging play a role in consumers’ decision-making. This has created a gap between consumers’ attitudes and behavior, which calls for intervention of behavioral sciences to change consumer behavior and consequently combat the climate crisis effectively. Consumer neuroscience methodology has been proposed as a potential tool to untangle the neural and psychological origins of consumers’ behavior since subjective reports may be biased by social desirability and therefore are not a reliable measure of pro-environmental behavior. Prior studies have shown that conditioning the consumer with information on the environmental impact of products can influence their buying behavior and brain activity. This paper provides an extended exploration of past works on consumer neuroscience, environmental behavior, and conditioning techniques. We aim to unite the current theories and common practices and uncover future research directions in an effort to develop a neuroscientifically supported conditioning intervention that could promote pro-environmental behavior in consumers.
Nikki Leeuwis, Maryam Alimardani, Tom Van Bommel
Open Access
Article
Conference Proceedings
FNIRS an emerging technology for design: advantages and disadvantages.
fNIRS is a functional neuroimaging technology that measures activations according to the oxygenation and deoxygenation of neural activities. A technique still little used within design, but that can contribute in neurodesign and affective, for example. Although emotions are universal, their way of perceiving and feeling is individual. The emotion design has some gaps, namely the lack of mastery of techniques and knowledge of human responses to emotions. In total, 44 articles were analyzed in a non-systematic way, with the aim to find the advantages and disadvantage of using fNIRS. As conclusion, it was possible to perceive that the fNIRS is a promising neuroimaging technique with 20 advantages points and 13 disadvantages points. The stimuli can be sensorial, cognitive and motor, handled in laboratory, in social environments or in real situations. fNIRS is already used in studies of emotions and can help to investigate the brain activations in the face of emotion processing and the affective design, enabling the possibility to design better experiences, products, services or environments focused on this affective parameter in front of neurocognition. fNIRS is an emerging and promising technique, which can help to understand some gaps in human beings as promote pleasure and well-being.
Bernardo Providência, Iara Margolis
Open Access
Article
Conference Proceedings
System immersion of a driving simulator affects the oscillatory brain activity
The technological properties of a system delivering simulation experience are a cru- cial dimension of immersion. To create a sense of presence and reproduce drivers behaviour as realistically as possible, we need reliable driving simulators that allow drivers to become highly immersed. This study investigates the impact of a system immersion of a driving simulator on the drivers’ brain activity while operating a con- ditionally automated vehicle. Nineteen participants drove approximately 40 minutes while their brain activity was recorded using electroencephalography (EEG). We found a significant effect of the system immersion in the occipital and parietal areas, prima- rily in the high-Beta bandwidth. No effect was found in the Theta, Alpha, and low-Beta bandwidths. These findings suggest that the system immersion might influence the drivers’ physiological arousal, consequently influencing their cognitive and emotional processes.
Nikol Figalová, Jürgen Pichen, Lewis L Chuang, Martin Baumann, Olga Pollatos
Open Access
Article
Conference Proceedings
Machine Learning-Based Gaming Behavior Prediction Platform
Brain disorders caused by Gaming Addiction drastically increased due to the rise of Internet users and Internet Gaming auditory. Driven by such a tendency, in 2018, World Health Organization (WHO) and the American Medical Association (AMA) addressed this problem as a “gaming disorder” and added it to official manuals. Scientific society equipped by statistical analysis methods such as t-test, ANOVA, and neuroimaging techniques, such as functional magnetic resonance imaging (fMRI), positron emission tomography (PET), and electroencephalography (EEG), has achieved significant success in brain mapping, examining dynamics and patterns in different conditions and stages. Nevertheless, more powerful, self-learning intelligent algorithms are suitable not only to evaluate the correlation between gaming addiction patterns but also to predict behavior and prognosis brain response depending on the addiction severity. The current paper aims to enrich the knowledge base of the correlation between gaming activity, decision-making, and brain activation, using Machine Learning (ML) algorithms and advanced neuroimaging techniques. The proposed gaming behavior patterns prediction platform was built inside the experiment environment composed of a Functional Near-Infrared Spectrometer (fNIRS) and the computer version of Iowa Gambling Task (IGT). Thirty healthy participants were hired to perform 100 cards selection while equipped with fNIRS. Thus, accelerated by IGT gaming decision-making process was synchronized with changes of oxy-hemoglobin (HbO) levels in the human brain, averaged, and investigated in the left and the right brain hemispheres as well as different psychosomatic conditions, conditionally divided by blocks with 20 card trials in each: absolute unknown and uncertainty in the first block, “pre-hunch” and “hunch” in the second and third blocks, and conceptuality and risky in the fourth and fifth blocks. The features space was constructed around the HbO signal, split by training and tested in two proportions 70/30 and 80/20, and drove patterns prediction ML-based platform consisted of five mechanics, such as Multiple Regression, Classification and Regression Trees (CART), Support Vector Machine (SVM), Artificial Neural Network (ANN), and Random Forest. The algorithm prediction power was validated by the 5- and 10-fold cross-validation method and compared by Root Mean Squared Error (RMSE) and coefficient of determination (R Squared) metrics. Indicators of “the best” fit model, lowest RMSE, and highest R Squared were determined for each block and both brain hemispheres and used to make a conclusion about prediction accuracy: SVM algorithm with RBF kernel, Random Forest, and ANN demonstrated the best accuracy in most cases. Lastly, “best fit” classifiers were applied to the testing dataset and finalized the experiment. Hence, the distribution of gaming score was predicted by five blocks and both brain hemispheres that reflect the decision-making process patterns during gaming. The investigation showed increasing ML algorithm prediction power from IGT block one to five, reflecting an increasing learning effect as a behavioral pattern. Furthermore, performed inside constructed platform simulation could benefit in diagnosing gaming disorders, their patterns, mechanisms, and abnormalities.
Denis Kornev, Roozbeh Sadeghian, Stanley Nwoji, Qinghua He, Amir Gandjbbakhche, Siamak Aram
Open Access
Article
Conference Proceedings
Natural Language Processing to Assess Communication Dynamics between Cooperating Dyads during Video Gameplay
Latent Dirichlet Allocation (LDA) and Sentiment Analysis have become prominent tools in natural language processing applications for both research and industry. While LDA is a generative probabilistic modeling methodology that is widely used in Topic Modeling to extract underlying themes and topics from a collection of words, Sentiment Analysis is defined as identifying the hedonic tone of a corpus of text. Here, supervised Sentiment Analysis is used to classify conversations between team gaming dyads in terms of valence. Additionally, LDA is utilized to label segments of cooperative conversation between dyads as topics. Fourteen participants were paired as dyads (7 teams) and instructed to complete thirty-two 150 second gaming scenarios (trials) in the first-person shooter (FPS) video game Overwatch. While completing the scenarios, participants were instructed to communicate with their respective teammate via a voice communication headset. The conversations from each scenario were transcribed from recorded voice channels before analysis was performed. Our approach examines the relationship between perceived task difficulty and both conversation sentiment scores and topic frequency in both novice experienced skill groups. Preliminary results indicate evidence that conversation topic, sentiment and perception dynamics are consistent with an encouragement and frustration sentiment paradigm.
Jan Watson, Adrian Curtin, Sukethram Sivakumar, Yigit Topoglu, Nicholas DeFlippis, Jintao Zhang, Rajneesh Suri, Hasan Ayaz
Open Access
Article
Conference Proceedings
Exploring the Link Between Emotional Arousal and Player Skill in Video Gaming Using Electrodermal Activity
Video games provide a high-octane competitive sports platform where players with diverse skills engage in tasks that require precise control of cognitive skills and emotional responses. Electrodermal activity (EDA) is a portable, non-invasive, and wearable physiological activity sensing modality that captures correlates of emotional responses. In this exploratory study, we analyzed the EDA skin conductance data that we collected from healthy adult participants over their left index and middle fingers, while they were playing a first-person shooter video game. Participants played solo against easy and hard AI opponents, where the objective was to either escort a truck to its destination or prevent the truck from being escorted to win the game. The in-game behavioral performance results showed that, as expected, novice players struggled with the game more than experienced players: novices had fewer kills, died more, and finished the scenarios slower. Furthermore, EDA skin conductance results showed that experienced players showed significantly higher electrodermal activity than novice players while playing the game, both in the context of the phasic and tonic activity.
Yigit Topoglu, Jan Watson, Adrian Curtin, Nicholas Defilippis, Jintao Zhang, Rajneesh Suri, Hasan Ayaz
Open Access
Article
Conference Proceedings
Social Robots and Performance Errors: Level of Empathy Distinguishes Changes in Trust
Much work has been done to engineer robots’ mechanical capabilities to best suit the general demands of their users and tasks. However, minimal research has addressed the impact of individual differences on perceptions of robot trustworthiness. These conclusions can provide guidance to optimize adaptive robotic systems in education, healthcare, and industry settings. This study examined the relationship between personality and human robot interaction in two contexts: (1) error-free and (2) errors. Assessment of individual differences were achieved via the Interpersonal Reactivity Index (IRI) (Davis, 1980) and robot trust assessed using the Multidimensional Measure of Trust (MDMT) (Ullman & Malle, 2018). This project provided a novel contribution in the field of human-robot interaction, highlighting the influence of technological failure on trust impressions of a social robot. Additionally, we sought to understand the degree to which empathy levels mediate these changes in trust.
Nina Rothstein, Ewart de Visser, Yigit Topoglu, Shawn Joshi, John Kounios, Frank Kruger, Hasan Ayaz
Open Access
Article
Conference Proceedings
Relationship between activation of prefrontal cortex and testosterone in N-back task
Purpose: Individual and gender differences are an important topic in the study of human cognition. In general, it has been shown that women are superior at verbal ability and associative memory, while men are better at tasks related to numbers, logic, and spatial recognition. Among them, it has been reported that cerebral blood flow in the prefrontal cortex increases during the N-back task, a working memory task. The relationship between sex hormones and cerebral blood flow remains unelucidated, although sex hormones and depressive mood are both factors that affect the function of the prefrontal cortex (cognitive function). Therefore, the present study aimed to clarify the relationship between the testosterone and cognitive function.Methods: The subjects included 6men and 16 women aged 20–35 years. Near infrared spectroscopy (NIRS), a brain function imaging device, was attached to the forehead of the subjects, and Oxyhemoglobin (Oxy-Hb) was measured during the N-back task. Oxy-Hb was measured during the N-back task using a NIRS system in the CHs 1-4 (right side), CHs 7-10 (center), and CHs 13-16 (left side) , and saliva was collected to measure the testosterone concentration. Subsequently, the correlation coefficient between the initial activation and testosterone concentration was calculated. This study was approved by the Institutional Review Board of the Kanagawa institute of technology (No. 20191011-01).Result: The items that were significantly correlated with testosterone were the initial activation CH13-16 in the 0-back task (r=48, p=.02). And the correct response rate (r=.42, p=.03), reaction time (p=.46, p=.03) and initial activation of CH13-16 (r=.68, p=.007) in the 1-back task. Conclusion: In adults, testosterone was associated with initial activation of the NIRS Oxy-Hb signal in the 0-back and 1-back tasks. Furthermore, in the 1-back task, testosterone was associated with correct response rate, the reaction time, and initial activation of the NIRS Oxy-Hb signal of CH13-16. This result suggests that testosterone may be related to left prefrontal cortex activation and responsiveness to presentation of the N-back task.
Aoki Makiko, Satoshi Suzuki, Ryouma Anzai
Open Access
Article
Conference Proceedings