A Multidimensional Framework for Assessing Human-AI Collaboration in Military Operations: The CogARMY Teamwork Index (CTI)
Abstract
The increasing integration of autonomous systems and artificial intelligence (AI) into military operations is transforming teamwork, requiring new approaches to assess performance in hybrid human–AI teams operating in dynamic, high-stakes environments. Existing teamwork assessment frameworks offer robust theoretical foundations for evaluating leadership, communication, coordination, and adaptability, but were developed primarily for human-only teams and are not fully suited to Human-Autonomy Teaming (HAT), defined by O'Neill et al. (2022) as interdependent collaboration between one or more humans and one or more autonomous agents, each fulfilling distinct roles while pursuing shared mission objectives. This paper proposes a CogArmy Team Index (CTI), a multidimensional framework for assessing teamwork effectiveness across human-only and hybrid human–AI military teams. Developed through the Cognitive Training and Teamwork Assessment of Army Personnel (CogARMY) initiative and NATO Science and Technology Organization Human Factors and Medicine research task group HFM-374, the CTI extends established teamwork theory with autonomy-specific dimensions, including trust calibration, explainability, human–AI coordination, adaptive task allocation, and human–AI synchronisation. The framework adopts a multilevel Input–Process–Output architecture linking contextual inputs to teamwork processes, coordinating mechanisms, autonomy management, regulation, and mission outcomes. The CTI leverages a multimodal, data-driven methodology integrating behavioural, physiological, and AI-derived metrics, overcoming limitations of self-report measures while enabling assessment of both human and autonomous team members. Composite scores combine theory-driven and empirical weighting to support adaptability across operational contexts. The proposed framework offers a scalable, scientifically grounded approach for evaluating and optimising teamwork in next-generation military environments, supporting operational readiness and future human–AI collaboration.
Keywords: Human-Autonomy Teaming, Teamwork Assessment, Military Human Factors, Trust Calibration, Multimodal Assessment
DOI: 10.54941/ahfe1008350
Cite this paper
More from this volume
- AI-Enhanced Live-Streamed English Language Learning: Designing and Evaluating an Intelligent Platform for Chinese Learners
- "They Will Move": Confirmation Bias, Expectation Frames, and Premature Monitoring Drop Off in COLREG Encounters
- Automation transparency communication does not improve supervisory control performance, but shapes operator trust
- The influence of neuroticism on startle and surprise response: a flight simulator study
- EEG-Based Classification of Reward, Penalty, and Movement Events Across Four Video Games of Varying Complexity
- The Effects of Task Similarity and Time Pressure on Task Performance Following Interruptions in Complex Human-Computer Interaction Systems
- Trusting the Chain, Not the Black Box: A Provenance-Based Architecture for Human-Machine Teaming in Military Autonomy
- Design of a Simplified Humanoid Robot Face: Cultural Differences and Emotional Synchronization
- From Visual Observation to Auditory Interaction: A Human-Centered Tangible Music System Based on Physical Objects
- The Impact of Scarcity-Based Dark Patterns on User Performance, Perceived Darkness, and Workload in Subscription Services
- A Behavioral Observation Tool for Diagnosing Human–AI Interaction Profiles in Engineering Design Education
- The Mediated Moon: Semantic Mediation Between Gesture and Generative Image


AHFE Open Access