Adaptive Autonomy in the Air Force: Testbed for Human-AI Collaboration

Abstract

Autonomous and Agentic AI enabled systems are fundamentally reshaping aerial operations—spanning intelligence, surveillance, reconnaissance, targeting, and weapons employment. Such rapid transformation raises a pivotal question: how can we preserve meaningful human control while ensuring that autonomy augments mission effectiveness? Past work has shown that important factors to consider for human-AI collaboration include reliability, dependability, predictability, and transparency as well as relational qualities like responsiveness and its effects on operator engagement in complex and uncertain task environments. In this paper, we introduce a new testbed called STAR-SKY (Sharing Task with Autonomous Resources) enabling investigations on how to dynamically adjust autonomy levels or task allocations between humans and machines can be done across multi-domain (SKY, LAND, SEA etc..). Its goal is to support gathering empirical evidence on how to calibrate and optimize that process based on complex factors such as human workload, fatigue, trust, situational awareness, doctrine, task complexity, authority, interdependence, and differences in mental models. We present initial benchmark data from a pilot study and show how the experiment design and metrics guide integration of meaningful human control within Human-Autonomy Teaming (HAT) systems in the Air Force. We conclude with a discussion on requirements and recommendations for human-in-the-loop experiments with the STAR-SKY simulation.

Keywords: Adaptive control allocation, Human-AI collaboration, Human-autonomy teaming, Dynamic AI tasking

DOI: 10.54941/ahfe1007175