Avoiding black box problems by assigning an active role to humans in the control of autonomous AI: A methodological approach

Abstract

From a human factors’ perspective, the combination of humans and autonomous AI presents a number of challenges. A major problem is the black box nature of AI. Humans are faced with the impossible task of evaluating AI-generated suggestions that they can no longer understand and taking responsibility for them. An effect even appearing when AI provides explanations. Further challenges include difficulties in developing adequate situation awareness, de-skilling, de-motivation, or automation complacency. In our research, we assume that these negative effects on humans are exacerbated by the black box nature of autonomous AI in conjunction with the passive role assigned to humans in terms of supervisory control. To address these two problems while still leveraging the benefits of autonomous AI, we turn to the concept of interpretable primitives. A primitive is an autonomous AI agent with reduced scope, so that its purpose and functioning are easy for humans to understand. To avoid the black box problem, many primitives that are understandable to humans are used instead of a comprehensive but incomprehensible AI. The human’s role is to orchestrate the primitives by defining strategies, setting priorities, or directing their deployment. In this way, humans are assigned an active role that includes task characteristics that are considered prerequisites for human engagement and up-skilling. The paper presents operationalized criteria and a method for identifying primitives.

Keywords: Human Systems Integration, Human-AI Collaboration, Automation Transparency, Autonomous AI, Primitives

DOI: 10.54941/ahfe1007985