Investigating the Impact of Confidence Scores in AI-based Decision Support Systems on Decision Quality and Reliance in Work Contexts

Abstract

Due to the rising demands in work contexts, for example shortage of skilled workers and the increasing complexity of work environments and decision-making context, the use of artificial intelligence (AI) -based decision support systems, is increasing. These interactions differ in several ways from interactions with AI in leisure contexts:For example, in work contexts, individuals make decisions for which they are held accountable. In contrast, when using AI in leisure contexts, the consequences of an incorrect decision may not reflect them in a professional way. Also, because individuals are experts in their respective fields, they tend to be more critical of using AI-based assistance systems in work context. Furthermore, using AI in leisure contexts is voluntary, attracting individuals who are interested in and familiar with such systems. In contrast, the use of AI systems in work contexts is often mandatory, requiring even those who are less interested and of differently competent in using AI systems to engage with them. This raises the question of how to best support individuals in work contexts where they have a high need for assistance but may be the most critical of new systems and have diverse competency levels in dealing with AI.One opportunity to support users of AI-based systems is to assist in the interpretation and critical evaluation of the output of the systems. A promising approach in this regard is the communication of uncertainty through confidence scores. These scores aid end users in interpreting the output by conveying the level of certainty of the AI recommendation. Ultimately, this approach aims to achieve a realistic calibration of trust in the system.Many studies in this field do not sufficiently consider the specific characteristics of work contexts and focus on artificially constructed decision-making situations that are not transferable to professional settings. Often, these studies involve binary decision scenarios, such as distinguishing between sick and healthy in medicine. However, in practice individuals are confronted with not only binary but also multidimensional decision situationsThe present paper addresses this gap by designing an empirical study that investigates how uncertainty communication, in the form of confidence scores, affects decision quality, reliance in the AI system, and perceived task load in the context of a multidimensional decision-making situation at work. The paper presents the current state of literature and derives hypotheses related to these questions, discusses the requirements for the experimental design, and finally addresses these through deriving a specific study design.This research aims to provide valuable insight for both academics and practitioners engaged in the interaction of human and AI-based decision support systems, their collaboration, and implementation in the practice.

Keywords: Decision Support, Uncertainty communication, AI support in work contexts

DOI: 10.54941/ahfe1005825