Trust and Calibration in AI-mediated Decision Support under Conditions of Risk

Abstract

AI-mediated decision support systems are increasingly deployed in domains characterized by risk, uncertainty, and time pressure. In such environments, appropriate reliance on AI recommendations requires not only initial trust formation but also dynamic recalibration when system performance fluctuates or conflicts with other information sources. Although determinants of perceived trust (e.g., explainability, authority cues, and ethical framing) have been widely studied, less attention has been given to how reliance behavior adjusts following observed system error. This paper presents a focused qualitative synthesis of empirical studies examining trust and reliance in AI-based decision support under conditions of risk or informational divergence. Across the included studies, trust was frequently operationalized as an attitudinal construct or predictor of adoption. In contrast, fewer investigations directly measured behavioral reliance following performance degradation or assessed calibration accuracy, defined as the alignment between perceived system capability and actual performance over time. Findings suggest that reductions in reported trust do not consistently translate into commensurate changes in reliance behavior. This divergence highlights the need to distinguish attitudinal trust from behavioral calibration when evaluating AI systems in safety-relevant contexts. We argue that calibration-aligned design (rather than trust maximization alone) should guide the development and assessment of high-stakes AI decision support.

Keywords: AI Agent-interaction, AI-generated Recommendations, Decision Support, Trust Calibration, High-risk Scenario

DOI: 10.54941/ahfe1007318