Decision Support Systems for Route Planning: Impacts on Performance and Trust

Abstract: Decision Support Systems (DSS) and other performance augmentation tools are increasingly leveraged by the military to recommend courses of action and improve analyst performance on critical tasks. This is particularly important for path planning operations, where analysts must consider complex tradeoffs and contingencies based on available assets, distance, and target priority. To emulate this environment in a more general applied context, we developed a path planning task that emulated long-range delivery truck dispatch. Participants conducted a quality control check on four scenarios, each with a simulated DSS that recommended truck allocations, which ranged from perfect (100%) accuracy to subpar (40%) accuracy. Each participant also received one of four explanations for how the DSS algorithm would determine where trucks should be allocated: (1) no explanation, (2) a simple written explanation, (3) a flowchart, or (4) annotated tables. Participants demonstrated appropriately lower trust of DSS that had lower accuracy. Despite this appropriate trust calibration, their quality control performance was significantly lower when exposed to a DSS that had below perfect accuracy. Further, participants who self-reported higher levels of experience with path planning and AI algorithms demonstrated lower quality control accuracy. This demonstrates that while participants were able to successfully calibrate trust in their DSS, they nevertheless experienced performance decrements, possibly due to anchoring on the DSS’s incorrect result. Participants demonstrated the greatest understanding, strongest trust, and highest subjective preference for the simple written explanation of the DSS’s algorithm over more complex presentations, including a flowchart or annotated tables. The findings of this study provide the groundwork to understand the relationship between automation-reliance, trust, and performance, to determine when it is most appropriate to allow automation to make recommendations to analysts in operational environments and when DSS under-reliability may impact or increase human error.

Keywords: decision support systems, quality control, human performance

DOI: 10.54941/ahfe1001562

Cite this paper:

​