Detecting Silent Automation Reliability Degradation - A Signal Detection Theory Analysis

Open Access
Article
Conference Proceedings
Authors: Cheng Tan, Liang Ma
Abstract

Silent reliability degradation provides no explicit alert, requiring operators to distinguish a genuine system-state change from ordinary stochastic errors. Using the previously reported first-report dataset (N = 60; 720 participant-scenario trajectories), this secondary signal detection theory (SDT) analysis formally decomposed terminal failure-report behavior into discrimination sensitivity (d-prime) and relative decision criterion (c). Reliability losses of 5% and 10% were treated as signal-present states, 0% loss as signal absent, and any first failure report within 100 formal monitoring trials as a positive response. Larger reliability loss was associated with higher d-prime. Higher initial reliability was associated with both higher d-prime and a more conservative c. Thus, the slower yet more accurate pattern previously observed under high initial reliability is consistent with the joint contribution of better terminal state discrimination and more conservative reporting. This analysis quantitatively complements prior time/accuracy and psychometric-threshold analyses of the same dataset.

Keywords: Automated Decision Aids, Silent Reliability Degradation, Signal Detection Theory, Sensitivity, Decision Criterion

DOI: 10.54941/ahfe1008474

Cite this paper
Downloads
0
Visits
1
Download PDF

More from this volume

← Human-Centred Transition Pathways for AI-Enabled Safety-Critical Systems: An Industrial Systems Engineering PerspectiveExperimental investigation of damping and mass parameters of an adjustable physical hand-arm model for reproducing human mechanical impedance →
View all articles in Human Factors in Design, Engineering, and Computing →