Detecting Silent Automation Reliability Degradation - A Signal Detection Theory Analysis
Abstract
Silent reliability degradation provides no explicit alert, requiring operators to distinguish a genuine system-state change from ordinary stochastic errors. Using the previously reported first-report dataset (N = 60; 720 participant-scenario trajectories), this secondary signal detection theory (SDT) analysis formally decomposed terminal failure-report behavior into discrimination sensitivity (d-prime) and relative decision criterion (c). Reliability losses of 5% and 10% were treated as signal-present states, 0% loss as signal absent, and any first failure report within 100 formal monitoring trials as a positive response. Larger reliability loss was associated with higher d-prime. Higher initial reliability was associated with both higher d-prime and a more conservative c. Thus, the slower yet more accurate pattern previously observed under high initial reliability is consistent with the joint contribution of better terminal state discrimination and more conservative reporting. This analysis quantitatively complements prior time/accuracy and psychometric-threshold analyses of the same dataset.
Keywords: Automated Decision Aids, Silent Reliability Degradation, Signal Detection Theory, Sensitivity, Decision Criterion
DOI: 10.54941/ahfe1008474
Cite this paper
More from this volume
- AI-Enhanced Live-Streamed English Language Learning: Designing and Evaluating an Intelligent Platform for Chinese Learners
- "They Will Move": Confirmation Bias, Expectation Frames, and Premature Monitoring Drop Off in COLREG Encounters
- Automation transparency communication does not improve supervisory control performance, but shapes operator trust
- The influence of neuroticism on startle and surprise response: a flight simulator study
- EEG-Based Classification of Reward, Penalty, and Movement Events Across Four Video Games of Varying Complexity
- The Effects of Task Similarity and Time Pressure on Task Performance Following Interruptions in Complex Human-Computer Interaction Systems
- Trusting the Chain, Not the Black Box: A Provenance-Based Architecture for Human-Machine Teaming in Military Autonomy
- Design of a Simplified Humanoid Robot Face: Cultural Differences and Emotional Synchronization
- From Visual Observation to Auditory Interaction: A Human-Centered Tangible Music System Based on Physical Objects
- The Impact of Scarcity-Based Dark Patterns on User Performance, Perceived Darkness, and Workload in Subscription Services
- A Behavioral Observation Tool for Diagnosing Human–AI Interaction Profiles in Engineering Design Education
- The Mediated Moon: Semantic Mediation Between Gesture and Generative Image


AHFE Open Access