Appropriate Reliance in Wearable AI for Parkinson’s Differential Assessment: Calibration, Selective Referral, and Subgroup Reliability
Abstract
Smartwatch-based machine learning for Parkinson's disease (PD) is often evaluated as forced-choice classification against healthy controls, although clinically similar disorders create the harder decision boundary. This retrospective study evaluates a human-centered selective-prediction framework on the Parkinson's Disease Smartwatch (PADS) dataset. The primary task included 276 participants with PD and 114 with differential diagnoses (DD); 79 healthy controls formed a secondary benchmark. From bilateral accelerometer and gyroscope recordings across 11 motor tasks, 3,366 participant-level time-domain, spectral, magnitude, and wrist-asymmetry features were derived, with demographic variables excluded from model inputs. Regularized logistic regression, Extra Trees, and radial-basis-function support-vector models were selected by nested five-fold outer and three-fold inner cross-validation and sigmoid calibrated. We evaluated discrimination, calibration, selective referral, subgroup uncertainty burden, class-conditional split conformal prediction, sensor/task ablation, and leave-one-DD-subtype-out shift. For PD versus DD, calibrated balanced accuracy was 0.674 and AUROC 0.714; Brier score and expected calibration error improved to 0.180 and 0.047. Referring the 20% least-confident cases reduced retained-case error from 30.5% to 25.0%. At 90% nominal conformal coverage, empirical coverage was 94.6%, but 76.7% of outputs were ambiguous. Under unseen DD subtypes, confidence referral did not consistently remove false-PD predictions, whereas conformal ambiguity increased. The results support evaluating wearable AI as a referral-and-review system rather than a forced-choice classifier.
Keywords: Parkinson's Disease, Wearable Technology, Human-Centered Design, Selective Prediction, Conformal Prediction, Probability Calibration, Human-AI Collaboration
DOI: 10.54941/ahfe1008275
Cite this paper
More from this volume
- AI-Enhanced Live-Streamed English Language Learning: Designing and Evaluating an Intelligent Platform for Chinese Learners
- "They Will Move": Confirmation Bias, Expectation Frames, and Premature Monitoring Drop Off in COLREG Encounters
- Automation transparency communication does not improve supervisory control performance, but shapes operator trust
- The influence of neuroticism on startle and surprise response: a flight simulator study
- EEG-Based Classification of Reward, Penalty, and Movement Events Across Four Video Games of Varying Complexity
- The Effects of Task Similarity and Time Pressure on Task Performance Following Interruptions in Complex Human-Computer Interaction Systems
- Trusting the Chain, Not the Black Box: A Provenance-Based Architecture for Human-Machine Teaming in Military Autonomy
- Design of a Simplified Humanoid Robot Face: Cultural Differences and Emotional Synchronization
- From Visual Observation to Auditory Interaction: A Human-Centered Tangible Music System Based on Physical Objects
- The Impact of Scarcity-Based Dark Patterns on User Performance, Perceived Darkness, and Workload in Subscription Services
- A Behavioral Observation Tool for Diagnosing Human–AI Interaction Profiles in Engineering Design Education
- The Mediated Moon: Semantic Mediation Between Gesture and Generative Image


AHFE Open Access