Appropriate Reliance in Wearable AI for Parkinson’s Differential Assessment: Calibration, Selective Referral, and Subgroup Reliability

Open Access
Article
Conference Proceedings
Authors: Mohammad Mahdi Nazari, Negar Heidari Matin, Yaser Michael Banad
Abstract

Smartwatch-based machine learning for Parkinson's disease (PD) is often evaluated as forced-choice classification against healthy controls, although clinically similar disorders create the harder decision boundary. This retrospective study evaluates a human-centered selective-prediction framework on the Parkinson's Disease Smartwatch (PADS) dataset. The primary task included 276 participants with PD and 114 with differential diagnoses (DD); 79 healthy controls formed a secondary benchmark. From bilateral accelerometer and gyroscope recordings across 11 motor tasks, 3,366 participant-level time-domain, spectral, magnitude, and wrist-asymmetry features were derived, with demographic variables excluded from model inputs. Regularized logistic regression, Extra Trees, and radial-basis-function support-vector models were selected by nested five-fold outer and three-fold inner cross-validation and sigmoid calibrated. We evaluated discrimination, calibration, selective referral, subgroup uncertainty burden, class-conditional split conformal prediction, sensor/task ablation, and leave-one-DD-subtype-out shift. For PD versus DD, calibrated balanced accuracy was 0.674 and AUROC 0.714; Brier score and expected calibration error improved to 0.180 and 0.047. Referring the 20% least-confident cases reduced retained-case error from 30.5% to 25.0%. At 90% nominal conformal coverage, empirical coverage was 94.6%, but 76.7% of outputs were ambiguous. Under unseen DD subtypes, confidence referral did not consistently remove false-PD predictions, whereas conformal ambiguity increased. The results support evaluating wearable AI as a referral-and-review system rather than a forced-choice classifier.

Keywords: Parkinson's Disease, Wearable Technology, Human-Centered Design, Selective Prediction, Conformal Prediction, Probability Calibration, Human-AI Collaboration

DOI: 10.54941/ahfe1008275

Cite this paper
Downloads
0
Visits
1
Download PDF

More from this volume

← Effects of Virtual Agent Style and Decision Control on Sense of Agency and Trust in AI-Enabled TelecareVisual Quality, Perceived Safety, and Spatial Vitality in Shichahai, Beijing: A Day–Night Assessment of a Cultural Waterfront →
View all articles in Human Factors in Design, Engineering, and Computing →