User Perceptions of Response Inconsistency and Trust in AI-Assisted Learning

Abstract

Generative AI chatbots are increasingly deployed in educational settings, yet their inherent response variability may undermine user trust and long-term adoption. This study examined how perceived response inconsistency and response structure influence user trust, perceived learning, and task performance in AI-assisted learning. Thirty-one graduate students completed GRE-style verbal reasoning tasks with AI assistance delivered via a Wizard-of-Oz chatbot that systematically varied response style (Standard, Lengthy, Unstructured, Ambiguous) across trials, creating response inconsistency by design. Post-task surveys assessed perceived inconsistency, trust impact, and learning experience, while task accuracy served as the objective performance measure. Spearman correlations were employed given the ordinal nature of the survey data. Results revealed that participants who noticed greater response inconsistency reported significantly higher trust damage (ρ = .617, p < .001) and more negative perceived learning impact (ρ = .499, p < .01). Critically, however, neither trust nor perceived learning impact correlated with actual task accuracy. This perception-performance disconnect indicates that users felt their learning was impaired when they noticed inconsistency and lost trust, yet their objective performance remained unaffected. Additionally, 86% of participants identified unstructured responses as detrimental to comprehension, while emoji use was rated highly effective for understanding (M = 4.61/5). These findings suggest that response inconsistency poses greater risks for user trust and long-term engagement than for immediate task performance. Users may abandon effective AI tools they do not trust, highlighting the importance of evaluating subjective user experience alongside objective performance metrics in educational AI systems. Design implications include prioritizing structural consistency in AI responses and incorporating visual cues to aid comprehension.

Keywords: Generative AI, Trust, Response Inconsistency, human-AI Interaction, Conversational Agents, User Perception, Educational Technology

DOI: 10.54941/ahfe1007507