When Helpful AI Feels Too Trustworthy: Engagement and Reliance in Design Ideation

Open Access
Article
Conference Proceedings
Authors: Ting Liao, Jiacheng Sun
Abstract

Artificial intelligence (AI) assistants are increasingly used in engineering ideation, but their effectiveness depends on both the AI model’s capabilities and how users interpret and act on AI feedback. This paper synthesizes human-factors findings from the development and evaluation of the AI Design Assistant (AIDA), a modular testbed for iterative engineering ideation, focusing on trust calibration, feedback presentation, user agency, and design exploration. Across studies, adaptive and socially responsive feedback generally increased trust and perceived usefulness, yet trust did not reliably correspond to measured design improvement. More feedback yielded mixed results: longer or highly structured responses were not consistently associated with stronger outcomes, while concise feedback and longer user revisions were associated with larger score gains in turn-level analyses. Users also applied AI guidance selectively; only 38.6% of actionable suggestions were implemented, although most revisions used at least one AI-generated suggestion. These findings suggest that the next generation of AI design assistants needs to support calibrated trust, reduce unnecessary information-processing demands, preserve explicit user control over suggestion use, and distinguish user perceptions from measured design outcomes. The results provide practical design guidance for human-centered AI systems that augment, rather than replace, human judgment during creative engineering work.

Keywords: Human-AI Collaboration, Decision Support, Trustworthy AI, Adaptive System

DOI: 10.54941/ahfe1008303

Cite this paper
Downloads
0
Visits
1
Download PDF

More from this volume

← Human-Centered Design of AI Decision Support: Algorithmic Vigilance and the Role of AI LiteracyUnderstanding the needs and constraints of speech-language pathologists when using technological devices in rehabilitating people with aphasia. →
View all articles in Human Factors in Design, Engineering, and Computing →