Data-grounded empathy: Simulating "the untouchable" to mitigate representational bias in user research

Abstract

Traditional user research in high-pressure service contexts is often constrained by logistical challenges and the pervasive influence of social desirability bias (SDB), which compromises data authenticity. This paper presents a reproducible workflow for developing and validating a high-fidelity AI interview agent designed to address these challenges. Built on a Retrieval-Augmented Generation (RAG) architecture, the agent is grounded in a multi-source knowledge base compiled from in-depth interviews, online community discussions, and multimedia content from service-industry workers. We describe the end-to-end process, from data collection and preprocessing to agent implementation and prompt engineering. The agent’s performance was assessed through a two-part validation study: an expert heuristic evaluation and a comparative Turing test involving 22 participants. The results show that the agent produced interview data that were perceptually indistinguishable from human-generated responses and were rated by participants as significantly more consistent and coherent. This work contributes a transparent and adaptable methodology for Human–Computer Interaction (HCI) and design research, offering a scalable tool to gather authentic user insights while mitigating known biases. The findings point to a new paradigm for human–AI collaboration in user research, particularly for accessing hard-to-reach populations.

Keywords: AI Agents, User Research, Conversational AI, Social Desirability Bias, Retrieval-Augmented Generation (RAG), Reproducibility, Human-Computer Interaction

DOI: 10.54941/ahfe1006859