Designing with ‘Intelligence’? Exploring the Limits of AI-Based UX Tools

Abstract

Motivation: Artificial intelligence (AI) is increasingly shaping UX practice, with numerous AI-based tools promising to streamline the UX process and improve efficiency, saving time, money, and resources for users. However, how effectively do these tools support UX professionals in UX design and research? This project explores the current limitations and possibilities of using AI-based tools within an integrated, end-to-end human-centered design process.Research Questions: - How can AI-based tools support the phases of the human-centered design process?- How much and what kind of human intervention is necessary when using AI-based tools?Methodology: To address these questions, 106 available AI-based tools for UX were identified and reviewed. The tools were evaluated according to key criteria: AI functionality, support for UX methods relevant to the case study, GDPR compliance, availability of a free trial, and positive user reviews. Based on these criteria, 23 tools were selected for further assessment, focusing on their AI capabilities and the transparency of AI-generated results. From this assessment, two tools were chosen for use in the case study. The case study utilized a draft of a university website as a test case. It included an AI-based analysis of university websites, followed by AI-moderated interviews with students. Based on the research findings, an interactive prototype was developed and subsequently tested in an AI-moderated usability study with students.Initial findings: The case study revealed gaps between the different phases of the process, rather than an integrated end-to-end workflow. Bridging these gaps required human intervention at several points, particularly when incorporating research results into the prototype. Furthermore, both the methods applied, and the quality of the results produced by the AI-based tools were found to be inferior to the work of experienced UX professionals. This underscores the current limitations of AI-based tools in fully supporting the human-centered design process.

Keywords: artificial intelligence, human-centred design, cost-benefit, quality, UX professional

DOI: 10.54941/ahfe1006698