Comparison of human and AI-driven interview data analysis in industrial work context

Abstract: The rapid development of artificial intelligence (AI), especially in the form of large language models (LLMs), has opened new possibilities for the analysis of qualitative data. This has traditionally relied on the expertise, contextual understanding and interpretation of human analysts. There is growing interest in the field of human-centred design on whether analytical processes can be accelerated and systematized with the help of AI. However, this must be done without compromising the quality and reliability of the results. Empirical evidence on the differences between human and AI-based analyses in real industrial environments is still limited.The aim of the study was to assess the suitability of AI for the analysis of interview data. The interview material had been collected in a study examining the use of emerging technologies in industrial work. The objective was to determine whether AI analysis produces results comparable to those obtained through human analysis. Data to be analyzed was collected in a user study, in which five workers performed a lifting task using a crane. The analysis of the interviews was first done by two human factors researchers with over 20 years of experience in user studies. AI analysis was performed according to the same specifications as the human analysis. As for background information, AI was given a description of the two technologies being tested. ChatGPT 5 pro was used for the analysis. The AI was provided with transcripts of the interviews.The results show that both the human analysis and large language models (ChatGPT 5 Pro) analysis find largely the same key findings. However, some of the findings differed, mostly at the level of perspective and abstraction. For example, AI emphasized safety and security issues more than human analysis. The data also revealed a clear AI interpretation error related to linking a participant’s comment to the wrong technology. This highlights the importance of both expert validation and careful prompt design. In conclusion, the study suggests that human and AI analysis do not replace but complement each other. The most promising solutions are found in a hybrid model, where the speed and systematicity of AI are combined with the human analyst ethical judgment and knowledge on the interview data. The findings show that AI can be used to enhance and diversify analysis. The results of the study can also be utilized in an industrial context.

Keywords: Artificial intelligence, Interview analysis, Human-centred design, Industrial field trial, Generative AI, Compare

DOI: 10.54941/ahfe1007187

Cite this paper:

​