Human-centered Explainable-AI: An empirical study in Process industry

Abstract: This paper presents an empirical study on the explainability of transformer models analyzing time series data, a largely unexplored area in the field of AI explainability. The study is part of an ongoing EU-funded project which applies a human-centered approach to developing explainable AI solutions for the process industry. Here, we investigate the choice of explainer mechanisms and human factor needs when developing eXplainable Artificial Intelligence (XAI) for operators of two industrial contexts: copper mining and paper manufacturing. On-site evaluations were conducted in these settings involving control room operators to test the prototype developed in the project. The results indicate that the method of feature importance alone was not sufficient to provide explanations that are tailored to individuals and situations, as required by users. Overall, our empirical data supports “social” explanations for AI users and demonstrates the value of involving end users in the design process of effective XAI solutions. We also provide design implications which address human factor needs for such solutions in industrial settings.

Keywords: Explainable Ai, Industrial Application, Explainability, Empirical Study, Human Factors

DOI: 10.54941/ahfe1004638

Cite this paper:

​