User Centered Design and Evaluation of an Artificial Intelligence based Process Recommender System in Textile Engineering

Abstract

Despite digitization and automation in everyday life and at the workplace, traditional craftsmanship continues to be primarily analogue and manual. This specifically applies to decision-making processes that are predominantly influenced by experience and intuition. As a result, established best-practice solutions are commonly used and promising alternatives are overlooked. AI-based decision support tools are a viable option to automate and objectify the decision-making process. Additionally, these tools can help stimulate decision makers to break with common best-practice solutions and consider novel, promising alternatives. However, using AI may lead to lower social acceptance among users, due to scepticism about effectiveness, workers’ fear of being eventually substituted, and missing comprehensibility of the suggestions due to the black box-models of many AI systems. Currently, there is a lack of grounded guideline for designing and implementing user-oriented AI-based decision support systems in traditional craftsmanship.This contribution investigates how a user-centred design of AI-based decision support influences user acceptance and usage intention. For this purpose, two AI-based process recommender systems for planning textile reinforced composite processes are designed with varying focus (user-centred and purely functional). Both applications are then benchmarked in a mixed-method user study with qualitative (think aloud) and quantitative (survey) parts and 17 domain experts. We used an Excel-based decision support system as a reference, since it realistically represents the currently prevailing planning support in manufacturing companies. In the user study we evaluate the planning efficiency, objectivity, and user orientation by measuring the duration of the planning process, the result quality, consistency, and reproducibility of the designed process chains and the usability of the system. Additionally, trust in automation, the performance expectation, as well as the intention to use are measured based on the acceptance models of Körber and Venkatesh et al. and are supplemented by additional items (e.g., comprehensibility). The results of the study suggest that an AI-based support system can increase the speed and objectivity of the decision-making process. However, it is also important to design the system in a user-centric way to ensure usability, trust, and acceptance. Further, we found that it is reasonable to leave the final decision-making authority with the decision maker, since our participants tended to less frequently question a completely automated result. Based on the results of our study, we derive actionable guidelines for the design of AI-based support systems in manufacturing.

Keywords: Artificial Intelligence, Process Recommender System, User-Centered Design, Carbon-Fibre Reinforced Polymers, Trust in Automation, Acceptance

DOI: 10.54941/ahfe1001709