Web-based Human-centred Explainability of NLP Tasks with Rationale Mapping Theory

Abstract

Recently, human-generated data has been used to explain machine learning and NLP models. Such methods usually focus only on labelling results with relevant human-generated tags, explicitly identifying objects, actions, or other elements in the output. Therefore, potential explanations only refer to the data elements and the model parts that produce them. The cognitive process applied by the human to perform the task is completely neglected. We claim the latter is essential to provide complete and human-understandable explanations of results, models, and processes. Some existing approaches studied in linguistics, such as rationale mappings, aim to achieve this objective by formalizing tree-based data structures to collect human rationale applied to NLP tasks. This work presents a web-based, human-centred approach to collect rationale mappings for various NLP tasks. Our contribution includes the formalization of the Rationale Mapping theory, the design of the human-computer interaction paradigm implementing the theory, the specification of the data collection process, its implementation as a crowdsourcing web application, and its validation with experimental studies showing its reliability and effectiveness.

Keywords: Human-centred Approach, Crowdsourcing, Web Application, NLP, Explainability

DOI: 10.54941/ahfe1006045