A Regulatory and Human Factors Approach to Overreliance in Legal Information Chatbots
Abstract
Chatbots are rapidly transforming the way individuals from diverse social and economic backgrounds access legal information. This free access to legal knowledge can act as a corrective mechanism within society, reducing inequalities among social groups. Notwithstanding their benefits, the growing use of chatbots for legal matters raises significant concerns. One major issue is the accuracy of the information provided, since correctly applying a rule is a complex exercise, not a simple act of reading. Additional risks involve the processing of personal and sensitive data, which may be leaked or over-shared without users' deliberate consent or awareness. Concerns also arise regarding algorithmic bias, since AI systems may reproduce discriminatory patterns embedded in training or design data. Furthermore, chatbots may raise issues concerning the unauthorized practice of law. This article argues that legal question-answering chatbots, regardless of their pre-designed purpose, may endanger users' fundamental rights, and that classifying them as limited-risk systems undermines the goal of ethical, responsible and trustworthy AI. It further argues that the transparency framework under Article 50(1) of the AI Act is insufficient and proposes embedding ethics-by-design and ethics-by-default through a set of concrete interaction safeguards to actively support verification of chatbot-provided information before reliance: cognitive forcing functions, interaction friction, scaffolded verification, and calibrated uncertainty and limitation cues.
Keywords: Legal Information Chatbots, Automation Bias, Trust, AI Act, Transparency, Ethics-by-Design
DOI: 10.54941/ahfe1008419
Cite this paper
More from this volume
- AI-Enhanced Live-Streamed English Language Learning: Designing and Evaluating an Intelligent Platform for Chinese Learners
- "They Will Move": Confirmation Bias, Expectation Frames, and Premature Monitoring Drop Off in COLREG Encounters
- Automation transparency communication does not improve supervisory control performance, but shapes operator trust
- The influence of neuroticism on startle and surprise response: a flight simulator study
- EEG-Based Classification of Reward, Penalty, and Movement Events Across Four Video Games of Varying Complexity
- The Effects of Task Similarity and Time Pressure on Task Performance Following Interruptions in Complex Human-Computer Interaction Systems
- Trusting the Chain, Not the Black Box: A Provenance-Based Architecture for Human-Machine Teaming in Military Autonomy
- Design of a Simplified Humanoid Robot Face: Cultural Differences and Emotional Synchronization
- From Visual Observation to Auditory Interaction: A Human-Centered Tangible Music System Based on Physical Objects
- The Impact of Scarcity-Based Dark Patterns on User Performance, Perceived Darkness, and Workload in Subscription Services
- A Behavioral Observation Tool for Diagnosing Human–AI Interaction Profiles in Engineering Design Education
- The Mediated Moon: Semantic Mediation Between Gesture and Generative Image


AHFE Open Access