SIGNPAL: A Human–AI Interaction Framework for Real-Time Sign Language Translation

Abstract

Advances in artificial intelligence and human–computer interaction have enabled significant progress in assistive communication technologies for deaf and hard-of-hearing individuals who rely on American Sign Language (ASL). However, a persistent communication gap remains between ASL users and non-signers in everyday interactions. This study preliminarily evaluates the usability and performance of SIGNPAL, a high-fidelity AI-driven ASL interpretation system designed to provide real-time gesture-to-text translation, text-to-speech output, and customizable accessibility features to support two-way communication. Three participants with basic, moderate, and advanced ASL familiarity completed four usability tasks: real-time sign-to-text translation, accessibility customization, gesture recording and playback, and text-to-speech reply to generation. Objective measures included task completion time, translation errors, and recognition accuracy, while subjective usability was assessed using a 10-item Likert questionnaire and the System Usability Scale (SUS). The results show that SIGNPAL achieved an overall translation accuracy of 83.33%, exceeding the predefined performance threshold, and a 100% task completion rate. Response times remained below two seconds, supporting real-time interaction. Likert-scale ratings indicated high user satisfaction (overall mean = 4.5/5), and the mean SUS score of 90.83 classified the system as having excellent usability. Qualitative feedback highlighted the clarity of the interface and the usefulness of the recording-and-playback feature, with minor recommendations for improving text visibility. These findings demonstrate that integrating human factors principles with AI-driven gesture recognition can produce effective and user-centered assistive communication systems, supporting inclusive real-time interaction between ASL users and non-signers.

Keywords: Sign Language Recognition, Human–AI Interaction, Usability Evaluation, Accessibility Design, Assistive Communication Systems

DOI: 10.54941/ahfe1007976