An Adaptive XR Platform for Multimodal Public Speaking Training and Performance Assessment

Abstract

Public speaking is a fundamental competence across academic, professional, and social contexts, yet an estimated 77% of the population experiences public speaking anxiety (PSA), manifesting through psychological symptoms – such as fear of judgment and avoidance behaviours – and physiological responses including increased heart rate and electrodermal activity. Traditional training approaches face limitations in scalability, reproducibility, and objective performance assessment, often relying on subjective rubrics and offering limited opportunities for repeated practice in realistic settings. Extended Reality (XR) technologies have emerged as a promising alternative, as immersive virtual environments can reliably elicit anxiety responses comparable to real audiences while providing safe, controllable, and repeatable practice scenarios. However, most existing XR platforms focus primarily on exposure, without systematically capturing or leveraging the multimodal signals that reflect a speaker's internal state and communicative behaviour. Incorporating behavioural data such as gaze patterns, gesture dynamics, and spatial movement, alongside psychophysiological signals and speech acoustics, is essential to move beyond subjective evaluation toward objective characterization of user states, enabling personalized feedback and real-time training adaptation. This paper presents the design, implementation, and preliminary assessment of an adaptive XR platform for public speaking training that integrates these multimodal data streams within a five-layer architecture (Environment Generation, Data Gathering, Analysis, Feedback, and Visualization), instantiated through a modular microservices approach. A preliminary usability assessment with ten participants demonstrates the platform's learnability and task completion effectiveness, advancing beyond exposure-only systems toward intelligent, data-driven, and personalized skill development.

Keywords: Extended Reality, Public Speaking Anxiety, Multimodal Data Analysis, Adaptive Feedback, Virtual Reality Training

DOI: 10.54941/ahfe1007283