Vocal Markers in Aviation of Workload, Stress, Fatigue, and Sleepiness: A Protocol Validation Study

Abstract: This study is composed of two parts: the first part is a systematic review, and the second part is a protocol validation study. The systematic review aims to summarize and consolidate evidence from existing studies on the impact of workload, stress, fatigue, and sleepiness on speech, focusing on identifying specific vocal markers associated with these states within the context of aviation. Using PRISMA guidelines, we performed a comprehensive search of electronic databases, including Scopus, ScienceDirect, PsycINFO, and Web of Science. Twenty studies met the inclusion criteria and were analyzed to extract consistent vocal features indicative of these psychophysiological states in pilots and air traffic controllers (ATCs). Key findings from the review indicate that stress and workload are associated with increased vocal intensity and pitch, reflecting heightened sympathetic nervous system activation. Conversely, fatigue and sleepiness manifest through reduced vocal energy, slower speech rates, and increased pauses, indicative of diminished central nervous system activity. Mel-frequency cepstral coefficients (MFCCs) were highlighted as reliable and versatile indicators across all states. Building on the insights from the systematic review, the second part of the study focuses on validating an analysis protocol designed to detect and classify psychophysiological states in real-world aviation scenarios starting from vocal behavior. This protocol builds on the vocal markers identified in the review and applies structured acoustic analysis techniques using Praat.Real-world audio recordings were collected from pilots and ATCs. These scenarios included high-stress emergencies and routine operations. The recordings were processed to extract vocal features, including pitch, intensity, speech rate, pause duration, and MFCCs. Machine learning models were trained and tested on these features to classify the vocal data into categories of workload, stress, fatigue, and sleepiness. Although preliminary analyses are still underway, the current phase focuses on feature extraction and classification strategy development. Performance metrics will be assessed in future phases once model training is finalized.The integration of this validated protocol into aviation safety protocols may offer promising prospects for enhancing performance monitoring and risk mitigation. Real-time vocal monitoring systems could provide immediate feedback to pilots and ATCs, enabling timely interventions to address stress or fatigue before they compromise safety. Future work will focus on testing the system in operational settings and exploring the integration of vocal monitoring with existing cockpit technologies and ATC systems to support real-world implementation.

Keywords: Vocal markers, Acoustic analysis, Aviation safety, Pilots and ATCs Communication

DOI: 10.54941/ahfe1006584

Cite this paper:

​