A Privacy-Preserving Edge Audio Analytics Framework for Contactless Operator Resilience Monitoring in Main Control Rooms

Abstract

Maintaining operator cognitive readiness in industrial main control rooms is essential for safe operation, yet continuous monitoring remains difficult because wearable sensors can burden workers and cloud analytics can conflict with data-sovereignty requirements. This paper presents a privacy-preserving, contactless edge audio analytics framework for operator resilience monitoring that processes all speech locally. The architecture uses a dual-gating front end that combines FRCRN-based speech enhancement with ECAPA-TDNN speaker verification to suppress acoustic interference and isolate operator-specific speech in multi-operator environments. Authenticated speech is then analyzed by three parallel modules: emotion2vec-based stress representation learning, Paraformer-based transcription and semantic intent detection, and an acoustic vocal-fatigue proxy derived from statistical and cepstral features. These indicators are fused through an allostatic-load-inspired risk model and regulated by a finite-state alert controller with persistence and cooldown logic. Prototype evaluation under representative noisy, overlapping control-room conditions indicates end-to-end latency below 200 ms for up to five concurrent audio streams on standalone local hardware, while reducing false positives relative to a conventional far-field baseline. The results support the feasibility of privacy-preserving, audio-only operator-state monitoring for safety-critical control rooms.

Keywords: Edge Computing, Cognitive State Assessment, Audio Signal Processing, Human Factors, Voice Biometrics

DOI: 10.54941/ahfe1007557