Design and Implementation of Gesture Interaction for a Command And Control System Based on Multimodal Data Fusion

Abstract

With the continuous improvement of people's demand for interactive experience,gesture interaction technology, as a natural and intuitive way of human-computer interaction, has attracted more and more attention and research. However, the research and application of gesture interaction in command and control system is still lacking. Firstly, the characteristics of command and control system and the key technologies of gesture interaction are investigated in this study.Then a series of operating gestures that meet the application scenarios of the command and control system are designed,and a gesture recognition algorithm based on the fusion of image data and skeletal data is proposed.By integrating with the command and control system, the function of controlling the command and control system through gestures is realized. Finally, the feasibility and effect of gesture interaction technology in the application scenarios of command and control system are evaluated through experiments.The experimental results show that the application effect of the technology in the command and control system system is good, which can reach 94.89% under typical tasks, improving the interaction efficiency and interaction experience.

Keywords: Human-computer Interaction, Command and Control System, Gesture Recognition, Multimodal Human-Computer Interaction

DOI: 10.54941/ahfe1005361