Articulated Spatial Audio for Minimally Invasive Surgery Training

Abstract

Contemporary spatial sound recording and reconstruction systems enable audiences to experience realistic 3D soundscapes from multiple speakers or binaural headsets. Spatial audio is especially useful in minimally invasive surgery training which represents the sound sources, dynamic patterns, and verbal communications. In our study, we recorded and reconstructed 3D sound from a laparoscopic surgery room with multiple cases. The spatial sound data contains ambient sounds, equipment sounds, and music in the OR environment. We then articulated the ECG sound based on the simulated patient’s conditions. For example, when the patient feels pain, the heart rate increases. Our experiments show that the articulated spatial audio helps to narrow the gap between the abstract training boxes and the actual OR environments. It is one step forward to advance Extended Reality (XR) with physical and physiological variables.

Keywords: Spatial Audio, 3D Sound, Minimally Invasive Surgery, MIS, Surgery Training, Laparoscopic Surgery, Articulated Sound

DOI: 10.54941/ahfe1004990