Symbolism in Extended Reality

Abstract

For centuries, humans used symbols to describe scenes, objects, and events. Some of the symbols have evolved into pictorial languages such as petroglyphs and Chinese. The rapidly growing Extended Reality (XR) technologies enable users to experience immersive photorealistic views with head-mounted displays, 3D projection screens, or holographic display panels. The purpose of Extended Reality (XR) is to make invisible to be visible to the user. In this study, we explore how to superimpose symbolic feedback to the user in forms of geometric shapes, trajectory traces, spectra image, semantic network, and multimodal alerts, including audio and tactile signals. We zoom into the laparoscopic surgery train as a case study for applications. Our preliminary experiment results show that the symbolism approach does not significantly increase the distraction of attention or computational load. We also found that Extended Reality, especially, multimodal alert would be a bridge between manual operations and autonomous operations.

Keywords: Symbolism, Simplicity, Augmented Reality, Virtual Reality, Overlay, Visualization, Extended Reality, Abstraction, Discovery

DOI: 10.54941/ahfe1004608