"Fall PreNoSys": Augmented Reality-based Tripping Hazard Notification System and Initial User Feedback Study

Abstract: Falls and falling remain a significant problem for people as they age. This work proposes a mobile augmented reality (AR) based system called “Fall Prevention via Notification System” (Fall PreNoSys) to detect likely tripping hazards around the wearer and provide notifications to help them avoid safety problems, along with two phases of user feedback to improve the system design. Blending mobile technologies and human-computer interaction requires significant work on human interface components to become an effective, calm, and useful tool in daily life. A series of studies involving human participants was conducted to gather feedback on the Fall PreNoSys interface design, its utility, and its underlying concepts. Current AR research in gerontechnology and in-home assessments represents a nascent field, and Fall PreNoSys offers a novel approach to fall prevention.Fall PreNoSys uses a Microsoft HoloLens v2 to gather real time 3D models of the space around the user. These models are segmented to identify potential tripping hazards, and the HoloLens scene understanding library is employed to classify objects using an AI classifier. The combination of the Fall PreNoSys algorithm for object segmentation and scene understanding results in a list of objects that can trigger notifications as the user moves around a room.To evaluate notifications style and to get feedback from possible users of the system, two pilot user studies were performed. These studies provided early-stage feedback, initial impressions, guided the continued design of notifications, tested the object detection algorithm's robustness, and evaluated user reactions to static and dynamic notification types developed for Fall PreNoSys.Notifications took the form of 3D visual objects projected onto the HoloLens' AR screen within the wearer's field of view. These notifications were shaped as arrows or OSHA safety-style triangles and were placed on or near identified potential hazards. Based on user feedback from the first phase of the user trial, notifications became interactive, changing color, bouncing in place, and reacting to the participant's relative location to orient their attention to hazards.The study used walking tracks with likely in-home tripping hazards, a combination of machine learning-based detection algorithms, and multiple styles of visual hazard notifications. Study data was collected through two phases of interviews, user feedback of their experiences with the technology, and measurements using the System Usability Study scale to help guide further development of Fall PreNoSys and similar systems in the future. Future work on Fall PreNoSys includes a series of studies with older adults after the latest user feedback from this study is incorporated into the interface design. Additional work includes using eye gaze notification acknowledgements, user path estimations, and out-of-view edge notifications to help people interact with notifications, adapt to the user's walking path, and handle issues with the AR screen's field of view limitations.

Keywords: Augmented Reality, Falls and fall prevention, gerontechnology, user interface design

DOI: 10.54941/ahfe1004444

Cite this paper:

​