Tactile feedback for controlling a robot with hand gestures

Abstract

This study investigates contactless gesture control combined with tactile ultrasound feedback as an alternative interaction method for controlling a robotic arm. A gesture alphabet was developed for translational and grasping movements. Hand motions were captured using a Leap Motion Controller, while tactile feedback forms were projected onto the user’s palm. 5 feedback forms were designed and evaluated regarding perceptibility, mental workload, directional changes, gripper opening width, task fulfillment, and operating time. The study included 32 participants, who first assessed the isolated feedback forms and then performed a grasping and placement task using a robotic arm. The results show that simple geometric forms, particularly points and lines, are perceived and interpreted more reliably than complex shapes such as triangles or helix. A line as Feedback form achieved the most favourable results across several variables, including directional perception, lower mental workload, and shorter average operating times. Rotational coding proved less intuitive and increased mental load, especially when used to represent linear robot movements. Overall, the findings indicate that tactile ultrasound feedback can support contactless robot control, provided that feedback forms are simple, continuous, and spatially intuitive. Future work should examine blind operation, larger samples, and person-specific adaptation based on tactile sensitivity.

Keywords: Human-Robot-Interaction, Huma-Machine-Interface, Tactile Feedback, Usability, Information Coding

DOI: 10.54941/ahfe1008219