Tactile information coding for touchless interaction with medical devices by means of hand gestures in the air

Abstract: Non-contact control of medical devices in the operating room offers essential advantages. However, touchless forms of interaction such as gaze, hand and voice gesture control are poorly accepted in practical use in medicine due to insufficient feedback. Current gesture control systems usually output audiovisual feedback about a performed selection of a function. Consequently, audiovisual feedback does not relieve the already heavily loaded audiovisual perception channel of the surgeon. Therefore, the frequent request of surgeons results in a haptic feedback similar to the manipulation via a physical control element, where a direct feedback about the selection of a function is given. In this research project, the advantage of contamination-free manipulation of hand gesture control is combined with tactile feedback based on ultrasonic waves to transfer information by the tactile perception channel.For the study, various 10-point tactile scales were generated, which were examined with regard to their suitability for the transmission of information. The focus of the investigation was the subject‘s recognition of the information contents "increase", "decrease" and the marking of a "middle" or a "preferred scale value". Therefore, a virtual slider with a tactile feedback based on ultrasound waves was implemented. This slider was provided with a discrete tactile feedback in the form of a 10-point tactile scale, which could be perceived by the hand during a sliding movement. For the coding of information, the recognition of differences in the tactile scale was relevant. To identify the differences, the tactile feedback could be modified in terms of feedback intensity and scale spacing. A total of 16 different coding features were tested. 4 coding features for coding an “increase”, 4 coding features for coding a “decrease” and 8 coding features for coding a “middle” or “preferred value”. The tactile feedback was tested on 30 test persons (15 f / 15 m, Ø = 24.33 years, SD = 3.74 years). The experiment consisted of a main and a secondary task. The main task was to perform a precision task on a medical phantom. As a secondary task, the test persons had to perceive the change in the scale and adjust the scale position directly afterwards. The secondary task was performed blindly, without visual or acoustic feedback.The evaluation of the objective data such as task completion or operating time as well as the subjective data such as recognition of the tactile coding feature or mental load show differences between the characteristics of the coding features. With an interference statistical analysis of the results, significant differences between the different characteristics of the coding features concerning the effectiveness, efficiency and user satisfaction are identified. It’s also shown that the most appropriate coding features for marking an “increase”, a “decrease” and a “middle” or a “preferred scale value” based on this study need to be investigated in more detail in further studies. A particular focus of further investigations will be on the difference threshold with respect to feedback intensity and scale spacing of tactile feedback.

Keywords: human, machine interaction, tactile information coding, haptics, gesture control

DOI: 10.54941/ahfe1003483

Cite this paper:

​