Remote Haptics: Introduction to a touchless remote inspection system for industrial experts

Abstract: Physical inspection poses significant risks, such as exposure to hazardous or even impossible working situations like the presence of rotating parts or a high-temperature environment. Remote inspection with robotic platforms is therefore an established method. This paper introduces a prototype of a touchless remote inspection system for industrial experts, combining both visual and haptic feedback. The system is split into two sub-systems, user- and environment-side.The user side consists of a wearable hand glove, arm exoskeleton, and a VR headset. The environment side consists of a robotic arm, stereo camera, and sensor system mounted on the TCP of the robotic arm that acquires the object properties touchless. The glove and the arm exoskeleton are used to teleoperate the robotic arm and the data gathered by the sensor system are transcoded into haptic signals and realized using actuators mounted on both the glove and the arm exoskeleton. A Robot Operating system (ROS) environment acts as a signal interface between the two sub-systems and gives the opportunity to create a digital twin for testing and applying new control algorithms.To allow a natural feeling in inspection, the sensor system with µm accuracy is used to acquire objects' properties, such as shape, texture information, and vibration. This system consists of a Structured Light (SL) camera with an accuracy of 30 µm at a range of 0.3 m, along with a Continuous Wave (CW) radar sensor.The glove and the arm exoskeleton are designed to allow the natural movement of the user's hand and arm. Twelve IMU sensors measure the fingers’ joints to calculate the fingertips' positions and the orientation of the palm, while the exoskeleton has five Degrees of Freedom (DOF), four for the shoulder, and one for the elbow. On the other hand, they are equipped with different types of actuators to allow the proper haptic feedback. The glove is equipped with Linear Resonance Actuators (LRA) to allow tactile feedback and a braking mechanism for kinesthetic feedback. Another braking mechanism is used in the arm exoskeleton in order to give the proper feedback in the case of collision, for example.With the accuracy of the SL camera, several algorithms are implemented, with the fingertips' positions as input, to extract the texture information and detect the object's shape from point cloud data created by the SL camera. Another algorithm is implemented to make use of the high accuracy of the radar sensor, especially in the case of a vibrating object, to allow the inspection of such dangerous and rather impossible objects physically. The output of these algorithms is then mapped into haptic signals and sent to the user side.The live visual feedback from the stereo camera is fed to a VR/AR environment. Also, the haptic glove is augmented in the environment. The stereo camera is mounted on a mechanism independent of the robotic arm. This mechanism allows mimicking of the user’s head movements while wearing the VR headset with a current latency of 400 ms.

Keywords: Remote inspection system, Haptic feedback glove, Robot teleoperation, Acquiring texture information touchless

DOI: 10.54941/ahfe1004288

Cite this paper:

​