Challenges in Research Projects on Augmented Reality in the Domain of Mechatronic and Robotic Applications

Abstract: The team of the Otto Rettenmaier Research Laboratory for Digitalization at Heilbronn University has been working for years on research projects for interaction with robots as well as a mechatronic factory system using augmented reality (and also virtual reality as well as quite simply via mobile devices - keyword BYOD). As AR Device usually a Microsoft Hololens is used, in the VR area a system of HTC Vive is available in the lab. Other mobile devices are smartphones as well as tablets from different manufacturers.A number of different implementation challenges have been identified in the projects, some of which are quite different. As a main issue is certainly the combination of industrial components such as the robots with the systems that are used for the augmented reality applications. The AR applications and systems typically originate from the gaming world and are not aligned with typical industrial communication standards in terms of issues such as data communication and more. As described, one technical hurdle that has emerged is the communication between the interaction devices and the mechatronic systems. In the industrial world - based on the topic of Industry 4.0 and the associated developments - the OPC UA communication architecture has emerged. However, OPC UA -quasi as an industrial standard- is of course established as a standard in entertainment technology.In addition, there are questions about the development environment. This is predetermined on the side of the mechatronic systems by the products from the control technology, but relatively free on the side of the AR/VR applications. The bridge between these worlds must be developed in the projects anyway. The bandwidth of the respective engines for the development of AR/VR applications, on the other hand, is very broad and very dynamic - meaning that the possible range of functions changes with every update.The biggest challenges, however, come from the actual application scenarios based on industrial production. The classical interaction concepts, which are given for example by the respective operating terminals of robots and are known to the user, can only be transferred to the virtual or extended space with great difficulty. Furthermore, the mapping of the actual production environment in the virtual world is of course a very big problem.The article explains the hurdles encountered in the various research projects and the solutions developed in the projects as well as the questions that are still open.

Keywords: Augmented Realtiy User Interaction Robotics Mechatronics

DOI: 10.54941/ahfe1002315

Cite this paper:

​