Motion Capture Body Tracking and Functional Safety in Dynamically Controlled Theatre Automation Systems

Abstract: Live theatrical performance is an ever-evolving art form in which visionary theater makers are incorporating evolving technologies into performances to connect and engage modern audiences. Recent developments in theatrical motion control systems are enabling vibrant and adaptive control through dynamic automation. Traditional theatrical motion control systems use a set path motion profile to produce predictable movements of scenery and people through space and time. New dynamic control systems utilize an external generated set point parameter to specify the desired motion. This flexibility enables a DJ to control the movement of flown digital chandeliers above a dance floor with their regular beat control or for Alice to control her flown decent as she tumbles down the rabbit hole by changing the position of her arms. Putting the control of the equipment into the hands of the performers is a significant leap in evolution stage automation control.To accomplish the motion tracking of performers on stage, the research team developed a unique set-up of Microsoft Kinect sensors to monitor the stage environment. The skeletal models of up to four performers were captured and transmitted via an ethernet network to the video game engine. The main benefit of this tracking method is that it allowed for both traditionally costumed performers and regularly dressed audience members to be captured by the system. Most motion capture systems in entertainment require the use of custom body suits which can disenchant and distract audiences. Our system maintains the traditional relationship between costume and character while collecting the necessary interactivity data.The natively 3D nature of the video game environment is optimally suited to handle the interaction between performer and the digital environment. Using the skeletal tracking models from the sensor system, the video game engine allowed the performers to dynamically interactive with the stage environment. They could open doors, swat away birds, and flap their wings to fly away. The added benefit of the system was that it allowed members of youth young audiences to be called onto stage to further the narrative with their actions. The resulting system enables a new performance methodology with exciting new options for theatrical storytelling, educational training, and interactive entertainment. This evolution of control adds risks beyond those present in deterministic systems.Dynamic control of entertainment automation systems requires strong conformance to functional safety principles to mitigate the risks to affected personal and environments. Functional Safety is the mitigation of unacceptable injury risks through the implementation of one or more automatic protection functions (often called safety functions). In traditional theatre environments, safety functions have historically been limited to “Emergency Stop” functions which halt all motion when an operator presses “the big red button”. As systems become more complex and utilize dynamic control, theatrical automation control systems need to utilize a functional safety evaluation process to better protect persons and the environment for systematic and random failures in the systems. Industry experts develop best practice procedures to address safety concerns which are written into documents called standards.The presenter conducts research at the intersection of entertainment and engineering, has actively participated in the development of numerous American national standards with the Technical Standards Program of ESTA (Entertainment Services and Technology Association), and is a dual certified functional safety engineer (TÜV Rheinland and Underwriters Laboratories). This presentation will go through the functional safety considerations required to implement dynamic control per national and international standards using examples from realized performance projects led by the presenter.

Keywords: theatre, safety, theater, live performance, motion capture, game engine, Kinect, dynamic, automation, body tracking, performance, functional safety

DOI: 10.54941/ahfe1004077

Cite this paper:

​