Improving the accuracy of automatic object tracking by using posture recognition

Abstract: In recent years, in addition to classical machine vision algorithms, AI-based techniques such as image segmentation and object detection and recognition have been utilised in the transport sector for various purposes. These include vehicle-on-board applications for driver assistance or autonomous “self-driving”, such as object detection for collision avoidance and traffic monitoring and management applications. One application in the field of aviation is “remote tower”. “Tower” air traffic controllers work by observing the traffic at airports directly from a control tower building. Remote tower uses video cameras to replace the “out-of-the-window” tower view, enabling the physical tower to be eliminated and allowing controllers to provide services from a remote location. Digitization of the image stream also gives the opportunity to develop functions to support the controller tasks and improve situational awareness.This paper describes the development of a system to support remote tower operators in airport traffic control. The system features pan-tilt-zoom (PTZ) cameras with an automatic tracking function for aircraft flying in the vicinity or moving on the surface of the airport. The function operates in two modes: “sensor” tracking based information from an aircraft surveillance sensor, and “optical” tracking based on recognising and tracking aircraft or ground vehicles from PTZ camera images.This paper discusses the challenges and solutions concerning the automatic tracking function of the PTZ camera utilising image recognition.Challenges in Automatic Object Tracking Using Image Recognition:This research has developed functionality for recognising target such as aircraft and vehicles at airports and their surroundings in video images using YOLO. YOLO incorporates both target extraction and recognition processes within images, having the advantage of enabling rapid processing up to target recognition.On the other hand, several challenges arise when using YOLO recognition results for target tracking. One such challenge is the occurrence of “switching” during target tracking. Target switching refers to instances, when multiple aircraft are present in the image, tracking shifts to an object with higher recognition accuracy, or where tracking mistakenly follows another target if the originally recognised and tracked target was lost by an obstacle. As our first trial, to suppress this “switch”, we attempted a method whereby the velocity vectors of each object are calculated from the image recognition results, and the difference between the velocity vectors of the target object and other recognised objects is used as a discrimination parameter.This method determined how closely “current movement” matched “past movement”. These were estimated as a cubic function between “current movement” and “past movement”, using information on the “last position of the past movement”, the “first position of the current movement”, the “velocity of the past movement”, and the “velocity of the current movement”. Movements were judged to be closer together when the coefficients for the cubic “change in acceleration” and the quadratic “acceleration” were smaller. However, as a result, when aircraft crossed at slow speeds, fluctuations in the image recognition segmentation frames interfered, preventing the detection of velocity vectors. Furthermore, in cases where detection was unstable due to factors such as aircraft becoming obscured, resulting in gaps in the data, the coefficients became smoother. This led to instances where other targets were misidentified.Object identification based on posture detection:This paper investigated whether continuous tracking of target aircraft could be achieved even in crossing situations by learning aircraft orientation using YOLO v8 and identifying the aircraft's orientation. To recognise aircraft orientation, the ‘Posing’ function was utilised for ‘Pose’ learning. Annotation data comprising six points—nose, fuselage, right wing, left wing, tail, and vertical tail fin—was created and used for training.In the case study, we could determine the aircraft's orientation through image recognition. By utilising this orientation recognition result as a determination parameter for tracking, we confirmed that continuous tracking is achievable even in situations where automatic target tracking would previously be impossible using only velocity vectors.

Keywords: Image recognition, Automatic object tracking, Machine learning, Posture recognition

DOI: 10.54941/ahfe1007098

Cite this paper:

​