Investigating AI Model Limitations in Recognizing Faces and Bodies in Ballroom Dance Settings

Abstract

The advancement in the face and body detection algorithms has sparked an interest in using them to assist physical education and sports training, where AI can analyze students' body postures and movements to offer corrective guidance and prevent injuries. However, ballroom dance settings are uniquely different from traditional settings often used for face and body detection. On the one hand, most traditional face and body detection algorithms detect individuals instead of a collaborative dyad. Moreover, specific dancing postures may pose additional challenges for AI algorithms to detect. In order to unlock the power of AI in enhancing real-time feedback for dancers on their movements, postures, and expressions, there needs to be a thorough understanding of the capabilities of AI in analyzing complex dance sequences and identifying subtle nuances in body language. In this work, we examined four widely adopted body and face detection models on their effectiveness in detecting ballroom dancers. We found that these models shared key limitations. First, they are more likely to detect the man than the woman in the dyad, especially when the woman is curved backwards. Second, they often detect the couple as one person, mixing different body sections. Third, errors are frequent when the dancers do not face the camera or when they are wearing specialized costumes. This project offers suggestions to diversify the training datasets of such algorithms to make them suitable for new settings including ballroom dance.

Keywords: Skeleton detection, Ballroom dancing, AI mistakes

DOI: 10.54941/ahfe1004599