Motion Tracking for the body and AI

Body tracking is a new technology that helps to track the movement of people, and can be used in the creative industries such as film and gaming to make animation and movement as realistic as possible.

However, during my time studying my module "Digital Media and the Senses", I started to question how accurate this body tracking may be, and even if AI may be more biased against some people than others. This also made me question points regarding the purpose of these technologies, such as how body tracking would track the movements of people with physical disabilities who need technology such as wheelchairs to move.

The aim of this project is to track these ideas using AI motion capture and learn about any potential biases within the application. After this, I aim to use AI such as Teachable Machine to examine how AI can recognise different elements and motions to discover and theorise about how AI could potentially overcome these biases. Furthermore, I aim to look into technological development and theorise a prototype of different technologies that could be used to overcome these biases.

Body Tracking AI, Rokoko and Bias

  • The Beginning

    Due to an interest in creative assets for media industries, I found it interesting to research what type of data AI can gather about humans by tracking our movements. The AI used in this project is on a platform named Rokoko and gathered data about how I moved through video recordings. I recorded this footage to analyse just what the AI is tracking and turning into data, including colour, movement and objects.

  • The Application

    I wanted to create a variety of virtual models using Rokoko's AI features to show what the AI did and didn't track when it came to movement. I recorded footage in multiple scenes wearing different outfits to see how much external factors affected tracking alongside a variety of movements to analyse which were the easiest for the AI to track. This gave me a wide variety of models ranging in quality, with many of them showing highly inaccurate movement due to the external changes inflicted on them.

  • Results

    After recording my models, I found that there were many biases within the AI application that made them less applicable to certain individuals. When tracking movement using a crutch and wheelchair, the AI failed to track any movement of the medical equipment, leading to the models appearing shakey and unstable. However, the AI did appear to be affected by external factors such as colour, showing that it was biased against some external factors.

The Human Body and the Tracking of Data

The results above help to show that AI body tracking is biased, but what can we do to combat this? Can AI learn how to process the movements of both objects and people? I aim to research this idea to create a prototype for how the motion tracking of the human body can be improved for maximum accuracy to prevent the identified biases from affecting the final animation, or at least make the final result more controlled.

The Prototype

After researching my idea I discovered more about body tracking and object detection, discovering the 3 key stages of the tracking process and why biases may occur. My prototype consists of a dataset containing information on people, objects and colour, all of which an AI can be biased against, in an attempt to get a full body tracking AI to detect and animate its previous biases. This dataset, imagined in full, would go on to help AI respond better to our senses and the media, with animation for media projects such as films and games becoming further embedded in our sensory world as a result.