Enabling mobile devices to be better compatible with augmented reality (AR) has been a big focus for Apple and Google lately, as they introduce the tools necessary to enable developers greater and easier access to AR features. Not to be left behind, the Facebook AI Camera Team is working on various technologies and tools. Adding make up and using filters is just the beginning though, as Facebook aim to create accurate full body tracking technology which uses just a mobile phone camera.
Being able to track body movements in real time is a difficult task, as the Facebook team explain in their blog post. Poses and body shapes are difficult to track across a wide range of people, not to mention items of clothing can help confuse matter, unlike facial features, which have generally uniform placements and features with fewer variable.
While still in the research phase, Facebook have managed to device technology which can detect body poses and accurately track a body against a background. The technology is based on the Mask R-CNN framework, which they’ve adapted for full body tracking and segmentation.
Unlike many solutions to body tracking, the application they’ve devised is only a few megabytes in size, and is lightweight in nature, meaning even conventional smart phones can actually run the technology.
What’s next for the team? Well, in their blog, the team state; “Developing computer vision models for mobile devices is a challenging task. A mobile model has to be small, fast and accurate without large memory requirements. We will continue exploring new model architectures which will lead to more efficient models. We will also explore models that can better fit in mobile GPUs and DSPs which has the potential to save both the battery and computational power.”
The new technology looks to be taking augmented reality on mobile devices to the next level – at this point, many apps feature AR features, whether they be comprehensive or smaller, and Facebook’s research might put them at the forefront of mobile AR technology. We hope to see a more finalised version of Facebook technology in the near future, but in the meantime, you can look at their blog for more technical information, and stay on VRFocus for more virtual reality (VR) and AR news.