A number of companies involved in immersive technology such as virtual reality (VR) and augmented reality (AR) have filed patents in the past year that relate to new developments for the next generation of AR and VR headsets. The latest to be revealed is a Google patent to track facial expressions using eye-tracking cameras.
The patent is called ‘Classifying Facial Expressions Using Eye-Tracking Cameras’. It works by using machine learning algorithms, combined with eye-tracking sensors in order to analyse the expressions on users faces.
The patent goes on to describe how the system would allow users to interact with each other in the virtual world through avatars, which would not only resemble the user, but would also be able to reflect their facial expressions.
It is noted in the patent that since VR headsets tend to obscure the user’s face, facial expressions can be difficult to ascertain when the user is wearing one. The machine learning algorithm is said be able to be trained to recognise expressions related to anger, happiness, surprise and more, as well as identifying more subtle expressions such as a raised eyebrow.
Eye-tracking has been widely recognised as an important part of the development for the future iterations of VR hardware, since eye-tracking will not only enable more accurate representations of user avatars, but can also introduce new interaction methods, as well as technology such as ‘foveated rendering’ which can save processing power by ensuring only the areas currently in view are fully rendered.
The introduction of eye-tracking into VR could be used to create more immersive experiences for VR users, but the technology is yet to be integrated into any of the commercially available VR headsets.
The Google patent could change this, but as with any patent, there are no guarantees that it will be used in a product.
For future coverage on new developments in VR technology, keep checking back with VRFocus.