Conveying what it’s actually like in virtual reality (VR) has been one of the biggest hurdles to adoption, with the most common video technique being a combination of green screen and mixed reality (MR) technology. While this helps viewers see what VR players are engaged in there was one other barrier left, the headset itself. Now Google Research and Daydream Labs have unveiled a new digital technique that allows a users face to be seen whilst wearing a head-mounted display (HMD).
What Google has come up with is a way to make a headset seem transparent so that viewers watching an MR video can see the range of emotions being portrayed by the player. To do this the development team uses a combination of 3D vision, machine learning and graphics techniques to build a model of the person’s face, capturing various facial variations. Then a modified HTC Vive is used that contains SMI eye tracking tech, recording gaze related data. This is then blended together to give the illusion for seeing a users face whilst they play in a virtual world.
The Google Research Blog goes into much greater detail with Vivek Kwatra, Research Scientist and Software Engineers, Christian Frueh, Avneesh Sud explaining the future applications of the technology: “Headset removal is poised to enhance communication and social interaction in VR itself with diverse applications like VR video conference meetings, multiplayer VR gaming, and exploration with friends and family. Going from an utterly blank headset to being able to see, with photographic realism, the faces of fellow VR users promises to be a significant transition in the VR world.”
The project will be an ongoing collaboration between Google Research, Daydream Labs and the YouTube team, with the technology set to become available across select YouTube Spaces for creators in the future.
For the latest updates from Google Research and Daydream Labs, keep reading VRFocus.