In what looked like the start of things to come for the Oculus brand, back in May it was announced that Oculus Research, the R&D division of the company was being rebranded as Facebook Reality Labs. Today, the new lab has made its first big announcement since the name change, introducing a new mirror reconstruction technology.
When creating a realistic scene one of the hardest elements to get right are mirrors and glass panes, making not only reflections look true but also the way they scatter light in a scene. So Facebook Reality Labs has created a new fully-automated pipeline to reconstruct mirrors and other reflective flat surfaces, which it’ll be discussing at SIGGRAPH 2018 this month.
Detailing the research on the Oculus blog, the team highlights the advances made over existing 3D scanning systems.
“If you look around the mirror section of a home decor shop, you’ll immediately see a wide range of shapes and sizes—almost no two mirrors are the same!” says Research Scientist Thomas Whelan. “It became obvious to us very early on that we needed a general solution that didn’t make too many assumptions about what was being reconstructed. We really wanted a solution that just worked in real-world environments because that’s where it’s most useful.”
“The sheer variety of mirror types and shapes was quite stunning in the beginning,” Research Scientist Julian Straub went onto note. “Designing a system that would be able to handle most or all mirror shapes and sizes was the main goal. Then we realized that the system would also work with glass surfaces with a minor additional classification step. That was pretty cool.”
They learnt that by identifying mirrored surfaces, they could then re-render a scene with correct geometry and reflections. This led to the creation of a capture rig which included an infrared depth capture camera, an RGB color camera, two SLAM cameras, an infrared projector and a special target the system could identify.This allowed a mirrors boundary to be identified even without a frame.
Check out the video below for a more hands-on description, or head over to SIGGRAPH 2018 for the full in-depth talk. This is just one of the ways Facebook Reality Labs is helping developers deliver more immersive content, as further advances are made VRFocus will keep you updated.