Facebook is now Meta but What Does That Mean?

Meta

So, Facebook announced its rebrand a couple of weeks ago. Let’s get into it. The social media giant will be positioning itself ready to build the metaverse, and in keeping with that goal, it changed its parent-company name to ‘Meta’. Meta will now be known as the overarching company that runs Facebook, Whatsapp, Instagram, and Oculus among others. 

Meta

Mark Zuckerberg, CEO and founder of Facebook, has said that he wants to build the metaverse; with it being a virtual world where people can do anything and everything, including work, game, and socialise. Zuckerberg said, “We believe the metaverse will be the successor of the mobile internet…We’ll be able to feel present – like we’re right there with people no matter how far apart we actually are.”

Meta also reported that the revenue of their VR segment had grown so substantially that they will now need to count this revenue separately and divide it into two categories. One category is the family of apps as mentioned above, and the second category is its Reality Labs products featuring their VR and AR technology. 

This change comes at a time when Zuckerberg is expecting the metaverse to reach a population of 1 billion people within the next 10 years. Therefore, this rebrand is to support his vision of building a metaverse to meet that demand. Users will be able to design their own avatars, decorate their own virtual space, meet with people, and even attend virtual concerts and events all from the comfort of their own homes. All of this is not too far stretched from what we see now, with both Fortnite and Roblox hosting virtual concerts very recently.

Meta

Over the last year, Facebook rolled out two platforms in beta to get the metaverse ball rolling. Horizon Worlds and Horizon Workrooms. The former allows users to invite their friends over to their digital world and the latter does the same but purely for professional environments. As well as this, Zuckerberg has also expressed interest in the NFT and Crypto space, working on how digital assets can be effectively represented within the metaverse.

However, this change comes at a time when Facebook is facing uphill public relations battles. This includes the recent whistleblower, Frances Haugen coming forward with leaked documents outlining the toxic business practices and evidence of the long term negative impact the platform will have on the public. Many believe that this rebrand is just to distract the public from what is really going on behind closed doors. Imran Ahmed, CEO of the Center for Countering Digital Hate had not-so favourable words about the Facebook reposition, saying the following, ‘…just imagine what Facebook could achieve if it devoted even a fraction of its metaverse investment on proper content moderation to enforce even the most basic standards of truth, decency and progress.’

There has been a mixed reaction from industry experts on this move from Zuckerberg. Here are a few examples of the negative comments:

“While it’ll help alleviate confusion by distinguishing Facebook’s parent company from its founding app, a name change doesn’t suddenly erase the systemic issues plaguing the company… If Meta doesn’t address its issues beyond a defensive and superficial altitude, those same issues will occupy the metaverse,” said Mike Proulx, VP and Research Director at Forrester.

Meta

Tom Bianchi, VP of Marketing at Acquia stated, “Big tech companies are taking steps towards consent-led data strategies, such as embracing first-party data and the phasing out of third-party cookies… But, given the scale of the Cambridge Analytica scandal and the recent whistleblower revelations, it’s clear Facebook has a long way to go before it regains consumer confidence.” 

“With social media under scrutiny, and with increasing regulation inevitable, the rebrand and pivot seem to be an attempt to deflect some of the attention from recent scandals, while also entering into a new arena where Facebook may feel better positioned to shape the rules,” said Arielle Garcia, Chief Privacy Officer at UM Worldwide.

It’s not all bad though, here are a few positive outlooks…

Jeff Sue, General Manager of Americas at Mintegral said, “The metaverse and virtual reality world is a great opportunity for Facebook and the new rebranding signifies their intentions… Traditional social media has been a crowded space recently, especially with the rise of TikTok capturing younger audiences. Facebook needed to pivot on many levels from product to PR, and this will allow it to continue growing.”

“Meta represents the future not only for Facebook but for marketing… The metaverse represents the next massive opportunity for brands to engage consumers in new ways,” Aaron Goldman, Chief Marketing Officer at Mediaocean opined. “The key to success for brands in the metaverse will be ensuring that their assets are built into the ecosystem for easy access by consumers.”

This future vision certainly has the tech industry divided. With Meta planning on spending $10 billion in the next year alone on this bet, the social media giant is most certainly committed no matter what other issues hamper it.

Leaked Facebook MR Headset Confirmed as Project Cambria, a “High-end” Device Coming Next Year

Facebook today confirmed Project Cambria, the MR headset which was spotted in a recent leak. The company says it’s a high-end headset designed to roll out more advanced technology before being able to bring it down to the price point of the Quest line. Project Cambria is said to be launching sometime next year.

If you followed along earlier this week you’ll already be at least a little bit familiar with Project Cambria; the headset’s look was revealed in leaked videos, but there were few details to be gleaned from it otherwise.

Today during Facebook Connect the company confirmed the headset, which is codenamed Project Cambria. Facebook CEO Mark Zuckerberg said that the headset “isn’t the next Quest,” though it will be compatible with the Quest platform.

Image courtesy Facebook

The way Zuckerberg explains it, Cambria is a high-end headset (and will be “at the higher end of the price spectrum”) which will include advanced technology that the company wants to experiment with before considering bringing it to the more affordable Quest product line.

Project Cambria Specs and Features

We don’t know a whole lot of specifics about Cambria yet, but the company did confirm a handful of things.

For one, the headset will include “high-res color mixed reality passthrough,” which will make it better for mixed reality applications. It will also use pancake optics to reduce the headset’s bulk.

Image courtesy Facebook

Eye-and face tracking will be included to provide a more realistic representation of the user within the virtual world.

From the leak and teaser photos we can also see that the controllers are ditching the tracking rings and are likely tracked with on-board cameras.

Project Cambria looks like it will lean much more into its mixed reality capabilities than Quest 2 is currently capable of.

Project Cambria Release Date

Facebook said that the Project Cambria headset will launch “next year,” but offered no additional details on timing or price, though the company says it’s already working with developers to begin building experiences for the headset.

The post Leaked Facebook MR Headset Confirmed as Project Cambria, a “High-end” Device Coming Next Year appeared first on Road to VR.

Facebook’s 2021 XR Investments to Total $10 Billion, Even More in 2022 & Beyond

In the company’s Q3 2021 earnings call Facebook today said it expects its XR investments this year to top $10 billion, and advised investors that it plans to spend even more than that in subsequent years.

Facebook’s top brass believe so deeply that XR and the metaverse will be a transformative computing platform that they are dropping tens of billions of dollars in investments aimed to give them a head start in the space.

Today during the company’s Q3 earnings call, Facebook said that it planned to change its future financial reporting to more granularly detail the aggressive investments its pouring into Facebook Reality Labs (FRL), the company’s XR and metaverse division which includes Oculus, FRL Research, and more. The company says it hopes this change will “provide investors with additional visibility into the investments that we’re making in augmented and virtual reality.”

While the first financial reports to include the more detailed look into FRL spending won’t come until the company’s Q4 2021 earnings report, Facebook CEO Mark Zuckerberg said on today’s call that the company expects its FRL spending to total some $10 billion in 2021 alone. What’s more, Zuckerberg said, investors should expect that number to “grow even further for each of the next several years.”

Facebook is spending exponentially more in this category than anyone else in the industry, at least as far as we know; Apple & Microsoft are perhaps the only two companies in the world that might match or exceed Facebook’s investment, though neither company has explicitly stated the kind of spending it’s doing in the field. Facebook, on the other hand, seems to want everyone within earshot to know.

The news comes just a week after Facebook announced plans for a 10,000 person hiring spree to bolster its workforce as it races to dominate the metaverse.

Also during the earnings call the company noted that it’s ‘Other’ revenue (a category which covers its non-advertising activities) is up 195% over the prior quarter, for a total of $734 million in Q3 2021. While the category includes non-Oculus hardware (like Facebook’s Portal products), the company specifically said that the revenue was “driven by strong Quest 2 sales.”

There’s plenty more news to come with the Facebook Connect conference set for Thursday where the company may reveal a new Quest headset that was apparently leaked.

The post Facebook’s 2021 XR Investments to Total $10 Billion, Even More in 2022 & Beyond appeared first on Road to VR.

Facebook Reality Labs has Teased 3 Prototype VR Headsets

Facebook - Boz Prototype 1

Not to be outdone with all the other hardware announcements going on at the moment, Facebook’s CEO Mark Zuckerberg and incoming CTO Andrew “Boz” Bosworth have teased three separate virtual reality (VR) prototypes the company is currently developing.

Facebook - Zuckerberg Prototype 1

Bosworth – who is head of Facebook Reality Labs – started with quite a stylish looking standalone prototype (in comparison to the others) with a shiny, almost mirrored black front. In a tweet, he said: “Proud of the research Michael Abrash’s team is working on at FRL-R Redmond—excited to get an early look at some of the technologies that will underpin the metaverse (we work on several prototype headsets to prove out concepts, this is one of them. Kind of. It’s a long story.)”

He then followed that up with the headset seen below, what looks like a heavily modded Oculus Quest with a halo strap and a bunch of sensors underneath – could those be for facial tracking? No further details were provided at this stage, so you’ll just have to stare at the images and ponder what this new tech might do.

Over on Facebook, Zuckerberg posted the image seen above, revealing that this particular model has a retina display: “I spent the day with the Facebook Reality Labs research team in Redmond to demo our next-generation virtual reality, augmented reality and artifical intelligence tech. This one is an early retina resolution prototype. The future is going to be awesome.”

Facebook - Boz Prototype 2

While it’s no surprise that FRL has several VR prototypes being worked on the timing is quite funny. HTC Vive has its special reveal event today, although it looks like that bubble has already been burst. Lynx-R1 launched a successful Kickstarter that easily hit its goal and enterprise-focused Varjo has its own unveiling event coming up. All of this before Facebook Connect on 28th October.

In the past, these sorts of prototypes would’ve been showcased at the annual conference but maybe with everything going on Facebook wanted in on the buzz. Plus, if the company is happily sharing these hardware snippets now, what else does FRL have up its sleeve? An Oculus Quest Pro maybe? There are exactly two weeks to wait to find out. For continued updates, keep reading VRFocus.

Facebook Execs Tease “Next-generation” Headset Prototypes Following Vive Flow Leak

Seemingly in an effort to keep the spotlight during an increasingly hot month of VR announcements, Facebook teased a look at two new XR headset prototypes just a day after an apparent leak of HTC’s upcoming Vive Flow headset.

Update (October 14th, 2021): Facebook shared yet another teasing glimpse of a VR headset prototype; we’ve included it further below.

Although Facebook is expected to make major XR announcements at its upcoming Facebook Connect conference later this month, it seems the company couldn’t help but tease some of what its been working on.

Facebook CEO Mark Zuckerberg today posted an image of himself looking into what he called an “early retina resolution prototype.”

Image courtesy Mark Zuckerberg

‘Retina resolution’ refers to a display which has enough pixel density that it meets or exceeds the resolving power of the human eye. With no cameras apparently on the outside of the headset, this particular prototype is likely a VR headset. However, Zuckerberg said he was checking out a range of “next-generation” XR projects from his company’s VR division, Facebook Reality Labs.

“I spent the day with the Facebook Reality Labs research team in Redmond to demo our next-generation virtual reality, augmented reality and artificial intelligence tech. This one is an early retina resolution prototype. The future is going to be awesome,” he wrote.

Zuckerberg’s post was coordinated with another from Facebook’s VP of XR, Andrew “Boz” Bosworth, who tweeted an image at what he says is another prototype from the FRL Research team in Redmond. The device appears to be a compact XR headset that takes on a goggles form factor

Image courtesy Andrew Bosworth

“Proud of the research Michael Abrash’s team is working on at FRL-R Redmond—excited to get an early look at some of the technologies that will underpin the metaverse (we work on several prototype headsets to prove out concepts, this is one of them. Kind of. It’s a long story.),” he wrote.

The wording suggests that the device isn’t itself an upcoming product but simply a prototype to “prove out concepts,” though he also teases that there’s a “long story” behind the headset, without going into detail.

While it could be something as simple as a non-functional ergonomic prototype, Facebook Reality Labs researcher Douglas Lanman seems to have confirmed that the headset is a display prototype in his own tweet. “I’m excited to see this preview of another headset prototype from the Display Systems Research team at FRL-R Redmond,” he wrote.

Update (October 14th, 2021): Bosworth shared another photo of a prototype headset which looks like an original Quest attached to a Rift S head-strap and with some extra hardware around it and what appears to be a rudimentary antenna sticking out of the top.

Image courtesy Andrew Bosworth

He didn’t offer any detail other than teasing, “So. Many. Prototypes.”

It doesn’t seem like coincidence that the Facebook teases happened today. Just yesterday, HTC’s own upcoming compact VR headset, Vive Flow, appears to have leaked ahead of the company’s event on Thursday. With pre-orders for Vive Flow purportedly starting on October 15th, the unspoken sentiment of Zuckerberg & Bosworth’s posts feel akin to ‘wait until you see what we’ve got up our sleeve’.

HTC likely got the message; the company’s President of Vive China quickly tweeted back to Bosworth, “Hey Boz, Nice looking research project. Want to trade for a production quality device hot out of our factory? 😎,” insinuating that HTC is ahead of the game by already having its compact headset ready for production.

With all this new XR hardware in the works, it’s certainly turning out to be an exciting month. Facebook will surely save its major announcements at Facebook Connect later this month and high-end headset maker Varjo has been teasing its own big announcement.

The post Facebook Execs Tease “Next-generation” Headset Prototypes Following Vive Flow Leak appeared first on Road to VR.

Andrew Bosworth Will Be Facebook’s New CTO in 2022

Andrew Bosworth

This week marks Facebook’s next big step towards its metaverse future as current Chief Technology Officer (CTO) Mike Schroepfer has announced he’ll be stepping down next year after 13 years at the company. Taking his place will be head of Facebook Reality Lab (FRL) Andrew ‘Boz’ Bosworth, continuing Facebook’s work in virtual reality (VR) and augmented reality (AR).

Mike Schroepfer
Image credit: Facebook/Mike Schroepfer

In a statement on his Facebook profile, Schroepfer said: “I have made the decision to step down as Chief Technology Officer and transition to a new part time role as Facebook’s first Senior Fellow at the company sometime in 2022.” This was a personal choice with him going on to say: “This change in role will allow me to dedicate more time to my family and my personal philanthropic efforts while staying deeply connected to the company working on key initiatives including recruiting and developing technical talent.”

This process will be very gradual by all accounts with Bosworth continuing to oversee FRL as he assumes more responsibilities at Facebook. “As our next CTO, Boz will continue leading Facebook Reality Labs and overseeing our work in augmented reality, virtual reality and more, and as part of this transition a few other groups will join Boz’s team as well. This is all foundational to our broader efforts helping to build the metaverse, and I’m excited about the future of this work under Boz’s leadership,” said CEO Mark Zuckerberg in a statement.

The announcement comes as Facebook continues to ramp up its hardware efforts across a number of product lines. This week saw new Portal devices revealed and a couple of weeks ago Facebook’s first entry into the smart glasses sector with Ray-Ban Stories, essentially camera-equipped sunglasses. While these weren’t AR glasses the company’s Project Aria is working towards that eventual solution.

Oculus Quest 2 top down

When it comes to that metaverse vision of Zuckerberg’s, the Oculus Quest platform is going to play an important role. Oculus Quest 2 recently saw and upgrade with the standard £299 model now sporting 128GB of internal memory. Facebook Connect is only a month away with biggest VR and AR news usually saved for the event. VRFocus is expecting either Bosworth or Zuckerberg to confirm new details regarding Oculus Quest Pro, a device both have teased will feature more sensors to make the experience more immersive.

As further details arise regarding Bosworth’s plan for his new role VRFocus will keep you updated.

Andrew Bosworth To Take Over As Facebook CTO In 2022

Facebook Vice President of AR/VR, Andrew ‘Boz’ Bosworth, will take over the role of Facebook Chief Technology Officer (CTO) after current CTO Mike Schroepfer announced plans to step down from the role sometime next year.

Schroepfer has been with Facebook for 13 years and after stepping away from his CTO role, he will move to a Senior Fellow role within the company. Here’s a copy of his full Twitter thread announcing his departure:

After 13 years at FB, I’ve decided to step down as CTO and pass the baton to @boztank at some point in 2022. I will stay on as long as it takes to ensure a successful leadership transition.

My new role as Senior Fellow will enable me to stay deeply connected to the company, working on key initiatives including recruiting and developing technical talent and fostering our AI investments in critical technologies like @PyTorch

It has been a privilege to lead our technology teams during a time of incredible growth & advancement. I am proud of what the team has achieved, from unleashing the benefits of AI & bringing VR to life to connecting more people around the world through technology.

I am still incredibly optimistic about the potential for AI and AR/VR to improve the lives of people every day. I am honored to be able to continue to support Facebook’s exciting future in my role as Senior Fellow.

Facebook VP of AR/VR Andrew Bosworth has taken on an increasingly public facing role over the last year or two, becoming a prominent spokesperson for Facebook’s VR/AR efforts. Bosworth has hosted Instagram AMAs (‘ask me anything’) with user-submitted questions on Facebook’s VR/AR efforts (his responses to which often flirt between informative, teasing and promotional all at once), moderated talks with Mark Zuckerberg and John Carmack on various VR and VR-adjacent topics, and dropped public hints and winks towards new products like Quest Pro.

In a statement, Facebook CEO Mark Zuckerberg said that as CTO, Bosworth will continue to lead Facebook Reality Labs:

As our next CTO, Boz will continue leading Facebook Reality Labs and overseeing our work in augmented reality, virtual reality and more, and as part of this transition a few other groups will join Boz’s team as well. This is all foundational to our broader efforts helping to build the metaverse, and I’m excited about the future of this work under Boz’s leadership.

Responding to Schroepfer’s thread on Twitter, Bosworth said, “Congratulations Schrep on an epic 13 year run. Lots to do still and I’m glad you’re sticking around to help.”

Facebook released Quest 2 near the end of 2020 and, last month, doubled the storage of the base $299 model to 128GB. The company also recently partnered to make Ray-Ban brand sunglasses that start at $299 and connect to your phone to answer calls or play music, or capture square stabilized 30 second videos and wide angle photos. As of the end of June 2021, Facebook employed more than 63,000 people with a significant percentage working on VR and AR technologies.

Facebook’s AR Research Aria Glasses Detailed In FCC Listing

An FCC listing for Facebook’s Aria Glasses, which are being used internally for AR research, provides new details via a user manual and various other testing and compliance documents.

The Aria Glasses were first announced at Facebook Connect last year. They are not a consumer product or a prototype for AR glasses, but are instead designed to be worn by Facebook employees as a research tool that will “help us collect data to uncover the underlying technical and ethical questions and start to look at answers to those,” as Facebook representatives explained when the project was announced. Aria Glasses do not feature a display and are intended to capture research data while the wearer goes about their day.

Project Aria “will capture the wearer’s video and audio, as well as their eye tracking and location information. The glasses’ on-device computing power will then be used to encrypt and store information that, when uploaded to separate, designated back-end storage space, will help our researchers figure out how AR can work in the real world,” Facebook explained. “Our future AR devices must be more perceptive in order to be more genuinely useful to us. In order for devices to understand where they are in relation to people and other objects, and how to make sense of any given situation, they need a virtual 3D map of the things around you. But it’s far too power-intensive to scan and reconstruct a space in real time from scratch, so AR glasses will need to tap into an existing 3D map we call LiveMaps. LiveMaps uses computer vision to construct a virtual representation of the parts of the world that are relevant to you. With these 3D maps, our future devices will be able to efficiently see, analyze, and understand the world around them and better serve those who use them. These devices will keep track of changes, like new street names, and update them in real-time. The Project Aria device is testing out how this can work in practice.”

Facebook also announced a partnership with Luxottica and Ray-Ban to produce “smart glasses”, which are unrelated to Aria and due for release later this year.

Now, we have a bit more information and our best look yet at Facebook’s Aria Glasses by way of documents within a FCC listing that was made public recently, as reported by Protocol’s Janko Roettgers.

The documents include a user manual, where the glasses are referred to as “Gemini EVT” and described as “prototype equipment … intended for limited distribution only for  purposes of testing and data collection.”

The manual also reveals that the glasses support prescription lenses, and that they do not fold down like a normal pair of glasses — the temples remain fixed at right angles from the main frame.

Project Aria AR Glasses Facebook

The user manual also includes photos labeling individual parts and external buttons. As you can see in the image embedded above, the glasses offer a mute switch, power button, proximity sensor and status LED.

When the mute switch is set to mute/privacy mode, red is visible on the LED indicator. The right hand side also features a capture button pictured below, but the document (at least at the time it was written) says it is only used to power on the device.

Project Aria AR Glasses Facebook

The glasses also come with a microfiber cloth, a power supply and a USB cord with a magnetic connector that attaches to the left temple of the glasses. In the manual, Facebook says this connector is “sensitive” and that users “may have to physically hold it together momentarily for the device to be enumerated and/or initiate charging.”

Project Aria AR Glasses Facebook

The glasses connect to an internal mobile companion app, called Ariane, and the footage collected from the cameras can be transferred and viewed on a computer as well.

Project Aria AR Glasses Facebook

While the Aria Glasses are not the consumer AR glasses many are expecting, they do give an insight into the research process Facebook is undertaking to move toward consumer AR. On the Project Aria site, Facebook is clear that the glasses are for research work only.

With Facebook Connect coming up in October, there’s a fair chance we might hear more about Project Aria and the resulting AR technology then. You can view the Aria FCC listing and all associated documents here.

Facebook’s ‘Reverse Passthrough’ VR Prototype Is Nightmare Fuel (Right Now)

Facebook Reality Labs will present new work on a ‘reverse passthrough’ VR concept at this year’s SIGGRAPH event which, honestly, looks quite terrifying in its current form.

A blog post from FRL’s research team posted this week showcases the concept. Reverse passthrough essentially shows a render of the VR user’s eyes on 3D displays at the front of the headset. The idea is to reduce the effect of VR users shutting off from people in the real world when they pull a device over their face. We’ve seen similar ideas before from companies like Google, who once showcased a means of showing the user’s face inside mixed reality capture. This project in particular was led by research scientist Nathan Matsuda.

Facebook Reverse Passthrough

Images in the blog post show very early versions of this work which, honestly, look quite frightening. But these are of course prototypes, not consumer products, meant more to display the idea in its raw form rather than present it as a consumer-ready concept.

According to the blog post, Matsuda started work on the concept in 2019, when he built a prototype Rift S headset with a 3D screen displayed on the front of the kit. The image displayed is of a 3D render of the engineer’s face, replicating eye movements with the help of eye-tracking cameras inside the headset.

Facebook Reverse Passthrough Rift S

In its current form, though, the prototype features “purpose-built optics, electronics, software, and a range of supporting technologies to capture and depict more realistic 3D faces.” The headset itself is a much smaller device than an Oculus Quest, looking comparable to Panasonic’s super slim VR glasses, but that’s negated by the fact it’s connected to a huge amount of wires and circuitry. Matsuda’s eyes, meanwhile, are still 3D renders and not the real thing. The displays themselves are made up of a pretty sophisticated stack, as shown in the below diagram.

headset desgin facebook research prototype

Here’s what it looks like in action which is, again, more than a little scary:

As with a lot of FRL’s work, though, this is a purely experimental prototype and definitely not something you should count on seeing in a consumer version anytime soon. Would you be interested in seeing the reverse passthrough concept in a consumer VR headset? Let us know in the comments below!

Facebook Researchers Show ‘Reverse Passthrough’ VR Prototype for Eye-contact Outside the Headset

Researchers at Facebook Reality Labs today published new work showcasing a prototype headset which has external displays to depict the user’s eyes to others outside of the headset. The goal is to allow eye-contact between the headset wearer and others in an effort to make it less awkward while wearing a headset and communicating with someone in the same room.

One of my favorite things to do when demoing an Oculus Quest to someone for the first time is to put on the headset, activate it’s ‘passthrough view’ (which lets me see the world outside of the headset), and then walk up and shake their hand to clearly reveal that I can see them. Because Quest’s cameras are at the four corners of the visor, it’s not easy to imagine that there would be any way for the user to see ‘through’ the headset, so the result from the outside seems a bit magical. Aftward I put the headset on the person and let them see what I could see from inside!

But this fun little demo reveals a problem too. Even though it’s easy for the person in the headset to see people outside of the headset, it isn’t clear to people outside of the headset when the person in the headset is actually looking at them (rather than looking at an entirely different virtual world.

Eye-contact is clearly a huge factor in face-to-face communication; it helps us gauge if someone is paying attention to the conversation, how they’re feeling about it, and even if they have something to say, want to change the topic, or leave the conversation entirely. Trying to talk to someone whose eyes you can’t see is uncomfortable and awkward, specifically because it robs us of our ingrained ability to detect this kind of intent.

But as VR headsets become thinner and more comfortable—and it becomes easier to use passthrough to have a conversation with someone nearby than taking the headset off entirely—this will become a growing issue.

Researchers at Facebook Reality Labs have come up with a high-tech fix to the problem. Making use of light-field displays mounted on the outside of a VR headset, the so called ‘reverse passthrough’ prototype system aims to show a representation of the user’s eyes that’s both depth and direction accurate.

Image courtesy Facebook Reality Lab

In a paper published this week for SIGGRAPH 2021, Facebook Reality Labs researchers Nathan Matsuda, Joel Hegland, and Douglas Lanman, detailed the system. While to external observers it appears that the headset is very thick but transparent enough to see their eyes, the apparent depth is an illusion created by a light-field display on the outside of the headset.

If it was instead a typical display, the user’s eyes would appear to float far away from their face, making for perhaps a more uncomfortable image than not being able to see them at all! Below researcher Nathan Matsuda shows the system without any eyes (left), with eyes but no depth (middle), and with eyes and depth (right).

The light-field display (in this case a display which uses a microlens array), allows multiple observers to see the correct depth cues no matter which angle they’re standing at.

What the observers see isn’t a real image of the user’s eyes however. Instead, eye-tracking data is applied to a 3D model of the user’s face, which means this technique would be limited by how realistic the model is and how easy it is to acquire for each individual.

Of course, Facebook has been doing some really impressive work on that front too with their Codec Avatars project. The researchers mocked up an example of a Codec Avatar being used for the reverse passthrough function (above), which looks even better, but resolution is clearly still a limiting factor—something the researchers believe will be overcome in due time.

Facebook Reality Labs Chief Scientist, Michael Abrash admits he didn’t think there was much merit to the idea of reverse passthrough until the researchers further proved out the concept.

“My first reaction was that it was kind of a goofy idea, a novelty at best,” Abrash said in a post about the work. “But I don’t tell researchers what to do, because you don’t get innovation without freedom to try new things, and that’s a good thing, because now it’s clearly a unique idea with genuine promise.”

– – — – –

It might seem like a whole lot of work and extra hardware to solve a problem that isn’t really a problem if you just decided to use an AR headset in the first place. After all, most AR headsets are built with transparent optics from the outset, and being able to see the eyes of the user is a major benefit when it comes to interfacing with other people while wearing the device.

But even then, AR headsets can suffer from ‘eye-glow’ which obstructs the view of the eye from the outside, sometimes severely, depending upon the optics and the angle of the viewer.

Image courtesy DigiLens

AR headsets also have other limitations that aren’t an issue on VR headsets, like a limited field-of-view and a lack of complete opacity control. Depending upon the use-case, a thin and light future VR headset with a very convincing reverse passthrough system could be preferable to an AR headset with transparent optics.

The post Facebook Researchers Show ‘Reverse Passthrough’ VR Prototype for Eye-contact Outside the Headset appeared first on Road to VR.