Facebook is rolling out its latest avatars, Oculus Avatars 2.0, which it hopes will finally unify virtual identify across Oculus applications and Facebook products. In addition to becoming the default avatars for Facebook Horizon and being optionally integrated into Oculus games, Facebook also teased that the new avatars will eventually find their way to the Facebook app, Messenger, Instagram, and more.
Given that a VR headset and controllers only tell the computer the location of your head and hands, it’s quite difficult to accurately predict where the rest of your body (like your elbows and shoulders) should be shown in the virtual world. Facebook says it has leaned on new methods based on machine-learning to create more believable and expressive avatar movements based solely on the movement of your head and hands.
Facebook says Avatars 2.0 offers far more customization than the previous system, now allowing users to individually customize skin tone, hair style, face shape/markings/lines, eye shape, eyebrows, eye makeup, and more. Combined with other options like clothing, glasses, and body types, the company says there’s more than one quintillion possible combinations of avatars.
While anyone running the latest version of the Oculus Quest software should be able to customize an avatar with the new system, it’ll be some time yet before they can be widely integrated into Oculus applications; Facebook is only offering the new system to select developers for the time being, with open access to the Avatars 2.0 SDK planned for “later this year.”
Three applications now support the Oculus Avatars 2.0 right out of the gate: Epic Roller Coasters, PokerStars VR, and Topgolf with Pro Putt. Facebook says that the new avatars will also soon show up in Synth Riders and ForeVR Bowling.
As for Facebook’s own VR products, the company says that later this year Oculus Avatars 2.0 will replace the avatars in Facebook Horizon, the company’s social VR app which remains in closed beta. Outside of VR, the company has also teased that Avatars 2.0 will come—in some form—to the Facebook app, Messenger, Instagram, and other, unspecified, Facebook products.
Facebook plans to improve its Horizon & Venues avatars and roll them into a new Avatars SDK to replace Oculus Avatars.
There’s no timeline on when that will happen. For now, Facebook is just showing a single image preview of this new avatar system:
It’s apparently an evolution of the current Facebook VR avatars used in Facebook Horizon and the beta for the new Venues. Staffers working on the project include former Pixar animators.
That means you’ll have a consistent virtual avatar across built-in Facebook social apps and any 3rd party apps using the SDK. Platforms like Bigscreen, VRChat, and Rec Room use their own separate avatar systems.
At UploadVR we use the current Oculus Avatars for our weekly from-VR podcast, as well as virtual interviews. Oculus Avatars was first introduced with a highly abstract monotone style following a “don’t show what you aren’t tracking” philosophy- realistic human appearance without realistic human motion can creep your brain out.
A current Oculus Avatar
With the release of Oculus Go in 2018 Facebook brought in color, and last year the avatars got a huge update with neural network estimated lip movement, simulated object-based eye gazing, and microexpressions. Some of this core technology appears to have transferred over to Horizon, though in our experience the lips don’t move quite as authentically yet.
Facebook said you’ll be able to use these avatars in its non-VR smartphone apps one day too, presumably including Messenger, which is coming to Quest.
There was a lot to take in from this week’s Facebook Developers Conference (F8) 2019, not just the official rollout of pre-orders and launch dates for Oculus Quest and Oculus Rift S. The company showcased some of the work it was doing across a range of challenges it was trying to tackle, one of them being social interaction in virtual reality (VR). During the second day’s keynote, Facebook unveiled its latest Oculus Avatars prototype which aims to bring someones entire body into VR.
The work is being done by Facebook Reality Labs (FRL) with the goal to be able to fully represent a user from head to toe in VR, making social interactions even more seamless and natural.
FRL’s Ronald Mallet began by discussing a project the team revealed a couple of months ago, Codec Avatars, which was about making avatars as photorealistic as possible. While certainly a massive leap from the current avatars available today, a massive amount of equipment was required to scan a user’s features, plus it only worked from the shoulders up.
Mallet noted that while plenty of information can be glean from someones facial expressions, the same can work for the entire body; where full motion can tell you about emotion, agreement, trust and empathy. The needed to create a body model that represented the human anatomy but which needed to be fully adaptive to match any individual automatically.
To build the model FRL started from the inside out, designing an anatomically correct skeleton with proper articulations, from the way the shoulders roll to joint movement. Then they matched the skeleton to a users body structure, which accurately followed the team members movements, all from a single sensor – Mallet didn’t specify which sensor if it was a normal Oculus one.
Of course, we can’t all be skeletons in VR so muscles were added to the model, helping to predict how they move and contract. Finally, skin and clothes were added as faithfully as possible with a physics-based model for cloth deformations, with the final prototype seen in the above image.
There’s still a way to go before this tech will be consumer ready, it does look very promising. For now, Oculus Rift users will have to be happy with the Expressive Avatars update. As further details are released, VRFocus will let you know.
Digital avatars are an important part of the virtual reality (VR) ecosystem, allowing users to chat and meet each other almost like they would in real life. It’s why Oculus Avatars were launched back in 2016 as a means of making social interaction more realistic. Today, that’s been taken a step further with a new update adding ‘Expressive Avatars’ as well as new customisation options.
Oculus Avatars have come a long way since 2016, with these new versions now able to simulate eye and mouth movement for that enhanced realism. For example, eye’s are highly important when talking to someone as they can display a whole range of emotions, turning a lifeless face into an expressive one. Up until now, Oculus has got around this by adding updates with eyewear, but with this update, the company aims to simulate a response.
Oculus’ headset’s don’t feature any eye tracking cameras inside so the Oculus Avatar team developed algorithms to accurately portray eye movement. Sophisticated models were built that allowed the team to codify these models of behaviour for VR, turning them into reactions that would seem as lifelike as possible.
It’s not just the eyes Oculus wanted to improve, lips are equally important when talking to another human being. So Oculus improved its lipsync technology to add even greater expression to a conversation. Before lips tended to just twitch to a users voice, now they behave and move in a more natural fashion.
Alongside the difficult job of trying to simulate emotions, Oculus has given users even more creative freedom over how their avatar looks. Enhanced personalization can now be achieved through new skin tones, lip and eyebrow colour, hairstyles, and clothing so that spending time with the growing VR community can be almost the same as real life.
Oculus Avatars are supported on Oculus Rift and Oculus Go, with the update going live today. The company does note on its blog that: “Over the coming days and weeks, a host of third party apps and games including Poker Stars VR, Tribe XR DJ, and Epic Rollercoaster will also update to support the new avatars”
So the face is almost sorted, now Oculus has to move from lifelike busts to full body avatars. When that happens, VRFocus will let you know.
Oculus is today launching their ‘Expressive Avatars’ update on Rift and mobile VR; it’s a significant step up in realism thanks to a few additions including cleverly simulated eye models, lipsync, and even microexpressions. If you’re also hurting for more hairstyles, clothes, and accessories, you might want to pop into your headset at some point today to check it all out once the update is live.
First unveiled at OC5 last year, the public release of Oculus’ avatar overhaul launches today which also includes an update to the Avatar Editor on PC and mobile. The new editor includes a range of new customization options such as lipstick, eye color, brow and lash colors, new hair, clothing and eye wear options.
Just like Oculus’ previous avatar system, third-party apps and games will also support the update. Over the course of a few days Oculus tells us games such as Poker Stars VR, Tribe XR DJ, and Epic Rollercoaster will all include support. While not stated specifically, it’s clear the company is hoping to appeal to more third-party developers with ‘Expressive Avatars’, as many games on the platform make use of their own avatar systems.
Oculus is set to release their blogpost that officially announces Expressive Avatars at some point today.
Express Yourself
Oculus first released their first version of Oculus Avatars in 2016, and while the company has since given users the chance to customize their persistent digital likenesses with a variety of textures, clothing, and hairstyles, without eye or face tracking avatars were essentially inarticulate masks that made the user rely upon motion controls and body language to transmit emotion.
Oculus previously used eye wear to avoid off-putting stares, Image courtesy Oculus
This was due to the fact that no Oculus devices actually feature face or eye-tracking, which would naturally give avatars a greater avenue for 1:1 user expression. And with the impending release of Oculus Quest and Rift S, that’s still going to be the case, as neither headset offers these things. Hardware notwithstanding, Oculus has been hacking away at just what they can get away with in order to better simulate realistic-looking eye movement, blinking, facial movements, lipsyncing—all of it in the name of making avatars more human.
“We’ve made a big step forward with this update,” says Oculus Avatars product manager Mike Howard. “Bringing together expertise in art, machine learning and behavioural modeling, the Oculus Avatar team developed algorithms to accurately simulate how people talk to, and look at, objects and other people—all without any cameras to track eye or face movement. The Oculus Avatar team were able to codify these models of behavior for VR, and then had the fun job of tuning them to make interactions feel more lifelike.”
Keeping It Real
Oculus’ Mike Howard penned a deep-dive article on the past, present and future of Oculus Avatars, which tells us a little more about what sort of challenges the company faced in creating not only more realistic avatars with its current hardware in mind—limited by the lack of on-board biometric tracking and user’s computers/mobile headsets—but doing it well within the bounds of the uncanny valley.
That’s something you can’t afford to brush up against if you want users to invest both the time into creating their digital likenesses and interacting with others, Howard maintains.
“In VR, when you see an avatar moving in a realistic and very human way, your mind begins to analyze it, and you see what’s wrong. You could almost think of this as an evolutionary defense mechanism. We should be wary of things that move like humans but don’t behave like us,” he says.
Making an avatar that simply moves its mouth when you talk and blink at regular intervals wasn’t enough for Oculus. The system needed to be tailor-made to infer when a user might conceivably blink, and make the best possible guess at how a user’s mouth should move when forming words. That last part is a particularly tough equation, as humans move their mouths before, during, and after producing a word, leaving the predictive capabilities with a hard ceiling of accuracy. More on that in a bit.
As for eyeballs, the realization that VR headset users typically only move their eyes about 10 degrees off-center, and use their head to accommodate the rest of the way to look at any given object, made it “easier to predict where someone was looking based on head direction and the objects or people in front of them in a virtual environment, giving us more confidence in being able to simulate compelling eye behaviors,” Howard maintains.
The system is said to simulate blinking, and a host of eye kinematics such as gaze shifting, saccades (rotating the eye rapidly usually during focal change), micro-saccades, and smoothly tracking objects with the eye.
And for simulated lip movements, the team discovered they could model intermediate mouth shapes between each sound and the following sound by controlling how quickly individual (virtual) mouth muscles could move, something the team dubs ‘differential interpolation’.
To boot, the team has also included micro-expressions to keep faces looking natural during speech and rest, although they’re clearly staying away from actual implied expressions like extremely happy, sad, angry, etc. An avatar looking bored or disgusted during a lively chat could cross wires socially.
What Oculus Avatars *won’t* do, Image courtesy Oculus
In the end, Howard makes it clear that more realistic-looking avatars are technically in the purview of current head and hand tracking hardware, although compute power across all platforms puts a hard barrier on the sort of skin and hair that can be simulated. Frankly put: a more detailed skin texture means you have to model that skin to look natural as it stretches over your face. Having more detailed skin also necessitates equally detailed hair to match.
“Given our learning to date, we determined that we would use a more sculpturally accurate form, but we’d also use texture and shading to pull it back from being too realistic, in order to match the behavioral fidelity that we were increasingly confident we could simulate,” Howard explains. “Our goal was to create something that was human enough that you’d read into the physiological traits and face behaviors we wanted to exemplify, but not so much that you’d fixate on the way that the skin should wrinkle and stretch, or the behavior of hair (which is incredibly difficult to simulate).”
There’s still plenty left to do. Oculus Avatars aren’t seamlessly available in all games on either the Oculus platform or Steam, requiring developers to integrate on a case-by-case basis. Not to be missed: they’re still basically floating torsos and hands at the moment. To that tune, the company is working on inverse kinematic models to make full body avatars a possibility.
If you want to read more about the history and possible future of Oculus Avatars, check out Howard’s deep-dive when it goes live later today.
Update (12:15 ET): In a previous version of this article, it was stated that Oculus Avatars aren’t cross-platform, however this isn’t accurate. Oculus made a cross-plaform option available to developers last year, although this integration must be done on a game-by-game basis. Developers can choose to use default Oculus avatars or allow unique Oculus platform user avatars in their game, although it’s far from the seamless integration that the word ‘cross-platform’ might imply. We’ve updated the offending bit to better reflect this distinction.
The long awaited ‘Expressive Avatars’ update to Oculus Avatars is launching in closed beta ‘shortly’. The update adds simulated eyes and overhauls the lipsync technology.
Oculus Avatars are available for free for any VR developer to use in their app. This new update was first announced back at Oculus Connect 5. It was described at the time as coming “later this year”, but that target seems to have slipped.
The most noticeable addition in the update is eyes. Currently Oculus Avatars avoid eyes by requiring opaque eyewear such as sunglasses or a virtual VR headset. The new eye simulation is based on real human behaviors such as “micro-saccades, smooth pursuit and ballistic gaze shifting”. Developers will be able to specify objects for the eyes to look at.
Current Oculus Avatars use eyewear to avoid eyes
The lip sync technology has also been overhauled. Previously the lip simply shifted like a wave in response to the volume of the user’s voice. It now uses a machine learning based technology which recreates real lip movements. The new renderer features “differential blending between individual muscle movements” and “facial micro-expressions”.
New avatar customization options are being added in the editor. Users will now be able to change eye, lip, lash, and brow color.
Oculus Avatars went cross-platform last year meaning other platforms will be able to experience these new features too. However currently only Oculus users can customize their avatar- other platforms must select from a predetermined list.
Expressive Avatars is launching as a developer-only closed beta for now. The last major update to Avatars was released alongside Oculus Go, so it seems possible this next update will release alongside Oculus Quest. If Facebook can expand cross-platform support and entice more developers, Oculus Avatars could one day become the standard for VR.
Oculus Go is a great headset for virtual reality (VR) beginners. Easy to use there’s not much that can go wrong. Sometimes though everyone needs a hand, which is fine if you’re on Oculus Rift looking at a monitor, but far trickier with the standalone headset. During Oculus Connect 5 (OC5) the company announced a new Casting feature to enable non-VR players to see what the person in VR was seeing via a mobile device. Today, alongside several other updates, the beta version of casting on Oculus Go is being rolled out.
Casting from VR should be fairly simple by the sounds of it, with VR users heading to the Oculus Go menu to select the ‘Cast’ option. This will send a push notification to your companion device. So long as both the headset and the phone or tablet are on the same WiFi network it should work.
In addition to the Casting update Oculus Avatars are getting into the spooky Halloween season allowing players to dress up as a witch, vampire, mummy or skeleton. The Avatar Editor supports Oculus Go, Samsung Gear VR, and Oculus Rift.
Lastly, the Oculus update includes an important safety feature. Users now have the ability to report abusive content or behavior from inside any VR app or videogame. You can capture video of your experience to provide context to your report, which is then reviewed by our Community Operations team.
This past weekend saw Oculus Venues begin its lineup of live NBA games including the Phoenix Suns playing Golden State Warriors. The next game is on 11th November between the Boston Celtics and the Portland Trailblazers. Don’t forget that everyone who attends an NBA game in Venues will unlock an NBA jersey from one of the teams they watched for their Oculus Avatar.
VRFocus will continue its coverage of Oculus Go and Oculus Venues, reporting back with further updates.
Oculus had announced almost a year ago now that they planned to take their Oculus Avatars system cross-platform. Now, with last week’s release of Oculus Avatar SDK 1.28, the package is finally ready to be deployed cross-platform, meaning that Oculus games also offered on other platforms can use a unified avatar system.
Oculus invested in building a platform-level avatar system to allow users to create a consistent representation of themselves that could be used across various games in the Oculus ecosystem while saving developers from needing to build their own avatar system for each game.
Unfortunately, while they are arguably some of the best looking avatars out there, Oculus Avatars has seen minimal adoption, due in part to the fact that it has been restricted to titles on the Oculus platform. Since cross-platform distribution is critical for developers at this stage in the VR market, implementing Oculus Avatars into apps which would ultimately be distributed on other platforms like SteamVR would mean extra work, as the developer would need to implement Oculus Avatars for the Oculus version of the app, and a separate avatar system for the non-Oculus version of the app.
With the release of the Oculus Avatar SDK 1.28, developers can finally build Oculus Avatars into their apps for cross-platform use, meaning apps can rely on a unified avatar system whether they’re distributed through the Oculus Store, SteamVR, or elsewhere. This also makes things potentially easier for apps which have cross-play (multiplayer between platforms). Oculus says cross-platform support for their Avatars system has been “one of [developers’] biggest requests.”
Even with the move to cross-platform support, Oculus Avatars themselves are configured at the platform level and tied to an Oculus user account; players do their avatar customizations in Oculus Home rather than inside individual apps. That means that non-Oculus users can’t make their own custom avatars, and will instead be stuck choosing from pre-configured avatars offered through each app.
Oculus says they’re “in the early stages of building out richer avatar support for non-Oculus users,” so perhaps in the future they will also make avatar customization cross-platform (let’s just hope it doesn’t take another year). The company also says they’re continuing to invest in the Oculus Avatars system, including working toward “developer-created avatar content” for game-specific avatar customization.
How you are represented in the virtual environment is a topic that is pretty dear to the hearts of many virtual reality (VR) users. It is especially important in a social setting, where it can form a first impression in social VR scenarios. Oculus Avatars were originally rolled out for the Oculus Rift before being ported to the Samsung Gear VR, but now their reach is extending even further.
Oculus have announced in a blog post that Oculus Avatars will be going cross-platform, allowing a user’s Oculus Avatar to follow them to any platform.
With the release of SDK 1.28, support is being rolled out for cross-platform Avatars on PC, with an included Unity sample and an updated SDL license. Oculus says this will allow developers to implement a unified avatar system with multiplayer experiences while reducing both costs and development times.
The Oculus Avatar app has been prepared to work right out of the box, and the SDK has gone through updates so it can support querying the user’s avatar from any platform.
This means that apps running on the HTC Vive or Windows Mixed Reality devices will also be able to use Oculus Avatars, giving users the chance to have a unified virtual identity across multiple videogames and applications that use the SDK, rather than having to create a new character each time.
According to the blog post, this initial release allows the Avatar for any Oculus user to be queries from any app that uses the SDK. The Oculus blog post also says: “We’re in the early stages of building out richer avatar support for non-Oculus users, but to provide some choices for non-Oculus users we’ve created 12 default avatars (included in the Unity sample) that can be provided as options to anyone without an existing avatar.”
According to Oculus, this is just the first step in supporting developers across multiple platforms. As always, VRFocuswill be there to bring you the latest developments in this area.
While the weeks after the Electronic Entertainment Expo (E3) 2018 tend to be a little slower when it comes to videogame news, Oculus has been fairly busy this week with several updates to its core platform, sculpting software Oculus Medium and Oculus for Business. To round the week of Oculus has unveiled its July roster of updates rolling out today, adding more social features, customisation options and more.
The updates are being pushed out through Oculus’ Public Test Channel (PTC) which means they’re not official just yet, and only available to those who’ve signed up to the channel.
First up Rift Core 2.0 – which launched in December 2017 – is getting social so users can hang out with friends inside Oculus Home. Explaining on the Oculus Blog the company states: “This initial release has basic functionality and will help us test the multiplayer infrastructure. Over the next several months, we’ll add more features and depth to make Home a great place to spend time with friends regardless of physical distance—and the perfect jumping off point to explore social VR content together.”
Other features being rolled out today that you can start testing include Group Hang-Outs where up to eight people can chill in the same Home, Broadcasting Oculus Desktop which adds experimental support for broadcasting embedded Dash panels and upgraded Oculus Avatars.
The avatar customisation has been improved with new hair, skin shading, clothing, and eyewear design options. Oculus Avatars don’t just appear in Oculus Home but in several videogames as well, including Brass Tactics, Epic Rollercoasters, REFLEX UNIT, SportsBar VR, and Drop Dead. Additionally, Oculus has added an ‘Avatar Editor’ mirror in the Special Items section of the inventory that can be placed anywhere in the Home space.
Fine tuning the platform even further beta language support has been added for Danish, Dutch, Finnish, Italian, Norwegian, Polish, Spanish (Latin America), Spanish (Spain), and Swedish. Plus, for those who don’t like teleporting new movement options like Walk Mode and Smooth Turning are now available. As Oculus continues to improve its platform, VRFocus will keep you informed.
