Category: AR hardware

Posted on

Facebook Cancels Head-Mounted BCI Research, Will Focus On Wrist

Facebook Reality Labs is sunsetting its research toward head-mounted thought reading, instead focusing on a wrist-based device.

First vaguely revealed at F8 2017, the project’s stated goal was to “create a system capable of typing 100 words-per-minute straight from your brain” without requiring implanted electrodes or other physically invasive techniques. Such a device could act as input for AR or VR.

This BCI (Brain Computer Interface) work initially happened in Facebook’s ‘Building 8’ advanced research division, run by former director of DARPA Regina Dugan. When Building 8 was disbanded in late 2018 the project moved to Facebook Reality Labs.

In July 2019 Facebook went into more detail in a technical blog post, explaining the technology as near-infrared light imaging used to sense “shifts in oxygen levels within the brain” caused by neurons consuming oxygen when active- an indirect measure of brain activity. “A decade from now, the ability to type directly from our brains may be accepted as a given,” the 2019 blog post stated. “Not long ago, it sounded like science fiction. Now, it feels within plausible reach.”

In a blog post this week, Facebook says it “reevaluated” its objectives for BCI research. It is no longer working on a head-mounted optical device for reading speech from thoughts. Instead it will focus on the wrist-based device it showed off earlier this year, which it says has “a nearer-term path to market”.

The wrist device uses EMG (electromyography) to sense the neural signals passing through your arm to your hand & fingers, replacing the need for optical hand tracking – and avoiding the limitations of cameras. Facebook claims this high bandwidth input will be “highly reliable, subtle, personalizable, and adaptable to many situations”. FRL Research Director Sean Keller goes so far as to suggest “it will be the core input for AR glasses.”

Facebook says it “still believes in the long-term potential of head-mounted optical BCI technologies”, even though it isn’t working on it. The company is making its BCI software open source and plans to “share head-mounted hardware prototypes with key researchers and other peers to help advance new use cases, such as assistive technologies”.

Posted on

Lynx R1: XR2 AR-VR Hybrid Ships 2022 For ‘A Few Hundred Dollars’

French startup Lynx is shifting strategy for its upcoming AR-VR hybrid headset, dramatically lowering the price to target consumers too.

Lynx R1 was initially announced in February 2020 as a $1500 product focused on businesses & professionals. It has the same Snapdragon XR2 processor found in Oculus Quest 2 & HTC’s Vive Focus 3. But whereas those headsets can only show a low resolution black & white view of the real world, Lynx R1 has two dedicated high resolution color cameras for passthrough AR.

https://www.youtube.com/watch?v=QGRybx3wFso&t=2s

This week, CEO Stan Larroque announced a radical shift in his company’s approach. The previous lenses needed expensive eye tracking sensors to account for a relatively small eyebox, but a new lens design has a much wider eyebox.

Since eye tracking is no longer needed for the lenses, Lynx R1 is ditching the feature. Larroque says this enables a price of “just a few hundred dollars” for consumers. The other specifications – including dual 1600×1600 LCD panels running at 90Hz with independent lens separation adjustment – remain unchanged.

Larroque expects headset sales to consumers will generate “very little” profit, but higher-priced sales to businesses and fees from a planned app store will subsidize this. He suggests the pivot to consumer is “a bet”, partly motivated by a need for competition in the VR space.

“On the privacy side of things, on the open ecosystem, I don’t see a lot of good things happening. I don’t want to live in that future where VR is Facebook Reality”

The new design also adds two IR cameras to the front of the headset for Ultraleap hand tracking, the device’s primary input method. Lynx R1 won’t ship with tracked controllers, but it will support the upcoming FinchShift controllers – announced at $249 in 2019. Larroque says Lynx R1 might build its own tracked controllers in future “depending on how the Kickstarter goes”.

Like Oculus Quests, Lynx R1 runs a modified version of Android and will have a boundary safety system similar to Facebook’s Guardian. Lynx has its own SDK, but also plans to support OpenXR content like Quest.

The app store on Lynx R1 will offer VR apps, AR apps, and even apps that can switch between each at will. Lynx plans to show off games running on the headset in the coming weeks. Larroque is already in talks with several Oculus Quest developers keen to port their titles – though given the front-oriented camera positioning Lynx R1 might not be able to support all games.

In VR mode, a separate facial interface can be attached to block out peripheral light.

At launch Lynx R1 will support SteamVR via USB-C cable to a gaming PC. A future update will add wireless SteamVR streaming. Virtual Desktop developer Guy Godin says he looks forward to supporting other OpenXR headsets.

Larroque acknowledges the enormous challenges involved of shipping consumer tech hardware at scale. He admits he initially underestimated the time needed to develop the product, but almost three years later, with the help of partners like Qualcomm, he’s confident his team can deliver.

“We have a mission to bring a product that people will love, and not be afraid to put on their head.”

Lynx plans to launch a 30 day Kickstarter campaign in September. Larroque expects the product will ship in February 2022. As with all hardware startups, this should be taken as a goal, not a promise.

UploadVR plans to get hands-on with Lynx R1 in the coming months to bring you our impressions of this exciting new device.

Posted on

Snap Reveals Narrow-FoV AR Spectacles For Creators With 30 Minute Battery

Snap just announced new Spectacles with AR capabilities, which it plans to ship to eligible AR experience creators this year.

Snap inc has shipped three Spectacles branded glasses products to date. The previous models didn’t have any form of display system – they functioned solely as wearable cameras, exporting captured clips to the Snapchat app.

The new reveal isn’t a total surprise- The Information reported on Snap’s plan back in March, though we didn’t know the details until now.

The new Spectacles have dual waveguide displays with a diagonal field of view of just 26.3 degrees, an impressive 2000 nits of brightness, and end-to-end latency of less than 15 milliseconds so virtual objects appear solid in place.

That’s a much narrower field of view than AR headsets like HoloLens 2 (52° diagonal) & Magic Leap One (50° diagonal) – and those already feel narrow. That said, at 2000 nits Spectacles are 4x brigher than HL2 and 10x brighter than Magic Leap, making them suitable for use outdoors.

Qualcomm’s XR1 chip is used with two RGB cameras, one on each side, to perform spatial tracking of the world as well as hand tracking and video capture.

They weigh 134 grams – around three pairs of regular specs – with a battery life of around 30 minutes. A touchpad on the side can be used for precise input, and there are dedicated area scan & clip capture buttons too.

Spectacles aren’t a general purpose computer, and don’t run “apps”. Instead they integrate directly with Snap’s existing AR platform, which has hundreds of millions of active users. Creators can build ‘Lenses’ for Spectacles & the Snapchat app with Lens Studio.

This isn’t a product you can buy, at least not yet, but you can apply for Spectacles as a prospective creator on Snapchat’s website.

Posted on

DigiLens Design v1: A Reference For Modern Standalone AR Glasses

DigiLens Design v1 is a modular reference design for standalone AR glasses, powered by the same XR2 chip used in Oculus Quest 2.

DigiLens has been around for decades, specializing in holographic waveguides as a supplier in the smartglasses market. Until recently, this market was mainly for militaries and niche business applications.

Design v1 Exploded View

As you can guess, Design v1 isn’t a product you can buy, it’s a modular solution available to prospective DigiLens partners including “software vendors and IoT programmers”. It’s essentially a fully functional showcase of the company’s latest ‘Crystal50’ waveguide, which it describes as industry leading, as well as a developer kit for standalone AR.

Crystal50’s efficiency enables a HoloLens 2 class field of view in glasses-like form factor, with less darkening of real world light, and four times less “eye glow” so other people can see your eyes.

Design v1 is a standalone AR computer like HoloLens & Magic Leap – though its processor has smartphone level clock rates rather than laptop level.

Qualcomm offers two mobile chip tiers for AR/VR; XR1 and XR2. XR1 provides baseline functionality like tracking for headsets that tether to a phone or PC, whereas XR2 is a full-fledged flagship mobile system-on-chip (SoC) for standalone headsets.

Back in February Qualcomm itself showed off a reference design for an XR1-powered AR viewer which connects via USB-C to high end Android smartphones.

DigiLens is using the high end XR2 chip, the same used in VR headsets like as Quest 2 & Vive Focus. It’s running standard Android with DigiLens’ Visualize Framework for developers to build on.

Design v1 also packs in 6GB RAM, 128GB storage, two monochrome side cameras for tracking, an 8MP color center camera for capture, speakers for spatial audio, multiple microphones, , WiFi, Bluetooth, USB-C, and batteries beside each temple.

The architecture is modular: the included Coretronic 720p projectors can be detached to be swapped out with alternatives or future versions.

Facebook and Apple are working on AR glasses that are still years away and a key Microsoft leader recently reinforced his company’s interest in the consumer AR market. Still, there are still huge hurdles to overcome in terms of field of view and battery consumption needed to appeal to the consumer market more broadly. This reference design from Digilens is one step on that path, enabling an experience similar to HoloLens & Magic Leap in a cheaper, more compact form factor.

Posted on

Microsoft ‘Absolutely’ Working On Consumer HoloLens – ‘It’s A Very Important Part Of Our Strategy’

Microsoft reconfirmed it is “absolutely” working on a “consumer journey” for its HoloLens AR headset, even if that’s some ways off.

Speaking to Wall Street Journal, Technical Fellow Alex Kipman touched on the future of the platform. “But you don’t get to lead a new medium of computing if you’re not going to be in consumer,” Kipman said. “So we are absolutely working on a consumer journey for HoloLens. I’m happy to confirm that and say that is a very important part of our strategy.”

Consumer HoloLens Coming

Kipman didn’t give any kind of roadmap for an eventual launch of a consumer HoloLens, but it’s likely not going to happen for a while. The HoloLens 2, which we reviewed earlier this year, is an excellent but expensive ($3,500) AR headset with a lot of features that don’t quite measure up to consumer expectations. Improving these features whilst also bringing the kit’s price down is a challenge not just Microsoft but every hopeful AR competitor faces in the coming years.

Magic Leap learned the hard way when it pivoted its AR glasses away from consumer and towards enterprise, while rumors and speculation around Apple’s AR headset have persisted for years, most recently suggesting it might not arrive until 2025. Even HoloLens itself was first introduced with AR videogames and entertainment apps before Microsoft truly positioned it towards businesses.

As for HoloLens 3, Kipman also assured Microsoft was working on it and that it had to be “another huge leap forward”. Again, no word on when we might see that device.

Would you buy a consumer HoloLens headset? Let us know in the comments below!

Posted on

FRL Still Hopes To Give Devs Quest Passthrough Access This Year

Facebook Reality Labs is still aiming to let Oculus Quest developers work with the headset’s passthrough camera system at some point this year.

News comes by way of the most recent Instagram AMA session with FRL VP, Andrew ‘Boz’ Bosworth. Asked when developers might get access to passthrough support so that they might make augmented reality content, Bosworth replied that he didn’t have a “firm date” for the feature just yet.

“We are working on it,” he explained, “it’s gonna come out– I’m hoping it’s gonna come out this year. It’s a little harder than it seems and obviously safety is caught up in it from a standpoint of keeping the user safe from a Guardian perspective. So there’s a few things we’ve got to work through still on that but we’re hoping for this year.”

Currently Quest’s passthrough feature is used by Facebook alone so that users can set up play areas with their Guardian system, or use the feature in the Quest’s menu, swapping out the virtual home environment. Last year, we reported that the company is aiming to offer a Passthrough+ API to developers to let them work with the feature too.

Quest’s spatial tracking would make it possible for some AR capabilities on the headset. It’s a little like what’s seen with Microsoft’s HoloLens and Magic Leap only, instead of see-through lenses that project virtual images into the real world, you’d be observing the world as captured through the cameras. This allows for a wider field of view than on those other devices, but Quest’s passthrough function is currently only shown in black and white – there’s no color option.

As for the mention of safety, Bosworth is likely referring to the issue of passthrough potentially revealing a user’s private and personal data to developers.

Would you try AR games and features through Quest’s passthrough feature? Let us know in the comments below!

Posted on

AR Is ‘Critically Important’ To Apple’s Future, Confirms Tim Cook

Apple CEO Tim Cook has confirmed that augmented reality is a “critically important part” of the company’s future.

Cook acknowledged as much in a recent interview with Kara Swisher on the Sway podcast. While he declined to talk about Apple’s specific plans for the technology, he did talk a little bit about its promise.

“But in terms of AR, the promise of AR is that you and I are having a great conversation right now,” Cook began. “Arguably, it could even be better if we were able to augment our discussion with charts or other things to appear. And your audience would also benefit from this, too, I think. And so when I think about that in different fields, whether it’s health, whether it’s education, whether it’s gaming, whether it’s retail, I’m already seeing AR take off in some of these areas with use of the phone. And I think the promise is even greater in the future.”

Asked if AR was a “critically important part of Apple’s future”, Cook simply replied: “It is.”

Speculation about Apple’s work with both VR and AR has persisted for years now. A few months ago, we reported on rumors that the company is aiming to release a headset that is “mostly VR” in 2022, but could be very expensive. Later reports have estimated the device to cost anywhere from $1,000 to $3,000, while the company’s full AR glasses may not arriving until around 2025.

When do you think we’ll finally see Apple’s work in AR revealed? Let us know in the comments below!

Posted on

Report: Snapchat To Launch AR Glasses Developer Kit This Year

The Information reports Snap will ship AR glasses to developers & creators, with an announcement planned in May.

Founded in 2011, Snap inc has never made a profit. The Snapchat app has over 300 million active users. Some financial analysts expect the company will reach profitability this year.

Snap has shipped three smartglasses products to date, branded Spectacles. The models so far don’t have any form of display system – they function solely as a wearable camera, exporting captured clips to the Snapchat app.

Snapchat Spectacles
Current non-AR Snapchat Spectacles (v2)

Spectacles 2 added water resistance but Spectacles 3 dropped that feature, instead adding a second camera for 3D video capture and effects.

Based on The Information‘s reporting, it seems like Spectacles 4 will make the jump from smartglasses to AR glasses, with a display system and spatial tracking capable of overlaying “filters” on the real world.

It’s not clear whether Snap is developing its own core tech or using Qualcomm’s XR1 reference design.

There’s no mention of hand tracking or a controller – the report suggests the specs will continue to be controlled by the Snapchat app on a smartphone.

Spectacles 3 is priced at $380, suggesting an AR model could be priced much higher. Snap’s reported focus on developers & creators could buy it time to get a lower cost product available for consumers.

Snap’s cofounder reportedly said this to investors:

“To fully realize this idea of computing overlaid directly onto the world will require a new device: a completely new kind of camera that is capable of rendering digital content right in front of us, with the power to instantly and continuously understand the world as our own eyes do, and all in a light, wearable form factor”

Posted on

Microsoft Mesh Brings Incredible AR Multiplayer To HoloLens

With Microsoft Mesh, HoloLens finally goes fully online.

If you have any experience in VRChat, Spatial, Rec Room, AltspaceVR or other social and collaborative VR platforms, the core of Microsoft Mesh might not seem immediately exciting to you. VR has enjoyed multi-user collaboration, socializing and gaming for years now, what exactly makes it so much more exciting in AR?

I had the same reservations going into demo Mesh last week. But, with a HoloLens 2 on my head, those reservations disappeared almost immediately.

Mesh is Microsoft’s new framework for multi-user immersive reality experiences across a range of platforms. It consists of a set of tools developers can use to implement support into existing applications or build new ones. At the core of it is the long-awaited ability for multiple HoloLens users to connect across the world and interact with the same virtual assets. We’ve seen HoloLens used for same-room collaboration and some limited online functionality in the past but, with Mesh, Microsoft is taking the next step.

So, for example, I trialed a collaboration app named Fenix that came with many of the features you’ve come to expect from social VR apps. Along with other members of the press, I could import 3D assets into the space, pass them off to other users or pinch my fingers to draw Tilt Brush-style lines with hand-tracking. A 3D render of a HoloLens 2 sat on top of a table I could anchor to a physical space. I could pinch the model with both hands, resize it, place it over my head, then slot it over a 3D model of the moon I’d imported to the scene. Moving the table itself also moved other avatar’s positions along with it, allowing me to find space for everyone else in my environment.

Again, all fairly familiar concepts, right? What isn’t familiar, however, is just how extraordinary this feels inside HoloLens, which much of the friction of VR removed – AltspaceVR avatars all stood in my room, passing assets between each other with seamless intuition, as if we were all really standing next to each other. Because I’m no longer in VR, I can quickly grab my phone to check through other messages or maybe access a nearby PC should I need to. Even as a simple showcase, Fenix felt like a genuinely natural and incredibly useful extension of existing online collaboration tools – there’s even planned support to bring PC users in via webcam.

Microsoft Mesh Capture (2)

Yes there’s the usual restrictions with hardware limitations (and, notably, Microsoft’s promotional materials still don’t realistically represent HoloLens 2’s limited field of view). But, with Mesh integrated, HoloLens starts to shed of some its early prototype stigma and embrace its full potential as a collaborative production tool — and Microsoft says the device is being adopted for production tasks in greater numbers. Obviously VR has its own benefits when it comes to collaboration, and that’s why Microsoft says Mesh isn’t limited to just HoloLens. Director of Mixed Reality Greg Sullivan name-dropped Oculus when talking about compatible devices and confirmed smartphone support was coming too.

Microsoft plans to roll Mesh out in preview at first, gradually adding new features for developers to integrate, including new tools for avatars.

There’s early plans for app integration too. You can request access to a Mesh-enabled version of AltspaceVR, for example, that integrates many of the new features into the social VR platform. Perhaps more excitingly, among other partners at today’s Ignite developer conference, Niantic demonstrated multiplayer Pokemon Go battles running on Mesh in a non-consumer concept demo, though it does make one wonder why Microsoft chose to shut down Minecraft World before introducing what could have been a crucial new feature.

Granted we’re still a ways off from something like Mesh having a truly big impact on consumer AR, mostly because of the limitations of that platform itself. Multiplayer Pokemon Go in full AR is a tantalizing concept, but HoloLens remains an enterprise-focused product that’s too expensive and simply not ready for consumer adoption, and Microsoft isn’t talking about the next steps in that area at Ignite today.

With Mesh, it feels like Microsoft just made a big leap to living up to HoloLens’ full potential and, along with it, the future of spatial computing.

Posted on

Through-The-Lens Clip Shows The Supremacy Of Passthrough AR

A short through-the-lens clip of the upcoming Lynx-R1 mixed reality headset shows the advantages of passthrough AR over transparent optics.

Lynx-R1 was announced in February 2020, slated to be priced at $1499. It’s targeted at professionals. Since that announcement the design has changed, becoming significantly slimmer. It uses the same Snapdragon XR2 processor seen in Oculus Quest 2, but places it in the rear alongside a much larger battery.

There are currently two fundamental types of AR headsets: see-through and passthrough. Most AR headsets, such as Microsoft’s HoloLens 2, are see-through. You view the real world directly, with virtual objects superimposed onto the glass.

The technology behind see-through AR optics is still in the very early stages. The field of view is narrow and virtual objects cannot be fully opaque.

Passthrough headsets, like Lynx-R1, use the same kind of display system as VR headsets, except instead of rendering an entirely virtual world they show the real world via cameras. While the real world won’t necessarily look as good, this allows for AR across a much wider field of view, as well as full virtual object opacity and lower cost (HoloLens 2 is priced at $3500).

That’s visible in the short clip posted this week. It was filmed by placing 2 cameras in front of the lenses. Remember, what you’re seeing is not a transparent optic. It’s a view from the cameras on the front, synthesized into a perspective-correct view by the Snapdragon processor onboard and displayed on the LCD display behind the lenses.

Long before we get the sci-fi AR glasses expected from Apple and Facebook, VR headsets will deliver richer augmented reality via high quality cameras. As the field of view of VR increases, it will be harder yet for transparent optics to catch up.