Third Gen Smart Ray-Bans Reportedly Getting A HUD And Neural Wristband

The third-generation Ray-Ban smartglasses will reportedly ship in 2025 with a HUD and neural wristband.

Ray-Ban Stories shipped in 2021 as the result of a collaboration between Meta and Luxottica. The current Stories are essentially camera glasses for taking hands-free first person photos and videos. They also have speakers and a microphone for music and phone calls but there is no display of any sort. Snapchat has been selling successive generations of a similar product, Spectacles, since 2017.

Ray-Ban Stories

The Verge’s Alex Heath reports Meta’s VP of AR Alex Himel told staff in an internal presentation that a second generation is set to release later this year. This was previously reported by The Information last year, which suggested it may take higher quality photos and videos. The first generation hardware yields poor results in low light.

More interestingly though, Himel reportedly said a third generation is planned to ship in 2025 which has a display and comes with a neural input wristband.

Called the “viewfinder”, this heads-up display will reportedly be used to show notifications, scan QR codes, and translate real-world text in real time. To be clear: this wouldn’t be true AR, it would be a small floating contextual display.

The neural wristband is based on the tech from CTRL-Labs, a startup Facebook acquired in 2019. Meta has discussed its development since openly. It works by using using EMG (electromyography) to read the neural signals passing through your arm from your brain to your fingers. Such a device could sense even incredibly subtle finger movements not clearly perceptible to people nearby. Himel reportedly said it will let the wearer “control the glasses through hand movements, such as swiping fingers on an imaginary D-pad”.

Himel reportedly said a higher-end model of the wristband will be offered which also acts as a smartwatch, with a display and wearer-facing camera. It will reportedly be deeply integrated with the glasses and support typical health and fitness features as well as video calls.

What About True AR Glasses?

A second generation of the wristband would come with the true AR glasses Meta plans to eventually ship in the latter half of the decade. The ultimate aim for the wristband is to unlock a new method of communication that’s faster than a typing on a keyboard.

Meta has reportedly been working on AR glasses for 8 years now, spending tens of billions of dollars on what Mark Zuckerberg hopes will be “an iPhone moment”. The first generation will likely be very expensive though, with a focus on early adopters and developers.

Last April Heath reported Meta’s target year for the first generation was 2024, but by June he said the target was pushed out to 2026, and Himel is reportedly now telling staff it’s 2027.

The company is apparently already thinking about monetization strategies, with Himel reportedly planning “a combination of selling virtual goods, optional add-ons like cloud backups, and AR ads.”

Apple Headset ‘A Macintosh Moment’ – Acquired Mixed Reality Startup Founder

Apple’s entry to VR will be “a Macintosh moment”, says the founder of Vrvana.

Vrvana was acquired by Apple in 2017. It was a startup working on Totem, one of the first headsets to use high-resolution color cameras for stereo passthrough mixed reality – a feature only now starting to arrive in consumer headsets.

Vrvrana founder Bertrand Nepveu told Radio CanadaI always say that when Apple goes into virtual reality, it will be a Macintosh moment”. Nepveu claims that when he left Apple in 2021, around 1000 people were working on the long-delayed headset. In November, The Information reported around 3000 people were now working on it as Apple gets closer to launch.

A Macintosh Moment

So what exactly does Nepvue mean by “a Macintosh moment”?

The launch of the Apple Macintosh in 1984 was a pivotal moment in the history of personal computing.

While the Apple Lisa was technically the first PC with a graphical user interface (GUI) aimed at individuals, its high price (around $30K adjusted for inflation) rendered it essentially irrelevant and forgotten, selling just a few thousand units in its first year. The Macintosh had a very different fate. Macintosh delivered a GUI operating system at around a quarter of the price just 12 months later and was aggressively marketed to non-technical prospective buyers, including in a famous Super Bowl ad.

While not the overwhelming sales success Steve Jobs had hoped for, Macintosh was hailed as a revolutionary user experience and popularized the idea of personal computers as a product the average person could use. While PCs wouldn’t reach mainstream adoption until Windows 95 and the iMac over a decade later, the Macintosh proved out the idea that they actualized.

Meta’s Quest headsets start at just $400 but are primarily driven by controllers that are essentially a gamepad split in half and are hampered by a clunky and fragmented software experience. If Nepveu’s analogy holds true, Apple’s headset could introduce a refined interaction paradigm that, while too expensive and limited for most consumers in its first iteration, sets the stage for the software experience of mass-market headsets in the coming years and decades.

Apple Headset Introduction Reportedly Delayed (Yet Again)

Apple has postponed the launch of its AR/VR headset yet again, Bloomberg’s Mark Gurman reports.

The introduction was apparently originally planned for April, but should now happen at the annual WWDC conference in June. It should then go on sale later this year, Gurman writes, matching what prominent supply chain analyst Ming-Chi Kuo claimed in December.

Kuo’s December note blamed the delay on unspecified “software-related issues”. Gurman describes the problem as “issues with sensors on the device to enable the hand and eye control mechanism”. Three weeks ago Gurman described the headset using a combination of eyetracking and finger pinching as its primary control mechanism, alongside Siri voice commands.

This is just the newest of many reported delays. In 2019 The Information reported Apple planned to ship a headset in 2022, as did Gurman in 2020 and Kuo in 2021. But come January 2022 Gurman reported this had been delayed to 2023 due to “challenges related to overheating, cameras and software”.

Last year The Information published an investigative report detailing the headset’s troubled development and changing form factor. The report outlined an evolution from a thin-client paired with a wireless console to a fully standalone device covered with sensors and an external display showing the wearer’s eyes. It also suggested Apple’s project suffered from “overengineering, too-complicated solutions that often result from poor planning”.

The Information Apple VR

In August an Apple-linked company filed to trademark ‘Reality One’, ‘Reality Pro’, and ‘Reality Processor’, suggesting the headset and its processor will be branded Apple Reality. Gurman previously reported Apple is working on a VR client for FaceTime with face tracking for driving avatars, a VR version of Maps, spatial versions of Notes and Calendar, the ability to view a Mac’s display in-headset, and AR/VR tools for developers.

The Information and Gurman have previously reported the product will be priced around $3000. Kuo claimed it will weigh noticeably less than current VR headsets and feature high resolution OLED microdisplays, while The Information has claimed it will be powered by the M2 chip seen in the latest MacBooks and feature hand tracking, eye tracking, face tracking, leg tracking and iris scanning for logins and payments. So if the reports so far are to be believed, Apple’s product will have higher resolution, a more powerful processor, more features, and a slimmer design than Meta’s Quest Pro – though at a higher price. And given Meta reportedly plans a successor for 2024, Apple’s headset might spend more of its lifecycle competing against Quest Pro 2, which might feature OLED microdisplays and a new chip too.

Tim Cook On AR Headsets: “Stay Tuned And You’ll See What We Have To Offer”

Apple CEO Tim Cook was interviewed by China Daily, and he made an interesting comment when asked about AR headsets.

Reporter: “Chinese consumers are highly enthusiastic about VR and AR technologies, but some of them are not very satisfied with products currently available on the market. What do you think are the key factors for AR products such as AR headsets to succeed in the consumer market?”

Cook: “That’s a great question. I am incredibly excited about AR as you might know. And the critical thing to any technology including AR is putting humanity at the center of it. And that is what we focus on every day. Right now as an example we have over 14,000 ARKit apps in the App Store which provide AR experiences for millions of people around the world. But I think despite that we’re still in the very early innings of how this technology will evolve. I couldn’t be more excited about the opportunities we see in this space and sort of stay tuned and you’ll see what we have to offer.”

The Information Apple VRLast year Bloomberg, supply chain analyst Ming-Chi Kuo Kuo and The Information released reports claiming Apple is working on a premium headset for VR and AR with high resolution color passthrough. Kuo claimed this headset will weigh less than Meta’s Quest 2, feature dual 4K OLED microdisplays, and use a new chip with “similar computing power as the M1 for Mac”.

Earlier this month Bloomberg reported Apple is working on realityOS (rOS) versions of its core apps as well as the ability to view a Mac’s display and AR/VR developer tools. References to realityOS were found in App Store upload logs and Apple code earlier this year, and an Apple-linked shell company trademarked RealityOS in May. At Apple’s WWDC conference a number of features tangentially related to AR were announced, but the company may be waiting for the headset reveal to show its full AR/VR strategy.

Earlier this year Bloomberg reported Apple’s headset was delayed to 2023 due to “challenges related to overheating, cameras and software”. The New York Times also reports the headset is delayed to 2023, with its sources saying this is due to “continuing challenges with its battery power”. Both Bloomerg and The Information report the product is set to be priced north of $2000.

Reports: Meta Cancels 2024 AR Glasses Consumer Release Amid Cost Cutting

Meta no longer plans release the AR glasses slated for 2024 to consumers, The Information and The Verge report.

At the Connect 2021 conference in October Meta announced Project Nazare, “our first full augmented reality glasses”. The company didn’t show a prototype or even real footage, instead showing a brief “simulated” concept clip.

“We still have a ways to go with Nazare, but we’re making good progress,” Mark Zuckerberg remarked.

In April The Verge’s Alex Heath released a report detailing Meta’s apparent plans for glasses over the next six years. Heath reported Project Nazare would launch in 2024. It apparently weighs 100 grams, around four times normal glasses, and resembles Clark Kent’s thick black frames. Nazare will apparently have a battery life of just 4 hours and be powered by an included wireless puck which can fit in your pocket. The original target field of view was 70 degrees, but “that goal likely won’t be met” Heath wrote.

In May Reuters reported CTO Andrew Bosworth told staffers in the Reality Labs hardware division the company “could not afford to do some projects anymore and would have to postpone others”.

At the time of the April report Heath noted that the expensive components – including eye tracking cameras, custom waveguides, and microLED projectors – would drive Nazare’s price to “multiple thousands of dollars”, so it would be a niche product aimed at early adopters and developers. But both The Information and Heath now report Nazare will not be sold as a product at all. The Information describes Nazare’s new purpose as a “a demonstration product”, whereas Heath says Meta will “distribute them to developers”.

In that same April report, Heath wrote Meta planned second generation AR glasses for 2026 and a third generation for 2028. The Information’s new report says that second generation is codenamed Artemis, and will now be the first to commercially launch. Heath describes Artemis as having “a less bulky design and more advanced display technology”.

Meta still plans to release cheaper non-AR smart glasses codenamed Hypernova as early as next year, which will pair with your smartphone to display notifications and contextually useful information in a small heads up display, Heath writes. In May Mark Zuckerberg met with the billionaire founder and chairman of Luxottica – the Italian company behind brands like Ray-Ban and Oakley – to plan “new smart glasses”. Hypernova may be a new generation of Ray-Ban Stories, adding a HUD. The current product does not have a display.

The shifting plans come amid widespread reports of general cost cutting at Meta, apparently due to slowing revenue growth and threat of a looming recession. This week Bloomberg reported Meta halted development of a smartwatch with cameras, and The Information reports the company will not make any more Portal video calling appliances for consumers.

Interestingly, The Information & Heath offer different reasons for the smartwatch cancelation. The Information wrote that Meta struggled with “the difficulty of making a watch that users could unclip without dropping it” and the battery wasn’t lasting a full day. But Heath says the detachable design made it difficult to integrate the EMG tech for controlling AR glasses with neural signals, writing that Meta “has pivoted to focus fully on a design that better supports EMG on the wrist”.

Meta CTO Andrew Bosworth indirectly responded to the reports on Twitter, confirming the company still plans to ship AR glasses and wrist devices.

As is common in our industry, we iterate on multiple prototypes in parallel & shift resources as we learn,” he wrote.

Project Cambria Has A Depth Sensor & Much Higher Resolution Cameras

Project Cambria has a depth sensor and cameras with 3 times the resolution of Quest 2’s.

Cambria is the public codename for Meta’s upcoming high end standalone headset, announced at Connect 2021 in October. It will be sold alongside Quest 2 with a price tag “significantly” higher than $800, aimed at remote workers and mixed reality enthusiasts. The headset looks to have a more balanced design than Quest 2 with a slimmer visor achieved through the use of pancake lenses instead of fresnel lenses. It also has built-in face and eye tracking to drive avatars in social experiences like Workrooms.

But Cambria’s headline new feature is high resolution color passthrough for mixed reality – Quest 2’s passthrough is grainy black & white. And today in a conversation with Jesse Schell (of Schell Games), Meta CEO Mark Zuckerberg revealed some of the hardware behind this mixed reality functionality.

Zuckerberg said Quest 2’s passthrough is based on “sensors that were not designed to give you anything more than just a very rough outline of what’s going on around you”.

Cambria, on the other hand will have “a bunch of new sensors” including “high resolution color outward facing cameras” as well as a dedicated depth sensor. “Right now on Quest 2 we hack it a little bit by looking at the cameras and trying to intuit what the depth is”. Zuckerberg told Protocol that this sensor is an IR projector for active depth sensing, and also said the regular cameras have three times the resolution of Quest 2’s.

Hardware level depth sensing is also “more optimized towards hands” Zuckerberg told protocol, though Meta has dramatically improved hand tracking even on Quest 2 by leveraging recent advances in computer vision.

Project Cambria still doesn’t have a product name, specific release window, or exact price; but Zuckerberg re-iterated it will launch “later this year”.

Zuckerberg Meets With Luxottica Chairman To Plan ‘New Smart Glasses’

Mark Zuckerberg met with Leonardo Del Vecchio to plan “new smart glasses” and demo Meta’s neural wristband.

“Great to be back in Milan to discuss plans for new smart glasses with Leonardo Del Vecchio and the EssilorLuxottica team. Here Leonardo is using a prototype of our neural interface EMG wristband that will eventually let you control your glasses and other devices. 😎

Del Vecchio is the billionaire founder and chairman of Luxottica, the Italian company behind brands like Ray-Ban and Oakley. Meta and Luxottica have already collaborated on a smart glasses product which shipped in September, called Ray-Ban Stories. Stories are essentially camera glasses – they also have speakers and a microphone for music and phone calls but there is no display of any sort. Snapchat has been selling a similar product, Spectacles, since 2017.

Ray-Ban Stories

Until now it was unclear just how important Meta saw this partnership to its long term ambitions. EssilorLuxottica has a near monopoly on the eyewear market worldwide – its brands like Ray-Ban and Oakley are some of the most recognizable on the planet. Maintaining this partnership could be a key element of Meta’s strategy to take on Apple later this decade.

Last month The Information reported Meta plans to launch a second generation of Ray Ban Stories in 2023. Separately, The Verge reported Meta plans two distinct glasses for 2024 – affordable heads-up display glasses codenamed Hypernova, and expensive true AR glasses codenamed Project Nazare.

The report also claimed both Nazare and Hypernova will be bundled with Meta’s in-development neural wristband – the device Zuckerberg is demoing to Del Vecchio. Meta has discussed its development openly. The device reads the neural signals passing through your arm from your brain to your hand, using EMG (electromyography). Such a device can track finger movement precisely before it even happens, and can even sense incredibly subtle gestures not clearly perceptible to others nearby.

Everyone I’ve talked to who has tried a prototype of the band Meta is working on says it’s one of the most impressive tech demos they’ve ever experienced. If it works at scale, the company thinks it could have the next mouse and keyboard” The Verge’s Alex Heath wrote.

Zuckerberg’s “will eventually let you control your glasses and other devices” statement seems to back up The Verge’s claim of this device being bundled with Meta’s future glasses – and demoing it to Del Vecchio suggests the EssilorLuxottica partnership will go far beyond today’s Ray-Ban Stories.

Watch What Developers Are Making With Snapchat’s AR Glasses

Snap shared a compilation of what developers have been building with its Spectacles AR glasses.

Spectacles was revealed almost a year ago. It’s a standalone device powered by Qualcomm’s XR1 chip featuring two cameras to perform spatial tracking of the world as well as hand tracking and video capture. A touchpad on the side can be used for precise input, and there are dedicated buttons for area scan & clip capture.

The displays have an impressive 2000 nits of brightness – 4x brighter than HoloLens 2 and 10x brighter than Magic Leap One – making Spectacles one of the only AR devices usable outdoors. However, the field of view is a tiny 26.3 degrees diagonal and the battery lasts just 30 minutes.

Spectacles aren’t a general purpose computer – you don’t build apps for them in a game engine. Instead they integrate directly with Snap’s existing AR platform, which has hundreds of millions of active users on smartphones. Developers build ‘Lenses’ – AR experiences – for both Spectacles & the Snapchat app using the company’s Lens Studio.

Over the past year Lens Studio has added new features, including speech recognition, 3D hand tracking, and (arguably) most importantly colocation- multiple Spectacles users in the same physical space seeing the same virtual objects in the same place, enabling multiplayer. Snap calls this Connected Lenses.

Spectacles is still only available to select developers – it isn’t yet a product consumers can buy. You can apply for Spectacles as a prospective developer on Snap’s website.

Zuckerberg Sees Project Cambria ‘Eventually Replacing Your Laptop’

Mark Zuckerberg said he sees Project Cambria “eventually replacing your laptop”.

Here’s how he described Cambria to Meta’s investors during the Q1 2022 earnings call:

“On the hardware side, Meta Quest 2 continues to be the leading virtual reality headset. Later this year, we’ll release a higher-end headset, codenamed Project Cambria, which will be more focused on work use cases and eventually replacing your laptop or work setup.

This premium device will have improved ergonomics and full color passthrough mixed reality to seamlessly blend virtual reality with the physical world. We’re also building in eye tracking and face tracking so that your avatar can make eye contact and facial expressions, which dramatically improves your sense of presence. It’s also a good example of why we’re developing hardware in addition to the social platforms.”

Project Cambria was announced in late October at Connect 2021 as a high end standalone headset launching in 2022 which will be sold alongside Quest 2. At one point it was called ‘Quest Pro’, at least internally.

But what exactly does he mean by “replacing your laptop”?

The short teaser for Cambria shared at Connect 2021 shows one of VR’s key potential use cases: a portable but powerful workspace. The idea is that instead of being limited to hunching over the one small screen built into your laptop, you can have multiple virtual screens of whatever size you want floating in front of you. Consulting CTO John Carmack described the eventual goal as “to be something that somebody hard up for money decides I’m going to buy a VR headset instead of a Chromebook or instead of a tablet”.

Meta has been slowly adding productivity features to the Quest operating system, branded as Infinite Office. If you have a Logitech K830 or Apple Magic Keyboard, they’ll show up in VR so you can type and use the trackpad to navigate. Triple window support was added to the web browser in August, with window resizing added in December. These windows can actually also be Android phone apps, but the Quest Store doesn’t serve these and there’s obviously no Google Play Store or Amazon Appstore, so such apps have to be sideloaded.

 

While the software is starting to support these use cases, Quest 2’s front-heavy design makes extended seated use uncomfortable and its passthrough view is low resolution black & white. Cambria appears to have a more compact and balanced design, with a smaller frontbox and a strap resembling Quest 2’s elite strap accessory. And as shown in the teaser, its high resolution color passthrough should allow interaction with real world objects as well as reading notes on paper.

Still, the focus on web apps limits the current scope of Meta’s VR productivity ambitions to taking on Chromebooks. A lot of work is done on Windows or Mac, leveraging power hungry native apps like Photoshop, Premiere, Unity, or programming IDEs. You can of course use apps like Virtual Desktop and Immersed to use your PC in VR, but neither is frictionless and both still work around an operating system designed for windows housed within physical screens.

Unofficial renders by Marcus Kane depicting Cambria’s design

Zuckerberg finished his statement by saying Meta will “share more details about Project Cambria in the months ahead as we get ready to launch it”.

Amazon Job Listings Reference ‘New-To-World’ AR/VR Consumer Product

Amazon job listings reference a “new-to-world” AR/VR consumer product.

Spotted by Protocol, one listing explains “You will develop an advanced XR research concept into a magical and useful new-to-world consumer product” while another references “developing code for early prototypes through mass production.”

Another job listing describes the role as working on “the core system interface along with end-user applications spanning from multi-modal interfaces to 3D AR entertainment experiences”.

Protocol also spotted that in March Amazon hired Kharis O’Connell to lead a “Futures Design” group, described as “helping Amazon experience what it’s like to live in the future, today”. O’Connell once worked for the now defunct Meta View startup, and then worked on Google’s AR operating system.

Amazon is the only consumer tech giant with no announced or rumored AR or VR headset product. Meta has its Quest VR line and is working on AR glasses too. Microsoft has its HoloLens AR headsets. Multiple reliable sources claim Apple is working on a mixed reality headset, and The Verge reported Google is too.

The company currently sells “smart glasses” called Echo Frames, but these lack any display system or cameras – the use cases are talking to Alexa, taking calls, and playing music. It’s possible – even arguably likely – that Amazon intends to develop this product line into AR glasses in the long term future.

If you’ve got information to share with UploadVR you can email us via tips@uploadvr.com.