Ming-Chi Kuo: Apple Likely to Release Mixed Reality Headset in January 2023

Industry analyst Ming-Chi Kuo, a respected figure in all things Apple supply chain leaks, says the Cupertino tech giant is likely preparing to launch its long-rumored mixed reality headset early next year.

In a Medium post, Kuo outlines a few key points based on how he gathers the industry is headed.

In short, Kuo posits that Meta is slowing down investment in VR hardware due to looming economic recession, but this will provide others opportunity to play catchup as market share shifts away from Meta to companies such as Sony, Valve, Pico, and HTC. It’s not VR, its Meta’s core business that’s taking a hit.

Kuo says there’s still a “vast” potential demand for VR headsets in China which could be filled by companies with ready access to the Chinese market, such as ByteDance subsidiary Pico Interactive and Taiwan’s HTC.

Apple is also tapped to fill growing demand. Codenamed N301, Apple’s MR headset will “likely release in January 2023,” Kuo maintains, and is set to “favor the continued rapid growth of the headset sector,” adding that it’s “the most complicated product Apple has ever designed.”

“Although Apple has repeatedly reiterated its focus on AR, I believe Apple AR/MR supporting video see-thru could also offer an excellent immersive experience,” Kuo says. “Therefore, the launch of Apple AR/MR will further boost the demand for immersive gaming/multimedia entertainment.”

N301 is said to combine VR displays with passthrough cameras for both VR and AR applications. Check out the roundup below for all of the rumors surrounding Apple’s MR headset:

What We (think we) Know About N301 Mixed Reality Headset

The post Ming-Chi Kuo: Apple Likely to Release Mixed Reality Headset in January 2023 appeared first on Road to VR.

Tim Cook On AR Headsets: “Stay Tuned And You’ll See What We Have To Offer”

Apple CEO Tim Cook was interviewed by China Daily, and he made an interesting comment when asked about AR headsets.

Reporter: “Chinese consumers are highly enthusiastic about VR and AR technologies, but some of them are not very satisfied with products currently available on the market. What do you think are the key factors for AR products such as AR headsets to succeed in the consumer market?”

Cook: “That’s a great question. I am incredibly excited about AR as you might know. And the critical thing to any technology including AR is putting humanity at the center of it. And that is what we focus on every day. Right now as an example we have over 14,000 ARKit apps in the App Store which provide AR experiences for millions of people around the world. But I think despite that we’re still in the very early innings of how this technology will evolve. I couldn’t be more excited about the opportunities we see in this space and sort of stay tuned and you’ll see what we have to offer.”

The Information Apple VRLast year Bloomberg, supply chain analyst Ming-Chi Kuo Kuo and The Information released reports claiming Apple is working on a premium headset for VR and AR with high resolution color passthrough. Kuo claimed this headset will weigh less than Meta’s Quest 2, feature dual 4K OLED microdisplays, and use a new chip with “similar computing power as the M1 for Mac”.

Earlier this month Bloomberg reported Apple is working on realityOS (rOS) versions of its core apps as well as the ability to view a Mac’s display and AR/VR developer tools. References to realityOS were found in App Store upload logs and Apple code earlier this year, and an Apple-linked shell company trademarked RealityOS in May. At Apple’s WWDC conference a number of features tangentially related to AR were announced, but the company may be waiting for the headset reveal to show its full AR/VR strategy.

Earlier this year Bloomberg reported Apple’s headset was delayed to 2023 due to “challenges related to overheating, cameras and software”. The New York Times also reports the headset is delayed to 2023, with its sources saying this is due to “continuing challenges with its battery power”. Both Bloomerg and The Information report the product is set to be priced north of $2000.

Apple Quietly Released One of The Most Impressive AR Room-mapping Tools

Apple has barely mentioned augmented or virtual reality in its big keynotes lately, however at WWDC 2022 earlier this month, the company quietly released probably one of the best 3D room-mapping tools for mobile AR yet.

Called RoomPlan, the ARKit Swift API uses the camera and LiDAR scanner on recent iPhones and iPads to create a 3D floor plan of a room, including key characteristics such as dimensions and types of furniture.

It’s not for consumers (yet) though. Apple says it’s aiming to appeal to professionals like architecture and interior designers for conceptual exploration and planning, as well as developers of real estate, e-commerce, or hospitality apps; developers can integrate RoomPlan directly into their AR-capable apps.

When it was released earlier this month, Jonathan Stephens, Chief Evangelist at spatial computing company EveryPoint, took RoomPlan for a test drive to see what it could do. The results are pretty surprising.

RoomPlan seems to be able to deal with a number of traditionally difficult situations, including the mirror seen above, but also messy spaces, open and closed doors, windows, and generally complex architecture. Still, Stephens’ house isn’t just a bunch of cube-shaped rooms, so there’s a few bits that just didn’t match up.

Vaulted ceilings, wall openings, multifloor areas like you might find in foyers were all a bit too difficult for RoomPlan to correctly digest. Although not perfect, it seems to at least autocorrect to some degree based on some assumptions of how things might best fit together.

RoomPlan isn’t just for app integrations though. Apple says it outputs in USD or USDZ file formats which include dimensions of each component recognized in the room, such as walls or cabinets, as well as the type of furniture detected.

If you’re looking to finetune the scan, dimensions and placement of each individual components can be adjusted when exported into various USDZ-compatible tools, such as Cinema 4D, Shapr3D, or AutoCAD, Apple says.

We’re still no closer to learning when the company plans to release its rumored mixed reality headset or its full-fledged AR glasses, however either AR or MR headset would need extremely robust space-mapping capabilities. Seeing Apple make these sorts of strides using its existent platforms certainly shows they’re on the right track.

If you haven’t been following along with the Apple rumor mill, check out some of the links below regarding the company’s mixed reality headset, codenamed N301:

What We (think we) Know About N301 Mixed Reality Headset


A special thanks to Hrafn Thorisson for pointing us to the news!

The post Apple Quietly Released One of The Most Impressive AR Room-mapping Tools appeared first on Road to VR.

Supply Chain Analyst: Apple Headset Delayed To Mid 2023

A prominent supply chain analyst predicts Apple’s headset will launch in mid 2023.

Ming-Chi Kuo is a TF International Securities analyst mostly known for predicting Apple products & moves over a year in advance using his supply chain sources.

I believe Apple’s AR/MR headset shipping date will postpone to 2Q23 (vs. 1Q23 of market consensus) because Shanghai lockdown interrupts the development” Kuo Tweeted this week.

Kuo predicts Apple will host a media event in January, deliver development kits 2-4 weeks later, and launch preorders in the second quarter of 2023 for a release before WWDC 2023. WWDC is Apple’s yearly software conference, almost always held in June.

Last year Kuo, Bloomberg, and The Information released reports claiming Apple is working on a premium headset for VR and AR with high resolution color passthrough. Kuo claimed this headset will weigh less than Meta’s Quest 2, feature dual 4K OLED microdisplays, and use a new chip with “similar computing power as the M1 for Mac”.

Over the weekend Bloomberg reported Apple is working on realityOS (rOS) versions of its core apps as well as the ability to view a Mac’s display and AR/VR developer tools. References to realityOS were found in App Store upload logs and Apple code earlier this year, and an Apple linked shell company trademarked RealityOS in May. At Apple’s WWDC conference a number of features tangentially related to AR were announced, but the company may be waiting for the headset reveal to show its full AR/VR strategy.

Earlier this year Bloomberg reported Apple’s headset was delayed to 2023 due to “challenges related to overheating, cameras and software”. The New York Times also now reports the headset is delayed to 2023, with its sources saying this is due to “continuing challenges with its battery power”. Both Gurman and The Information report the product is set to be priced north of $2000.

NYT: Mandalorian Director Favreau Building Dinosaur Experience For Apple’s Headset

The New York Times reports that Jon Favreau is developing “video content” for Apple’s upcoming mixed reality headset, expected to launch next year.

The content will be related to the recent Apple TV+ show Prehistoric Planet, which debuted on the streaming service last month with Favreau as showrunner and David Attenborough as the narrator.

The report contains few other details on the nature of the Prehistoric Planet content for Apple’s headset, except that Favreau is “working to bring that show’s dinosaurs to life on the headset.” That being said, some kind of 3D immersive experience seems likely.

Favreau is an acclaimed Hollywood filmmaker, director and actor. His directing credits include Elf (2003), Iron Man (2008), Chef (2014) and The Jungle Book (2016), as well as most recently working on popular Star Wars series The Mandalorian for Disney+. He also been heavily involved in various capacities with Marvel Cinematic Universe films.

However, Favreau is no stranger to virtual reality either. In 2016, he worked on a VR narrative experience called Gnomes & Goblins for the HTC Vive. Even more recently, he’s been using VR headsets to visualize shots on the sets of movies like the 2019 remake of The Lion King.

“[We] created this multiplayer VR filmmaking game where all the crew put on headsets and they were able to walk around and look around the Pride Lands and watch the animated performances and set cameras inside VR,” he told Jimmy Kimmel in 2019. “So it felt like we were making a live action film inside virtual reality.”

Speaking to UploadVR in 2016 about Gnomes & Goblins, Favreau said he wasn’t sure if he could see himself spending the majority of his time as a director working in an immersive medium. Nonetheless, it’s clear virtual and mixed reality has kept his interest since then.

According to industry analysts and publications, Apple’s mixed reality headset is set to be revealed either this year or early next, for a release in 2023.

Bloomberg: Apple Building Headset Versions Of FaceTime, Maps & More

Apple is working on headset versions of its core apps as well as the ability to view a Mac’s display, Bloomberg’s Mark Gurman reports.

Last year GurmanThe Information, and supply chain analyst Ming-Chi Kuo released reports claiming Apple is working on a premium headset for VR and AR with high resolution color passthrough. Kuo claims claimed this headset will weigh less than Meta’s Quest 2, feature dual 4K OLED microdisplays, and use a new chip with “similar computing power as the M1 for Mac”.

In May Gurman reported Apple recently ramped up development of realityOS (rOS), the operating system the headset will run, and previewed it to the board of directors. References to realityOS were found in App Store upload logs and Apple code earlier this year, and an Apple linked shell company trademarked RealityOS last week. At Apple’s WWDC conference today a number of features tangentially related to AR were announced, but the company may be waiting for the headset reveal to show its full AR/VR strategy.

The Information Apple VR

In a newsletter published over the weekend Gurman wrote:

Apple’s headset initiative isn’t simply the device and its operating system. It’s an entire set of new VR- and AR-powered Apple apps and experiences, a slew of input paradigms never seen before on the company’s products and a completely new platform for third-party developers.

Apple is reportedly working on a VR client for FaceTime with face tracking driving Memoji avatars, as well as a VR version of Maps (similar to Google Earth VR?) and spatial versions of Notes and Calendar.  You’ll also apparently be able to extend a Mac’s display into the headset. Leveraging its existing suite of first party cross-platform applications, and making them work seamlessly across devices, could be one of Apple’s key advantages competing with Meta’s Project Cambria.

Gurman also says Apple is “leveraging its entertainment arm” and acquisition of live events livestreaming startup NextVR to produce content for the headset. Over the weekend The New York Times also reported Apple is enlisting Hollywood directors like Jon Favreau to create immersive VR content, such as a version of Prehistoric Planet.

RealityKit

For developers, Apple will reportedly ship a spatial version of Apple’s SwiftUI user interface framework, which should make it much easier to build AR and VR applications compared to using a game engine like Unity or Unreal. This UI framework would integrate into RealityKit, Apple’s existing high level AR framework that handles physics, spatial audio, all aspects of rendering including materials, shadows, reflections, and even camera motion blur. RealityKit also handles networking for multiplayer AR apps, meaning developers won’t need to be network engineers to develop shared AR experiences.

Developers will also apparently get access to a Mac based headset simulator so they can develop before having the device in-hand.

Earlier this year Gurman reported Apple’s headset was delayed to 2023 due to “challenges related to overheating, cameras and software”. The New York Times also now reports the headset is delayed to 2023, with its sources saying this is due to “continuing challenges with its battery power”. Both Gurman and The Information report the product is set to be priced north of $2000.

Apple’s iPhone Will Soon Scan Your Ear to Solve a Big Problem with Spatial Audio

Today during Apple’s WWDC 2022 keynote, the company announced that iOS 16 will allow users of modern iPhones to scan the shape of their ear to create more accurate spatial audio. Likely implemented as an HRTF, creating custom HRTFs for consumers was once impractical due to the need for sophisticated equipment, but advances in computer vision are making the technology much more accessible.

When it comes to digital spatial audio, there’s a limit to how accurate the sense of ‘position’ or ‘3D’ the audio can be without taking into account the unique shape of the user’s head and ears.

Because everybody has a uniquely shaped head, and especially ears, elements of incoming sound from the real world bounce off your head and into your ears in different and very subtle ways. For instance, when a sound is behind you, the precise geometry of the folds in your ear reflect sound from that angle in a unique way. And when you hear sound coming to your ear in that particular way, you’re attuned to understand that the source of the sound is behind you.

To create a highly accurate sense of digital spatial audio, you need a model which accounts for these factors, such that the audio is mixed with the correct cues that are created by the unique shape of your head and ears.

Audiologists have described this phenomena mathematically in a model known as a Head-related Transfer Function (also known as an HRTF). Using an HRTF, digital audio can be modified to replicate the spatial audio cues that are unique to an individual’s ear.

So while the math is well studied and the technology to apply an HRTF in real-time is readily available today, there’s still one big problem: every person needs their own custom HRTF. This involves accurately measuring each ear of each person, which isn’t easy without specialized equipment.

But now Apple says it will make use of advanced sensors in its latest iPhones to allow anyone to scan their head and ears, and create a custom spatial audio profile from that data.

Apple isn’t the first company to offer custom HRTFs based on a computer-vision model of the ear, but having it built into iOS will certainly make the technology much more widespread than it ever has been.

During the WWDC 2022 keynote, Apple announced the feature as part of the forthcoming iOS 16 update which is due out later this year. It will work on iPhones with the TrueDepth camera system, which includes the iPhone 10 and beyond.

But just having an accurate model of the ear isn’t enough. Apple will need to have developed an automated process to simulate the way that real sound would interact with the unique geometry of the ear. The company hasn’t specifically said this will be based on an HRTF implementation, but it seems highly likely as it’s a known quantity in the spatial audio field.

Ultimately this should result in more accurate digital spatial audio on iPhones (and very likely future Apple XR headsets). That means that a sound 10 feet from your left ear will sound more like it should at that distance, making it easier to differentiate between a sound 2 feet from your left ear, for instance.

This will pair well with the existing spatial audio capabilities of Apple products; especially when used with AirPods which can track the movement of your head for a head-tracked spatial audio experience. Apple’s iOS and MacOS both support spatial audio out of the box, which is capable of taking standard audio and making it sound as if it’s coming from speakers in your room (instead of in your head), and accurately playing back sound that’s specially authored for spatial audio, such as Dolby Atmos tracks on Apple Music.

And there’s another potential upside to this feature to. If Apple makes it possible for users to download their own custom HRTF profile, they may be able to take it and use it on other devices (like on a VR headset for instance).

The post Apple’s iPhone Will Soon Scan Your Ear to Solve a Big Problem with Spatial Audio appeared first on Road to VR.