Cyberpunk is science fiction subgenre in a dystopian futuristic setting that tends to focus on a “combination of lowlife and high tech”, featuring futuristic technological and scientific achievements, such as artificial intelligence and cybernetics, juxtaposed with societal collapse, dystopia or decay (per Wikipedia). What Cyberpunk also often features are advanced demonstrations and uses of XR technologies: Augmented Reality, Virtual Reality and Mixed Reality.
What we found more interesting was the Top Cyberpunk Films You Haven’t Seen article as it has some new and obscure AR & VR films we haven’t seen yet. There’s a recent film called Karmalink that has some advanced AR concepts, a film called Hardwired that features AR advertising and display concepts (via a brain implant) and Terminal Justice which features old school VR HMD’s, a virtual reality crime scene and AR eye implants for infrared and night time vision. There were also two additional films we would recommend every AR & VR fan check out: Virtual Nightmare and Natural City. Virtual Nightmare uses VR similar to The Matrix and Natural City is similar to Blade Runner and shows many advanced AR concepts integrated into a futuristic society.
KBZ also has a short video that highlights some of these films (with Cyberpunk AR, VR and MR concepts) and you can watch the video below.
We’ve referenced the KBZ Films site before as they’ve posted some lists of lesser-known Augmented Reality & Virtual Reality films. Recently, KBZ has posted an article of the Best AR & VR Films that has some great films listed (and that we’ve highlighted in the past on our blog here). From the article’s Top 20 rankings are some great AR & VR films including Avalon (VR), Auggie (AR), Brainstorm (VR), Sleep Dealer (MR) and Anon (AR). It’s worth checking the article out if you’re new to the XR field and want to see a list of the Best AR & VR films. KBZ also has a list of every imaginable film about AR, VR & MR that you can find here.
The KBZ site also has some other interesting articles to check out including a list of some great films about Artificial Intelligence (AI) and another article that highlights some obscure films about the Multiverse and alternate realities. The AI list of films is especially relevant as AI technology is also usually found in AR & VR films. The film Auggie comes to mind as the AR projection through eyewear is based on AI models of the person’s subconscious. Below are videos of some of the AI and Multiverse films referenced in the articles.
Finally, if you’re into Sci-Fi, there’s also some other great articles from the site including a list of the best time travel films and best time loop films. We’ve always found those films interesting as more recent time travel films have also included aspects of XR technologies.
If the avalanche of recent reports can indicate anything at all, it seems Apple is entering the VR/AR headset market fairly soon, bringing along with it the most inflated expectations the industry has ever seen. It’s probably going to be expensive, but whether it flops or becomes a big hit, the mere existence of Apple in the space is set to change a lot about how things are done.
The iPhone wasn’t the first smartphone. That award goes to an obscure PDA device called the IBM Simon, released in limited numbers in 1994. The Apple Watch wasn’t the first smartwatch either. That was debatably the Seiko Raputer, which was released in 1998 in Japan. Its monochrome LCD wasn’t capable of touch, instead offering up a tiny eight-direction joystick and six function buttons to browse files, play games, and set calendar appointments. Similarly, iPad wasn’t the first tablet. Mac wasn’t the first home computer. iPod wasn’t the first MP3 player. But all of these products have become nothing short of iconic. There’s very little benefit to being first, at least as far as Apple is concerned.
And while it seems the company’s first mixed reality headset could finally debut at its Worldwide Developers Conference (WWDC) in June, like all of its other products, it won’t be the first MR headset. Just the same, like everything else the fruit company makes, it’s going to be the one everyone is talking about—for better or worse.
In case you haven’t noticed, Apple is a big deal. It has an ecosystem of products which connect to each other, design-forward hardware that has helped it maintain brand name cache, and a philosophy that puts user-friendliness at the core of its software experience. Oh, and it’s the most valuable company in the world.
And while the irrational exuberance for successive device generations has mostly petered out since its heydays in the early 2000s, reducing its famed long-line launch extravaganzas to more chill online pre-order releases, becoming an Apple apostate is still unthinkable to many. Once you’re in, you’re in. You buy the phone, the laptop, the headphones, and now, maybe you’ll get the newfangled headset too. Maybe. Let’s put aside the rumors for now. Forget about the spec breakdowns, hardware design leaks, software capabilities, etc. There are plenty of them out there, and you can read about those here. The only thing we know for sure is Apple is… well… Apple. Here’s what you, and probably everyone else is expecting.
Apple’s BKC Store in Mumbai, India | Image courtesy Apple
For Better: What Should Happen
Unless the company is making a drastic departure here, its first mixed reality headset should be built with this same level of user friendliness as all of its other devices, which means it should connect to the Apple ecosystem easily, and have a simple and intuitive UI. Log in with Apple ID. No muss, no fuss (whatever ‘muss’ is). Privacy should be a giant focus for the headset from the outset, since it will almost certainly pack eye-tracking in addition to a host of cameras to get a glimpse of the inside of your immediate surroundings, messiness and all. Apple has its fair share of data collection scandals, yet it seems to inspire enough confidence for privacy to be a big historical selling point for all of its devices.
If you want to avoid drawing the ire of tech reviewers everywhere though, wearing it should be fairly simple and very comfortable, and the experiences within should be of high enough value to overcome that inherent friction of charging it, putting it on, setting up a tracking volume, and wearing it for extended periods of time—everything we expect from any mixed reality headset at this point. It should fit most people, and offer up a clear picture to people with heads and eyes of all shapes and sizes.
Meta Quest Pro | Image courtesy Meta
An obvious analogue here is Meta Quest Pro, which is relatively low friction, but things like a halo strap that forces too much weight on your brow, or a passthrough that’s just a little too grainy, or a display that doesn’t have a high enough pixel per degree (ppd) for staring at text—all of these things make it less appealing to users in the day-to-day, introducing what you might call accumulative friction. You use it a bunch at first until you figure out all of the niggles, at which point you may revert to traditional computing standards like using a laptop or smartphone. The thing isn’t really the all-purpose device you hoped it would be, and the company thinks twice about when to send the better, more improved version down the pipeline.
One would hope that Apple’s headset, on the other hand, should have a mature design language and have obviously useful features from day one. While there’s bound to be some stutters, like with the first Apple Watch, which was critiqued for its slow software, short battery life, and lack of customization, it should all be there, and not require a ton of feature updates to enhance after the big launch day.
It should sell well out of the gate—at least by the standards of the existing XR industry—even if everything isn’t perfect. And it should be so cool that it’s copied. Like a lot. And it should drag top-level studios into the XR scene to start making innovative and useful apps that aren’t just straight ports of ARkit or ARcore apps made for mobile, but things people need and want to use in-headset. A big win from Apple should not only spur its new mixed reality product category, but kick off a buzz among developers, which would include those who currently work in the XR industry and Apple’s existing cohort of dedicated iOS developers.
But more than merely being the latest shiny new headset within the existing XR industry, Apple’s entrance into the field has a real chance of radically expanding the industry itself, by showing that the world’s most iconic tech company now thinks the medium is worth pursuing. That’s the way it happened when Apple jumped into MP3 players, smartphones, tablets, wireless earbuds, and more.
As the saying goes, a rising tide lifts all boats. The inverse is also true though….
For Worse: What Could Happen
Apple’s headset is reportedly (okay, maybe just one rumor) priced somewhere near $3,000, so it probably won’t be the sort of accessory that initially attracts people to the ecosystem; that would be the job of a peripheral like Apple Watch. It will likely rely on the pool of built-in Apple users. Despite the price, the first iteration very likely won’t offer the sort of power you’d expect from a workhorse like Apple MacBook Pro either.
At the outset, any sustained draw from prosumers will invariably hinge on how well it can manage general computing tasks, like you might have with an iPad or MacBook, and everything else current mixed reality headset should do too, namely VR and AR stuff. That includes a large swath of things like fitness apps, both AR and VR games and experiences, productivity apps, standard work apps, everything. Basically, it has to be the Quest Pro that Meta wanted to release but didn’t.
AR turn-by-turn directions on an iPhone | Image courtesy Apple
And if not, it leaves Apple in a pretty precarious situation. If their headset can’t find a proper foothold within its ecosystem and attract enough users, it could lead to low adoption rates and a lack of interest in the technology as a whole. Mixed reality is largely seen as valuable steppingstone to what many consider the true moneymaker: all-day AR glasses. And despite some very glassses-shaped AR headsets out there, we’re still not there yet. Even if Apple is willing to take a hit with a bulky device in service of pushing use cases for its AR glasses yet to come, the short term may not look very bright.
And perhaps most importantly for the industry as a whole are the (metaphorical) optics.
After all, if the iconic Apple can’t manage to make MR something that everybody wants, the rest of the world watching from the sidelines may think the concept just can’t be conquered. In turn, it may mean capital investment in the space will dry up until ‘real’ AR headsets are a thing—the all-day glasses that will let you play Pokémon Go in the park, do turn-by-turn directions, and remind you the name of that person you met last week. The steppingstone of mixed reality may get waterlogged. Those are a lot of ifs, coulds, shoulds, and won’ts though. The only thing truly certain is we’re in for a very interesting few months, which you can of course follow at Road to VR.
Apple’s entrance into XR has the potential to expand the industry by demonstrating its viability, just as Apple has done with previous technologies. It stands a good chance at carving out a sizeable claim in the space, but it’s a gamble that could equally backfire if both sales and public perception aren’t on their side.
Is Apple’s XR headset going to be the “one more thing?” we’ve all been waiting for at WWDC this year? Will it live up to the Apple name, or be an expensive dev kit? Let us know in the comments below!
Details about Meta Quest 3 were leaked to The Verge.
Alex Heath reportsMeta’s VP of VR Mark Rabkin told staff in an internal presentation that Quest 3, launching later this year, will be two times thinner and at least twice as powerful as Quest 2 – but cost “a bit more.” CEO Mark Zuckerberg previously said Quest 3 will be “in the price range of $300, $400, or $500, that zone”, suggesting $500 as the likely entry price.
“We have to prove to people that all this power, all these new features are worth it” Rabkin reportedly said.
Rabkin also reportedly said Quest 3 will have a “Smart Guardian”. The report doesn’t elaborate, but this could potentially involve automatically seeing a 3D outline of furniture, people, and pets when you get close to them. Currently, Quest 2 and Quest Pro let you arduously manually mark out furniture to see a basic rectangular bounding box when near, and you can enable a crude 2D outline feature that doesn’t distinguish between kinds of objects. Late last year, leadership at Meta said it was a goal to make room scanning an automated process for mixed reality.
Apparent schematics leaked to YouTuber SadleyItsBradley
In September,apparent schematics of Quest 3were leaked to YouTuber SadlyItsBradley. Unlike Quest Pro, the headset depicted didn’t have either eye or face tracking. It did, however, include pancake lenses to achieve a slim design, and a depth sensor for advanced mixed reality.
Mixed reality will reportedly be a major selling point of Quest 3. “The main north star for the team was from the moment you put on this headset, the mixed reality has to make it feel better, easier, more natural,” Rabkin reportedly said. “You can walk effortlessly through your house knowing you can see perfectly well. You can put anchors and things on your desktop. You can take your coffee. You can stay in there much longer.”
Rabkin also reportedly said there will be 41 new apps and games shipping for Quest 3. Meta has acquired 8 game studios over the past 3 years but has only vaguely teased 1 new game from them. We’ve been speculating for a while now that Meta has been directing these studios to build content for Quest 3.
Microsoft technical fellow and mixed reality figurehead Alex Kipman is reportedly leaving the company in the wake of misconduct allegations, leading to a reorganization in its MR efforts.
Late last month Insider reported [paywall] on claims of inappropriate behavior from Kipman, who heads up Microsoft’s HoloLens and mixed reality work. One alleged incident included Kipman watching an “overtly sexualized pillow fight” in VR in front of staff, with another claiming the executive rubbed the shoulders of a female employee as she “looked deeply uncomfortable”.
Another article from Insider today now says that Kipman has resigned from the company. GeekWire has since published an apparent internal email from Scott Guthrie, the head of Microsoft’s Cloud & AI Group, both announcing the departure and a shake-up for the wider mixed reality team. Microsoft has not responded to either article.
The reported email explains that Kipman will remain at Microsoft for two more months to help with the transition, but doesn’t make note of the allegations against him. “Over the last several months, Alex Kipman and I have been talking about the team’s path going forward,” it reads. “We have mutually decided that this is the right time for him to leave the company to pursue other opportunities.”
The rest of the email, meanwhile, explains that Microsoft’s Mixed Reality Hardware teams are being integrated into the Windows + Devices organization. “This move will integrate our Mixed Reality hardware teams as part of Microsoft’s broader end-user device hardware organization,” the email reads. “Both HoloLens and IVAS are built using Windows, and this move further aligns our client platform efforts.”
It also states that the Mixed Reality Presence and Collaboration teams will join the Teams organization.
Earlier this year another report from Insider claimed a HoloLens 3 device had been canceled in favor of a partnership with Samsung, and that Microsoft’s mixed reality efforts were plagued with “confusion and strategic uncertainty”.
Quest developers can now use your walls & furniture from Room Setup.
Room Setup shipped last week as an experimental feature. It lets you mark out your walls, doors, windows, and furniture from inside Quest’s real world passthrough view, using the VR controllers.
With the release of v40 of the Quest SDK developers can now access these walls, doors, windows – as well as your couch and desk – to build mixed reality applications. Marking your couch was added to the Quest OS in February 2021, and marking your desk in April 2021, but developers haven’t been able to access these until now.
Quest 2’s passthrough view is low resolution and black & white – it was originally only intended for setting up your Guardian boundary. So while apps using this new scene understanding will run on Quest 2, it’s clear the real purpose of the new functionality is for Project Cambria, Meta’s upcoming “high end” headset with high resolution color passthrough. From a practical perspective, Quest 2 seems to be a low end development kit for mixed reality, not the ideal target device.
Having to mark out walls, doors, windows, couches, desks and furniture manually is a fairly arduous and imprecise process. Last week Meta’s CTO explained why this isn’t yet automatic, saying “Segmentation is getting better all the time but still has error. The risk of getting it wrong is a concern as it relates to how people can safely navigate a physical space.” Consulting CTO John Carmack went into more detail on Twitter, saying “There are a good number of smart people at Meta working on understanding the world from camera images, but most of it isn’t production ready.”
But will this always be the case for Project Cambria? It will feature much higher resolution cameras and active depth sensing, which Meta says “helps reconstruct surfaces with higher fidelity and accuracy”. In the announcement of the v40 SDK, Meta Product Manager Rangaprabhu Parthasarathy hinted “in the future, devs will be able to add more bells and whistles to their apps that are only available on this device”.
Lynx, the French XR hardware startup known for crowdfunding the Lynx R-1 mixed reality headset, today announced it’s secured $4 million in its Series A round, led by social VR platform Somnium Space.
Alongside Somnium Space, participants in the latest funding round also include what Lynx calls early supporters of the company and “other investors involved in the AR/VR field such as ex-Meta and Google engineers.”
This brings the company’s total outside funding to $6.8 million, according to Crunchbasedata, following the R-1 headset’s $800,000 Kickstarter campaign back in late 2021 and a seed round of $2 million in early 2019. As a part of the deal Artur Sychov, founder & CEO of Somnium Space, is joining the company’s board of directors.
Built on Qualcomm’s XR2 chipset, Lynx R-1 combines high quality cameras and virtual reality displays to achieve passthrough AR in addition to standard VR (aka, mixed reality), making it an early pioneer of the category. The headset also ditches the standard Fresnel lenses for a novel optic called a “four-fold catadioptric freeform prism,” which is said to slim down the size of Lynx R-1 seemingly beyond what current Fresnels can do.
“At Somnium Space we truly believe in the future of open and decentralized Metaverse which empowers its users. This includes, not only software, but also very importantly hardware,” says Artur Sychov, Founder & CEO of Somnium Space. “The Lynx team led by Stan has created an extraordinary AR / VR device (Lynx-r) with openness in mind which will change and revolutionize the way we all think and interact with this market category. I am very happy to support this company and believe that together we will push the boundaries and potential of the VR/AR industry forward.”
Founded in 2019, Lynx has high ambitions for its R-1 headset too. Priced at $600 for its consumer version, the company is looking to lead the way into the same product category that many established players are soon to enter, including Meta with Project Cambria and Apple with its rumored headset, reportedly code named N301.
“We have this opportunity right here to create the European Champion of Mixed Reality with our work at Lynx, supported by a vibrant community of users and developers desperate to see alternatives to Big Tech companies products and their closed ecosystems.” says Stan Larroque, founder and CEO of Lynx. “What’s the point of creating a European Metaverse if the underlying platform, the door we use to access it, remains in the hands of the same big players with their damaging business models?” he adds.
Since finalizing the design of Lynx R-1, the company has also completed an office expansion in Paris, now at more 200 square meters of R&D, and established a new office in Taiwan.
Apple is ramping up development of the operating system for its headset, with a plan to launch in 2023, Bloomberg’s Mark Gurman reports.
In 2021 Bloomberg, The Information, and supply chain analyst Ming-Chi Kuo released reports claiming Apple is preparing to release a premium headset for VR and AR with high resolution color passthrough. Recent notes from Kuo claim this headset will weigh significantly less than Meta’s Quest 2, feature dual 4K OLED microdisplays, and use a new chip with “similar computing power as the M1 for Mac”.
Apple has in recent weeks ramped up development of realityOS (rOS), its operating system that the headset will run, and previewed the headset to the board of directors, Gurman reports. The board of directors includes CEO Tim Cook. References to realityOS were found in App Store upload logs and Apple code earlier this year.
Separately, The Information released the first half of a report detailing the history of the project. The product was apparently originally supposed to launch in 2019 but this has slipped several times. Early prototypes in 2016 used jury-rigged HTC Vives and software running on Microsoft Windows, with one being so heavy it had to be suspended by a small crane. The report also cites five sources revealing Tim Cook rarely visits the team working on the headset – a stark difference to Meta where some employees are reportedly frustrated at Zuckerberg’s obsession with VR and AR.
The lack of a top level Apple executive championing the project has apparently made it harder to get engineering staff & resources allocated compared to the iPhone and Mac. To get support for the project, team members apparently warned that companies like Facebook and Magic Leap could end up owning the sector.
Both Gurman and The Information report Apple’s headset is set to be priced north of $2000. It should end up competing with Meta’s Project Cambria, slated to launch later this year for “significantly” above $800. If Kuo’s notes are to be believed, though, Apple’s product will have higher resolution, a more powerful processor, and slimmer design.
Project Cambria has a depth sensor and cameras with 3 times the resolution of Quest 2’s.
Cambria is the public codename for Meta’s upcoming high end standalone headset, announced at Connect 2021 in October. It will be sold alongside Quest 2 with a price tag “significantly” higher than $800, aimed at remote workers and mixed reality enthusiasts. The headset looks to have a more balanced design than Quest 2 with a slimmer visor achieved through the use of pancake lenses instead of fresnel lenses. It also has built-in face and eye tracking to drive avatars in social experiences like Workrooms.
But Cambria’s headline new feature is high resolution color passthrough for mixed reality – Quest 2’s passthrough is grainy black & white. And today in a conversation with Jesse Schell(of Schell Games), Meta CEO Mark Zuckerberg revealed some of the hardware behind this mixed reality functionality.
Zuckerberg said Quest 2’s passthrough is based on “sensors that were not designed to give you anything more than just a very rough outline of what’s going on around you”.
Cambria, on the other hand will have “a bunch of new sensors” including “high resolution color outward facing cameras” as well as a dedicated depth sensor. “Right now on Quest 2 we hack it a little bit by looking at the cameras and trying to intuit what the depth is”. Zuckerberg told Protocol that this sensor is an IR projector for active depth sensing, and also said the regular cameras have three times the resolution of Quest 2’s.
Hardware level depth sensing is also “more optimized towards hands” Zuckerberg told protocol, though Meta has dramatically improved hand tracking even on Quest 2 by leveraging recent advances in computer vision.
Project Cambria still doesn’t have a product name, specific release window, or exact price; but Zuckerberg re-iterated it will launch “later this year”.
Meta teased its next virtual reality (VR) headset, codenamed Project Cambria, at last year’s Connect conference. In the following months, the company has released further snippets such as its enterprise, rather than gaming focus, and that it won’t be cheap. Today, CEO Mark Zuckerberg has showcased Project Cambria in action, or at least the mixed reality (MR) part.
In the promo video, the actual headset has been blocked out so there’s no cheeky reveal of its physical aesthetics. What Zuckerberg is demonstrating is the full-colour pass-through cameras the enterprise headset will feature, rather than the black and white experience you get on the Meta Quest.
To demonstrate Project Cambria’s mixed reality (MR) capabilities Meta created an app called The World Beyond using its Presence Platform. The platform is a set of machine learning and AI tools designed to help developers create MR experiences that utilise natural hand and voice interactions. The World Beyond looks to be a virtual pet simulator, where you can throw toys and pet your digital companion – which looks very similar to Oppy, who made an appearance at Connect 2021.
Better yet, this might be a Presence Platform demo but The World Beyond is coming to Meta Quest’s App Lab very soon, Zuckerberg mentions. The app will be open source so developers can use it to help them build their own experiences for Project Cambria,
That’s it in terms of Project Cambria news, unfortunately. Meta did release an interview between Zuckerberg and Schell Games’ CEO Jesse Schell where they discussed all things VR but there was no accidental slip of specs, price or availability.
Those details are likely to arrive during Connect 2022 later this year. Meta has confirmed Project Cambria will arrive in 2022, keep it on track to release a reported four devices by 2024 if rumours are to be believed. As further details regarding Cambria are released, gmw3 will keep you updated.
