We had previously seen photos of Magic Leap 2, but this new video gives a full 360 degree overview. Plus, it gives a clearer look at the headset’s accompanying controllers. As reported earlier this month, the controllers feature cameras on the sides, used for onboard inside-out tracking.
We had seen some unofficial pictures of the controllers at the time, but this new video gives us our first official look. The two cameras are present on the sides, but you can also see what looks to be a trackpad on the top of the controller.
This week, we’re going to run through our list, in no particular order, of existing Apple features that we’d love to see support on the company’s mixed reality headset. Apple is all about parity and integration across its ecosystem of devices, so it’s fair to expect that it will leverage many existing features (and the familiar branding behind them) to bolster the user experience of its headsets.
Keep in mind — some of the features listed below are fairly safe bets, while others might be further down the pipeline or simply more speculative/hypothetical in nature. Here’s our list:
AirDrop
AirDrop is one the best features across Apple’s ecosystem and it would make perfect sense on a headset.
People mostly use AirDrop to share photos between phones, but its functionality extends well beyond that – you can use it to send links to a secondary device, share contacts, send files between devices, and much more. Integrating AirDrop into Apple’s headset would allow users to quickly share content with each other and between their existing Apple devices and the headset. This would come in handy when trying to send your headset a link from your phone, for example, or when trying to quickly transfer a VR screenshot or video recording across from the headset to another device.
iCloud
iCloud support seems like a no-brainer, if not near guaranteed, inclusion on an Apple headset. Like other Apple devices, this would seamlessly sync content between all devices as well as back up your headset to the cloud in case it needs to be reset or you upgrade to a new headset in the future.
Likewise, this would allow system-level integration with iCloud Files, allowing you to access the same files from your headset, phone and computers at all times. It would also sync your VR screenshots, videos and app data across all devices, providing another easy way to access content you create in VR from another device at any time.
Sidecar
Sidecar is one of Apple’s recent features allowing an iPad to operate as a mirrored or second display for a Mac computer. It works wirelessly and remarkably well, in my experience, providing users with an easy two-monitor setup while on the go.
We’d love to see Sidecar’s functionality extended with new features for the mixed reality headset. Instead of using another device as a second monitor for a computer, it would be awesome to see Sidecar add support for using an iPad, iPhone or other Apple device while in mixed reality. Perhaps something similar to Horizon Workrooms’ remote desktop, allowing iPads and iPhones to be tracked, represented and usable in mixed or virtual reality.
Pushing the idea even further, it would be cool to see Sidecar allow an iPad or iPhone to work as customizable peripheral accessory for mixed reality — a physical device that you could pick up and interact with, tracked by the headset, displaying some kind of custom content while using the headset.
FaceID
FaceID remains one of the most reliable and fast methods of face-recognition on the smartphone market. As VR avatars get closer to photo-realism, user authentication and authorization is going to be increasingly crucial. While we don’t know what sensors to expect in Apple’s first-generation headset, it would be great to one day see FaceID adapted for VR using face tracking sensors to verify the owner of the headset. It would be equally useful as a way recognize different users on one headset, allowing the headset to automatically switch profiles for each.
iMessage
Apple’s now-infamous blue bubble iMessage system is standard among Apple users. Much like how users can send Facebook Messenger messages on Quest 2, it would only make sense to see iMessage supported on Apple’s headset.
Facetime & Memoji Support
On existing Apple devices, Facetime now supports audio and video calls. Being able to accept audio Facetime calls while using Apple’s headset would be great, but it would also be fantastic to see Facetime expanded with additional made-for-VR functionality. One option would be to add a new option for VR calls, allowing headset users to talk and interact with each other on call in 3D virtual space with personal avatars. Apple’s Memoji system seems like a natural system to use for VR avatars in these instances, akin to Meta’s recently updated avatar styles.
SharePlay
SharePlay is a newer feature, only recently launched as part of iOS 15. Tied together with Facetime, it lets users sync up video and audio content with each other, so they can watch/listen together at the same time. The obvious next step for SharePlay would be allowing headset users to join a SharePlay session together in VR cinemas or home environments, similar to Horizon Home.
AirPlay with VR Casting Support
One of Quest 2’s best features is the ability to cast your view from VR onto a computer, TV or other Chromecast-enabled device, so that others can follow along. It would be remiss of Apple not to include a similar feature at launch for its own headset, and AirPlay would be the obvious way to do it.
AirPlay works similarly to Google Cast, allowing you to share your screen or content with other AirPlay-enabled devices. Being able to seamlessly share your view in VR to a Mac computer, iPhone, iPad, Apple TV or other device would be fantastic.
2D iOS App Support
One of Meta’s big 2021 Connect announcements was expanded support for 2D apps, like Instagram and Dropbox, coming to Quest 2. However, the app selection is still quite small and still expanding. Apple has a slam dunk opportunity to one-up Meta instantly here, by adding support to run all, or at least most, existing iOS and iPad OS apps in 2D on its headset.
This could even work similarly to how iOS app support worked on the iPad at launch. Some apps had iPad-specific designs and features at launch, but many didn’t. To this day, iOS apps that don’t have iPad-specific support can still be run on the system — instead of a native iPad app, you simply use the app as it’s designed for iOS, but scaled up and enlarged to fit as much of the iPad’s screen as possible. Developers can choose to add support for a native iPad version of their iOS apps, which will automatically run instead of the iOS version, once implemented.
A similar approach could be taken for 2D iOS and iPad OS apps on Apple’s headset — supported at launch, but mostly running the same iPhone and iPad versions you’re used to. Developers could then choose to add headset-native versions of the apps over time, which would take full advantage of the platform.
Apple Wallet/Apple Pay
Entering details like a card number while in VR is a huge hassle and switching quickly between real life and VR to enter some text into your headset is never fun. If implemented, Apple Pay would remove the need to enter any card details in your headset and would use automatically suggest cards that are already stored in your Apple Wallet. Having this connected functionality in VR would be a huge time saver, allowing new headset owners to purchase experiences in a hassle-free way just by linking their Apple account.
What features do you want to see on Apple’s upcoming headset? Let us know in the comments below.
This year has seen some absolutely incredible games release for VR and AR platforms, alongside a smattering of new hardware releases as well.
As the year comes to a close, we’re looking back at some of the biggest stories and releases of 2021 and we’ve compiled all of our reviews from 2021 in one place.
Read on for the breakdown of UploadVR’s 2021 game and hardware reviews – we’ve highlighted some of the biggest releases in bold and included video reviews for some titles as well.
January – March
Mare Oculus Quest Review: A Breathless Beauty Of A VR Game Read Here
Battlescar Review: An Explosive Must-See Read Here
Hitman 3 VR Review – A (Mostly) Clean Kill Read Here
Yupitergrad Review: Innovative VR Platforming That’s As Much Hard Work As It Is Fun Read Here
Contractors Quest Review: VR Gets An Approachable Call of Duty-Style Shooter Read Here
Ryte: The Eye of Atlantis Review: Watered-Down Myst Read Here
Vanishing Grace Review: A Cozy Firewatch-Lite To Drink In Read Here
Terje Haakonsen’s Powder VR Early Access Review: A Fine Winter Sports VR Game Read Here
A Wake Inn Review: Tedious Pacing Overshadows Immersive Design Read Here
Angry Birds VR: Isle of Pigs Review – A Shooting Success Read Here
Crashland Review: A Deliciously Creepy If Overly Tough VR Critter Killer Read Here
The Climb 2 Review: Quest Gets A Real Cliffhanger Read Here
Flow Weaver Review: Clever Time-Weaving Doesn’t Escape Boredom Read Here
Hyper Dash Review: Dynamic And Intense Competitive VR Shooter Read Here
Cosmodread Review: Roguelike VR Horror Where No One Can Hear You Scream Read Here
Doom 3: VR Edition Review – A Serviceable Port Of A Game Never Intended For VR Read Here
Zombieland VR: Headshot Fever Review — Slightly Off-Target Read Here
April – June
Floor Plan 2 Review: A Henson-Esque Marvel Read Here
Hand Physics Lab Review: Experiments In Joy And Frustration At Your Fingertips Read Here
Carly And The Reaperman Review – VR’s Best Asymmetrical Co-Op Overcomes Its Scrappy Side Read Here
Alvo PSVR Review: Surprisingly Rewarding Barebones Shooter Read Here
Maskmaker Review – Wonderfully Intricate But Familiar VR Fairy Tale Read Here
Wraith: The Oblivion – Afterlife Review – Solid Survival Horror For VR Read Here
Star Wars Pinball VR Review: Making the Bump(er) to Hyper Space Read Here
The Wizards: Dark Times Review – Action-Packed Magical RPG Adventure Read Here
Demeo Review – A Social VR Masterclass In An Engaging Tabletop RPG Read Here
Zero Caliber: Reloaded Review – Fantastic Gunplay Held Back By Design Issues Read Here
Solaris: Offworld Combat Review – Sci-Fi Quake For The VR Age Read Here
Puzzle Bobble VR Review: A Fun If Unnecessary Take On A Classic Read Here
Carve Snowboarding Review – A Thrilling Take On An Addictive Sport Read Here
ForeVR Bowl Review: A Great Take On A Sport That Isn’t Ready For VR Read Here
Stones Of Harlath Review: Novel But All-Too-Brief Dungeon Crawling Throwback Read Here
Larcenauts Review: A Slick, Rich Shooter For Competitive Play Read Here
Sentenced Review: A Brilliantly Bloody Bit Of VR Theater Read Here
July – September
Chess Club Review – Almost All You Could Ask For From VR Chess Read Here
A Rogue Escape Review: Brilliantly Stuffy, Mercilessly Tough Puzzler Read Here
Sniper Elite VR Review: Old Dog, New Tricks Read Here
Sam & Max: This Time It’s Virtual Review – Light On Interaction, Heavy On Hilarity Read Here
Google is hiring in several positions working on an ‘Augmented Reality OS’ for an AR device, as reported by 9 To 5 Google.
There’s a variety of open positions involved in building the software for this AR device, with the listing for Senior Software Developer, Embedded, providing a description of the team working on the OS:
Our team is building the software components that control and manage the hardware on our Augmented Reality (AR) products. These are the software components that run on the AR devices and are the closest to the hardware. As Google adds products to the AR portfolio, the OS Foundations team is the very first software team to work with new hardware.
The full job listing is here. Likewise, postings for Senior Software Engineer, Camera and Senior Software Developer Tech Lead, Input describe building software for an “innovative AR device.” The listings say that this OS team “is the very first software team to work with new hardware.” The nature of this device is unclear, but the operating system and new hardware angle seems to imply a dedicated AR device, such as a headset. Apple is working on a still-unannounced VR/AR mixed reality headset while Meta is preparing a higher end headset code-named Cambria, and we’d expect Google to be very interested in the AR product category.
A new update for Mario Kart Live: Home Circuit brings splitscreen mulitplayer functionality, allowing players to use two physical karts with one Nintendo Switch console.
Mario Kart Live: Home Circuit released last year and brings the famous arcade racing franchise to your home, using a camera-equipped toy kart to turn your home into an AR Mario Kart course on your Switch.
Using a Nintendo Switch console, players can connect to the kart and control it using the Switch, mapping out an AR-powered course around your home using cardboard gates that can be placed around you on the ground.
One Home Circuit kit comes with the game, the kart and the required gates to map the course. Multiplayer was supported at launch, but only with multiple Switch consoles — each player would have to use their own Switch along with their own physical kart.
Now, with a free update, splitscreen multiplayer is supported, allowing two physical karts to compete against each other using just one Switch system. This means that a family with just one console can purchase two karts and play on the same system. In addition, the base Home Circuit kit is now discounted for Black Friday at multiple retailers, bringing the price of one kit down to $59.99 from $99.99.
There’s also some other additions in the 2.0 update, including a new Relay Race multiplayer mode, which allows 2-4 players to use one kart, taking turns controlling the kart via mid-race switches.
There’s a new Luigi Cup added to the Grand Prix as well, along with some new cosmetics and customization unlocks.
An update to Horizon Workrooms on Oculus Quest 2 adds the ability to overlay AR labels onto your tracked keyboard when using passthrough mode.
The feature is one of a few added in the Workrooms 1.1 update. As previously reported, the update includes support for M1 Mac models, but also introduces the AR keyboard labels, quicker connection to your computer via Oculus Remote Desktop, and “more intuitive whiteboard design” that lets you use your hands more.
You can see some photos of the AR keyboard labels spread throughout the article — on a flat 2D photo, they look hard to distinguish from the normal light-up keys on the MacBook Pro. However, the depth perception in VR makes the labels look just slightly elevated from the keys themselves.
This only applies to tracked keyboards (which currently only includes the Logitech K830 and Apple Magic Keyboards and laptop keyboard) when using the desk passthrough cut-out. If you’re just using the standard tracked keyboard in Workrooms without passthrough there’s no need for the AR labels — a 1:1 digital model represents and displays your keyboard in VR, matched to the position of your physical keyboard.
It’s only when using the passthrough desk cut-out feature that the AR labels come to life. This makes the passthrough cut-out feature much more useable, as you retain full visibility of your keyboard keys while also being able to see other desk accessories such as your mouse, drinks etc.
Over the last week or so, AR mobile game developer Niantic has opened one door and closed another – Pikmin Bloom is now available, but 2019’s Harry Potter: Wizards Unite will be shut down in 2022.
Back in March, Niantic announced it was working on a Pikmin mobile AR title in partnership with Nintendo. As of last week, Pikmin Bloom began rolling out to several countries — it’s now available in almost all major markets, including Europe, Asia Pacific, Africa, Middle East, the US, Canada.
Like many other mobile AR games, Pikmin Bloom follows the rough framework and precedent set by Pokemon Go, Niantic’s first and most successful mobile AR title. However, Bloom takes a slightly different approach to the concept — Bloom is more idle, focused more on simply walking around than actively visiting locations or searching for something. The more you walk, the more pikmin you’ll obtain.
Overall, the game seems to markets itself more as a glorified pedometer with a few game elements thrown in. There’s also minimal AR content compared to other Niantic titles. In fact, the only true use of mobile AR appears to be when you send Pikmin to fight mushrooms, which can be viewed as an AR overlay similar to catching Pokemon in Pokemon Go.
In the same week, Niantic also announced that it will be shutting down its 2019 mobile AR game, Harry Potter: Wizards Unite.
Wizards Unite, the studio’s first game to follow in Pokemon Go’s footsteps, announced that it will close on January 31, 2022 in a post on the game’s website. The game will be removed from store fronts even earlier, from December 6 2021.
We weren’t huge fans of Wizards Unite, but it’s just one game of many — Niantic has announced and released a few IP-focused mobile AR titles in the wake of Pokemon Go, but so far none have managed to catch on in the same way.
Let’s hope that changes with Transformers: Heavy Metal, the next Niantic AR game set to release sometime this year.
A demo video of Nazare showed some familiar AR experiences, like communicating with friends in virtual windows and even playing multiplayer with avatars appearing in the user’s living room. There was no actual picture of the hardware itself, but expect more information in the future.
Speaking about Project Nazare, Meta CEO Mark Zuckerberg described the glasses as the company’s “first full augmented reality glasses,” but also indicated that they are still a work-in-progress:
“There’s a lot of technical work to get this form factor and experience right. We have to fit hologram displays, projectors, batteries, radios, custom silicon chips, cameras, speakers, sensors to map the world around you and more into glasses that are about 5mm thick. So we still have a ways to go with Nazare, but we are making good progress.”
To watch the full segment from today’s keynote, check out the video embedded above — if it doesn’t start in the right place automatically, skip to 1:07:40.
True AR is shaping up to be a competitive market – Microsoft and Magic Leap already have full but compromised AR devices like HoloLens, which are bulky and have limited field of view. We also know that other companies like Apple are working on AR devices as well.
Developers interested in experimenting with mixed reality on Oculus Quest 2 can now use RoomMapper with Unity to explore designs mapped to your physical surroundings.
Developer Bob Berkebile just released RoomMapper for free and Unity developers can use it with Oculus Quest 2 as a $299 standalone AR and VR development kit with software that’s given “an extremely optimized model of the room.” Berkebile recognizes Facebook will likely make the “solution obsolete with their advancements to public access of Passthrough,” but the RoomMapper solution may help people explore mixed reality concepts and get started now with AR applications on Quest 2 that incorporate more about a physical environment. Videos Berkebile posted on Twitter have been shared widely showing virtual zombies moving down a physical hallway, or seeing things in the mirror which aren’t actually there.
Berkebile shared the following walkthrough video giving an overview of how the mapping system works, and the second half of the video really shows off how it can offer a kind of mixed reality playground.
“I immediately noticed that the experimental access to the passthrough composition on Quest 2 is extremely barebones; Oculus is taking this path carefully and I appreciate that,” Berkebile wrote to UploadVR. “I wanted to accelerate the communities’ ability by brining a solution for environmental mapping to them now. The solution presents a solid UX flow that walks a user through the necessary steps to measure, map, and align an extremely optimized model of the room. From there colliders are added, materials can be swapped, and suddenly your room has physicality within the AR/MR that the passthrough solution offers.”
Facebook is due to announce updates for its AR and VR technologies on October 28 with a keynote from Mark Zuckerberg and more planned throughout the day.
The United States Army says it recently conducted a test of its Microsoft HoloLens-based AR platform and will move further testing of the hardware to 2022.
The Integrated Visual Augmentation System or IVAS is based on Microsoft’s HoloLens AR technology and the contract to supply the United Stated military with the technology represents a major vote of confidence in Microsoft’s platform and a key testing ground for the company to develop ruggedized AR tech. A report from Janes suggested the Army is “essentially doing a reset of” the program to figure “out what is the appropriate timeline and where is the technology.”
“The Army is fully committed to its partnership with Microsoft to advance specific technologies to meet operational requirements and maximize warfighter impact,” a press release from the Army states. “The Army conducted an Adversarial Electronic Warfare and Cybersecurity Test in September 2021, and plans to execute testing regularly throughout FY22. This decision allows the Army and Industry team to continue to enhance the IVAS technology platform ensuring Soldiers achieve overmatch in Multi Domain Operations. The Army intends to continue developing and fielding this revolutionary, first-of-its-kind technology in FY22.”
The U.S. Army provided a project timeline with May 2022 listed as the month for an operational test and September of 2022 for the first unit equipped with the hardware. Here’s the full timeline as provided by the Army:
Project Timeline:
OCT / NOV 2020: Soldier Touchpoint 3 (STP 3) & Distinguished Visitor Days
DEC 2020: Rapid Fielding Decision
JAN 2021: Vehicle Integration VE2 – Stryker & Bradley
FEB 2021: Cold Weather Test
MAR 2021: Production Award
MAR 2021: Tropical Weather Test
APR 2021: Soldier Touchpoint 4 (STP 4)
JUL 2021: OT Entrance Criteria User Jury
SEP 2021: Vehicle Integration VE3 – Bradley
SEP 2021: Adversarial Electronic Warfare & Cybersecurity Test
