Quest 2 Can Now Label Keyboards With AR In Passthrough

An update to Horizon Workrooms on Oculus Quest 2 adds the ability to overlay AR labels onto your tracked keyboard when using passthrough mode.

The feature is one of a few added in the Workrooms 1.1 update. As previously reported, the update includes support for M1 Mac models, but also introduces the AR keyboard labels, quicker connection to your computer via Oculus Remote Desktop, and “more intuitive whiteboard design” that lets you use your hands more.

quest ar keyboard label

You can see some photos of the AR keyboard labels spread throughout the article — on a flat 2D photo, they look hard to distinguish from the normal light-up keys on the MacBook Pro. However, the depth perception in VR makes the labels look just slightly elevated from the keys themselves.

quest ar keyboard label

This only applies to tracked keyboards (which currently only includes the Logitech K830 and Apple Magic Keyboards and laptop keyboard) when using the desk passthrough cut-out. If you’re just using the standard tracked keyboard in Workrooms without passthrough there’s no need for the AR labels — a 1:1 digital model represents and displays your keyboard in VR, matched to the position of your physical keyboard.

It’s only when using the passthrough desk cut-out feature that the AR labels come to life. This makes the passthrough cut-out feature much more useable, as you retain full visibility of your keyboard keys while also being able to see other desk accessories such as your mouse, drinks etc.

In a future update, Workrooms will get support for Zoom conference call integration and room customization.

Niantic Launches Pikmin Bloom, Harry Potter: Wizards Unite To Shut Down

Over the last week or so, AR mobile game developer Niantic has opened one door and closed another – Pikmin Bloom is now available, but 2019’s Harry Potter: Wizards Unite will be shut down in 2022.

Back in March, Niantic announced it was working on a Pikmin mobile AR title in partnership with Nintendo. As of last week, Pikmin Bloom began rolling out to several countries — it’s now available in almost all major markets, including Europe, Asia Pacific, Africa, Middle East, the US, Canada.

Like many other mobile AR games, Pikmin Bloom follows the rough framework and precedent set by Pokemon Go, Niantic’s first and most successful mobile AR title. However, Bloom takes a slightly different approach to the concept — Bloom is more idle, focused more on simply walking around than actively visiting locations or searching for something. The more you walk, the more pikmin you’ll obtain.

Overall, the game seems to markets itself more as a glorified pedometer with a few game elements thrown in. There’s also minimal AR content compared to other Niantic titles. In fact, the only true use of mobile AR appears to be when you send Pikmin to fight mushrooms, which can be viewed as an AR overlay similar to catching Pokemon in Pokemon Go.

In the same week, Niantic also announced that it will be shutting down its 2019 mobile AR game, Harry Potter: Wizards Unite.

Wizards Unite, the studio’s first game to follow in Pokemon Go’s footsteps, announced that it will close on January 31, 2022 in a post on the game’s website. The game will be removed from store fronts even earlier, from December 6 2021.

We weren’t huge fans of Wizards Unite, but it’s just one game of many — Niantic has announced and released a few IP-focused mobile AR titles in the wake of Pokemon Go, but so far none have managed to catch on in the same way.

Let’s hope that changes with Transformers: Heavy Metal, the next Niantic AR game set to release sometime this year.

Meta Announces AR Glasses Prototype Project Nazare

Alongside the reveal of the new Project Cambria VR headset, Meta (formerly Facebook) just gave a codename to its first pair of consumer AR glasses. Meet Project Nazare.

A demo video of Nazare showed some familiar AR experiences, like communicating with friends in virtual windows and even playing multiplayer with avatars appearing in the user’s living room. There was no actual picture of the hardware itself, but expect more information in the future.

Speaking about Project Nazare, Meta CEO Mark Zuckerberg described the glasses as the company’s “first full augmented reality glasses,” but also indicated that they are still a work-in-progress:

“There’s a lot of technical work to get this form factor and experience right. We have to fit hologram displays, projectors, batteries, radios, custom silicon chips, cameras, speakers, sensors to map the world around you and more into glasses that are about 5mm thick. So we still have a ways to go with Nazare, but we are making good progress.”

A Connect blog post says the Project Nazare glasses are “are still a few years out.

To watch the full segment from today’s keynote, check out the video embedded above — if it doesn’t start in the right place automatically, skip to 1:07:40.

Last month Facebook also released a pair of glasses in partnership with Ray-Ban named Ray-Ban Stories. However, these are not real AR glasses, and don’t even feature simplistic overlays. The Ray-Ban Stories’ main feature is its built-in camera and microphone for point-of-view photo and video recording.

True AR is shaping up to be a competitive market – Microsoft and Magic Leap already have full but compromised AR devices like HoloLens, which are bulky and have limited field of view. We also know that other companies like Apple are working on AR devices as well.

Enter the metaverse: the digital future Mark Zuckerberg is steering us toward

The company, now rebranded Meta, already has a foothold in the digital world. How far will it go to see it succeed?

Facebook CEO Mark Zuckerberg on Thursday outlined his vision for the future of the social media giant, formalising the company’s focus on the metaverse.

In a presentation at the company’s annual Connect conference, Zuckerberg announced the company is rebranding as Meta and detailed how his company aims to build a new version of the internet.

Continue reading...

Facebook Boosts AR Developers Tools for Forthcoming AR Glasses

Today Facebook announced that it plans to expand the capabilities of its Spark AR platform to give developers more tools to build experiences that leverage existing smartphone-based AR and the company’s upcoming AR glasses.

Facebook has made no secret that its Ray-Ban Stories camera glasses are just its first step toward building full blown AR glasses. But just as the hardware continues to be refined, the software must evolve too.

Facebook’s Spark AR Studio toolset was initially built largely around AR face filters but has been steadily expanding with more features. The latest updates improve the platform’s understanding of things beyond the user’s face, like their entire body and the world around them. The company says it’s “working toward fully-featured AR glasses” and hopes the new capabilities of Spark AR will “illuminate the path to AR glasses.”

The updated toolset will soon support geo-mapped experiences for public spaces which will allow for location-based AR experiences that will be accessible from specific locations. This could enable experiences like a scavenger hunt, guided tour around a city, or even a murder mystery where you have to go find clues in the real world.

Image courtesy Facebook

Facebook says it’s testing this geo-locked AR capability with select partners, but will open it up to everyone in 2022.

Beyond geo-awareness, the company says it’s adding body and hand-tracking to Spark AR Studio which will allow developers to easily anchor AR effects to a person’s body. The capability will support “20 different 2D key points” which can be applied to a single person or multiple people.

Hands can be tracked specifically as well, and Facebook says a hand bounding-box will be the first function of a “multi-phased capability rollout” which will ostensibly add additional hand-tracking features (like, perhaps, finger-tracking) in the future.

Facebook says that body and hand-tracking in Spark AR Studio will be available “soon.”

The company is also building out what it calls a “virtual objects pipeline” for AR, which will make it easy to place virtual 3D objects in the scene which can include text, characters, GIFs, stickers, and more, all of which will support an understanding of depth, occlusion, and plane tracking to convincingly existing in the space. This developer feature will come in 2022, according to Facebook.

The post Facebook Boosts AR Developers Tools for Forthcoming AR Glasses appeared first on Road to VR.

Pikmin Bloom Arrives to Brighten Up Any Walk

Pikmin Bloom

Niantic Labs’ videogames continually focus on getting players out and about and back in March the company announced a new project in conjunction with Nintendo based on its Pikmin franchise. Today, the pair have begun the global rollout of Pikmin Bloom, a colourful augmented reality-capable experience all about building up a squad of Pikmin simply by walking.

Pikmin Bloom

Offering a much more casual, combat free experience in comparison to Niantic Labs’ other AR titles, Pikmin Bloom is all about nurturing these tiny plant-like creatures which have appeared in several Nintendo videogames. To do this players need to walk, finding Pikmin seedlings along their path which can then be picked, adding to the Pikmin entourage. All the Pikmin will follow the player making flowers bloom on their journey.

When players take their daily stroll they’ll need to collect fruits along the way to feed their Pikmin, in turn making beautiful flowers bloom atop their heads. Their petals can then be collected and planted to create even more flowers with each step, unleashing a cascading effect of colour. That’s not the only interactive element, players will be able to utilise their red, yellow, blue and more Pikmin to remove giant mushrooms to gain more fruit.

As you may have realised, fitness is an integral part of the experience. Pikmin Bloom will have basic pedometer functionality, recording the number of steps players take as well as the routes they walked. Building upon that healthy lifestyle image they can add notes and photos to a lifelog, take pics with the Pikmin who can also bring back postcards of the places players have visited, shared with friends in the app. 

Pikmin Bloom

Whilst Pikmin Bloom is a free app Niantic Labs monetizing the title with in-game purchases, selling booster items to enhance the experience. There will also be monthly Community Day events.

Pikmin Bloom is available to download for iOS and Android devices on the App Store and Google Play respectively in Australia and Singapore, to begin with. Other regions will slowly be incorporated as Niantic Labs expands the launch. For continued updates on the latest AR projects, keep reading VRFocus.

RoomMapper For Quest 2 Maps Your Surroundings For Mixed Reality

Developers interested in experimenting with mixed reality on Oculus Quest 2 can now use RoomMapper with Unity to explore designs mapped to your physical surroundings.

Developer Bob Berkebile just released RoomMapper for free and Unity developers can use it with Oculus Quest 2 as a $299 standalone AR and VR development kit with software that’s given “an extremely optimized model of the room.” Berkebile recognizes Facebook will likely make the “solution obsolete with their advancements to public access of Passthrough,” but the RoomMapper solution may help people explore mixed reality concepts and get started now with AR applications on Quest 2 that incorporate more about a physical environment. Videos Berkebile posted on Twitter have been shared widely showing virtual zombies moving down a physical hallway, or seeing things in the mirror which aren’t actually there.

Berkebile shared the following walkthrough video giving an overview of how the mapping system works, and the second half of the video really shows off how it can offer a kind of mixed reality playground.

“I immediately noticed that the experimental access to the passthrough composition on Quest 2 is extremely barebones; Oculus is taking this path carefully and I appreciate that,” Berkebile wrote to UploadVR. “I wanted to accelerate the communities’ ability by brining a solution for environmental mapping to them now. The solution presents a solid UX flow that walks a user through the necessary steps to measure, map, and align an extremely optimized model of the room. From there colliders are added, materials can be swapped, and suddenly your room has physicality within the AR/MR that the passthrough solution offers.”

Facebook is due to announce updates for its AR and VR technologies on October 28 with a keynote from Mark Zuckerberg and more planned throughout the day.

Army Shifts Testing Of HoloLens-Based AR Gear To 2022

The United States Army says it recently conducted a test of its Microsoft HoloLens-based AR platform and will move further testing of the hardware to 2022.

The Integrated Visual Augmentation System or IVAS is based on Microsoft’s HoloLens AR technology and the contract to supply the United Stated military with the technology represents a major vote of confidence in Microsoft’s platform and a key testing ground for the company to develop ruggedized AR tech. A report from Janes suggested the Army is “essentially doing a reset of” the program to figure “out what is the appropriate timeline and where is the technology.”

“The Army is fully committed to its partnership with Microsoft to advance specific technologies to meet operational requirements and maximize warfighter impact,” a press release from the Army states. “The Army conducted an Adversarial Electronic Warfare and Cybersecurity Test in September 2021, and plans to execute testing regularly throughout FY22. This decision allows the Army and Industry team to continue to enhance the IVAS technology platform ensuring Soldiers achieve overmatch in Multi Domain Operations. The Army intends to continue developing and fielding this revolutionary, first-of-its-kind technology in FY22.”

The U.S. Army provided a project timeline with May 2022 listed as the month for an operational test and September of 2022 for the first unit equipped with the hardware. Here’s the full timeline as provided by the Army:

Project Timeline:

  • OCT / NOV 2020: Soldier Touchpoint 3 (STP 3) & Distinguished Visitor Days
  • DEC 2020: Rapid Fielding Decision
  • JAN 2021: Vehicle Integration VE2 – Stryker & Bradley
  • FEB 2021: Cold Weather Test
  • MAR 2021: Production Award
  • MAR 2021: Tropical Weather Test
  • APR 2021: Soldier Touchpoint 4 (STP 4)
  • JUL 2021: OT Entrance Criteria User Jury
  • SEP 2021: Vehicle Integration VE3 – Bradley
  • SEP 2021: Adversarial Electronic Warfare & Cybersecurity Test
  • MAY 2022: Operational Test (IOT)
  • SEP 2022: First Unit Equipped (FUE)

Official Magic Leap 2 Details Released, $500m New Funding Raised

Magic Leap 2

It certainly seems to be hardware season where virtual reality (VR) and augmented reality (AR) is concerned. HTC Vive has a special event planned this week, Lynx-R1 launched a Kickstarter, Canon EOS VR was unveiled and now the first proper look at Magic Leap’s next device. Not only that, the company has managed to raise a further $500 million USD in funding.

Magic Leap 2

Magic Leap CEO Peggy Johnson initially teased the first image on Monday before an interview with CNBC. This has been followed up with further imagery of a much sleeker looking device. In a similar arrangement to the original model, the Magic Leap 2 has a pair of central cameras but this time there are two additional sensors just below them. There’s an additional lens on the corner that looks tilted outwards by around 45-degrees, presumably for a wider capture field.

Actual specifications regarding the Magic Leap 2 haven’t been released just yet, Johnson has mentioned that the: “headset boasts critical updates that make it more immersive and even more comfortable,” increasing its field of view (FoV) – as seen in the image above – vertically rather than horizontally. It’ll also come with a dimming feature that the CEO says is: “a first-to-market innovation that enables the headset to be used in brightly lit settings.”

Johnson is also keen to point out that the Magic Leap 2 aims to achieve the company’s goal of a device suitable for “all day, everyday use”. “All day” would be quite the achievement considering the Magic Leap 1 was good for 3.5 hours of continuous use. That could indicate a much larger battery, improved power consumption or hot-swappable battery for users.

Magic Leap 2

To help bring the Magic Leap 2 to market, Magic Leap has managed to raise an additional $500 million on top of the billions previously raised.

“This investment is an important step in advancing Magic Leap’s mission to transform the way we work,” said Peggy Johnson in a blog post. “With ongoing support from our existing investors, Magic Leap will have greater financial flexibility and the resources needed to continue our growth trajectory as we expand on our industry-leading AR technology.”

Magic Leap 2 is already in select enterprise customer hands via an early access program. Sales of the headset will begin at some point in 2022. For continued updates, keep reading VRFocus.

Magic Leap 2 Teased For 2022 With Taller Field Of View

Magic Leap teased the first images of its next generation AR headset, Magic Leap 2.

A blog post by Magic Leap CEO Peggy Johnson features an image, pictured below, comparing the field of view of the first and second generation AR headsets. While Magic Leap 2 seems to have small gains in horizontal field of view, vertically the augmentation of your vision should be far more significant with the new device. The company is said to have raised another $500 million to roll-out the second generation product focused toward business markets in 2022. “Select customers” are “already leveraging its capabilities through an early access program,” according to the company.


We can also see in the images of the new device that it apparently keeps its wired design. The original Magic Leap One headset shipped starting in August 2018 priced at $2,295 with a single handheld controller and wired computing puck accompanying the lightweight glasses.

magic leap 2

Here’s the first generation device for comparison:

Apple, Facebook, and Microsoft are just a few of the companies working to develop next-generation AR headsets geared toward different use cases. A new generation of VR headsets are also likely to have advanced AR passthrough modes likely to overlap with the feature sets of systems like Magic Leap 2. We’ll be curious to see, then, what price Magic Leap’s second generation system arrives at when it launches, and how it holds up against other products available at the time it ships.

“Magic Leap 2 will be the industry’s smallest and lightest device built for enterprise adoption,” Johnson wrote. “This more advanced headset boasts critical updates that make it more immersive and even more comfortable, with leading optics, the largest field of view in the industry, and dimming – a first-to-market innovation that enables the headset to be used in brightly lit settings, in addition to a significantly smaller and lighter form factor. These updated features lend themselves to achieving our goal of all day, everyday use, which is what the enterprise market has been asking for – a device that you can put on your head in the morning and wear all day long.”

Magic Leap appointed Johnson as CEO in July 2020 after former CEO Rony Abovitz stepped down, citing a need for the company to pivot to a new ‘focused’ direction.