Category: mr

Posted on

Project Cambria Has A Depth Sensor & Much Higher Resolution Cameras

Project Cambria has a depth sensor and cameras with 3 times the resolution of Quest 2’s.

Cambria is the public codename for Meta’s upcoming high end standalone headset, announced at Connect 2021 in October. It will be sold alongside Quest 2 with a price tag “significantly” higher than $800, aimed at remote workers and mixed reality enthusiasts. The headset looks to have a more balanced design than Quest 2 with a slimmer visor achieved through the use of pancake lenses instead of fresnel lenses. It also has built-in face and eye tracking to drive avatars in social experiences like Workrooms.

But Cambria’s headline new feature is high resolution color passthrough for mixed reality – Quest 2’s passthrough is grainy black & white. And today in a conversation with Jesse Schell (of Schell Games), Meta CEO Mark Zuckerberg revealed some of the hardware behind this mixed reality functionality.

Zuckerberg said Quest 2’s passthrough is based on “sensors that were not designed to give you anything more than just a very rough outline of what’s going on around you”.

Cambria, on the other hand will have “a bunch of new sensors” including “high resolution color outward facing cameras” as well as a dedicated depth sensor. “Right now on Quest 2 we hack it a little bit by looking at the cameras and trying to intuit what the depth is”. Zuckerberg told Protocol that this sensor is an IR projector for active depth sensing, and also said the regular cameras have three times the resolution of Quest 2’s.

Hardware level depth sensing is also “more optimized towards hands” Zuckerberg told protocol, though Meta has dramatically improved hand tracking even on Quest 2 by leveraging recent advances in computer vision.

Project Cambria still doesn’t have a product name, specific release window, or exact price; but Zuckerberg re-iterated it will launch “later this year”.

Posted on

Amazon Job Listings Reference ‘New-To-World’ AR/VR Consumer Product

Amazon job listings reference a “new-to-world” AR/VR consumer product.

Spotted by Protocol, one listing explains “You will develop an advanced XR research concept into a magical and useful new-to-world consumer product” while another references “developing code for early prototypes through mass production.”

Another job listing describes the role as working on “the core system interface along with end-user applications spanning from multi-modal interfaces to 3D AR entertainment experiences”.

Protocol also spotted that in March Amazon hired Kharis O’Connell to lead a “Futures Design” group, described as “helping Amazon experience what it’s like to live in the future, today”. O’Connell once worked for the now defunct Meta View startup, and then worked on Google’s AR operating system.

Amazon is the only consumer tech giant with no announced or rumored AR or VR headset product. Meta has its Quest VR line and is working on AR glasses too. Microsoft has its HoloLens AR headsets. Multiple reliable sources claim Apple is working on a mixed reality headset, and The Verge reported Google is too.

The company currently sells “smart glasses” called Echo Frames, but these lack any display system or cameras – the use cases are talking to Alexa, taking calls, and playing music. It’s possible – even arguably likely – that Amazon intends to develop this product line into AR glasses in the long term future.

If you’ve got information to share with UploadVR you can email us via tips@uploadvr.com.

Posted on

Hands-on: Varjo’s Unique Masking Tool Lets You Bring the Real World into VR

Varjo’s XR-3 headset has perhaps the best passthrough view of any MR headset on the market thanks to color cameras that offer a fairly high resolution and a wide field-of-view. But rather than just using the passthrough view for AR (bringing virtual objects into the real world) Varjo has developed a new tool to do the reverse (bringing real objects into the virtual world).

At AWE 2021 this week I got my first glimpse at ‘Varjo Lab Tools’, a soon-to-be released software suite that will work with the company’s XR-3 mixed reality headset. The tool allows users to trace arbitrary shapes that then become windows into the real world, while the rest of the view remains virtual.

This makes it possible to bring parts of the real world into virtual reality. For instance, as the company shows, you can trace the outline of a physical steering wheel peripheral so that you can see the real wheel inside of the virtual world.

The Varjo Lab Tools software supports three core masking modes: static, depth, and marker-based. With the static mode you can draw a passthrough window that will essentially be tracked to your head. With the depth mode you can have the system automatically pull in anything that’s within a certain distant to the headset (judged by the lidar sensor). You can imagine this being used to automatically show your real hands holding something when you raise them up in front of you, without pulling in the background of your room.

In the marker-based mode you can create a window that’s tracked to a simple fiducial marker and moves with the marker. This would be great for something like the steering wheel example above, or allowing your real keyboard to persistently appear in the virtual world. The markers can be easily printed and mounted anywhere… and you can track up to 1,000 of them.

At AWE 2021 I got to try the system for myself, and I was impressed with the user-friendliness of the tool. To draw a marker-tracked mask, for instance, you just use a VR controller to select the marker you want to track, and then trace out the mask with a simple point-and-click system. For example, I used the tool to mask out an area around a gamepad that had one of the markers attached to it. Then when I jumped into VR I could see a window around the gamepad, allowing me to look down and see my real hands on the gamepad and easily reference the position of my fingers with regard to the buttons.

And what’s particularly cool is that all of this passthrough magic is happening completely independent of the VR application, which means the passthrough windows that you draw can function inside of any app without a special integration by the developer.

It’s a very cool system that makes passthrough fundamentally more useful, but it isn’t yet flawless. One of the most obvious limitations right now is the update rate of the masked area. While the view through the passthrough window (ie: the view of the real world) updates at a perfectly fast rate, the shape of the window itself (when attached to a marker) only seems to update a few times per second. So if you’re moving the marker even slightly fast you’ll see the passthrough window lag behind as it tries to keep up.

I’m not sure what the limiting factor is on the update rate for the passthrough window, but I take it this is something that will be improved in the future. A much faster update rate would make real-world objects appear much more seamlessly within the virtual world.

– – — – –

Varjo says that the passthrough masking capabilities can also be used with the chroma key functionality it has introduced previously, making the XR-3’s passthrough system very functional and flexible compared to anything else on the market today. Indeed, just as the impressive resolution of Varjo’s flagship headsets has long felt like a glimpse of the eventual future of consumer VR headsets, we’d also hope to see this kind of advanced passthrough functionality come to consumer headsets, eventually.

For now the capability is only on Varjo’s high-end enterprise XR-3 headset (as it’s the only one with all the passthrough hardware necessary to make it all happen). The company says the Varjo Lab Tools software, which will enable these features, will launch alongside the next major update to its core ‘Varjo Base’ software.

The post Hands-on: Varjo’s Unique Masking Tool Lets You Bring the Real World into VR appeared first on Road to VR.

Posted on

LIV iOS Open Beta Offers Easy Oculus Quest Mixed Reality Capture

LIV’s iOS app is in open beta, offering an easy way to capture mixed reality footage with select games using just an iPhone and an Oculus Quest headset.

An early version of the software was available back in March 2020, but was taken down and improved to make it more reliable and accessible. Now, the new open beta gives anyone with a Quest and an iOS device an accessible way to start capturing mixed reality footage.

The app is available in open beta through Apple’s beta testing platform, Testflight. Users can install the app on their iOS device through Testflight and will then also need to install the LIV Capture app for Oculus Quest, which is available for sideloading through SideQuest.

Once everything is installed, the LIV Capture app on Quest runs the user through a quick calibration process to align everything. Once that’s complete, the LIV app can show a mixed reality view of select Quest games, where the player is overlaid onto a third-person perspective view of the virtual world.

Mixed reality capture has been available in varying capacities across many VR systems for some time now, but the LIV iOS app makes it much more accessible for Quest users. Only a phone and a Quest headset are needed — no additional equipment is required, not even a green screen. The app is able to identify the user against any background and dynamically place them into a third-person perspective of the virtual world, with generally positive results. Users can record the mixed reality view by using iOS’s built-in screen recorder.

For now, only select Quest games support mixed reality with the LIV app — Beat Saber, Crisis VRigade, Cubism, FitXR, Hyper Dash, OhShape, Real VR Fishing, Smash Drums, Space Pirate Trainer, Superhot and Synth Riders.

The LIV Beta app for iOS arrives at a time when there seems to be increasing support and interest in mixed reality capture. Facebook is seemingly trying to build its own answer to LIV’s mixed reality tools, but the features are segmented and don’t quite work together cohesively yet. Live Overlay lets you view a cut-out of the user playing VR (taken from the phone’s camera) on top of the first-person VR view cast to a phone. However, this misses the essential third-person perspective needed for mixed reality.

Meanwhile, its Spectator Camera feature will allow users cast a Quest to a phone and reposition the camera to third-person positions (that offer a different perspective to the VR user’s first-person view), but lacks any implementation of mixed reality features. The building blocks are there, but all remain separate and don’t come together cohesively like in LIV’s iOS app.

Likewise, Fabio Dela Antonio’s Reality Mixer app is another community project offering mixed reality capture on Quest, which Mark Zuckerberg seemingly used in a recent Facebook video showing him playing Beat Saber captured in mixed reality.

LIV’s iOS app is available in open beta now through Testflight. You can read more on the LIV blog.

Posted on

Canon kündigt MREAL S1 an

Canon kündigt eine neue AR-Brille mit der MREAL S1 an. Das Headset für Unternehmen bietet eine AR-Durchgangsfunktion (passthrough) und kann sowohl an einer Kopfhalterung befestigt werden, um die Hände frei zu haben, als auch an einem Gehäuse, das für das Halten in den Händen gedacht ist.

Canon kündigt MREAL S1 an

Canon Mreal S1 Mixed Reality

Im Gegensatz zum MREAL Display MD-20, das Anfang dieses Jahres vorgestellt wurde, könnte man das MREAL S1 technisch als ein VR-Headset mit AR-Sicht beschreiben, welche durch den Einbau von stereoskopischen Kamerasensoren auf der Vorderseite des Geräts erreicht wird, die dem Benutzer eine “gemischte Realität” bieten. Das Headset enthält auch zwei zusätzliche optische Sensoren für die Positionsverfolgung.

Bisher hält sich Canon zu den Spezifikationen bedeckt. Jedoch sagt das Unternehmen, dass es das kleinste und leichteste MR-Gerät in der Geschichte von MREAL sei. Zum Betrieb der MR-Brille soll jedoch auch ein PC notwendig.

Aktuell ist nicht bekannt, wie und wo Canon das MR-Headset zukünftig anbieten wird. Es ist aber davon auszugehen, dass sich das Unternehmen zunächst auf den japanischen Markt konzentrieren wird.

(Quelle: Road to VR)

Der Beitrag Canon kündigt MREAL S1 an zuerst gesehen auf VR∙Nerds. VR·Nerds am Werk!

Posted on

Schedule For Microsoft’s Virtual Mixed Reality Dev Days Revealed

Microsoft revealed the full schedule for their upcoming Mixed Reality Dev Days, which is set to take place virtually in Altspace VR on Thursday and Friday.

Microsoft’s Mixed Reality Dev Days were originally going to be hosted in Seattle this month, but with the ongoing COVID-19 pandemic, the event has been moved online and will take place in VR, using Altspace.

As previously reported, the event will begin with a keynote speech from Alex Kipman, a Technical Fellow at Microsoft, working on mixed reality, AI and the HoloLens AR headsets. Including the keynote, there are an 21 events scheduled across Thursday and Friday with a variety of speakers from Microsoft and other companies.

Here are some notable events as listed in the full schedule:

  • Intro to Azure Mixed Reality Services: Azure Remote Rendering: Azure Remote Rendering just entered public preview. Learn how you can use ARR to render interactive 3D models with hundreds of millions of polygons and stream them to devices like HoloLens 2 in real-time.
  • Getting started with the HoloLens 2 and Unity: Learn the basics of setting up Unity and building for the HoloLens 2. This presentation will cover best practices, basic features of the HoloLens 2 and how to quickly add hand tracking support and interactivity with native Unity API’s
  • Fireside Chat with Alex Kipman and René Schulte: Chat about topics the community is interested in. René has been gathering questions from the community for about a week, and we anticipate it’s going to be a great conversation.

These are only a small slice of the various sessions on offer. You can check out the full schedule here. All of the events are free to attend but require registration ahead of time.

The post Schedule For Microsoft’s Virtual Mixed Reality Dev Days Revealed appeared first on UploadVR.

Posted on

Microsoft Mixed Reality Dev Days Names Keynote For AltSpace VR Event

Microsoft announced more details and plans for their Mixed Reality Dev Days, which will now take place fully online in AltSpaceVR. The event will take place on May 21-22, however due to COVID-19, the event will be conducted from within VR instead of in person in Seattle, as was originally planned.

The Mixed Reality Dev Days were originally announced in February, with a plan to host two days in the US in Seattle and two later dates for international developers hosted in Germany and Japan later in the year. However, the ongoing pandemic makes it seem unlikely that the planned international events will go forward at all this year. That being said, the new virtual event in AltSpace will mean that more users are able to join virtually than what would be possible with a physical event. Therefore, Microsoft made registration open to anyone and available now for free.

Microsoft also announced that the virtual keynote presentation that kicks off the two-day event will be hosted by Alex Kipman, a Technical Fellow working on Mixed Reality and AI. Kipman is also known for his work on the enterprise AR headset HoloLens. We tried out the HoloLens 2 at MWC in February 2019, before it began shipping later that year in November.

Microsoft is requesting that anyone who is interested register online beforehand to ensure they receive more information before the event, including the forthcoming full schedule and lineup.

The post Microsoft Mixed Reality Dev Days Names Keynote For AltSpace VR Event appeared first on UploadVR.

Posted on

Microsoft To Host US And International Mixed Reality Dev Days

Microsoft announced that it will be hosting two Mixed Reality Dev Days in Redmond, Washington May 16 and 17, in addition to some international dev days held in Japan and Germany at some point during the year.

The Mixed Reality Dev Days will be held at the Microsoft Conference Center in Redmond for the second year running. The 2019 Mixed Reality Dev Days ran a bit earlier in May at the same location last year.

One notable similarity to last year is that this year’s Mixed Reality Dev Day sessions run just a few days before Microsoft’s annual Build developer conference, which is focused on developers of any kind, not just those involved in VR and AR.

However, for those not in the US, Microsoft will also host international Mixed Reality Dev Day tours in Germany and Japan this year – however, dates have not been announced yet, as locations and venues are still being finalized.

Mixed Reality is the term used by Microsoft to cover everything from the HoloLens 2 AR headset to their PC VR headsets that are produced in conjunction with third-party partners like HP or Samsung. At the Dev Days events, there will likely be an opportunity for developers to get hands-on with hardware like the HoloLens, while also providing Microsoft with feedback on their experience developing for Mixed Reality platforms.

First introduced at the Mobile World Congress last year, Microsoft’s biggest Mixed Reality product, the HoloLens 2, is an enterprise-focused AR headset that started shipping late last year. You can read our impressions of the headset from MWC last year.

The post Microsoft To Host US And International Mixed Reality Dev Days appeared first on UploadVR.

Posted on

LYNX-R1 Is A $1500 Standalone Passthrough AR Headset

LYNX-R1, from France-based startup Lynx, is the first 6DoF standalone passthrough video AR headset. It is priced at $1499.

Passthrough AR

There are currently two fundamental types of AR headsets: see-through and passthrough. Most AR headsets, such as Microsoft’s HoloLens 2, are see-through. The user views the real world through the glass directly, with virtual objects superimposed onto that glass.

The technology behind see-through AR optics is still in the very early stages. The field of view is narrow and virtual objects cannot be fully opaque.

Passthrough headsets, like LYNX-R1, use the same kind of display system as VR headsets, except instead of rendering an entirely virtual world they display the real world via cameras. While the real world won’t necessarily look as good, this allows for AR across a much wider field of view, as well as full virtual object opacity and lower cost (HoloLens 2 is priced at $3500).

Some, including Lynx, call this kind of product a “mixed reality headset”.

Fully Standalone

LYNX-R1 is not the first passthrough AR headset. Varjo XR-1 and XTAL both promote similar capabilities, as was the Vrvana Totem acquired by Apple.

But other 6DoF passthrough headsets have to be tethered to a PC in order to function. Like the Oculus Quest, LYNX-R1 is standalone and wireless — it has the computing hardware, battery, and storage onboard.

Lynx is using Qualcomm’s new Snapdragon XR-2 chipset. That’s a for-XR variant of the Snapdragon 865, roughly twice as powerful as the Oculus Quest’s Snapdragon 835. This is paired with 6GB of RAM and 128GB of onboard storage.

Other Specifications

The headset uses dual 1600×1600 LCD panels running at 90Hz- LCD provides better sharpness than PenTile OLED, but with less rich colors. The lenses are unique “4-fold catadioptric freeform prisms” with a claimed circular field of view of 90°.

There are four cameras on the exterior. Two black & white cameras provide positional tracking, and two color cameras are used for the passthrough and computer vision tasks such as occlusion mapping and hand tracking, including gesture recognition.

Inside the headset are eye tracking cameras. The headset also has two speakers and two microphones, enabling positional audio and voice communications.

Charging is done via the USB Type-C port, and Lynx claims the battery lasts for two hours of “active use”.

Gap In The Market?

LYNX-R1 could fill a gap in the enterprise market for a wireless, standalone AR headset which offers a relatively wide field of view for a relatively affordable price. For consumers, the idea of video passthrough may be unappealing, but professionals might not care whether they’re looking through glass or a camera.

LYNX-R1 is slated to ship in summer of this year. Preorders are available now from the company’s website.

The post LYNX-R1 Is A $1500 Standalone Passthrough AR Headset appeared first on UploadVR.

Posted on

Join Us LIVE On YouTube To Discuss The Week’s News & The XR/VR/AR/MR Debate

UploadVR’s weekly podcast, The VR Download, is LIVE on YouTube today at 10:30am PST (18:30 UTC)!

Unlike regular video podcasts, The VR Download is broadcast from virtual reality! Our team are together in a virtual space, giving us many of the benefits of a studio even though we live on different continents.

This week’s Hot Topic: What do the terms VR AR MR & XR actually mean? Does it matter?

If you want to know more about The VR Download, head on over to our new webpage for the show!

As always, we’ll also be making it available for audio-only listening on Apple, Google, Spotify, TuneIn/Alexa, Stitcher, and more within a couple of hours of airing.

Watch In VR With Bigscreen!

Every episode, you can watch The VR Download LIVE in virtual reality with an audience of other VR users on any major VR headset (including Oculus Quest and Oculus Go!), via the Bigscreen platform.

Click on the image above to subscribe to the event.

The post Join Us LIVE On YouTube To Discuss The Week’s News & The XR/VR/AR/MR Debate appeared first on UploadVR.