Hand Tracking – Page 3 – Augmented & Virtual Reality Confabulation

November 19, 2021

Leading Hand-tracking Company Ultraleap Raises $82 Million Investment

Ultraleap, a leading company focused on hand-tracking interfaces, this week announced it has secured a £60 million (~$82 million) Series D investment, with the goal of expanding its hand-tracking and mid-air haptic tech in the XR space and beyond.

Formerly known as Ultrahaptics, Ultraleap was formed after the UK-based haptics company acquired leading hand-tracking company Leap Motion back in 2019. The new name clearly defined the merger’s unique combination of mid-air ultrasonic haptics now underpinned by some of the best hand-tracking tech in the industry.

This week Ultraleap announced it has raised a £60 million (~$82 million) Series D investment with participation from new investors Tencent, British Patient Capital, and CMB International, alongside existing investors Mayfair Equity Partners and IP Group plc.

“With this investment round, Ultraleap will continue to bring Gemini to different operating systems and increase their investment in tooling to enable developers to build more applications using the best interface—your hands. Ultraleap will also continue to invest in R&D to drive their machine-learning-based hand tracking even further ahead,” the company said in its investment announcement.

Ultraleap is betting that hand-tracking will be the primary input for XR and the metaverse. Last month the company released its latest revision.

While the company has been trying to get its tech into the XR space for many years now, it has yet to find significant traction. Though Ultraleap hand-tracking can be found on a few headsets like those from Varjo and Pimax, leading devices implementing hand-tracking—like Quest 2, HoloLens, and Magic Leap—are using their own solutions, as far as we know.

However, with a growing number of XR devices on the market and the steady march toward consumer-friendly AR glasses, the company seems poised to find the right fit eventually.

Ultraleap is also looking to find a home for its tech outside of the XR realm. The company has long been angling its tech in the automotive space as an in-car interface, as well as the out-of-home space in areas like exhibits, marketing installations, and touchless self-service kiosks.

The post Leading Hand-tracking Company Ultraleap Raises $82 Million Investment appeared first on Road to VR.

October 27, 2021

Ultraleap Gemini Brings 5th-Gen Hand Tracking to Windows

Earlier in the year, hand tracking experts Ultraleap unveiled a preview of its fifth-generation hand-tracking software offering a number of improvements including two-handed interactions. Called Gemini, today Ultraleap has announced the official launch of this new hand tracking platform, fully compatible with Windows systems.

Already available in Varjo headsets and integrated into Qualcomm’s Snapdragon XR2 5G reference design, Gemini marks the first full software release from the company in three years. Completely rebuilding its hand-tracking engine, the core improvements see refinement in two-handed tracking, where occlusion can impair the sensors ability to distinguish between either hand when close together.

Another important enhancement is the speed at which hands are detected. Some location-based entertainment (LBE) venues utilise Ultraleap hand tracking, if you’ve experienced one then you may have noticed that when your hands are out of sight bringing them back would take a few seconds. That should now be rectified with Gemini.

Ultraleap also notes that Gemini offers: “Improved robustness to challenging environmental conditions,” so a user’s hands should be much more easily tracked if lighting conditions suddenly change, for example. To showcase these new features Ultraleap check out the new XR launcher experience in the trailer below.

“Ultraleap Gemini is the next step in bringing intuitive and seamless XR experiences to life by making natural interaction between humans and technology possible. With ten years of continuous innovation and experience from supporting over 350k developers, we have refined our hand tracking to deliver the best and most flexible hand tracking platform available,” said Matt Tullis, Product Director, Spatial Computing, Ultraleap, in a statement. “We are looking forward to working with software and hardware partners to bring hand tracking and XR to the masses.”

The new Gemini system is available to download today and works with both the Leap Motion Controller and Stereo IR 170 camera module. For further Ultraleap updates, keep reading VRFocus.

September 21, 2021

Preview: Unplugged – Rocking VR Air Guitar

Everyone has strummed out a little air guitar at some point in their lives – come on admit it – whether that’s in the shower or when you hear an awesome solo over the airwaves. But mimicking your wild arm flailing into a cohesive, hand tracked virtual reality (VR) videogame is another endeavour entirely. It’s a challenge indie team Anotherway decided to tackle, and with the help of Vertigo Games behind them, has begun to showcase what rock dreams are made of in Unplugged.

Unlike every other rhythm action guitar title where you had some sort of plastic controller with fret buttons and a whammy bar to hold, giving that pseudo sense of being an ace axe player, Unplugged’s use of hand tracking is bold. That’s because up until now hand tracking on Oculus Quest has revolved around slower, more methodical genres like puzzle videogames; Cubism’s recent implementation is testimony to that.

Without having a guitar to “feel” where your hands are on the neck going into Unplugged for the first time is like stepping into the unknown, as the expectation is that this level of complexity can’t work (or work well). And first impressions definitely are mixed when it comes to playing a hand tracked guitar in VR.

The demo of Unplugged VRFocus got to play offered the main gist of the experience, an introductory tutorial as well as four songs to try and master, each with three difficulty levels. It must be said that Unplugged looks extremely polished, from the tattoos on your virtual fingers to the inclusion of Satchel from Steel Panther as your rock guide, it is very well presented. Even the buttons to select the various menu options have a nice push to them, a small but important touch.

When it comes to actually playing the virtual guitar the neck is split into five sections with each of your fingers colour coded so you can play specific notes. That means you have to pay attention to where the notes are going to hit the neck as well as the appropriate finger combination. You also have to strum of course. However, from what’s been shown so far there are no epic individual finger solos like you used to get on Guitar Hero, most tend to be all four fingers, three and the occasional two-finger notes. And that’s certainly enough.

Even on easy Unplugged isn’t particularly straightforward. Without that physicality, playing tracks like Creedence Clearwater Revival’s Fortunate Son or The Offspring’s The Kids Aren’t Alright requires focusing all of your attention on the guitar neck so you know exactly where to place your hand whilst ensuring optimal tracking of your fingers. The downside to this was forgetting where that important strumming hand needed to be. Quite often notes were being missed not because of incorrect finger placement but that pick hand moving out of place during an awesome rock solo.

You can play Unplugged both seated and standing with the latter tending to be the easier option. There’s also the option to adjust where the guitar is placed in relation to you, moving it up/down, in/out depending on preference. While this does really help, strumming still seemed to be the main issue as it’s difficult keeping your hand in very near the same point mid-air for an entire song. Or maybe much, much more practice is required.

Thankfully, Unplugged doesn’t just have notes you need to strum. The Pull-Off notes are by far the easiest to play as you can move your hand up and down the neck for some true air guitar rocking! The same goes for the Virtuoso notes where you’re given a blank flaming box to wiggle those digits however you see fit. These are the moments where Unplugged comes alive, coordination and precision go out the window, allowing you to enjoy the song at its fullest.

At the end of each track, you can then pump up the crowd for more points and hopefully a top leaderboard position.

Unplugged is going to be the greatest test of Oculus Quest’s hand tracking and likely very divisive as to whether it can really offer a viable alternative to those physical, guitar rhythm action games of old. There’s no doubt that it works with some flashes of brilliance but the learning curve is huge, especially trying to complete those higher levels. With a launch planned for fall 2021, there isn’t long to wait to see if Unplugged is the hardest air guitar you’ve ever played.

May 24, 2021

Hand Physics Lab Adds Accessibility Options, 120 Hz Mode Planned

Hand Physics Lab now includes a number of accessibility options, including a colorblind mode and the ability to map finger movements to other fingers.

The finger mapping feature should make the game playable for people who have a finger-related disability. The game is one of the few available on Oculus Quest that’s entirely playable with either Oculus Touch controllers or hand tracking. I reviewed the game when it debuted in April and, while it is held back by the quality of Oculus Quest’s cameras, Facebook has since released an update for Quest 2 enabling developers to increase the quality of the hand tracking for the newer headset.

Dennys Kuhnert, the co-founder of Holonautic and creator of the game, already added the high frequency hand tracking to Hand Physics Lab and is also planning to support 120 Hz experimental frame rate as well in a coming update. In the current version of the game, you can set the refresh rate to 72, 80, or 90 Hz on Quest 2.

Kuhnert says the game sold more than 100,000 copies in the six weeks after release, meaning that the $9.99 title made more than $1 million in gross revenue before Facebook took its cut of sales. An early experimental version of the title hit SideQuest last year and he wrote via direct message that its sales on the Oculus Store were “honestly not expected. I started this as a side project in 2020 just for fun with zero expectations. This will allow the studio to grow and work on new exciting projects.”

“After 5 months of experimenting and building the first prototype of Hand Physics Lab, I just wanted to have it out there and get feedback from the VR community to improve it further. Combining fully physics-based interactions with the experimental hand tracking from the Quest (quite unstable a year ago, it improved a lot since) was a big challenge and many things could go wrong,” he wrote. “It was important to minimize the barrier of entry and have safety systems in place to handle bad tracking events for the players. Shane Harris and his platform SideQuest were a big help to help reach a bigger community and basically validate the concept. The platform is really great and allowed many experimental VR concepts like Hand Physics Lab and others to quickly reach a large audience and get valuable feedback to make the experience better!”

Facebook’s head of VR and AR Andrew Bosworth says the company is pursuing an “Oculus Quest Pro” that won’t release this year but is likely to add more powerful sensing abilities to the Quest line of headsets. Facebook CEO Mark Zuckerberg has expressed interest in future headsets including face and eye tracking, and we’d expect Facebook to improve the quality of hand tracking in future hardware as well. Consulting Oculus CTO John Carmack recently commented that Facebook is on a “multi-year path to phasing [physical controllers] away as a core feature, where we want to be able to have a controller -free SKU in the future where we rely just on hand tracking.”

Hand Physics Lab is available on the Oculus Quest Store.

May 1, 2021

Vacation Simulator Adds High Frequency Hand Tracking Support on Quest 2

A new Vacation Simulator update adds support for the Quest 2’s new High Frequency Hand Tracking mode, increasing performance and decreasing latency.

The update is available now and improves the reliability and performance when using hand tracking, thanks to an increase in the rate of tracking. The feature is only available to Oculus Quest 2 players – the original Quest still supports hand tracking, but without the high frequency tracking mode.

Human! Today we released the High Frequency Hand Tracking update for Vacation Simulator on Oculus Quest 2.

This update brings improved tracking and lower latency to deliver a more immersive experience. Explore Vacation Island with just your human hands from start to finish! pic.twitter.com/J5jyknRPCV

— Owlchemy Labs (@OwlchemyLabs) April 29, 2021

Previously, the Quest 2 camera’s hand tracking was limited to a rate of 30Hz. Earlier this week, Oculus gave developers the option to enable a ‘High Frequency Hand Tracking’ mode that ups the rate to 60Hz and reduces end-to-end latency by 10%.

Overall, the new mode is meant to be more effective for tracking fast hand movements. It does possess a slight increase in jitter in low-light situations, but Facebook says this will be fixed in a future update.

Vacation Simulator is one of very few games that has added support for the new 60Hz hand tracking mode. Hand Physics Simulator is another, which rolled out support fairly quickly after Facebook announced the feature. In our anecdotal tests, we found that the high frequency tracking really improved the hand tracking experience in Hand Physics Lab.

It’s uncertain whether all or most games that support hand tracking support will be able to include support for the new high frequency tracking mode. It does come with an increased performance cost, which means that some games with less performance overhead might have to stick with the standard 30Hz rate.

What do you think of the new High Frequency Hand Tracking mode? Let us know what you think in the comments.

April 29, 2021

Hand Physics Lab Updates With 60Hz Hand Tracking Support On Quest 2

Hand Physics Lab pushed out an update that adds support for the new High Frequency Hand Tracking mode that became available to developers yesterday.

The update integrates a doubling of the hand tracking rate, but comes at a performance cost that means it’s only available for Quest 2 users.

Previously, the rate of hand tracking using the Quest 2’s cameras was limited to just 30Hz. Yesterday, Oculus gave developers the option to enable a ‘High Frequency Hand Tracking’ mode that ups the rate to 60Hz and reduces end-to-end latency by 10%.

Overall, the new mode is meant to be more effective for tracking fast hand movements. It does possess a slight increase in jitter in low light situations, but Facebook says this will be fixed in a future update.

Hand Physics Lab started as a series of experimental demonstration interactions to show off the capabilities (and limitations) of the Quest’s then-new hand tracking support. It was available on SideQuest and received a few updates adding new interactions and situations.

However, earlier this month, a new and improved revamped version of the experience launched on the Oculus Store, with support for controllers and hand tracking. It presents the interactions as puzzles that form a larger campaign, along with a sandbox mode for the puzzles that you’ve completed. For more info, check out our review.

The addition of High Frequency Hand Tracking should improve the experience dramatically for most users. After testing the update earlier today, I can say that I found the hand tracking felt noticeably better and smoother overall.

Have you tried high frequency hand tracking on Quest 2? Let us know what you think in the comments below.

April 29, 2021

Oculus Quest 2 Hand Tracking jetzt mit 60Hz möglich

Ab sofort ist es für Entwickler und Entwicklerinnen möglich, die Tracking-Rate von 30Hz auf 60Hz zu erhöhen. Dies soll besonders bei schnelleren Bewegungen ein zuverlässigeres Tracking ermöglichen.

Oculus Quest 2 Hand Tracking jetzt mit 60Hz möglich

Wie Facebook sagt, soll der neue Modus die Qualität des Trackings und die Latenz um 10% verbessern. Da jedoch “nur” die Rate beim Tracking erhöht ist, bleiben andere Probleme bestehen. So ist es auch weiterhin ein Problem, wenn eure Hände oder Finger sich gegenseitig verdecken.

Aktuell ist der neue Modus nicht automatisch aktiviert, sondern muss von den Entwicklern und Entwicklerinnen für die jeweiligen Anwendungen freigeschaltet werden und kann nur mit der Oculus Quest 2 genutzt werden. Die Oculus Quest 1 kann die erhöhte Rate nicht liefern.

Die Oculus Quest 2 ist in Deutschland nicht verfügbar. Ihr könnt sie aber bei Amazon Frankreich bestellen (Anleitung). Hier geht es zu unserem Langzeittest zur Oculus Quest 2.

(Quelle: Upload VR, Oculus)

Der Beitrag Oculus Quest 2 Hand Tracking jetzt mit 60Hz möglich zuerst gesehen auf VR∙Nerds. VR·Nerds am Werk!

April 28, 2021

Oculus Quest 2 Now Has A 60 Hz Hand Tracking Mode

Oculus Quest 2 developers can now enable a new ‘High Frequency Hand Tracking’ mode in their apps, increasing the tracking rate from 30 Hz to 60 Hz.

Facebook claims the mode increases hand tracking quality, as well as reducing its end-to-end latency by 10%. Its internal ‘Strike Team’ enabled the new mode in the Tiny Castles demo and reported the following results:

“In Tiny Castles, we observed that high frequency hand tracking results in a slight reduction in perceived latency but also significantly improved tracking quality during fast hand movements. There were no perceived changes regarding jitter and occlusion.“

However, Facebook says there is a “slight increase in jitter” for low light conditions, which it plans to fix in a future software update.

Oculus Quest 2 Now Has A 60 Hz Hand Tracking Mode

Developers can enable High Frequency Hand Tracking by adding this line to the Android manifest:

Oculus apps can set the CPU & GPU clock rate – not to a specific value but to 5 predefined levels (0-4). This lets developers balance performance and battery life.

High Frequency Hand Tracking requires more background computing power to process the cameras at double the normal rate. Facebook says this can cause headset overheating (leading to a sudden shutdown), so to avoid this, apps using the feature will be capped to CPU & GPU Level 2.

That means while enabling this feature is technically a one-line change, actually making it work in real apps could require significant optimization work.

Facebook says that going forward, even apps using low frequency Hand Tracking will be limited to CPU & GPU Level 3, to avoid overheating.

April 21, 2021

Carmack Hints At ‘Controller-Free’ SKU For Oculus Headsets

Facebook may be on a “multi-year path to phasing away” reliance on Oculus Touch controllers and turning them into a “separable feature”, Consulting Oculus CTO John Carmack suggested in recent comments.

Carmack and Facebook’s head of AR/VR Andrew Bosworth recently participated in a Twitter Spaces audio chat where the duo offered some of the most insightful comments into Facebook’s long-term strategy for VR. One part of the discussion revealed a Quest Pro won’t ship in 2021 while Quest 2 will be available to buyers for “a long while” and another segment pointed to how Facebook’s VR headsets are likely to keep adding new software features to make them more fully functional personal computers which could go after the market for Chromebooks and tablets.

We’ve got a recording of the talk and have been working to transcribe the conversation and edit out portions unrelated to VR. We talked about some of the revelations in our show earlier this week, embedded below:

We’ll be posting the entire transcription and audio recording later this week, but for now we’ve been posting about some of the most notable segments.

Next Generation Oculus Hardware

One section of the talk saw Bosworth ask Carmack “What’s the next gen hardware feature that you’re most excited about for virtual reality?”

Carmack contends that, essentially, an improved version of everything in Quest 2 is what Facebook should be focusing on, while “alternate points of view” suggest new capabilities like depth cameras, eye tracking, face tracking and more should be a priority in future hardware. The conversation then covers how a Quest Pro could introduce some of those extra sensors in a higher end device, and the conversation turns to the value of tracked hand controllers — aka Oculus Touch — and how Carmack “completely missed the value of how much we get out of tracked controllers.”

“But it’s interesting how we are on this multi-year path to phasing that away as a core feature, where we want to be able to have a controller-free SKU in the future where we rely just on hand tracking for people that want to use keyboard and mouse and don’t want to pay for the controllers,” Carmack said. “So it is clearly more valuable for gaming than I initially expected. But I think that we will wind up in those cases where lots of the users will not wind up taking advantage of those in the future. And that’ll be nice to have that as a separable feature.”

Bosworth adds:

“The good news is we can do both. We can do both, you can have a set of techniques and development that are going to put something out there that has a more featureful presence. And this is going to go at maybe lower volume in terms of the number of units, but also advances the state of the art, inspires developers, I think unlocks a lot more use cases. And then as that technology matures, finds its way into these scale units that get out to so many people, which we want to continue to advance the drum beat on that.”

We’ve seen Facebook approve for release a number of hand-tracking focused applications in recent weeks, with Hand Physics Lab showcasing an exciting yet frustrating example of what’s possible with current generation hand tracking technology, a new version of its First Steps app that shows developers how to adapt their controller-focused input to hand tracking, and the forthcoming Unplugged: Air Guitar game bringing one of the first arcade games to Quest later this year that’s focused entirely around the open air input method.

April 1, 2021

Hand Physics Lab Review: Experiments In Joy And Frustration At Your Fingertips

Hand Physics Lab is probably the game you want if you’re looking for a hand tracking introduction on Oculus Quest to demonstrate the cutting edge of VR. Read on for our full Hand Physics Lab review to find out why.

The entire collection of puzzles—85 of them in total at launch alongside a sandbox mode—is playable either with hand tracking or Touch controllers. The very first “puzzle”, if you can call it that, is simply to press the button. The big red circle is sitting right there in front of you so just reach out and press it down. But the truth of VR is that when you’re not actually touching real physical objects and have no haptic feedback whatsoever, even something as simple as “press the button” is not necessarily doable on the very first try.

“You mean I actually have to, um, move my hands?” Some people trying VR for the first time will have to learn that. “And you mean I have to push my hand down with a little force in mid-air after positioning it right above the button?” Yes, and even if you’ve owned several VR headsets at this point that’s still a lesson you might need to learn with your bare hands.

So follows dozens of puzzles built by Dennys Kuhnert at Holonautic Studio exploring the very cutting edge of physics-laden open air interaction design.

Hand Physics Lab – The Facts

What is it?: A physics-based puzzle game with full hand-tracking support
Platforms: Oculus Quest
Release Date: April 1, 2021
Price: $9.99

Pushing through the entire puzzle collection is as maddening as it is interesting. For example, a couple puzzles put you in control of a remote control car you operate by pinching together your fingers in one hand to accelerate the tiny vehicle and wagging a finger left or right on the other hand to direct which way it turns. It’s an absolute joy to realize “whoa this works!” But then, a later puzzle requires driving the car over ramps to your right and left. So you turn your head away from your hands to look, but this moves the headset’s cameras away from your fingertips—effectively destroying your remote control in the process.

Oops, stupid head.

Hand Physics Lab Review – Comfort

Hand Physics Lab comes with a large set of options and adjustments, including dark mode, adjusting hand size, adjusting your play space, and vignetting your view. There’s a third-person view you can optionally activate that might be uncomfortable for some where you essentially operate your own arms remotely, and a puzzle with climbing. The vast majority of puzzles keep the player completely stationary, however, and overall this is a very comfortable game.

While most puzzles can be satisfyingly solved in seconds, I spent probably an hour on a single puzzle in which I’d convinced myself there was some mystical power in my fingertips that I needed to master. My hands weren’t allowed to penetrate a force field but I’d been empowered with telekinesis to move blocks on the other side of it. I’d come to believe that carefully precise movement and spacing between my fingertips would master the telekinesis like Wanda Maximoff, and allow me to sort the colored blocks into the correct cups on the far end of the table.

Instead, I would again and again either drop the blocks behind the cups, where I couldn’t retrieve them, or pull the blocks into the grip of my hand. It was one of the only puzzles in the collection where I grabbed my Touch controllers to solve it and, with Touch controllers in hand, the puzzle was solved very quickly.

Hand Physics Review – Final Verdict

Hand Physics Lab can be intensely frustrating but also incredibly satisfying. It is also a one-of-a-kind experimental playground that’s unlike anything else Facebook has allowed into the Oculus Quest Store. We recommend grabbing the game to explore the cutting edge of VR game design with hand tracking or controller-based interactions. Its earliest puzzles can be a wonderful introduction to new VR users while its later puzzles — particularly those with mirrored hands — can be revelatory for more experienced VR fans interested in something that scratches an itch for a Portal-like game in VR.

For more on how we arrived at this score, read our review guidelines. This review was conducted using a complimentary early copy provided by the developer on a Quest 2 headset.

Hand Physics Lab is out now on Oculus Quest for $10.