Ultraleap, the company behind the Leap Motion hand-tracking module, informed staff on Wednesday that it was proposing a layoff amid a potential restructuring of the business that could see the company split in two.
The Bristol, UK-based company acquired Leap Motion in 2019, prompting a rebrand from its original name Ultrahaptics to Ultraleap. Prior to the acquisition, the company was best known for pioneering its mid-air haptic technology, which uses ultrasound to project tactile sensations onto the user’s hands.
As reported by Sky News, Ultraleap is allegedly now seeking to sell off its hand-tracking business entirely, and spin out its mid-air haptics division into a new company, which would be owned by Ultraleap’s existing shareholders and also seek additional external funding.
Leap Motion 2 in front of its predecessor | courtesy Ultraleap
The company hasn’t publicly confirmed the sale of Leap Motion or the specifics surrounding the restructuring of its haptics business, however it has confirmed layoffs are coming:
“After much consideration, we have made the difficult decision to reshape some of our divisions and reduce the size of our team,” an Ultraleap spokesperson told Sky News. “This decision has not been taken lightly, but it is necessary for us to adapt our business to better serve our market and our customers.
Initially released in 2013, Leap Motion was one of the first viable hand-tracking modules to come to market. While it was originally created to work as an input method for PCs, a few years later the then still independent company would hard pivot into the VR space, providing hand-tracking to headsets which at the time had none.
Fast-forward to today, and many standalone headsets pack in their own onboard hand-tracking thanks to the requisite bank of optical sensors that are also used for tracking the user in room-scale environments. The shift has made bespoke modules like Leap Motion less desirable for consumers overall, leaving the company to focus on integrating its tech with boutique headset manufacturers such as Varjo, Pimax, and Vrgineers.
10 years after the launch of Leap Motion—which garnered praise for offering some of the best hand-tracking in the industry—the company has announced a next-generation version of the device which now supports standalone XR headsets in addition to Windows and MacOS.
Years before the modern era of VR, Leap Motion set out to build a hand-tracking module that it hoped would revolutionize human-computer interaction. Launched initially in 2013, the device was praised for its impressive hand-tracking, but failed to find a killer use-case when used as an accessory for PCs. But as the VR spark began anew a few years later, Leap Motion’s hand-tracking started to look like a perfect input method for interacting with immersive content.
Between then and now the company pivoted heavily into the VR space, but didn’t manage to find its way into any major headsets until well after the launch of first-gen VR headsets like Oculus Rift and HTC Vive (though that didn’t stop developers from attached the Leap Motion module and experimenting with hand-tracking). Over the years the company kept honing their hand-tracking tech, improving its software stack which made hand-tracking with the first generation of the hand-tracking module better over time.
First generation Leap Motion | Image courtesy Leap Motion
(It should be noted that Leap Motion was once both the name of the device and the company itself, Leap Motion was merged with another company to form Ultraleap back in 2019.)
More recently the company has built newer versions of its hand-tracking module—including integrations with headsets from the likes of Varjo and Lynx—but never sold that newer hardware as a standalone tracking module that anyone could buy. Until now.
Leap Motion 2 is the first new standalone hand-tracking module from the company since the original, and it’s already available for pre-order, priced at $140, and expected to ship this Summer.
Purportedly built for “XR, desktop use, holographic displays, and Vtubing,” Ultraleap says the Leap Motion 2 is its “most flexible camera ever” thanks to support for Windows, MacOS, and standalone Android headsets with Qualcomm’s XR2 chip.
Image courtesy Ultraleap
From a specs standpoint, the company says the new tracker has “higher resolution cameras, increased field-of-view, and 25% lower power consumption, all in a 30% smaller package for optimum placement and convenience.”
Ultraleap says that Leap Motion 2 will give developers an easy way to experiment with high-quality hand-tracking by adding it to headsets like Varjo Aero, Pico Neo 3 Pro, and Lenovo’s ThinkReality VRX. The company also plans to sell a mount for the device to be attached to XR headsets, as it did with the original device.
Image courtesy Ultraleap
And with the launch of this next-gen hand-tracking module, Ultraleap says it’s moving on from the original Leap Motion tracker.
“Existing customers [using the first Leap Motion module] may continue to access the latest compatible software including the soon-to-be-released Gemini for macOS. Support will also continue to be provided. Future versions of the software will not deliver any performance improvements to the original Leap Motion Controller device,” the company says.
Ultraleap said it has sold more than 1 million Leap Motion trackers to date, with some 350,000 developers having build apps and experiences using the company’s hand-tracking tech.
Pico Neo 3 Pro is getting a high quality hand tracking accessory from Ultraleap.
Pico was a China-based startup which launched the first standalone VR headset, Pico Goblin, a year before Oculus Go. Last year it was acquired by ByteDance, the Chinese tech giant behind the TikTok social media platform. Pico’s latest model, Neo 3 Pro, has remarkably similar specs and design to Meta’s Quest 2. But in western markets Pico currently only sells to businesses, not consumers, and it lacks built in controller free hand tracking.
The Ultraleap hand tracking bundle will be formally released in summer. It includes the headset, Ultraleap’s Stereo IR 170 tracker, and a bespoke mount pre-attached with Ultraleap’s software pre-installed.
I had the chance to try Ultraleap’s latest technology at CES 2022. The hand tracking quality was significantly better than Quest 2 – the virtual hands seemed to match my own precisely and with no perceptible latency, and I could even interlock my fingers. Only by almost entirely occluding one hand did tracking start to fail.
For now an “early access” bundle is available for “developers, evaluation, and proof of concepts”, but Ultraleap cautions this is not suitable for scalable deployments as the tracker currently lacks the European Economic Area’s CE regulatory marking. The bundle is priced at €999 in Europe, sold by Bestware and VR Expert. In the US it’s sold by Mace Virtual Labs.
Businesses may be able to buy the tracker separately and attach it to their own Pico Neo 3 Pro, but that wouldn’t include the mount or software license.
One of the definitive leaders in hand tracking technology is Ultraleap, with its tech integrated into devices such as Varjo’s headsets or available as a third-party accessory. It’s the latter that Ultraleap is announcing today, bringing hand tracking to Pico Interactive’s Neo 3 Pro and Pro Eye headsets.
Image credit: Pico Interactive
As you can see in the image above, the setup consists of an Ultraleap Stereo IR 170 camera inside a bespoke mount, with a power cable running to the Pico Neo 3’s USB-C socket. The setup will then run Ultraleap’s fifth-generation hand tracking software Gemini, with Unity and Unreal platforms supported for developers.
The Ultraleap Hand Tracking Accessory won’t be sold as an individual unit it seems for current Neo 3 Pro and Pro Eye owners to upgrade to. It’ll be sold as a new bundle with one of the aforementioned headsets (Gemini coming pre-installed) through select retailers, available now in early access for developers and enterprise customers. An official launch will then take place this summer, with prices yet to be revealed.
“VR for training is on the cusp of mainstream adoption and we truly believe hand tracking plays an important part in tipping it over the edge. We’re already seeing significant wins from customers who have deployed VR training programmes or LBE experiences with hand tracking,” said Matt Tullis, VP, XR at Ultraleap in a statement. “This first phase of the Pico relationship will mean more developers and organisations will be able to test, pilot and refine their applications to unlock the true power of VR now and deploy at scale in a few months.”
“We’re very excited to bring Ultraleap hand tracking to our latest VR headsets through this accessory. When applications need the highest performing hand tracking for complex interactions or challenging environments, Ultraleap’s hand tracking really is world-class. We can’t wait to see what developers and organisations will create from this joint effort,” adds Leland Hedges, GM for Pico Interactive Europe.
Hand tracking has been gaining ground of late, featuring in devices like the HTC Vive Focus 3 whilst the upcoming Lynx-R1 utilises hand tracking (Ultraleap’s again) as its default input method. And, of course, let’s not forget about Meta Quest 2 which supports hand tracking out the box with titles like Cubism, Vacation Simulator and Clash of Chef’s VR all adding hand tracking updates.
gmw3 will continue its coverage of hand tracking as further announcements are made.
The leading UK amusement and entertainment trade event holds many XR secrets, industry specialist Kevin Williams, in his latest Virtual Arena column – looks at the trends shaping an industry that is gaining ground after a period of lockdown.
The Out-of-Home (OOH) entertainment scene has a wide array of weapons at its disposal. Be it Virtual Reality (VR), Augmented Reality (AR), Projection mapped immersive enclosures, being Mixed Reality (MR) – all under the general umbrella term of Extreme Reality (XR). All covered by the parapluie of these applications were on show with the return of the physical exhibition format at Entertainment Attraction & Gaming Expo 2022 Expo (EAG). An event dedicated to the best in amusement, entertainment, and leisure developments for the trade, held in London.
Image credit: KWP
Virtual Reality
First off, the return to a physical trade event allowed many of the trade buyers to get their hands on the latest VR offerings from the international market launched last year – and making UK landfall for the first time.
One of the major trends shaping the amusement adoption of VR is the ability to offer a self-service, standalone kiosk. The obvious savings in not having to constantly man the game system, as well as a level of automation to the VR experience have proven a draw.
The VRsenal ‘V2’ is the latest incarnation of this approach. The company building on their previous VR Kiosk system, have now created a two-player standalone platform, supporting an automated controller and headset retraction system after each use.
VRsenal Kiosk. Image credit: KWP
Content is king in the land of immersive entertainment and the ‘V2’ hoped to bank on its available games. The system offers the chance for players to try Vader Immortal – Lightsaber Dojo: A Star Wars VR Experience. A specially commissioned version of the Star Wars property developed byILMxLAB for the amusement scene.
Also available on the platform, is a special VR arcade version of Rhythmatics – offering a highly active VR music game – and Pirate Trainer; bringing full-on blasting action. The ‘V2’ is sold under special agreement in the UK and Europe throughBANDAI NAMCO Amusement Europe (BNAE). The system uses a converted HTC Vive headset and controllers.
Another automated VR kiosk with a unique presentation was ‘VR Agent’. Developed in collaboration between Chinese studio 3MindWave, and SEGA Amusement International (SAI). The platform is a unique approach to immersive gaming – with the player viewing the virtual environment through the headset mounted to the weapon they are wielding. This alleviates the need for players to wear a cumbersome headset.
3MindWave’s VR Agent. Image credit: KWP
On the SAI booth, the kiosk-style amusement platform also incorporates a headset retraction system. Offering a pure shooting game experience, reminiscent of Virtua Cop – the player traversing an office block populated with robotic enemies. This new approach to immersive gaming proved compelling. The platform uses a cutdown 3Glasses VR headset.
Returning to the BANDAI NAMCO Amusement Europe (BNAE) show booth, two examples of the other trend in VR amusement were on show. The deployment of what has been called VR Ride Systems – a genre of two-rider player systems with motion seats and special effects, have found popularity with operators.
On display was King Kong: Of Skull Island (developed by Raw Thrills) – which has two players using their hands to complete activities, while racing on the back of a jeep through the home island of the giant beast, avoiding monsters and obstacles.
King Kong: Of Skull Island. Image credit: KWP
Also, on the BNAE booth was STORM (developed by TRIOTECH) – which also had two players sitting on special motion seats, waving their hands to collect points and gain a high score as they traversed a world of confectionary, in the latest game on the platform, called Sugary Slope.
TrioTech’s Storm. Image credit: KWP
While the Raw Thrills platform uses the HP G2 headset, the TRIOTECH system used the HTC Vive Pro. Though both platforms incorporated the UltraLeap motion tracking system to represent the players’ hands in the game. Along with wind and motion effects to add to the experience.
Though not on the show floor, a new VR attraction developed by Frontgrid was just being rolled out in Qatar. The ParadropVR: Pod system is a compact virtual reality paragliding experience, based on the older brother platform. This new version of the system comprises exhilarating motion, rising, and lowering the virtual flyer as they soar through the landscape, collecting points in competition.
Image credit: KWP
It is the application of physical effects to the virtual experience that single outs many of the successful VR amusement and attraction platforms. Far more people have experienced VR over the years through amusement and attractions, accelerating the development of new systems in the field. While EAG exhibitor Harry Levy offers examples of the latest Chinese VR motion-ride film systems that provide a passive VR experience.
Image credit: KWP
Mixed Reality
Beyond the traditional VR experiences, the Out-of-Home entertainment sector has embraced the other elements of immersive entertainment through MR applications. EAG exhibitor Sports Simulations had their active entertainment enclosure, using a large projected screen and object tracking. This offers a virtual sports simulator and even includes a bowling game.
Sports Simulations. Image credit: KWP
Another immersive projection system presented was from Interactive 365, with their ‘Augmented Reality Air Hockey’ system. Using the latest object tracking technology and short throw projector, the company has taken the traditional amusement and given a major new twist. The platform offers different styles of mini-game incorporated into the air hockey action.
AR Air Hockey by Interactive 365. Image credit: KWP
Concerning the wider immersive entertainment scene beyond EAG, drawing on all the elements and technologies of XR, the London scene has seen a growth in investment in Immersive Live Experiences. Away from just the MR hardware, these new “Immersive Theater” experiences are popping up across London offering high-profile IP, with immersive technology elements in puzzle solving, and presentation.
Most recently we have seen popup experiences including Tomb Raider, and Money Heist from Netflix. There is also a brand new Gun Powder Plot experience in development by Layered Reality that will incorporate new XR elements; the same team that developed the War of The World’s immersive experience. Expect a full report in the coming weeks.
If the only controller-free hand tracking you’ve used is Quest 2, you may not have the best opinion of it.
Sure it’s cool to see your actual hands in VR for the first time. But try to navigate Quest’s menu system or grab objects in apps like Hand Physics Lab and you’ll soon realize the tracking quality leaves a lot to be desired.
At CES 2022 I tried Ultraleap’s fifth generation technology, which the company calls Gemini. If you’ve been following VR since before the release of consumer headsets you’ll be familiar with Leap Motion, a startup that launched a desktop hand tracking accessory in 2014 that could be mounted to the front of the Oculus Developer Kit 2. It brought hand tracking to VR before most users even had access to motion controllers.
Leap Motion had a small range of demo applications, and it was even supported in the social platform AltSpaceVR. But when the HTC Vive & Oculus Touch controllers launched in 2016 it quickly faded from relevance, despite a major algorithm quality update called ‘Orion’. By 2019 even AltSpace dropped support for Leap Motion – developers prioritize for hardware users actually own, and even though Leap Motion was relatively inexpensive (around $80) it still faced the classic chicken & egg problem of input accessories.
Leap Motion was acquired in 2019 by UK-based haptic technology firm UltraHaptics, merging to become UltraLeap. A year later, Facebook shipped a software update to Oculus Quest adding hand tracking. Quest’s hand tracking leverages the onboard grey fisheye cameras (which can also see in the infrared spectrum). It does work, and millions of people have used it, but if your hands are at the wrong angle or too close together it quickly breaks down. Trying Ultraleap showed me just how much better hand tracking can be.
VR-quality hand tracking requires sensor overlap – the algorithm compares the perspective of each camera to determine the relative position of each hand. Quest’s cameras have a wide field of view, but since they’re positioned at the edges of the headset pointing outwards the hand tracking area is relatively small. That’s why it only works properly with your hands directly in front of you, and why you can notice the system re-establishing tracking as you bring your hands back into your view.
Ultraleap’s latest Gemini hand tracking tech is the successor to Orion, and Ultraleap claims it was re-written from the ground up. It was announced in 2020 alongside a multi-year agreement with Qualcomm integrating and optimizing it for the Snapdragon XR2 chip. 95% of the algorithm runs on the XR2’s DSP (Digital Signal Processor), freeing up the CPU for the actual VR & AR applications.
I tried Gemini as an attachment to Vive Focus 3 virtual reality headset and as an integrated aspect of the upcoming Lynx R1 mixed reality headset – both headsets use the XR2 chip. The two infrared fisheye cameras face forward, not to the sides, giving almost total overlap over a 170 degrees field of view. That’s wider than either headset’s lenses (and almost every headset in existence).
The result was that just like when using motion controllers, I could stretch my arms out naturally and no longer had to worry about going out of the tracking range.
The hardware, like Leap Motion before it, also has active infrared emitters to illuminate your hands for a better view. And further, the sensors are sampled at 90 frames per second. The combination of these specs and the Gemini algorithm meant the virtual hands seemed to match my own precisely and with no perceptible latency. Gone was the lag and glitching of Quest’s hand tracking – Gemini felt generations ahead. I could even interlock my fingers. Only by almost entirely occluding one hand did tracking start to fail.
And speaking of occlusion, one of the most impressive aspects of the demo was the re-acquisition time. I purposely occluded one of my hands by passing it under a table, but by the time it was visible again tracking had almost instantly resumed.
I have always believed controller-free hand tracking will play an important role in mainstream virtual reality. Some people object to this on the basis of the lack of haptics, a view I can’t really argue with. But others are skeptical on the grounds of a perceived lack of precision. Gemini proves that’s really just the tracking quality of Quest, not a fundamental limit of the technology.
Gemini will be pre-integrated in the Lynx R1 headset, which starts at around $500 for Kickstarter backers. Simply put, there has never been hand tracking of this quality at that price. I’m excited to see what developers will do with it, and for future devices leveraging this tech. Meta has some catching up to do.
Ultraleap, a leading company focused on hand-tracking interfaces, this week announced it has secured a £60 million (~$82 million) Series D investment, with the goal of expanding its hand-tracking and mid-air haptic tech in the XR space and beyond.
Formerly known as Ultrahaptics, Ultraleap was formed after the UK-based haptics company acquired leading hand-tracking company Leap Motion back in 2019. The new name clearly defined the merger’s unique combination of mid-air ultrasonic haptics now underpinned by some of the best hand-tracking tech in the industry.
“With this investment round, Ultraleap will continue to bring Gemini to different operating systems and increase their investment in tooling to enable developers to build more applications using the best interface—your hands. Ultraleap will also continue to invest in R&D to drive their machine-learning-based hand tracking even further ahead,” the company said in its investment announcement.
Ultraleap is betting that hand-tracking will be the primary input for XR and the metaverse. Last month the company released its latest revision.
While the company has been trying to get its tech into the XR space for many years now, it has yet to find significant traction. Though Ultraleap hand-tracking can be found on a few headsets like those from Varjo and Pimax, leading devices implementing hand-tracking—like Quest 2, HoloLens, and Magic Leap—are using their own solutions, as far as we know.
However, with a growing number of XR devices on the market and the steady march toward consumer-friendly AR glasses, the company seems poised to find the right fit eventually.
Ultraleap is also looking to find a home for its tech outside of the XR realm. The company has long been angling its tech in the automotive space as an in-car interface, as well as the out-of-home space in areas like exhibits, marketing installations, and touchless self-service kiosks.
Every weekend VRFocus gathers together vacancies from across the virtual reality (VR), augmented reality (AR) and mixed reality (MR) industry, in locations around the globe to help make finding that ideal job easier. Below is a selection of roles that are currently accepting applications across a number of disciplines, all within departments and companies that focus on immersive entertainment.
Don’t forget, if there wasn’t anything that took your fancy this week there’s always last week’s listings on The VR Job Hubto check as well.
If you are an employer looking for someone to fill an immersive technology related role – regardless of the industry – don’t forget you can send us the lowdown on the position and we’ll be sure to feature it in that following week’s feature. Details should be sent to Peter Graham (pgraham@vrfocus.com).
We’ll see you next week on VRFocus at the usual time of 3PM (UK) for another selection of jobs from around the world.
The latest version of Ultraleap’s hand-tracking tech is finally available today on Windows for use with the Leap Motion Controller accessory and promises to improve two-handed interactions, speed, and robustness. The release includes a demo experience showcasing how hand-tracking can be used as a primary input for a standalone XR device.
Ultraleap today publicly released ‘Gemini’, the company’s fifth-generation hand-tracking software which was initially made available in a developer preview earlier this year. The improved hand-tracking software has already been deployed to headsets like Varjo’s and been made available for devices based on Qualcomm’s Snapdragon XR2, and now it can be downloaded on Windows to be used with the company’s existing Leap Motion Controller accessory which can be mounted to VR headsets. Support for MacOS and Linux are expected further down the road.
While the Leap Motion Controller is by now quite old, the company has continued to refine the software that underlies it, improving on what is already recognized as some of the best hand-tracking tech available in the industry. More recently, Ultraleap has released improved hand-tracking modules with a wider field-of-view and other improvements, though these aren’t available as a standalone accessory.
Image courtesy Ultraleap
With the Gemini update, Ultraleap says it has improved two-handed interactions, initialization speed, and the robustness of its hand-tracking. Alongside the Windows release of Gemini, the company is also making available an ‘XR Launcher’ demo experience which shows how the hand-tracking tech can be used for a fully functional XR interface.
Earlier in the year, hand tracking experts Ultraleap unveiled a preview of its fifth-generation hand-tracking software offering a number of improvements including two-handed interactions. Called Gemini, today Ultraleap has announced the official launch of this new hand tracking platform, fully compatible with Windows systems.
Already available in Varjo headsets and integrated into Qualcomm’s Snapdragon XR2 5G reference design, Gemini marks the first full software release from the company in three years. Completely rebuilding its hand-tracking engine, the core improvements see refinement in two-handed tracking, where occlusion can impair the sensors ability to distinguish between either hand when close together.
Another important enhancement is the speed at which hands are detected. Some location-based entertainment (LBE) venues utilise Ultraleap hand tracking, if you’ve experienced one then you may have noticed that when your hands are out of sight bringing them back would take a few seconds. That should now be rectified with Gemini.
Ultraleap also notes that Gemini offers: “Improved robustness to challenging environmental conditions,” so a user’s hands should be much more easily tracked if lighting conditions suddenly change, for example. To showcase these new features Ultraleap check out the new XR launcher experience in the trailer below.
“Ultraleap Gemini is the next step in bringing intuitive and seamless XR experiences to life by making natural interaction between humans and technology possible. With ten years of continuous innovation and experience from supporting over 350k developers, we have refined our hand tracking to deliver the best and most flexible hand tracking platform available,” said Matt Tullis, Product Director, Spatial Computing, Ultraleap, in a statement. “We are looking forward to working with software and hardware partners to bring hand tracking and XR to the masses.”
The new Gemini system is available to download today and works with both the Leap Motion Controller and Stereo IR 170 camera module. For further Ultraleap updates, keep reading VRFocus.
