Here’s a video of one of VR’s best games being played with its most promising new controllers.
Brian Lindenhof, a VR developer best known for his work on Climbey, has uploaded a video of him playing Ready at Dawn’s Oculus Rift exclusive, Lone Echo, using the Revive app. The new controllers, which were shipped out to developers earlier this month, seem to work pretty perfectly with the game, right down to replicating Oculus Touch’s limited finger tracking (but obviously not providing the full extent of Knuckles’ own finger tracking).
Lindenhof also has an older video of himself using the Knuckles with Bethesda’s Fallout 4 VR (after failed attempts to boot up Skyrim in VR). There’s a bit of tweaking at the start of both videos but, at the very least, Knuckles seems to be more than capable of imitating the controllers that have come before it.
Of course, it’s difficult to judge just how improved the experience is without seeing a game that natively supports Knuckles. Right now the only app to do that is Valve’s own demo for the controllers, set in its Portal universe.
We don’t yet know when and how Valve plans to ship Knuckles out to VR fans, nor how much they’ll cost. Hopefully with kits now in developer’s hands, though, it won’t be too much longer.
A new generation of the Valve Knuckles controllers are rolling out to developers with thumbsticks added on. The touchpad is reduced to a small strip on the updated controllers.
The new “force sensors can be used in conjunction with capacitive sensors to enable higher fidelity interactions. With these two sensors working together, we can detect the full range of the hand position – from completely open, to holding the controller lightly, to gripping it tightly. This helps us understand user intent and makes things like ‘pickup and throw’ with Knuckles EV2 much improved.”
Here are some videos showing the new controllers in action:
As part of the roll-out, a new tech demo called Moondust set in the Portal universe will be available to “hundreds” of developers receiving the new controller prototypes.
Software developer Jimmy Gray is working on a modification to HTC Vive’s controllers so they work much like Valve’s Knuckles prototypes, with finger tracking.
Gray is working on a number of VR input devices. With Tactil (formerly known as Dactyl), he’s built a modification that allows “users to have finger tracking with the current Vive controllers,” Gray wrote in a message.
My original goal was to see if I could emulate the Valve Knuckles prototype using the existing Vive controllers, and to see if it could make sense functionally and ergonomically. From the start, I knew that it would be easiest to start prototyping with flex sensors, and since I had a working hardware and software stack with another DIY project I was working on I then found it easy to get up and running in terms of a wireless hardware solution. I started doing some CAD modeling for the actual plastic pieces that mount to the Vive controllers. I ended up coming with a working design prototype, although the finger straps are a bit too short still, and I’m currently working on those.
He took inspiration from a Kickstarter that eventually closed down and then “rigged some rubber bands to the wrist strap included with the Vive controller.”
Gray posted to the r/Vive subreddit and was encouraged by the response. He then made a video showing the design in action.
Gray is considering a crowdfunding campaign for the accessory with the possibility of open-sourcing the design for the VR community to work with “while still funding a batch of kits to send out to devs and/or everyday Vive users who eventually might want to emulate the Knuckles controllers with their existing Vive controllers.”
He wants to build a software development kit so creators can use the extra functionality of his controllers in their designs. He’s also hoping to eventually emulate the actual Knuckles controllers “so that they’d more or less just work out of the box.”
Gray is still trying to improve the modification, hoping to make it work with the Vive Tracker and offer full five finger tracking.
Right now the Vive controller accessory is what I’m trying to focus on getting into the hands of Vive owners and developers. Once the Valve Knuckles hit the market, if they ever do, I hope my solution could not only have parity with them via being able to emulate them, but that my solution could also provide other features that developers and users would be interested in utilizing.
Let Gray know in the comments what you think and we’ll bring you any big updates about the controllers.
Valve is developing a pair of more immersive hand controllers currently called “Knuckles” that promise to offer more precise finger tracking and grasping sensations in VR. Though the controllers are still in a prototype phase and likely to change, Valve released early versions to developers over the last few months and interest is high to see how they can be used to make virtual worlds more immersive.
Last month Denny Unger from Cloudhead, the studio behind The Gallery, shared some thoughts based on his hands-on experience with the controllers and over the last several weeks Climbey developer Brian Lindenhof has been testing the controllers in a variety of virtual worlds from both the Oculus store and Steam. I asked Lindenhof over direct message whether, even in this prototype state, he prefers Knuckles to the Oculus Touch or the Vive wands.
“Definitely the Knuckles,” he wrote.
The controllers fit around the palm of your hand and strap in snugly behind your knuckles. You pull a short cord to tighten the fit. Initially, Lindenhof noted the controllers could get slightly uncomfortable after a few hours and pinch his hand just a bit. Now he says he’s “grown more used to them” and doesn’t feel the pinching anymore.
“[I] regularly play with them until they drop dead now,” Lindenhof wrote. “Which is 3-4 hours currently.”
That is just one developer’s opinion at this point but that sort of rigorous use and preference for the hardware even in a prototype state bodes very well for Valve’s efforts to push immersion further than Facebook’s Oculus Touch controllers. It sounds like the current controllers are great for simulating the feeling of grasping and releasing objects with handles requiring several fingers, but pinching an object between just your index finger and thumb might not be ideal yet. In other words, it may be perfect for grasping a bow or a cup’s handle but a picking up a spoon from a drawer with just two fingers may not feel exactly like the real world.
The developer behind the Revive hack which lets people access Oculus store games with an HTC Vive headset reportedly also received the controllers, and Lindenhof has been using the hack in both Lone Echo and Echo Arena, two Oculus exclusive titles. He’s been recording his experience and thoughts at length on his YouTube channel and shared a few of the videos with us.
User interfaces in virtual reality start with your hands. We know that now with the Oculus Touch and the HTC Vive, but even when VR was simply a screen strapped to your head many felt that hands were the future. We began to develop The Gallery on the first Oculus devkit with the Razer Hydra (a Sixense technology) to deliver surrogate hand tracking and body presence back in 2013. And when Valve invited Cloudhead and the first wave of developers to see what would eventually be known as the Vive, Valve showed a commitment to that same vision. Now Valve has invited us and a new wave of developers to begin working with their latest prototype—the SteamVR Knuckles, a wearable VR controller that tracks not just your hand, but each finger too.
Back in 2014, we didn’t realize exactly how accurate SteamVR tracking was—the whole notion of roomscale VR was almost incomprehensible. Up until that point, there was still some motion latency in VR, so you never felt completely attached to the actions in your hands. But once the SteamVR Lighthouses started tracking objects in a 3D space, it was a deep and immediate connection of, “Holy crap, that’s actually my hand in VR.”
Our goal then was to deliver an experience where the player doesn’t have to think about the controller, and has only natural, gestural interactions. We wanted to demonstrate why this kind of input—your hands themselves—was meaningful. When we received our first Vive devkit (wired at the time) we were taping them to our hands in order to feel more immersed, and we even spoke to Valve about crude ways they could strap the controller on.
Those early Vive prototypes already showed an incredible level of fidelity, capable of measuring the tiniest fraction of a movement. It’s like the Moore’s Law of motion control; each incremental improvement in tracking brings with it new possibilities. So as soon as you have that kind of fidelity with your hands in VR, you need your fingers to be more purely represented. And that’s what the Oculus Touch started to do by bringing capacitive, gestural input to the controller.
Where Touch differs from what the Knuckles offer, however, is that you’re still pushing a binary button in the end; Touch feels more grounded in traditional gamepad design. Specifically, you always feel like you’re holding something with buttons—and that works perfectly for gun games and sword games. But the Knuckles take that further by removing the abstractions of first-gen VR inputs. Even though it looks like a more complicated device, it’s actually a much simpler one.
With the Knuckles, you’re not holding a controller; it simply straps to your hand and rests in your palm. If you relax your hand into a natural flex, the controller stays put and keeps tracking your fingers. If you reach out to grab an object in VR, your hand wraps around the base of the controller, giving a tangible feeling of grabbing something. That physicality is something you don’t get from data gloves, or vision based inputs without any device, and that feeling can then be fine-tuned with haptic feedback. Plus, you’re not passing around a sweaty data glove between your friends.
When reaching out to an object with the Knuckles, I’m not thinking about the controller I’m holding in my hand, because I’m not holding one. I’m not thinking about how to use my fingers either, because they’re not assigned to a button press. I’m not even thinking about my hand, and that’s where the magic comes in—I’m just thinking about grabbing the object, as I would in real life. That entire grasping motion is represented in VR, whether I pinch with two fingers, scoop with my hand, or close my fist around it. The Knuckles track your fingers by the distance they are from the base of the controller (your palm), and represent that movement in VR. It’s second nature.
As developers, when we receive prototype hardware like the SteamVR Knuckles, it makes us want to push the capabilities. In the past, that’s meant radically rethinking our stack of interactions and locomotion systems—virtually redesigning the game. We’ve written about what these kinds of changes have meant for The Gallery in the past, but the long story short is that new controllers like the Knuckles aren’t just affecting the complexity of interactions. We now have new possibilities for game mechanics and design that haven’t and couldn’t have been done before. It’s to be seen how these controllers will impact Heart of the Emberstone in September, but they’ll be a core focus in designing Episode 3.
Think of an interface-heavy app like Tilt Brush. Dials can be intuitive, but using your fingers is organic. There’s a possibility for gestural movements to call functions and navigate dense data; there could be an entire language built out of using your hands to manipulate paint brushes and pencils and sizes and colors. Once you take the mental load of an interface off the player—once they stop thinking about the controller—you can leverage that partition into experiential design and organic controls.
The kind of technology that the SteamVR Knuckles offer is not just impactful to the future of input in gaming, but also the future of output. I can go to a social VR space and point to something, or offer a peace sign, or tell somebody to hang loose without having to think about it. The controller doesn’t guess your gesture, or snap to a new position, it represents your fingers based on the distance it calculates. The more natural and intuitive the interface, the less we think about hardware. And the less we fixate on hardware, the more present we can be in VR.
Prior to the Knuckles, hardware developers were looking for something that would be more broadly accepted by the general public. Something that resembled a Wiimote, like the Vive wands, or something that when put together resembled a gamepad, like the Touch controllers, meant that VR input was familiar. Strapping an alien device to your hand in first-gen VR would have been too much too fast. But I think it took the evolution of those two controllers to get VR to the point where the public could be comfortable with the idea of a controller strapped to your hand.
There are so many moments in life in which using your hands is a vital part of the experience. There are implications for education and communication—with audiences who don’t generally understand videogame controllers—because the SteamVR Knuckles open the door to that broader audience. These are pick-up-and-play controllers where you don’t have to think about the input, you just reach out and interact in virtual space.
User interface in virtual reality starts with your hands. And once users are empowered in that way, and don’t have to be told how to use the technology, the next generation of virtual reality is here.
—
This is a guest post not produced by the UploadVR staff. It’s a contribution submitted by Denny Unger, the Co-Founder, CEO, and Creative Director of Cloudhead Games. No compensation was exchanged for the creation of this content.
Less than a week ago new models of Valve’s prototype Knuckles controller surfaced in SteamVR Home. The device is an attempt to innovate in the VR space with controllers that allow you to grab and release in VR without being concerned with dropping a controller.
The key to the Knuckles controller is being able to freely manipulate items in a virtual experience without worrying about dropping a handheld controller while doing so. VR controllers, in their current forms, have straps simply because some people may accidentally have the natural inclination to release them when mimicking certain types of actions like swinging a sword or throwing an object. We’ve seen what the Knuckles controllers will look like and this demo shows off how they work.
There are sensors on the Knuckles grip that will activate individual fingers and the video starts off showing how, using your index finger and thumb, you can pick up something like a coffee mug in a natural way and then toss it across the room. We also get to see how we can hold an object in place in our hands using different fingers or use our open hands to spin a globe without having to press a button to interact with it.
This type of intuitive control is going to open up lots of doors for VR creators as long as Valve finds a way to get it into homes at a decent price. If you have a prototype and/or want to check out the demo and the source for the interaction, Zulubo has made it all available via GitHub.
Valve confirmed to UploadVR the company has progressed to sending developers prototypes of its next generation VR hand controllers.
This was to be expected but provides us with a rough sense of the advancements being pursued by the VR technology company. Recent updates from Valve, which partnered with HTC to create the Vive,
The controllers are being seeded to developers to gather their feedback, and only being built in a limited quantity in this version, according to Valve. The new controllers certainly look slim and ergonomic, and we wonder what kind of release timeline Valve has for them.
We know Valve is developing multiple VR games internally alongside the controllers, and exploring game design alongside these new interaction paradigms puts the company at the forefront of VR development with Facebook, Microsoft and Google.
While Valve is more commonly known as the company behind Steam, which is effectively the global App Store for PC gaming, the company also has millions of fans of its games like DOTA 2, Half-Life, Portal, Counter-Strike, and Team Fortress. As a privately-owned company with a flat management structure, Valve also remains a bit of a wild card in the fast-developing VR ecosystem. Valve released its innovative tracking technology royalty-free, and is releasing a second generation that has potential to upend the market currently occupied by expensive cameras used to track VR headsets through warehouse-sized spaces.
Nevertheless, the Vive headset Valve partnered with HTC to make remains incompatible with content purchased from Facebook’s Oculus store. The Rift remains compatible with content purchased from both Steam and Facebook’s own Oculus store. There is a hack to work around the division on Vive, but that is far from direct compatibility with the Oculus store and makes using some of the Oculus-exclusive games difficult to access with HTC’s headset. The issue has been a frustrating one for early adopters who want to buy the best content available and play it on whichever headset they own.
This is relevant because Microsoft is soon to enter the fray with its own controllers and line of headsets, and Windows 10 is evolving to support Microsoft-backed VR headsets from some of its PC manufacturing partners. It is unclear where Vive and Rift will sit in the evolving Windows ecosystem even as Microsoft plans a push for its plans by the holidays this year. It should be noted Valve also partnered with Apple now and is bringing SteamVR system to Mac computers.
We will report back when we know more about the distribution and release plans for Valve’s new “knuckle” controllers. Let us know what you think down in the comments below!
