Augmented & Virtual Reality Confabulation
What Augmented Reality and Virtual Reality developers are talking about
Multiple patents awarded to Facebook this year suggest the company is researching a range of technologies which could enable force feedback gloves for VR.
Gloves may be the ultimate goal for VR input. The term “haptic gloves” usually refers to finger tracking gloves with vibration motors on each finger. Force feedback gloves, though, go further by restricting the movement of fingers in response to a simulated object or surface.
Facebook previously showed research involving a haptic glove, but not force feedback gloves. These three patents, however, provide insight into some of the company’s research.
Note these are not applications, but actual awarded patents.
In March of this year, Facebook was awarded a patent titled Switchable fluidic device.
The patent describes a glove with “soft materials that use millimeter or smaller channels filled with fluid“. By controlling the flow of fluid through these tiny channels, the system adapts the pressure it applies to the finger joints.
When the pressure is high, it “prevents or enables a physical movement of a portion of a user in contact with the virtual object in the virtual space“, according to the patent description. “For example, if a user’s finger is in contact with a virtual object (e.g., a virtual wall) in a virtual space, the haptic assembly prevents a physical movement of the user finger to move in a direction through the virtual object in the virtual space. Accordingly, the user can receive a perception of contacting the virtual object.”
One “embodiment” of the gloves is described as covered with infrared LEDs for positional tracking from cameras, just like an Oculus Touch controller.
In June, Facebook was awarded another patent related to haptic gloves, titled Haptic devices that simulate rigidity of virtual objects.
Almost all gloves to date use vibration motors to mimic the feeling of touching virtual objects, however, this is not particularly realistic. The glove in the patent instead uses an array of plates which dynamically actuate to touch the user’s finger with a force simulating the object the user’s finger is touching.
When pressing against a hard virtual object such as a button, the plates would resist the pressure of the finger. When pressing a less rigid virtual object the plates can “give” much more easily to pressure, providing the feeling of softness.
Last week, Facebook was awarded yet another patent. This time the idea covers an alternative to microfluidics for force feedback: Pneumatically controlled haptic mechanisms with nested internal structures for haptic feedback.
The glove is covered in an array of stacked “pods” which each contain a pneumatic “bladder” made of “a durable, puncture resistance material, such as thermoplastic polyurethane (TPU)”. By adding and removing air from each bladder, the pressure against the user’s hand at that position can be altered: “Depending on a posture of the user’s finger when the pressure inside the bladder is increased, the user may experience his or her finger becoming stiff and rigid, bending downwards, or bending upwards (e.g., pushing and pulling sensations).”
This pneumatic technology, because of the higher pressure it can generate, could simulate more intense events, even pushing back on the fingers rather than simply restricting them. The patent describes how this could be used in VR games:
“just prior to releasing an arrow from a bow in real life, a tremendous force is applied to the pads of the fingers drawing the bow string. Therefore, in virtual reality, the haptic stimulation created by the wearable device would need to be intense to provide some realism to the virtual reality experience (e.g., one or more pods on each string-contacting finger push against the string-contacting fingers and attempt to straighten these fingers, as would the bow string in real life)”
While such a system may not be reliable for hours of usage per week, the patent describes designing with this in mind: “Due to the ever-changing nature of virtual and augmented reality, the pods may be required to transition between the two states hundreds, or perhaps thousands of times, during a single use. Thus, the pods described herein are durable and designed to quickly transition from state to state.”
At Oculus Connect 3 in 2016, Oculus Chief Scientist Michael Abrash made a series of predictions about the future of VR technology. Among these was the claim that Touch-like VR controllers would be the state of the art “for at least 5 years”, and “maybe” much longer. Abrash postulated that controllers could potentially be “the mouse of VR”, and remain so even “40 years from now”. He followed this by saying:
“The only thing I can see replacing Touch-like controllers is the ability to use your hands as direct physical manipulators as you do in the real world, and I don’t see that happening in the next 5 years because it requires haptic and kinematic technology that isn’t even on the distant horizon.”
At Oculus Connect 5 in 2018, however, when revisiting these predictions, Abrash changed his outlook:
“I still don’t think it’ll happen in the next 4 years, but something interesting may in fact be on the distant horizon. […] I believe we’ll have useful haptic hands in some form within 10 years”
It is possible research related to these patented ideas — or perhaps some similar research underway at Facebook — may have informed Abrash’s updated guidance. Of course, organizations of Facebook’s size file patents all the time on lots of ideas that never see actual use.
Facebook Reality Lab, the company’s VR/AR research division led by Abrash, has been on a hiring spree for years now. In Facebook earning calls, the company mentioned increasing investment in VR/AR research.
The post Facebook Patents Reveal Deep Research On True Haptic VR Gloves appeared first on UploadVR.
At the World Haptics Conference 2019 Facebook showed a prototype wrist-worn haptic VR/AR device called ‘Tasbi’.
The device was created by a team of six Facebook Reality Labs researchers along with an interning PhD student from Rice University.
Tasbi is worn on each wrist. It uses a combination of vibrations and squeezing (dynamic tension adjustment) to mimic haptic feedback for virtual objects. Each “tactor” in the band includes an individual linear actuator, so vibrations can be precisely controlled.
In software, visual tricks can suggest the press of a button by showing a finger ceasing movement once it collides with the object. The researchers found the visual tricks only impacted perceived realism in a minor way. The wrist-based haptic device provided most of the sensation even though it doesn’t touch any fingers.
Unlike previous attempts at this technique, each tactor in the Tasbi uses a smooth pin such that it is decoupled from the band, which the researchers claim produces “pure, uniform normal forces while maintaining tactor radial positions”.
The device does not perform finger tracking in itself. This could potentially be done by computer vision software analyzing the imagery from headset mounted cameras. Another interesting possibility is that the device could use a probing electrical signal to detect the position of your fingers, a technique Facebook explored enough to file a patent on earlier this year.
Of course, a device like this on its own would not be a suitable direct replacement for Touch controllers. It would not be a great fit for traditional gaming given the lack of physical resistance, thumbstick or buttons. However, if the haptics are as convincing as the researchers say, then Facebook might be able to use the research alongside, or as part of, a next generation version of Oculus Touch. The company could also ship it as a non-gaming controller for a future Oculus Go, or even for its upcoming AR glasses.
A Facebook representative told us that the research paper for Tasbi will be published “in a few weeks”. We’ll make sure to give it a read and update the article with any new insights it provides.
The post Facebook’s Researchers Made A Wrist-Worn Prototype For Haptic Feedback And Free Hand Movement appeared first on UploadVR.