Augmented & Virtual Reality Confabulation
What Augmented Reality and Virtual Reality developers are talking about
Welcome to Episode 13 of the VRecap! Sorry we’re a few days late! These usually post on Fridays but we ran into some scheduling issues on the team that delayed things.
Competition: https://uploadvr.com/vrecap-13/ Prizes provided by 2K and Gearbox
Narrated by David Jagneaux
The post VRecap: Microsoft Research, Budget Cuts 2 Date, and Borderlands 2 VR Giveaway appeared first on UploadVR.
Welcome to Episode 13 of the VRecap! Yeah, we know, it’s Monday and not Friday and this is David not Jamie or Zeena, but you make due with what you’ve got.
This episode is a few days later than usual, we know, stop yelling! Last week we had a lot of weird scheduling issues on the team and were never really at full force at any point, so Zeena edited the video itself, I did the VO for the first time on this show, and we never had time to finish and post it — but here it is!
There were some really cool news stories last week for VRecap, like the city-scale research project, new eye-tracking revelations, and haptic feedback prototypes all from Microsoft. And Borderlands 2 VR finally launched on PC — but it had a slew of issues right out of the gate. We’ve also got a ton of new VR release dates for you as well for some big titles like Budget Cuts 2 and more.
Finally — this week’s giveaway is for Borderlands 2 VR on PC via Steam! Now if you’ve got a Valve Index or HTC Vive, proceed with caution, but it seems to be working okay for Oculus Rift users at this time. You can either click here or down in the widget form below:
GIVEAWAY: Win A Free Copy Of Borderlands 2 VR On Steam
Thanks so much for tuning into another VRecap! You can expect the VRecap to return to Fridays, more than likely, this week and make sure you tune into The VR Download every week for the latest news and big interviews all from within our virtual studio.
Have fun in VR!
The post Microsoft VR Research And Borderlands 2 VR Launch Issues And Giveaway! – VRecap appeared first on UploadVR.
Microsoft unveiled an experiment this week that explores the future of always-on virtual reality. Building a system called DreamWalker, Microsoft researchers walk around in the physical world while still being fully immersed in VR, essentially taking the first steps into replacing your morning walk with something that’s not only reactive to your physical surroundings, but ultimately more interesting.
To do this, DreamWalker fuses a Windows VR headset’s inside-out tracking, two RGB depth sensors, and GPS locations. The system, the researchers maintain in their paper, can continuously and accurately position the user in the physical world, sense walkable paths and obstacles in real-time, and represent paths through a dynamically changing scene in VR to redirect the user towards the chosen destination.
Created by Jackie Yang, Eyal Ofek, Andy Wilson, and Christian Holz, the experiment shows it’s clearly still early days for always-on VR—the system requires a backpack-mounted computer and a load of other gear—however DreamWalker poses some interesting questions related to how such a system could (or rather should) be shaped around a dynamic world.
In the paper, the researchers pose a few methods of keeping you safely on your path, including obstacle avoidance techniques like digital pedestrians moving you away from potential danger and controlling a user’s paths with dynamic events such as vehicles being parked in front of you (video linked below). And while randomly spawning traffic cones or a wild gang of pedestrians herding you to your destination may seem like inelegant solutions for now, there’s no telling what a smarter, more integrated system may hold in the future.
For all its marketing bloat, Magic Leap poses such a highly integrated system in its hypothetical Magicverse that would necessarily require a fairly complete understanding of the physical world, including pre-mapped and digitized streets, buildings—everything so you could potentially ‘reskin’ the world to a varying extent.
Microsoft’s method is decidedly less involved than Magic Leap’s moon-shot idea, which thus far has only been presented as a hypothesis more than object of active experimentation. The researchers instead take the user’s planned walking area and match it up as best they can with a digital map, introducing re-directed walking when needed to curb the user from veering off course.
Of course, neither AR or VR hardware is capable of doing all of this for now, however it’s not out of the question for future devices. Whatever off-the-shelf parts the researchers have strapped together are more than likely to find themselves on future headsets in some form, which includes greater rendering power, better computer vision, and on-board GPS for better location-based play.
Moreover, these early steps are predicated on the very real assumption that the better AR/VR gets, the more time we’ll spend interacting in digital environments, making something mundane like a walk through the park as exciting and novel as creators can make it.
Check out the four-minute video below to see the research group’s findings it in action:
The post Microsoft ‘DreamWalker’ Experiment Takes First Steps into Always-on World-scale VR appeared first on Road to VR.
Microsoft Research showed off a prototype VR controller which provides force feedback for grasping small objects with the thumb and index finger.
The researchers claim that most approaches to resistance in VR hardware are bulky, costly, and require significant power to reach human-scale forces. This is because they use large powerful actuators.
CapstanCrunch’s solution to this problem is to use a “capstan brake and clutched spring”. This is a kind of mechanism which allows the force of the relatively small motor to be amplified 40 times such that it can sufficiently resist human applied forces. This would use significantly less energy than previous approaches.
The mechanism applies to the thumb and index finger. CapstanCrunch can simulate holding rigid objects by applying the force when the user’s finger reaches the position of the virtual object. It can also simulate softer objects such as stress balls by increasing the force as the user presses in further.
But the mechanism also allows for more advanced interactions. Two examples Microsoft gave were connecting two lego blocks or cutting a material like cardboard with scissors. These kind of interactions are simply not compelling with current VR hardware, so like many real world interactions, VR developers currently just avoid them.
Back in May, Microsoft showed off another VR controller prototype, TORC. TORC is also used for grabbing small objects, however its focus is on simulating their textures and edges rather than the resistance of grabbing them. We’d be interested to see if these two prototypes could be integrated to provide both at the same time.
It’s important to note that like TORC, CapstanCrunch is just a feature prototype. That means it incorporates one feature that a next generation VR controller might have, but lacks other essential functionality such as tracking, thumbsticks, buttons, or any handling of the other three fingers.
So it’s unlikely that CapstanCrunch will directly become a product. It’s just one piece in a much larger puzzle. But Microsoft could incorporate the concept into a future controller for Xbox ‘Scarlett’ or a second generation of Windows MR. We’ll be keeping a close eye on Redmond for any signs of next generation VR hardware.
The post Microsoft Shows Off Force Resistance VR Grasping Feature Prototype appeared first on UploadVR.
When it comes to combining both the real and digital worlds companies are making great strides in both software and hardware, from videogames like Reality Clash to devices like HoloLens 2, augmenting the world around us. Microsoft has been experimenting with taking this one step further, combining the real world with virtual reality (VR), allowing someone to take a normal walk yet wander around a completely different location.
In a recent blog posting this week Microsoft Research revealed several new VR solutions the company has been working on, one of which is called DreamWalker. Imagine walking to work, the same old route you’ve walked countless times, suddenly turning into a journey through an exotic city.
Using a variety of sensing technology including a dual-band GPS sensor, two RGB depth cameras, and a Windows Mixed Reality headset with inside-out tracking, DreamWalker allows for a real walk to be virtual adapting to a particular route to maintain the immersion. “To accomplish this task, the technology first plans users’ paths in the virtual world and then uses real-time environment detection, walking redirection, and virtual world updates once users have embarked on their journeys,” the blog notes.
Should the system discover an object either stationary or moving it’ll introduce a virtual one to stop a user bumping into something or walking into danger. Creators can also introduce options for controlling a users path so the virtual and real align, such as animals or dynamic events like a parking car.
Another project being developed is CapstanCrunch, a haptic controller which magnifies the force supplied by a hand to create sensory feedback. Normally this is done by way of a large motor but in CapstanCrunch only a small motor is used alongside a linear and directional brake.
“A key component of the brake—and the inspiration behind the controller’s name—is the capstan, a centuries-old mechanical device originally used to control ropes on sailing ships. “It’s an old technology that people used to tie boats, but we use it in a different way that right now enables us to multiply the force of a small internal motor,” says Eyal Ofek, a Senior Researcher.”
Using this method the team notes: “The capstan magnifies an input force by around a factor of 40”, helping reduce the size and weight of the design.
Microsoft Research will be in New Orleans at the ACM Symposium on User Interface Software and Technology (UIST) 2019. to showcase the projects. For further updates keep reading VRFocus.
In Steven Spielberg’s 2018 adaptation of Ernest Cline’s 2011 novel Ready Player One, on the Planet Doom in the network of simulated places known as The OASIS there’s a climactic battle for control over that fictional multiverse.
In the film, set decades in the future, fans of The OASIS run down physical streets decked out in VR gear while their systems seamlessly map virtual worlds onto their physical surroundings — thus maintaining the illusion indefinitely. Sure it looks pretty socially awkward and raises uncomfortable questions about the value people place on their physical environment, but it still represents a remarkable technical achievement if it could actually be done:
Even technical viewers, though, steeped in VR technology might look at such depictions and wonder if this sort of thing would ever be feasible. It isn’t readily apparent, for instance, how a virtual world could compensate and change to allow for navigation while strolling through public spaces outdoors.
Turns out Microsoft researchers are already figuring out those answers. The publication of a new paper presented as part of the ACM Symposium on User Interface Software and Technology (UIST) titled “DreamWalker” comes from Stanford University PhD student Jackie Yang, who was a Microsoft Research intern during the work, and Microsoft researchers Eyal Ofek, Andy Wilson, and Christian Holz. The paper reveals in detail how they built a city-scale redirected walking system for VR on top of a Samsung Odyssey headset and additional sensing hardware.
The paper’s full title is“DreamWalker: Substituting Real-World Walking Experiences with a Virtual Reality” and you can read the entire 14-page PDF yourself.
In summary, though, “we explore a future in which people spend considerably more time in virtual reality, even during moments when they walk between locations in the real world.” DreamWalker builds on earlier research while working “in unseen large-scale, uncontrolled and public areas on contiguous paths in the real-world that are void of moving vehicles.”
According to the paper:
“The Windows Mixed Reality system provides inside-out tracking on a Samsung Odyssey VR headset, updating sensed 6D locations at 90 Hz. Empirically, we measured 1 m of drift over a course of just 30 m through the inside-out tracking alone. Two Intel RealSense 425 cameras provide RGB depth images, slightly angled and rotated 90 degrees to achieve a large field of view (86◦ × 98◦). We built a custom adapter for the backpack computer that converts Thunderbolt 3 to four USB 3 ports and thus supports the bandwidth required to stream both RGB depth cameras at a resolution of 640×480 (depth) and 640×480 (RGB) at 30 Hz. Finally, GPS data comes from the sensor inside a Xiaomi Mi 8 phone…”
The paper describes how measurements from all these devices is combined and analyzed to keep the person in VR on a safe and comfortable path through the real world without ever taking the headset off such that “8 participants walked across campus along a 15-minute route, experiencing a virtual Manhattan that was full of animated cars, people, and other objects.”
In the video below you can see different ways the software encourages people to stay on the right path, including using “humanoid” animated virtual characters which “move into the location of detected obstacles and guide the user towards the destination.”
The conclusion for the paper states “each participant confidently walked for 15 minutes in DreamWalker, which showed the potential of our system to make repetitive real-world walking tasks more entertaining.”
The post Microsoft Researchers Built A City-Scale Redirected VR Walking System appeared first on UploadVR.
This prototype haptic VR controller from Microsoft Research suggests where Valve’s Index controllers could go next.
Developed at KAIST University, TORC is a haptic controller offering ‘High-Dexterity Finger Interaction’. In its prototype stage, it uses a Vive Tracker for movement. The controller has a trackpad for your thumb protruding from one side. The user places their index and middle fingers on the back on this pad.
In VR, this simulates grabbing an object with those two fingers and your thumb. Sliding your thumb over the trackpad rotates the object in question. It doesn’t look entirely natural, but it is an interesting solution for object manipulation in VR.
Crucially, TORC uses a pressure sensor to detect when the user is squeezing an object. For elastic objects, this could simulate squeezing. Whereas Valve’s Index controllers can only detect when your fingers are touching the controller and when they aren’t, TORC could ensure users are physically gripping virtual objects. The thumbpad, meanwhile, is able to simulate textures when moving it.
This being a prototype, though, TORC doesn’t seem to have other classic VR features like triggers and buttons. There’s no thumbstick for locomotion, for example. We’d also like to see a version of the controller that utilizes all of the user’s fingers.
TORC certainly doesn’t have all the answers for haptic feedback in VR, then. But it does look like an interesting middle ground between something like the Index controllers and more robust haptic systems like HaptX. The latter systems are still complex, expensive and likely someways off from consumer viability. TORC, meanwhile, certainly looks like it would at least fit alongside a modern VR controller. Whether or not anything will come of it remains to be seen.
The post Microsoft Research Demos Index-Like Haptic VR Controller appeared first on UploadVR.
Microsoft Research showed off a new haptic VR controller which simulates grabbing small objects with the thumb and first two fingers, delivering rich haptic feedback. The controller is codenamed TORC— TOuch Rigid Controller.
The user’s thumb rests on a capacitive surface. The two fingers rest against linear actuators (also used in the Oculus Touch and HTC Vive controllers) to give targeted haptic feedback. Inside the device near the thumb is a force sensor.
Users can grab virtual objects by simply applying a grabbing force, or let go by simply removing that grabbing force. Objects held in the hand can even be rotated by swiping the thumb across the capacitive surface.
TORC was designed to even allow for squeezing of objects. While your fingers don’t move, Microsoft claims the targeted haptic vibrations combined with the squeezing you see in VR trick your brain into feeling as though you’re actually squeezing. The company explained that this emerged from their neuroscience research on “dynamic visuotactile stimulation”.
The prototype uses an HTC Vive Tracker for positional tracking, but that can be swapped out for any tracking system wanted. The innovations here are in the application of force sensors and haptics, not related to tracking.
The company also showed a drawing depicting how this technology could look directly integrated into a product:
The drawing bares a striking resemblance to the existing Windows MR controllers used on headsets such as the HP Reverb. While Microsoft describes TORC as a “proof of concept”, if it can be miniaturized and mass produced, it’s possible it could become part of the next generation Windows MR controllers one day.
Valve recently opened preorders for the Index Controllers, which feature basic finger tracking and allow the user to let go. But VR controllers still have a long way to go to reach their ultimate goal of simulating arbitrary objects- projects like TORC are a small step on that journey.
Tagged with: haptic controller, microsoft research, vr controllers
The post Microsoft Shows Off Haptic VR Controller, Simulates Grabbing With Thumb & Two Fingers appeared first on UploadVR.
Microsoft Reseach, in partnership with Cornell University, developed a range of techniques to make virtual reality accessible to the visually impaired.
VR is a heavily visual medium. Most VR apps and games assume the user has full visual ability. But just like in real life, some users in virtual environments are visually impaired. In the real world a range of measures are taken to accommodate such users, but in VR no effort has yet been made.
The researchers came up with 14 specific tools to tackle this problem. They are delivered as engine plugins for Unity. Of these tools, 9 do not require specific developer effort. For the remaining 5, the developer of each app needs to undertake some effort to support them.
It’s estimated that around 200 million people worldwide are visually impaired. If Microsoft plans to release these tools as engine plugins, it could make a huge difference in these user’s ability to use virtual reality. For VR to succeed as a medium it must accommodate everyone.
Magnification Lens: Mimicking the most common Windows OS visual accessibility tool, the magnification lens magnifies around half of the user’s field of view by 10x.
Bifocal Lens: Much the same as bifocal glasses in the real world, this tool adds a smaller but persistent magnification near the bottom of the user’s vision. This allows for constant spatial awareness while still enabling reading at a distance.
Brightness Lens: Some people have different brightness sensitivity, so this tool allows the user to adjust the brightness of the image all the way from 50% to 500% to make out details.
Contrast Lens: Similar to the Brightness Lens, this tool lets the user modify the contrast so that low contrast details can be made out. It is an adjustable scale from 1 to 10.
Edge Enhancement: A more sophisticated way to achieve the goal of the Contrast Lens, this tool detects visible edges based on depth and outlines them.
Peripheral Remapping: This tool is for people without peripheral vision. It uses the same edge detection technique as Edge Enhancement but shows the edges as an overlay in the center of the user’s field of view, giving them spatial awareness.
Text Augmentation: This tool automatically changes all text to white or black (whichever is most appropriate) and changes the font to Arial. The researchers claim Arial is proven to be more readable. The user can also change the text to bold or increase the size.
Text to Speech: This tool gives the user a virtual laser pointer. Whichever text they point at will be read aloud using speech synthesis technology.
Depth Measurement: For people with depth perception issues, this tool adds a ball to the end of the laser pointer, which lets them easily see the distance they are pointing to.
Object Recognition: Just like “alt text” on images on the 2D web, this tool reads aloud the description of virtual objects the user is pointing at (using speech synthesis).
Highlight: Users with vision issues may struggle to find the relevant objects in a game scene. By simply highlighting them in the same way as Edge Enhancement, this tool lets those users find the way in games.
Guideline: This tool works alongside Highlight. When the user isn’t looking at the relevant objects, Guideline draws arrows pointing towards them.
Recoloring: For users with very serious vision problems, this tool recolors the entire scene to simple colors.
Tagged with: Accessibility, microsoft, microsoft research, VR research
The post A Closer Look At What Microsoft’s SeeingVR Offers The Visually Impaired appeared first on UploadVR.
Microsoft researchers have created a tool that could make VR more accessible to users with impaired vision.
Called SeeingVR, the tool kit is designed to address a problem that affects users with low vision, or vision that can’t be corrected by prescription glasses.
As reported by Engadget, Microsoft researchers teamed up with Cornell Tech and Cornell University to create SeeingVR, a kit intended for Unity developers to integrate into their VR projects. SeeingVR includes 14 different tools that address a variety of vision issues.
The team’s purely visual tools include a magnification window, bifocal magnification, brightness and contrast enhancements, edge enhancement, an overlay that lets you see a miniature outline of you peripheral vision, configurable text (color, boldness, and size), depth measurement tool, a ‘guideline’ tool for better user orientation, a whole-scene recoloring tool, and an object highlight tool—similar to edge enhancement, but for specific objects in the scene.
The team also included audio-based enhancements including text-to-speech, audible object descriptions, and AI and human-assisted object descriptions leveraged from existent mobile apps such as VizWiz and Seeing AI.
“A user can select, adjust, and combine different tools based on their preferences, the team’s research paper states. “Nine of our tools modify an existing VR application post hoc via a plugin without developer effort. The rest require simple inputs from developers using a Unity toolkit we created that allows integrating all 14 of our low vision support tools during development.”
The researchers’ evaluation included 11 participants with low vision; the team concludes that their various methods show that SeeingVR “enabled users to better enjoy VR and complete tasks more quickly and accurately.”
The team will be presenting their findings in May at the ACM CHI Conference on Human Factors in Computing Systems conference in Glasgow, Scotland.
The post Microsoft Aims to Improve VR for Users with Vision Problems appeared first on Road to VR.