Oculus Rift In Space Will Be Used For Hand-Eye Coordination Experiment

Oculus Rift In Space Will Be Used For Hand-Eye Coordination Experiment

An Oculus Rift is set to arrive at the International Space Station aboard a SpaceX cargo ship, according to a report by Air & Space Magazine.

Microsoft’s HoloLens previously made a trip to the space station after a failed launch, and custom VR headsets have been used by astronauts in the past. The arrival of a modified Rift, though, brings consumer-grade VR to the station for neuroscience experiments to be conducted by French astronaut Thomas Pesquet.

Like Oculus Research itself, the system is bypassing the Rift’s own infrared tracking system in favor of a different solution. It’ll be running on HP Zbooks which, the article notes, aren’t “ideal” for VR. The modified Rift and associated hardware is called Perspectives, and Pesquet will use the gear in space as part of an experiment called GRASP that will explore “how important gravity is, compared to the other senses, when reaching for an object,” according to the NASA page describing the experiment.

From that page:

The research can help researchers to better understand the workings of the human vestibular system and how it connects to the other sensory organs. In other words, it will achieve a better understanding of the physiology behind eye-hand coordination, as well as shedding light on how to best treat the loss of vestibular function on Earth. This research will also be useful in helping astronauts during spacewalks.

After an aborted rendezvous with the International Space Station on Wednesday, the unmanned SpaceX cargo ship is set to arrive at the station around 3 am Pacific on Thursday, according to a report by Space.com. We look forward to seeing the results of the experiment and hope photos of a Rift in space aren’t too far off — the featured photo of the Perspectives gear at the top of this page is provided by the European Space Agency.

The Air & Space report also notes that Pesquet has been using a Giroptic 360-degree video camera to capture videos on the station as well, and those should be coming online within a couple weeks.

Tagged with: ,

Google Can Recreate Your Face For Better Mixed Reality Footage

Google Can Recreate Your Face For Better Mixed Reality Footage

We are big fans of mixed reality here at UploadVR. It is a great way of showing what people in VR are doing.

Startups like Owlchemy Labs and LIV are attempting to make the capture process easier while pushing for higher quality, but current approaches are all limited by one major roadblock. The most expressive part of the the human body, the face, is mostly blocked during capture. You largely have to imagine the expressions of people as they interact with a virtual world.

Google, however, showed off some impressive research that takes the technology to the next level. Using a collection of techniques, including a modified HTC Vive with SMI eye tracking, Google digitally recreates your face in place of the VR headset that is blocking it.

This work is the result of an “ongoing collaboration” between Research, Daydream Labs and the YouTube teams at Google. According to a blog post diving into the research, here is how it works:

The core idea behind our technique is to use a 3D model of the user’s face as a proxy for the hidden face. This proxy is used to synthesize the face in the MR video, thereby creating an impression of the headset being removed. First, we capture a personalized 3D face model for the user with what we call gaze-dependent dynamic appearance. This initial calibration step requires the user to sit in front of a color+depth camera and a monitor, and then track a marker on the monitor with their eyes. We use this one-time calibration procedure — which typically takes less than a minute — to acquire a 3D face model of the user, and learn a database that maps appearance images (or textures) to different eye-gaze directions and blinks. This gaze database(i.e. the face model with textures indexed by eye-gaze) allows us to dynamically change the appearance of the face during synthesis and generate any desired eye-gaze, thus making the synthesized face look natural and alive.

Here’s a video showing the approach in action:

Team members behind the project, including research scientist Vivek Kwatra and software engineers Christian Frueh and Avneesh Sud, believe the approach holds enormous promise to “enhance communication and social interaction in VR itself with diverse applications like VR video conference meetings, multiplayer VR gaming, and exploration with friends and family.”

Tagged with:

Report: Microsoft Might Not Update HoloLens Hardware Until 2019

Report: Microsoft Might Not Update HoloLens Hardware Until 2019

A report on Thurrott.com indicates Microsoft is skipping an incremental hardware update to its Hololens in favor of a generational upgrade in 2019.

The post by Brad Sams says tips he received from “several sources” indicated Microsoft canceled its second version of Hololens to focus on a third version that will take longer to ship but provide a bigger improvement. The article frames this as Microsoft deciding it has “a large lead in the AR space and isn’t feeling pressure to release a product that is only an incremental update.”

From the post:

By skipping what was version two on their roadmap, the company can accelerate version three which will be closer to a generational leap and help keep Microsoft ahead of the competition. My sources are telling me that this version of Hololens will not arrive until 2019.

This framing of the information could be representative of Microsoft’s outlook. After all, Hololens as a self-contained unit with solid inside-out positional tracking is in a class of its own right now. But we’ve also heard reports suggesting that not many of the $3,000 headsets have been sold. The high price of Hololens could mean only developers contracted to make business, medical or industrial applications are likely to find success in the near-term. Now early adopters are essentially being told not to expect an upgrade soon. Developers who hoped for a rapid follow up from Microsoft that would be less expensive and appeal to a larger segment of buyers are going to have to bootstrap their ideas for longer, or find another way to head to market more quickly.

I reached out to Microsoft for comment and received the following prepared statement:

Mixed reality is the future of computing, and Microsoft HoloLens is the future and present of mixed reality. Our commitment requires no roadmap.

Of course, developer kits for VR headsets compatible with new versions of Microsoft’s Windows are set to be released at GDC. These upcoming headsets will feature a range of specifications, allowing Microsoft partners to release a variety of hardware solutions with different capabilities priced all the way down to $300. These systems could provide an alternate route to market for some developers already familiar with Windows and its mixed reality tools.

It is also worth noting we are waiting to see how Microsoft will work with the HTC Vive and Oculus Rift going forward. As owner of the Windows operating system, Microsoft has enormous power in issuing future updates to the software and determining how VR and AR hardware will interact with it. With Hololens hardware updates potentially pushed to 2019, is it more important that Microsoft start generating revenue right now from all VR and AR headsets which use Windows?

Tagged with: , ,

New Worlds To Visit And Games To Play In VR This Weekend (2/17/17)

New Worlds To Visit And Games To Play In VR This Weekend (2/17/17)

Keeping up with everything new happening in VR is getting more challenging with each passing week. We track new releases on Oculus, Steam/Viveport, and PlayStation, but there is also a constant flow of new 360-degree video projects as well as live events and short demo experiences being broadcast in VR with increasing frequency, not to mention incredible hand-made worlds being shared more rapidly as well as meetups being planned in social VR experiences.

Take artist Danny Bittman’s Painting Within a Painting as an example of this new medium’s growth beyond the boundaries of existing distribution platforms. Bittman is exploring interesting ways of creating both 2D and 3D art using VR, and he says he spent around 100 hours creating this world by hand in Tilt Brush, letting “you explore an immersive painting of a sci-fi mountain range, and then paint paintings of that painting.” You can check out the creation for yourself in Tilt Brush, as he’s offered up the world for visitors to check out with instructions included for setting it up.

360-Degree Projects

The Oscars are next weekend so if you haven’t had the chance to see Pearl yet, be sure to give yourself a few minutes to check out the 360-degree version of the Oscar-nominated animated film.

We also found ourselves laughing pretty hard at the idiot gang from It’s Always Sunny In Philadelphia performing a “badass” jump in 360 on YouTube. You almost feel like a part of the group until they do something awful to you, which is pretty much exactly what the show is about so it is a fitting extension in VR.

Also check out this clever Internet Surfer video.

NBA All-Star Weekend

NextVR is showing highights from most of the events at the NBA’s All-Star Weekend, including the dunk and three-point contests. When the cameras are placed in the right spots we’ve found NextVR’s VR broadcasts to be among the best, and the various events held during All-Star weekend could let you see the action from some fun vantage points. Check it out on Daydream or Gear VR via the NextVR app.

IMAX VR in Los Angeles

If you happen to be in the Los Angeles area, we’d recommend picking up tickets to check out the IMAX VR center that opened recently — they have a good selection of experiences to try out in the sampler for walk-around VR.

New For Gamers

And speaking of those new releases list, we’d highly recommend checking out DiRT Rally on PlayStation VR, Syren on Rift and Vive and Harbinger Trail on Gear VR. Onward players should check out the field guide and if you’re a long-time fan of video games looking for a stroll through a bit of nostalgia, there’s a virtual world made from a map from The Legend of Zelda you can visit. The Art of the Fight also just exited Early Access as a full release, making it a great option for fans of competitive shooters.

Social VR

AltSpace has its full calendar of events lined up to watch with others inside VR, including a SpaceX Falcon rocket launch party in VR early Saturday morning. The company is adding Daydream support and support for non-VR Android phones as well — so there are more ways than ever to jump in and meet with people.

Tagged with: , , ,

Inside The First IMAX VR Arcade

Inside The First IMAX VR Arcade

If VR arcades in the United States are going to find success, it is looking like IMAX VR is going to give it the best try.

We went inside the big-screen company’s flagship VR center in Los Angeles this week, and got a good look at their plans for arcade roll-outs this year including centers spread across the globe. In short, I saw a series of well-designed spaces equipped with some of the best virtual reality hardware and software available.

Visiting virtual worlds in the center costs about $1 per minute right now (“Each experience is approximately 5 to 15 minutes long” and “featured experiences range between approximately $7-$10”). That could change as more content is introduced to the location over time.

The first arcade in Los Angeles is not connected with a multiplex, but future IMAX VR centers in Los Angeles, New York and overseas will be connected to movie theater locations. The idea is you’ll head out to the movies and before or after the big show you’ll visit the VR arcade for a few minutes.

Photo provided by IMAX.

HTC is betting big on VR arcades in China. An arcade version of its Viveport VR content store is designed to make it easier for groups to set up a location equipped with a variety of worlds to visit. There’s an assumption that out-of-home VR experiences are going to be very popular in Asia where many people have less space and money to install their own equipment. It remains to be seen whether similar arcades in the United States and elsewhere will find success, but IMAX is looking to make a strong go of it.

“This will be a breakthrough for the U.S. I think,” said HTC’s Viveport President, Rikard Steiber.

A lot of thought clearly went into the design of this first flagship IMAX location, and it is equipped with hardware which should exceed or at least rival the setups early adopters have at home.

For VR enthusiasts in the Los Angeles area looking to introduce friends, family or coworkers to VR — this center should be strong competition for anything you’ve set up personally.

IMAX is making clear it is providing a premium experience partnering with the HTC Vive for a series of room-scale pods complemented by Subpac haptic vests and a D-Box motion chair. Powerful PCs are stored out of sight overhead with wires running down on “balancers” meant to manage the wires and keep them out of the way while moving freely around the room.

Here’s the roll-out plan for the next pilot centers:

IMAX has signed agreements to open an additional five IMAX VR centres in the coming months, including a pilot Centre in the UK with ODEON & UCI Cinema. The Company is also currently in conversations to open additional pilot centres in Japan, the Middle East and Western Europe. These pilot locations will test several factors including the overall customer experience, pricing models and the types of content featured. If successful, the intent is to roll out the concept globally to select multiplexes as well as to commercial locations such as shopping centers and tourist destinations.

Photo provided by IMAX.

While most of the pods were equipped with HTC Vives, a pair of StarVR headsets were also at the IMAX location, powered by some of the beefiest graphics cards available hidden away in computers overhead.

Photo provided by IMAX.

StarVR is a headset from Starbreeze that features an ultra-panoramic view compared to the first headsets you can buy for your own home. It is still very much a developer kit, though, and only runs at 60 frames per second (FPS) compared to the Vive’s 90 FPS.

StarVR should probably come with an “experimental” sign so visitors to the center know it is not at the same level of polish as the Vive. I tried a rooftop assault version of the John Wick Chronicles VR game in the headset with a realistic gun accessory tracked using PhaseSpace.

Inside The First IMAX VR Arcade

Inside The First IMAX VR Arcade

Update: This article was originally published on February 2, 2017 and has been republished now to coincide with the news that the (surprisingly good) Justice League VR experience will be available for people to enjoy later this year at its locations.

Original: We don’t know yet if VR arcades in the United States are going to find success, but it is looking like IMAX VR is going to give it a great try.

We went inside the big-screen company’s flagship VR center in Los Angeles this week, and got a good look at their plans for arcade roll-outs this year including centers spread across the globe. In short, I saw a series of well-designed spaces equipped with some of the best virtual reality hardware and software available.

Visiting virtual worlds in the center costs about $1 per minute right now (“Each experience is approximately 5 to 15 minutes long” and “featured experiences range between approximately $7-$10”). That could change as more content is introduced to the location over time.

The first arcade in Los Angeles is not connected with a multiplex, but future IMAX VR centers in Los Angeles, New York and overseas will be connected to movie theater locations. The idea is you’ll head out to the movies and before or after the big show you’ll visit the VR arcade for a few minutes.

Photo provided by IMAX.

HTC is betting big on VR arcades in China. An arcade version of its Viveport VR content store is designed to make it easier for groups to set up a location equipped with a variety of worlds to visit. There’s an assumption that out-of-home VR experiences are going to be very popular in Asia where many people have less space and money to install their own equipment. It remains to be seen whether similar arcades in the United States and elsewhere will find success, but IMAX is looking to make a strong go of it.

“This will be a breakthrough for the U.S. I think,” said HTC’s Viveport President, Rikard Steiber.

A lot of thought clearly went into the design of this first flagship IMAX location, and it is equipped with hardware which should exceed or at least rival the setups early adopters have at home.

For VR enthusiasts in the Los Angeles area looking to introduce friends, family or coworkers to VR — this center should be strong competition for anything you’ve set up personally.

IMAX is making clear it is providing a premium experience partnering with the HTC Vive for a series of room-scale pods complemented by Subpac haptic vests and a D-Box motion chair. Powerful PCs are stored out of sight overhead with wires running down on “balancers” meant to manage the wires and keep them out of the way while moving freely around the room.

Here’s the roll-out plan for the next pilot centers:

IMAX has signed agreements to open an additional five IMAX VR centres in the coming months, including a pilot Centre in the UK with ODEON & UCI Cinema. The Company is also currently in conversations to open additional pilot centres in Japan, the Middle East and Western Europe. These pilot locations will test several factors including the overall customer experience, pricing models and the types of content featured. If successful, the intent is to roll out the concept globally to select multiplexes as well as to commercial locations such as shopping centers and tourist destinations.

Photo provided by IMAX.

While most of the pods were equipped with HTC Vives, a pair of StarVR headsets were also at the IMAX location, powered by some of the beefiest graphics cards available hidden away in computers overhead.

Photo provided by IMAX.

StarVR is a headset from Starbreeze that features an ultra-panoramic view compared to the first headsets you can buy for your own home. It is still very much a developer kit, though, and only runs at 60 frames per second (FPS) compared to the Vive’s 90 FPS.

StarVR should probably come with an “experimental” sign so visitors to the center know it is not at the same level of polish as the Vive. I tried a rooftop assault version of the John Wick Chronicles VR game in the headset with a realistic gun accessory tracked using PhaseSpace.

The Void Upgrades Display, Aims For 20 Installations This Year

The Void Upgrades Display, Aims For 20 Installations This Year

If you wanted to see one of the most complete VR experiences available last year you would have paid for a ticket to Madame Tussauds in New York for around $50 and included the extra Ghostbusters: Dimension VR experience. You would enjoy catching ghosts with three other players moving freely around the virtual world alongside you.

What makes the experience so immersive is that it is a completely untethered multiplayer walk-around experience complemented by real-world effects, like heat or wind. So you aren’t tethered to a computer, you aren’t alone and when you walk up to a flame you can actually feel it. The experience is a product of The Void, a Utah-based startup.

If you try out the same experience in the next few months you’re going to see a much improved experience, according to co-founder James Jensen. That is because the Ghostbusters experience last year was, according to hands-on reports, initially put together using a modified Rift development kit 2 (DK2) combined with expensive Optitrack tracking. This allowed the system to be deployed in a large area. While the approach allowed The Void to deploy the Ghostbusters: Dimension experience relatively early, it came online as Oculus and HTC started shipping improved consumer hardware. As consumers started receiving headsets that displayed 90 frames per second using the highest resolution displays possible, The Void was using a headset with an older display that only showed 75 frames per second.

This meant that despite wearing a backpack that feels just like a Ghostbusters proton pack, and catching ghosts with three other players in a virtual world enhanced by impressive environmental effects, you could still get an experience at home that was superior in some respects. But 2017 is shaping up to be a big year for The Void as the startup looks to provide the ultimate “hyper-reality” experience. The startup’s newest “Rapture” unit features more pixels in front of your eyes and more frames each second smoothing out the experience. They have more than 100 people working at The Void now, though Jensen declined to discuss financial backing.

“We spent a good part of 2016 getting all of our equipment on manufacturing lines so that we could just pump this stuff out and install in locations worldwide,” said Jensen. “Each of our ‘stages’ can have an experience on it, so locations could have two or three stages at them. We’re hoping to install 20 stages this year.”

Jensen said they wanted to get their first location up and running quickly, which is why they launched using the hardware they did. They also needed to build a VR helmet for safety reasons that would let them get people in and out of a headset very quickly, according to Jensen. 

“Our HMD is actually a helmet because you’re walking around untethered in a space with multiple people…if you take a hit on the front of it then it acts like a bicycle helmet. It bounces off your forehead, there’s padding in here,” said Jensen. “If you look at those [business] models where people are just taking an at-home system and deploying it in a mall, if you really break it down on the mechanics of how long people are in the experience, how long it takes them to get out of the experience… then those businesses kind of fall apart. It’s all in the details of getting that equipment on, getting it off and providing an amazing experience for the end user.”

The unit notably also includes a detachable facial interface so you aren’t sniffing a previous wearer’s sweat when you put it on. Jensen also said their backpack computer should be around 1.5 pounds lighter and the headset should be half a pound lighter. Lights are embedded along the surface of the gun accessory and headset so they can be spotted by the Optitrack cameras for tracking. The Void showed the newest gear for the first time at an event in Los Angeles earlier this week where we tried on the vest and took the photos embedded throughout this article. The photo at right shows the facial interface being inserted, and you can see stickers over the lenses which look exactly like those included with the consumer Rift. I asked The Void if it is a consumer Rift buried under their modifications but they declined to comment.

With this upgraded hardware rolling out and more locations planned, it is looking more and more like The Void will be bringing the ultimate VR experience to a number of cities around the world this year. We can’t wait to check out what they have in store.

Tagged with: , , ,

Mark Zuckerberg Teases Glove-Based Advanced Hand Tracking Research From Oculus

Mark Zuckerberg Teases Glove-Based Advanced Hand Tracking Research From Oculus

Mark Zuckerberg just posted to his personal account on Facebook a first-ever peek inside Oculus Research, a lab based in Redmond, Washington dedicated to VR and AR advancements.

The lab is headed up by Michael Abrash, who left Valve Software in 2014 to join Facebook/Oculus and lead this team exploring what’s needed to improve on the state of mixed reality technology.

In fact, the featured photo above shows gloves allowing Zuckerberg to “draw, type on a virtual keyboard, and even shoot webs like Spider Man.” Notably, the researcher’s hardware appears to use fairly expensive Optitrack cameras for the system rather than the buggy Oculus Sensor tracking system the company used for the consumer Rift.

Hand tracking is an incredibly difficult problem to solve in VR given the many quick and precise movements your fingers do that might be hard to spot using a camera, or a pain to calibrate using gloves. Developing software and an inexpensive hardware system that can track these movements accurately on anyone could push VR and AR forward.

Zuckerberg’s caption on this photo reads “When you manufacture really small pieces, you have to keep every surface clean to avoid defects. This clean room filters out particles 1000x smaller than a speck of dust,” giving a hint at the core manufacturing work they are doing to work on new technologies. 

“We’ve built labs that let us quickly make new kinds of lenses and devices to push the boundaries of virtual and augmented reality. The includes a Diamond Turning Lab that cuts metal with a gemstone quality diamond, and this 5-axis CNC milling machine,” his caption on the below photo reads.

Abrash has spoken publicly in the past to outline what the future holds for VR, and his work makes him one of the world’s leading experts on the technical constraints facing VR. Though this look inside his lab at Oculus Research is little more than a tease, it does offer us a new perspective on just how serious Facebook is in its ambitions for better VR and AR.

Tagged with: , ,

Developers Showcase Incredible Full Body Tracking With Vive

Developers Showcase Incredible Full Body Tracking With Vive

We have yet to see a solution for full body tracking that is affordable, easy to use, and can work in a wide range of VR experiences. That’s not keeping developers from forging ahead anyway though.

The prize for developers creating virtual worlds and implementing full body tracking is a more immersive experience overall. You’d have an experience in which your shadow on the ground, reflection in the mirror and arms and legs when you look down all accurately represent the movements of their real-world counterparts. Other companies specializing in motion capture, like IKinema, might also be able to put together products and services that are more accessible to a larger group of creators.

One method of tracking full body movement is by wearing a suit that takes a lot of time to put on and calibrate. A pair of videos released in the last few days, however, show how Valve Software’s promising tracking technology is usable for the same purpose.

We covered the first test by Steve Bowler at CloudGate Studio, developer of Island 359 [Early Access Review: 8/10], and now Bowler released a second experiment showing just how good the movement capture works with the HTC Vive when you have six points of data to collect. The first test didn’t track the hips, but now Bowler is putting a Vive controller in a tool belt at his waist in addition to the ones on his feet and his hands. When combined with the head’s position via the headset itself, these six points provide an incredibly realistic real-time body capture system.

Here’s Bowler:

We’re pushing to get an incredibly robust fullbody awareness commercial system out to our Island 359 players. It should scale with how many tracked points we see when they boot up the game, so that if players only have the HMD and Controllers, they should still be able to look down and see a torso, arms, and hands. If they own two trackers, they’ll have legs, and if they own three trackers, they’ll see even more robust body tracking. This way people don’t have to invest in trackers at any set level if they don’t want to, but places with a bit more capital like VR Arcades could offer the fully tracked experience easily, without a ton of complication. We’re also in the process of devising a “templating” system to make sure that the user doesn’t have to do any technical work to make the full body magic happen. Once we have the trackers, they should just be able to attach them to their shoes/belt, do a quick template check, and be playing with full body awareness.

Bowler is far from the only one attempting this. IKinema is also developing a solution for “low-cost, full body mocap from HTC Vive hardware.” The latest video from IKinema, embedded at the top of this post, uses six tracking points as well.

There is still a lot of work to be done before we’ll start seeing these types of solutions in a lot of VR games. HTC has just started distributing Vive Trackers to developers, which would provide a slightly more ergonomic solution for this sort of tracking than buying extra controllers. We’re also still waiting for revelations from hundreds of partners which are working with Valve’s SteamVR Tracking technology on what would likely include even more ergonomic solutions built for this specific purpose.

Tagged with: , , ,

Vive Tracker Includes USB Add-on To Connect The Additional Accessory

Vive Tracker Includes USB Add-on To Connect The Additional Accessory

HTC recently released a document (PDF) detailing how its upcoming Tracker is supposed to function. The small puck-sized unit is meant to be attached to a range of objects and accessories, bringing physical guns, baseball bats, phones, cameras and much more into virtual reality with you.

The Tracker holds enormous promise for extending 1:1 physical simulation to a wide range of activities. An example we saw at CES was a firehose, with the Vive Tracker attached to the end for a realistic firefighting experience.

The system could dramatically lower the cost for high-end simulation systems that look, sound and feel very much like the real thing. The device could potentially lower the cost of training so a wide range of professionals could practice their jobs using VR. The Tracker could also dramatically expand the types of games seen at VR arcades. Imagine, for instance, batting cages, but instead of mechanical arms throwing the balls, batters are swinging at virtual pitches.

Owners of the HTC Vive who have the headset in their homes already will also benefit from extra accessories as well. The Trackers should be compatible with the base stations and headset already in the homes of HTC Vive buyers. We’ve also confirmed with HTC that a Tracker can connect wirelessly through the headset in place of one of the existing Vive controllers. In addition, though, the Tracker will ship with a USB add-on that plugs into your PC to wirelessly connect the unit. This should make adding the Tracker and its related accessories relatively easy for most setups since you won’t have to disconnect your existing controllers to add the new piece. That said, if you did want to save the extra USB port on your PC and only use the new accessory, you could do so through the headset’s connection and leave out the traditional controller, according to HTC.

We can’t wait to see how these devices end up changing the VR ecosystem when it ships in the coming months. It is also notable, however, that Oculus is attaching its Oculus Touch controllers to the Rock Band guitar so that object can be tracked in VR as well. Could that be the first of many tracked Rift accessories?

Tagged with: , , ,