Eye tracking experts SensoMotoric Instruments (SMI) have collaborated with Valve to bring their technology to OpenVR. Eye tracking-enabled HTC Vive units are being shown at GDC 2017, as part of an R&D effort to integrate eye-tracking support into Valve’s OpenVR API.
At this week’s Game Developers Conference, along with LG’s ‘next generation’ VR headset prototype, Valve are also showing some upgraded HTC Vive headsets at their booth, fitted with eye tracking technology by SensoMotoric Instruments (SMI). The German computer vision provider is a world leader in eye tracking technology, and has produced kits for the Oculus DK2, Gear VR and HTC Vive HMDs.
A peek inside one of SMI’s Modded Gear VR units, from E3 2016.
Eye tracking is a major technology hurdle that must be conquered in order to take VR to the next level of visual quality. Performance requirements for VR rendering exponentially increase with display resolution and field of view, so eye-tracked foveated rendering is essential – a technique that removes unnecessary fidelity from parts of the image that miss the detailed part of the retina (the fovea). While SMI demonstrated 250Hz eye tracking and foveated rendering over a year ago, Michael Abrash described it as “not a solved problem at all” at his keynote at Oculus Connect in October.
SMI are showcasing their latest work with ARM at their own booth, to highlight the benefits of foveated rendering on mobile VR devices, and Valve are giving demos of OpenVR eye-tracking features using modified Vive headsets. Integrating eye-tracking into the OpenVR API is an indication that Valve want to move forward swiftly with the technology, although it remains unclear whether the next round of consumer headsets will include eye tracking. Speaking to Tom’s Hardware before the event, Valve developer Yasser Malaika said “Eye tracking opens up several interesting possibilities to both VR developers and customers. Our collaboration with SMI on R&D, as well as on SMI’s efforts to make eye tracking enabled Vive units available to the larger VR community, have been critical to our growing understanding of how HMDs with integrated eye tracking will positively impact the future of VR.”
Mage’s Tale is a VR Action RPG set in the world of classic fantasy RPG The Bard’s Tale. It is being shown at GDC 2017 for the first time on Oculus Rift and Touch and here’s six minutes of gameplay footage to whet your appetite for the title.
As one of six new titles announced at GDC for Oculus platforms, Mage’s Tale is built from the ground up for Oculus Touch. The Action RPG is developed by inXile Entertainment, set in the world of The Bard’s Tale, a classic RPG series. Delving through dungeons, solving puzzles and mastering the arcane arts sounds like a good time; the game promises to deliver 10 hours of gameplay through hand-crafted dungeons littered with secrets. A crafting system that lets you design hundreds of variations of four base spells, all of which are cast with a physical hand motion. While a release date hasn’t been set, inXile plan to launch on Touch first, but will bring the game to other platforms in the future. The game is shaping up well too, as you can see from this 6 minutes of gameplay footage.
Road to VR‘s Ben Lang also spoke to inXile’s David Rogers this week at GDC to find out more about the design ethos behind the game.
AR/VR Austin returns for its third successive year, presenting a mini-conference named ‘Alternative Realities’. Taking place as an unofficial event during the SXSW 2017 festival on March 11th, this year’s event will question the impact of social and political change on the AR/VR industry.
Following last year’s celebration of technology innovation, the AR/VR Austin event returns on March 11th with a totally different tone; discussing the implications of the changing social and political climate on these new technologies. The event will include cutting-edge demonstrations, but the focal point is a ‘serious gathering of the best minds in AR/VR/social thought’ taking a deep dive into ‘resistance, privacy, analytics, deception and surveillance’.
Founder Simon Solotko believes that rapid political change will impact AR/VR, and has invited several industry experts and special guests to offer their insights across three sessions during the evening: ‘Resistance & Organization Intersect Surveillance’, ‘Synthetic Worlds Intersect Real Minds’, and ‘Constructing Groundbreaking Experiences in the New Republic’.
The Alternative Realities mini-conference is an unofficial event taking place during this year’s South By Southwest festival in Austin, Texas, which itself includes 45 AR and VR panels, sessions, and workshops scheduled between the 10th and 16th March, including some with similar agendas.
Tickets for AR/VR Austin 2017 are available via Eventbrite.
Announced at AMD’s GDC 2017 press conference, Radeon GPUs will support Asynchronous Reprojection on SteamVR in the next update. The technology helps to maintain smooth head motion when framerate drops below the headset’s refresh rate.
Roy Taylor, Corporate Vice President – Alliances at AMD took to the stage with some passionate announcements about AMD and its VR support, promising ‘120+ fps, billions of entities and 16K graphics’ in the future. Today, their focus is on delivering the most seamless VR experience possible on existing hardware, and a much-needed feature is an effective way of dealing with performance drops. Taylor invited Dan O’Brien, General Manager of HTC Vive, onto the stage to announce Asynchronous Reprojection for Steam VR on Radeon GPUs.
While AMD’s LiquidVR technology has supported Asynchronous Spacewarp on the Oculus Rift since December, Asynchronous Reprojection for SteamVR has been limited to Nvidia GPUs since its introduction in October 2016. The technique is very effective at mitigating the impact of small performance drops below 90fps, maintaining totally smooth head orientation tracking, which can result in a significant improvement to comfort and immersion. AMD is aiming to include the update in the next release of Radeon software.
Debuted at GDC 2017 and announced via their official blog, Oculus offer a first look at some of the titles coming to Rift and Gear VR in 2017. This includes 6 new game reveals across multiple genres, with 4 optimised for Touch and 2 “must haves” for Gear VR.
Having recently teased ‘months of high-profile VR content’ starting with Rock Band VR, Oculus have begun to deliver on that promise at this week’s Game Developers Conference, revealing a list of 2017 titles, including 4 new games for Touch and 2 for Gear VR.
Oculus Rift Games
Blade & Soul
The way of battle is simple: summon the units by grabbing them with your hands. Choose your units wisely, have a strategy of your own and fight until the end. Strong units are crucial to winning the battle. Strengthen your units by training and upgrading them in the lab.
Designed by: NCSoft
Genre: TCG, RTS
Platform: Touch
Release Date: TBA
Price: TBA
Comfort: TBA
Brass Tactics
Brass Tactics takes real-time-strategy to the next level by placing you in the middle of the action on a fantastic clockwork battlefield. Experience the thrill of directly moving and interacting with your clockwork creations: grab structures and place them on the battlefield, and direct your units with the sweep of your hand. Just when you think you’ve mastered the game, raise the stakes by teaming up with other players in co-op mode or go head-to-head in PvP mode.
Designed by: Hidden Path Entertainment
Genre: Classic Real-Time Strategy (RTS)
Platform: Touch
Release Date: Fall 2017
Price: TBA
Comfort: Comfortable
From Other Suns
Your own ship. A crew. Steady work. Things were going well until the Collapse. Now you and half of humanity are trapped on the far side of the wormhole with ruthless pirates, scheming corporations, and worse—new threats from outside known space.
There’s danger at every jump on this side of the wormhole. You and up to two of your friends will tour the sector, upgrading your ship, stockpiling weapons, and fighting for your lives. And when you all die, you’ll discover new challenges in your next playthrough.
Fight and try to save humanity, or just joyride through the galaxy until its extinction. Your call.
Developed By: Gunfire Games
Genre: Action Adventure
Platforms: Oculus Rift and Gear VR
Release Date: Fall 2017
Price: TBA
Comfort Rating: Comfortable
Mage’s Tale
Welcome, apprentice. Don your wizard’s robe and become a mighty conjuror. The corrupted wizard, Gaufroi, has kidnapped your master, Mage Alguin, and it’s up to you to save him. To win the day you must conquer eleven dungeons, from the stinking sewers of Skara Brae to the living tombs of the evil Charn. Mind bending puzzles, terrifying traps, and deadly monsters stand in your way, all perfectly capable of sending you to an early grave.
But worry not. You wield raw elemental power in the palm of your hand, allowing you to sling gouts of flame, javelins of ice, arcs of lightning, and swirling tempests which can finish off any fiend that stands in your way– from the snarkiest goblin to the burliest giant. And as you delve deeper into the depths you’ll find forgotten secrets, ancient lore, and powerful spell reagents with which you can craft increasingly exotic spells to defeat even greater foes. Yes, you may be an apprentice now, but to save your master, this must become your Mage’s Tale.
Designed by: inXile Entertainment, Inc.
Genre: VR Adventure, Puzzle Solving, Action RPG
Platforms: Rift
Release Date: TBA
Price: TBA
Comfort: TBA
Gear VR
Augmented Empire
Augmented Empire is a story-driven tactical RPG set on the island of New Savannah, an isolated neo-noir city divided into three tiers by a social grade system. While the citizens deemed of high societal value live in luxury at the summit, outliers and criminals are forced to live in squalor at the island’s depths.
From the armchair of your secluded hideout, command a team of 6 bioelectronically-enhanced misfits in ‘augmented reality;’ a diorama-scale version of the city rendered before your eyes. Explore the city, develop your skills and battle law-makers and law-breakers alike to mastermind the Revolution.
Designed by: Coatsink Software
Genre: Strategy, Action, Adventure
Platform: Gear VR
Release Date: TBA
Price: TBA
Comfort: Comfortable
Term1nal
You are Flynn Lightman, a highly skilled Avatar Pilot who can control androids from the safety of his apartment using advanced VR hardware. You are contacted by a client to infiltrate STRIDE Industries, a company that specializes in data security and advanced robotics. Your journey takes you into the depths of a heavily defended, high-tech facility as you discover its darker intent.
Dual-character 3rd-person stealth gameplay and immersive 1st person puzzles designed exclusively for Gear VR.
Designed by: Force Field VR
Genre: Action Puzzler
Platform: Gear VR
Release Date: Q2 2017
Price: TBA
Comfort: TBA
The new titles add to a list of compelling upcoming content for the Oculus platform, promising to deliver ‘depth and polish’, covering genres such as RTS, RPG, stealth, puzzle and shooter. Announced via the Oculus Blog, the ‘Year of VR Gaming’ is showcased in this new montage, with more news coming tomorrow.
Today at Mobile World Congress in Barcelona, Samsung announced the brand-new Gear VR with Controller, with 70 titles coming soon. The controller is said to add a deeper level of immersion, using motion sensors, and an intuitive touch pad and trigger input.
Announced today at the Mobile World Congress in Barcelona, and revealed on the Oculus Blog, Samsung’s new Gear VR has received a major new feature in the form of a motion controller.
The Gear VR Controller is a compact wand that uses motion sensors, a circular touchpad and trigger to interact with Gear VR content. With integrated home, volume and back buttons, the controller will be compatible with all existing Gear VR apps, but ‘more than 70’ titles designed for the controller are in development.
The single-handed controller has clear similarities with the controller that comes with Daydream VR, Google’s mobile VR platform. This input parity will certainly help developers create mobile VR apps that support motion controllers across both platforms. What’s more, every Gear VR shipped will include the controller as standard, a reassuring move for potential developers looking to support the new devices.
The new Gear VR headset still supports the same smartphones as the existing model, suggesting that the controller could be available separately for owners of the current hardware.
This story is breaking – more details will be added as we receive them.
Six Flags Entertainment Corporation—which runs theme park properties across the americas—and Samsung Electronics America, Inc. recently launched a renewed collaboration on new rides at two Six Flags theme parks which place users on real roller coasters while using a VR headset to add virtual visuals. ‘Galactic Attack’, which improves on the existing ‘New Revolution’ VR rides, introduces a ‘mixed reality’ element and ‘complex gameplay’.
Six Flags, which calls itself “the world’s largest regional theme park company,” has been at the forefront of the ‘VR Coaster revolution’, having launched virtual reality-based rides at nine locations in North America last year. Partnering with Samsung Electronics America, Six Flags offers the use of Gear VR headsets in combination with some of their existing roller coasters, which transforms the visual experience and, in the case of the ‘New Revolution’ presentation, offers a level of interactivity—the VR environment is a fighter jet cockpit, and riders can look around and fire weapons in a combat scenario as the coaster is in motion. The video heading this article gives an idea of what the experience is meant to be.
The collaboration has been refreshed for 2017, with a new experience called ‘The New Revolution Galactic Attack’ opening to the public this week at Six Flags Magic Mountain, near Los Angeles. The ride has been running since the 20th at Six Flags Discovery Kingdom near San Francisco, and was well-received by thrill-seekers at the preview event.
The new VR ride, which uses the ‘New Revolution’ coaster at Magic Mountain and the ‘Kong’ coaster at Discovery Kingdom, promises a “one-of-a-kind mixed reality experience” with “complex gameplay”, where your decisions during the battle to save the planet from alien drones will affect your experience and score at the end.
The ‘mixed reality’ element is enabled via the Gear VR’s passthrough camera, allowing riders to see the real world, including the person seated next to them. The virtual content begins to be overlaid in the form of a heads-up display, showing weapons, time codes, fuel cells, and a countdown clock. As the ride reaches the top of its first climb, a “massive, swirling wormhole” appears, and once the ride drops, the view transitions to a full VR environment. The ‘complex gameplay’ relates to the three drone bays that a rider can find themselves in during the space battle, providing a “completely different gaming experience and three different endings”.
“Six Flags is proud to be partnering with Samsung to develop the newest, most innovative thrill ride experience in the theme park industry,” said Brett Petit, Senior Vice President of Marketing and Sales. “This mixed reality technology is truly groundbreaking and like nothing our guests have ever experienced. Six Flags and Samsung changed the game last year with VR on twelve roller coasters and now we are breaking new ground yet again.”
“Our strategic partnership with Six Flags enables millions of consumers to experience virtual reality for the first time by bringing Gear VR to real roller coasters at theme parks across the country. We are always aiming to create immersive, never been done before experiences with our Samsung VR ecosystem as the focal point, so these new experiences at Six Flags are completely complementary to that key objective. We are thrilled to continue to work with Six Flags and bring VR to the mainstream,” said Marc Mathieu, Samsung Electronics America’s Chief Marketing Officer.
Zappar, a London-based developer in the Augmented Reality space, is announcing the closing of its Series A funding round. The $3.75m investment will accelerate the development of several new products along with increased support for ZapWorks, their flagship content creation tool.
Having worked with over 200 partners from the retail, television, film and packaged goods industries including Warner Bros, Coca-Cola and Manchester City Football Club, Zappar is a leading developer of Augmented Reality experiences and tools. Their free ‘Zappar’ AR smartphone app reacts when the camera is pointed at ‘zapcodes’ found on printed materials, allowing AR content to appear over any surface. Some of this content is produced in-house, but their most exciting venture is ZapWorks, a new creation suite that allows brands, marketing agencies and other external creators to quickly produce their own AR, VR and Mixed Reality content. They have also successfully crowdfunded a Google Cardboard-inspired $30 Mixed Reality solution called ZapBox.
Today, Zappar is announcing the closing of its $3.75m Series A funding round, led by London-based investors Hargreave Hale, along with brand technology business You & Mr Jones and China’s largest independent mobile game publishing platform iDreamSky.
“We’re at a unique moment in time as AR, VR and MR content and technology moves into the mainstream,” said Oliver Bedford, Fund Manager at Hargreave Hale. “There are a number of players entering this space. Zappar are a clear front runner, grounded in an established, effective, robust, and highly scalable content creation technology that caters to all markets and developer needs.”
The funding will accelerate the development of several new products, and will see the expansion of its SaaS Sales team, User Success specialists, R&D and Marketing divisions, extending Zappar’s international reach. The flagship product ZapWorks, which is already being used by leading brands such as Samsung and Vodafone, will receive further development, global marketing and support.
“Zappar is a best-in-class business that empowers businesses and creatives to build best-in-class augmented reality experiences”, said George Prest, Partner at You & Mr Jones, an investor in Niantic, creators of Pokémon Go. “Since launching ZapWorks earlier this year, the platform’s ease of use, breadth of tools and affordable cost structure have resonated very strongly with the market, resulting in clear belief from us in their business potential. Zappar has what it takes to truly democratise AR creation, and we’re looking forward to the company’s successes.”
“This funding round is another great milestone for the business”, said Caspar Thykier, CEO of Zappar. “Zappar’s mission is to democratise augmented reality. AR represents an exciting future for consumer engagement connecting devices to the world around us. With this round our established institutional and trade investors lend further credence to this future as we continue towards AR delivering a digital discovery channel through mobile devices and head-mounted displays.”
Prolific YouTuber Tom Scott recently visited the Mixed Reality Lab at the YouTube Space studio in New York in order to showcase mixed reality capture. The lab is using a technique involving a green screen and a tracked camera to combine real world footage with a virtual reality environment, and helpfully lays out the basic concepts of the technique.
One of the persistent hurdles when explaining or promoting augmented and virtual reality technology is the need to convey the experience when viewed on a conventional display. One effective way to achieve this is with mixed reality, a combination of capture techniques that merge footage of the real world with the virtual world.
Northway Games, developers of Fantastic Contraption, were among the first to embrace the technique, and their variousguides are a good place to start for those already versed in digital capture and streaming. Valve’s introduction video to SteamVR at the launch of the HTC Vive last year remains a superb illustration of the impact of presenting VR in this way.
YouTube Space New York, one of several official learning and creative YouTube studios around the world, has a Mixed Reality lab using an HTC Vive setup, which received mainstream exposure on late night talk show Conan in November. Recently Tom Scott, a popular YouTuber, visited the lab, providing a somewhat more sensible presentation about the equipment as part of his ‘Amazing Places’ series. The video is seen heading this article.
A Vive Tracker attached to a camera lets the computer sync the virtual and real footage together spatially.
Tom Small, manager of New Technology Programs at YouTube Spaces and Travis Butler, manager of Technology at YouTube Spaces in The Americas were on hand to explain some of the details. Small describes how a third Vive controller attached to the handheld camera filming the green screen allows the software to create a virtual camera that tracks in the same way, and then both types of capture can be combined to create the mixed reality footage. The upcoming Vive Tracker is ideal for this application, rather than having to use a third controller.
Butler explains the need for a powerful PC to achieve best results; in this case they’re using a high-end Intel CPU, 32GB RAM, and an NVIDIA GTX 1080 graphics card in order to render at 4K. The system is capable of generating the composite image in realtime, and a higher quality output can be created with post processing. The high system requirements are also due to the additional views being rendered alongside the normal VR output; the virtual camera view and a foreground and background layer used to provide a basic depth effect.
This ‘video sandwich’ method is achieved by rendering a view of the ‘background’ and ‘foreground’ separately, determined by the position of the headset relative to the virtual camera. Any geometry that appears between the headset and the camera is considered foreground, and anything ‘behind’ the headset is background.
It’s a demanding and inaccurate solution, and a problem that Owlchemy Labs, developers of Job Simulator, have addressed using a more advanced approach. Their first blog entry explains the difference between the common mixed reality solutions of green screen overlay and the foreground-background sandwich method, and why their depth-sensing solution is so much better; a custom shader and plugin uses depth information from a stereo camera to render the user in-engine in realtime, meaning no external compositing is required, and accurate depth is achieved. In addition, their second blog entry shows how this depth information means the user can receive dynamic lighting from the game engine in realtime, as well as solving transparency problems and removing green screen boundaries.
During a recent media event, Valve revealed that only 30 VR apps have made over $250,000 so far on Steam. Now focusing his company heavily on VR development, Valve president Gabe Newell remains bullish on the future of VR, but isn’t shying away from sharing frank assessments of the still young industry.
Sorting content ‘by VR’ through the Steam store, the number of titles that support the technology comfortably exceeds 1,000. While a few early indie VR titles have seen a few million in revenue, according to Valve only 30 of Steam’s VR apps have made over $250,000.
This could be a fairly discouraging figure for aspiring VR developers, as Steam is surely the most popular source for PC VR content, considering the apparent sales advantage of the HTC Vive over the competition, combined with the fact that SteamVR works with other headsets like the Oculus Rift and OSVR. Yet, Valve president and co-founder Gabe Newell doesn’t sound disappointed.
“We’re optimistic. We think VR is going great,” Newell told Polygon among others at a recent media briefing. “It’s going in a way that’s consistent with our expectations”. However, the forthright Newell-esque statement that followed—”We’re also pretty comfortable with the idea that it will turn out to be a complete failure”—set the tone for some very interesting and candid comments about where he believes VR stands right now.
Valve’s Gabe Newell | Photo courtesy Kotaku
Offering some explanation for his wariness, Newell describes some of the drawbacks of the VR space, in particular that the high-end will continue to be where the most compelling stuff happens. High cost of entry is already a problem, but due to the tech advancements required in future VR hardware, it will continue to be, Newell believes.
“We’re at the beginning of this. Vive is the most expensive device on the market. It’s barely capable of doing a marginally adequate job of delivering a VR experience. We have to figure out all sorts of other problems before even the hardware question gets answered, much less what’s going to be the compelling content.”
In other words, it’s far too early to introduce major cost reductions as a solution; the hardware simply has to remain expensive in order advance to the point where VR becomes good enough to be an accepted, mainstream technology.
“If you took the existing VR systems and made them 80 percent cheaper, that’s still not a huge market. There’s still not a really incredibly compelling reason for people to spend 20 hours a day in VR”, Newell said. “Once you’ve got something that will really cause millions of people to be excited about VR, then you start worrying about cost reductions. […] Some people have got attention by going out and saying there’ll be millions of [VR unit sales] and we’re like, wow, I don’t think so. I can’t point to a single piece of content that would cause millions of people to justify changing their home computing.”
Valve’s breakthrough SteamVR Tracking technology pushed the hardware beyond what many were expecting from a first generation VR package, and Newell’s statements clearly indicate that their hardware research continues unabated.
“We’re going to go from this weird position where VR right now is kind of low res, to being in a place where VR is higher res than just about anything else, with much higher refresh rates than you’re going to see on either desktops or phones. You’ll see the VR industry leapfrogging any other display technology. You’ll start to see that happening in 2018 and 2019 when you’ll be talking about incredibly high resolutions.”
But Valve remains a software company at heart, and their three full VR games in development promise to address the need for compelling content. Newell doesn’t provide any specific details about the projects, but quashes the idea of returning to gameplay that the studio became famous for.
“We got Half-Life 2 and Team Fortress running in VR. It was kind of a novelty, purely a development milestone. There was absolutely nothing compelling about them. Nobody’s going to buy a VR system so they can watch movies. You have to aspire and be optimistic that the unique characteristics of VR will cause you to discover a bunch of stuff that isn’t possible on any of the existing platforms.”
And optimistic developers are, Newell says.
“Developers are super excited. There’s nobody who works in VR saying, ‘oh I’m bored with this.’ Everybody comes back. For every idea they had in their first generation product, they have ten ideas now.”
