The benefit of a tethered, desktop-powered virtual reality setup is that you can get significantly more visual fidelity from a PC with a GeForce GTX 1080 graphics card plugged into a power outlet as opposed to the tiny, battery-powered GPU in a phone. But Google is claiming that could change.
Google has developed a technology called Seurat that makes it easier to render lifelike, highly detailed 3D environments on a mobile VR headset. While companies like Nvidia and Qualcomm have beefed up the graphics capabilities of smartphones in recent years, those tiny chips still lag behind massive PC cards. But instead of trying to brute force pixels onto its Daydream headsets with more power, Google is using software to enable developers to render beautiful scenes in real time. And Hollywood effects house Industrial Light and Magic’s experimental division, ILMxLab, has already used Seurat to bring some of the assets it built for Rogue One: A Star Wars Story to life in a VR experience.
“When xLab was approached by Google, they said they could take our ILM renders and make them run in real-time in VR,” ILMxLab senior UX engineer Lewey Geselowitz said in a video about Seurat. “Turns out that’s true. I really think we’re onto something.”
Google’s not explaining exactly how this technology works — although it is promising to reveal more later this year. Bavor did imply that the magic is happening on the software side of things.
“It uses some clever tricks to help you achieve desktop-level graphics or better with a mobile GPU,” Google VR boss Clay Bavor wrote in a blog post. “Seurat enabled ILMxLAB … to bring the cinema-quality world of Rogue One to a mobile VR headset.”
If this technology works as advertised, then Google could expedite the process of making PC-based VR obsolete. That’s especially true if the company combines it with its dedicated headset that it is building with inside-out tracking that doesn’t require external sensors to determine your position in the world. When all of those elements come together, you could have a believable cyber world to explore anywhere that you can bring along a battery-powered VR headset.
Das nächste Update von Daydream mit dem Namenszusatz Euphrates erhält erhebliche Verbesserungen, von denen alle Anwender der VR-Plattform profitieren werden – auch indirekt, denn unter der Haube stecken ebenfalls wichtige Neuerungen. Bei einer Presseeinladung auf der gerade laufenden Entwicklermesse I/O hat Google die neuen Funktionen des Updates vorgestellt, Daydream 2.0 „Euphrates“ soll mit Android O verfügbar sein.
Google Chrome in Daydream
Eine eigene App bringt den Google-Browser Chrome nativ in VR. Praktisch: Lesezeichen und Einstellungen bleiben erhalten, die Version soll identisch mit der Smartphone-App sein und dann flüssig und komfortabel im VR-Headset laufen.
Daydream Home
Damit man auch in der virtuellen Realität nicht ganz vom echten Leben abgeschottet ist, führt Google in Daydream 2.0 ein Dashboard ein, das man in jeder VR-Anwendung aufrufen kann. Im Dashboard kann man aber nicht nur Benachrichtigungen checken, sondern auch Einstellungen ändern und von App zu App wechseln. Außerdem passt Google die Oberfläche für die angekündigten All-in-One-Headsets von HTC und Lenovo an, bei denen eine Touchsteuerung wegfällt. An den Store legt Google ebenfalls Hand an und möchte die Benutzeroberfläche ändern sowie mehr Kategorien einführen, womit sich Inhalte besser finden lassen. Google will mit den Änderungen zu den Stores von Oculus und SteamVR von Valve aufschließen.
Daydream Cast
Daydream Cast nutzt Google Cast, um Inhalte aus der virtuellen Realität heraus zu streamen. So kann beispielsweise die ganze Familie am Fernsehgerät an der VR-Erfahrung teilhaben, wenn auch nur in 2D.
Daydream Sharing
In eine ähnliche Richtung geht Daydream Sharing. Hier lasen sich Screenshots anfertigen und über soziale Medien teilen. Als Beispiel nannte Google das VR-Malprogramm Tiltbrush: Internetnutzer würden vor allem dann Tiltbrush entdecken, wenn Anwender der Software etwas darüber posten würden.
360-Grad-Videos teilen
Die beliebteste virtuelle Anwendung seien 360-Grad-Videos, erklärt Google. Damit Anwender gemeinsam eine Erfahrung erleben können, lassen sich Freunde zu YouTube-360-Grad-Videos einladen. Das ist allerdings keine Funktion von Daydream, sondern funktioniert über die YouTube-VR-App. Auch ein soziales Element hält die Software bereit, Zuschauer können miteinander kommunizieren und Videos teilen.
Unter der Haube
Neben den für Daydream-Experten sichtbaren Änderungen gibt es auch für Entwickler wichtige Verbesserungen. Beispielsweise einen Multi-Prozessor unterstützenden VR-Compositor, die Bereitstellung der Vulkan-Schnittstelle, Multi-Layer-Unterstützung sowie Multiview-Stereo-Rendering.
Im Audio-Bereich fügt Google die Unterstützung von Unity und Unreal sowie Wwise und FMOD hinzu. Außerdem gibt es einige eigene Audio-Effekte. Beispielsweise lässt sich die Raumakustik vorberechnen, was Rechenzeit spart. Eine Geschwindigkeitsoptimierung und Verringerung der Latenz verspricht eine direkte Anbindung der Sensoren und ein Echtzeit-Sensor-Buffering.
Damit die neuen Daydream-Headsets von HTC und Lenovo funktionieren, unterstützt das Update VR-Systeme mit zwei Bildschirmen sowie den Qualcomm Snapdragon 835 und die Grafikeinheit ARM Mali-G71.
Daydream’s 2.0 software update, called Daydream Euphrates, is going to bring along with it some major changes that are set to reach down to the core of the upcoming Android ‘O’ operating system. In a AR/VR press briefing, Google told us there are a number of new systems the update will add across all Daydream headsets, regardless of whether they’re standalone or smartphone-based.
The Android operating system was primarily designed to work on devices with a touchscreen, but with the addition of standalone headsets, this is about to change in a big way. To make the upcoming Android ‘O’ OS more VR-friendly, Google is integrating some things you might think are pretty basic, but in the end are intended to give VR users the tools they need to manage the ‘always in VR’ nature of standalone headsets.
Daydream Home
2.0 will add a dashboard on top of any VR experience so you can check notifications, change settings, pause experiences, and most importantly switch from app-to-app like on a smartphone. This includes support for 2D system UI.
As for the store itself, Google says they want to put content front and center of the Daydream Home experience with an update to the Daydream UI.
image courtesy Google
While the company contends that 360 video is the most popular type of content people are consuming in VR right now, they’re adding more ways to categorize so that users can easily browse the store for all types of content, bringing Daydream Home more in line with digital marketplaces like Oculus Home and Valve’s SteamVR UI.
Daydream Cast
Google Cast-support is also coming to Daydream devices so people outside of VR can see what you’re doing.
image courtesy Google
This also means app developers can create cooperative games that involve people outside the headset like VR party game Keep Talking and Nobody Explodes (2015).
Daydream Sharing
Daydream Sharing adds the ability to capture screenshots and screencaptures, and then share them via social media.
“This worked really well in Tiltbrush,” a Google spokesperson remarked. “It’s actually one of the number one ways people discover and learn about Tiltbrush – is their friends who are in it post something about it.”
Developer Updates
Developers creating apps for the Daydream platform should take note; there’s also a number of graphics, audio and general performance updates as well.
Announced during the second day of Google’s annual I/O developer event, Daydream VR will soon have access to the Android Chrome web browser. Because of Chrome’s official support for WebVR, this means that users will be able to navigate their way to their next VR-optimised web experience without having to remove the headset.
Google has actively worked on bringing web-based content to VR for some time, having co-authored the WebVR specification (enabling rich VR content to be written with JavaScript and WebGL) three years ago, but the user experience has felt incomplete without proper browser support.
Soon, the Chrome browser will be available within Daydream, and it is the same application; sharing the same bookmarks, history, and login information as the ‘2D’ version you would normally use on your phone. The Daydream controller will enable easy navigation on any normal web page, and as soon as you hit WebVR content, it will launch into a more immersive mode. Eventually, the transition between 2D web pages and VR-optimised web content should become more seamless.
AR features are also expected to make their way into the landscape of web browsing, starting with an experimental build of Chromium launching on GitHub today, which has WebAR features enabled. Andrey Doronichev, Product Director of VR and AR apps at Google, explained how it should work similarly to WebVR, “being able to easily integrate AR features into your existing websites so that as a developer, you don’t have the friction of teaching the user new behaviours or making them download a new app”.
As an example, he showed a shopping experience running in WebGL and Javascript from a Wayfair website, which allows the user to mark out their room space and then place a virtual item (only the ones that fit), such as a coffee table, to see if it suits the room before they buy.
Over the last 12 months or so we’ve been keeping an eye on Google’s aggressive hiring for its AR/VR team which, among other things, suggested that the company was working on new “mass production” hardware that would go beyond the simple smartphone snap-in Daydream headsets. This week we learned that there was indeed something to all that hiring, as Google has announced new ‘standalone’ headsets coming to the Daydream platform, fully self-contained VR devices, the first of which will come from HTC and Lenovo. Until those headsets launch later this year, Road to VR was among a select group that got to see one of Google’s early Daydream standalone prototypes.
At Google’s Mountain View campus, a stone’s throw away from the hustle and bustle of I/O 2017, I sat at a literal round-table with core members of Google’s AR/VR team. No pictures were allowed. There, I learned about the latest Daydream and Tango developments that have happened since last year’s I/O. Core to the discussion was the new ‘standalone’ VR headsets coming to Daydream.
And while Google has announced that HTC and Lenovo are working on consumer versions of the device, and that the company has build a reference version with Qualcomm (for other companies to use as a foundation to make their own), the device I got my head in was even earlier than that; an internal prototype of roughly a year old. Lots of caveats were given: ‘the displays are more than two years old, the latency still needs to be optimized, you might see some dropped frames due to the old hardware and software’, I was told. And yet I was still very impressed with what I saw.
In the same conference room were two large circular carpets, maybe 10 feet in diameter. At the center I was handed an entirely black headset with a ‘halo’-style head strap (like PSVR) which tightened with a knob in the back and rested comfortably on the head. I could see that the lenses were shaped similar to PSVR’s, except they used a Fresnel design like the Vive. I could also see a clear indication that these headsets had eye-tracking hardware on board, but Google’s AR/VR team didn’t talk about that feature and it isn’t clear yet if we can expect eye-tracking in the final products.
Google didn’t let me snap pictures of the prototype, but the lenses looked similar to those on this Qualcomm reference headset | Photo by Road to VR
As I put the headset on and tightened the strap for comfort I found myself immersed in an underwater scene with translucent jellyfish floating about and a curious sea turtle who would swim by from time to time. Though Google hasn’t said much about the headset’s specs (and they are likely to differ for the consumer versions), the field of view looked close to what I’d expect from PSVR (~100 degrees diagonally), which is a huge improvement over the Daydream View’s more limited field of view.
Unlike prior Daydream headsets, which can only track rotation, the standalone Daydream headsets will use Google’s newly announced ‘WorldSense’ inside-out tracking to achieve positional tracking too, allowing you to walk around your environment just like you’d expect with the Vive, except with no external beacons for tracking.
I was able to comfortably roam the entire area of the circular carpet, getting up close to inspect the jelly fish floating around me. If I stepped toward the edge of the carpet, the virtual world would fade to black to let me know I was leaving the designated playspace. Theoretically WorldSense tracking could go much further than the carpet, but Google seems to be positioning it for now as a room-scale-capable system that you can use anywhere, rather than something you might roam your entire house with in one session. It isn’t clear yet if this is a design or technical limitation.
Though I only had my head inside this particular demo for five minutes or so, I was very impressed with the accuracy and latency of the tracking. At least for the relatively slow moving case of walking around and looking at things, it worked very well and felt much like I’d expect from a Vive. However, there were few static near-field objects in the demo with which to get up close and get a good assessment of any jitter, and it’ll take more time with the headset to see how it handles faster motions like ducking and dodging.
To achieve the inside-out tracking, Google tapped their Tango team to create a version of their optical inside-out tracking that’s optimized for VR, the result of which is WorldSense. On this prototype standalone Daydream headset, the tracking was achieved with two forward-facing cameras which are the only sensors—save for the usual on-board IMUs—that derive the positional information; Google confirmed the system doesn’t rely on a depth sensor.
To see how well the tracking could keep up, I tried covering of the two cameras on the front, and much to my surprise it still worked relatively well. Google said there’s a monoscopic mode which can kick in if occlusions like that were to happen, but generally the two cameras are working in tandem to do the tracking. I also tried looking straight down at the ground with my head just a foot or two above it (trying to give the camera’s less distinct visuals to work with), and found that the tracking held of perfectly.
Google said the room was not pre-mapped and that this demo was specifically designed to wipe any WorldSense data so that I was experiencing the headset as if it was the first time it saw the room. That said, the company did confirm that WorldSense will learn over time and get even better if you use it in the same place repeatedly.
For a year-old device, the Daydream standalone VR headset prototype is very promising. Newer iterations in work by Google’s partners, which will use Android O (which is further optimized for VR), as well as newer and more powerful hardware (based on Qualcomm’s Snapdragon 835), are likely to be a significantly more ‘premium’ experience compared to Daydream headsets that rely on a snap-in smartphone, thanks to their made-for-VR optimization and positional tracking which adds an entire new dimension to the mobile VR experience.
During Day 2 of Google I/O 2017, following major announcements of support for standalone VR headsets and a ‘2.0’ release of the Daydream software platform, Mike Jazayeri, Director of Product Management for Daydream, stated that “tens of millions” of Daydream-Ready devices are expected in the market by the end of the year.
After Daydream VR’s positive start in November 2016, it has taken a while for additional third-party Daydream-Ready devices to appear, perhaps due to the demanding minimum hardware requirements. Today, there are eight Daydream-Ready devices on the market, and momentum will continue to build throughout the year as flagship products from Samsung and LG begin to support the platform. The announcement of Samsung’s flagship S8 and S8+ receiving Daydream support is particularly notable, as they represent the best devices for use on the competing Gear VR platform. LG’s flagship smartphone due in the second half of 2017 is also confirmed to be Daydream-Ready.
image courtesy Google
In addition, “existing partners such as Motorola and ASUS will launch more Daydream-Ready phones by the end of the year” says Jazayeri. This means Google rightfully expects multiple millions of Daydream-Ready devices in the market by the end of year—which may also include the estimated sales figures for the newly-announced standalone headsets from HTC and Lenovo.
Thanks to its massive head-start, Samsung’s Oculus-powered Gear VR platform has dominated the high-quality mobile VR market since its launch in November 2015, but the announcements at this year’s I/O developer event represent a major momentum swing in Google’s direction.
Before my demo at Google I/O, Google’s head of VR Clay Bavor outlined a series of caveats and expectations for what I was about to experience.
First, he said, I was going to use a generation older hardware from internal prototypes that likely would be improved upon in practically every way before its release as a reference design to developers and eventually to consumers (Google hopes at least HTC can achieve consumer release in 2017 with its first standalone headset). Second, latency on this hardware was said to be under 30 milliseconds but would be improved to under 20 milliseconds — a critical threshold for comfort — by the time the system reaches broader deployment. The system was also running an older display at 75 frames per second. Bavor declined to comment as to the frame rate of the finished hardware.
Yet, for 120 seconds in one demo on a rug-sized space large enough for two to three steps in any direction I became one of the first journalists in the world to try WorldSense — an evolution of Google’s Tango efforts and an answer to the inside-out tracking technology Microsoft developed for its $3,000 HoloLens and then adapted for use in a series of far less expensive VR headsets starting with one from Acer. Facebook, too, is working hard on inside-out technology and showed its Santa Cruz prototype last year using this more convenient approach to VR headset design. Intel is working on it as well, and says it will have devices out by the end of the year.
WorldSense
The demo, according to Google, would reset after each journalist so the hardware would see the room fresh each time after each person donned the headset. WorldSense is designed to intelligently look for anchors in the real world and grow smarter over time — discarding parts that might have moved while gathering more detailed information about the world around the person wearing the headset. I saw this immediately as I started to move around in VR — with tracking initially stuttering just slightly and then within 10 seconds or so turning rock solid with full freedom of movement in all directions. Though, of course, I was limited to my rug — if I moved outside the world would dim into blackness.
I covered both the outward facing cameras and Google’s tracking technology continued to track my head movements in every direction accurately for about one second. Then it reverted to the kind of 3DOF head tracking seen with current Daydream and Gear VR headsets. After removing my hands and letting the pair of forward-facing cameras see the world again I also got down on the ground facing toward the ceiling, looked to my sides on the floor, and noticed no jumps in tracking as I got up from the floor.
The scene was a simple underwater one with turtles and jellyfish floating around me. I was disappointed to note that there were no objects that seemed to move within my volume of movement freedom. In other words, the jellyfish and turtles seemed to stay just out of reach, so I couldn’t gauge whether an object that seems to float in the center of my virtual space would seem to drift out of place over time.
The space afforded for movement freedom was greater than I’ve experienced in Acer’s Microsoft-powered headsets but smaller than a carefully lit and arranged room shown by Facebook’s Oculus last year. Only Intel’s demo at CES this year placed objects in the center of the room allowing me to gauge the drift of virtual objects.
According to Google, the company isn’t expecting to initially offer active detection and warning about cats or kids that enter your space. It will, however, offer an established volume in which you can move around with complete freedom and then warn you if you move outside.
Bottom Line
I left my demo impressed by Google’s progress with inside-out tracking for VR headsets, but scratching my head about the market Google hopes to serve with these initial standalone systems.
Google’s guidance is that when the WorldSense standalone systems arrive — supposedly this year from at least HTC — is that they are likely best used seated and standing in place. This sets the systems up as an improvement on Google’s Daydream platform featuring better immersion, but still a small step on a long path that will still take years to realize. It is a lot to ask someone to invest $400-$800 in a dedicated headset that is a marginal improvement upon a phone-powered headset without the benefits of a fully featured phone you can use separate from VR. It is unclear what market Google hopes to serve with these initial standalones.
I was not able to see the headset paired with the Daydream controller and this is a critical component that will define which market segment this first generation of standalone headsets will target. Many developers should find their apps built for Daydream will extend to these standalone headsets without too much modification. However, developers will have to figure out how to design around the fact that you cannot naturally reach out with your arm using a Daydream controller to interact with an object.
Educational or exploratory apps like Google Earth could conceivably work incredibly well with these types of standalone headsets from HTC and Lenovo, but leading apps like Tilt Brush and Job Simulator will likely need to wait until some time after 2017 before a standalone system powered by Google’s technology is able to offer the kind of experience we’ve come to know with something like an HTC Vive.
Daydream-compatible standalone headsets are coming, revealed on stage at Google I/O yesterday, but the big question remains: how much is one of these inside-out positional tracking headsets going to cost? Google tells that although prices are dependent on the individual manufacturers, they expect it to cost about as much as a PC VR headset like Oculus Rift or HTC Vive.
Being able to pull a singular, bespoke VR headset out of a backpack—with no added wires or sensors—is pretty awesome, but the addition of inside-out positional tracking, or the ability to physically move your head forward, backward and side-to-side, is an even bigger step for mobile VR.
We sat down with Google’s VR team at a press briefing to learn more about how much the convenience (and immersion) was going to cost.
“It will vary by OEM, depending on what screens they decide to go with and so on,” a Google spokersperson told us. “I would expect the price-point to be around the same as you have with the desktop VR headsets today – minus the cost of the PC, drilling holes in your wall, and all that kind of stuff.”
This falls in line with what we’d expect, considering the guts of a standalone headset need to pack as much as (if not more than) the graphical horsepower of a VR-ready flagship smartphone like those already taking part in the smartphone-focused Daydream program. In any case, running acceptable frame rates while tracking the world around you in real-time didn’t exactly sound cheap to begin with.
For reference, the Oculus Rift when bundled with Touch costs just under $600. HTC prices the Vive at the higher end of the spectrum at $799 for their PC VR system. There’s no telling which end of the price spectrum the first two headsets, made by Lenovo and HTC, will take. HTC maintains their standalone headset will launch sometime later this year, so we’re sure to get a better idea closer to launch.
This of course doesn’t include the startup cost of a VR-capable PC, which although cheaper than ever thanks to software updates and cheaper GPUs from AMD and NVIDIA, is still a pretty large investment.
Wie wir gestern berichteten, gehen HTC und Google eine Partnerschaft ein und HTC möchte noch in diesem Jahr eine VR-Brille veröffentlichen, die nicht mit einem PC oder einem Smartphone verbunden werden muss. Außerdem wird die Brille auf Inside-Out-Tracking setzen und den Weg für Mobile-VR mit Room Scale Tracking (bzw. WorldSense Inside-Out-Tracking) ebenen.
HTC Vive für Google Daydream
Die neue Brille von HTC wird auf Android-Basis arbeiten und sollte sich auch bei der Leistung in den Bereichen der aktuellen High-End-Smartphones bewegen. Der Vorteil einer solcher Lösung ist, dass eine höhere Qualität mit niedrigeren Kosten ermöglicht werden kann. Eine Kombination aus Smartphone und Brille ist dennoch gefühlt günstiger, da wir das Smartphone auch für andere Dinge verwenden können, während die All-In-One-Brille nur zur Flucht in die Virtual Reality genutzt werden kann.
Doch vermutlich wird die Brille auch kein Schnäppchen werden, denn HTC sagt, dass es sich um eine „Premium“-Brille handelt.
HTC wird demnächst weitere Informationen über die Brille veröffentlichen und bereits in diesem Jahr soll die VR-Brille auf den Markt kommen. Spannend bleibt die Frage, welche Controller es für das System geben wird. Wir können uns nur schwer vorstellen, dass HTC dauerhaft auf den Daydream Controller mit 3DOF setzen wird, welcher nicht im Raum getrackt werden kann, während die Brille seine Position im Raum bestimmen kann. Zunächst sollen die Brillen von HTC und Co. aber mit dem herkömmlichen Daydream Controller auskommen. Da auch Qualcomm bei dem Projekt mitwirkt, könnte auch ein System für das Hand-Tracking zukünftig verwendet werden. Da HTC sehr stark betont, dass die Brille ein perfekter Begleiter sein soll, wäre jede zusätzliche Ausstattung auch zusätzliches Gepäck und würde von der Idee des Marketings wegführen. Wir dürfen gespannt sein.
During the Google I/O keynote, it was revealed that Google teamed up with Qualcomm to create a stand-alone mobile Daydream VR headset. The first units will be built by HTC and Lenovo, and will be available later this year.
