Standalone Virtual Reality Headsets Will Bring VR Into Focus

As some of you may know, I have a background in software engineering and CAD and have been active in emerging tech field over the last ten years through investments as well as growing start-ups in mobile IoT, AI and, recently, augmented reality (AR) and/or virtual reality (VR). We then have set up the Realities Centre to help filter through the hype and offer environments for the start-ups and corporates to harness those technologies for industry-specific verticals for AR/VR innovation and to learn more about the technologies.

This has enabled us to see where there is real traction, and we have been lucky to run great events, such as many hackathons, to validate industries and ideas as well as build what is probably the largest ecosystem of developers, mentors and corporate partners. I have seen first-hand how new users react to technology, interfaces and experiences through our many public and private conferences, and also having been personally involved in a company that offers virtual reality tours for multi-user property. I have been scaring many IT directors with the account of the number of involved cables as well as the software restarts and computer requirements for the package that would be needed for them to run our experiences. Moreover, I have seen companies finding ways around that problem by on-boarding clients first with a preloaded mobile VR package, which is an all-in-one “case” to reduce this hardware pain, and then trying to upgrade them to a full high spec solution.

6 DoF Room-scale VR Headsets Solve A Lot Of Problems

It is, therefore, out of genuine sentiment that I think that this year’s upcoming standalone headsets that include room-scale tracking (called 6 degrees of freedom or “6DoF”, as opposed to 3DoF like in mobile VR), such as the HTC Vive Focus, Pico Goblin and Google Mirage Solo with Daydream (pictured below), are going to really solve a lot of the difficulties that the potential large-scale B2B2C users experienced. As those were eliminated, their decision to adopt VR at scale for their organisation and to deliver to end-users shall be much easier.

Standalone VR HeadsetsOf course, high-end tethered headsets still have an important role to play for engineers, designers, arcades and gamers as they benefit from higher specifications, such as the HTC Vive Pro, which delivers optimal graphics and positional tracking with wireless options coming soon. Those still have very legitimate uses and innovations, such as eye tracking, Brain-Computer Interface (BCI) and more, are key. However, that would be specified in another article.

HTC Vive Pro 2.0 KitInstead, this standalone adoption is meant for the schools, retailers, training academies and departments and entertainment centres (although arcades can be included here too), and, therefore, they are meant for commercial mass use. Note that I am mentioning 6DoF here, as I think that any headset that does not offer positional tracking – to ensure that the head and body are fully tracked in space due to the presence of the inside-out sensors – should not be considered. 6DoF allows a minimum degree of immersion, which comprises the VR experience. I believe that 3DoF, although it is cheaper (at the moment), is a dangerous offering for the mass market, and, therefore, it is a no-go as follows: there is a lot more chance of getting sick, it is not as immersive and the experience is a lot more passive. It is bound to disappoint, and we all know the importance of first impressions …

3 DoF Is A No-go For B2B2C

Although Oculus, with the release of the 3DoF “Go” headset, has made some great tilt features to mimic head tracking and incorporate some great UX features as well as social tools, this is not enough. Consequently, we have found that some first-time users have become pretty sick after using it, and it is far from being a 6DoF device. Therefore, I think that for delivering VR experiences at scale for education, training, commercial experiences, such as experiential marketing (travel agents, property walkthroughs, etc.), 6DoF standalone headsets eliminate the difficulty of having to run a gaming PC/laptop, setting up the headset and sensors and installing the whole software stack. Standalone headsets solve that and, although they might be missing 6DoF hand controllers, already upgrades that are being announced to make those possible are present, as we have seen recently for the Vive Focus.

The last thing entails choosing which one should be used as a few varieties are available. Although they are all quite similar, they come with content platforms, accessories and an existing ecosystem. However, I would first look at the professional services and innovations that they would be bringing to the forefront. If you look at the use cases that involve education, training, entertainment and arts, HTC Vive has been a lot more innovative and active in those sectors to offer the following: recharge and cleaning cabinets for schools and training academies, centralised remote software control applications for synchronising experience delivery for groups (useful for schools, entertainment and more), an Art and Studio program as well as partnerships with big sports institutions (McLaren and Major League Baseball has been recently announced), and, most importantly, it has made continuous innovations which I will cover later on in this article.

Vive Focus And EducationThere has clearly not been as much engagement for education, arts, entertainment and technological innovations from the other standalone makers, such as Google, although they do have the requisite education and professional services’ infrastructure to be able to make those possible.

A great recent example of the education sector was given by Paola Paulino at the International School of Nanshan Shenzhen in which she used the Vive Focus to enable students to generate content they had created as well as integrate hand gestures integrations by the XR Pioneer program’ students (see picture below). The elementary, middle and high school students were involved in showcasing their VR Storybooks that they had created with Vive Paper. Moreover, they were the first students to pipe K-12 student-generated content to the new Vive Focus 2.0, which leverages hand gesture interactions.

And this also removes the difficulty in the past to setup large PC VR classes (like Vivedu did last year), whereas now it takes minutes to setup similar size classes with the Vive Focus. In terms of education, such a user friendly and easy setup VR system makes kids more engaged and learn more easily.

Vivedu

Offering Complete Professional Services As Part Of The Offering Is Key

It will take time, as we are still in the early stages of adoption, trial and understanding user onboarding and user experience development; however, those really offer a chance to deliver immersion at scale, albeit of a slightly lower quality (although the Vive Focus is 3K, which is higher than most tethered headsets). But this will keep improving with the increase in the better performing chipsets and also the upcoming 5G and cloud VR infrastructure, thereby delivering all that high-end graphic performance from the cloud itself with the help of machine learning, as I wrote earlier this year on VRFocus.

However, already, a great example of how remote streaming of the experiences is done has also been demonstrated in Vive’s latest announced feature on the Vive Focus as follows: the ability to stream VR from a PC is also key to solving content issue and also wirelessly use it (called Riftcat), so that one can play all Viveport and, especially, all Steam VR games on the Focus headset. HTC advises using this in a modern 5G Wi-fi local area network to have proper visual quality. This also solves the initial low amount of content that can be currently found on the Vive Focus ‘wave’ content platform by enabling tapping into Steam, the largest 3D experiences platform that is present.

Moreover, I have not even touched on the open space mixed reality experiences’ potentials that those headsets can offer while blending VR with the real world for facilitating fully spatial experiences, such as commerce and property simulation as well as entertainment. The inside-out cameras offer a lot of potential for space and object recognition. The most obvious and hoped for (as the device originally only came with one controller) feature was hand gesture tracking for control, similar to Leap Motion. This was also announced and teased as an additional feature on the update for the Vive Focus.

Vive Focus Hand TrackingEven more features are coming out, such as the ability to upgrade the Vive Focus’ current 3 degrees of freedom (3DoF) controller to behave like a 6DoF controller without the need for any additional hardware by leveraging the device’s existing front-facing cameras and AI computer vision technology as well as demonstrating the option of streaming the phone screen’s content from the latest HTC U12+ phone to the Vive Focus. This enables millions of existing mobile applications, video games and video content to be enjoyed on a super-sized screen on the top of making phone calls and creating cool social VR experiences.

China Is Leading The Way With Adoption And Implantation Of New Use Cases

It seems that VR innovation out of China really fuels from the adoption that the market has been undergoing (and, recently, it has been highlighted by Alvin Wang Graylin, President of Vive’s China business, showing 85% awareness of VR and great demographics. We can perceive improvement occurring at a great pace, which should result in optimising content and hardware improvements at scale, which is great as it holds the great promises for the standalone headsets. This should, importantly, enable the development of professional services around the setup, content production and maintenance of pools of standalone VR headsets, as the graphic given below shows, in the case of Vivedu’s ‘turnkey solution’ in China.

ViveduThat’s a great example of potential models that will be released for education and training and will be marketed as professional services in various industries. I cannot wait to contribute to that development as well as benchmark the best practices and lessons that can be learnt from the Chinese market to effectively grow these professional services in Europe at scale.

Cloud VR & 5G: More Work Still Required

There has been a lot of talks and great articles in the past on web augmented reality (AR) and virtual reality (VR). From a content and user experience point of view, it makes great sense to have VR content web based in the Cloud instead of downloading on a PC, especially as standalone VR headsets are being released. But how do you do that to deliver great experiences as good as from the ones downloaded on a PC?

Last year, HTC Vive partnered with Dalian Television and Beijing Cyber Cloud in China to launch the world’s first Cloud VR service for a commercial trial in Dalian; rather than plugging a Vive VR system to a PC, it was hooked up to a set-top box with access to the carrier’s VR Cloud content store and customers were offered a broadband package in the process.

Although there was not much feedback from those tests, it seemed that any drops in connection meant latency rate drops as well as image definition, therefore impacting the VR experiences where reliable high quality is a must.

Huawei: 5G LatencyMore recently, Verizon ran trials involving streaming VR over 5G of a Super Bowl experience at 50 megabits per second per HTC Vive VR headset, which is relatively the low end of 5G but still much higher than 4G.

This month, at the Winter Olympics, KT Corporation and Intel are also showcasing the technology. High-definition pictures and 360 Video VR  from the Games in Pyeongchang are streamed through 5G from multiple cameras.

It is said that VR will flourish thanks to 5G, because headsets can work free of wires and debilitating motion sickness because latency reduces to almost zero with 5G.

But as I found out at a pre-Mobile World Congress 2018 (MWC) briefing by Huawei recently in London, it is not as simple. First of all, connection drops and variations in bandwidth like the ones experienced over fibre broadband in the Cloud-based Chinese trial could be the same over 5G: there could be signal drops and switching from 3G to 4G back to 5G, really potentially impacting the experience.

So, in a world where fibre is very likely to be taken over by mobile telecoms and therefore 5G, how can we make sure that we get a compelling Cloud-based experience for VR? And therefore for 360 degree video and traditional video streaming as those will also most probably deliver all our media in the future, think terabytes monthly plans for 24K video feeds. Especially as 24K has been theoretically identified in the past as one of the important element to deliver true realistic VR experiences.

via Huawei Pre MWC 2018 Brief

To solve those problems, it was interesting to first learn at the Huawei brief about Massive MIMOs (Massive Multiple Input Multiple Output) technologies which are basically mobile data masts with very clever multi antennas which are working together following algorithms to ensure that 5G signals do not get dropped. Those have been used in 4G already and it seems that they are just getting better and better with more security, smaller form factors and more power efficient.

The second important aspect to ensuring that the signal is stable and the VR content is of high quality is an AI Cloud which is capable at managing GPUs (the Graphics Processing Units running high quality video) and content in such a way that the latency output and resolution stay of high enough quality. There is currently more and more competition in AI Clouds but what seemed really interesting about the Huawei AI Cloud (called Atlas), is that it is directly feeding from the rest of the network infrastructure data in order to optimise itself using machine learning. It looks at the massive MIMOs 5G clients connections quality as well as the video GPU throughput from the Cloud to make sure the content delivery is optimised.

Network Cloud Engine 5G
via Huawei Pre MWC 2018 Brief

This end to end hardware / software optimisation using AI machine learning is surely the way forward in order to make sure we get compelling VR experiences in our future standalone high-quality VR headsets, free of wires and computers and I cannot wait to experience more of it. MWC 2018 will showcase more demonstrations of those technologies and it will be exciting to start getting those to the end user. Some partnerships such as last year’s announced TPCast & Huawei X Labs Cloud VR rendering solution partnership, are surely exciting developments to watch as well as the launch of the Open Lab cooperation plan launched last year to focus on innovation around Cloud VR. Those will be a game changer as offering a lot less friction for the user to experience VR with stable high quality and therefore making adoption much easier for applications such social VR, eSports, live broadcast, business remote collaboration, and much more.

Omnichannel Realities

In my last VRFocus article from September, I stressed the importance of Virtual Reality (VR) applications in focusing on usefulness and superseding reality. Then going on to highlight how content should be delivered via accessible (cheap and easy-to-use) hardware such as VR headsets connected to media boxes (e.g., Netflix) to reach mass market adoption.

Well, cases of such VR hardware are coming into play this year: Microsoft announced their VR OEM Windows “Mixed Reality” headset plans last year (previously called “Holographic”) and just provided more details at Game Developers Conference in San Francisco, beginning with key partnerships with Dell, Acer, Lenovo as well as launching their developers kits. These easy-to-setup and more affordable devices have the potential to become a home accessory for the mass market (I am not covering the gaming or B2B industries, nor their customer base or high spec VR & Augmented Reality (AR) hardware in this article, and therefore not referring to those).

The headsets don’t require external trackers and instead use their on-board sensors to provide indoor tracking, as well as other technologies, to enable what Microsoft has coined ‘6 Degrees of Freedom’. Although they are still tethered – for the moment at least as the wireless technology has been changing a lot in the past few months with cheaper solutions being offered by many different providers – their setup seems to be as simple as plug and play.

 

Microsoft announcing their VR headsets in 2016

Microsoft Acer Headset

Although their specifications are yet to be announced, at a price point of $300 one would hope they will be sold as bundles with new laptops and desktop computers. Indeed, as they are OEM and therefore built and distributed by computer manufacturing partners such as HP, Dell, Lenovo and more, it would make sense for Dell (as an example) to sell them as a PC with VR headset bundle this upcoming Christmas season. However, they could also lower the margins so much so that when someone is shopping for a computer the additional cost to add a VR headset would be even lower.

Also, one can expect GPU/CPU requirements and parts costs to go down, especially for the screens and chipsets; therefore, this will dramatically increase the accessibility in terms of cost and lower spec PCs requirements in future versions.

Example of a Dell online purchase bundle options, VR coming soon too?

As part of the Microsoft developer community, the Windows “Mixed Reality” or “Holographic” developer program also offers the promise of attracting an enormous pool of Microsoft developers to develop news apps, as well as extensions and browsers toolkits.

Perhaps the most important aspect here is the potential for the Windows “Mixed Reality” VR headsets to become a home accessory sitting next to one’s printer. Imagine you are browsing a website and there is a VR button to visualise the items on your basket at their real size or to watch a preview of a potential holiday; one would just click, put the headset on, experience the products and services, then remove or continue to finish your purchase in the VR mode!

The headset could become a tool which improves the customer journey experience, especially in terms of e-commerce – this is where there is truly mass-market adoption potential. Therefore, I don’t believe these VR headsets will be purchased by the mass market as a gaming or entertainment device (unlike the headsets which would be twinned with media boxes or gaming consoles, but also the Windows “Mixed Reality” VR which will be compatible with the Xbox gaming console), but instead as a tool being used sporadically to improve the internet browsing experience or through some VR apps experiences.

The browsing experience will also be seamless, with VR call to action buttons integrated within existing browsers – such as Internet Explorer – to create a seamless experience. We’ve already seen Google integrating VR functionalities in its’ Chrome browser and, therefore, it seems logical that Microsoft Explorer will also have these VR integrations. Given that there is a whole VR/Augmented Reality (AR) Windows Mixed Reality integrated development platform, we will be sure to see more and more AR, Mixed Reality (MR) and VR integrated features within the Windows Operating System and its’ core applications, such as Explorer, Apps, Office, Skype, LinkedIn and more.

At this stage, VR becomes part of the e-commerce customer journey which, amazingly, extends into an AR/MR/VR/Artificial Intelligence (AI)/Internet of Things (IoT)/Wearables circle:

A customer uses a mobile or wearable Augmented/Mixed Reality device to gain more information in a shop about a product or location, or just special offers. To do that, AI computer vision and IoT provide more information about the product whilst also learning about the customer’s behaviour. While doing this, an updated 3D pointcloud of the shop and the product has been scanned. All this information can be used in a Virtual Reality version of the shop by another customer who is shopping fully or partly in VR (i.e., browser mode).

Of course, more detailed scanning and updates will also be carried over by specific staff (and drones) in shops, with the VR versions will be customised and adapted using machine learning to deliver a personalised experience.

On the AR and MR side, which company is better positioned to provide cloud point data and then a VR rendering and version of a location such as a business? The answer is a company who has had AR products tested long before the current wave of AR and VR buzz.

Google

It seems logical that Google will be (or already is?) a central provider of those AR cloudpoints through existing data; but also of AR wearables and mobile devices, such as the hybrid DayDream/Tango phones like the Asus Zenfone AR. It’s also logical that it will release a successor to the Glass product for the mass market, since it arguably has the most experience in that area (with companies like ODG, a very experienced AR glasses maker).

ASUS ZenFone AR with Google Daydream integration

Also, bear in mind that there is already a VR version of Google Earth on Steam for the HTC Vive, which shows that having Google Maps VR is not far-fetched at all and that all AR scanning would update outdoor and indoor datasets. Google also has relationships with businesses that are mapped and on the internet through its’ SEO; this provides a great advantage for existing information and relationships to integrate those within the AR/MR information systems, as well as VR e-commerce experiences.

 

This illustrates how close and connected AR/MR and VR have become, as well as how intrinsic AI, IoT & wearables technologies are to the whole system.

From a hardware perspective, it also shows that Microsoft Mixed Reality VR OEM headsets are not the only potential mass market devices; it seems logical that future Google Daydream VR headsets and their wearable AR products will be fully integrated with Google Tango phones as a hybrid (beyond the current two modes in one phone).

Therefore, Google and Microsoft will have strong multi-platform AR/VR capabilities that harness their operating systems, technologies and ecosystems.

Most importantly, this means the Omni channel strategy for brands and marketers is more streamlined and effective if they ensure they harness those AR/MR/AI/IoT/Wearables interactions and prepare accordingly.

Consequently, instead of calling this a ‘circle’ or a ‘system’, it seems to be more a strategic AR/MR/VR vision relying on a product/service’s ‘omni-channel presence’ or ‘omnichannel realities’.

To prepare for their presence on those various technologies, brands and agencies must prepare for seamless integrations of AR and VR features within their marketing and e-commerce channels. It starts, for example, with adopting 3D scanning technologies to make the products available for visualisation, as well as to integrate those assets for narrated/interactive marketing experiences. However, these are not simple integrations as they require different skillsets and product management systems.

Also, by making products available in 3D, their design is out in the open, which is no different from stocking a product physically in a shop for a customer to observe. However, the most conservative brands may be slower to accept this, although they will eventually be required to adapt.

These are exciting times to prepare the grounds for augmented customer journeys, in which the focus really comes back to usefulness and personalisation.

I don’t believe in providing more information to visitors/customers in augmented shops or on e-commerce websites with VR functionalities, but instead a more seamless and customised information delivery system providing much higher satisfaction and conversion rates.