We’ve reported several times on the efforts of World of Tanks developer Wargaming who have previously experimented with 360 degree video, augmented reality (AR) and virtual reality (VR) in tandem with various videogames in its World of… series. This has usually been courtesy of Wargaming’s ‘Special Projects Division’ which has taken on a number of partnerships with institutions such as museums, as Wargaming seek to not only be a gaming platform but use that to help educate on the realities behind their title.
One of the museum partners is the Tank Museum at Bovington, found in the in the South West England county of Dorset. As part of the recent Tankfest 2017 event the Special Projects Division revealed a brand new experience featuring World of Tanks’ ultra-rare model the German-made Sturmtiger or ‘Assault Tiger’, part of the series of assault vehicles built on the design platform of the devastating Tiger I tank. Wargaming’s team, in collaboration with Gravity Jack, this time utilised support for both Microsoft’s Hololens head-mounted display (HMD) and Google’s Tango platform. Having the Sturmtiger roll straight into the museum before showcasing its movement, firing and gives both an ‘exploded’ and cross section view of what makes up the tank.
You can see a video of what visitors to Tankfest could enjoy below. VRFocus will bring you more updates on the latest uses of MR and the the Hololens and Tango platforms as we get it.
Augmented Reality has played a huge role at the developer conferences for Microsoft, Apple, Facebook, and Google, which is a great sign that the industry is moving towards spatially-aware computing. Microsoft is the only company to start with head-mounted AR with the HoloLens while the other three companies are starting with phone-based AR. They are using machine learning with the phone camera to do six degree-of-freedom tracking, but Google’s Project Tango is the only phone solution that’s starting with a depth-sensor camera. This allows the Tango to do more sophisticated depth-sensor compositing and area learning where virtual objects can be placed within a spatial memory context that is persistent across sessions. They also have a sophisticated virtual positional system (VPS) that will help customers locate products within a store, which is going through early testing with Lowes.
I had a chance to talk with Tango Engineering Director Johnny Lee at Google I/O about the unique capabilities of the Tango phones including tracking, depth-sensing, and area learning. We cover the underlying technology in the phone, world locking & latency comparisons to HoloLens, virtual positioning system, privacy, future features of occlusions, object segmentation, & mapping drift tolerance, and the future of spatially-aware computing. I also compare and contrast the recent AR announcements from Apple, Google, Microsoft, and Facebook in my wrap-up.
LISTEN TO THE VOICES OF VR PODCAST
The Asus ZenPhone AR coming out in July will also be one of the first Tango & Daydream-enabled phones.
Wo finde ich in diesem riesigen Supermarkt bitteschön die Tapenade? Manchmal weiß sogar die freundliche Verkäuferin keine Antwort und man irrt gemeinsam verzweifelt durch die Gänge, bis man die Olivenpaste endlich findet. Google hat auf seiner Entwicklerkonferenz I/O 2017 einen Lösungansatz für dieses und ähnliche Probleme vorgestellt: VPS (Visual Positioning Service) beginnt dort, wo GPS aufhört, und leitet einen in Innenräumen zum angestrebten Ziel. Dabei stellt VPS eine GPS-artige turn-by-turn-Navigation zur Verfügung, nutzt aber zusätzlich die Möglichkeiten von Augmented Reality.
VPS: Tanze Tango mit mir
Mit Tango entwickelte Google eine AR-Plattform, die Tiefeninformationen im Raum aufnehmen, sowie Bewegungen tracken und auswerten kann. VPS nutzt das Inside-out-Tracking-System von Tango und kombiniert es mit Google Maps, um die Navigation in Innenräumen zu ermöglichen. Dabei verspricht Google, dass das System zentimetergenau arbeitet und beispielsweise Kunden im Baumarkt exakt zum gesuchten Schraubenzieher führen kann. In ausgewählten Geschäften einer amerikanischen Baumarktkette soll das schon heute funktionieren, verspricht Google, außerdem in einigen Museen. Bisher benutzt VPS lediglich das Display, um über die Darstellung des Raumes seine Informationen zu legen, allerdings stellt der Google-VR-Entwickler Clay Bavor auch ein Audiosystem für Sehbehinderte in Aussicht.
Einen Haken gibt es allerdings bei der Sache, bevor man einen Tango aufs Parkett legen kann: Um VPS nutzen zu können, benötigt man zwingend ein Smartphone mit Tango. Vorreiter mit einem ersten marktfähigen Produkt war Lenovo. Der Hersteller veröffentlichte vor knapp einem Jahr mit dem PHAB2 Pro ein Tango-fähiges Smartphone.
Google setzt auf Innenraum-Navigationssysteme
Auf den Marktstart wartet noch das wesentlich interessantere und zeitgemäßere Zenfone AR von Asus, das neben Tango auch Daydream unterstützt und damit zu den spannendsten Neuerscheinungen im Hardware-Markt gehört. Das Smartphone mit 5,7-Zoll-WQHD-Bildschirm kann sich zeigen lassen: Den Snapdragon 821 unterstützen je nach Konfiguration 6 oder 8 GB Arbeitsspeicher, für die Tiefeninformationen für Tango sorgt das TriCam-System, das wie der Name schon vermuten lässt aus gleich drei Kamerasysteme auf der Rückseite besteht: Die 23-Megapixel-Hauptkamera, eine Kamera für das Motion-Tracking und die dritte für die Erfassung von Tiefeninformationen. Dann lässt sich beispielsweise auch das Tango-Spiel Phantogeist zocken. Im Gegensatz zu anderen AR-Spielen, können sich virtuelle Gegner beispielsweise hinter realen Gegenständen verstecken.
Times change, even for buildings and organisations whose entire purpose is chronicling the passing of said times. Nothing stays the same, even for museums. Exhibits change, are moved, the museum itself evolves. Now thanks to Google Tango visitors to one museum are getting the chance to travel back in time to see how things used to look.
The museum in question is the National Museum of Singapore who have teamed with Google and tour guide firm GuidiGo. Using scans taken by Google utilising the area learning capabilities of Tango the team has created a number of locations in the building where visitors, provided with a Lenovo Phab 2 Pro for their free hour long tour, can use it to see an augmented reality (AR) view of how things used to look. Even allowing them to take photos of exhibits that aren’t even there anymore.
Speaking to The Straits Times, Museum Director Angelita Teo explained the move. “The emergence of digital and future technology has opened many doors for museums worldwide and we now have the opportunity to redefine the conventional museum experience. Enhanced architectural tours will enrich our visitors’ experience of our stories and their connection to the past”.
Tango will no doubt be a major focus of Google’s I/O event later this month. For all the latest news be sure to check back with VRFocus.
The Unity video game development engine is set to receive official, native support for Google's Project Tango augmented reality platform via an update set for release later this year.
The Unity video game development engine is set to receive official, native support for Google's Project Tango augmented reality platform via an update set for release later this year.
Some shops are sufficiently large that it is easy to get lost in them, or at least very hard to locate the item you are looking for in a store that covers several acres. Hardware store Lowe’s believe they have a solution with their new in-store navigation app.
Using Google Tango-enabled phones, the app uses motion tracking and area learning to guide customers through the store using an augmented reality (AR) interface. Any Tango-enabled smartphone is able to use the app, which can create and store lists of required items, access product reviews and comparison information in addition to simply providing store navigation.
The technology is planned to be rolled out to stores in California and Washington in April, where it will be evaluated and tested before further roll outs.
“Our research shows that helping make it easier for customers to find products in stores not only makes for a better shopping experience, it allows our associates to spend more time advising on home improvement projects,” said Kyle Nel, executive director of Lowe’s Innovation Labs. “With Lowe’s Vision: In-Store Navigation, we’ve created a more seamless experience using breakthrough technology so customers can save time shopping and focus more on their project.”
Lowe’s have used virtual reality technology in their stores before, as they have previously introduced VR experiences to some of its stores to help customers with learning how to tackle tricky DIY projects. Lowe’s even has its own research arm, Lowe’s Innovation Labs, which focusses on new ways to improve the experience for Lowe’s customers.
VRFocus will keep you updates with new of VR and AR use in retail.
In my last VRFocusarticle from September, I stressed the importance of Virtual Reality (VR) applications in focusing on usefulness and superseding reality. Then going on to highlight how content should be delivered via accessible (cheap and easy-to-use) hardware such as VR headsets connected to media boxes (e.g., Netflix) to reach mass market adoption.
Well, cases of such VR hardware are coming into play this year: Microsoft announced their VR OEM Windows “Mixed Reality” headset plans last year (previously called “Holographic”) and just provided more details at Game Developers Conference in San Francisco, beginning with key partnerships with Dell, Acer, Lenovo as well as launching their developers kits. These easy-to-setup and more affordable devices have the potential to become a home accessory for the mass market (I am not covering the gaming or B2B industries, nor their customer base or high spec VR & Augmented Reality (AR) hardware in this article, and therefore not referring to those).
The headsets don’t require external trackers and instead use their on-board sensors to provide indoor tracking, as well as other technologies, to enable what Microsoft has coined ‘6 Degrees of Freedom’. Although they are still tethered – for the moment at least as the wireless technology has been changing a lot in the past few months with cheaper solutions being offered by many different providers – their setup seems to be as simple as plug and play.
Microsoft announcing their VR headsets in 2016
Although their specifications are yet to be announced, at a price point of $300 one would hope they will be sold as bundles with new laptops and desktop computers. Indeed, as they are OEM and therefore built and distributed by computer manufacturing partners such as HP, Dell, Lenovo and more, it would make sense for Dell (as an example) to sell them as a PC with VR headset bundle this upcoming Christmas season. However, they could also lower the margins so much so that when someone is shopping for a computer the additional cost to add a VR headset would be even lower.
Also, one can expect GPU/CPU requirements and parts costs to go down, especially for the screens and chipsets; therefore, this will dramatically increase the accessibility in terms of cost and lower spec PCs requirements in future versions.
Example of a Dell online purchase bundle options, VR coming soon too?
As part of the Microsoft developer community, the Windows “Mixed Reality” or “Holographic” developer program also offers the promise of attracting an enormous pool of Microsoft developers to develop news apps, as well as extensions and browsers toolkits.
Perhaps the most important aspect here is the potential for the Windows “Mixed Reality” VR headsets to become a home accessory sitting next to one’s printer. Imagine you are browsing a website and there is a VR button to visualise the items on your basket at their real size or to watch a preview of a potential holiday; one would just click, put the headset on, experience the products and services, then remove or continue to finish your purchase in the VR mode!
The headset could become a tool which improves the customer journey experience, especially in terms of e-commerce – this is where there is truly mass-market adoption potential. Therefore, I don’t believe these VR headsets will be purchased by the mass market as a gaming or entertainment device (unlike the headsets which would be twinned with media boxes or gaming consoles, but also the Windows “Mixed Reality” VR which will be compatible with the Xbox gaming console), but instead as a tool being used sporadically to improve the internet browsing experience or through some VR apps experiences.
The browsing experience will also be seamless, with VR call to action buttons integrated within existing browsers – such as Internet Explorer – to create a seamless experience. We’ve already seen Google integrating VR functionalities in its’ Chrome browser and, therefore, it seems logical that Microsoft Explorer will also have these VR integrations. Given that there is a whole VR/Augmented Reality (AR) Windows Mixed Reality integrated development platform, we will be sure to see more and more AR, Mixed Reality (MR) and VR integrated features within the Windows Operating System and its’ core applications, such as Explorer, Apps, Office, Skype, LinkedIn and more.
At this stage, VR becomes part of the e-commerce customer journey which, amazingly, extends into an AR/MR/VR/Artificial Intelligence (AI)/Internet of Things (IoT)/Wearables circle:
A customer uses a mobile or wearable Augmented/Mixed Reality device to gain more information in a shop about a product or location, or just special offers. To do that, AI computer vision and IoT provide more information about the product whilst also learning about the customer’s behaviour. While doing this, an updated 3D pointcloud of the shop and the product has been scanned. All this information can be used in a Virtual Reality version of the shop by another customer who is shopping fully or partly in VR (i.e., browser mode).
Of course, more detailed scanning and updates will also be carried over by specific staff (and drones) in shops, with the VR versions will be customised and adapted using machine learning to deliver a personalised experience.
On the AR and MR side, which company is better positioned to provide cloud point data and then a VR rendering and version of a location such as a business? The answer is a company who has had AR products tested long before the current wave of AR and VR buzz.
Google
It seems logical that Google will be (or already is?) a central provider of those AR cloudpoints through existing data; but also of AR wearables and mobile devices, such as the hybrid DayDream/Tango phones like the Asus Zenfone AR. It’s also logical that it will release a successor to the Glass product for the mass market, since it arguably has the most experience in that area (with companies like ODG, a very experienced AR glasses maker).
ASUS ZenFone AR with Google Daydream integration
Also, bear in mind that there is already a VR version of Google Earth on Steam for the HTC Vive, which shows that having Google Maps VR is not far-fetched at all and that all AR scanning would update outdoor and indoor datasets. Google also has relationships with businesses that are mapped and on the internet through its’ SEO; this provides a great advantage for existing information and relationships to integrate those within the AR/MR information systems, as well as VR e-commerce experiences.
This illustrates how close and connected AR/MR and VR have become, as well as how intrinsic AI, IoT & wearables technologies are to the whole system.
From a hardware perspective, it also shows that Microsoft Mixed Reality VR OEM headsets are not the only potential mass market devices; it seems logical that future Google Daydream VR headsets and their wearable AR products will be fully integrated with Google Tango phones as a hybrid (beyond the current two modes in one phone).
Therefore, Google and Microsoft will have strong multi-platform AR/VR capabilities that harness their operating systems, technologies and ecosystems.
Most importantly, this means the Omni channel strategy for brands and marketers is more streamlined and effective if they ensure they harness those AR/MR/AI/IoT/Wearables interactions and prepare accordingly.
Consequently, instead of calling this a ‘circle’ or a ‘system’, it seems to be more a strategic AR/MR/VR vision relying on a product/service’s ‘omni-channel presence’ or ‘omnichannel realities’.
To prepare for their presence on those various technologies, brands and agencies must prepare for seamless integrations of AR and VR features within their marketing and e-commerce channels. It starts, for example, with adopting 3D scanning technologies to make the products available for visualisation, as well as to integrate those assets for narrated/interactive marketing experiences. However, these are not simple integrations as they require different skillsets and product management systems.
Also, by making products available in 3D, their design is out in the open, which is no different from stocking a product physically in a shop for a customer to observe. However, the most conservative brands may be slower to accept this, although they will eventually be required to adapt.
These are exciting times to prepare the grounds for augmented customer journeys, in which the focus really comes back to usefulness and personalisation.
I don’t believe in providing more information to visitors/customers in augmented shops or on e-commerce websites with VR functionalities, but instead a more seamless and customised information delivery system providing much higher satisfaction and conversion rates.
3D interactive sales and marketing firm Kaon Interactive has announced it will be introducing augmented reality (AR) experiences into it’s High Velocity Marketing Platform.
Kaon AR will allow the introduction of 3D Product Models that are scaled and designed to be the same size as the physical product to be seamlessly introduced to real-world environments with the use of Tango-enabled mobile phones and tablets.
This allows customers who, for example, may be shopping for new furniture to look through the smartphone and see what a specific sofa or coffee table might look like if placed in their living room.
“The effectiveness of marketing and sales experiences are dramatically increased when customers can see your products appear in their actual physical space and can then contextualize the benefits of that solution because they have that clear mental image of how that product fits and works,” said Gavin Finn, President & CEO of Kaon Interactive. “It truly creates an emotional connection with prospects that transcends traditional sales and marketing relationships.”
Rachel Zerilla, Marketing Analyst at Seagate Technology added;”At CES the WOW factor of demonstrating our cloud systems using augmented reality was amazing. Investors, customers, partners and analysts could walk right into the system to explore unique features and visualize it in the physical space at scale. It fit in perfectly as to how we were deploying technology innovation at the event.”
Kaon AR is available immediately for customers as part of the High Velocity Marketing Platform.
VRFocus will continue to bring you the latest on AR and VR marketing.
Developer Legacy Games is re-launching Crayola Color Blaster, it’s augmented reality (AR) mobile game. The re-launch will contain a new content pack and a special ‘arcade mode’.
Crayola Color Blaster was originally launched on November 1st 2016 and was created as part of the Google App Incubator Project. It is specifically designed for Tango-enabled Android smartphones. Tango is an AR platform for Android phones and tablets to support AR applications and games.
The game is essentially a virtual colouring book that uses AR to map characters onto the real world for children to colour in to progress through levels. The new storyline content that will be introduced alongside the re-launch is called Paint My Dragon, which includes fairies, tree monsters, ogres and, as you would expect, dragons.
The arcade mode is also a new addition, a faster-paced game mode featuring zombies, which can be enjoyed for free. The Paint My Dragon story pack is available as an in-app purchase.
“We are thrilled to have worked with Crayola and Google in the creation of this innovative children’s app,” says Ariella Lehrer, CEO of Legacy Games. “Crayola Color Blaster is one of the first kid’s titles to demonstrate how Tango’s technology enables new and exciting play patterns. Our game is a mixture of coloring and tag…only possible when your mobile device understands its position relative to the world around it.”
“At Crayola, we’re all about inspiring creativity through color and innovative play patterns, and we are delighted to partner with Legacy on the new and exciting Tango platform,” added Warren Shorr, Vice President of Business Development & Licensing. “This new app is a fun, relevant and highly engaging activity for kids of all ages.”
VRFocus will continue to bring you news on Augmented Reality products.
One of the museum partners is the Tank Museum at Bovington, found in the in the South West England county of Dorset. As part of the recent Tankfest 2017 event the Special Projects Division revealed a brand new experience featuring World of Tanks’ ultra-rare model the German-made Sturmtiger or ‘Assault Tiger’, part of the series of assault vehicles built on the design platform of the devastating Tiger I tank. Wargaming’s team, in collaboration with Gravity Jack, this time utilised support for both Microsoft’s Hololens head-mounted display (HMD) and Google’s Tango platform. Having the Sturmtiger roll straight into the museum before showcasing its movement, firing and gives both an ‘exploded’ and cross section view of what makes up the tank.
Augmented Reality has played a huge role at the developer conferences for Microsoft, Apple, Facebook, and Google, which is a great sign that the industry is moving towards spatially-aware computing. Microsoft is the only company to start with head-mounted AR with the HoloLens while the other three companies are starting with phone-based AR. They are using machine learning with the phone camera to do six degree-of-freedom tracking, but 
