The mantra goes: If it’s possible, someone will eventually build it. That statement rings true for everything from the weird and spastic games like Mosh Pit Simulator or hardware add-ons like the pulsating haptic skin from Omnipulse, and while I’m just not sure why ’80s Norwegian synthpop band A-ha really needed to be realized in AR, I feel so much better about the world now that I’ve seen it.
Created by Chip Sineni of Trixi Studios, the augmented reality app was built using Apple’s ARKit, a developer tool recently released by the company running on the iOS 11 beta that lets you create AR apps and games. Because of the wide-spread availability of Apple devices supporting the iOS 11 beta, the company claims millions of iPhones and iPads are already in the hands of prospective developers around the world. So it’s no wonder when weird and wild things come creeping out of the woodwork.
Reflecting what happens in A-ha’s “Take On Me” music video, a man beckons to you from the inside of a comic book, which then opens to the iconic hand-drawn world parodied in popculture ever since the music video’s release in 1984.
Like the inter-dimensional portals we’ve seen built using ARKit, the app lets you walk through to the ‘other side’ and experience the world just like in the music video.
In case you haven’t experienced the glory of ’80s synthpop first-hand:
Created by Helios Interactive, Echelon is a multi-player board game played in Hololens mixed-reality that, as an proof of concept, gives us a look into what sort of games we might be playing in the future when we’re not engaged in the many augmented reality productivity-based activities like navigating with turn-by-turn directions or skyping with a work colleague.
Using cards to spawn creatures, the game’s developers say Echelon builds on gameplay mechanics as seen in many traditional physical board games but “augments” them with mixed reality creatures and game pieces, enabling the players to experience an immersive augmented reality gaming experience.
Helios Interactive Producer Devin Fuller Knight says the project highlights “the fact that you can actually interact with another player in a HoloLens and [while] being in a HoloLens is really fun, it’s such a solo experience. The idea of actually being able to interact with another player and both being able to see the same world exist in the same [space] is really fun.”
“When I saw the HoloChess game in Star Wars,” says Helios developer Kristafer Vale, “I immediately wanted to play that game, and I had to wait until now for us to have the technology like Unity and HoloLens to be able to realize that dream.”
While HoloLens presents the user with a fairly limited field of view (as seen above), a near-field object like a game board plays to the headset’s strengths. Still costing $3000 however is a big barrier of entry for prospective AR developers, which is precisely why so many people are experimenting with Apple’s ARKit.
Nexus Interactive Arts, an immersive media division of VFX production studio Nexus Studios, have used Apple’s ARKit working on an iPhone 7 in an experiment that creates basic inside-out positional tracking and pass-through AR for a Google Cardboard headset.
Announced last month at Apple’s annual Worldwide Developer Conference (WWDC), ARKit is an iOS 11 tool allowing developers to create AR applications thanks to the device’s computer vision capabilities. With ARKit, iOS 11 devices are able to map surfaces in real time, and allow users to superimpose digital objects onto the physical world—replete with interactive animations and dynamic lighting.
Using ARKit, the team reports their inside-out positional tracking solution clocks in “at around 60 frames per second,” or right around mobile VR’s current target framerate. This, according to the team, means Apple has created the foundations for a cheap, but still ultimately reliable positional tracking solution for mobile VR headsets.
In the video, they demonstrate inside-out positional tracking for VR and pass-through AR by touring a conceptual ‘art museum’ in a park. When in VR, walking close to a boundary like a tree results in a point cloud materializing into the otherwise closed-off experience—essentially acting as a guardian system to keep you from bumping into things as explore the infinite (or sufficiently large) tracking volume afforded by the device’s machine vision. In the AR demonstration, the digital skybox is lifted to reveal digital scenery affixed to the park’s trees and landscape.
The AR headset capabilities presented in the video, while an impressive use of ARKit, are less useful in this case because of the lack of stereoscopic vision afforded by the iPhone 7’s monoscopic rear-mounted camera. The developers aren’t couching this as a verified AR headset solution however, but rather showing the versatility of ARKit itself.
Allowing developers free reign to create applications for AR—and thanks to this experiment, now free-roaming VR experiences currently puts Apple back into competition despite its lack of discrete AR/VR headsets.
We’re all waiting for the day when you can put on a singular, tether-free headset and experience all the ‘R’s that VR/AR/MR have to offer—and NormalVR, a “small but focused” group of passionate immersive media creators just gave us a peek into what’s possible when you mash it all together.
Known recently for their whimsical open source keyboard (and magnificently weird blobby-guy), Normal is a group of remotely-located developers that are using their own technology to make developing from separate locations an easier experience. By their own admission, they aren’t entirely sure what they’re doing, but it’s clear from the video that surfaced yesterday that they’re hitting on some very big ideas and executing them with serious flair.
In a recent tweet, the group shows what at first appears to be another fun video demonstrating the potential of creating apps with Apple’s recently released ARkit. But this is more than just a dancing hotdog-guy. Zooming out, we see the blobby-guy avatar is in fact controlled by a person using an HTC Vive standing just out of frame, creating a digital sunflower with some species of art program like Tilt Brush (2016).
Why is this so important? Normal is building this on readily available, cost-effective hardware and it seems to work perfectly.
As a launchpad for universal Windows apps that are designed to work across HoloLens and the company’s wallet-friendly fleet of VR headsets, Microsoft’s Windows Mixed Reality Platform is taking the first baby steps into making sure there’s a strong base of apps in the AR/VR shared ecosystem. More important to the scope of the article, Microsoft also spearheaded a neat way to turn your DSLR into an AR capture device so you can see what goes on in the digital realm, but this requires you to buy an extra HoloLens, which at $3000 is a pretty steep price to pay.
By picking up an iPhone and a VR headset like the Oculus Rift, which now costs even lower than ever at $400, developers can start building the future of games and apps for all immersive platforms—hopefully coming sooner rather than later.
Funomena’s Woorld won the ‘Best AR Experience’ category at the recent 2017 Google Play awards. In the game you scan your room with a Google Tango-enabled phone, and then you’re encouraged to decorate your space with extremely cute art and characters designed by Katamari’sKeita Takahashi. Part of the gameplay in Woorld is to figure out how to combine different objects together in order to unlock new objects and portions of the story in your space.
LISTEN TO THE VOICES OF VR PODCAST
Funomena had to innovate on a lot of augmented reality user interaction paradigms and spatial gameplay in designing this game. I had a chance to catch up with Funomena co-founder and CEO Robin Hunicke at Google I/O to talk about her game design process, as well as her deeper intention of bringing sacredness, mindfulness, calmness, worship, spirituality, love, empathy, and kindness into your environment through augmented reality technology. She takes a lot of inspiration from Jodorowsky’s The Technopriests as well as the sci-fi novel Lady of Mazes by Karl Schroeder.
Hunicke also sees that there’s a split that’s emerging between the commercial VR and the indie VR scene with the character of content that’s being funded, and she talks the importance of supporting indie game creators.
This year’s 2017 Google Play Awards include new categories for top VR and AR apps, to which Google has nominated five apps each. The award ceremony will take place on Thursday, May 18th at 6:30PM PT during Google I/O, the company’s annual developer festival.
Among 12 categories, the company had added two new ‘Best Experience’ categories for AR and VR to the Google Play Awards 2017.
Nominees were selected much like last year by cross-functional teams throughout Google who work hand-in-hand with the relevant categories and product areas. While category specific criteria can be found below, the common requirements across all categories focused on high star rating, technical performance and freshness, requiring a launch or major update since April 2016.
Augmented reality is conceptually so broad, there’s almost no limit to what you could imagine doing with the technology. But rather than throwing out the playbook entirely, we’ll to our grand visions through useful incremental steps that begin with integration into existing content built around today’s paradigms. A concept video using HoloLens and Final Fantasy XIV, shows how AR could be incredibly useful as a ‘Second Screen’ companion for gaming and more.
While one day we’ll want AR games that are entirely based in augmented reality, today’s reality is that much of the world already has high-end, high resolution displays in their home, and powerful computers or consoles already hooked up to them to play all manner of games and content. Making use of those screens—especially while AR display technology and content design is still advancing—could be a great stopgap to a fully augmented futured.
The ‘Second Screen’ idea—putting additional content on a mobile device that’s relevant to what’s playing on the main screen—has been around since smartphone usage has become mainstream in the last decade. We’ve seen it attempted with games and film/TV content in the last several years, but it hasn’t quite caught on as something truly useful. AR, however, could change that.
Tanufuku has reserved the laptop screen entirely for real-time 3D action, while exploding the game’s dizzying array of interface windows to the surrounding area with AR via HoloLens.
Now one might ask ‘Well then, why not just play the game on an AR screen too?’ Great question. The reason is that AR display technology is still advancing, and the laptop’s screen is almost certainly the superior display option for the time being.
For instance, while the video makes it seem like you’d be able to see all the AR windows in your periphery while looking at the main screen, the actual HoloLens field of view is around 35 degrees diagonally (making its available viewing area much smaller than most static displays that people use in their home today) which means if you replaced the laptop screen with an AR screen, you’d get clipping if you turned your head even slightly (and the AR screen would disappear completely from your field of view if you went to look at the surrounding AR windows).
Keeping the most important parts of the game on a real screen means the less important info (occasionally-access parts of the interface) will clip out of the field of view while you focus on the most important part of the game, but when you go to look at the AR windows, you’ll still be able to see that important real-time game action on the screen in your peripheral view.
Microsoft’s HoloLens AR headset
Not to mention the fact that many users will have monitors or TV’s that far exceed the effective resolution of AR displays for some time, and, if you are running a contemporary game, there’s no reason to use a transparent display (which would degrade the quality of the content due to interference with the real world in the background).
So, leaving the most important part of the content (the real-time action) on a real screen—since they’re inexpensive, high-resolution, and we already own them!—while giving the less important parts of the game (like maps, inventory, menus, scoreboards, and more) some breathing room via augmented reality makes a lot sense for AR’s use as a ‘Second Screen’. Especially with some smart transitions between the real screen and the augmented space; you can imagine how, with some work, a game could allow users to seamlessly drag windows ‘out of their laptop’ and into augmented reality.
Funomena’s 
