Apple CEO on AR Headsets: ‘We don’t want to be first, we want to be the best’

Apple CEO Tim Cook thinks augmented reality is going to be something big, a fairly simple claim to make thanks to the company’s recent inclusion of its AR tech into every Apple device capable of updating to iOS 11. But when we talk about AR, the natural terminus isn’t the smartphone’s screen, but rather a perfectly immersive pair of AR glasses that have yet to come. Speaking to The Independent, Cook says the technology to create an AR headset in a “quality way” just isn’t possible yet, but when it is, Apple will be the best.

“The products themselves have to have a lot of processing power, and a fair amount of different sensor technology in order to do this locational stuff,” Cook tells The Independent. “So having it on iPhone changes the game for developers, because instantly they had hundreds of millions of potential customers. If it were on a different device then you would never have a commercial opportunity, and without the commercial opportunity you’d never have 15 million people that say, ‘I want to design my passion with AR’.”

image courtesy Apple

As a ready-made launchpad for AR interactions with a built-in audience, the iPhone and iPad are great, but what about AR headsets? Cook is doubtful in the short-term, saying the technology itself “doesn’t exist to do that in a quality way.”

Despite some telling patents made by the company, Cook says consumers shouldn’t really expect an Apple AR headset in the short-term in spite of the growing number of AR headsets already coming to market.

“The display technology required, as well as putting enough stuff around your face – there’s huge challenges with that. The field of view, the quality of the display itself, it’s not there yet,” he says. “We don’t give a rat’s about being first, we want to be the best, and give people a great experience. But now anything you would see on the market any time soon would not be something any of us would be satisfied with. Nor do I think the vast majority of people would be satisfied.”

“Most technology challenges can be solved, but it’s a matter of how long,” he says.

The post Apple CEO on AR Headsets: ‘We don’t want to be first, we want to be the best’ appeared first on Road to VR.

Google’s New Pixel Phones Are “factory calibrated and optimized for AR”, Now With 60 FPS Tracking

Google’s event today saw the unveiling of its newest smartphones, Pixel 2 and Pixel 2 XL. According to the company, the Pixel 2 camera has been “factory calibrated and optimized for AR.”

Google says its specially calibrated Pixel 2 camera enables robust tracking, even in low-light conditions, allowing the phone to track the environment and render AR objects at 60 frames per second (FPS).

Apple made a similar claim when they unveiled the iPhone 8 last month, saying that each camera is “individually calibrated, with new gyroscopes and accelerometers for accurate motion tracking.” The new iPhone line also boasts 60 FPS tracking.

image courtesy Google

Featuring a single 12.2MP camera with f/1.8 aperture lens, Google’s Pixel 2 nearly matches the same portion of the spec sheet as iPhone 8’s 12MP camera with f/1.8 aperture lens, putting the two on par in terms of AR rendering ability, at least on paper.

Google’s AR function, thanks to the ARCore SDK, is said to support “millions of devices” capable of running Android 7.0 Nougat and above. Apple, with its ARKit SDK, has also said they’ve pushed AR to “hundreds of millions of iPhones and iPads” capable of running its newest operating system iOS 11, which includes pretty much everything back to the iPhone 5s and even the 6th generation iPod Touch.

Pixel 2 is priced at $649 (64 GB) and $749 (128 GB), with its XL-sized bigger brother selling for $849 (64 GB) and $949 (128 GB). Pixel 2 is slated to launch October 19th, and Pixel 2XL on November 15th.

Only user reports will be able tell just how Google’s flagship Pixel 2 stacks up with Apple’s iPhone 8 as it takes the fight to the realms of augmented reality.

Pixel 2 Specs

Spec Category Pixel 2 Pixel 2 XL
Display Cinematic 5.0 inch display / Cinematic 127 mm display
FHD (1920 x 1080) AMOLED at 441ppi
16:9
2.5D Corning® Gorilla® Glass 5
Always-on display
Fullscreen 6 inch displayQHD+ (2880 x 1440) pOLED at 538ppi18:93D Corning® Gorilla® Glass 5 Always-on display
Size and Weight 5.7 x 2.7 x 0.3 inches
145.7 x 69.7 x 7.8 mm
143 gm
6.2 x 3.0 x 0.3 inches
157.9 x 76.7 x 7.9 mm
175 gm
Battery 2700 mAh battery

Up to 7 hours of go with 15 minutes of charge

*Approximate battery life based on a mix of talk, data, and standby use with always on display off. Requires use of included charger. An active display or data usage will decrease battery life. Actual results may vary, see website for details.

3520 mAh battery

Up to 7 hours of go with 15 minutes of charge

*Approximate battery life based on a mix of talk, data, and standby use with always on display off. Requires use of included charger. An active display or data usage will decrease battery life. Actual results may vary, see website for details.

Design Aluminum unibody with hybrid coating IP67 water and dust resistant

**Pixel has a water protection rating of IP67 under IEC standard 60529. Charger and accessories are not water resistant.

Memory 4GB LPDDR4x RAM
Display Characteristics 95% DCI-P3 coverage100,000:1, super contrast ratio True black levelFull 24-bits depth or 16.77 million colors 100% DCI-P3 coverage
100,000:1, super contrast ratio
True black level
Full 24-bits depth or 16.77 million colors
Storage 64 or 128GB
Processors Qualcomm® Snapdragon™ 8352.35Ghz + 1.9Ghz, 64Bit Octa-Core Adreno 540Security module
Rear Camera 12.2MP
1.4μm
Autofocus with laser + dual pixel phase detection
Optical + electronic image stabilization
f/1.8 aperture
Front Camera 8MP
1.4μm
f/2.4 aperture
Fixed focus
Video Rear Camera:
1080p @ 30fps, 60fps, 120fps
720p @ 30fps, 60fps, 240fps
4K @ 30fpsFront Camera:
1080p @ 30fps
720p @ 30fps
480p @ 30fps
Sensors Active Edge™ Proximity / Ambient light sensor Accelerometer / Gyrometer MagnetometerPixel Imprint – Back-mounted fingerprint sensor for fast unlockingBarometer Hall effect sensor Android Sensor Hub Advanced x-axis haptics for sharper / defined response
Charging USB Type-C™ 18W adaptor with USB-PD 2.0 18W charging
Ports and Slots USB Type-C™ USB
3.1 Gen 1
Single Nano SIM
Media Stereo front-firing speakers3 Mics Noise suppression
Connectivity Wi-Fi 2.4G + 5GHz 802.11 a/b/g/n/ac 2×2 MIMOBluetooth 5.0 + LENFC eSIMGoogle Cast
In box [GLOBAL]
USB Type-C™ 18W adaptor with USB-PD
C-C cable (USB 2.0)
SIM tool
Quick Switch Adapter
3.5mm to USB-C Headphone Adapter (aka Headphone Adapter)
OS Android Oreo
Hearing Aid Compatibility M4/T3 HAC rating
Google’s devices meet the hearing aid compatibility (HAC) requirements set by the Federal Communications Commission (FCC).
Security Updates Minimum 3 years of security updates**[US and GLOBAL] Pixel’s software and security updates for three years. See website for details.
OS Updates Minimum 3 years of OS updates**[US and GLOBAL] Pixel’s software and security updates for three years. See website for details.
Daydream Ready Daydream-ready: Built for VR to work with Google Daydream View headset.

The post Google’s New Pixel Phones Are “factory calibrated and optimized for AR”, Now With 60 FPS Tracking appeared first on Road to VR.

‘Pigeon Panic’ Proves That AR is Only Going to Make the World a Stranger Place

As first to market with a robust augmented reality function baked right into its mobile operating system, the various projects we’ve seen using Apple’s smartphone-based AR platform have tended to range from the useful, to the artful, to the downright silly. Targeting the ‘downright silly’ category is Pigeon Panic, a deliciously strange game from AR studio Combo that lets you chase virtual pigeons through the park, making everyone around you wonder what the hell you’re doing with your life.

To play, all you have to do is pick a spot on the ground, chuck out a mass of hamburgers, donuts and pizza, and watch as the dopey virtual pigeons arrive for the feast. Waiting for just the right moment, the objective is to scare off as many of skyrats as possible before the timer hits zero.

The game, while not exactly high on replay value, demonstrates AR’s unique ability to immerse you in an invisible digital world that really leaves people wondering why you’re having so much fun running around the park. While it doesn’t offer the sort of game mechanics you’d stick around for like Pokemon Go, it certainly gives you a taster for what’s in store for the future of AR games.

Sam Piggott, developer on Pigeon Panic and co-founder of Combo, recalls the moment when the idea was born. “Pretty much the day after the announcement at WWDC, we were sat in an East London coffee shop, coming up with a concept for the new game; something fairly simple to grasp in concept, but dependent on AR for execution. There were a bunch of pigeons outside pecking at some bread outside the coffee shop window, and one of us was like, ‘what if….?’”.

“The whole experience was designed to be quite silly,” says Piggott. “We wanted to pull together an experience that let anyone relive that juvenile feeling of scaring off flocks of birds – without the crushing fear of judgement from onlookers.”

“In retrospect, it’s probably equally as strange to be seen running around open spaces with an iDevice, though”, he added.

Pigeon Panic is free, and currently available for download on the App Store for most devices that can run iOS 11.

The post ‘Pigeon Panic’ Proves That AR is Only Going to Make the World a Stranger Place appeared first on Road to VR.

Apple is Bringing AR to “Hundreds of Millions” of iPhones and iPads Starting September 19th

Apple’s iOS 11 is coming to compatible devices starting September 19th, which means that if you own a recent Apple iPhone or iPad, you may have an augmented reality-capable device in your hands before the new iPhone line even launches.

Apple’s big iPhone 8/8 Plus/X unveiling this year promised a bevy of information surrounding augmented reality, thanks to the release earlier this summer of ARKit. As a tool that lets developers make AR games and apps on what Apple says will amount to “hundred of millions of iPhones and iPads,” we had our hopes pretty high for a slew of app announcements.

While we only saw four AR apps revealed on stage demonstrating the phone’s AR capabilities, Apple has said in the past that they’re working with Pokemon GO creators Niantic, IKEA, and Lego to name a few to bring AR apps to the App Store. To that end, starting this month any iPhone, iPad or iPod that can upgrade to iOS 11 will be able to get in on the action, which the company says will let you do things like “redecorate your home, explore a city you’ve never visited, or even try on a new tattoo.”

image courtesy Apple

Apple is advertising the new iPhone line as custom designed “for the ultimate augmented reality experience,” featuring specially calibrated cameras, a screen low on bezels, and the new A11 Bionic processor that drives the room and face-mapping power of the new devices. That may not be enough for many to take the $1000 iPhone X upgrade, but if you’re looking for the most capable AR-capable phone out there, you can bet it’s going to be an Apple product until other manufacturers get in the game.

Google also recently released an AR developer kit, ARCore, which aims to give similar AR abilities to “100 million [Android] devices at the end of preview.” Google is working with Samsung, Huawei, LG, ASUS and unnamed others to accomplish it, making AR the next battle ground for the competing brands.

The post Apple is Bringing AR to “Hundreds of Millions” of iPhones and iPads Starting September 19th appeared first on Road to VR.

Apple Reveals Short List of AR Apps Coming to App Store This Month

With ARKit in the wild for a few months now, Apple’s special iPhone 8 event was the place to announce the coming wave of augmented reality apps. Senior VP of worldwide marketing Phil Schiller took the stage to introduce a few apps coming to the App Store later this month, which left us wanting more.

Calling augmented ‘the smartphone camera’s third category’ behind its ability to take video and photos, Schiller only introduced four AR apps—less than we’d hoped for, but pretty cool none the less.

There’s likely plenty more coming, as the company has already revealed that they’re working with IKEA, Lego, and a number of others to bring AR apps and games to the App Store—not to mention the slew of creators developing AR apps on the “hundred of millions of devices” that can already run iOS 11. Critically missing was anything from Niantic, the creators behind breakout success Pokemon Go.

The Machines

image courtesy Apple

Coming from Directive Games was the star of the show, The Machines. Warranting an on-stage demo, the competitive multiplayer RTS lets you duke it out with your friends in the same room in augmented reality. Using two Apple 8s, the developers played a quick game to show off the little tabletop game.

Warhammer 40k: Freeblade

image courtesy Apple

Much like a true AR-version of Pokemon Go’s ‘AR mode’, Pixel Toys’ Warhammer 40k: Free Blade lets you activate a special AR camera mode to bring characters from the game into the physical world and play it right where you’re standing.

Major League Baseball – At Bat

image courtesy Apple

So you’re already watching a game a baseball game on TV, but with MLB’s At Bat app, you’ll soon be able to see real-time player info and stats overlayed on the game you’re watching.

Sky Guide

image courtesy Apple

One of the coolest app updates we’ve seen is Sky Guide’s new AR function, which lets you quickly learn the constellations by simply gazing up into the sky and looking at your phone.


This story is breaking. We’ll fill in more info as it comes in.

The post Apple Reveals Short List of AR Apps Coming to App Store This Month appeared first on Road to VR.

Devs Use ARKit to Make an iPhone Motion Controller for HoloLens

Ok, these ARKit experiments and mashups are getting darn cool. Here VR development studio Emergent linked together HoloLens and an iPhone with ARKit to turn the iPhone into a 6DOF motion tracked controller.

The tracking may be somewhat rudimentary, but it’s becoming increasingly clear the many possibilities that come with smartphones that are able to understand their position in the world. From a fusion of AR and VR to a portal into volumetric video, we’ve seen some really cool ARKit experiments, but none that have used an iPhone as a motion controller instead of a display.

Emergent has demonstrated the possibility by turning an iPhone into a motion controller for a HoloLens game. In the game the HoloLens is represented as the player’s helmet, while the iPhone is a sci-fi laser gun that’s tracked in all dimensions. Taking things a step further, the studio demonstrates how bullet laser holes can be convincingly overlaid onto objects in the environment, thanks to the devices’ ability to map the geometry of the environment.

It’s clear from the video that there’s a good deal of latency and that the precision isn’t particularly high (after all, ARKit isn’t intended for immersive experiences), and keeping the tracking systems of both HoloLens and ARKit in sync over distance and time could post a challenge. But the potential for the use-case is clear—especially as smartphones begin to hit the mainstream with more precise motion tracking instruments—a well tracked smartphone can be a flexible and widespread tool for AR and VR alike.

The post Devs Use ARKit to Make an iPhone Motion Controller for HoloLens appeared first on Road to VR.

ManoMotion Brings Hand Gesture Input to Apple’s ARKit

ManoMotion, a computer-vision and machine learning company, today announced they’re integrated their company’s smartphone-based gesture control with Apple’s augmented reality developer tool ARKit, making it possible to bring basic hand-tracking into AR with only the use of the smartphone’s onboard processors and camera.

With Google and Apple gearing up for the augmented reality revolution with their respective software developer kits, ARCore and ARKit, developers are fervently looking to see just how far smartphone-based AR can really go. We’ve seen plenty of new usecases for both, including inside-out positional tracking for mobile VR headsets and some pretty mind-blowing experiments too, but this is the first we’ve seen hand-tracking integrated into either AR platform.

image courtesy ManoMotion

Venture Beat got an early look at the company’s gesture input capabilities before they integrated support for ARKit, with ManoMotion CEO Daniel Carlman telling them it tracked “many of the 27 degrees of freedom (DOF) of motion in a hand.” Just like their previous build, the new ARKit-integrated SDK can track depth and recognize familiar gestures like swipes, clicking, tapping, grab, and release—all with what ManoMotion calls “an extremely small footprint on CPUs, memory, and battery consumption.”

In ManoMotion’s video, we can see the ARKit-driven app recognize the user’s hand and respond to a flicking motion, which sends a ping-pong ball into a cup, replete with all of the spatial mapping abilities of ARKit.

A simple game like beerpong may seem like a fairly banal usecase, but being able to interact with the digital realm with your own two hands (or in this case, one hand) has a much larger implication outside of games. AR devices like HoloLens and The Meta 2 rely upon gesture control to make UI fully interactive, which opens up a world of possibilities including productivity-related stuff like placing and resizing windows, or simply turning on Internet-connected lights in your house with the snap of the finger. While neither Google nor Apple have released word on future AR headsets, it’s these early experimental steps on the mobile platforms of today—which necessarily don’t have access to expensive custom parts—that will define the capabilities of AR headsets in the near future.

“Up until now, there has been a very painful limitation to the current state of AR technology – the inability to interact intuitively in-depth with augmented objects in 3D space,” said Carlman. “Introducing gesture control to the ARKit, and being the first in the market to show proof of this, for that matter, is a tremendous milestone for us. We’re eager to see how developers create and potentially redefine interaction in Augmented Reality!”

ManoMotion says ARKit integration will be made available in the upcoming SDK build, which will be available for download “in the coming weeks” on the company’s website. The integration will initially be made available for Unity iOS, followed by Native iOS in subsequent updates.

The post ManoMotion Brings Hand Gesture Input to Apple’s ARKit appeared first on Road to VR.

ARKit Project Shows Turn-by-Turn Directions in AR, Giving a Glimpse into Future of AR-enabled Maps

What’s that building in the distance? Where’s the nearest Starbucks? How do I get to there from here? These are questions you’d likely find answers to in your smartphone’s map application—but with Apple’s ARKit, one developer showed it could be even easier with a recently revealed project that demonstrates the practical application of turn-by-turn directions and basic landmark-based orientation displayed in augmented reality.

Created by London-based iOS developer Andrew Hart, the project was built using two Apple tools; ARKit and Core Location, the latter of which provides devs access to data from the phone’s onboard hardware including Wi-Fi, GPS, Bluetooth, magnetometer, barometer, and cellular hardware so it can be used to gather relevant positioning data. Integrated it into a map project, Hart was able to demonstrate landmark orientation and even turn-by-turn directions that appear with the click of a single button.

Hart says he’ll be publishing the project on GitHub soon, and that he’d like to make sure the experimental app is in “a solid place before opening it up.”

Apple ARKit was first revealed at the company’s annual Worldwide Developer Conference (WWDC), and works on newer iPhone and iPads that have the ability to opt into the iOS 11 beta. While Apple is working with third-parties such as IKEA, Lego, and Niantic to build apps using ARKit, the company also released the tool for general use so enterprising developers can start building AR apps today—many of them going viral on the Internet like AR inter-dimensional portals and a project that built positional tracking for a VR headset using ARKit.

There’s no telling what Apple has up its sleeves hardware-wise, whether it’s simply a new iPhone with ARKit integrated at its core, or an AR headset that some patents would suggest is currently in the works. In the meantime though, we’ll be casting wistful stares at the iOS developers of the world to see what they come up with next.

The post ARKit Project Shows Turn-by-Turn Directions in AR, Giving a Glimpse into Future of AR-enabled Maps appeared first on Road to VR.

The 10 Coolest Things Being Built with Apple’s ARKit Right Now

Apple’s ARKit has been out for developers since it was announced last month. Because of the mind-boggling number of iPhones and iPads capable of running ARKit, the new tool is already spreading its augmented reality wings across the Internet thanks to a diverse set of developers. Here we take a look at some of the coolest publicly-shared AR projects being built right now using ARKit.

Keep in mind: because ARKit essentially gives the device the ability to accurately map your surroundings, something made possible by simultaneous localization and mapping (SLAM), there’s no need for external equipment to run these sorts of AR experiences. Apple, with a single flip of the wrist, has however started down the path of farming AR applications for whatever device it has planned next.

10 – Robot Dancing

Ok, so a dancing robot isn’t the most innovative application we’ve seen, but being able to see a 3D figure do the boogie in your living room has its appeal, and the dynamic lighting only makes it that much cooler. Dancing to Vengaboys (the song from the Six Flags commercial), this happy little robot was created by Tomas Garcia, also known for many other cool ARKit experiments.

9 – Room Measurement

Created by Smart Picture 3D, their Measurement Cloud enables what they call “dimensioning intelligence for the interior world.” There’s no denying the utility of pulling out your smartphone to see if that IKEA dresser is going to fit in your office (of course leaving enough space for VR).

8 – Inter-dimensional Portal

Fancy a walk into another dimension? This inter-dimensional portal built by French consulting agency Nedd shows you just how extraordinary a lonely allée can be when you have the ability to see another world at a moment’s notice.

7 – Volumetric Video in AR

Using 3D animations from 4DViews, China-based developer 应高选 created this with his iPhone 6s and Unity. 4DViews is known for creating dynamic, volumetric captures for VR/AR/MR market.

6 – Portals in Play

Showing some gaming applications to the always cool portal-trick, Helsinki-based creative developer lingoded is by his own admission “searching [for] the next big thing” as he works with ARKit and Unity. Look inside the portal to find a key to unlock another door: classic gaming brought to life.

SEE ALSO
'Budget Cuts' Uses Portals to Solve Room-Scale Locomotion Problem

5 – Dance Practice

You don’t know how to dance. I don’t know how to dance. But with AR, we’ll all be dancing in perfect step with a modern take on the old-timey footstep diagrams. This little two-step number was created by Dance Reality App, and projects a moving diagram on the floor so you can learn all the right moves.

4 – Apartment Walk-through

Virtual apartment walk-throughs have been in VR for a while now, but with the limited room-scale space and relatively expensive kit required to display (not excluding the difficulty of teaching someone how to move in VR for the first time), Stockholm-based firm 3D Interactive Sthlm brought their visualization technique into AR, allowing you the ease of movement you’ve been used to since you started walking.

3 – Positional Tracking for VR

Nexus Interactive Arts, an immersive media division of VFX production studio Nexus Studios, have used Apple’s ARKit working on an iPhone 7 in an experiment that creates basic inside-out positional tracking and pass-through AR for a Google Cardboard headset.

2 – Minecraft in AR

Minecraft is already available in VR, so it was only a matter of time until it came to AR too. Created by Matthew Hallberg with an iPhone and Unity 3D, he says “I love that you are able to place life size objects because the tracking with ARkit is so good.” Stay tuned to Hallberg’s channel for upcoming tutorials and a breakdown of how he created Minecraft for AR.

1 – Real-time Mixed Reality with Vive

Built in Unity by Normal VR, the mashup between the HTC Vive-driven art program and the ARKit’s ‘window’ into the virtual world is truly inspiring—almost as if it were giving us a peek into the future of mixed reality when AR/VR unite in a single hardware platform.

SEE ALSO
Video Gives Us a Peek Into the Future of How AR and VR Will Work Together

What’s your favorite ARKit project? Show us some cool ones in the comment section!

The post The 10 Coolest Things Being Built with Apple’s ARKit Right Now appeared first on Road to VR.

Using Apple’s ARKit, This Company Built Inside-Out Tracking for a VR Headset

Nexus Interactive Arts, an immersive media division of VFX production studio Nexus Studios, have used Apple’s ARKit working on an iPhone 7 in an experiment that creates basic inside-out positional tracking and pass-through AR for a Google Cardboard headset.

Announced last month at Apple’s annual Worldwide Developer Conference (WWDC), ARKit is an iOS 11 tool allowing developers to create AR applications thanks to the device’s computer vision capabilities. With ARKit, iOS 11 devices are able to map surfaces in real time, and allow users to superimpose digital objects onto the physical world—replete with interactive animations and dynamic lighting.

Using ARKit, the team reports their inside-out positional tracking solution clocks in “at around 60 frames per second,” or right around mobile VR’s current target framerate. This, according to the team, means Apple has created the foundations for a cheap, but still ultimately reliable positional tracking solution for mobile VR headsets.

In the video, they demonstrate inside-out positional tracking for VR and pass-through AR by touring a conceptual ‘art museum’ in a park. When in VR, walking close to a boundary like a tree results in a point cloud materializing into the otherwise closed-off experience—essentially acting as a guardian system to keep you from bumping into things as explore the infinite (or sufficiently large) tracking volume afforded by the device’s machine vision. In the AR demonstration, the digital skybox is lifted to reveal digital scenery affixed to the park’s trees and landscape.

The AR headset capabilities presented in the video, while an impressive use of ARKit, are less useful in this case because of the lack of stereoscopic vision afforded by the iPhone 7’s monoscopic rear-mounted camera. The developers aren’t couching this as a verified AR headset solution however, but rather showing the versatility of ARKit itself.

Allowing developers free reign to create applications for AR—and thanks to this experiment, now free-roaming VR experiences currently puts Apple back into competition despite its lack of discrete AR/VR headsets.

The post Using Apple’s ARKit, This Company Built Inside-Out Tracking for a VR Headset appeared first on Road to VR.