Former Felix & Paul chief content officer Ryan Horrigan and former Milk VR and VRSE.works producer Armando Kirwi have banded together to found Artie, a new startup with the mission of bringing AI avatars to augmented reality.
In the video below, you can see a bulbous little avatar named ‘Moke’ responding to voice commands and doing the Moonwalk, Makarena, or YMCA dance.
But what’s especially interesting is the little assistant ‘Winnie’, which uses machine learning to recognize what you’re looking at in real-time, be it a friend smiling ear-to-ear, or a Coke can on your desk.
“With Artie’s deep-learning insights and analytics platform, creators can monitor engagement with their avatars in real-time,” the company says in a press statement. “The platform also captures user behaviors to drive neural networks that automatically train and improve an avatar’s performance over time, enabling more entertaining and lifelike interactions for each subsequent user.”
The company tells Variety their software is capable of detecting seven prime emotions based on facial expressions, which Artie says should expand to 30 emotions in the coming months. The little purple onion-guy Winnie can only detect 80 objects at the moment, but that’s also a number that’s sure to grow in the near future.
“VR and AR are the first media where you can be part of the story,” Horrigan tells Variety. “But you can’t talk to characters. You can’t have human interactions with characters. We felt this was a big problem.” And it’s a problem the company wants to address on a pretty large scale—not by just creating their own avatars and throwing them on Google Play or Apple’s App Store willy nilly.
“Right now, AR experiences are stuck in the App Store,” said Horrigan. “The distribution angle needs to change to really see meaningful discovery and shareability and thus virality.”
To address this, Artie is developing something it calls the ‘Wonderfriend Engine’, a suite of tools that will help companies build their own interactive avatars for mobile AR experiences. The company hopes one day that Hollywood will pick up on the idea of building their own AI avatars for a deeper level of brand engagement.
Artie is backed by investors in AI, AR, and entertainment, including Founders Fund (with partner Cyan Banister also investing personally), The Venture Reality Fund, WndrCo, M Ventures, Metaverse Ventures (a DCG company), YouTube Co-founder and former CEO Chad Hurley, and Ubiquity6 Co-founder and CEO Anjney Midha.The company plans on launching the platform in Q1 of 2019.
AR optics maker WaveOptics has announced a manufacturing partnership with Goertek, one of the VR industry’s leading manufacturers. The deal will enable mass production of WaveOptics’ waveguides for integration into AR headsets which the companies say could be market-ready at a $600 price point in 2019.
China-based Goertek is not a widely recognized name within the VR industry but the company has quietly become a key enabler, providing manufacturing services behind some of the most prominent VR products on the market.
Now, as Goertek looks to play a similar role in the growing AR space, the company has announced that a partnership with WaveOptics which will enable Goertek to offer mass produced AR optics, either as individual components to be used in other AR headsets, or in their own ODM products.
AR optics are often a large contributor to the cost of an AR headset, and with the partnership the companies say they will be able to enable market-ready AR headsets in 2019 at a price point of $600.
WaveOptics is a designer of diffractive waveguide optics. Waveguides can enable extremely thin optics by using structures within the lens which can redirect light in arbitrary directions. In addition to thin optics, the waveguide’s capabilities mean that the display source itself can be mounted in a way that makes an AR headset much more compact than would be possible with other approaches, like Meta 2’s fishbowl optics.
Image courtesy WaveOptics
Waveguides are seen as a promising approach to delivering truly glasses-sized AR headsets which are thought to be key to taking the technology mainstream.
WaveOptics announced that it had raised a $16 million Series B investment last year, and claims to have raised $25 million to date. In the newly announced Goertek partnership, WaveOptics stands to clearly demonstrate the claimed scalability of its approach to manufacturing affordable waveguide optics using a lithographic approach.
According to the company’s website, its current optic, the ‘Phlox 40’, offers a 40 degree diagonal field of view across a 16:9 aspect ratio. That’s in the same class as devices like HoloLens and Magic Leap, which lack a wide enough field of view to be particularly immersive. Assuming the Phlox 40 is the optic that Goertek will be offering through the partnership, it’s likely that it will initially see more use in smartglasses than true AR headsets.
WaveOptics expects to be able to expand the field of view of their offering at some point in the future, but pushing beyond 50 degrees has been a challenge for most waveguide technology to date; in the VR industry, a 90 degree field of view is considered the minimum by many for high levels of immersion, but surely even smaller fields of view can be useful and immersive if paired with the right content design.
Jaunt was once a cinematic VR company which produced high-quality 360 video, and to boot even a professional-grade 360 camera dubbed NEO. Taking a step in a decidedly more AR direction with its recently revealed volumetric video capture solution, Jaunt further announced it’s also acquired both Personify’s ‘Teleporter’ volumetric video streaming tech and the engineers behind it.
As a talent and IP-driven acquisition, the move is said in a press statement to directly support Jaunt’s volumetric R&D initiatives for its Jaunt XR Platform, a solution that lets businesses create and deliver their own branded volumetric video content like livestreamed avatars of real people, deliverable to both VR headsets and AR-capable devices like flagship Apple smartphones and tablets.
According to Venture Beat’s Dean Takahashi, who visited Jaunt’s San Mateo, California headquarters last month, the company has created a pipeline that uses six Intel RealSense depth cameras; the resultant images are then automatically stitched into a single 3D avatar and livestreamed to supported devices.
Jaunt CTO and Founder Arthur van Hoff says adding both Teleporter and the talent behind it allows them to “increase the speed and scope of our research and development as we move further into the extended reality arena with the Jaunt XR Platform at the core of our business.”
Continuing: “We’re honing in on fully immersive virtual, mixed, and augmented reality experiences, and are thrilled to advance those technologies with the help of our new Chicago-based team.”
The deal includes seven Personify engineers, who will join Jaunt’s R&D team, four pending patents developed around Personify’s Teleporter technology, and Personify’s office in Chicago. Jaunt hasn’t disclosed the acquisition price.
Jaunt’s evolution to a B2B-focused company coincided with the late-2017 announcement of their Jaunt XR platform. The company has been involved in the VR cinematic space since its founding in 2013.
Meta, makers of an AR headset dev kit, was close to raising a new round of funding before a Chinese lead investor backed out, Bloomberg reports. The Chinese investment firm had reportedly put up $20 million for the deal. Meta claims that the Chinese government urged the investment firm to reconsider the deal following trade tensions brought on by the Trump administration.
Meta, makers of the Meta 2, a tethered AR headset dev kit with a notably wide field of view, last raised $50 million in Series B funding back in mid 2016. According to Bloomberg the company had secured a lead investor for its next funding round, a Chinese private-equity firm and real estate entity which agreed to bring $20 million to the deal, with additional funding to be filled out likely in lesser increments from other investors.
But the deal hit a stumbling block when the lead investor backed out of the deal. Meta claims that the reversal was a result of trade tensions incited by the Trump administration.
“The Chinese government sent an official request to our lead investor to re-evaluate the deal based on the recent actions from the Trump administration,” Gribetz told Bloomberg. “This was a big shock to us.”
The stagnanting deal led Meta to furlough two thirds of the company’s roughly 100 employees, Bloomberg reports, suggesting the startup is short on cash.
The Meta 2 AR headset is bulky compared to other AR headsets on the market but offers a much wider field of view| Photo by Road to VR
Chinese investment in the US startup and tech space has been growing steadily, and has been a key source of capital for AR and VR companies like Magic Leap, which saw investment in both its Series C and Series D rounds from Chinese online mega-retailer Alibaba. Those rounds totaled nearly $1.3 billion, according to Crunchbase.
Meta itself counts China-based Tencent and Lenovo among its existing investors.
Over the last year the US has imposed tariffs on products from China, Mexico, Canada and the EU in an effort to make international goods less competitive in favor of domestic ones, according to BBC. President Trump says the goal of the tariffs on Chinese goods is to stop the “unfair transfers of American technology and intellectual property to China” and protect US jobs. All of the aforementioned countries, including China, have initiated retaliatory trade policies.
Meta, like any other US company, has to try to survive despite bumpy trade relations between the US and China. According to Bloomberg, the company plans to establish a Chinese subsidiary independent from its US operations which would allow the Chinese lead investor to back the company without making an investment in the US.
As the VR and AR sector goes, Meta has been around for quite a long time. Founded in 2012, the company ran a successful Kickstarter in 2013 for an early AR development headset. In the company went on to raise $73 million between its Series A and Series B investments. In 2016 Meta announced its newest AR dev kit, the Meta 2, a tethered AR headset development kit with a wide field of view, which the company currently sells for $1,500.
While Palmer Luckey provided his own Magic Leap One headset to iFixitfor their tear down, the founder of Oculus certainly had his chance for a proper review before the spudgers and screwdrivers came out, and he isn’t pulling any punches either. According to Luckey, the company’s much awaited AR headset is “a tragedy in the classical sense.”
Posting on his personal blog, Luckey is critical about the overall usability of the headset, calling it “less of a functional developer kit and more of a flashy hype vehicle that almost nobody can actually use in a meaningful way, and many of their design decisions seem to be driven by that reality.”
First the good, then the bad (as determined by Luckey).
Giving the Lightpack computing unit an “A+,” Luckey was fairly happy with the design decision to keep the guts of the computer off the user’s head, which could easily cause fatigue on neck muscles if the creators weren’t careful.
image courtesy Magic Leap
The robustness of the cables was also a positive point. While he admits Magic Leap should have made the battery replaceable, he maintains “nobody is going to use their ML1 long enough for that to matter to anyone but collectors with an aim to preserve the history of AR and VR.”
Biting. Prophetic. Very Palmer Luckey.
The “Tragic”
The headset, dubbed ‘Lightwear’, doesn’t use such a unique display system as previously touted by the company, Luckey says. As evidenced by the iFixit tear down, Lightwear uses a standard AR display technology called waveguides, which are then paired with reflective sequential-color LCOS displays and LED illumination. So Magic Leap One offers a few things more than HoloLens, but only just.
Captured by Road to VR, image courtesy Magic Leap
One of the headset’s goals was to solve the vergence-accommodation conflict, a fact of current mono-focal displays with fixed render distances. If you want to learn more about it, check out our primer here. To do this, Lightwear’s optics offers two focal planes, which doesn’t entirely solve the issue, Luckey says. “Mismatch occurs at all other depths. In much the same way, a broken clock displays the correct time twice a day.”
Tracking, image quality, and environmental meshing, or scanning the room for 3D geometry, are all acceptable, he says.
“Have you seen Hololens? Think that, but with slightly larger FOV. The rainbow artifacts are a bit worse owing to the large number of stacked waveguides and the black levels are a bit better, but Magic Leap is playing in the ballpark as everyone else. Despite drawing enough power to keep the headset nice and toasty (seriously, it is hard to touch the magnesium shell if you are in a warm room), the display is far too dim to use outdoors. That is a shame, since the transparency is about the same as a pair of dark sunglasses – not exactly indoor material. How does the eye-tracking work? Impossible to say, because nothing uses it. That is not a great indicator.”
Luckey mentions the system’s controller, dubbed ‘Control’, is plainly “bad,” and that it suffers from interference from ferrous materials like those often found in industrial environments.
Image courtesy FCC
The lack of click-able touchpad is also a sore spot for Luckey, as he raises doubts about precision UI-selection due to having to either tap the touchpad or use the trigger for selection, two movements that can cause you to miss your intended target. Counterbalances to Control’s magnetic transmitter, Luckey maintains, help give it a premium feel, but isn’t suited for long-term use due to the weightiness of the controller.
As for the operating system, called LuminOS, Luckey posits it “is actually just Android with custom stuff on top, the same approach most people take when they want to claim they have built a whole operating system.” The UI is dominated by flat windows, something he calls “some of the worst parts of phone UI slammed into some of the most gimmicky parts of VR UI, and I hope developers create better stuff in the near future.”
Dev Kit or Expensive Prototype?
According to his own data gathering, Luckey estimates Magic Leap only sold around 2,000 units in the first week of sales, with “well under 3,000 units” as of this writing.
If true, this doesn’t bode well for the platform’s developers, as hardware adoption is somewhat of a self-fulfilling prophesy; if a company doesn’t motivate enough developers to create apps for a device, very few apps will exist and no one will buy the headset, meaning less developers are interested in the future and the platform essentially peters out as developers move on to more lucrative projects.
“Magic Leap needed to really blow people away to justify the last few years. The product they put out is reasonably solid, but is nowhere close to what they had hyped up, and has several flaws that prevent it from becoming a broadly useful tool for development of AR applications. That is not good for the XR industry. It is slightly better than Hololens in some ways, slightly worse in others, and generally a small step past what was state of the art three years ago – this is more Hololens 1.1 than Consumer AR 1.0. Consumer AR can’t happen without advancement, and it seems those advancements will be coming from other companies. There is, of course, a chance that Magic Leap is sandbagging us; maybe the real deal is just behind the next curtain! Past experience suggests otherwise…”
When it comes to revolution-launching developer kits, Luckey has some room to speak here. The Oculus Rift Developer Kit 1, the VR headset that started it all when it was released in 2013, only cost $300—a smidgen cheaper than Magic Leap One’s $2,300 price tag.
That said, Magic Leap still likely has enough funding to weather a less than resounding first product launch, although the true consumer generation offering will need to not only beat the upcoming the second gen HoloLens in terms of unique selling proposition, but also present a price-point that’s more accessible to early adopters looking to finally meet that big whale in the gymnasium—or at least a 4:3 window of it.
Magic Leap today announced that AT&T, one of the leading cellular carriers in the US, has invested in the company and solidified a partnership making AT&T the “exclusive wireless distributor of Magic Leap products for consumers in the U.S.”
Magic Leap, which is building a highly anticipated AR headset, today announced that it has “completed” its Series D equity funding round with an investment from AT&T. The company had announced its $502 million Series D back in 2017, but it isn’t clear if the round had grown to include AT&T’s investment, or today simply marks the conclusion of investment matters for AT&T’s portion (we’ve reached out to Magic Leap for clarification). As part of the investment, AT&T Communications CEO John Donovan joins the Magic Leap board as an observer.
The investment also forms a partnership between the two companies, with AT&T becoming the “exclusive wireless distributor of Magic Leap products for consumers in the U.S.” That means the consumer version of the Magic Leap headset will be sold in AT&T stores, but that may not include the company’s very first headset, the Magic Leap One, which is being positioned more as a development kit than a consumer product.
“When available for consumers, AT&T customers will be among the first to experience [the Magic Leap product] in select AT&T stores in Atlanta, Boston, Chicago, Los Angeles, and San Francisco, with more markets to follow,” AT&T said.
The partnership also gives us reason to believe that future Magic Leap headsets (if not the ML1) will include cellular radios for wireless data.
“We’ve joined with AT&T because we believe in a combined vision of expanding high-speed networks, edge computing, and deep integration with creative content,” said Rony Abovitz, CEO of Magic Leap. “Coupling the strength of the evolving AT&T network with Magic Leap’s spatial computing platform can transform computing experiences for people.”
The move is similar to Apple’s play when it launched its first iPhone. The company struck up an exclusive distribution partnership with US cellular carrier Cingular, promising that the two companies would work together to push the network infrastructure forward to enable new applications and uses for the iPhone. Cingular merged with AT&T in 2007.
Magic Leap is among the top funded tech startups in the world, having raised more than $2 billion, according to Crunchbase.
Adobe recently unveiled a new project that aims to take the company further into the realm of augmented reality. Called Project Aero, the newly announced AR authoring tool and multi-platform system that will soon deliver a way for developers to build simple AR scenes and experiences for Apple’s ARKit.
In collaboration with Apple and Pixar, Adobe is also adding ‘usdz’ support to Adobe Creative Cloud apps and services, a file format that is a zero compression, unencrypted zip archive for 3D content such as AR/VR objects. The integration of usdz support was first announced at Apple’s WWDC, which saw the release of ARKit 2.0.
Users, the company says, will be able to create AR content using industry standard tools such as Photoshop CC and Dimension CC, and then “convert assets into usdz that can be natively consumed in the Apple ecosystem,” writes CTO Abhay Parasnis in an Adobe blogpost.
Essentially, Adobe is taking one step further into a world still largely dominated by 3D game engines such as Unreal and Unity, which could foretell and interesting forward march from 2D creation to a decidedly more 3D-focused business. Project Aero is heading into early access soon, and is available right now by request-only.
To show off what’s possible with Project Aero, Adobe has partnered with 15 artists for The Festival of the Impossible, a three-day immersive art exhibition in San Francisco which is featured in the video above.
“This is just the beginning of our journey to extend digital experiences beyond the screen and I couldn’t be more excited about what’s ahead,” Parasnis says. “We’ll have much more to share at the Adobe MAX Creativity Conference later this fall.”
This year’s Game Developers Conference (GDC) is nearly upon us, and now that talks for the week-long event are published on the GDC website, it appears Google will be taking the stage to introduce a number of new products and tools for game devs. While it’s not explicitly stated whether the talk will be primarily focused on AR/VR, Google’s product manager for AR/VR developer tools Nathan Martz will be speaking, making us think there might be something interesting around the corner.
Presented at Google Developer Day keynote on March 19th, Martz will be joining Instant Apps PM Lead Jonathan Karmel and Mark Vela, Head of Games Merchandising at Google Play Americas. Here’s the brief, but tantalizing listing:
Google Developer Day: Keynote (Presented by Google): The Google Developer Day kicks off with new product announcements, tools and opportunities for game developers to build high quality, engaging experiences for a growing global audience.
While there’s only a quick blurb on the schedule itself, when talking to Martz recently GDC organizers called the keynote an “announcement-filled extravaganza.” Martz then told GDC organizers that “[t]his year we’ll be focusing on helping developers get the most out of Android, including deep dives into new platforms like AR Core and best practices for optimizing your app both pre and post launch.”
Google most recently released its 1.0 version of ARCore, which brings the augmented reality developer tool to a number of flagship smartphones including Samsung Galaxy S8, S8+, Note8, S7 and S7 edge, LG’s V30 and V30+, ASUS’s Zenfone AR, and OnePlus 5.
Martz’s team has also recently built a number of tools to help developers create, iterate, debug, and release their AR/VR apps including; Poly – the free 3D asset repository, Instant Preview – the tool for rapidly iterating on device), and GAPID – a tool to help debug GPU behavior.
We’ll have feet on the ground at this year’s GDC, so check back soon for the full details. GDC is taking place March 19-23 in San Francisco, CA.
GREE VR Capital, the investment firm backing startups such as VRChat, TheWaveVR, and Littlstar, today announced the completion of its first immersive media fund, totaling $18.3 million in allocated funds. Previously called GVR Fund, the company is now changing its name to GFR Fund as it prepares for a second fund later this year.
The fund has invested in 17 early stage companies over the course of its two-year existence, primarily in the North American VR/AR/MR market. According to a press release provided to Road to VR, the fund will be “actively seeking AR and MR startup companies for investment with a particular focus on mobile AR platforms, AR Cloud, consumer AR applications, and enterprise AR, including AR glasses.”
GREE VR says the team is continuing with its focus on helping U.S. startup companies looking to reach out to Japanese investors and market leaders. The name change, the company says, broadens the fund’s scope to more than just VR, putting more emphasis on ‘frontier technology’, a term intended to encompass VR/AR/MR.
“GFR invested in our Series A round and has been instrumental in helping our company grow both in the U.S. and Asia,” said Tony Mugavero, co-founder and CEO of VR video streaming platform Littlstar. “They brought us to Japan in 2016 and introduced us to nearly a dozen accredited investors, helped us secure Sony and others as new investors, and lined up a speaking engagement for us at the Japan VR Summit. The connections and visibility that GFR was able to bring to us has been critical in making Littlstar the success that it is and has afforded us the opportunity to expand into new markets.”
To date, the GFR Fund has invested in 17 companies, including VRChat, SPACES, Sliver.tv, Littlstar, Insidemaps, YBVR, Immersv, Experiment 7, TheWaveVR, Sturfee, InsiteVR, Streem, DottyAR, Upload and Torch3D. In addition to GREE, Colopl VR Fund, mixi, Inc., Maruhan, Yahoo Japan and Akatsuki are among Limited Partners (LPs) of the fund.
Starting toward the end of 2017, the number of listed job openings at Oculus has nearly tripled, with the current number of openings floating around an all time high.
Across Oculus’ 15 locations, the company is currently listing 443 available openings, spanning 20 categories from Research to Software Engineering to Computer Vision, and more. The number of openings has spiked significantly since November, data collected from the last 20 months by Road to VR shows.
Data collected by Road to VR
The spike comes at the end of 2017 and into the New Year. While that might coincide with annual budget planning, job openings at the company during the 2016 to 2017 transition didn’t show anything comparable to what’s been seen from Oculus’ hiring since November 2017.
A spokesperson for Oculus tells Road to VR, “Overall, the growing number of job openings across AR and VR are an indication of the level of commitment that Facebook/Oculus is making in the VR space.”
Corresponding with the company’s hiring spree, a promotional recruiting video was released last week extolling the benefits of working in the Oculus Research department, which currently has a record high 80 job openings:
Back in 2014, shortly after Facebook’s acquisition of Oculus and the formation of Oculus Research, the company’s chief scientist, Michael Abrash, called it the “first complete, well funded VR research team in close to 20 years.” In early 2017, Facebook CEO Mark Zuckerberg publicly toured the Oculus Research facility and shared an inside look.
Oculus’ clear room for nanofabrication | Image courtesy Oculus
With some 60 internship, co-op, and PHD student openings, it’s clear that Oculus hopes to take advantage of the Reality Lab as an incubator for future AR/VR research talent.
