Tilt Five CEO Jeri Ellsworth is working with her team to ship their patented AR technology to Kickstarter backers as they build out from a focused vision starting as AR glasses for tabletop gaming.
About a decade ago, however, Ellsworth was developing some of the core of that technology while working at Valve with a small team of hardware engineers doing research on forward-facing ideas like AR and VR. The idea was to invent “novel user interactions that broaden the Steam user base” that also “bring the entire family together in the living room”, as Ellsworth describes on her LinkedIn page.
Then Ellsworth and her colleagues were fired, leaving the engineer to contemplate the value of her voice and ability to speak openly about her time at the company against the amount of money offered in a severance payment packaged with a non-disclosure agreement. Separate to this consideration, what would losing her job at Valve mean for the future of the technology she worked on there? Would she need to work on something else?
Ellsworth made a key decision the day she was fired. In a recent interview with UploadVR, here’s how she describes her memories of what happened during her last day at Valve:
“I feel very fortunate that I made a split second decision the day that Valve did their big layoff. I showed up at the office. I met someone in the elevator and they said, ‘did you hear what happened to Ed? They laid him off today’. And I’m like, ‘that’s my mechanical engineer, working on my project, how could they do that?’ So I stormed upstairs and then it was like a bomb had gone off in the middle of the room. Everyone’s just sitting around moping. I hadn’t even opened my email yet to see the HR request to come see them. And someone’s like, ‘you’re getting fired today.’ I’m like, what? How can that be? That was the strangest layoff I’ve ever been around…they just let us kind of hang out in the building for like eight hours and we were just assigned a time to go talk to HR.
Folks that knew they were getting laid off were like angry and sad, and there were just tons of emotions. I was later in the day when I was going to receive the bad news and so I go up and I was like prepared to chew someone’s ass out about it. I walk in the door and Gabe’s there in the room with a lawyer/HR/somebody, and I started off like super aggressive. I was like, well, ‘so this is it?’ You know? And then immediately broke down into tears and I was getting emotional. And I’m like ‘Gabe, you gave me this mission to bring the family together and I can’t believe you’re doing this to me…I was onto something amazing.’ And he said some things like ‘I’m always going to be a fan’, like, oh, okay. And got myself back under like emotional control. And as I was walking out the door, I think my back was even turned to him. And I was like, ‘you should just sell me the technology.’ And I turned around and he was like, ‘okay.’ And that was it.
He made the decision on the spot to let me take this optical technique out of Valve. It was pretty incredible. I could have just walked out the door and just moved on to whatever project after that and never thought back on it…I don’t think folks around Valve understood what we were really onto at the time, like how we could generate this light field and how comfortable and vivid and how it solved all these problems. It was just back in those days there was still this notion that somebody is going to just stumble onto a way to make this perfect AR system. And you won’t need to use anything like a game board and still people are still dreaming and hoping that they’ll stumble onto some way to make that happen. And it’s the laws of physics. It’s really difficult.
The agreement was for $100, Ellsworth said, plus the cost of lawyers to make it legal, and “we basically got everything in our lab dedicated to the retroreflective glasses which included the prototypes, optical components, software, computers, etc.” The agreement would essentially help launch development efforts at CastAR, a company the ex-Valve employees co-founded to continue developing their approach to AR.
“The biggest win was the legal documentation that gave us freedom to operate,” Ellsworth wrote in a direct message.
Years later, Ellsworth and other CastAR veterans would essentially need to buy it all again — now with actual patents backing their retroreflective optical technique — after CastAR went defunct. Now at Tilt Five, Ellsworth is determined to “take on the big players.”
“At castAR and TiltFive we made improvements to the original prototype designs and have about a dozen patents covering the current design,” Ellsworth wrote.
Tilt Five is working on drivers to enable multiple glasses to run from a single PC as well as driving the system from an iOS or Android smartphone, features which — if well supported among developers — could offer an approachable way for AR “to bring the entire family together in the living room.”
Finally, after several years of hosting virtual events due to the COVID pandemic, the Game Developers Conference (GDC) held its annual event in San Francisco, in person. And gmw3 was there to scout out all the latest and greatest virtual reality (VR), augmented reality (AR) and Web3 content and hardware being demonstrated for the first time.
GDC 2022 – Unity. Image credit: gmw3
While great to see one of the biggest events in the industry calendar back on, it was clearly noticeable that GDC 2022 wasn’t going to be able to match pre-pandemic levels just yet. The main South Hall was nowhere near full capacity and crowds had plenty of space to walk around, making for a far more relaxed event.
Even so, there was plenty to keep GDC attendees occupied during the course of the week-long event, with the first couple of days dedicated to developer talks covering all the latest topics; from core sessions like game engines, monetization and diversity to more VR centric talks on OpenXR or using live VR actors in projects.
But that was just the warm-up to the main event, the sprawling expo held inside San Francisco’s Moscone Centre. And it’s here (mostly) that gmw3 found new VR videogames to keep an eye out for, AR hardware that provided some exciting gameplay opportunities and a rise in Web3 companies showcasing their latest Blockchain titles.
VR, AR & Beyond
So let’s get straight into the fun stuff, what VR videogames were on show worth a mention? Firstly, Japanese developer Thirdverse Inc. was there showcasing its follow-up to Sword of Gargantua with Altair Breaker. This is another sword fighting experience from the team but with more of an emphasis on arcade-style battles rather than precision sword fighting simulations. That’s not to say features such as parrying with a shield for a tactical return attack aren’t there, just this time around you’ve got more moves to play with. Keep an eye out as gmw3 will be doing a deeper dive later in the week.
Next up is Bootstrap Island by Maru VR. This is a roguelite survival experience placing you on a desert island, back in an era of flintlock pistols and dodgy rum. The demo on the show floor provided some basic mechanics, gathering wood to start a fire, smashing open a locked chest to find a pistol inside then quickly learning how to load and use said weapon as night fell. Clearly an early demo, one aspect that was very clearly important to Maru VR was visual fidelity. Bootstrap Island was gorgeous, with golden sandy beaches, crabs scuttling around and the wreck of a ship off in the far distance setting the scene. Bootstrap Islandwill only be coming to PC VR by the looks of it.
For those that love a bit of VR horror, there was Do Not Open for PlayStation VR. Designed to be an old school survival horror, you play an infamous zoologist trapped within a warped version of his own house. For demonstration purposes Do Not Open ran on an Oculus Rift S but the final version won’t support PC VR. The demo section took place around the kitchen, challenging players to solve a knife puzzle within a set time limit, failing to do so saw a giant creature appear to end the session. Had a similar feel to Intruders: Hide and Seek, so definitely worth a watch.
GDC 2022 – Tilt Five. Image credit: gmw3
There were several other VR titles but one of the show highlights came from the AR corner. Having announced a content partnership with Asmodee Digital at the beginning of the week, Tilt Five had its AR glasses on hand, providing one of the best hardware demos on the show floor. With the lightweight glasses on, Tilt Five presented three games to test, a third-person fantasy adventure where I held the slimline controller horizontal – just like the Wii remote – to hack through several enemies. Held in a vertical pointer fashion, a shooting gallery where you had to hit green blocks offered a fun distraction whilst the third title offered a platforming experience where I had to precisely build my own platforms.
All simple yet effective games, what really impressed me with Tilt Five was the clarity and depth of the visuals. Graphics on AR glasses can tend to be a bit washed out, but because the system requires looking at a specialised mat, that effect was greatly reduced. If you love the idea of AR, then Tilt Five could well provide a breakthrough gaming experience.
Sticking on the hardware side of things, French company Actronika was at GDC to give guests a taste of its haptic vest, the Skinetic. Currently running a Kickstarter campaign, Actronika demoed Skinetic with a range of software; from its own simulation environment that shot bullets at me, saw rain clouds shower me in water, and even dropped a nuke, the haptics work exceptionally well. A demo built inside a Half-Life: Alyx mod gave some on-the-fly haptic feedback and they’d even got the vest rigged up to the movie Avatar.
GDC 2022 – Skinetic. Image credit: gmw3
The highlight of the Skinetic demo, for me, was the laser, getting shot whilst simulating the effect that the beam was going right through me thanks to the front and back haptics. This kind of accessory isn’t easy to pull off but Actronika is going in the right direction.
The wonderful world of Web3
There was no doubt that Web3 and Blockchain gaming were going to be a part of GDC 2022. As mentioned, the previous GDC physical event was that long ago that this area of the videogame industry simply didn’t exist. Whatever your feelings towards blockchain gaming – a very divisive subject that even GDC’s own State of the Game Industry Survey shows most developers don’t have a love for – the sector is here and it’s here to stay by the growing number of booths.
Now, this is a best of GDC roundup and to be honest, none of the blockchain stands had a demo to play, unfortunately. Mainly because most were already available to download and start playing on your mobile or PC. Some like Seascape, for example, showcased their own titles as well as helped developers look towards this industry as a way to increase revenue.
And that was the overriding reason most were at GDC, education. Web3 gaming is so new that it can be a bewildering minefield that does offer genuine opportunity – not just scams.
Behind closed doors…
GDC isn’t purely about what’s on the show floor as any veteran will tell you, some of the most interesting demos and info lay behind closed doors. While gmw3 can’t talk about a couple of the things it was privy to during the event (at the moment), you’ve probably already heard about the one giant reveal that the press wasn’t allowed to see, the PlayStation VR2.
The below tweet from StrayBombay’s Chet Faliszek confirmed Sony Interactive Entertainment (SIE) and the headset was there, with select developers getting a chance to see the device – although there was no indication it was in its final form. To know that it was so close yet so far away…
Had one of those VR moments today playing in the new PSVR2 hmd…
You know where the world just feels different when you return?
Sooooo good… thanks @yosp and @GregRicey for the demo and chat.
Every weekend VRFocus gathers together vacancies from across the virtual reality (VR), augmented reality (AR) and mixed reality (MR) industry, in locations around the globe to help make finding that ideal job easier. Below is a selection of roles that are currently accepting applications across a number of disciplines, all within departments and companies that focus on immersive entertainment.
Don’t forget, if there wasn’t anything that took your fancy this week there’s always last week’s listings on The VR Job Hubto check as well.
If you are an employer looking for someone to fill an immersive technology related role – regardless of the industry – don’t forget you can send us the lowdown on the position and we’ll be sure to feature it in that following week’s feature. Details should be sent to Peter Graham (pgraham@vrfocus.com).
We’ll see you next week on VRFocus at the usual time of 3PM (UK) for another selection of jobs from around the world.
Tilt Five, the startup behind the eponymous AR headset for tabletop gaming, today announced it has secured $7.5 million in Series A funding.
The funding round was led by SIP Global Partners with participation from BITKRAFT Ventures, Galaxy Interactive, and games industry veteran Ken Birdwell, who is known for his previous work on the Half-Life series at Valve. Jeffrey Smith of SIP Global Partners is said to join Tilt Five’s board of directors.
The company says its Series A will be used to hire additional employees, build partnerships, and expand available content.
Founded in 2017 by Jeri Ellsworth, Tilt Five launched its Kickstarter campaign late last year hoping to attract $450,000 to bring its AR headset to life. Within only 24 hours Tilt Five was fully funded, going on to garner over $1.7 million throughout the month-long campaign. Including current pre-orders for the AR headset, this brings the startup’s overall funds to nearly $10 million.
Image courtesy Tilt Five
Tilt Five uses a similar projection-based technology to Ellsworth’s first AR headset system, CastAR, which attracted $15 million in venture funding before closing down in 2017. Similar to CastAR, Tilt Five includes AR glasses, a special retroreflective game board, and a 6DOF wand controller—all of which is made to work with multiplayer interactions in mind, both local and online play.
Tilt Five’s tech isn’t based on the same sort of wave-guide optics as conventional AR headsets though, which gives it a few important benefits over conventional designs; the headset actually projects the image via two head-mounted micro projectors and tracks the wand controller via inside-out sensors.
Image courtesy Tilt Five
This allows Tilt Five to achieve a reported 110-degree field of view, which is far and away larger than HoloLens 2, Magic Leap 1, or even the Nreal Light headset. Also, because it relies on PC, tablet, or smartphones to drive the experience, it only weighs 85 grams. Still, it’s limited to the playspace of the special gameboard, however judging by the startup’s early crowdfunding success, it’s clear there’s plenty of tabletop gamers out there looking for a more immersive experience.
“Tilt Five was founded on the idea that we can bring family and friends together to play in the real world, enhanced with the magical capabilities of immersive 3D technology,” said Jeri Ellsworth, Tilt Five CEO. “While the tech giants invest billions in far-off solutions, this investment expands our ability to deliver on a proven set of technologies resulting in an affordable, social and wearable device.”
Tilt Five says it’s ramping up production now with the intent of shipping kits to Kickstarter backers and pre-order customers over the next two quarters. It’s base model kit currently available for pre-order, Tilt Five XE, includes a pair of AR glasses, a game board and single wand— priced at $359. Larger format kits including three AR glasses, three wands and a gameboard, are said to arrive sometime next year.
AR glasses company Tilt Five raised $7.5 million from investors in a Series A round led by SIP Global.
The investment marks a major moment for Jeri Ellsworth’s patented approach to tabletop augmented reality. For those just catching up, Ellsworth is an electrical engineer who worked at Valve in the early days of its VR and AR research. She developed a novel approach to AR delivery with a pair of tracked glasses that project light onto a special retroreflective surface. The result is a field of view for AR that’s on par with modern day VR headsets — as long as you’re looking at things with the glasses that are backed by that special reflective surface. You’re also not blocked off from the real world like a VR headset, so you’re able to see your friends around the table with you through the AR glasses.
Jeri Ellsworth
If you’re wearing these glasses an otherwise still and flat board game could transform into one filled with life, movement and the sense of seeing real depth below anything within the board’s borders. An expansion piece extending the reflective surface vertically could even add some height to the virtual content displayed by Tilt Five.
A Kickstarter project for Tilt Five raised $1.7 million in October 2019, with delivery of the first PC-powered AR glasses kits originally promised to arrive in mid-2020. That timeline has come and gone with the global pandemic affecting hardware development timelines everywhere. In its funding announcement, the company wrote that it was ramping up production and would “commence shipping kits to Kickstarter backers and pre-order customers over the coming two quarters.”
Tilt Five is still offering its XE kits for pre-order, promising a pair of AR glasses, game board and wand for $359. Sometime in 2021 the company is planning to sell “Social Kits” with “3 AR glasses, 3 wands and a gameboard,” according to the announcement. Tilt Five’s investment round is led by SIP Global Partners with SIP’s Jeffrey Smith joining the startup’s board. BITKRAFT Ventures, Galaxy Interactive, Logitech, and Ken Birdwell participated in the round. The money will be used “to hire additional employees, build partnerships, and expand available content.”
“Our team has been so small for the last few years,” Ellsworth wrote in a direct message. “I’m so happy we’ll be able to bring on some more folks.”
$7.5 million isn’t exactly a huge amount compared to the hundreds and hundreds of millions of dollars Rony Abovitz spent pursuing an AR dream at Magic Leap in Florida. The funding round is also half of what Ellsworth’s previous company, CastAR, raised in 2015 from Playground Global while working on what amounts to an earlier generation of Tilt Five’s technology.
Tilt Five “Through The Lens” image provided by the startup.
“The environment for XR startups is much different these days,” Ellsworth wrote. “There was this notion that this new market would evolve much faster than other markets in the past. Companies were raising/spending billions and making claims that within a few short years they’d completely replace all the ways we do our computing. I can’t point to a time in tech history where any tech change happened this fast. It was like everyone was ignoring Moore’s law and forgetting how difficult it is to change consumer habits. This is why we’re in a good position right now (post-hype) with a very practical system designed specifically for entertainment, with a clear use case and marketing message that customers will understand.”
Tilt Five still needs to deliver on its Kickstarter promises of a great tabletop gaming experience. And even then? The company’s path to success is unclear beyond successful delivery. Still, if Tilt Five does find any measure of success in 2021 or beyond it might represent one of the most remarkable underdog stories in modern tech history.
AR startup Tilt Five confirmed in a backer update sent out recently that the coronavirus may impact the shipment of some early kits which were scheduled to be sent to backers this month.
Tilt Five is a tabletop, consumer AR system which finished its Kickstarter fully funded in October 2019. The majority of kits were scheduled to be sent to backers in the summer, however, one tier of the campaign included shipment of early backer kits sent out in February of this year.
Startup representatives wrote in a backer update that the coronavirus may impact the product of some of these early backer kits. The statement is as follows:
“As you may have read in the news, concerns about the potential spread of the coronavirus has led to quarantines, transportation disruptions, and mandatory extended New Year’s holidays for businesses in several regions of China. The truth is that businesses and factories in China are, understandably, taking precautions by stopping factory operations and having their workers stay at home safe. Obviously this was unforeseen, and it will have some impact on your timeline and ability to get some of the parts that we need. The good news is we have enough parts on hand to continue forward with our refinement of the assembly process.
Until this health crisis abroad is resolved, we won’t have every part that we need to assemble all of the Beta units. We’re hoping the situation will be resolved soon, and we can quickly get back on track.”
We reached out to Tilt Five Co-founder and CEO Jeri Ellsworth for comment, who added “I’ve never seen a supply chain situation like this before,” she wrote in a direct message. “I suspect almost every industry is going to be impacted because of the manufacturing reliance on China.” She also wrote that they are working on ways to deliver the promised units with only limited delays.
The World Health Organization offers advice to the public to protect themselves and others from the “Novel Coronavirus (2019-nCoV)” including washing your hands frequently, covering your mouth with a flexed elbow or tissue and maintaining space of around 1 meter between yourself and people who are coughing.
It is unclear whether there will be any further impact on Tilt Five shipments beyond the early backer kits but the virus is an ongoing and unexpected situation, so anything is possible. Keep an eye out for future updates.
CES is a strange conference. Across endless aisle after endless aisle a mixture of companies looking for suppliers, distributors, press or investment present wares in hope of making the year ahead one of major success for their business. The conference always presents a roadmap for the future of technology and it has been that way since the ’60s. Seeing the actual pathway ahead, though, is difficult to find through the manufactured hype and “best of” awards.
So I understand the comments we see from folks who want to know what was “good” at the conference who might be confused by vastly differing reports and reactions. Is the reporter able to describe what actually happened in their demo, or is their vocabulary lacking in describing if something went wrong? Do they gloss over issues? Are they sold the promise of the technology if bugs are worked out in a couple more years?
With this in mind, and the joy I know is already delivered so rapidly by Valve Index at home and Oculus Quest in a carrying case in my hotel room, I found myself largely unimpressed by almost all the AR on display at CES and much of the VR too.
“There is no doubt Oculus Quest has shown what the right mainstream consumer device is and while there was nothing immediately that will give the Quest a run for it’s money, I did try the new Qualcomm reference design and some other tech that suggests next year there should be many more all-in-ones that should get consumers and enterprise excited for the near future of VR,” AR and VR investor Tipatat Chennavasin explained in a direct message.
There were still hints of the future at CES 2020, but I need to address the difficulty and cost involved in polishing these technologies to the point of actual usefulness for businesses or mass appeal to consumers. That’s what this post is about.
So read on to understand what’s truly going on in VR and AR.
VR: Advances In Field Of View, Controller Tracking, And Headset Size
VR headsets at CES 2020 demonstrated advances on several fronts. None of these were entirely new, or entirely perfect, but all point to possibilities for the future.
Smaller Headset Designs
Several companies at CES demonstrated pancake lenses which dramatically reduce the physical size of VR headsets. These also generally featured a reduced field of view compared with current designs.
The one I found most interesting was Panasonic’s “VR Eyeglasses” which combined an incredibly light design with an innovative lens separation adjustment. Read my story about the Panasonic glasses.
While Pico showed a similar conceptual design alongside their Neo 2 and Neo 2 Eye I didn’t see any physical adjustment for lens separation on their glasses design. So the Pico glasses design made me extremely uncomfortable in a matter of seconds. The distance between my pupils is wider than many and I usually need that adjustment (which is found on both Index and Quest) to have a comfortable experience in a VR headset.
Huawei’s slim VR Glass design was also demonstrated by Nolo combined with their front-facing 6DoF tracking system as an add-on. BoxVR worked reasonably well in this configuration, but the $500 VR Glass plus Nolo kit only tracked my hands while facing forward. Under no circumstances would I be interested in a VR headset in 2020 or beyond that doesn’t let me turn around.
Combining pancake lenses with microdisplays is nothing new of course — eMagin showed this off in 2015 before even the release of the Oculus Rift and HTC Vive. Since then, microdisplay suppliers have claimed that headsets using this kind of display system are just around the corner. But now we’re finally starting to see major consumer electronics companies explore this idea, which may mean it is now closer than ever to being in an actual product.
Expanded Field of View
Some VR headsets at CES 2020, like the nearly $8000 Xtal and all the Pimax systems, featured an expanded field of view compared with headsets like Oculus Quest and Valve Index.
Xtal demonstrated an impressive $50,000 motion simulator with their headset integrated into a flight helmet with its wide field of view and 4K per eye resolution offering incredible sights flying in an F-18 over Dubai. While enjoyable, I noted some distortions at the edges of the lenses.
The Pimax Demo
Pimax frequently makes promises then delays delivering on them again and again. We’ve also heard numerous reports of problems with their hardware ranging from minor to major.
While my colleagues found Pimax demos at previous events, my first Pimax Demo was at CES 2020 and the first time I put a Pimax 8KX on my head I tried Boneworks.
The first thing I noted is that content popped in at the edges of the display a second or two after the rest of the scene. When I mentioned it, the demo (said to be running from an RTX 2080 TI) was switched to parallel processing and that solved the problem, but within about five minutes the right eye started blinking to black every few seconds.
I asked for another 8KX to try with Boneworks and this one didn’t have the problems I saw before. Its wide field of view invited me to look around a little more and the high resolution of the panels were certainly nice, but I found myself also constantly distracted by the significant distortions and bending of the scene at the edges of the lenses — more significant than the distortions I saw with Xtal. My right hand also lost tracking with 8KX, but that’s not entirely unusual for SteamVR Tracking in convention settings.
We’ve heard some people with Pimax headsets artificially reduce the field of view of the system through software adjustment to avoid these distortions while still enjoying a relatively expanded field of view. My colleague Tatjana returned to the Pimax booth later in the CES week to try both the 8KX and Artisan. Aristan is Pimax’ newest headset which ends up priced comparable to Valve Index when you include controllers and base stations. She played Boneworks and Fruit Ninja and found herself sick to her stomach afterward. I, too, left the Pimax booth a bit woozy.
There are Pimax defenders out there and those who will find the expanded field of view a good fit for their strong stomachs and top of the line graphics cards. Indeed, I got sick repeatedly over my 12-hour play through of the Boneworks campaign with Valve Index, but didn’t get uncomfortable as quickly as I did with that Pimax demo at CES.
Surely a wider field of view is something we want in future VR headsets but I remain unconvinced we’ll see it at an affordable price other than from a major platform like Facebook, Valve, Sony, Google, or Apple with large teams developing both optics and eye tracking technologies which could make these devices more comfortable.
Overall, the repeated delays for Pimax products and less than stellar impressions at CES 2020 means that we will keep using words like “claim” to describe plans for products pitched by this company.
Alternative Tracking Systems With Pico
The Pico Neo 2 and Neo 2 Eye, priced at $700 and $900 respectively, featured occlusion-resistant electromagnetic tracking. In the Neo 2, for example, I played Angry Birds and was able to pull back the slingshot while keeping my head turned away from the controllers. This is something I can’t do in an Oculus Quest. The Neo 2 also balances weight to the back better than Quest’s front-heavy design.
The Neo 2 Eye demonstrated Tobii’s eye tracking for the first time I’ve seen in a standalone design. Also for the first time I’ve seen, the headset tracked my eye movements without calibration. Typically, Tobii asks each wearer to point their eyes at a few dots at different corners of the scene to calibrate, and I was told that option is still available. The eye tracking without calibration seemed to work alright — I was able to target enemies in a scene by just pointing my eyes at them — but when I pointed my eyes downward it seemed to lose tracking.
I’d need more time with the $700 Neo 2 to say more, but I found it to be the only Pico device I’ve ever tried I might actually be interested in using. However, Neo 2 is only sold to businesses.
AR: Tracking Lacking, Power Problems And Severe Field Of View Limits For Years To Come
During the course of CES my colleague David Heaney described simply something that’s hard to grasp about the market for AR head-worn gadgets: Recognizing occlusion in your environment is to AR what 6DoF tracking is to VR.
This is to say that if an AR headset with see-through optics inserting digital content into your real-world environment cannot reliably understand when a person or object blocks simulated content from your view, the illusion meant to be provided by the hardware is broken. The same is true of VR headsets that do not track the position of your head. For example, if you wear an Oculus Go VR headset and decide to lean forward or stand up, the illusion of VR that you were enjoying is instantly broken.
Discomfort or confusion often follows in either case of severely limited VR or AR, and this represents a death sentence for the hardware. Failing to provide 6DoF tracking, or recognize occlusion, is so uncomfortable, I believe some headset wearers are discouraged from wearing the hardware ever again. Of course, some can learn to work around this limit.
There are fans of Oculus Go who understand its limitation and use the device as a low-cost personal media viewer. Nreal AR glasses, for example, provide a relatively large field of view for a similar genre of content. Still, without a complete and constantly improving understanding of the environment around the wearer, AR glasses like the $500 Nreal (and there were a lot of copycats at CES) are likely to consistently fail this test. Without passing this test walking out in the real world, the appeal of these kinds of AR hardware platforms is minimal to developers, to businesses, and to consumers.
“I would agree that for AR glasses to be truly useful and game changing, they would need not only solid 6DOF tracking (which none of the slim AR glasses I tried had), object occlusion, and object permanence but also really intuitive and reliable 6DOF input which was not also shown,” Chennavasin wrote. “I appreciate the effort being done by all the companies but it still looks like consumer AR is still years away.”
This roadblock for AR relates to two others — power consumption and field of view. Most AR glasses fail to deliver anything close to the amount of digital content to your eyes seen through most VR headset designs. This means that even in HoloLens 2 and Magic Leap 1 — two dedicated AR headsets — you need to move far away from the digital content to get a full view of it and truly enjoy the sense of immersion it brings. Again, some can get used to this limitation but the bulky size and high price of these AR systems ($2,300 for ML1 and $3,500 for HoloLens 2) also prevents them from appealing to consumers.
Lastly, building the most detailed map of your environment typically means on-board cameras need to keep scanning the room to keep the map updated. Using those cameras drains power. One of the key reasons the first generation of phone-powered VR headsets were retired so quickly is because those headsets drained power from a device you needed to last the day for other tasks. So do phone-powered AR systems like Nreal have a shot if they drain phone battery power just like the already-defunct Gear VR, and are used for the same types of content as the $200 Oculus Go starting at more then double the price?
“I agree that power is a (possibly the) key constraint for HMDs. It’s one reason why active depth cameras are not the best solution for occlusion, and we are seeing 6D’s approach of using low power RGB sensors, combined with efficient Neural Net co-processors providing solutions to occlusion,” explained Matt Miesnieks, the CEO of spatial recognition company 6D.ai, in a direct message. “The first low cost AR headsets do need refinement, but we have some visibility into roadmaps, and as that refinement is primarily driven by software, we will see improvements very rapidly.”
With Nreal in particular I saw software that needed significant refinement, both in tracking and stability, to become more usable.
Short Term Potential Advances In AR And VR
CES 2020 confirmed to me what we’ve already seen elsewhere.
Tilt Five’s forthcoming consumer AR system uses a novel retro reflective method delivered as a game board that tunes for both a relatively wide field of view and the specific case of tabletop games. These aren’t glasses you’ll take out into the real world and since you are localized to the game board, occlusion isn’t as much of a concern. Tilt Five cost $879 during its recent Kickstarter for a three-pack of glasses and there are extensions to the board to expand it vertically to provide more height to the augmentation effect. Altogether, Tilt Five represents one of the most promising short-term AR projects we’ve seen. I contacted Tilt Five CEO Jeri Ellsworth and she suggested there may even be a way to charge companion phones, with a a pass through hub, while also powering the glasses.
“We have a huge advantage by containing our system.,” Ellsworth wrote in a message. “We also limit the compute on the device…by doing re-projection and in headset tracking which saves power.”
Shipping a hardware crowdfunding product isn’t easy, so there’s a lot to prove for Tilt Five, but after CES 2020 I’m still left thinking Ellsworth’s company remains the most interesting AR project approaching the consumer market in the near-term.
When it comes to VR, we know the future of the medium features wireless, higher field of view, more tracking of body features in a wider range of conditions and smaller, more well-balanced headsets. But taking all the best pieces of CES 2020 VR systems — Pico’s better balance, Xtal’s wide field of view, Panasonic’s slim design — and then polishing these features, adding on more and supplying them with content is an entirely different level of challenge than preparing an interesting demo.
I’m talking about spending billions of dollars and there are only a few companies positioned to make that sort of investment in the next couple years.
Tilt Five successfully finished its Kickstarter campaign fully funded, far exceeding the $450,000 USD initial goal. The campaign reached a total of $1,767,301 pledged from 3345 backers, unlocking four stretch goals.
A $299 pledge promised everything you need for one person to use the system – one pair of the lightweight, high-FOV AR glasses, the square game board to frame the AR content and a wand to interact with what appears on the board, along with some others accessories like the required cables. Three pairs of glasses and three accompanying wands required a pledge of $879.
Tilt Five CEO and Co-founder Jeri Ellisworth also posted an endorsement of the system by Gabe Newell with six hours of the campaign left, who said he was “super impressed” with what the Tilt Five team had built. Tilt Five reached its fourth stretch goal with under two hours left of the campaign.
I asked Gabe Newell to share some of his thoughts about @tiltfive AR glasses after his visit a few days ago. It was a proud moment for me to show him how far we've come!
Tilt Five will launch with several free introductory games, but it has also announced several content partners that bring games to the tabletop AR system, including Monocle Society, Tabletopia and others. As the campaign exceeded its initial funding goal, it also went on to reach three out of four stretch goals that reward backers even more, outlined as follows:
Stretch Goal #1 (at $750,000): One new board color available – backers who pledged $359 or more can now choose between a black border or one other color scheme, which will be decided upon in a vote conducted with backers.
Stretch Goal #2 (at $1,250,000): One new glasses color and one new wand color – backers who pledged $359 or more can choose between the black and white color scheme and a second color scheme, which will be decided upon in a vote conducted with backers.
Stretch Goal #3 (at $1,500,000): Another glasses and wand color available, decided upon in a vote conducted with backers.
Stretch Goal #4 ($1,750,000): Three games – Wartile, Chuck’s Challenge 3D and Cubiti pARti – are now included for free with the systems that will ship to Kickstarter backers.
Tilt Five expects to ship most kits to backers in the middle of 2020.
Did you back the Tilt Five Kickstarter campaign? Let us know in the comments.
One of the big questions accompanying the Kickstarter was what tabletop games would actually be supported, with the project page originally promising a partnership with Fantasy Grounds, a virtual tabletop system that sells officially licensed digital add-ons for games such as Dungeons & Dragons. In a mid-campaign update, Tilt Five says Monocle Society and Tabletopia will join as partners. Monocle Society plans to bring their storytelling card game Weave to the platform, which they describe as “half tarot, half roleplay, all story.” Tabletopia, meanwhile, is a service offering a variety of games like chess and various card games alongside a large collection of modern games. In the update, Tilt Five says that with Tabletopia even if “your friends don’t have Tilt Five, they can still join in via PC, phone, or tablet.” The standard online web version of Tabletopia is available now if you want to see how it works.
As of this writing, Tilt Five raised $1,357,825 from 2,602 backers.
Weeks of fundraising on Kickstarter remain for tabletop AR project Tilt Five but they’ve already surpassed some notable milestones.
The project ends fundraising on Oct. 28 with a $299 pledge promising a pair of lightweight wide field of view AR glasses that work with a tabletop board. Tilt Five promises a three-pack of glasses for an $879 pledge, and neither of these prices include the processing computer(s) needed to draw the augmentations you can see through the glasses. Tilt Five says their hardware supports “phone, tablet, or PC” with delivery planned for mid-2020.
The project cleared its $450,000 goal in less than 24 hours and recently eclipsed the amount raised in 2013 by Tilt Five’s predecessor — CastAR. That original project raised $1,052,110 from 3,863 backers and never ultimately delivered. They did, however, offer refunds. The new project is backed by 2,118 accounts at the time of this writing and those pledges already funded the next generation Tilt Five product past the amount from 2013, with $1,075,581 raised.
“My biggest mistake was to not believe in my ability to be CEO,” Tilt Five co-founder and CEO Jeri Ellsworth wrote to me of her time at CastAR. “We may have cracked all of the things that’s keeping AR from going mass market.”
As we reported previously, hardware Kickstarters often face challenges and the risk can be higher due to more expensive support levels. The Tilt Five project page states they’ve done “extensive prototyping” as well as “multiple pilot runs” and its “risks” section on the Kickstarter page says they “promise to be fully transparent about any issues that come up, and we’ll do our best to address and solve them quickly.”
