Apple’s iPhone X Facial Recognition Will Revolutionize VR/AR Development

Apple’s iPhone X Facial Recognition Will Revolutionize VR/AR Development

One of the co-founders of groundbreaking Los Angeles-based VR startup Kite & Lightning received his new iPhone X on Saturday. By Sunday night, Cory Strassburger produced a proof-of concept using the phone’s landmark real-time facial recognition system to animate a model for a character from their upcoming game, Bebylon: Battle Royale. While the proof-of concept is not perfect, there appears to be enough room for improvement achievable with the approach that he can use it for his specific purpose. A real-time facial capture system embedded in your phone could dramatically lower the cost to make digital characters look more alive and expressive in a wide range of content, including VR and AR software.

For those unfamiliar, the premise of Kite & Lightning’s Bebylon is a future in which immortality has been achieved. The drawback is that newborns stop aging at the baby stage. Thus, a society of “bebies” that live forever spend their days entertaining themselves with a vehicular combat game focused on over-the-top attitude and spectacle. These bebies have enormous personalities, and the iPhone X face capture system — the same one used to make animated emoji — could be how they are brought to life.

“I need fast and easy,” Strassburger wrote in an email. “The free bonus would be I can easily make a super cool AR promotional app where Bebylon players (or anyone for that matter) can capture some fun clips with their Beby character… and eventually, when this tech is more widely adopted, have players record expressions into their ‘character’s expression library’ using a complementary mobile app, that can then be uploaded to the game and used by them in gameplay for funny taunts and such.”

It’s still an early test and there may be further improvements he can achieve, but Strassburger wrote that “honestly its better data than most anything I’ve seen when it comes to real-time” capture and “I’m pretty confident that I’ll use this process for our game, cinematics and marketing content.”

Check out the video below where he does a pretty good impression of Christopher Walken while animating one of these bebies.

Apple’s iPhone facial recognition system is based on its 2015 acquisition of startup Faceshift. The startup showed how it could animate digital characters in real-time without any setup. The tech also made use of PrimeSense, another Apple acquisition that originally built the motion capture system in Microsoft’s Kinect. It appears that over the last several years Apple has been totally focused on miniaturizing the tech so that it could be used in a device as small as an iPhone. Over the next few years it is likely we’ll see the technology proliferate across Apple’s entire product line.

Using the iPhone X facial capture in combination of different technologies including Eisko and an Xsens motion capture suit, Strassburger believes there is an “an incredibly cheap” capture pipeline for human performance “that is super fast and easy to set up with very impressive results.”

Follow Kite & Lightning on Twitter for future updates, and we’ll keep following as well.

Tagged with: ,

Bebylon: Battle Royale Is Weaving The Worst Of Online Behavior Into A VR Party Brawler

Bebylon: Battle Royale Is Weaving The Worst Of Online Behavior Into A VR Party Brawler

On our list of 50 VR games we can’t wait to play in 2017, Bebylon: Battle Royale certainly stood out. With babies (or characters that appear to be babies) engaging in vehicular melee, one can’t help but be curious about what’s going on here. At PAX South 2017, Kite & Lightning co-founder Ikrima Elhassan was excitedly demoing Bebylon for passersby and we jumped at the chance to speak with him about the game and get some hands-on time.

We had no idea what we were getting into.

The game is still a ways off from release for this year, so what we played was a very early build. Nevertheless, Bebylon is wild and exciting. In Bebylon’s science fiction future, humanity has discovered an immortality pill that stops aging, which includes newborns. In Battle Royale, we’re introduced to the society of Bebylon which was created by adults that still look like babies who grew tired of being ridiculed and not taken seriously. No one can die anymore in this world, so things like status, fame, and wealth are the focus of the inhabitants of Bebylon and they feed their ego through battle royales.

The battles themselves are the core of the gameplay and it’s essentially a party brawler in the vein of Super Smash Brothers. With ego and status as prime focus in the meta of Bebylon’s world, it’s hilarious to see that the core gameplay involves a great deal of trolling abilities based on popular culture. For example, when you use a powerful uppercut you take a selfie that immediately goes up on the fighting venue’s huge screen. As you are more narcissistic and show-boat during combat, your ego meter fills up and you are able to do even more powerful attacks. The audience is influential as well and, as you play to their excitement, you earn multipliers for cash and gaining followers (the game’s experience system) for the end of matches.

“The whole game, the subtext, is a bit of a satire on how everyone in society may be vying for attention all the time,” Elhassan explains. “The more terrible you are and the more terrible things you do, the more you get rewarded for it. For the game design aspect, we took everything that kind of annoys us or the worst of modern online behavior and were like ‘let’s make that the foundation of a game, what could go wrong’.”

With the third-person nature of Bebylon: Battle Royale, Elhassan seemed to expect the question of “why VR” but he and the team are taking a more immersive approach. Elhassan says, from the outside looking in and from screenshots, it’s third-person with a fixed perspective and seems like it could be a non-VR game (the team is considering a non-VR version).

“VR adds a couple things we’re really excited about,” he says. “From a pure technical fighter perspective, the depth perception VR gives you allows you to fight in this 3D arena in ways you couldn’t do before.” He goes on to reference games like Power Stone and Smash Brothers that have a fixed and limited space that, while it is 3D, is still on a 2D plane to a degree.

The arenas in Bebylon can change and evolve depending on in-fight events, even spinning around quickly which is something that affects VR users in a different way from someone playing on a standard screen.

“It allows, from a creative and design perspective, us to really mess with the sense of scale and the sense of environment as a new axis to move the emotional needle in different directions,” he said.

The controls seemed unfinished in this build, but the foundation of the gameplay was fun. With the fixed perspective, comfort wasn’t a concern at all but you do still get a taste of VR immersion when the map’s floor disintegrates and everyone falls down to a lower level. We’ll be looking forward to getting our hands on a more complete build sometime later in the year.

Tagged with: , , , , ,

Kite & Lightning’s Senza Peso Now Available for Oculus Rift and HTC Vive

Back in August VRFocus reported on Kite & Lightning resurrecting its virtual reality (VR) mini opera Senza Peso for current head-mounted displays (HMD). Today the experience has gone live on Steam with support for HTC Vive and Oculus Rift.

Originally debuting back in 2014 on the Oculus Rift Development Kit 1 (DK1), Senza Peso is described: “As a soul transitioning into the afterlife, Senza Peso shepherds you on a spiritual journey through a beautiful and foreboding dark world of lost souls and redemption.”

The experience was a VR adaption of a short film by Cory Strassburger & Alain Vasquez – Strassburger co-founded Kite & Lightning with Ikrima Elhassan – in which viewers become the protagonist and personally experience the metamorphosis from death to eternity.

Kite & Lightning is currently working on multiplayer brawler Bebylon: Battle Royale, which involves babies remote-controlling a vehicle-mounted baby in an arena of death. These vehicles are armed with giant fists which can punch and slap opponents along with other offensive weaponry.

Senza Peso can be downloaded now for free, and for all the latest Kite & Lightning news keep reading VRFocus.

‘Senza Peso’ Is A Must-Download Available Now For Rift And Vive

‘Senza Peso’ Is A Must-Download Available Now For Rift And Vive

Kite & Lightning released an updated version of their 2014 demo experience Senza Peso on Steam. If you haven’t tried it, the download is free and certainly worth a viewing on either Rift and Vive.

This is one of the many environments seen in Senza Peso.

The surreal journey down the river Styx is a fantastic sit-down demo VR experience with incredible music. The software was a go-to demo for Rift development kit owners and likely introduced a great many people to VR for the first time.

Senza Peso was shown at some of the earliest big VR meetups by Kite & Lightning before the startup took on some work-for-hire projects. That gave them the cash they needed to make a year-long pilgrimage to Paris for artistic exploration. Ikrima Elhassan and Cory Strassburger, the two co-founders, returned to Los Angeles this summer funded with $2.5 million and an idea to create an expansive multiplayer VR world centered around immortal babies with bad behavior.

You can almost feel the chilly wind blowing during this very memorable environment seen in Senza Peso.

Senza Peso, though, brought the co-founders a lot of early attention in the VR industry and featured some pioneering work using a mixture of captured performances set inside an environment developed with a game engine. All set to an enveloping soundtrack, even on a DK2 headset the experience made a strong case for the power of virtual reality. The experience features no interactivity and it is a bit like the It’s A Small World ride at Disneyland, but instead of a tour of different cultures you get to see different regions of the afterlife.

We highly recommend you give the free experience a try. It’s available now on Steam.

The VR Job Hub: The Secret Location, Kite and Lightning & Cubical Ninjas

Whether you’re an experienced designer, programmer, engineer, or maybe you’ve just been inspired after reading VRFocus articles – either way, you have stumbled across VRFocus’ VR Job Hub. The jobs listed here are located worldwide, from major game players to humble indie developers – the one thing they all have in common is that they are all jobs in VR.

View the new listings below for more information:

Location Company Role Link
Toronto, Canada The Secret Location Technical Artist/Shader Artist Click here to apply
Chicago, US Cubical Ninjas VR Engineers Click here to check it out
LA, US Kite & Lightning VR Engineer Enquire: hello@kiteandlightning.la
London, UK Powster Web VR & Creative Front-End Developer Click here to apply

Look back at last week’s post for ongoing listings.

Check back with VRFocus 4pm BST every Sunday for the latest positions in one of the most progressive industries.