In a move that seeks to combine the expertise of two very different companies working towards a similar goal, the camera and virtual reality (VR) technology company Lytro have joined forces with narrative animation experts at Limitless, taking Limitless into the fold of the company, bolstering their forces and preparing them to make their own unique VR content in-house. The joiner was announced on Limitless’ website.
Up to now, Lytro have been focused on crafting their light-field technology. It enables both animatied and live action capture, utilising 6 Degrees of Freedom (6DoF) technology to allow for more flexible environments in a 3D space. Users can lean in and out of live action 3D environments, something impossible with traditional VR capture of live action locations.
Limitless meanwhile have been honing their interactive character technology, which allows users to create interesting, dynamic interactive characters in a VR environment without having to be an expert in AI programming. It allows for immersive, believable interaction experiences and animations.
With Limitless joing Lytro, the two technologies can combine and create new immersive and interactive experiences, with believable characters and the quality of animation that Limitless are known for. We previously saw Limitless’ expertise at work in Gary the Gull, which displayed their new technology.
We saw Lytro’s tech at work in the recent release Hallelujah, which reimagined an iconic ballad in a VR experience format. The choir makes for an immersive and intense VR experience, only topped by being there in person. You can view the experience by downloading the Within app, which is available for most VR head-mounted displays (HMDs).
Way back in 2015, Lytro’s Jason Rosenthal said that VR was the next wave of cinematic content, and the recent join with Limitless reaffirms the belief he shared then; “VR is the next wave in cinematic content, and immersive storytellers have been seeking technology that allows them to fully realize their creative vision. We believe the power of Light Field will help VR creators deliver on the promise of this new medium. Lytro Immerge is an end to end Light Field solution that will provide all the hardware, software and services required to capture, process, edit and playback professional grade cinematic content.”
Hopefully we’ll see much more from Lytro in the near future which comes as a result of this wise and strategic acquisition. When we hear more from Lytro, you’ll read it first on VRFocus.
Last month VRFocus went to AR and VR on the Lot, (previously known as just VR on the Lot) to uncover what Hollywood’s high-end blockbuster creatives are looking at, and caught up with Lytro, a company specialising in light-field cameras and technology.
The changing of the event’s name, may tell you how the introduction of augmented reality (AR) applications such as Apple’s ARKit and Google’s ARCore will be shaping the future of content in the entertainment industry. VRFocus spoke to Orin Green, the VR/VFX Supervisor at Lytro Inc. to talk about what they’re focusing on, the virtual reality (VR) experiences they were showcasing as well as what they were doing at AR & VR on the Lot.
Green explains that Lytro’s focus is on live action and animated capture, enabling content creators to have the freedom to use both 3D models or real-life locations for future projects. He said that through Lytro’s ability to capture light-fields, it opens up a whole new world for individuals looking to use true 6 Degrees of Freedom (6 DoF) for their VR experiences, allowing the user to move around freely in a more natural way, for a more immersive experience.
The mechanical bird and laser-like head from ‘One Morning’
A lot of 360 VR experiences have problems with creating realistic environments or experiences; where a user is normally stuck to the position of where the camera is situated, therefore unable to move around in a space. The freedom of movement allowed in 6 DoF virtual reality would enable a user to walk around a 3D environment that looks highly realistic. Examples of the environments Lytro have created with their technology can be seen in the video below.
Lytro showcased three VR experiences at AR and VR on the Lot. The first is Hallelujah, which is also available on Within’s website and app. With singer and composer Bobby Halvorson performing Leonard Cohen’s classic track. The musical experience premiered at Tribeca Film Festival in the U.S. and internationally at Cannes Film Festival in France. When in the experience you will see Halvorson start singing in-front of you, then several versions of him as he starts to add a unique five-part a cappella around you. The piece escalates to suddenly finding yourself inside a church before Halvorson is joined by a choir.
The second VR experience allowed users to walk around fully rendered CGI environment in 6 DoF. “Right now when you get animated environments, they often get generated from game engines which have very limited rendering capabilities.” Green explains. “With our system, you don’t have to be limited by the game engine.”
Their third experience One Morning, is the first light field animated short film in history by director Rodrigo Blaas. Blaas has done animation work on Wall-E (2008), Up (2009)and Finding Nemo (2003) and worked for years at Pixar and Dreamworks Animation. The short VR experience has a small mechanical bird with a red laser head come around a blue car, it comes to inspect you then suddenly a larger mechanical bird (presumably it’s mother of father) appears.
Lytro attended AR and VR on the Lot not only to showcase their technologies and future capabilities of VR for future content, but to also make new contacts and meet creators interested in starting a partnership with Lytro for future content creation. Watch the video below to find out more.
After premiering at Tribeca Film Festival and Cannes this year, immersive production company Within and partner Lytro have now released 360-degree music experience Hallelujah to the public.
A reimagining Leonard Cohen’s iconic ballad, the companies created Hallelujah using Lytro’s light-field technology, a 95-camera lens array which is rotated five times on set to create a full 360-degree experience. Performed by composer/vocalist Bobby Halvorson and directed by Zach Richter, the 6DoF in-venue film is centered around a 5-part a cappella arrangement sung mostly by Halvorson before a choir is introduced.
“In honor of Leonard Cohen’s birthday, we are sharing this immersive acapella rendition of ‘Hallelujah’ on the WITHIN app. Combining the transportive qualities of music with virtual reality results in an incredibly emotional and gripping experience. We hope the launch of ‘Halleujah’ will provide a chance for people to connect with the iconic ballad in an entirely new way,” said Chris Milk, WITHIN Co-Founder and CEO in a statement.
“In Hallelujah, viewers are placed inside a virtual world that feels and looks like our reality, and are able to physically move around within it. While most live action VR systems keep the viewer fixed in a single point in space, Lytro’s technology breaks that boundary by giving the user freedom to move, allowing for a lifelike sense of presence that hasn’t been possible in virtual reality until now,” Richter explains. “We were also able to create a mobile version that immerses the viewer in incredible 3D sound, allowing anyone with a smartphone to explore the five parts of the a cappella and engage with this powerful music experience.”
The Within app can be downloaded for iOS and Android for Google Cardboard, or HTC Vive, Oculus Rift, Samsung Gear VR, and PlayStation VR head-mounted displays (HMDs). Or you can watch Hallelujah on the Within website.
For the latest Within announcements, keep reading VRFocus.
Hallelujah is a new experience by VR film studio Within that’s captured using Lytro’s latest Immerge light-field camera which captures volumetric footage that makes for a much more immersive experience than traditional 360 video. Hallelujah is a performance of Leonard Cohen’s 1984 song of the same name, and mixes the latest in VR film capture technology with superb spatial audio to form a stunning experience.
Update (9/22/17, 3:49PM PT): A spokesperson for the project has confirmed that the version of the experience just released is mastered from the original light-field capture, but unfortunately takes the form of a 360 video rather than true volumetric video, even on the desktop VR headsets that support positional tracking. We’ve inquired if and when the volumetric version will be made available.
Update (9/22/17): The light-field captured piece, Hallelujah, is finally available to the public through the Within app on just about every mobile and desktop VR platform, for free. Head to the Within website to be directed to the app for your platform of choice.
Photo courtesy Lytro
Original Article (4/23/17): Lytro’s Immerge camera is unlike any 360 camera you’ve seen before. Instead of shooting individual ‘flat’ frames, the Immerge camera has a huge array of cameras which gather many views of the same scene, data which is crunched by special software to recreate the actual shape of the environment around the camera. The big benefit of which is that the playback puts the viewer in a virtual capture of the space, allowing for a limited amount of movement within the scene, whereas traditional 360 video only captures a static viewpoint which is essentially stuck to your head. Not to mention the Immerge camera also provides true stereo and outputs a much higher playback quality. The result is a much richer and more immersive VR film experience than what you’ve seen with traditional 360 video shoots.
After a recent visit to Lytro check out the latest version of the Immerge camera, I concluded, “All-in-all, seeing Lytro’s latest work with Immerge has further convinced me that today’s de facto 360-degree film capture is a stopgap. When it comes to cinematic VR film production, volumetric capture is the future, and Lytro is on the bleeding edge.”
In that article I talked a lot about the tech, but I couldn’t say much about the experience that I saw which left me so impressed. Hallelujah was that experience, and now I can talk about it.
Created by VR film studio Within, Hallelujah pairs Immerge’s volumetric capture capabilities with finely mixed spatial audio that forms the foundation for a stunning performance of Leonard Cohen’s 1984 song Hallelujah (perhaps most famously performed by Jeff Buckley). Lytro has provided a great behind-the-scenes look at the production here:
In Hallelujah, singer Bobby Halvorson starts as the solo lead of the song directly in front of the viewer against a pitch black background. As the only object in the scene, it’s easy to feel the volume and shape of the singer thanks to the volumetric capture. As you lean left and right, you see the sides of Halvorson’s face in a way that would be utterly impossible with traditional 360 capture techniques. With that sense of depth and parallax comes a feeling that the singer is really there right in front of you.
The entire song is sung a capella with no instrumentation, and copies of Halvorson are duplicated to the left, right, and behind the viewer, singing accompanying tracks.
Photo courtesy Lytro
There you are, in a void of blackness, with Halvorsons at arms length on all sides, singing a stirring version of the uplifting song. You can look all around you to see each version of Halvorson singing a different track as he creates all the notes of the song. As you turn your head, the careful audio mixing becomes apparent; each track accurately sounds like it’s coming from each of the respective singers. This excellently mixed spatial audio significantly enhanced the sense of Presence for me; Halvorson doesn’t just look and feel like he’s there in front of you, he also sounds like it.
As the song progresses, you may noticed a distinct and fitting reverb coming from Halvorson’s voice. It isn’t a digitally applied effect though; as the song progresses, the black void around him eventually fades away and you find yourself in the midst of a beautifully adorned church. Behind the singer is a full church choir that joins into the song all at once, adding a swelling of new voices to Halvorson’s self-accompanied solo.
Photo courtesy Lytro
Turning around to explore this newly unveiled environment, you can see the ceiling, walls, and windows in detail; behind you are rows of empty pews, flanked by huge columns supporting arches that run from the back of the church to the front. Returning your gaze forward, the song reaches its climax and eventual conclusion.
Lytro’s Immerge camera | Photo by Road to VR
The whole thing is but the length of the song (about 5 minutes), but the stirring performance feels like it was done for you alone, one which feels uniquely immersive thanks to some special technology and a carefully planned and well executed production.
Hallelujah is the kind of experience that is likely to be used for a long while as a demo not only to show the benefits of volumetric capture, but also to stir the imaginations of other creators working in the medium of VR film. Sadly, there’s no word yet as to when (or on what platforms) the experience will launch to the public.
