Eyes-On: Light Field Lab Built A Tantalizing Holographic Display For The 2020s

Eyes-On: Light Field Lab Built A Tantalizing Holographic Display For The 2020s

There’s a monarch butterfly in the dark, suspended behind the glass panel of a decorative wooden lantern about a meter away from me. I am not wearing glasses. Light Field Lab representatives say they aren’t tracking my head’s position.

Like a practiced magician, Light Field Lab CEO Jon Karafin removes the lantern from its perch to reveal the butterfly is floating in the open air.

“This is a complete reconstruction of the light field such that your eye focuses on the holographic object from within the holographic viewing volume,” Karafin said.

Future vision concept art of a room scale holographic display from Light Field Lab.

I move my head to the left and right inside the small “visual acuity” area outlined with glowing tape on the floor. This is the sweet spot. I also move my head both up and down in this area about a foot. The butterfly’s wings seem to respond with the correct parallax as I shift. I move slightly closer and backward — still inside he box — and the butterfly seems to grow and shrink exactly as I would expect. This is what they said they would show me. The eight hours I needed to spend in a car driving to and from this demo in San Jose were not a waste of time.

“There are other volumetric display technologies that may leverage a surface to form 2D-only pixels in space (e.g. smoke, water, screens, mirrors, moving surfaces, etc.),” Karafin said. “However, these are not holograms.”

Real Holograms

Karafin really baked my noodle next.

He grabbed a magnifying lens off a tray and placed it inside the butterfly. How the fuck does a standard magnifying glass warp light from the inside of a holographic object? My eyes and brain had never had to process the way light was behaving relative to the lenses he was putting “through” what my brain perceived as a physical solid object.

“When using the magnifying glass in front of the butterfly, it responds exactly as it would in the real world,” Karafin explains. “When you pass the magnifying glass through the hologram, you see things that only a real hologram can achieve.”

Ok, so now the trip was worth it.

Next Steps

I noted deficiencies in the visuals of the display only outside the optimal viewing cone provided by the small display. Everything inside that cone — at least a meter away — looked great. I should note they also showed a moving fish that was less detailed than the butterfly, and as certain pieces of it moved closer to me I noted a bit of fuzzy softness there.

“The amount of movement depends on how close you are to the hologram, the size of the holographic surface, and the optical prescription for the holographic waveguides,” Karafin explained.

The field of view for the Light Field Lab display prototype is very small, but the units should be able to be stacked up like bricks for much larger panels.

I left the room and realized I should have asked to see the demo with the lights on. Light Field Lab representatives were crystal clear in saying that I would see something small float believably in front of me with no glasses, goggles or head tracking of any kind. But they also have a lot to prove with so many other volumetric display technologies promising things which can’t be delivered. Many competing ideas for Light Field Lab are low resolution when viewed up close, like Looking Glass. That’s why Karafin showed me up close how lenses affect his holograms.

“The holographic volume is two inches, but that’s only limited by the submodule prototype,” Karafin said. “That grows substantially in product.”

Eventually enough of the Light Field Lab “bricks” could be stacked up for an entire holographic wall, or room.

LFL representatives believe the first use cases for this tech will be for location-based entertainment and education, like theaters, museums or theme park attractions. I didn’t expect the lights-on demo to be impressive and it wasn’t, but it did give me a good look at the overall rig. Photographs weren’t allowed, but the panel looks a lot like it does in the image at the top of this post except that it is mounted on a tall server rack.

Bricks In A Wall

The word “brick” is useful here because that’s how I’d describe the box from which light emerges.

Karafin and crew believe they’ve uncovered a “recipe” for producing proper light fields from one side of this brick. My demo — reminiscent in some ways of my first Oculus Rift demo back when the company just had seed funding in late 2012 — was about showing me a proper light field display and proving it to me that it is real. Light Field Lab accomplished that. The next phase for the company is about expanding the size of the area where the full holographic effect can be viewed by stacking them up like bricks in a wall. That also means the tower portion of the installation needs to go away.

“We already do everything within the prototype that we will do to manufacturer the product,” Karafin said.

So next year my expectations for Light Field Lab is to see the same demo with a friend able to stand side by side with me. I want to observe a couple live butterflies with the same visual acuity as 2018 but with only wires running to a couple of these bricks with the computer towers in a separate area.

Karafin says that’s the plan. Light Field Lab is also working with rendering company OTOY to build out a content development pipeline for the technology.

“You will be able to move octane light field objects interactively on the display in a month once we hook up our custom build of our viewer to the panel,” OTOY CEO Jules Urbach said. “Been waiting to get multiple RTX 6000 before going up to LFL to finish this work.”

Last year Karafin said they would use their first funding to deliver a proof-of concept hardware prototype. Earlier this year they announced $7 million in funding to complete it. This week I looked at it. If some portion of $7 million is what it took to produce what I saw, then I need to see what Light Field Lab does with more money and another year to build a much bigger device.

Karafin showed slides before the demo depicting various  possible configurations.A holographic table so you can have something like Holochess from Star Wars, or a holographic theater where a shark seems to come out into the audience like Jaws in Back To The Future Part II. Or perhaps one day many years from now a fully holographic room — a holodeck — where redirected walking techniques and haptic effects might be used to simulate touch in a seemingly endless space. Karafin’s slides also include explanations of why most of these holographic pop culture reference points — including R2-D2 projecting Leia into the open air — probably can’t work in the real world. As Karafin explains, you need a surface behind the holographic volume “spraying” out light which enters the eye as naturally as if it had bounced off a real object.

Any doubt I have (and I don’t have too much of it now) the company can deliver this ground-breaking display technology to locations like theme parks will evaporate whenever they announce more funding and I see Light Field Lab successfully stack these units together.

Now Boarding: Hype Train To The Holodeck

A toy mirascope.

I’m handed a bag at the end of my meeting with a cheap plastic toy inside that was made in China. It is called a mirascope. Karafin says it is the analog equivalent of what they are doing with each point of light from their display.

“The mirascope is a dual parabolic reflector that forms a relayed real image from a real object. The real image you see results from the convergence of ray bundles, identical to how our system collimates and converges energy to form the hologram,” Karafin said. “Essentially, the mirascope illustrates how a real image is formed — but we replace the real object and mirascope with a completely solid state, holographic, digital projection backplane.”

I brought their little toy home and it completely blew my family away to try and grab the gem and discover it really isn’t there (it is at the bottom of the mirascope). The toy taught my household both the Light Field Lab name and its promise of true glasses-free virtual worlds not too far off from the Holodeck.

Next time I make that trip to see Light Field Lab, I’m bringing my family.

“Our vision is to enable the Holodeck, which makes the depictions of holographic interfaces in science fiction a reality,” Karafin said.

Tagged with: , , , ,

The post Eyes-On: Light Field Lab Built A Tantalizing Holographic Display For The 2020s appeared first on UploadVR.

OTOY Partners With Light Field Lab On Holodeck Display Tech

OTOY Partners With Light Field Lab On Holodeck Display Tech

Los Angeles-based rendering company OTOY is teaming up with startup Light Field Lab to build a pipeline for holographic content creation and display.

Light Field Lab raised $7 million earlier this year to complete a prototype of its display technology which is meant to fulfill a few key features of Star Trek’s holodeck. Namely, Light Field Lab’s technology could theoretically be equipped along the walls of a room to show multiple people optically correct virtual worlds without any specialized eyewear. According to Light Field Lab CEO Jon Karafin they’ve been demonstrating prototypes to “key industry leaders throughout the year.”

“No headgear, no head-tracking, no vergence/accommodation conflict, no motion latency, no headaches or eye strain, nor other artifacts associated with other non-holographic technologies,” Karafin wrote in an email. “The panels are flat panel, solid state and directly emissive— meaning there are no moving parts or arrays of projectors or cumbersome accessories. When you see the holographic objects with the prototype system, your eye naturally focuses on the actual object in space, and your brain tells you it’s floating right there in front of you — because it is a dense holographic projection with converging ray bundles that form real images.  The current prototype represents the foundational building block that produces large format panels, walls, and surfaces as we head into manufacturing next year.”

Realizing Star Trek’s holodeck is also the goal of OTOY’s founder Jules Urbach, whose technology is used by creatives to render photo-realistic visuals for various purposes, like the haunting opening sequence of Westworld.

Urbach’s work crosses into a lot of different areas and he’s also the driving force behind RNDR — a blockchain-based token built around the idea of distributing rendering tasks to unused graphics processing units around the world.

The partnership between Light Field Lab and OTOY aims to combine all these technologies to supply the visuals emerging from Light Field Lab displays. We haven’t seen Light Field Lab’s technology yet but I’ve spent a fair amount of time with Urbach and he believes they’ve had a breakthrough:

When you wrote that article, I knew this was where we wanted to be a year out. I was waiting for the Light Field Lab prototypes and to see them with my own eyes, running our content, before being certain this was going to work. We also very much need the scale that RNDR provides for this pipeline to be feasible (both for pre-computing LF data and composing and streaming LF media to the display).

The principle behind the Light Field Lab displays is something that many have considered over the years as a path to holographic panels – but getting 1000’s of DPI and the optics/mechanics of bending rays without showing tiling is mechanically very difficult and so daunting (especially in full color, and full motion) that few thought it might be possible even in prototype form today. I have tested enough of our stack on the 6×4 inch prototype to see that LFL has solved the hardest problems needed to make this work – tiling is one of these issues. If you can tile the displays like LED walls, there is no limit to how big or what configuration that can be, hence the holodeck is viable.

According to Karafin, Light Field Lab partnered with OTOY because Urbach “is one of the most brilliant people I have ever worked with, and we have a shared vision to enable the Holodeck together. OTOY has industry-leading physics-based GPU rendering technologies that are already integrated into most of the content creation tools that enable content creators to produce holographic objects extremely efficiently. We will start with smaller scale installations with these holographic objects, characters and experiences to mature the technology and ecosystem with the ultimate goal of enabling the Star Trek Holodeck.”

Content to be shown on these new displays is in active development by Rod Roddenberry (Executive Producer on Star Trek: Discovery and son of Star Trek creator Gene Roddenberry) as well from OTOY investor Ari Emanuel, who is also the CEO of Endeavor (one of Hollywood’s most influential talent companies).

“OTOY has created the capture, rendering and streaming technology for the industry to transition to holographic content development, as well as the RNDR blockchain for IP rights and distribution,” Emanuel said in a prepared statement. “We’re excited to use this platform to bring true holographic content to Light Field Lab’s displays, which will give consumers unbelievable experiences, without the burden of 3D glasses or VR headsets.”

Tagged with: , ,

The post OTOY Partners With Light Field Lab On Holodeck Display Tech appeared first on UploadVR.

Hands-on with the Holodeck: Ready Player One – Escape Room

The motion-picture release of Ready Player One has, unsurprisingly, a number of virtual reality (VR) tie-ins to accompany it. One of those which home users may not get to jump into any time soon is NVIDIA’s own Ready Player One – Escape Room, which premiered at the company’s GTC 2018 event, San Jose.

Ready Player One HeroAs the name suggests, Ready Player One – Escape Room is a puzzle challenge set in a single room. In the version available at GTC 2018 up to three people could enter the space at one time, playing with HTC Vive Pro head-mounted displays (HMDs), with the idea that they would work together to solve a series of puzzles by interacting with objects within a virtual recreation of an area from the Ready Player One universe; namely a living room with a decidedly 1980s feel.

The objects players can interact with are limited; if you can touch it, it’s likely part of a puzzle you will have to solve. The team of players has 10 minutes to search the room for the correct objects at the correct time, and with even a small amount of knowledge of 1980s popular culture the challenges are relatively simple.

To begin with, there are a group of VHS cassette tapes laid on a counter with series of chalk outlines for those which are missing, with the task simply being to find and place the remaining tapes. Solving this clue makes a Batman symbol project from a Rubik’s Cube, requiring you to insert the VHS cassette tape of Tim Burton’s 1989 Batman motion-picture into the VHS player, which then makes a poster fall off the wall to reveal a Thundercats logo. As you can tell from this short explanation (which sadly is about half of the experience) the puzzles are far from challenging.

Ready Player One - HolodeckOf course, with NVIDIA’s Holodeck it’s less about the pre-designed experience and more about the player interaction. To that end, Ready Player One – Escape Room is one of the most enjoyable co-operative VR videogames around. With full voice communication and the ability to hand objects to one another, Ready Player One – Escape Room’s game of what is essentially an Easter egg hunt is most certainly collaborative. Six eyes are better than two, after all.

Ready Player One – Escape Room is also a visual treat, aptly showcasing the additional clarity in resolution between the HTC Vive Pro and the HTC Vive. Being able to read the text on posters and magazine covers as well as make out incidental details in the environment – showcasing the attention to detail of the artists who worked on the project – makes a huge difference to the level of immersion. A question has to be asked regarding the design of the players’ avatars – the clean white-and-green visage is certainly a modern interpretation and ill-fitting with the 1980’s aesthetic prevalent elsewhere – but it’s unlikely that many partaking in Ready Player One – Escape Room as one of their first VR experiences will suffer much umbrage from this disparity.

With social VR having become a key talking point within the industry of late it’s important that co-operative experiences such as Ready Player One – Escape Room take centre stage, showcasing that even the simplest of experiences can be improved by the added human factor. It’s a shame then, that NVIDIA doesn’t currently have any plans to showcase the piece within the public domain. This may of course change down the line, and VRFocus most certainly hopes that in time an expanded version of Ready Player One – Escape Room will be offered to VR’s early adopters at home.

 

 

Driving a Real Car in VR Takes a Step Towards Reality with NVIDIA’s Holodeck

Racing around tracks in virtual reality (VR) can be great fun, really showcasing how immersive the technology can be. But how about taking that principle and combining it with the real world, so you’re actually driving a real car. Well that’s just what NVIDIA has been showing off today during CEO Jensen Huang’s keynote address at the GPU Technology Conference (GTC) 2018.

Nvidia Holodeck Car1

A big part of NVIDIA’s keynote address was focused on self-driving cars and its AI autonomous vehicle platform DRIVE, which has been designed to help automakers create safe self-driving vehicles by putting the AI through a series of virtual environments, simulating various hazards along the way.

But that’s not all the company has been working on. Combining NVIDIA’s Holodeck with a VR setup including a HTC Vive, racing seat and steering wheel, plus a car kitted out to control it remotely, NVIDIA showcased someone inside the convention hall driving a car parked outside whilst wearing the headset.

What they could see was a digital representation of the car inside Holodeck, with cameras side the vehicle providing a view of the outside world. They could then drive the car around as if they were playing an actual videogame, albeit very slowly as they were in a car park and didn’t want to hit anyone.

Nvidia Holodeck Car2

It’s certainly an interesting use of the technology and could have massive uses in the future when it comes to sports. Drivers would no longer be in danger from crashing so it would become very safe. As such safety features for the cars could be dispensed with, making them lighter and faster. On the extreme side, emulating films like Death Race, the cars could be equipped with all sorts of offensive and defensive capabilities to make it a visual spectacle for the audience.

Drone racing for example already uses VR headsets so that racers all sit together in one location, while the drones are flown around a set course in a warehouse. With NVIDIA’s tech this could just be upscaled to bigger vehicles.

Of course there a far more practical and serious applications for the technology. Being able to transport yourself inside an autonomous vehicle or robot that can reach dangerous areas or to help save a life. Whatever happens, VRFocus will always keep you updated on the latest use cases for VR.

Ready Player One Gets the NVIDIA Holodeck Treatment at GTC 2018

Today marks the start of NVIDIA’s GPU Technology Conference (GTC) 2018 event in San Jose, California, with the technology manufacturer ready to start four days of talks and sessions focused on virtual reality (VR), AI, graphics cards and a whole lot more. To get the ball rolling, and with Ready Player One due to launch this weekNVIDIA is treating attendees to a VR experience on the company’s Holodeck.

Ready Player One Hero

Teaming up with Warner Bros. and HTC Vive, NVIDIA Holodeck will use 3D assets from Ready Player One, helping transport players to the year 2045 and “Aech’s basement,” where, in an escape room-style experience, they join a quest in which solving one puzzle triggers the next. Teams will have to work together to complete the challenge within the allotted time to win.

“Combining physics with natural interactions, NVIDIA Holodeck creates a virtual world that looks and feels real to players, who can interact with virtual objects while exploring richly detailed scenes. It imports complex, detailed models consisting of tens of millions of polygons in real-time VR, making it the perfect environment to showcase the cinema-quality assets of Ready Player One,” states NVIDIA.

Holodeck was first unveiled during last years keynote address by founder and CEO, Jensen Huang. Designed as a VR collaboration platform that brings people together from around the world in ultra-realistic virtual experiences, Holodeck  has previously been demonstrated by Swedish supercar maker Christian Koenigsegg.

Then in October 2017 NVIDIA launched the application process for product designers, application developers, architects or any other type of 3D content creator to sign up for early access.

GTC 2018 is likely to feature much more of Holodeck as the week progresses, with NVIDIA having previously mentioned plans to address deep learning techniques in virtual environments, including capabilities for AI-based training, simulation and content creation using the platform. VRFocus will be at GTC 2018 to bring you all the latest news and announcements as they happen.

Early Access now Available for NVIDIA’s Holodeck VR Platform

During NVIDIA’s GPU Technology Conference (GTC) earlier this year the company revealed Project Holodeck to the world, a virtual reality (VR) platform that would allow users to build and explore creations together in a VR environment, wherever they are in the world. Today it has announced that applications are now open for the early access version.

Whether you’re a product designer, application developer, architect or any other type of 3D content creator, NVIDIA Holodeck offers an immersive, photoreal experience with real-world physics so that customers can streamline their review processes and reduce the need for physical prototyping when dealing with scenes or models for example.

Holodeck-web-672x448

“NVIDIA Holodeck empowers designers to bring peers, partners and customers along the design journey to explore intricate, life-like 3D worlds together and ensure that the best ideas are discovered,” said Bob Pette, vice president of Professional Visualization at NVIDIA in a statement. “It’s an unparalleled environment for deploying and testing AI-based agents — and will only get better as we add more AI capabilities.”

For design professionals, NVIDIA Holodeck early access is the first opportunity for them to experiment with the platform’s capabilities and help guide future development. For those interested, sign up for Holodeck early access here.

“During our design process, teams of our engineers and scientists work together to imagine an idea, plan a design, create that model, experiment and test that solution, then take time to reiterate and improve the original – all steps that are crucial to mission success at NASA,” said Frank Delgado, lead for NASA’s Hybrid Reality Lab. “With Holodeck, we will be able to clearly visualize our models, easily collaborate in a physically simulated environment, and review to ensure the efficiency and safety of our designs.”

In the future, NVIDIA’s Holodeck updates aim to address deep learning techniques in virtual environments, including capabilities for AI-based training, simulation and content creation.

For further updates, keep reading VRFocus.

Sony Magic Lab: VR auf dem Weg zum Holodeck

In einem Interview mit Glixel verrät Dr. Richard Marks, wie er sich die Zukunft von VR vorstellt. Vor allem in der Spracherkennung sieht der Leiter des Sony Magic Lab R&D-Teams unglaubliches Potential auf dem Weg zum Holodeck: Natürlich gesprochene Anweisungen sollen nicht nur die Navigation in künstlichen Welten ermöglichen, sondern mit ihnen könnten Anwender in Zukunft auch jedes Element der virtuellen Erfahrung in Echtzeit aufrufen.

Von Black Mirror zum Holodeck

Über die Weihnachtsfeiertage hat Marks von Magic Lab sich mit einem Staubsaugerroboter sowie Amazon Echo und Google Home beschäftigt. Garniert wurde die Forschungszeit mit der Serie Black Mirror, die einen kritischen und satirischen Blick auf die sozialen Aspekte der technischen Entwicklung wirft. Die drei Staffeln der ursprünglich britischen Serie lassen sich übrigens auf Netflix ansehen. Marks Ziel ist das Holodeck, welches teilweise aus fertigen, hochwertigen Inhalten besteht sowie in Echtzeit berechneten Elementen. Um sie erscheinen zu lassen, verwenden Anwender lediglich ihre Stimme und geben natürlich gesprochene Anweisungen, was man in der virtuellen Welt erleben will.

Zwar gäbe es bereits Apps wie Virtual Desktop und Oculus Home, bei denen man via Spracheingabe VR-Software starten könne, allerdings geht Marks darüber hinaus: Cybernauten könnten alles aus einer riesigen Bibliothek in der virtuellen Realität entstehen lassen und mit den künstlichen Charakteren interagieren. Wenn man beispielsweise sagt: „Bring mich zur Burg“, Schwupps, wäre da eine Burg. „Mit Zugbrücke, bitte“ – alles klar, Zugbrücke wäre da. Er gibt zu, dass das alles nach Science Fiction klingt, aber es sei nicht zu sehr Science Fiction.

Marks spricht auch über den VR-Titel Star Trek: Bridge Crew, der Ende Mai erscheinen soll, und betont, wie wichtig soziale Interaktion in der virtuellen Realität sei. Das Gefühl der Präsenz anderer Mitspieler wäre wichtig und würde sich im Laufe der Zeit verbessern, je mehr Spieler den Weg in VR finden. Aber auch künstliche Charaktere würden immer eine Rolle spielen und die Ansprüche an sie wären in VR noch höher. Im Magic Lab hätte das Team bereits eine Demo eines „glaubhaften Charakters“ in Form eines Roboters, mit dem man über Gesten und Körpersprache interagieren könne.

Das ganze Interview mit weiteren Details findest du auf Glixel.

(Quelle: RoadToVR)

Der Beitrag Sony Magic Lab: VR auf dem Weg zum Holodeck zuerst gesehen auf VR∙Nerds. VR·Nerds am Werk!

Project Holodeck From NVIDIA Is A Social VR Space With Realistic Physics

Project Holodeck From NVIDIA Is A Social VR Space With Realistic Physics

NVIDIA CEO Jensnen Huang took the stage at GTC 2017 today to share what the GPU manufacturer is up to concerning deep learning, ray tracing, and virtual reality.

To this latter point, Huan announced Project Holodeck — an upcoming social VR Experience that will allow multiple users to occupy a single digital space.

To demonstrate Project Holodeck, Huan showed a feed of four NVIDIA representatives inside of the experience. Together, they examined and discussed a high-resolution model of a sports car. The four then began to interact with the model directly by opening the doors and entering the car. Huang then requested a closer look at the sports cars internal parts. On command, the car exploded into hundreds of pieces and parts that could be studied individually.

Shared experiences and car visualizers are nothing new for VR. However, Huang explained that what will set Project Holodeck apart is its focus on photo-realistic graphics and consistent, believable physics.

Most readily available social VR experiences now prioritize connectivity over image fidelity. As a result, things like Altspace VR and Facebook Spaces utilize low-poly avatars and assets in order to ensure a quick connection over the internet. NVIDIA’s social VR experiment, however, will be committed to making the visuals as photo realistic as possible, according to Huang.

In addition to realistic visuals, Huang added that Project Holodeck’s sense of realism will be protected through a suite or believable physics that are consistently applied. For example, in Project Holodeck when you want to touch the steering wheel of a car with your hand, your avatar’s hand will be stopped by the wheel rather than passing through it. Objects will have a rigidity inside Project Holodeck that NVIDIA hopes will make spending time inside of it more enjoyable and believable.

NVIDIA has already shown off this type of physics-based VR in its VRWorks collection of demos.

Project Holodeck will be available for early access in September, 2017.

Tagged with: ,