Fraunhofer Demonstrates VR Microdisplay Prototype

Last month, European research and development firm Fraunhofer announced it was working on using microdisplays in virtual reality (VR) headsets in a bit to improve image clarity, reduce the weight of the headset and even eliminate simulation sickness. Now the first reports of how its prototype is faring have begun to emerge.

A German VR specialist publication called VRODO managed to get hands-on with the nameless prototype device and reported on its findings.

Apparently the device uses four OLED microdisplays, each offering a resolution of 1,920 x 1,200, which equated to 2,300 pp, or pixels per inch, with a frame rate of up to 120 Hz.

The headset uses two OLED/CMOS WUXGA microdisplayer per eyes, which is capable of offering a potential maximum resolution of 4,800 x 1,920 pixels. VRODO reports that the displays being shown at the AugmentedWorld Expo (AWE) were running at only 60Hz, which may have been to make sure the GPU was able to meet the huge requirements for such high resolution and frame rate.

Project partner Limbak provided the optical system, and the fused optics of the headset are said to present the user with a seamless view, with up to 100 degree horizontal field-of-view, and a 75 degree vertical field of view, which matches up to what the HTC vive and Oculus rift offer horizontally, though it is smaller on the vertical plane. However, the prototype is said be be lighter than either the HTC Vive or Oculus Rift, with some reports saying it is only a quarter of the weight.

There VRODO report does mention there are some problems with optical distortion and blurriness, but the device is still an early prototype, make using 3D printing, and reportedly only assembled fully four weeks ago.

Some analysts are predicting that some VR companies may choose to go down the route of VR microdisplays, chasing the promise of lighter, more comfortable head-mounted displays. AS usual, VRFocus will keep you updated on new developments.

Fraunhofer Reveals Plans To Use Microdisplays To Overcome VR Motion Sickness And Headaches

One of the most comment problems that a small but significant number of users encounter when using virtual reality (VR) is that of motion sickness and headaches. This is something that researchers and developers have been trying to tackle since the early days of the technology and one that now Fraunhofer believe can be dealt with by using high-speed microdisplays that are positioned closer to the users’ eyes.

Fraunhofer Institute VR microdisplay

As reported by venturebeat, the work is being funded by a European project to popularize tiny but cost-effective high-speed screens which the German R&D institute will put on display next month. The key to the innovation is a chip design that enables multiple high-resolution OLED-on-silicon microdisplays to work quickly together at high 120Hz framerates. The displays will also offer a 100,000:1 contrast ratio and ultra-low power consumption and will also include a mode that eliminates flicker and motion artifacts. Because of their speed and quality, the displays promise to reduce motion sickness and headaches.

Fraunhofer are also trying to get more pixels out of these new one-inch microdisplays than that of a 1080p display by offering a 1920 x 1200 pixel resolution with a density of 2300 pixels per inch. As two of these displays are used per eye, four displays are used in total with a total pixel resolution of 4800 x 920, which offers a wide field-of-view of over 100 degrees.

Fraunhofer Institute VR microdisplay

These screens are then paired with a new optics system which has been designed by Limbak, a third-party partner on the project. Your typical VR headset will have screens placed around 60 to 75 millimeters from a user’s eyes, were Limbak’s optics will cut the distance down to 37 millimeters, reducing the overall size of the headset. This is one are that Fraunhofer are keen to see developments made with an expect headset weight with this solution to be around three-quarters smaller than a typical headset, without compromising the field-of-view or resolution.

Fraunhofer are planning to exhibit the new displays and a prototype headset at Munich’s AWE Europe 2018 event on October 18th and 19th. Though details such as release date and pricing have yet to be revealed, VRFocus will be sure to bring you all this and more when it becomes available.

Fraunhofer HHI New Mixed Reality Technology Allows For Natural Interaction Between Worlds

The future of virtual reality (VR) will be allowing people to interact with each other with more natural interaction, similar to the real world. This is the goal for Fraunhofer Heinrich Hertz Institute (HHI) who are planning to demonstrate a new technology that makes it possible for interaction and feedback to be transferred between the real world and the virtual world. at CeBIT.

 

The power of VR, augmented reality (AR), and mixed reality (MR) allows for people to collaboration regardless of the distance between them, using applications to engage in a range of activities. In the example of a remote assistance working with a technician to repair equipment, the future technology that Fraunhofer HHI are working on could allow for the assistance to provide scenery input to the repair. This includes hand movements and feedback on the surrounds of the technician, with more applications made possible depending on the situation.

“The solution we developed for this purpose can connect a simulated world with the real world in real time and high quality and open up new perspectives or collaborations,” explains Paul Chojecki, project manager at Fraunhofer HHI. “Physical interaction without disturbing controllers is more natural and more comfortable. The solution can be adapted more flexibly to the height of the user and increases immersion. At the same time, it can reduce symptoms of motion sickness that are often caused by VR scenarios.”

In the real world, eight cameras, four pairs each, capture the scene from all sides and generate depth maps with up to 30 Hertz. Gestures and dynamic movements are detected and the data is combined by algorithms, coded and transmitted to the VR station in real time, with the corresponding 3D textures. For the user wearing the VR headset, another 3D camera records the users. Thanks to the Fraunhofer HHI algorithms for 3D body detection and gesture interpretation, it is naturally interacted in the VR scene without disturbing controllers or markers.

The feedback from the virtual world is displayed in the real scene by means of a projection, with special image processing algorithms from Fraunhofer HHI used for this AR projection. This combination of two exciting technologies results in an impressive and immersive experience which offer a unique solution for MR interaction and applications.

VRFocus will be sure to bring you all the latest on this development from Fraunhofer HHI in the future so stay tuned for more.

Fraunhofer HHI To Present The Future Of Immersive Imaging At NAB 2018

The Fraunhofer Heinrich Hertz Institute (HHI) is presenting their current innovations in the field of immersive imaging technologies at the NAB 2018 event in Las Vegas.

As resolution and framerates increase so to does the data needing to be streamed and requiring compression. Current compression encoding such as High Effficieny Video Coding standard (HEVC) is only just capable of managing the task and this is where Fraunhofer HHI come in.

As part of a collaboration between the ITU Video Coding Expert Group (VCEG) and the ISO/IEC Moving Pictures Expert Group (MPEG) known as the Joint Video Experts Team (JVET). Fraunhofer HHI submitted a proposal for a cutting edge coding technology to address the problem and be included in the final standard by 2020.

At NAB 2018, Fraunhofer HHI will be displaying the codec to the publich for the time and already shows significant coding efficiency improvements over HEVC for content ranging from standard High Definition (HD) to High Dynamic Range Ultra-HD content. This also means that it is far more suited to the likes of 360-degree video and virtual reality (VR) application thanks to the improved efficiency. At NAB 2018 Fraunhofer HHI will also present the world’s first demonstration of continues live video streaming of VR 360-degree videos with a resolution beyond 4K.

OMNICAM-Fraunhofer

This demonstration will be made possible thanks to high-resolution VR 360-degree 10K video capturing and live rendering from the Fraunhofer HHI Omnicam-360, tile-based live encoding with the Fraunhofer HHI HEVC encoder, packaging according to the MPEG-OMAF Viewport-Dependent Media Profile and high-quality playback on VR glasses and TV screens. All of this is made possible thanks to the continued improvements and developments that Fraunhofer HHI have made within the immersive technology sector.

Elsewhere, Fraunhofer HHI has developed a new and uniquely integrated 360-degree multi-camera capture and lighting system for the creation of highly realistic Volumetric Video content of moving persons. This is achieved thanks to a set of 16 stereo cameras that capture and create 3D information from all different angles, before it is then processed to create a natural and dynamic 3D representation of the person. This process is completely automatic meaning the generated meshes can be integrated directly into VR and augmented reality (AR) applications.

For more on immersive technology and Fraunhofer HHI in the future, keep reading VRFocus.

Fraunhofer HHI Unveil Volumetric Video Studio to aid Rapid Market Development of VR/AR

Last month VRFocus reported on the Fraunhofer Institute for Telecommunications (Fraunhofer HHI) developing a new volumetric video variant known as 3D Human Body Reconstruction (3DHBR) to create naturally moving dynamic 3D models. Now the institute has announced a specialist volumetric video studio will be put into operation after an initial launch in October 2017.

VoluCap GmbH

Called VoluCap GmbH, it was founded by the shareholders ARRI Cine Technik GmbH & Co. KG, Fraunhofer-Gesellschaft, Interlake System GmbH, Studio Babelsberg AG and UFA GmbH. The creation of the studio was down to the necessity to relocate the commercial production to a company is due in particular to the rapid market development of virtual reality (VR) and augmented reality (AR) and the high demand for volumetric video content.

In November 2017, Fraunhofer HHI carried out the first professional commissioned production for VR experience Ein ganzes Leben (A Whole Life), which premiered at the Deutsche Kinemathek, the film and television museum in Berlin. The company plans on showcasing additional productions from the volumetric video studio to the public in several talks and presentations at this year’s Berlinale.

Plans are underway for the studio to be put into full operation as a commercial volumetric video studio in mid-2018, possibly the first on European continent.

Gateway to Infinity screenshot

The 3DHBR technology which VoluCap GmbH will be using captures real persons with multiple cameras at the same time and creates naturally moving dynamic 3D models, which can be observed from arbitrary viewpoints in the virtual scene. The HBR capture system consists of 16 stereo cameras that create 3D information from all different viewpoints around the person, similar to how humans perceive with their two eyes. This information is fused and transformed to a consistent, natural and dynamic 3D representation of the person.

Beyond the processing of dynamic 3D models, an integrated multi-camera and lighting system for full 360-degree acquisition of persons has been developed. The system enables a controlled, freely configurable and diffuse illumination of the persons, which allows both a subsequent synthetic illumination of the dynamic 3D-models in VR worlds (re-lighting), as well as a targeted dynamic scenic illumination with special lighting moods (campfire, blue light, etc.). As VRFocus learns more about  Fraunhofer HHI’s for 3DHBR we’ll keep you updated.

Fraunhofer HHI Develop Volumetric Human Body Reconstruction Technology

Volumetric video is regarded as the next important development step in the field of media production, especially in the context of the virtual reality (VR) and augmented reality (AR) industries. The researchers from Fraunhofer HHI have developed a new variant of this technology known as 3D Human Body Reconstruction (3DHBR). 3DHBR transmits the realistic image of a person into a virtual world.

Gateway to Infinity screenshot

3DHBR technology captures real persons with multiple cameras at the same time and creates naturally moving dynamic 3D models, which can be observed from within the virtual scene. The volumetric video capture system consists of 16 stereo cameras that create 3D information from all different viewpoints around the person, similar as humans perceive with their two eyes. This 3D information is fused and transformed to a consistent, natural and dynamic 3D representation of the person.

Post-processing modules adapt the data structures and formats and allow for direct integration in standardised post-production workflows and VR software. In contrast to classic animation of virtual characters, facial expressions as well as moving clothes are captured from video information and reconstructed at high geometrical detail and texture quality, thereby a naturalness of the visual impression is achieved.

The latest version of the technology features a complete workflow and is fully automatic. Due to the standardised output format, further post-processing of the dynamic 3D model is possible where previously the image that is produced had been far too large and complex for most software to handle.

Gateway to Infinity screenshot

Beyond the processing of dynamic 3D models, an integrated multi-camera and lighting system for full 360-degree acquisition of persons has been developed. The system supports diffuse lighting from any direction, automatic keying and flexible multi-camera arrangement. 3DHBR does not require a green screen, and thus provision of diffuse lighting from all directions can offer the best possible conditions for re-lighting of the dynamic 3D models at the design stage of the VR experience.

The 3DHBR technology was used for the first time in cooperation with UFA GmbH in the course of a joint test production for the immersive film Gateway to Infinity as a volumetric VR experience. In this test production, set in a science fiction world on a foreign planet, the 3D models were created based on real actors so that gestures, facial expressions and textures (skin, hair and fabric) could be recorded in detail. The 3D models of people were subsequently integrated into a virtual scene, and viewers with VR head-mounted displays (HMDs) were immersed directly in the scene, experiencing the story close up and personally.

The next steps for Fraunhofer’s 3DHBR technology are not yet known, however the company is showing interest in expanding its portfolio of VR experiences. When the technology will become available to be licensed to development studios working within the VR medium is not currently known, but VRFocus will keep you updated with all the latest details.