Category: 3DHBR

Posted on

Fraunhofer HHI Unveil Volumetric Video Studio to aid Rapid Market Development of VR/AR

Last month VRFocus reported on the Fraunhofer Institute for Telecommunications (Fraunhofer HHI) developing a new volumetric video variant known as 3D Human Body Reconstruction (3DHBR) to create naturally moving dynamic 3D models. Now the institute has announced a specialist volumetric video studio will be put into operation after an initial launch in October 2017.

VoluCap GmbH

Called VoluCap GmbH, it was founded by the shareholders ARRI Cine Technik GmbH & Co. KG, Fraunhofer-Gesellschaft, Interlake System GmbH, Studio Babelsberg AG and UFA GmbH. The creation of the studio was down to the necessity to relocate the commercial production to a company is due in particular to the rapid market development of virtual reality (VR) and augmented reality (AR) and the high demand for volumetric video content.

In November 2017, Fraunhofer HHI carried out the first professional commissioned production for VR experience Ein ganzes Leben (A Whole Life), which premiered at the Deutsche Kinemathek, the film and television museum in Berlin. The company plans on showcasing additional productions from the volumetric video studio to the public in several talks and presentations at this year’s Berlinale.

Plans are underway for the studio to be put into full operation as a commercial volumetric video studio in mid-2018, possibly the first on European continent.

Gateway to Infinity screenshot

The 3DHBR technology which VoluCap GmbH will be using captures real persons with multiple cameras at the same time and creates naturally moving dynamic 3D models, which can be observed from arbitrary viewpoints in the virtual scene. The HBR capture system consists of 16 stereo cameras that create 3D information from all different viewpoints around the person, similar to how humans perceive with their two eyes. This information is fused and transformed to a consistent, natural and dynamic 3D representation of the person.

Beyond the processing of dynamic 3D models, an integrated multi-camera and lighting system for full 360-degree acquisition of persons has been developed. The system enables a controlled, freely configurable and diffuse illumination of the persons, which allows both a subsequent synthetic illumination of the dynamic 3D-models in VR worlds (re-lighting), as well as a targeted dynamic scenic illumination with special lighting moods (campfire, blue light, etc.). As VRFocus learns more about  Fraunhofer HHI’s for 3DHBR we’ll keep you updated.

Posted on

Fraunhofer HHI Develop Volumetric Human Body Reconstruction Technology

Volumetric video is regarded as the next important development step in the field of media production, especially in the context of the virtual reality (VR) and augmented reality (AR) industries. The researchers from Fraunhofer HHI have developed a new variant of this technology known as 3D Human Body Reconstruction (3DHBR). 3DHBR transmits the realistic image of a person into a virtual world.

Gateway to Infinity screenshot

3DHBR technology captures real persons with multiple cameras at the same time and creates naturally moving dynamic 3D models, which can be observed from within the virtual scene. The volumetric video capture system consists of 16 stereo cameras that create 3D information from all different viewpoints around the person, similar as humans perceive with their two eyes. This 3D information is fused and transformed to a consistent, natural and dynamic 3D representation of the person.

Post-processing modules adapt the data structures and formats and allow for direct integration in standardised post-production workflows and VR software. In contrast to classic animation of virtual characters, facial expressions as well as moving clothes are captured from video information and reconstructed at high geometrical detail and texture quality, thereby a naturalness of the visual impression is achieved.

The latest version of the technology features a complete workflow and is fully automatic. Due to the standardised output format, further post-processing of the dynamic 3D model is possible where previously the image that is produced had been far too large and complex for most software to handle.

Gateway to Infinity screenshot

Beyond the processing of dynamic 3D models, an integrated multi-camera and lighting system for full 360-degree acquisition of persons has been developed. The system supports diffuse lighting from any direction, automatic keying and flexible multi-camera arrangement. 3DHBR does not require a green screen, and thus provision of diffuse lighting from all directions can offer the best possible conditions for re-lighting of the dynamic 3D models at the design stage of the VR experience.

The 3DHBR technology was used for the first time in cooperation with UFA GmbH in the course of a joint test production for the immersive film Gateway to Infinity as a volumetric VR experience. In this test production, set in a science fiction world on a foreign planet, the 3D models were created based on real actors so that gestures, facial expressions and textures (skin, hair and fabric) could be recorded in detail. The 3D models of people were subsequently integrated into a virtual scene, and viewers with VR head-mounted displays (HMDs) were immersed directly in the scene, experiencing the story close up and personally.

The next steps for Fraunhofer’s 3DHBR technology are not yet known, however the company is showing interest in expanding its portfolio of VR experiences. When the technology will become available to be licensed to development studios working within the VR medium is not currently known, but VRFocus will keep you updated with all the latest details.