Display manufacturer Kopin recently demonstrated its latest VR display and pancake optic which promises higher resolution and more affordability for future VR headsets.
Most modern VR headsets take on the ‘box on your face’ form-factor because of a simple display architecture which necessitates a certain distance between the display and the lens. In the effort to make VR headsets more compact in the near-term, so-called ‘pancake optics’ are emerging as a leading candidate. These more complex optics reduce the distance required between the display and the lens.
It’s natural to wonder why even the latest VR headsets are essentially just as bulky as the first generation launched back in 2016. The answer is simple: optics. Unfortunately the solution is not so simple.
Every consumer VR headset on the market uses effectively the same optical pipeline: a macro display behind a simple lens. The lens is there to focus the light from the display into your eye. But in order for that to happen the lens needs to be a few inches from the display, otherwise it doesn’t have enough focusing power to focus the light into your eye.
That necessary distance between the display and the lens is the reason why every headset out there looks like a box on your face. The approach is still used today because the lenses and the displays are known quantities; they’re cheap & simple, and although bulky, they achieve a wide field-of-view and high resolution.
Many solutions have been proposed for making VR headsets smaller, and just about all of them include the use of novel displays and lenses.
Pancake Optics (AKA Folded Optics)
What are pancake optics? It’s not quite what it sounds like, but once you understand it, you’d be hard pressed to come up with a better name.
While the simple lenses in today’s VR headsets must be a certain distance from the display in order to focus the light into your eye, the concept of pancake optics proposes ‘folding’ that distance over on itself, such that the light still traverses the same distance necessary for focusing, but its path is folded into a more compact area.
You can think of it like a piece of paper with an arbitrary length. When you fold the paper in half, the paper itself is still just as long as when you started, but its length occupies less space because you folded it over on itself.
But how the hell do you do that with light? Polarization is the key.
It turns out that beams of light have an ‘orientation’ which is referred to as polarization. Normally the orientation of light beams are random, but you can use a polarizer to only let light of a specific orientation pass through. You can think of a polarizer like the coin-slot on a vending machine: it will only accept coins in one orientation.
Using polarization, it’s possible to bounce light back and forth multiple times along an optical path before eventually letting it out and into the wearer’s eye. This approach, known as pancake or folded optics, allows the lens and the display to move much closer together, resulting in a more compact headset.
Kopin is an electronics manufacturer best known for its microdisplays. In recent years the company has been eyeing the emerging XR industry as a viable market for their wares. To that end, the company has been steady at work creating VR displays and optics that it hopes headset makers will want to snatch up.
At AWE 2022 last month, the company demonstrated its latest work on that front with a new plastic pancake optic and flagship VR display.
Kopin’s P95 pancake optic has just a 17mm distance between the display and lens, along with a 95° field-of-view. Furthermore, it differentiates itself as being an all-plastic optic, which makes it cheaper, lighter, more durable, and more flexible than comparable glass optics. The company says its secret sauce is being able to make plastic pancake optics that are as optically performant as their glass counterparts.
Photo by Road to VR
At AWE, I got to peak through the Kopin P95 optic. Inside I saw a sharp image with seemingly quite good edge-to-edge clarity. It’s tough to formulate a firm assessment of how it compares to contemporary headsets as my understanding is that the test pattern being shown had no geometric or color corrections, nor was it calibrated for the numbers shown.
You’ll notice that the P95 is a non-Fresnel optic which should mean it won’t suffer from the kind of ‘god-rays’ and glare that almost every contemporary VR headset exhibits. Granted, without seeing dynamic content it’s tough to know whether or not the multi-element pancake optic introduces any of its own visual artifacts.
Even though the test pattern wasn’t calibrated, it does reveal the retina resolution of the underlying display—Kopin’s flagship ‘Lightning’ display for VR devices.
Photo by Road to VR
This little beauty is a 1.3″ OLED display with a 2,560 × 2,560 resolution running up to 120Hz. Kopin says the display has 10-bit color, making viable for HDR.
Photo by Road to VR
Combined, the P95 pancake optic and the Lightning display appear to make a viable, retina resolution, compact display architecture for VR headsets. But it isn’t necessarily a shoe-in.
For one, the 95° field-of-view is just barely meeting par. Ostensibly Kopin will need to grow its 1.3″ Lighting display larger if it wants to meet or exceed what’s offered in today’s VR headsets.
Further, the company wasn’t prepared to divulge any info on the brightness of the display or the efficiency of the pancake lens—both of which are key factors for use in VR headsets.
Because pancake lenses use polarized light and bounce that light around a few times, they always end up being less efficient—meaning more brightness on the input to get the same level of brightness output. That typically means more heat and more power consumption, adding to the tradeoffs that would be required if building a headset with this display architecture.
Kopin has been touting its displays and optics as a solution for VR headsets for several years at this point, but at least in the consumer & enterprise space they don’t appear to have found any traction just yet. It’s not entirely clear what’s holding the company back from break into the VR space, but it likely comes down to the price or the performance of the offerings.
That said, Kopin has been steadily moving toward the form-factor, resolution, and field-of-view the VR industry has been hoping for, so perhaps the P95 optic and latest Lightning display will be the point at which the company starts turning heads in the VR space.
Kopin revealed the first all-plastic pancake lenses, which it says can enable much more compact VR headset designs.
“This major breakthrough represents a foundational milestone for the emerging VR markets. Bulky, heavy headsets have been a major barrier to faster consumer adoption for many years,” said Kopin’s CEO and founder John C C Fan.
Kopin Corporation produces OLED microdisplays which are highly suitable for pancake lenses.
Pancake Optics?
The primary driver of the size and bulk of today’s VR is the optical design. Most headsets today use relatively large, thick fresnel lenses to magnify a smartphone-sized display that is sometimes split in two. A large gap between the lenses & display is required. After adding housing, even the most streamlined designs end up over 400 grams.
Diagram from Oculus Connect 5
Pancake lenses have been used in head mounted displays (HMDs) for decades in labs and by militaries. Today, they are seen as a promising path to compact headsets, because the display can be much closer to the lenses and physically smaller. They do present their own challenges, however.
Panasonic and Pico more recently showed concepts of compact headsets using pancake lenses. Huawei launched a product with pancake lenses in China in December 2019, using small LCD panels. Without a tracking system or battery, these headsets end up around 150 grams.
Huawei’s VR Glass, sold in China since late 2019, weighs 166 grams
Pancake lenses use multiple elements. To avoid an image “ghosting” effect one of those had to be glass, driving up the weight & cost.
Kopin P95
Kopin’s new pancake lens, called P95, uses two plastic elements instead. Kopin says a “major breakthrough” has almost eliminated the ghosting problem, while delivering “better image quality, much smaller size, lighter weight and lower cost than anything previously available”.
So will all VR headsets move to P95 optics soon? Probably not. Pancake lenses have very low efficiency – they block most light, making typical displays look dim and washed out. This can be worked around with very bright displays, but that increases the power requirement, which could be a major barrier to battery-powered devices.
Smaller, closeup displays also make it challenging to deliver a wide field of view. Kopin says P95 has a field of view 95° diagonal – around 20% narrower than typical consumer VR today.
Kopin says it filed multiple patents on this new lens design, and says it has one “design-in partner” so far – though didn’t elaborate on specific plans.
Kopin’s microdisplays have long been used in a range of augmented reality (AR) and virtual reality (VR) products such as the SKULLY FENIX AR motorcycle helmet and the Google Glass Enterprise Edition. Like many companies working within the XR field, Kopin is working towards that dream scenario of a small, lightweight headset not too dissimilar to a pair of glasses. Its latest reveal takes another step towards that future with its new all-plastic Pancake optics.
Pancake optics are nothing new as they allow for a thin form factor, ideal for XR applications. However, this optical solution tends to require a glass lens to avoid image artefacts caused by birefringence when used in conjunction with plastic materials. Kopin has managed to create an all-plastic solution using two elements which not only reduces weight but also provides “virtually no birefringence” the company claims.
Currently, the company has applied for three patents on the design with more to follow. Its first all-plastic Pancake optics is called the P95 as it provides a 95° field of view (FoV), whilst weighing in at 15g per lens set and 16mm thick.
“This major breakthrough represents a foundational milestone for the emerging VR markets. Bulky, heavy headsets have been a major barrier to faster consumer adoption for many years. No more,” said Dr. John C.C. Fan, CEO and founder of Kopin Corporation in a statement. “After thorough review of the available optics designs, we believe all-plastic Pancake optics are the best for VR applications that require a very large field of view, excellent image performance, and a super compact size. The challenge of finding a suitable plastic material with the required optical properties has been solved.”
Google Glass Enterprise Edition 2 uses Kopin’s nHD Display.
“We believe our Pancake optics are perfect for VR headsets. Our first design, P95, is specifically optimized for use with our 2.6K x 2.6K OLED high-brightness microdisplays. This new technology would be ideal for VR, but we believe this unique optics would also be useful for MR and AR applications,” Fan continues. “With all-plastic Pancake optics and duo-stack OLED microdisplays, the dream of stylish, super lightweight (< 100 g), compact VR/MR/AR high-performance headset products is finally reachable. We plan on offering our new optics as either a stand-alone component or in a module with our OLED microdisplays. We are delighted that we already have the first design-in partner for our P95 optics.”
As patents are still being applied for it might be a while before Kopin’s P95 Pancake Optics start showing up in products. There are plenty of others including Facebook and Apple looking to get into the AR market over the next couple of years whilst those already in the field like Microsoft are working towards consumer-friendly devices. As further details are released, VRFocus will let you know.
During CES 2020 one of the most surprising virtual reality (VR) related reveals came from Panasonic with its VR Glasses. Shown in early prototype form, for CES 2021 this week a slightly tweaked design has appeared alongside several specs.
Still, without a decent name, the new Panasonic VR Glasses have distinctly chunkier arms which look to house cameras as the company now states the device uses: “optical technologies used in LUMIX digital cameras.” Those bigger arms no longer tuck behind the ears like normal glasses, gripping the back of your head instead. There’s no mention if this is adjustable though.
In fact, overall details are still fairly sparse when it comes to the new design. Panasonic has said the VR Glasses feature 2.6K micro-OLED panels for each eye with High-Dynamic Range (HDR) capability. These come from Kopin which announced its Lightning 2.6K x 2.6K OLED display (2560 x 2560 resolution) last year. The duo-stack OLED has 10-bit colour control so users can enjoy UHD high-quality images without the dreaded screen door effect hampering the experience.
And just like before it’ll be employing acoustic technologies from its Technics audio products with an ‘original dynamic driver’. Whether this means built-in speakers or merely headphones remains to be seen – previous marketing images did show cabled in earphones.
Panasonic VR Glasses CES 2020
The glasses will connect to either PC’s or 5G-capable smartphones to deliver content via a USB-C cable so you’ll be able to enjoy UHD visuals wherever you go. However, the design doesn’t support the wearing of actual glasses but there will be diopter adjustment options alongside that bridge wheel for adjusting Inter-Pupillary Distance (IPD). VRFocus found it work quite well when testing the prototype in 2020.
As for availability and price? Well, you’ll have to wait for those, Panasonic has yet to even indicate a possible release year let alone how much the VR Glasses will cost. Maybe it’ll do a 5G smartphone bundle like Neal Light? As and when further details are released, VRFocus will let you know.
I’ve written plenty of words complaining about the shameful state of a VR market heading into 2020 in which Oculus Go is a best seller with its limited interactions and a nearly defunct 3DoF development platform. So I’ll accept being called a hypocrite to report Panasonic’s 3DoF headset at CES 2020 is one of the most interesting VR prototypes I put in front of my eyes at the consumer electronics event this year.
Panasonic calls these “VR Eyeglasses” a prototype so consider any specifications shared in this article as placeholders for whatever could ship from the company or a partner. For example, a Panasonic representative said they could achieve a 100 degree field of view with this slim design using larger OLED microdisplays than the ones inside the headset I tried. This is to say the smaller OLED panels in the CES prototype produced a noticeably smaller field of view than most current consumer designs — 77 degrees measured diagonally, according to Panasonic.
Panasonic VR Weight and Limits
So why is this interesting if Panasonic’s prototype glasses are worse than Oculus Go in a critical way? I’m not really interested in VR with a significantly smaller field of view than current designs. So my interest relates to Panasonic’s suggestion they can expand on what I saw to levels comparable with current designs. With that in mind, I found very alluring this sleek lightweight approach coupled with lush visuals. Panasonic says it uses multi-element pancake lenses with panels produced in partnership with Kopin.
The weight loss is considerable compared with current VR headset designs. Panasonic reps said they are somewhere in the range of 150 grams, which is only about a third of one pound, but that could change based on the path they take to turn this prototype into an actual consumer product. The Oculus Quest, by comparison, is about 570 grams, which is about 1.25 pounds–roughly four times as heavy.
Though I only spent a few minutes with its 60 Hz display, I twisted the interpupillary distance (IPD) mechanism (said to adjust from below 60 millimeters to more than 70 millimeters) near the bridge of the glasses and tuned the lenses directly in front of my eyes. Then I found myself observing crisp text and recall no impression of a screen door effect. I enjoyed a look inside a temple that seemed to live up to the promise of extremely high resolution high-dynamic range panels.
I should note that I saw some kind of “ghosting” visual artifact in a few places where there were white text subtitles or white lights. A second almost transparent image of the text or lights seemed to float at an angle nearby.
USB-C
This was running from a high-powered PC but Panasonic intends for these VR glasses to operate over a USB-C connection running to a phone from the tip behind an ear. Panasonic representatives said they have “multiple” candidate designs for the addition of 6DoF tracking, though that would likely add some weight. The connected phone could obviously be used as an input device and provide content to the glasses but this is where things get sort of dicey for this approach.
What platform is this?
Google and Facebook just spent the past two years receding from the phone-based VR market, with the latter devoting 2019 to its Quest standalone platform and the former abandoning Daydream. Plus, Apple doesn’t always play well with partners and Samsung develops its own display technology. So what exactly would people do with this hardware and where would that content come from? Those are great questions I’m not able to answer with this early look.
Even if these glasses added some cameras and somehow became compatible with, say, Oculus Touch controllers and Oculus Quest apps — could you really play active games like Beat Saber and keep a pair of sunglasses from flying off your head? Or if Panasonic sticks with 3DoF would these glasses reach a low enough price and have enough software and content support they could actually appeal to a decent number of people who just want a private movie viewer?
I don’t know the answers to any of those questions from my short demo, but this slim design fit well enough I hope we get to find out.
At CES 2020 this week, Panasonic revealed ultra compact VR goggles with HDR displays. However, details are light and it’s not yet clear whether this is an actual product or just a concept.
The headset uses OLED microdisplays jointly developed by Panasonic and Kopin. Microdisplays are significantly smaller than regular panels and thus allow for significantly more compact headsets.
However, microdisplays tend to have a much higher production cost and their small size makes magnifying them over a large field of view challenging. Most existing headsets using microdisplays have had a relatively narrow field of view, and short of a breakthrough in optics Panasonic’s is likely no exception.
Panasonic describes the resolution as simply ultra high definition (UHD), which usually means 3840×2160. It’s unclear whether this is per-eye or shared between each eye.
The displays support High Dynamic Range (HDR) content. HDR is a feature which allows digital content to show a greater range of colors and luminance compared to the standard range we’re all used to, which makes the colors and ranges of brightness better resemble what the human eye sees.
Audio is provided by earbuds developed by Panasonic’s Technics subsidiary. The drivers use magnetic fluid to allow “the accurate stroke of the diaphragm” and achieve “ultra low distortion reproduction”.
The goggles are not described as being either a product or a concept. No description of whether this is powered by a smartphone or PC was given, nor what kind of content platform it will support. In the images provided there is no sign of a port where a cable would even plug in.
In a statement, Panasonic said it will “continue to further develop the new VR glasses so that they can be used in a variety of applications, thereby creating new customer value“- whatever that means.
VR headsets today are bulky and heavy, making wearing them for extended periods uncomfortable. For VR to become a mainstream device the form factor will need to shrink. Projects like this show what that future could look like, but it likely makes sacrifices in field of view to get there. Not to mention there is no mention of tracking, a microphone, or other features you’d expect in an actual product.
Don’t forget to check out our CES 2020 Coverage Hub right herefor all of the latest announcements for VR and AR tech from the show!
The manufacturer of microdisplays and smart glasses components, Kopin, have announced a new augmented reality (AR) device that runs of a mobile device and is able to be voice controlled, attaching to any suitable glasses.
Dubbed the Kopin Golden-i Infinity, the smart screen is intended as an enterprise level device that is able to interface with existing hardware to offer a small, affordable and salable solution to workplace AR integration. By attaching the screen to a pair of suitable eye wear, a user is able to transform that into an AR display which will provide useful information all while being connected and powered by a device such as a smartphone. This means that the Golden-I Infinity is able to remain small but off-loading unneeded components and keep it all within the attached smartphone, leveraging that hardware via a USB-C cable.
Planned to be made available by the third quarter of this year, the Golden-i Infinity is said to retail for around $899 (USD) with a device weight of 1.5 ounces. The attachment to eye wear is done so magnetically and is compact enough to be easily carried around without causing much fatigue or stress on the users.
Thanks to the built in support for voice controls it further enhances the opportunity for hands-free user interaction by allowing applications to controlled without the need to pull the smartphone out of the users pocket. Of course, this means of interaction is still supported and can be used should the user wish to. On Windows 10 and Android devices the on board voice assistant can also be accessed with the voice controls on the Kopin Golden-i Infinity.
It features a 854×480 resolution microdisplay with a 20-degree field of view and voice-controlled zoom, able to work at up to 2,000 nits in outdoor use. The Kopin Golden-i Infinity also features an embedded 13-megapixel camera, microphones and speaker which can allow for video calls if required by the user.
With the move to power the Kopin Golden-i Infinity via a USB-C cable, users on an enterprise level or not will be able to see benefits it brings to the device. With quick and effective handling of audio, video, data, and power, the Kopin Golden-i Infinity has made a bond move opting for this type of connection. The headset will work with Android device running a Snapdragon 820 and 835 chip with support for the 845 planned as well.
For all the latest on the Kopin Golden-i Infinity in the future, keep reading VRFocus.
CES 2018 is the most exciting showcase of innovative technology every year. With over 170,000 attendees and over 3,900 exhibitors, there has to be some virtual reality (VR), augmented reality (AR) and mixed reality (MR) here, right? Today, VRFocus is doing a Post CES #XR event Review at the Realities Centre in London, UK bringing some of the companies together for those that couldn’t attend the main event. However if you aren’t attending the Post CES event, VRFocus has put a little recap together. From head-mounted displays (HMDs), VR platforms, VR content, 360 cameras, AR, audio, and MR to accessories to accompany new immersive technologies, you’ll find a sizable list in the run down below.
Big Announcements at Press Conferences
The HTC Vive Pro was the most exciting pre-CES announcement from HTC Vive.
Before CES even started VRFocus attended the Nvidia press event, where the company showcased its latest technology in autonomous cars. The biggest trend by far at this year’s CES, was how both VR and AR are being used to help ensure the safety of future cars, but also testing autonomous cars in environments that otherwise could not be controlled. Nvidia announced AutoSim, a VR simulator for self-driving cars as well as Drive AR, software that gives drivers information about what is around the car. All this information will be featured on the windscreen or dashboard screen. Think Google Maps on your heads-up-display (HUD).
It was HTC Vive that really stole the show that day after its initial Twitter tease of a HMD with new resolution, announcing the new HTC Vive Pro and Wireless adapter. The promise of no screen door effect and going untethered? It was a big day for VR lovers around the world. To add a cherry on top, the HTC Vive Pro will feature two microphones for noise cancellation capabilities and it’ll work with both 1.0 and 2.0 base stations.
Not exactly room-scale virtual reality, but a quick and easy access to transport users into a digital re-created world.
With the announcement of the HTC Vive Pro, Dell revealed at its press conference a partnership with HTC Vive to make it easier for consumers to get a full VR package with their desktop by allowing customers to purchase both the VR-Ready Dell Inspiron Gaming Desktop and an HTC Vive.
With no new Oculus Rift being teased or showcased, VRFocus managed to catch up with Kopin, showcasing the Elf VR headset. With a pixel density of 2,940 PPI, dual 2K resolution, 120 Hz for each eye, and a promise of smaller, lighter size and no screen door effect, Kopin is making ambitious promises for VR applications in the future. The Elf VR HMD is not yet available for consumers, but Kopin has said to keep an ear out for news just before the Electronic Entertainment Expo (E3) 2018.
Pico Interactive showcased its standalone headsets Pico Goblin and Pico Neo CV. Both headsets will be compatible with the Vive Wave platform, giving Pico customers access to 400 titles from the Vive owned store. A taster perhaps of what other standalone headsets might be such as Oculus Go and Vive Focus. The Pico Goblin is retailing for $269 USD and the Pico Neo CV can be pre-ordered for $749 on Pico’s website.
Lenovo also demonstrated its standalone device, Lenovo Mirage Solo. The headset utilises the Qualcomm Snapdragon 835, has 4GB RAM, with 110-degree field-of-view (FoV). The headset also uses Google’s WorldSense technology for inside-out tracking and comes with a wireless Daydream controller. The HMD is set to be released in Q2 this year.
Zeiss VR One Connect is now also compatible with Steam VR. Supporting both iOS and Android, the Zeiss VR One will need two wireless three degrees of freedom (3DoF) controllers that are linked via Bluetooth to enjoy full VR. The Zeiss VR One Connect is set to be available at major US retailers in the spring of 2018 at $129. No details on an international launch date are currently available.
Then there was Timescope’s terminal, which wasn’t what one would call “true VR”, in the sense that it’s not room-scale, you are not able to teleport or physically pick up objects with controllers – but still a form of new VR for public spaces. From the age of seven, members of the public are able to use the Timescope terminal to then see into the past or future of certain locations. All 3D content is as historical accurate as possible, with historians and engineers that have worked hard on making the terminals suitable for everybody – including people wearing glasses.
VR Platforms, VRContent, Training and Social VR
It might become easier to start a platform when making numerous amounts of VR content, and trying to find a solution to distribute the content to their clients and audience. This seems to be exactly the case for Jaunt, Ikonospace Art and Serious Factory.
Jaunt has created the Jaunt XR platform, which gives clients an easy way to distribute large amounts of VR content. With a media manager, player engine and intelligence suite to track information of how users interact with 360 content – its biggest benefit is that it allows all 360, VR films to be distributed across various VR HMDs.
Ikonospace is a small Dutch start-up company that use VR to create virtual art exhibitions. The platform would enable any artist around the world to build, create, digitise and finally curate their own art exhibition. CEO and Co-Founder Joris Demnard, personally comes from a family that specialise in curating and putting together art exhibitions. With simple design tools, one can create a building, decorate the interior and then place their art in it.
Serious Factory have a special toolkit that enables users to create social scenarios. Think of it as directing a social space where you can control the characters interactions, facial expressions, reactions and create the perfect social training scenarios. The platform allows for full customisation of characters from ethnicity, outfit, and interactions to the 3D environment where the interactions would take place. The final scene also has a point system, which would allow recruiters to see in which category users failed in or excelled in.
Fibrum is the largest Russian developer for VR games and applications, having dipped its toes into e-sports as well as creating the Fibrum Pro HMD. Fibrum now focuses on a platform called Desirium, a mobile VR app and platform that’s similar to Jaunt’s app, where the company can host 360-degree and VR content. You can download the app on Google Play, App Store, Google Daydream or Samsung Gear VR for a free trial period now.
Make your dreary gym workout plan fun with VR, gaming and highscores.
VRFocus got into the grind of CES with Black Box VR, a new concept that combines gaming with gym workouts. Instead of getting bored at the gym, trying to make your exercise routine more interesting by listening to your ‘Gym Workout Session 1’ playlist on Spotify or attempting to enjoy the visual images on the TV screen, you’ll soon be able to dive into a HTC Vive headset (accompanied with special trackers on each arm) and carry out various work outs through Black Box VR. Several Black Box VR gyms will be opening on the West coast of the United States, to make your average gym workout more interesting.
Anybody who’s been trying to use VR for exercise will have heard of Sprint Vector, Survios’ action-packed multiplayer VR title. Survios announced several Sprint Vector tournaments and that it would be coming to various VR arcades soon, which can also be experienced in California.
Serious VR takes things very serious indeed, specialising in VR training and performance data focusing on industrial companies such as defence, metal and automotive sectors. The plan is to change manuals and e-learning into an information schedule to implement VR for training processes. The benefits of immersion that VR offers seem be very cost-effective and more efficient than conventional training for industrial companies at the moment.
French company Light and Shadows use both VR and AR to provide solutions to major industrial enterprises, having created various devices and training applications for companies that are looking for ways to train or operate in VR. With a background in 3D environments, they showcased a training simulation that combined leap motion with the HTC Vive. Using just your hands, VRFocus managed to pick up mechanical and industrial pieces in a factory.
vTime gets an audio boost with DTS custom audio technology.
All in all, VR companies exhibiting at CES showcased various platforms that wanted to be cross-platform friendly and were looking to have users create and generate their own content in order to boost their platform. For the most part, it seems like these platforms are oriented at large industrial companies looking to train staff or employees, whether it be in factories or in social situations. With it comes a lot of responsibility to create easy-to-use toolkits, quick production work flows as well as the tools to create the best immersion. For the most part, it has been HTC Vive that’s been the winner when looking at how companies are using the HMD to help introduce VR to the masses.
360 Cameras
Kodak’s three lense camera shoots 8K and will have a waterproof case to film underwater in.
There were only two cameras that really showcased what 360 VR could offer in the future. Besides Sony and Zeiss’s previously mentioned, it was Chinese company Pisoftech and Kodak that offered 8K capabilities. Lenovo also dips into VR, but uses 180 degrees instead.
Pisoftech has a background in mass surveillance and data. They decided to venture into the realm of hardware when they saw that the products their clients were making could be more efficient if they made it. The Pilot Era camera can shoot 8K and has the capability to stitch internally. In other words, no need for an external device or software to stitch for you anymore. It’s quite big as well, 61mm x 61 mm x 159mm and weighs around 565g, however it may be worth it if you don’t want the hassle of stitching your images and if you want 180 minutes of battery life. 8K comes at a huge price though, its estimated price is at £3,000 GBP, not exactly affordable for the average consumer. The product is expected to be available on Amazon in April 2018.
What Pisoftech offer in 8K internal stitching, Kodak makes up for in size. Showcasing two brand new cameras that as of yet have no name, Kodak is also dedicated to creating 8K VR footage but in stereoscopic 3D VR. The first to be shown is the three-lens 8K PixPro 360 camera that is able to film at 30 frames per second (FPS) or 4K at 60FPS, potentially even at 120FPS. A waterproof version is said to be released around the same time to enable underwater filming. Half the size of the Pilot Era camera, this camera does not offer internal stitching but is very small and could potentially last a long time if combined with external batteries.
The second camera shoots 4K 30 FPS and can fold out and do 3D stereoscopic 180 images. It was designed specifically to please the push YouTube and Google are making into 3D on their platforms. You can also tether the cameras together and make a full 360 3D unit, if you were to have three of them. When speaking to Kevin Kunze, he mentioned that the three-lens 8K camera would be $1,000-$2,000 cheaper than other 8K cameras on the market and both cameras would hopefully be coming out in Q4 this year.
Lenovo hasn’t exactly created a 360-degree camera but instead opted for the Lenovo Mirage Camera. The device enables users to capture VR video in broad 180-degrees from its two front facing lenses, which can then be viewed on the Lenovo Mirage Solo HMD (mentioned in the HMD section).
It really depends on what you’re looking for when it comes to cameras, 8K, size, weight, price or the effort of stitching. It’s interesting to note that both Lenovo and Kodak have created VR 180 degree cameras due to the push that Google have made for this content on the YouTube platform. Lenovo’s Mirage Camera for example natively integrates VR 180 images and videos onto both YouTube and Google photos.
Augmented Reality
AR was a big buzz word when it came to autonomous cars. Nvidia showcased this with their Drive AR, but it was WayRay that really got people on the show floor queuing up to have a try.
WayRay, is a Swiss developer of holographic AR navigation system for connected cars. They showcased NAVION, the world’s first ever holographic AR navigator, and the advanced version of the AR Infotainment System for cars. Placed on a dashboard, NAVION shows directions, trip details and display real-time indicators for things like pedestrians, POIs or alerting to hazards exactly where the driver needs them to be — on the road ahead — with no headgear or eye wear required. NAVION works hands-free and responds to the simple voice commands or to unique hand gestures.
Hololampon the other hand depends on interactions with your hands. Tracking individual faces, it’s able to change and alter the perspective of holograms to appear 3D. This can include anything, from food to architectural buildings. The holograms can be interacted with as well, so you can choose to change the colour of a room for example by tapping on the hologram.
AstroReality takes 3D printed objects and then uses their AR app AstroReality to help inform users about planets. AstroReality showcased how this was done by using 3D printed planets such as Mars, Earth and the Moon for example at CES. They hinted at working on other 3D printed objects and AR for other uses cases such as Dinosaurs, Rocks and other things. They’ve been very popular with astrologists, space-enthusiasts and classrooms.
uSens has not only partnered with Pico interactive on hand-tracking for mobile VR, but has managed to find a way to bring AR to almost all Android phones with the uSensAR app. uSens also announced a partnership with Chinese technology firm Spreadtrum, and uSensAR will be bringing AR camera effects to the Spreadtrum SC9853 chipset platform which will be in “hundreds of millions” of smartphones in 2018.
AiFishowcased the Wonderlens app which remarkably manages to capture individuals from their phones or tablets to anywhere they want without the need for green screen. This is all done in real-time as well, so no need for Chroma keying or masking – it’s all done on your phone or tablet. For those who aren’t filmmakers – this is remarkable and ground breaking technology at the tips of your consumer fingers and should definitely be tried (it can be argued that Wonderlens is actually MR). AiFi also showcased Holo Messenger turning you into a hologram which then gets projected in a personal video message to friends and family. Founded by former Google and Apple engineers, AiFi is currently working on the advanced checkout-free shopping experiences in the US. Apparently they’re doing Amazon Go, but on a much larger scale in US retail stores.
XXII is a French company that has taken the profits made in VR and AR and have gone into Artificial Intelligence (A.I.). Its created various escape room VR experiences and have now gone into smart retail, where the system uses sensors to analyse customers. They use AR to show and explain how their smart retail AI can be used. Similar, but smaller in size to AiFi it looks like the combination of immersive technologies and A.I. are proving to be a great combination for selling products.
The feedback VRFocus was getting from exhibitors was that AR was the first step to bring VR into the mass market. AR was an easier and less drastic way of introducing the technology to consumers. The buzz around autonomous cars and AR definitely looks like a new exciting possibility for future entertainment – right there on your dashboard. Stepping into an autonomous car in the future, will possibly be like stepping into a theatre. The combining of AI with both VR and AR can be very interesting, but is just at the start. Booth AiFi and XXII are blurring the lines and we could potentially see various interesting outcomes from both companies if they were to apply it in new and exciting ways.
Audio
Can you imagine being in a busy and loud room where only you hear the sound and voices?
Audio is equal, if not more important when it comes to feeling immersed. You can hear a mistake a lot quicker than seeing a mistake. If audio doesn’t quite work the way it does in real life, it breaks the immersion for many people. Besides vTime’s partnership with DTS, Dirac and Noveto aim to help create more immersive environments for both VR and AR.
Some tech needs to be experienced first hand, VR is certainly one while Noveto’s audio tech is another. The company showcased what it called ‘virtual headphones’, named Sowlo, projecting sound into your ear like a pair of headphones. The system tracks the users’ ears as well, so even if you turn your head a certain degree, the sound and volume would stay the same. Noveto demonstrated this for a conference call and interior of car use cases, explaining that the days of ‘headphones’ would soon be over. The system works with no need for any accessories on the user at all, no headphones, cables or added accessories. It just needed you and your ears.
Swedish company Dirac launched Dirac VR last year, a 3D audio platform for VR. This year at CES Dirac showcased the next iteration of the Dirac VR platform. The audio rendering of a virtual environment showcase how the location and volume level of the audio adjusts as the head rotates with a VR headset. In other words, if you move your head, sounds bounce off the walls the same way they would if you were doing this in real life.
Noveto’s Sowlo has the potential to not only change the way users interact with audio for VR, but also for everyday use cases. It has the potential to really enable social VR and really alter the way we can experience audio as a whole. Dirac’s experience in audio means that the re-creation of environments will create more immersive and realistic sound environments. Both look very promising to enhance VR worlds through means of audio.
Mixed Reality
Being hands-free would enable health care providers to work more efficiently.
There was not too much MR that VRFocus saw on the show floor at CES. Luckily the company that did showcase MR are doing something that could revolutionise the future of public health care. Making it more efficient, quicker and potentially lifesaving.
Exelus, a French telemedicine solution company has created a comprehensible mobile diagnostic platform using Microsoft’s Hololens. Partnering with Holoforge, they’ve created Nomadeec. It’s focused on delivering and providing the most accurate information as efficiently and quickly as possible between healthcare providers. Designed for nursing homes and out-of-hospital care facilities, it would allow nurses, care assistants, night watchmen and doctors to communicate with one another. Nomadeec is currently being used in France and could be a glimpse of what future healthcare methods healthcare providers will be using in future.
The last part of this recap is dedicated to accessories that are made to enhance your VR experience or can be used by developers to help develop and create new VR content.
When it comes to untethered VR, or VR without wires and cables that attach you to a laptop or desktop, HTC Vive users can rejoice at the announcement of the HTC Vive Wireless Adapter. However TPCast are also providing another option. The TPCast 2.0, which according to TPCast will utilise the latest generation of ultra-low latency codecs and upgraded real-time data control protocol. This will allow the company to deliver high quality video with low latency and real-time transmission at a range of distances. TPCast’s General Manager for Americas Udi Yuhjtman confirms that regardless of which HTC Vive HMD you’re using (Vive or Vive Pro), you’re able to use the TPCast 2.0 for both.
Feeling and Touching in VR: Gloves and Haptic Feedback
French company Go Touch VR uses haptic feedback to create the illusion of touch, except you don’t need to wear a glove. So no matter what size your hands or fingers are, you will be able to feel haptic feedback. It’s probably important to note, that long fingernails do not work with the system. By wearing small devices at the user’s fingertips, the units provide pressure on the fingers, allowing for multiple haptic feedback sensations. The VRTouch Developer Kit began selling in October 2017 and has since sold to several companies that include BMW.
Dutch start-up company Sense Glove is also seeking to find a solution for haptic feedback. However instead of putting items on your fingers, you need to put on what looks like a very complicated exoskeleton of a glove. It not only provides haptic feedback though, but force feedback as well. This was demonstrated at CES by having users trying to crack an egg and various other circular objects. The idea is to help the user feel the shape and density of virtual objects.
BeBop Sensors showcased their newly announced BeBop Sensors Marcel Modular Data Gloves that allow for either 6 or 9 degrees of freedom with a sensor sub-frame latency at 120 Hz. The data glove uses smart fabrics, with sensors tracking force, location, size, weight, bend and twist.
TreeTech is a Dutch company that realised that PC gamers were moving to the couch with VR and Steam Link box. So they created the PO!NT Controller, a modular game controller that brings the keyboard and mouse shortcuts to a controller. TreeTech also create custom module controls, and can be approached by developer for custom experiences. They will start producing the PO!NT controller in Q3, and on the market in Q4 retailing at $149.
Jamzone is a Dutch company that’s created a videogame to help train users to learn and cope with stress. Stressjamuses a special waistband that measures heart rate variability to read your body’s state of mind. So in other words it knows when you are calm or stressed. Stressjam then takes you on a guided journey where you can only complete certain tasks by making yourself calm or stressed. Jamzone want to use it for B2B purposes and want large companies to use it to help prevent stress, burnout and depression in their employees by training them to control their stress levels in VR.
VRFocusstumbled upon the Yaw VR Motion Simulator which makes VR more like a Cinema 4D experience. The Yaw VR offers full 360 degrees swivel, tilts, shakes, spins and motions that make you feel like you’re in real life. You can pre-order the Yaw VR and it is compatible with the Gear VR, Oculus Rift and HTC Vive for £629 if you get an Early Bird version.
On a special note, Pico Zense, AiFi and various other companies seem to be working on depth sensing technology at CES. The ability for VR or AR headsets to recognise the location around them could change and alter the way we interact with digital objects as well as pave the way forward for mixed reality content. All in all, it’s clear that there are numerous exhibitors who are looking for new solutions to solve problems in the fields of VR and AR.
Conclusion
CES 2018 showcased untethered VR without the need to link to a PC or laptop, trying both the HTC Vive Wireless Adapter and TPCast 2.0 was very strange in the sense that the usual ‘safety net’ or umbilical cord that kept you grounded to the sense of existing reality was gone. However with Oculus’s Santa Cruz and HTC Vive’s Wireless Adapter coming to the market – TPCast may struggle to stay relevant by the end of the year. The most common trend amongst HMD manufacturers such as Kopin or Zeiss One and VR content producers such as Jaunt, Fibrum, Ikonospace or vTime was that they wanted to be cross-platform and enable user generated content. This means that users need access or full customisation to what they’re creating in VR, but also make VR a much more accessible platform. It was very refreshing to see Timescope take VR outdoors and to public places where numerous tourists, and anybody from the age of seven is able to experience a location in a new way. It just goes to show that no matter which HMD you have, it’s all about content.
With regards to 360-degree cameras the trend is looking to get higher quality resolution and bumping up the cameras to 8K. It’s then just a matter of size, prize, weight and intent. There’s a definite trend towards using 3D 180 degree cameras for YouTube and Google’s sake, as is seen in Lenovo Mirage Camera – although not many except the Lenovo Mirage Solo seem to be discussing being a platform for the 3D 180 degree video or image content that would be created with the cameras. The foldable Kodak camera seems to try and combine both stereoscopic 180-degree 3D as well as full 360-degree filming – but at what cost? Stitching is still a massive problem when it comes to creating 360 content, but is paying $3,000 worth the price for not having to stitch?
When it comes to AR, the autonomous cars were the winners. AR on the dashboard or windscreen were two of the biggest hypes at CES when Nvidia and WayRay discussed them. Not only would information become easier, but it seemed like a natural progression having all this information accessible. Within five years, it will be expected that every new vehicle should have AR integrated in it. This means that in some sense AR will have reached the general consumer population to a certain degree. uSens as well as AiFi do deserve a special mention. uSens not only showcase impressive finger tracking, but uSens AR would also bring AR to cheaper android phones – especially in China opens up the AR market to various new types of content, users and market. No longer will AR be something only iPhone users can afford, but something that is for everybody. AiFi is one to watch with regards to how they are using A.I. with AR and VR. Though Wonderlens and Holo Messenger seem like small quirky apps, the technology behind it really showcase the intelligent sensors and software created by AiFi that enable them to go into the mass smart retail market.
When it comes to audio, it’s clear that vTime is trying to address immersion through realistic audio. It’s great to see a company such as Dirac who have built custom made audio for various locations and applications intently try and create a realistic audio as possible for VR. It was however Noveto that really blew VRFocusout of the water. Not only could this revolutionise the way users interact with audio as a whole, but it would also enable various users to go into a public space without the need to wear headphones on their VR headsets to enjoy an experience. No unnecessary wires, hygiene problems or headphones that aren’t suited to your specific ear shape.
There was only one real contender that truly showcased a working platform and model that is revolutionising the healthcare industry, and that is Nomadeec. A small thing such as enabling hands-free care whilst giving real time data back to the hospital as well as being able to call a doctor regarding the patient can be life changing and perhaps even life saving for patients that need immediate care. Though Nomadeec would be implemented in large business such as care homes or out of-hospital care facilities, it’s clear to see the potential of it as Nomadeec roles out. Hopefully Nomadeec will be brought to more countries and have a quick as well as immediate positive impact on patients in need. The only barrier to entry then becomes the price as well as training of staff the use the expensive Microsoft Hololens.
And last but not least, accessories. Now it’s hard to say which glove or accessory was the best at CES. When we reached BeBop Sensors it was nice to see something that felt complete, I personally was unable to use Go Touch VR because I have long fingernails and my colleague Peter demoed the Sense Glove. However after much discussion VRFocusconcluded that when it comes to being prepared for consumers, BeBop Sensors was the closest to being ready. Go Touch VR and Sense Glove both felt like early prototypes which needed several iterations whilst BeBop Sensors was almost there, had haptics on the fingers, was small and worked for both myself and Peter who have very different hand sizes.
A personal favourite of mine was Stressjam. This was probably because of how drastic and immediate of a change was asked from me whilst at CES in my state of mind. When I came out of Stressjam, it had a similar effect to how I felt when I used Noveto’s Sowlo system. Personally, I also believe that being able to control stress is a very healthy and useful ability for a population that suffers a lot from stress, depression and other illnesses as a result of that. It would be great to see VR implemented in a healthy and fun way that is not only associated with shooting zombies, scary horror VR videogames or training simulations.
Finally one thing that was most prominent from the whole show, was the continuous showcasing of VR experiences by using HTC Vive. Almost every single experience used the headset, from Black Box VR to training simulations that combined it with magic leap. Another thing that was noticeable was the incredible number of Asian or Chinese people at CES. It’s very clear that China is a huge market to be tapped, and it will be interesting to see more Chinese VR and AR tech coming out of the continent as well as seeing how HTC Vive will fare in the Eastern market.
When VRFocus last met with Stuart Nixdorff, Senior Vice President of Kopin to discuss Elf VR it was at last year’s Electronic Entertainment Expo (E3). Nixdorff was holding a prototype then, and was able to show VRFocus the ultra-small 4K display chip as well as their Elf VR head mounted display (HMD). Roughly six months on and Nixdorff is able to show a working prototype.
Since E3, Kopin have been working on making Elf VR into a working unit that is compatible with the SteamVR, Oculus and Microsoft platforms. Wanting to not only bring videogames to the headset but to do so at a high quality. Elf VR uses OLED microdisplays that boast a pixel density of 2,940 PPI which is approximately 400% higher than conventional TFT-LCD, OLED and AMOLED displays at 20% of the size. With dual 2K resolution, 120hz for each eyes Nixdorff says that is no screen door effect and therefore gives the user a film-like experience that is both rich and beautiful. This would make fast action games, movies or sports a lot more enjoyable to experience.
Here are some more technical specs:
Elf VR weighs only 300 grams and is less than half the size and weight of popular VR headsets in the industry.
Elf VR is equipped with Kopin LightningTM OLED microdisplay panels, which feature 2048 x 2048 resolution of each panel, to provide binocular 4K image resolution at 120Hz refresh rate. Combined with both 4K Ultra-High image resolution and 120Hz refresh-rate, Elf VR provides very smooth images with excellent quality, and effectively reduces the sense of vertigo.
The microdisplay panels are manufactured with advanced ultra-precise processing techniques. Its pixel density (2,940 pixels per inch) is approximately 400% higher than conventional TFT-LCD, OLED and AMOLED displays, and the screen size reduced to approximately 1/5 at similar pixel resolution level.
Elf VR also adopts an advanced optical solution with a compact multi-lens design that enables it to reduce the thickness of its optical module by up to 60% as well as reduce the total weight of the VR HMD by approximately 50%.
The reference design supports two novel optics solutions – 70 degrees FoV for film-like beauty or 100 degrees FoV for deeper immersion.
Nixdorff mentions that he met a lot of filmmakers at E3, and what they said was necessary for good storytelling, cinematic reality and realism was a product that would enable this emotional engagement. He says that filmmakers want to not only tell a story but want less separation from the environment in contrast to gamers who want to be fully and highly immersed. Nixdorff says that they’re working together with partners to bring in more ambient light and awareness for HMD’s to avoid the separation and making VR more social, or less separated. Kopin have worked together with various OEM’s and will be shipping it to developers soon. Nixdorff hope that developers can make Elf VR even smaller and lighter in industrial design as well as creating content that leverages the resolution.
Kopin is certainly very involved in the virtual reality (VR) and augmented reality (AR) space. “2018 will be a very exciting year for Kopin and represents the culmination of years of development,” said Dr. John C.C. Fan of Kopin. “At CES we will unveil a range of Innovation Award-winning technologies and devices that will transform the worlds of AR,VR and wearables.” When asked about when Elf VR would be coming to consumers, Nixdorff said they are doing something with partners and will hopefully be announcing something before this year’s E3.
Über die ultrahochauflösenden Minidisplays von Kopin haben wir schon öfter berichtet: Schließlich liefern derzeitige Modelle eine Pixeldichte von 2900 ppi und ermöglichen durch ihre Bauart kleine und leichte VR-Headsets. Allerdings gab es beim bisherigen Elf Headset noch diverse Probleme, beispielsweise das Sichtfeld im Zusammenhang mit der Helligkeit. Der chinesische Hersteller nahm die CES 2018 zum Anlass, einen neuen Prototypen vorzuführen.
Kopie und Elf Headset: Noch nicht perfekt
Spätestens nach dem Hands-on zum Pimax 8K Headset dürfte klar sein, dass man die perfekte Kombination aus Helligkeit, Auflösung, Linsen und Sichtfeld nicht aus dem Ärmel schütteln kann. Das Beispiel Kopin zeigt ebenfalls, was für Probleme noch entstehen können. Auf der CES 2018 in Las Vegas hat der Hersteller einen neuen Prototypen seines Elf Headsets vorführen können. Die VR-Brille besitzt zwei ultrahochauflösende Bildschirme, für jedes Auge einen. Auf lediglich ein Zoll tummeln sich auf dem Microdisplay 2048 x 2048 Pixel, was zu einer enormen Pixeldichte von 2900 ppi führt. Und offensichtlich noch zu Problemen.
Die Kollegen von Upload VR haben den Prototypen auf der CES 2018 testen können. Theoretisch erreicht das 2K-Display eine Bildwiederholfrequenz von 120 fps, auf der Messe in Las Vegas schaffte das Elf Headset jedoch nur 60 fps. Außerdem stellte der Redakteur zwei dünne horizontale Linien fest, die den eigentlich guten Bildeindruck störten.
Trotzdem ist die Technologie vielversprechend: Der kleinere Formfaktor der Displays ermöglicht es, leichtere und laut Hersteller um bis zu 40 Prozent kleinere VR-Headsets zu bauen. Gleichzeitig steigt die Auflösung insgesamt auf 4096 x 2048 Pixel, wodurch je nach der Weite des Sichtfelds ein äußerst scharfes Bild dargestellt werden könnte. Kopin räumt allerdings ein, dass Eye-Tracking notwendig sein könnte, um lediglich die Darstellung im Fokus in höchster Detailstufe rendern zu müssen. Das Unternehmen erwartet, noch dieses Jahr einige Hersteller vom Display überzeugen zu können und hofft, auf der CES im nächsten Jahr VR-Headsets diverser Anbieter ausstellen zu können.
