This year’s Virtual Reality Strategy Conference (VRS) will see Dr. Yue Fei, Chief Technology Officer and co-founder at uSens deliver a keynote speech discussing 3D skeleton hand-tracking. Usens is a company that does extensive work in human-computer interaction solutions, including hand-tracking for virtual reality (VR) and augmented reality (AR).
Dr. Fei’s keynote will be titled ‘Leveraging Naturalistic Interfaces in XR’ and will concentrate on the future of hand interaction in AR and VR. Dr. Fei has stated a belief that users need more intuitive ways to interact with virtual objects and environments.
The uSens 3D skeleton hand-tracking solution was launched in May of 2018, and has been taken up by some businesses, such as a partnership between a German autotmotive company to build a ‘virtual cockpit’ as a 3D prototyping platform. This project is expected to save $1million (USD) in prototyping costs.
The uSens hand-tracking system is also said to be utilised by an OEM firm involved in the manufacture of smart electronics products, with the uSens system as part of a smart device platform designed to educate and entertain children.
“The power and future of AR/VR lies in our ability to create the most natural and intuitive interactions. uSens is thrilled by the surge of demand from customers seeking to integrate our solutions on everything from sign language interpretation, enterprise remote assistance, gesture controls for robots/drones, AR stroke therapy programs and more,” said Anli He, uSens CEO and co-founder. “I believe our skeleton hand-tracking—the world’s first 3D, deep learning based, mobile skeletal hand-tracking solution—is on the way to empowering millions to ditch the touch screen to better interact with virtual objects.”
The VRS Conference will take place between 16th-17th October, 2018, at the Park Central Hotel in San Francisco, where uSens will be exhibiting its products. Further information can be found on the VRS website. For future coverage on uSens and other VR and AR projects, keep checking back with VRFocus.
Computer vision company uSens revealed last year that it was working on a method of hand-tracking. The company have now taken this idea even further, offering the possibility of hand-tracking technology using only a smartphone camera.
At the Augmented World Expo (AWE 2018), uSens revealed a beta version of the uSens Hand Tracking SDK, which was created to allow developers to integrate 3D motion recognition and tracking into smartphone apps.
The technology uses a smartphone RGB camera along with uSens computer vision and deep learning algorithms to track the full dynamic of the skeleton within the hand, instead of just the fingertips.
“uSens is proud to push AR to the next frontier, by enabling developers to create engaging, enjoyable and entertaining augmented reality experiences made more intuitive, for the smartphone user—by simply moving their hands and fingers in the air,” said Anli He, the co-founder and CEO of uSens, via a statement.
“This opens a whole new world of possibilities for developers, enabling them to create a truly one-of-kind experience for a mainstream audience,” continued He, “Similar to how touchscreens enabled even the most technologically challenged to embrace smartphones, providing an easy and natural way for users to engage with AR/VR objects and environments will play a major role in boosting consumer adoption.”
A company spokesperson has said that uSens is already working with an unspecified major game studio to integrate its tools into existing social media environments and upcoming videogames and apps.
Experts have noted that hand-tracking could offer new methods of interaction with AR apps and videogames, particularly if combined with ARKit and ARCore functionality, which could allow AR apps to transcend the current paradigms.
For further news and updates from AWE 2018, keep checking back with VRFocus.
At the 2018 Game Developer Conference this week uSens Inc, a pioneer in HCI and computer vision augmented reality (AR) and virtual reality (VR) announced that it was opening up a beta program to it’s upcoming Smartphone AR Engine, uSensAR.
Back at the start of the year at the Consumer Electronics Show (CES), uSens revealed their uSensAR engine and how they planned to bring it to 2 billion Android devices. With the recent release of ARCore developers now have access to over 100 million devices but with uSensAR developers will be able to reach many more. As uSenseAR is a single camera AR engine that is optimized for low-ends camera, sensors and IMUs, it make it possible for almost all Android devices to experience the thrill of AR.
“The number of AR-capable Android smartphones is still quite small, with estimates putting it at 100M by the end of 2018,” said Anli He, uSens co-founder and CEO. “In order for the AR industry to thrive, it is essential that game developers have a larger target market to cater to – uSensAR is aiming to solve this problem by making every Android smartphone, an AR device. Charles joins us at an exciting time in our company’s development, where his imagination and understanding of the gaming industry will help us connect with the most creative and innovative content developers in the industry, starting with our beta program.”
Alongside the announcement of the beta program, Guitar Hero co-found Charles Huang has been named Board Advisor to help drive uSensAR adoption into the videogame industry. Thanks to almost two decades worth of experience in the industry, Huang also serves on the board of startups in the videogame, connected toy, music technoloy industries, and also advises several VC and PE funds.
“Since 1999, I have seen the potential of video gaming technology to immerse users in a new world, where they can be center stage,” said Huang. “AR is such an exciting technology because it furthers immersion, blending real and virtual worlds so users can experience the game, as opposed to having it presented to them. While other AR engines have focused on top-tier phones and hardware, uSensAR will bring in hundreds of millions of new customers to the AR market. Leading gaming, toy, and ecommerce brands have already initiated projects to harness the potential of AR and with the customer base growing from platforms like uSensAR, more investment will flow into the market. It’s an exciting time to be involved!”
USensAR is designed for mid and low-tired phones, with even better performance on the higher-end models. Thanks to a robust set of features including markerless tracking, low CPU and battery consumption, advanced plane detection, environmental understanding via SLAM technology, millimeter-level position accuracy and support for C++, Java and the Unity engine, uSensAR is capable of countless possibilities. Developers and content creators who are interesting in signing up for the beta program can do so via the uSensAR website.
VRFocus will make sure to bring you all the latest on uSensAR in the future, so stay tuned for more.
CES 2018 is the most exciting showcase of innovative technology every year. With over 170,000 attendees and over 3,900 exhibitors, there has to be some virtual reality (VR), augmented reality (AR) and mixed reality (MR) here, right? Today, VRFocus is doing a Post CES #XR event Review at the Realities Centre in London, UK bringing some of the companies together for those that couldn’t attend the main event. However if you aren’t attending the Post CES event, VRFocus has put a little recap together. From head-mounted displays (HMDs), VR platforms, VR content, 360 cameras, AR, audio, and MR to accessories to accompany new immersive technologies, you’ll find a sizable list in the run down below.
Big Announcements at Press Conferences
The HTC Vive Pro was the most exciting pre-CES announcement from HTC Vive.
Before CES even started VRFocus attended the Nvidia press event, where the company showcased its latest technology in autonomous cars. The biggest trend by far at this year’s CES, was how both VR and AR are being used to help ensure the safety of future cars, but also testing autonomous cars in environments that otherwise could not be controlled. Nvidia announced AutoSim, a VR simulator for self-driving cars as well as Drive AR, software that gives drivers information about what is around the car. All this information will be featured on the windscreen or dashboard screen. Think Google Maps on your heads-up-display (HUD).
It was HTC Vive that really stole the show that day after its initial Twitter tease of a HMD with new resolution, announcing the new HTC Vive Pro and Wireless adapter. The promise of no screen door effect and going untethered? It was a big day for VR lovers around the world. To add a cherry on top, the HTC Vive Pro will feature two microphones for noise cancellation capabilities and it’ll work with both 1.0 and 2.0 base stations.
Not exactly room-scale virtual reality, but a quick and easy access to transport users into a digital re-created world.
With the announcement of the HTC Vive Pro, Dell revealed at its press conference a partnership with HTC Vive to make it easier for consumers to get a full VR package with their desktop by allowing customers to purchase both the VR-Ready Dell Inspiron Gaming Desktop and an HTC Vive.
With no new Oculus Rift being teased or showcased, VRFocus managed to catch up with Kopin, showcasing the Elf VR headset. With a pixel density of 2,940 PPI, dual 2K resolution, 120 Hz for each eye, and a promise of smaller, lighter size and no screen door effect, Kopin is making ambitious promises for VR applications in the future. The Elf VR HMD is not yet available for consumers, but Kopin has said to keep an ear out for news just before the Electronic Entertainment Expo (E3) 2018.
Pico Interactive showcased its standalone headsets Pico Goblin and Pico Neo CV. Both headsets will be compatible with the Vive Wave platform, giving Pico customers access to 400 titles from the Vive owned store. A taster perhaps of what other standalone headsets might be such as Oculus Go and Vive Focus. The Pico Goblin is retailing for $269 USD and the Pico Neo CV can be pre-ordered for $749 on Pico’s website.
Lenovo also demonstrated its standalone device, Lenovo Mirage Solo. The headset utilises the Qualcomm Snapdragon 835, has 4GB RAM, with 110-degree field-of-view (FoV). The headset also uses Google’s WorldSense technology for inside-out tracking and comes with a wireless Daydream controller. The HMD is set to be released in Q2 this year.
Zeiss VR One Connect is now also compatible with Steam VR. Supporting both iOS and Android, the Zeiss VR One will need two wireless three degrees of freedom (3DoF) controllers that are linked via Bluetooth to enjoy full VR. The Zeiss VR One Connect is set to be available at major US retailers in the spring of 2018 at $129. No details on an international launch date are currently available.
Then there was Timescope’s terminal, which wasn’t what one would call “true VR”, in the sense that it’s not room-scale, you are not able to teleport or physically pick up objects with controllers – but still a form of new VR for public spaces. From the age of seven, members of the public are able to use the Timescope terminal to then see into the past or future of certain locations. All 3D content is as historical accurate as possible, with historians and engineers that have worked hard on making the terminals suitable for everybody – including people wearing glasses.
VR Platforms, VRContent, Training and Social VR
It might become easier to start a platform when making numerous amounts of VR content, and trying to find a solution to distribute the content to their clients and audience. This seems to be exactly the case for Jaunt, Ikonospace Art and Serious Factory.
Jaunt has created the Jaunt XR platform, which gives clients an easy way to distribute large amounts of VR content. With a media manager, player engine and intelligence suite to track information of how users interact with 360 content – its biggest benefit is that it allows all 360, VR films to be distributed across various VR HMDs.
Ikonospace is a small Dutch start-up company that use VR to create virtual art exhibitions. The platform would enable any artist around the world to build, create, digitise and finally curate their own art exhibition. CEO and Co-Founder Joris Demnard, personally comes from a family that specialise in curating and putting together art exhibitions. With simple design tools, one can create a building, decorate the interior and then place their art in it.
Serious Factory have a special toolkit that enables users to create social scenarios. Think of it as directing a social space where you can control the characters interactions, facial expressions, reactions and create the perfect social training scenarios. The platform allows for full customisation of characters from ethnicity, outfit, and interactions to the 3D environment where the interactions would take place. The final scene also has a point system, which would allow recruiters to see in which category users failed in or excelled in.
Fibrum is the largest Russian developer for VR games and applications, having dipped its toes into e-sports as well as creating the Fibrum Pro HMD. Fibrum now focuses on a platform called Desirium, a mobile VR app and platform that’s similar to Jaunt’s app, where the company can host 360-degree and VR content. You can download the app on Google Play, App Store, Google Daydream or Samsung Gear VR for a free trial period now.
Make your dreary gym workout plan fun with VR, gaming and highscores.
VRFocus got into the grind of CES with Black Box VR, a new concept that combines gaming with gym workouts. Instead of getting bored at the gym, trying to make your exercise routine more interesting by listening to your ‘Gym Workout Session 1’ playlist on Spotify or attempting to enjoy the visual images on the TV screen, you’ll soon be able to dive into a HTC Vive headset (accompanied with special trackers on each arm) and carry out various work outs through Black Box VR. Several Black Box VR gyms will be opening on the West coast of the United States, to make your average gym workout more interesting.
Anybody who’s been trying to use VR for exercise will have heard of Sprint Vector, Survios’ action-packed multiplayer VR title. Survios announced several Sprint Vector tournaments and that it would be coming to various VR arcades soon, which can also be experienced in California.
Serious VR takes things very serious indeed, specialising in VR training and performance data focusing on industrial companies such as defence, metal and automotive sectors. The plan is to change manuals and e-learning into an information schedule to implement VR for training processes. The benefits of immersion that VR offers seem be very cost-effective and more efficient than conventional training for industrial companies at the moment.
French company Light and Shadows use both VR and AR to provide solutions to major industrial enterprises, having created various devices and training applications for companies that are looking for ways to train or operate in VR. With a background in 3D environments, they showcased a training simulation that combined leap motion with the HTC Vive. Using just your hands, VRFocus managed to pick up mechanical and industrial pieces in a factory.
vTime gets an audio boost with DTS custom audio technology.
All in all, VR companies exhibiting at CES showcased various platforms that wanted to be cross-platform friendly and were looking to have users create and generate their own content in order to boost their platform. For the most part, it seems like these platforms are oriented at large industrial companies looking to train staff or employees, whether it be in factories or in social situations. With it comes a lot of responsibility to create easy-to-use toolkits, quick production work flows as well as the tools to create the best immersion. For the most part, it has been HTC Vive that’s been the winner when looking at how companies are using the HMD to help introduce VR to the masses.
360 Cameras
Kodak’s three lense camera shoots 8K and will have a waterproof case to film underwater in.
There were only two cameras that really showcased what 360 VR could offer in the future. Besides Sony and Zeiss’s previously mentioned, it was Chinese company Pisoftech and Kodak that offered 8K capabilities. Lenovo also dips into VR, but uses 180 degrees instead.
Pisoftech has a background in mass surveillance and data. They decided to venture into the realm of hardware when they saw that the products their clients were making could be more efficient if they made it. The Pilot Era camera can shoot 8K and has the capability to stitch internally. In other words, no need for an external device or software to stitch for you anymore. It’s quite big as well, 61mm x 61 mm x 159mm and weighs around 565g, however it may be worth it if you don’t want the hassle of stitching your images and if you want 180 minutes of battery life. 8K comes at a huge price though, its estimated price is at £3,000 GBP, not exactly affordable for the average consumer. The product is expected to be available on Amazon in April 2018.
What Pisoftech offer in 8K internal stitching, Kodak makes up for in size. Showcasing two brand new cameras that as of yet have no name, Kodak is also dedicated to creating 8K VR footage but in stereoscopic 3D VR. The first to be shown is the three-lens 8K PixPro 360 camera that is able to film at 30 frames per second (FPS) or 4K at 60FPS, potentially even at 120FPS. A waterproof version is said to be released around the same time to enable underwater filming. Half the size of the Pilot Era camera, this camera does not offer internal stitching but is very small and could potentially last a long time if combined with external batteries.
The second camera shoots 4K 30 FPS and can fold out and do 3D stereoscopic 180 images. It was designed specifically to please the push YouTube and Google are making into 3D on their platforms. You can also tether the cameras together and make a full 360 3D unit, if you were to have three of them. When speaking to Kevin Kunze, he mentioned that the three-lens 8K camera would be $1,000-$2,000 cheaper than other 8K cameras on the market and both cameras would hopefully be coming out in Q4 this year.
Lenovo hasn’t exactly created a 360-degree camera but instead opted for the Lenovo Mirage Camera. The device enables users to capture VR video in broad 180-degrees from its two front facing lenses, which can then be viewed on the Lenovo Mirage Solo HMD (mentioned in the HMD section).
It really depends on what you’re looking for when it comes to cameras, 8K, size, weight, price or the effort of stitching. It’s interesting to note that both Lenovo and Kodak have created VR 180 degree cameras due to the push that Google have made for this content on the YouTube platform. Lenovo’s Mirage Camera for example natively integrates VR 180 images and videos onto both YouTube and Google photos.
Augmented Reality
AR was a big buzz word when it came to autonomous cars. Nvidia showcased this with their Drive AR, but it was WayRay that really got people on the show floor queuing up to have a try.
WayRay, is a Swiss developer of holographic AR navigation system for connected cars. They showcased NAVION, the world’s first ever holographic AR navigator, and the advanced version of the AR Infotainment System for cars. Placed on a dashboard, NAVION shows directions, trip details and display real-time indicators for things like pedestrians, POIs or alerting to hazards exactly where the driver needs them to be — on the road ahead — with no headgear or eye wear required. NAVION works hands-free and responds to the simple voice commands or to unique hand gestures.
Hololampon the other hand depends on interactions with your hands. Tracking individual faces, it’s able to change and alter the perspective of holograms to appear 3D. This can include anything, from food to architectural buildings. The holograms can be interacted with as well, so you can choose to change the colour of a room for example by tapping on the hologram.
AstroReality takes 3D printed objects and then uses their AR app AstroReality to help inform users about planets. AstroReality showcased how this was done by using 3D printed planets such as Mars, Earth and the Moon for example at CES. They hinted at working on other 3D printed objects and AR for other uses cases such as Dinosaurs, Rocks and other things. They’ve been very popular with astrologists, space-enthusiasts and classrooms.
uSens has not only partnered with Pico interactive on hand-tracking for mobile VR, but has managed to find a way to bring AR to almost all Android phones with the uSensAR app. uSens also announced a partnership with Chinese technology firm Spreadtrum, and uSensAR will be bringing AR camera effects to the Spreadtrum SC9853 chipset platform which will be in “hundreds of millions” of smartphones in 2018.
AiFishowcased the Wonderlens app which remarkably manages to capture individuals from their phones or tablets to anywhere they want without the need for green screen. This is all done in real-time as well, so no need for Chroma keying or masking – it’s all done on your phone or tablet. For those who aren’t filmmakers – this is remarkable and ground breaking technology at the tips of your consumer fingers and should definitely be tried (it can be argued that Wonderlens is actually MR). AiFi also showcased Holo Messenger turning you into a hologram which then gets projected in a personal video message to friends and family. Founded by former Google and Apple engineers, AiFi is currently working on the advanced checkout-free shopping experiences in the US. Apparently they’re doing Amazon Go, but on a much larger scale in US retail stores.
XXII is a French company that has taken the profits made in VR and AR and have gone into Artificial Intelligence (A.I.). Its created various escape room VR experiences and have now gone into smart retail, where the system uses sensors to analyse customers. They use AR to show and explain how their smart retail AI can be used. Similar, but smaller in size to AiFi it looks like the combination of immersive technologies and A.I. are proving to be a great combination for selling products.
The feedback VRFocus was getting from exhibitors was that AR was the first step to bring VR into the mass market. AR was an easier and less drastic way of introducing the technology to consumers. The buzz around autonomous cars and AR definitely looks like a new exciting possibility for future entertainment – right there on your dashboard. Stepping into an autonomous car in the future, will possibly be like stepping into a theatre. The combining of AI with both VR and AR can be very interesting, but is just at the start. Booth AiFi and XXII are blurring the lines and we could potentially see various interesting outcomes from both companies if they were to apply it in new and exciting ways.
Audio
Can you imagine being in a busy and loud room where only you hear the sound and voices?
Audio is equal, if not more important when it comes to feeling immersed. You can hear a mistake a lot quicker than seeing a mistake. If audio doesn’t quite work the way it does in real life, it breaks the immersion for many people. Besides vTime’s partnership with DTS, Dirac and Noveto aim to help create more immersive environments for both VR and AR.
Some tech needs to be experienced first hand, VR is certainly one while Noveto’s audio tech is another. The company showcased what it called ‘virtual headphones’, named Sowlo, projecting sound into your ear like a pair of headphones. The system tracks the users’ ears as well, so even if you turn your head a certain degree, the sound and volume would stay the same. Noveto demonstrated this for a conference call and interior of car use cases, explaining that the days of ‘headphones’ would soon be over. The system works with no need for any accessories on the user at all, no headphones, cables or added accessories. It just needed you and your ears.
Swedish company Dirac launched Dirac VR last year, a 3D audio platform for VR. This year at CES Dirac showcased the next iteration of the Dirac VR platform. The audio rendering of a virtual environment showcase how the location and volume level of the audio adjusts as the head rotates with a VR headset. In other words, if you move your head, sounds bounce off the walls the same way they would if you were doing this in real life.
Noveto’s Sowlo has the potential to not only change the way users interact with audio for VR, but also for everyday use cases. It has the potential to really enable social VR and really alter the way we can experience audio as a whole. Dirac’s experience in audio means that the re-creation of environments will create more immersive and realistic sound environments. Both look very promising to enhance VR worlds through means of audio.
Mixed Reality
Being hands-free would enable health care providers to work more efficiently.
There was not too much MR that VRFocus saw on the show floor at CES. Luckily the company that did showcase MR are doing something that could revolutionise the future of public health care. Making it more efficient, quicker and potentially lifesaving.
Exelus, a French telemedicine solution company has created a comprehensible mobile diagnostic platform using Microsoft’s Hololens. Partnering with Holoforge, they’ve created Nomadeec. It’s focused on delivering and providing the most accurate information as efficiently and quickly as possible between healthcare providers. Designed for nursing homes and out-of-hospital care facilities, it would allow nurses, care assistants, night watchmen and doctors to communicate with one another. Nomadeec is currently being used in France and could be a glimpse of what future healthcare methods healthcare providers will be using in future.
The last part of this recap is dedicated to accessories that are made to enhance your VR experience or can be used by developers to help develop and create new VR content.
When it comes to untethered VR, or VR without wires and cables that attach you to a laptop or desktop, HTC Vive users can rejoice at the announcement of the HTC Vive Wireless Adapter. However TPCast are also providing another option. The TPCast 2.0, which according to TPCast will utilise the latest generation of ultra-low latency codecs and upgraded real-time data control protocol. This will allow the company to deliver high quality video with low latency and real-time transmission at a range of distances. TPCast’s General Manager for Americas Udi Yuhjtman confirms that regardless of which HTC Vive HMD you’re using (Vive or Vive Pro), you’re able to use the TPCast 2.0 for both.
Feeling and Touching in VR: Gloves and Haptic Feedback
French company Go Touch VR uses haptic feedback to create the illusion of touch, except you don’t need to wear a glove. So no matter what size your hands or fingers are, you will be able to feel haptic feedback. It’s probably important to note, that long fingernails do not work with the system. By wearing small devices at the user’s fingertips, the units provide pressure on the fingers, allowing for multiple haptic feedback sensations. The VRTouch Developer Kit began selling in October 2017 and has since sold to several companies that include BMW.
Dutch start-up company Sense Glove is also seeking to find a solution for haptic feedback. However instead of putting items on your fingers, you need to put on what looks like a very complicated exoskeleton of a glove. It not only provides haptic feedback though, but force feedback as well. This was demonstrated at CES by having users trying to crack an egg and various other circular objects. The idea is to help the user feel the shape and density of virtual objects.
BeBop Sensors showcased their newly announced BeBop Sensors Marcel Modular Data Gloves that allow for either 6 or 9 degrees of freedom with a sensor sub-frame latency at 120 Hz. The data glove uses smart fabrics, with sensors tracking force, location, size, weight, bend and twist.
TreeTech is a Dutch company that realised that PC gamers were moving to the couch with VR and Steam Link box. So they created the PO!NT Controller, a modular game controller that brings the keyboard and mouse shortcuts to a controller. TreeTech also create custom module controls, and can be approached by developer for custom experiences. They will start producing the PO!NT controller in Q3, and on the market in Q4 retailing at $149.
Jamzone is a Dutch company that’s created a videogame to help train users to learn and cope with stress. Stressjamuses a special waistband that measures heart rate variability to read your body’s state of mind. So in other words it knows when you are calm or stressed. Stressjam then takes you on a guided journey where you can only complete certain tasks by making yourself calm or stressed. Jamzone want to use it for B2B purposes and want large companies to use it to help prevent stress, burnout and depression in their employees by training them to control their stress levels in VR.
VRFocusstumbled upon the Yaw VR Motion Simulator which makes VR more like a Cinema 4D experience. The Yaw VR offers full 360 degrees swivel, tilts, shakes, spins and motions that make you feel like you’re in real life. You can pre-order the Yaw VR and it is compatible with the Gear VR, Oculus Rift and HTC Vive for £629 if you get an Early Bird version.
On a special note, Pico Zense, AiFi and various other companies seem to be working on depth sensing technology at CES. The ability for VR or AR headsets to recognise the location around them could change and alter the way we interact with digital objects as well as pave the way forward for mixed reality content. All in all, it’s clear that there are numerous exhibitors who are looking for new solutions to solve problems in the fields of VR and AR.
Conclusion
CES 2018 showcased untethered VR without the need to link to a PC or laptop, trying both the HTC Vive Wireless Adapter and TPCast 2.0 was very strange in the sense that the usual ‘safety net’ or umbilical cord that kept you grounded to the sense of existing reality was gone. However with Oculus’s Santa Cruz and HTC Vive’s Wireless Adapter coming to the market – TPCast may struggle to stay relevant by the end of the year. The most common trend amongst HMD manufacturers such as Kopin or Zeiss One and VR content producers such as Jaunt, Fibrum, Ikonospace or vTime was that they wanted to be cross-platform and enable user generated content. This means that users need access or full customisation to what they’re creating in VR, but also make VR a much more accessible platform. It was very refreshing to see Timescope take VR outdoors and to public places where numerous tourists, and anybody from the age of seven is able to experience a location in a new way. It just goes to show that no matter which HMD you have, it’s all about content.
With regards to 360-degree cameras the trend is looking to get higher quality resolution and bumping up the cameras to 8K. It’s then just a matter of size, prize, weight and intent. There’s a definite trend towards using 3D 180 degree cameras for YouTube and Google’s sake, as is seen in Lenovo Mirage Camera – although not many except the Lenovo Mirage Solo seem to be discussing being a platform for the 3D 180 degree video or image content that would be created with the cameras. The foldable Kodak camera seems to try and combine both stereoscopic 180-degree 3D as well as full 360-degree filming – but at what cost? Stitching is still a massive problem when it comes to creating 360 content, but is paying $3,000 worth the price for not having to stitch?
When it comes to AR, the autonomous cars were the winners. AR on the dashboard or windscreen were two of the biggest hypes at CES when Nvidia and WayRay discussed them. Not only would information become easier, but it seemed like a natural progression having all this information accessible. Within five years, it will be expected that every new vehicle should have AR integrated in it. This means that in some sense AR will have reached the general consumer population to a certain degree. uSens as well as AiFi do deserve a special mention. uSens not only showcase impressive finger tracking, but uSens AR would also bring AR to cheaper android phones – especially in China opens up the AR market to various new types of content, users and market. No longer will AR be something only iPhone users can afford, but something that is for everybody. AiFi is one to watch with regards to how they are using A.I. with AR and VR. Though Wonderlens and Holo Messenger seem like small quirky apps, the technology behind it really showcase the intelligent sensors and software created by AiFi that enable them to go into the mass smart retail market.
When it comes to audio, it’s clear that vTime is trying to address immersion through realistic audio. It’s great to see a company such as Dirac who have built custom made audio for various locations and applications intently try and create a realistic audio as possible for VR. It was however Noveto that really blew VRFocusout of the water. Not only could this revolutionise the way users interact with audio as a whole, but it would also enable various users to go into a public space without the need to wear headphones on their VR headsets to enjoy an experience. No unnecessary wires, hygiene problems or headphones that aren’t suited to your specific ear shape.
There was only one real contender that truly showcased a working platform and model that is revolutionising the healthcare industry, and that is Nomadeec. A small thing such as enabling hands-free care whilst giving real time data back to the hospital as well as being able to call a doctor regarding the patient can be life changing and perhaps even life saving for patients that need immediate care. Though Nomadeec would be implemented in large business such as care homes or out of-hospital care facilities, it’s clear to see the potential of it as Nomadeec roles out. Hopefully Nomadeec will be brought to more countries and have a quick as well as immediate positive impact on patients in need. The only barrier to entry then becomes the price as well as training of staff the use the expensive Microsoft Hololens.
And last but not least, accessories. Now it’s hard to say which glove or accessory was the best at CES. When we reached BeBop Sensors it was nice to see something that felt complete, I personally was unable to use Go Touch VR because I have long fingernails and my colleague Peter demoed the Sense Glove. However after much discussion VRFocusconcluded that when it comes to being prepared for consumers, BeBop Sensors was the closest to being ready. Go Touch VR and Sense Glove both felt like early prototypes which needed several iterations whilst BeBop Sensors was almost there, had haptics on the fingers, was small and worked for both myself and Peter who have very different hand sizes.
A personal favourite of mine was Stressjam. This was probably because of how drastic and immediate of a change was asked from me whilst at CES in my state of mind. When I came out of Stressjam, it had a similar effect to how I felt when I used Noveto’s Sowlo system. Personally, I also believe that being able to control stress is a very healthy and useful ability for a population that suffers a lot from stress, depression and other illnesses as a result of that. It would be great to see VR implemented in a healthy and fun way that is not only associated with shooting zombies, scary horror VR videogames or training simulations.
Finally one thing that was most prominent from the whole show, was the continuous showcasing of VR experiences by using HTC Vive. Almost every single experience used the headset, from Black Box VR to training simulations that combined it with magic leap. Another thing that was noticeable was the incredible number of Asian or Chinese people at CES. It’s very clear that China is a huge market to be tapped, and it will be interesting to see more Chinese VR and AR tech coming out of the continent as well as seeing how HTC Vive will fare in the Eastern market.
Today I learned that uSens, a leading computer vision company working on exciting new AR initiatives, wants to bring accessible augmented reality tech to over two billion Android devices around the world through the use of a new single-camera AR platform called uSensAR. Today I also learned that there are supposedly over two billion Android devices on the planet.
Since uSensAR aims to be optimized for low-end cameras through the use of computer vision and machine learning, it can run on a vast array of Android devices, instead of just the higher-end smartphones, such as those that ARCore requires.
“ARcore currently only serves about 30M Android phones, which is just 5% of the entire Android smartphone ecosystem,” said Yue Fei, uSens CTO and Co-founder, in a prepared statement. “For the AR industry to thrive, it is essential that as many people as possible have access to AR ready devices, which in turn will entice more AR content to be developed. With the release of uSensAR, we are allowing developers, smartphone manufacturers and developers to create those AR experiences not just for iPhone, Pixel and Samsung S8 users, which ARkit and ARcore only serve, but the entire Android ecosystem.”
With support for Unity, C++, and Java already, as well as an SDK coming in Q1 2018, uSensAR wants to make AR even more widespread than it already is.
You can find out more about the new platform at the official uSensAR website. Let us know what you think so far down in the comments below!
uSens, the tracking technology specialist behind the Fingo, has announced development of a Simultaneous Localisation and Mapping (SLAM) based mobile inside-out 6DOF positional tracking solution.
Designed specifically for mobile devices, using advanced machine learning and computer vision techniques, uSens’ algorithm constructs a virtual map of an environment without the need for markers. This environment can then be customised by adding digital content. The inside-out technology tracks a user’s location by continuously scanning and ‘learning’ about where they are, allowing a user’s position to be tracked in unknown and changing environments.
This kind of technology is far more suited to mobiles unlike marker-based or outside-in SLAM, which requires markers or sensors to be placed around a room in order to interact with a VR system like Oculus Rift or HTC Vive.
“We carry around a supercomputer in our pocket at all times – our smartphones – and giving developers the tools to enhance the user experience on hardware we all already possess is key to increasing adoption,” said Dr. Yue Fei, chief technology officer and co-founder of uSens in a statement. “Apple entering the market with ARKit, joining Samsung and Google, has focused the industry on creating untethered, mobile content. Consumers are excited for the coming wave of AR content and we’re focused on building the core interactive technologies for developers so they can create experiences that feel real and immersive, which is what consumers are demanding.”
This is the kind of tech mobile VR headsets like Gear VR or Goggle Daydream would need to provide more immersive experiences like their bigger high-end cousins. This would allow users to lean into virtual worlds, or walk around them just like HTC Vive’s roomscale, just with one added bonus, no wires.
Dr. Fei continued, “Positional tracking is required to give objects and experiences real meaning in virtual environments. Computing systems can create color, shape, size, etc., but what’s missing in the mix is position. The world is not flat, so as our technology begins to blend the physical and virtual world, it’s increasingly important that it interacts with the world in three dimensions. We’re already in discussions with a number of consumer-level companies to utilize our SLAM technology. Accurately tracking a user’s position will allow the entire industry to take a giant leap forward and our solutions will help bring on the next generation of amazing ARVR content.”
The company hasn’t said how early in development this inside-out 6DOF tracking is so its unclear exactly when it might be rolled out. VRFocus will continue its coverage of uSens, reporting back with further announcements.
Auf der gestrigen Augmented World Expo (AWE) gab es diverse Ankündigungen. Dazu zählte auch die Verkündung der Gewinner der uDev Challenge in den USA durch die VR-Tracking-Spezialisten von uSens.
uSens und die uDev Challenge
Die uDev Challenge wurde Ende 2016 von uSens gestartet und forderte Teilnehmer zur Entwicklung von Apps für Augmented Reality oder Virtual Reality auf. Grundvoraussetzung war die Nutzung des uSens SDKs (Software Development Kit) für die Entwicklung der Apps. Dabei konnten sich die Teilnehmer aussuchen, ob sie ihre Software für die Samsung Gear VR, Google Cardboard, HTC Vive oder Oculus Rift entwickeln wollten. Die Teilnahme bei dem Projekt lohnte sich, denn dem Gewinner winkte ein Preisgeld von 50.000 USD.
Die Challenge zog sich über mehrere Phasen bis zum Halbfinale. Dort bewertete eine Jury bestehend aus Mitgliedern von uSens, Samsung und einer Gruppe von VR-Spezialisten aus dem Silicon Valley die Projekte der übrig gebliebenen Entwickler.
Die eingereichten Projekte waren sehr beeindruckend, wie auch der CTO und Co-Gründer von uSens Dr. Yue Fei berichtet: „Wir waren sehr beeindruckt von der Qualität und Kreativität der eingereichten Projekte unserer Halbfinalteilnehmer. Die Gewinner integrierten unsere Fingo-Module [uSens Inside-Out-Tracking-Modul] großartig und demonstrierten dadurch, wie man unsere Tracking-Technologie auf ansprechende Weise zugunsten des Endnutzers anwenden kann. Ich bin sehr zuversichtlich bezüglich zukünftiger Apps für unsere Tracking-Technologie.“
Der Gewinner der uDev Challenge ist Chris Wren mit der App MonsterShop:
Doch auch der zweite und dritte Platz ging nicht leer aus. Den zweiten Platz sicherte sich Wallace Lages mit seinem Mobile Game Krinkle VR, welches ihm 25.000 USD einbrachte.
Auf Platz drei befindet sich Tom Leahy, der eine App für medizinische Zwecke entwickelte, um die Makuladegeneration – eine Erkrankung der Netzhaut – zu demonstrieren. Auch er erhielt ein Preisgeld ihn Höhe von 10.000 USD.
Auch weitere Teilnehmer erhielten diverse Summen, so wurden nach dem Wettbewerb insgesamt 200.000 USD an die Teilnehmer verteilt. In China findet übrigens ein ähnlicher Wettbewerb statt, welcher hoffentlich ebenfalls großartige Apps hervorbringt.
The Augmented World Expo (AWE) played host to a number of announcements today including, amongst others the reveal by VR tracking specialists uSensof the winners uDev Challenge for America. This followed a sister competition in China.
The projects, which required use of uSens’ own Software Development Kit (SDK) for augmented reality (AR) and virtual reality (VR) and could utilize either the Samsung Gear VR, Google Cardboard, HTC Vive or the Oculus Rift. And were initially narrowed down to a selection of semi-finalists by a judging panel consisting of members of the uSens, Samsung and the Silicon Valley Virtual Reality Meetup group.
“We were impressed by the quality and creativity of the projects submitted by our semifinalists.” Explained uSens CTO and company co-founder Dr. Yue Fei . “Our winners integrated Fingo modules,” uSens’s tracking device, which recently received an upgrade, “in imaginative ways and demonstrated the ability to make engaging, exciting use of our tracking technology to benefit end-users. I’m very pleased about the promise of future applications of our tracking tech.”
$50,000 (USD) was won by the grand prize winner, Chris Wren who developed MonsterShop an application for VR retail.
Wallace Lages came in second for spellcasting mobile game Krinkle VR, netting the developer $25,000 (USD).
Third place was won by a medical related tool called Embodied Labs developed by Tom Leahy. Which shows the degeneration of the macular, a very small but important part of the retina. It’s a condition that results in people losing their central vision.
The US prizes bring the total amount awarded across both challenges to $200,000. VRFocus will bring you more news from AWE 2017 throughout the week.
Was fehlen Samsungs Gear VR und Googles Daydream, was die großen Systeme Oculus Rift und HTC Vive mitbringen oder per Zubehör erhalten? Richtig, ein Kopf- und Hand-Tracking. Fingo will dies ändern und bringt eure Hände in die virtuelle Realität. Dabei erfasst das System sogar Finger und soll deshalb auch HTC Vive Nutzerund Oculus Rift Besitzer ansprechen. Die Lösung lässt sich jetzt als Entwickler-Kit für knapp 100 US-Dollar bestellen.
Fingo: Inside-Out-Tracking mit Fingererkennung
Mit dem Inside-Out-Tracking-Modul Fingo kann man herkömmliche VR-Brillen mit einem Tracking-System nachrüsten. Dafür wird das Modul einfach auf der Brille aufgesteckt. Das System verwendet zwar lediglich ein Kameramodul mit zwei Linsen, schafft aber laut Dr. Yue Fei von uSense erstaunliche Ergebnisse und kann insgesamt 22-Fingergelenke erkennen. Fingo soll 6-DOF (degrees of freedom) beim Head-Tracking und 26-DOF beim Hand-Tracking ermöglichen.
Damit wird das System auch für Besitzer von HTC Vive und Oculus Rift interessant. Allerdings fügt der Cheftechniker und Mitbegründer von uSense hinzu, dass man beim Tracking erst am Anfang der Entwicklung stehe. Aktuell ist nicht klar, ob das System deutlich bessere Ergebnisse als der Leap Motion Controller beim Hand-Tracking erzielen kann. Zudem würde es uns überraschen, wenn Fingo ein ähnliches Inside-Out-Tracking wie die HoloLens von Microsoft bereitstellt.
Während Systeme wie HTC Vive, PSVR und Oculus Rift externe Kameras oder Laserstationen benötigen, ermöglichen Inside-Out-Systeme die räumliche Erfassung ohne zusätzliche Hardware im Zimmer. Das macht die Installation wesentlich komfortabler und flexibler. Die kommenden Microsoft Mixed-Reality-Brillen beispielsweise von Acer, HP und Lenovo verwenden das Inside-Out-System.
Neben der Bestellseite hat uSense auch ein Entwicklerportal freigeschaltet, auf dem sich das SDK herunterladen lässt.
Most current PC-based virtual reality (VR) systems rely on tracking sensors for head and hand tracking, which means positioning several tracking modules in the right position, which can be awkward for people with small or unusually-shaped rooms. Tracking technology specialist uSens is aiming to remove that requirement with advances in its Fingo camera technology.
VRFocus have covered the Fingo before, but the technology has since made an advance. Combining the dual-lens Fingo camera with inside-out six degrees-of-freedom (6DoF) technology, it is now possible to achieve full head and hand tracking with only one Fingo camera. uSens claim that the Fingo system offers better accuracy and flexibility than other inside-out systems such as the HoloLens.
“While VR and AR has exploded in the popular conscious in the last year, realistic tracking technology is still in its early stages of development. The proprietary algorithms in the next generation of our Fingo tracking module will enable this new, combined 6DOF head tracking plus 26DOF hand tracking technology,” said Dr. Yue Fei, chief technology officer and co-founder of uSens, “With only a single stereo camera module, our solution is far more compact and cost-effective for developers seeking to generate more realistic content with lifelike mobility.”
The Fingo module is currently available for developers from the uSens website. Fingo is capable of tracking 22 finger joints at low latency times, allowing users to interact with virtual objects. The Fingo supports both mobile VR systems such as Samsung Gear VR and Google Daydream as well as PC-based systems such as Oculus Rift and HTC Vive. The development kit for the Fingo module costs $99 (USD). You can watch a demonstration video of the technology below.
VRFocus will bring you further news on the Fingo module and other advances in VR tracking technology as it becomes available.
