Ivan Zubritskiy. (Image courtesy Program-Ace.)
One of the devices that generates the greatest interest among the tech community at the moment is Microsoft HoloLens. By revolutionizing the way we perceive reality-altering solutions, the technology that stands behind HoloLens is as fascinating as it is complex. In simple terms, it can be described as the merging of real and virtual worlds where objects from both realities can exist side-by-side and interact with each other in real time. This technology has been revealed to the world as “mixed reality.”
Many developers are thrilled to get started with HoloLens, even though there are not as many early birds purchasing a developer kit worth $3,000 USD. The good news actually is that it is not necessary to own a device to program for it, thanks to Universal Windows Platform – once developed for one device, your application will run on HoloLens as well.
Preparing tools, SDKs and IDEs
From space exploration to architecture and gaming to education, Microsoft HoloLens has been crafted to innovate the inside of these industries and has already found a practical application across a variety of inside procedures. As HoloLens uses relatively new technology, there not as many tools that allow for developing for mixed reality. A majority of familiar development kits and software are suitable to be used when creating a holographic app. Based on the Program-Ace experience, a set of the minimum necessary tools is quite small and can be limited to:
3D design: Autodesk Maya, 3ds Max, Adobe PhotoShop, Substance Painter
Animation: Biped for 3ds Max
Development: Unity, Vuforia, Microsoft Visual Studio, Windows 10 SDK, HoloLens Emulator (if you do not have a device)
All development tools that are mandatory to have in order to build a holographic experience can be found on the Microsoft official website as well as the minimum system requirements. Once you are finished with installation and preparation, you can consider yourself ready to produce new mixed reality experiences.
Adapting holograms
The conventional approaches to 3D modeling and rendering stay the same whenever you create 3D objects for VR apps, games, or HoloLens. The differences appear only when these three-dimensional models are being integrated into the holographic environment.
Even though the visual experience delivered by HoloLens is obviously amazing, it must be delivered properly in order to avoid a frequently encountered phenomenon — cybersickness. This illness, also called “virtual reality sickness,” may occur while a user is inside the VR system and experiences eye discomfort, headache, nausea, dizziness, and disorientation.
In our development adventures, Program-Ace experts have also faced similar problems when HoloLens apps do not get users interested because of the way they were built that resulted in physical illness. Here is where a near clip plane becomes helpful. When using Unity, it is extremely important to set the near clipping plane to no less than 0.85m. If it is less than this, then objects will appear far too close to the user’s eyes and cause them to cross.
In addition to making the visual experience difficult to deal with, it can cause severe discomfort for the viewer. Also, the ideal distance from holographic content to the viewer’s eyes should be kept as close to 2m as possible because this distance is free from a binocular rivalry.
In simple words, two pictures projected onto the left and right part of the transparent combiner lenses will be fully overlapped at 2 meters from the user. In cases when it is impossible to place holographic content at recommended distance, you can use the most favorable zone — that is a range of 1.25 to 5 meters.
(Image courtesy Program-Ace.)
Developers should also pay careful attention to another essential element of the solid visual experience such as frame rate. To ensure a pleasant and safe HoloLens experience, we advocate setting a frame rate around 30 frames-per-second.
However, be careful with deviating from the recommended value since a too small frame rate will cause cybersickness mentioned before, while a too high value cannot be processed quickly enough due to the device’s limited capabilities compared to wire-based headsets.
Interacting with the holo-world: Gaze
Tracking where the user is looking is the way of managing the reality-altering environment migrated from virtual reality headsets, particularly HTC Vive and Oculus Rift.
In order to implement this technology, HoloLens tracks the position of the user’s head in space, processing a gigabyte of real-time information each and every second. For better understanding, imagine a simple application that involves multiple choices: for instance, a standard HoloLens app that explains our Solar System. When the user looks at a certain planet, it becomes highlighted and more information is displayed next to it.
By default, inside the HoloLens, a small translucent circle that is actually a 3D cursor, indicates where the user is gazing; however, it may inherit a form of any object whatever it is, real or virtual.
Unity helps developers to manipulate this HoloLens feature in many ways. For instance, the known “billboarding” technique allows the hologram to always face the user when they hit his or her field-of-view. It is especially advantageous for UI elements such as text or buttons. Also, developers can use 3D cursors such as gaze indicators that catch the user’s attention and guide it to the necessary hologram or object.
Interacting with the holo-world: Voice
Using voice command is the most unusual way to interact with the head-mounted devices. However, it feels much more natural and intuitive and provides an entirely new spectrum for navigating applications and even device’s UIs. Program-Ace experts created a set of rules that developers must adhere to when using Unity to program voice recognition:
- Keep the voice commands as short and simple as possible to simplify the pronunciation for non-native speakers and not confuse the users with complex phrases or sentences.
- One predefined command should be used throughout the entire app for the similar actions such as clicking and opening.
- Duplicate voice commands with gesture-based alternatives to improve the overall user experience and avoid situations when the pronunciation is wrong or unclear.
Interacting with the holo-world: Gestures
Gestures are a progressively new approach to interacting with reality-altering technologies. Besides the ability to leave the user hands-free, they are proved to be useful and effective. With a wide set of discrete and continuous motions, HoloLens can be controlled and maintained.
Discrete gestures introduce immediate actions such as clicking on the icon or open the menu — HoloLens has a special unique gesture for going to the menu — while continuous gestures are the ones that incorporate rotating or scrolling.
Ivan Zubritskiy. (Image courtesy Program-Ace.)
When developing for the HoloLens, you should make sure that the device responses promptly to the gestures by highlighting icons, opening applications, or performing other appropriate actions. Similarly to the way that allows for programming a user’s gaze, developers can include a few lines of code to warn the user that the gestures he/she performs are out of view.
However, due to the fact that mixed reality is not a solid technology yet, HoloLens has certain limitations. For instance, hand position or object placement is not always accurately recognized. This should be taken into account when developing for Microsoft headset and ensure that difference in inches will not have a significant impact on the functioning of the app.
HoloLens and the business world
Microsoft has taken a huge step forward in the technological field with the introduction of its mixed reality smartglasses. Such technology can be widely applied across multiple industries ranging from education and the military to construction and entertainment.
However, today, due to the novelty of this headset, it is quite hard for the community to conceive HoloLens as being universal.
To dispel the common misconception that top-notch technologies can be used by industries such as gaming, entertainment, or software development rather than construction, manufacturing, or government services and instead of the conclusion, here will be a couple of cases that illustrate the successful implementation of HoloLens solutions beyond traditional frameworks.
Education is the industry that can benefit most from getting HoloLens involved in the process.
First of all, it is extremely useful for step-by-step training in each and every industry. By wearing HoloLens, you can receive instructions from a system or a mentor in real time and immediately perform the necessary actions.
Moreover, it can be used as a visualization tool that helps trainees to explore complex equipment or dangerous processes, without putting their lives at risk or withdrawing expensive machinery from service to demonstrate how it operates.
In addition to the showcasing function, HoloLens also allows for interacting with holographic content as well as real objects.
HoloLens-based training has been so successfully implemented by Japan Airlines that uses it for training employees to handle massive and complex aircraft parts such as jet engines or avionics systems.
(Image courtesy Microsoft.)
Construction and manufacturing industries are natural fits for the HoloLens. There is the ability for this headset to share the same reality with other HoloLens users who might be in the same room or at a remote location, facilities envisioning of the future project. It also helps all decision-makers to reach a consensus by allowing them to make modifications with the presented holograms and share the results with the audience in a matter of milliseconds. Furthermore, as HoloLens is capable of producing new environments in real time, it might be the next step toward real-time construction.
According to HoloLens, Volvo already uses holographic computing in multiple ways, including construction and prototyping.
(Image courtesy Volvo.)
Medical holography is a truly magical use case of Microsoft HoloLens. Similarly to virtual and augmented reality, it has a solid potential in the healthcare industry and, in particular, operations and microsurgery.
But unlike other technologies, HoloLens has more solutions than challenges. To name a few, it eliminates the problem of surgeons moving back and forth during an operation and provides full freedom of actions, because they can use voice commands to control holograms. It also can show the doctors a full, three-dimensional view of any organ and its features that might be different from person to person. In that way, the doctors will know for sure what to expect, thus the proportion of unsuccessful operations is expected to be diminished.
HoloLens is our new reality and the faster we recognize and understand it, the quicker we can move toward a better, progressive future.
