Mixed Reality Lets Any Surface Become A Touch Screen

The market is still awaiting a consumer version of the Microsoft HoloLens, which is currently one of the most expensive mixed reality (MR) devices on the market, priced at $3000 (USD). This hasn’t stopped developers and researchers experimenting with the device to see what it can do. According to a team of Microsoft Researchers, the HoloLens can now make any flat surface into a touch screen.

Using a newly developed interaction technique dubbed MRTouch, the researchers have enabled tactile touch for MR content on the HoloLens, with ambitions to eventually bring it to other MR devices.

MRTouch Hololens

Since smartphones became nigh-universal, most people have become accustomed to the touch-based control scheme such as that used by Android and Apple devices. The MRTouch software allows a similar type of interaction into MR environments by utilising the environment mapping and hand-tracking technology built into the HoloLens in order to identify flat surfaces that can be used as an input device.

The researchers used the HoloLens ‘short-throw’ tracking mode, which is used for gesture tracking for distances of under 1m. The information from the short-throw depth camera was combined with information from the infrared camera feed to allow for accurate mapping of flat surfaces.

To open an app using MRTouch, touch the surface you wish to work with and drag your finger down to the right, this creates a virtual window that can open the application. The physical surface means users have a better idea of when a virtual button has been pressed, and MRTouch also allows for multi-touch interactions with up to ten digits.

Like most apps for HoloLens, MRTouch windows remain anchored to their position, so your application window will be where you left it should you need to move around for whatever reason.

MRTouch Hololens

Microsoft have yet to give an indication on where, or even if, the MRTouch application would be made available publicly, however a research paper has been published on the subject.

Many experts are hopeful that the MRTouch project makes it to market, and any further news on the topic will be reported here on VRFocus.