According to a recently-published patent, Microsoft is working on a device that will give you tactile feedback in both mixed and virtual realities.
The patent, published last week but originally filed back in June 2017, is for an ‘Electrostatic Slide Clutch’ that appears to fit around the user’s hand and arm. A series of electrodes stimulate the user’s hand with varying voltage, stopping them from fully closing a fist when they hold a virtual object, for example. The patent notes that this tech could be used in full virtual reality experiences and ‘mixed reality’, suggesting it could be compatible with HoloLens, too.
In its own images, Microsoft gives an example of holding a basketball, with the user’s hand stretched to simulate the curves of the ball. “In particular, haptic device may be configured to provide resistive contact sensation in response to detection of contact between the body of the user in a virtual display object projected into a field of view,” the patent reads.
Haptic feedback is one of the most important areas of VR R&D right now. Currently, input devices like Oculus Touch and the HTC Vive wands allow us to bring out hands into VR, but there’s nothing to stop us from putting our hands through surfaces, and no way to replicate the exact feeling of holding certain objects. With Microsoft’s tech, though, VR worlds can come a step closer to reality by giving players realistic feedback.
Still, as with most patents we report on, we have no way of knowing if Microsoft’s work will ever see the light of day. For now, we can just hope.
VR headsets from Acer, Lenovo, Samsung, HP, ASUS and Dell might be easier to set up than Oculus Rift or HTC Vive, but integration with Valve’s SteamVR is what makes these headsets worth consideration. Without it, owners can visit some nice virtual worlds from the Windows Store, but they’d be missing the much wider and deeper library available from Steam.
With its latest April update to Windows today, Microsoft is also taking this integration out of “early access” — meaning it should provide an overall smoother and more stable experience compared with when the software first launched late last year.
“Windows Mixed Reality for SteamVR will remain in Early Access until consumers and developers have had a chance to test titles and the experience is stable for the majority of games and apps,” Microsoft originally wrote when releasing the software early.
According to a blog post, haptic feedback is improved with the controllers in the latest updates. I’ve asked Microsoft for details about why the company chose today to have the software graduate from early access. At the time of this writing, the software is still listed as being in early access, with the minimum specification listing the need for an NVIDIA GTX 970 or better and the recommended specification listing an NVIDIA GTX 1070 or better. Though, of course, specific experiences you want to try from Steam in these Windows-based “Mixed Reality” headsets may have different requirements.
Microsoft is teasing big things for its work in the social VR and AR space this year.
Last year, as the company launched its new line of Windows ‘Mixed Reality’ VR headsets, Microsoft acquired social VR app, AltspaceVR, which had been facing closure just months beforehand. This seemed to confirm the company’s interest in the social VR space, giving it an answer to Facebook Spaces and more. But Microsoft Technical Fellow Alex Kipman says that acquisition was “just the beginning” of its work in the space.
In a blog on LinkedIn, Kipman recently wrote about what to expect from immersive technologies in 2018. The post discusses advances in VR, AR and Microsoft’s HoloLens headset, but Kipman seemed most enthusiastic about what he dubbed ‘Immersive Communication’. He wrote about how current VR can be isolating and that the killer app for these new technologies with “tear down the walls” to open up communication.
“I also think about how mixed reality will change the way I communicate in my personal life,” Kipman said. “Space and cost are limitless in mixed reality. As a father with a 7-year old daughter, with most of my family living in Brazil, I love imagining a reality where she can interact with my entire family on the weekends, to play a board game around the table with all her cousins across the globe in a socially present and immersive way.
“Our work with AltspaceVR is just the beginning,” he added. “You will see us bring this innovation forward in new ways.”
Kipman ended by noting that there would be “a lot more to share” in the coming months, suggesting that Microsoft will have big updates in the social VR and AR space this year.
Exactly what those updates could be remains a mystery. No doubt we’ll see AltspaceVR continue to grow under Microsoft’s wing, but could the company be working on new software and services that take social VR and AR to new places?
Microsoft’s powerful 4K console, the Xbox One X (formerly known as Project Scorpio) just released to the public this week. As the most powerful video game console ever made it outpaces even the PS4 Pro and virtually any PC that you could build or buy for the same $499 price tag. But one thing is sorely missing that was touted all the way back at E3 2016: virtual reality support.
Back then, a whole year and a half ago, VR support was confirmed and Microsoft even announced at its press conference that Fallout 4 VR would be making its way to the console at some point. Well, Fallout 4 VR is set to launch on the HTC Vive next month and we’ve got no indication of when exactly it’s coming to the Xbox One X, although we do know it is coming. All we’ve heard in the past year is that Windows Mixed Reality content, otherwise known as Windows VR in all honesty, is coming to Xbox One X at some point in 2018. According to Microsoft, it doesn’t want to “distract” developers with VR support just yet.
Recently we reached out to Microsoft for comment on the status of VR support for the Xbox One X now that it has officially launched and a spokesperson had the following to say:
“Microsoft is committed to delivering great mixed reality gaming experiences. We believe that right now a Windows PC is the best platform for mixed reality as its open ecosystem and enormous installed base offer the best opportunity for developers, and Windows offers the most choices for consumers. Therefore, our primary focus is making our Windows Mixed Reality experiences a success. We have games from Microsoft Studios in development for Windows Mixed Reality, and several game developers are working closely with us to bring their titles to Windows Mixed Reality. Because of the opportunity with Windows Mixed Reality, and because we believe the user experience will be best on PC right now, that is where our focus is. We have nothing to share about mixed reality for console at this time.”
Basically they’re committed to supporting the budding line of Windows Mixed Reality headsets (note: we refer to these as Windows VR headsets because “mixed reality” is often used to refer to forms of augmented reality, but that’s not what the Windows headsets do. They’re VR headsets through and through so we call them as much.) We’ve already reviewed the Acer, covered the HP, recently received the Lenovo and area trying out the Samsung Odyssey very soon. Naturally, they want to nail support for these first and foremost. And notably, it sounds like Microsoft Studios is working on multiple different games. Hopefully they’ll all be better than the disappointing Halo Recruit experience.
All in all we don’t have much of an update yet, but we’re trying to get answers. When we know more, we’ll keep you informed. How do you feel about the prospect of Windows VR and Xbox One X VR at this point? Let us know down in the comments below!
Microsoft is planning to make a preview of its SteamVR integration available on Nov. 15.
The preview should make hundreds of VR apps available on Steam playable on headsets from Acer, Lenovo, HP, Dell and Samsung. So far, Microsoft has only offered access to developers through closed access to a Steam app.
We recently reviewed the Acer VR headset and found it lacking in the content department, so this feature could have a huge effect on sales as manufacturers just started shipping headsets to early adopters. Microsoft is launching the feature as a “preview”, which suggests it won’t be finalized and we’ve not heard an official launch date for the finished functionality.
We previously heard from Valve’s how this integration will work:
Microsoft is providing an OpenVR driver that allows SteamVR to automatically identify Windows MR hardware. From a consumer’s experience, this should allow SteamVR titles to ‘just work’ on these devices. On the dev side it is also possible for developers to tune their application for specific aspects of the headsets and/or for the Windows MR controllers.
Microsoft is jumpstarting its social VR effort by acquiring AltspaceVR and its remaining team.
Altspace was a pioneering social VR service available on all major VR headsets as well as traditional screens. It allowed people to meet up in a virtual world together and play games or share other activities. The startup burned through a hefty chunk of cash and nearly shuttered in recent months before it was unexpectedly resurrected. Now we know why they kept the lights on.
Microsoft will continue the service.
According to Microsoft, “AltspaceVR will stay AltspaceVR. Microsoft is most interested in preserving the current community that uses AltspaceVR to connect and interact with new and old friends. These first few months will focus on fostering our community and making sure AltspaceVR remains a friendly, welcoming and vibrant place to hang out in virtual reality.”
While that might be the case immediately it is easy to imagine Microsoft fast tracking an effort to make its virtual homes into places where you can invite friends, or integrating services like Skype. Such advancements would help Microsoft match some of the most compelling aspects of competing VR platforms. For example, Facebook’s Spaces lets you call people in the real world through Facebook Messenger and conference them into your virtual world while Valve’s SteamVR Home offers a customizable virtual home where you can invite friends and show them your gaming trophies. If Microsoft’s Xbox ends up ultimately supporting VR that latter feature would be a fun part of the experience.
We’ll of course have to wait and see what becomes of Altspace over time.
Over the next two weeks we are expecting major announcements related to VR from Microsoft, Google and Facebook.
It couldn’t come at a better time for VR developers who spent the summer reading headline after headline about how VR is dead, fulfilling the prediction of Unity CEO John Riccitiello that journalists couldn’t stop themselves from writing negatively about the gap between analyst estimates and actual sales in the first year or so of VR’s mainstream adoption. The so-called “gap of disappointment” was made possible by analysts who model the growth of the industry in a straight line whereas Riccitiello suggested it is far more likely to see more of a curve to adoption — a slow start that suddenly ramps at an accelerated pace.
What will drive the adoption curve is the arrival of affordable standalone VR systems that require no PC or phone to operate. We’ll see the first of these arrive in the next few months. These, too, might start off with meager adoption as the first systems will likely make sacrifices in functionality, but over the next year or two we can expect a rapid increase in capabilities as new systems arrive that will offer an all-in-one package and none of the drawbacks of current VR systems.
Here’s what to expect from announcements over the next two weeks.
Microsoft
Halo is coming to Microsoft-based immersive systems.
On Oct. 3 (Tuesday morning) Microsoft will give one final overview of its mixed reality strategy before its manufacturing partners, later this month, start shipping inexpensive but surprisingly capable PC-based virtual reality headsets. We know headsets are on the way from Dell, Lenovo, HP, Acer, ASUS and we’ve seen another leaked from Samsung. Pricing starts around $300 and they work with hand controllers which dramatically increase a sense of interactivity and overall immersion. These systems are compatible with the Microsoft’s Windows 10 “mixed reality” platform, promising compatibility with both the Windows Store and Steam, with a stellar line up of content on tap.
While all these headsets are expected to have a wired connection to a PC we know Microsoft is keenly aware of the value of a standalone system. It leads the augmented reality market with the standalone HoloLens. While HoloLens isn’t going for mass appeal as it costs $3,000, its tracking system locked Intel out of the market as it was being adapted to these less expensive PC-based VR headsets.
Microsoft is also expected to roll out the Xbox One X in November. At E3 2016, Microsoft suggested its future Xbox would be capable of high quality VR, but by E3 2017 there wasn’t a word about this feature. In between these events Microsoft revealed its partner-based strategy for Windows-based tethered VR headsets. Meanwhile AMD, which provides some of the processing chips inside the Xbox One X, purchased Nitero. Nitero made wireless technology specifically tuned to make VR headsets wireless while still offering high fidelity. The technology was good enough that it received investment from Valve before the acquisition. Could the Xbox One X have Nitero technology buried inside?
Microsoft is in a tricky place because its partners might be hurt if the company revealed a plan for the Xbox One X to support wireless VR in 2018 at the same time partners start shipping wired PC headsets. Nonetheless, Microsoft’s late 2017 entry into the VR market aims to hit a sweet spot between price, ease of setup and functionality that Facebook’s Oculus and HTC’s Vive have yet to match. The biggest remaining limitation to Microsoft’s approach — and thus the size of the VR market — is the wired connection. And Microsoft has the partnerships and technology lead needed to make that wire disappear very soon.
Google
On Oct. 4 (Wednesday morning) Google will reveal its product lineup for the holiday season.
Google’s Daydream View
Google just acquired a team from HTC responsible for the creation of the Pixel smartphone. Earlier this year the ad giant also acquired Owlchemy Labs, the creator of Job Simulator and Rick & Morty VR. During its developer conference earlier this year, Google revealed its WorldSense tracking technology ideal for VR and AR headsets.
We can reasonably expect a new version of the Pixel phone, perhaps a new Daydream headset and details around the first standalone VR headsets powered by Google expected to arrive at the end of this year from manufacturers like HTC. That said, we don’t know how you’ll interact with virtual worlds you visit inside these standalone systems.
Intel’s cancelled Alloy VR headset used an awkward combination of a headset that allowed full freedom of movement throughout a room with a hand controller that could only sense limited movement. It doesn’t feel right to be able to move your head around a virtual world freely but when you reach out and try to grab something that movement isn’t captured in the virtual world. That’s what it might feel like if Google used WorldSense tracking for a headset paired with its current Daydream controller.
Google is likely very aware of this limitation and spent years working on its computer vision technologies. So it is not out of the realm of possibility for the company to have figured out an immersive solution to this limitation with its next VR headsets.
Facebook’s Oculus
On Oct. 11 and 12 Facebook’s Oculus will be hosting Oculus Connect 4. The fourth developer conference from Facebook’s VR division will see a range of sessions and presentations outlining next steps from the company. It will also likely unleash a new VR headset prototype for developers and/or journalists to try. Last year this prototype was called Santa Cruz — a standalone headset that mashed together parts of the Rift and mobile efforts at Facebook. It offered movement freedom throughout a room.
Earlier this year reports emerged that Facebook was aiming for a $200 standalone that might be similar to the Gear VR. This system is likely not the future of Santa Cruz. In other words, Oculus is actively developing a range of standalone systems targeting both high and low ends.
In some ways Facebook is playing catch up to Microsoft and Google. The company was acquired in 2014 and has gone through an integration process that saw many employees upend their lives in Orange County, where Oculus was headquartered, and move closer to Facebook’s offices in Silicon Valley. Meanwhile, a huge injection of cash from Facebook led to the addition of lots of new people to the team alongside management restructuring. All this happened while teams at Google and Microsoft toiled away at some of the core problems, like computer vision, that Facebook would ultimately need to solve too.
What makes Facebook different from the efforts we’re seeing from Microsoft and Google is that the company doesn’t have any of the baggage from Android or Windows to support in developing its new ecosystem of VR headsets It’s a fresh start. Facebook’s VR decision makers are theoretically free to make choices that are best for the developers building for these systems and for the people who are buying them. When it comes to standalone VR headsets, Facebook’s teams likely won’t have to decide whether something works well with Android or with Windows.
Hopefully on Oct. 11 and 12 we’ll see what that difference really means.
Mixed reality (MR) is a new computing platform that‘s emerging from the current fields of augmented and virtual reality. MR blends the physical and the digital worlds into a single space, using a combination of cutting edge optical hardware and artificial intelligence (AI) software. Over the next 5 to 10 years, most industry analysts agree that the AR/VR market will be worth well north of $100 billion, with the vast majority coming from the AR side — in the form of consumer innovations in retail, location, or AI services. In terms of the enterprise market, that pie will be smaller, but will lean even more heavily toward AR and MR.
Currently, at the forefront of the MR revolution is the Microsoft HoloLens. Microsoft has uncharacteristically been early to market in this sector, with the HoloLens dominating almost all coverage of actual deployments of the technology. Although Microsoft enjoys an industry-leading position, it does face other threats from both new hardware entrants and established platform holders. Read on to see how the battle lines are being drawn — and what potential outcomes could emerge over the medium term.
In pole position
HoloLens moves the game on from the previous-generation MR devices — such as Google Glass — which provided little more than the ability to bring up text or simple icons over the user’s view. By contrast, the HoloLens is capable of both rendering full 3D objects and data as holograms, and of understanding the world around it in order to place those holograms in physical space. The self-contained nature of the HoloLens (it’s a full Windows 10 computer that doesn’t require wires or a network connection to function) makes it just as capable on a factory floor, or the middle of a construction site as it is in an office.
Thanks to the current generation’s $3,000 price, it is more likely it’ll be the next generation of MR devices that will revolutionize the consumer market. Meanwhile, industries as diverse as transportation, healthcare, industrial design and education are currently adopting MR via Microsoft’s device to disrupt existing business processes.
Current contenders
Consumer VR devices costing several hundred dollars from Oculus, HTC, and Sony run on powerful household desktop PCs that likely cost at least $1,000. They offer immersion in 3D worlds at what can be around $2,000 total, which could previously only be achieved with specialist installations costing $20,000 or more. Meanwhile, AR systems, such as Apple’s recent ARKit, allow mobile users to view the world through a window, with digital objects superimposed. Both mediums are potentially disruptive: VR primarily in the consumer space; AR primarily in the enterprise space — and have acted as gravity wells for venture funding over the 5 years, though both seem to remain perennially a few months from a great inflection point.
The emerging challengers
In new MR hardware, the two big players are Magic Leap, the $4 billion Florida-based behemoth; and Meta, the plucky Y-Combinator alumni with a comparatively teeny $73 million war chest. Despite their David-and-Goliath mismatch, Meta seem closer to actual product in the market, though neither have so far shipped devices to allow a true side-by-side comparison with Microsoft’s technology. If Magic Leap ever does ship a product, expect an all-out turf war for the enterprise space, as all that venture money may well end up being deployed to compete with an even more capable Hololens 2 that may emerge any time from late 2018.
The more interesting angle of attack on Microsoft’s current position may come from the very capable augmented reality support which both Apple and Google have added to their respective mobile platforms in 2017. The applications of ARKit and ARCore have mostly focused on the traditional ‘hold your phone up and look through it’ ergonomics, but manycompanies are looking past this by creating inexpensive head-mounted cradles which can transform a phone into a mixed reality device (think Google Cardboard for MR).
Disney’s recent collaboration with Lenovo to create a Star Wars head-mounted AR experience is based around this concept. If this approach proves popular with consumers, it could drastically narrow the considerable lead Microsoft has built up over its rivals Google and Apple, and bring MR to the consumer market — many years earlier than most analysts have forecast.
Final thoughts
Uptake of mixed reality has begun, but the sprint toward ubiquity has yet to start. Historically technologies take a while, then hit hard and transform fast. With so many major tech players investing in MR, that future seems assured — but the likely big winners right now are less clear. Ultimately, as with any new medium, it’ll be the players who find the most disruptive, high value use cases for the new capabilities MR enables. With its more consumer-focused siblings riding high in the headlines, challengers to the single dominant platform are lining up and enterprise adoption is accelerating, mixed reality is lined up for an interesting next few years.
Fracture Reality chief technical officer Rob Minson began his career in academia and holds a Ph.D in interactive simulation systems before moving into the video game industry, building interactive experiences and scalable online systems. This post by Rob Minson originally appeared on VenutreBeat.
Microsoft confirmed today it is working with 343 Industries to bring its epic Halo series to its Windows Mixed Reality platform.
The franchise is one of the biggest in gaming and a major driver of Xbox console sales over the years. Its arrival on Microsoft-backed devices could be a major driver of headset sales, though Microsoft is not providing any additional details about the project, such as its size and scope, if it’s a port of an existing game, a new entry, or something else entirely.
The Halo name appears in the below image that details the first wave of content coming to Windows Mixed Reality.
This seems to confirm that Halo VR is coming, though it needs to be pointed out that this could be referring to a Halo experience on HoloLens. That seems very unlikely given that practically every other app and company on the list is working in VR and HoloLens isn’t yet a consumer product, but it’s still a possibility.
Microsoft hasn’t even clarified whether it is working on the experience as a fully immersive world for VR headsets or for AR devices like HoloLens that insert digital objects into your view of the real world. Microsoft continues to use the term “Mixed Reality” to refer to the entire spectrum of immersive gadgets. While confusing now with very different VR and AR devices on the market, Microsoft claims to use the language as forward-looking terminology for when a single gadget can span both types of experiences.
Microsoft’s “Mixed Reality” spectrum spans a range of devices from the Acer VR headset and hand controllers (shown left, priced around $400) which hook up to a PC (starting around $500) to operate, all the way up to the standalone HoloLens AR headset (shown right) which costs $3,000 as a development kit. As technology evolves, AR devices are likely to come down in price and become more immersive while VR gadgets become standalone. Depending how long Halo is in development, it might be targeted for a future device that sits somewhere else on the spectrum.
The term “Mixed Reality” used in the context of this Halo announcement, with little more known, means there are a great number of potential ways the Halo franchise might be realized ranging from putting you inside Master Chief’s helmet for fast-paced multiplayer to something, well, not quite as dream-fulfilling. Microsoft uses the artificial intelligence character Cortana from the games as the name of its voice assistant. Cortana might ultimately become important to Microsoft’s immersive platform over time, and more than two years ago the company used HoloLens for a Halo marketing AR experience.
But the hope of gamers, of course, will be that Microsoft and 343 realize a more substantial gaming experience in VR. The Halo franchise is best known for its fast-paced first-person shooters but also expanded out to real-time strategy with Halo Wars. The latter might be a more comfortable fit for VR, but other developers are exploring a variety of options to bring well-known titles to VR. Bethesda is porting some of its biggest first-person titles like Doom, Fallout and Skyrim to VR while Valve offers a VR spectator mode for its popular Dota 2 battle arena game.
Of course, 343 and Microsoft could be working on something completely fresh for immersive headsets. Whatever the companies are cooking up, we hope for more details to emerge soon.
Windows-based VR headsets on the way from Microsoft’s partners will work with Valve Software’s Steam.
The announcement today from the companies could have long-lasting implications for the evolution of the VR industry, firmly establishing Valve’s leading PC software store as a one-stop shop for content that can run on any major PC-based VR headset. Steam already supports Vive, Rift and OSVR.
It is unclear whether the Steam support is automatic for all VR titles when a Windows-based VR headset and motion controllers are present, or whether individual developers will have to manually add support for Windows headsets one app at a time. For now, Valve and Microsoft declined to provide additional details.
“The introduction of Windows Mixed Reality headsets is [a] big step forward for VR,” Valve’s Joe Ludwig said in a prepared statement. “Working with Microsoft to include SteamVR compatibility with these devices is also a big step in growing VR as an open platform for developers and consumers. With a broad range of hardware options available from leading PC manufacturers, the Steam community will have more choice than ever to experience the amazing potential of VR.”
Microsoft’s VR hand controllers are like a mashup of of the Vive and Rift controllers. Unlike those systems, though, they don’t require sensors or markers outside the playspace to operate.
We’ve been skeptical of Microsoft’s approach to hand controllers for VR, given that they are expected to work best when in view of the headset’s forward facing sensors, but Games Editor David Jagneaux went hands-on with them recently and found them capable even when held behind his back. Many of the most immersive VR games rely on gestures like throwing or grabbing items from a virtual backpack. These gestures work great with Vive and Rift because those tracking systems are able to pinpoint the controller positions from the outside. If Microsoft’s system is able to match these capabilities the headsets could hit a sweet spot with easier setup, lower cost and great content.
