Microsoft’s HoloLens-based headset built on contract for the US Army has passed an important round of field testing by soldiers. The company is now set to fulfill a larger order to be used in more rigorous testing slated to take place in 2025.
Awarded in 2019, Microsoft’s $22 billion defense contract is aiming to supply the US Army with a tactical AR headset for soldiers based on HoloLens 2 technology, or what’s called an Integrated Visual Augmentation System (IVAS),
According to Bloomberg, 20 prototype versions of the newer 1.2 IVAS headsets were field tested by two squads of solders in late August specifically to check for improvements in reliability, low-light performance and ergonomics. It was reported in early 2022 that Microsoft was bracing for negative field testing, citing concerns with low-light performance and nausea.
Taking place at Fort Drum, New York, those tests “demonstrated improvements in reliability, low light sensor performance, and form factor,” a US Army spokesperson told Bloomberg, saying further that “soldier feedback was positive.”
The Army awarded Microsoft another contract on September 5th for the 1.2 IVAS to see if the company could scale production. Its $22 billion contract indicates an upper target, and not the full amount granted to Microsoft at present.
Remember those mixed reality combat goggles that Microsoft was building for the Army that made soldiers nauseous? (See our previous story here.) Well, they’re back.
Microsoft got its hands slapped and had to go back to the drawing board after reports came out last year that its new AR goggles were making soldiers literally sick to their stomachs. But it seems they’ve fixed the issues, and are ready for round two.
The Army just placed a new order for the Integrated Visual Augmentation System or IVAS goggles, Bloomberg reports.
Microsoft sent the Army some new prototype headsets this summer. The company apparently fixed the issues that caused headaches, nausea and pain.
The Army spokesperson said the new headsets showed “improvements in reliability, low light sensor performance, and form factor.”
I’m sure Microsoft was sweating bullets about this contract, since its consumer AR efforts seem to be dying on the vine. Apple has been grabbing all the hype with its upcoming mixed reality headset — but at $3,500 a pop, I don’t know how much traction those headsets are going to get, either. Which just leaves Meta and the Quest 2.
Meanwhile, Microsoft laid off a bunch of the HoloLens team earlier this year.
The next steps for IVAS include adding in cloud computing, the Army Times reports. This will let soldiers download apps for specific mission needs.
The Army wants to avoid overloading IVAS by offloading apps to the cloud instead of the device. During testing, soldiers used the goggles for assault planning, mission practice, targeting, and more.
IVAS lets them ditch the sand table to quickly scout and rehearse missions virtually.
Rather than an MRE-box sand table, a unit could virtually “see” the terrain in their heads-up display and rehearse a mission in their patrol base before leaving the wire,” Brig. Gen. Christopher Schneider told Army Times.
“Now we have to make this system producible and affordable,” he added.
Earlier issues around night vision, size, and weight are getting fixed bit by bit. The goal is to nail down cost and manufacturing in 18 months.
If all goes well, IVAS could start hitting units by 2025. Of course, that’s assuming the cloud tech actually works as advertised. And that Congress keeps funding the project.
IVAS has night vision, heat sensors, 3D mapping and other HoloLens features. It’s meant to give soldiers more awareness by layering digital info onto the real world, the company said.
To get input, Microsoft engineers did mock bootcamps in 2019, where they learned skills like navigating at night. This helped them design IVAS to handle tough conditions soldiers face.
After soldiers tested IVAS for around 80,000 hours by early 2021, Microsoft had a headset ready for combat use.
I’m not sure why they missed the whole nausea thing the first time around. Maybe the engineers had been using the headset so much themselves, during the whole development process, that they were used to it? Or it was so much better than the early iterations, that the nausea didn’t even register as a problem any more?
At least from the outside, it appears Microsoft isn’t actively competing for a seat at the XR table, which is fairly odd coming from a company that pioneered enterprise AR while simultaneously wrangling some of its top OEM partners to make a fleet of PC VR headsets for consumers in 2017. Microsoft gained a great early start, but now the Redmond-based tech giant is positioned to play catchup, which historically hasn’t worked out that well. Could we be in for another ‘Zune moment’? If Microsoft goes in half-cocked, maybe.
Microsoft released the first-gen Zune in 2006, an MP3 player that looked to compete with Apple’s largely dominant line of iPods. By “largely dominant,” I mean Apple not only had majority market share of the product category, making it synonymous with portable music at the time, but had already produced numerous generations of iPod Classic, iPod Mini, iPod Nano and iPod Shuffle. Apple wasn’t the first to make a portable MP3 player, although it was the first to make one everyone wanted.
Now, I can hear the Zune defenders in my head, and I sympathize. Zune wasn’t terrible, and it came at a time when full-color screens in MP3 players were just becoming a thing. It had a compelling reason to exist, which is why Microsoft directly competed against iPod Touch over the course of three device generations before eventually giving up the goat in 2011 and discontinuing the third-gen Zune. Many chalk it up to poor marketing, lack of brand cache, and not enough music to choose from. Zooming out, Zune’s ultimate defeat belies a larger pattern of behavior.
Image courtesy Digital Trends
Zune didn’t generate the sort of loyal customer base that Apple had in spades because entering rapidly evolving product categories isn’t easy. By the time platforms solidify, companies that come too late are usually tasked with flipping what’s left of undecided users or attracting users away from other ecosystems with unique selling points. Even with viable hardware on your side, it’s not an easy thing to do.
To put it into perspective, Zune entered the market one year before Apple announced the first iPhone. From that moment Microsoft was forced to play catchup not only with its MP3 players, but with its widely maligned Windows Phones which came afterwards, of which there are famously few defenders. Needless to say, Apple’s iPhone is still kicking, and that iPod/iPhone success story is why Apple is largest company in the world.
Breaking the Zune Curse?
Don’t get me wrong, Microsoft has success stories. Windows is still the world’s largest PC operating system. Azure Cloud Platform competes alongside AWS and Google Cloud. There’s a reason why we call digital slideshows a PowerPoint no matter which program you use to make them, and that’s thanks to Microsoft’s ongoing dominance in the general computing space. When Microsoft gets in early and sticks it out, you don’t generally get a Zune.
To its credit, the company had the foresight to release HoloLens in 2016, a full two years before unicorn startup Magic Leap could get its first standalone AR headset out the door. Three years later it released HoloLens 2, which directly competes today against Magic Leap Two. When HoloLens 3 will arrive, or whether it’s even in the works, still isn’t clear. We’re hoping they stick it out and it doesn’t turn into a ‘Zune moment’ down the line.
The first wave of WMR headsets launched in 2017 | Image courtesy Microsoft
In 2017, Microsoft also managed to assemble a host of major OEMs to create what would be the first Windows VR headsets, which included PC VR headsets from Dell, Lenovo, Acer, HP, Samsung, and Asus. It was a good opening gambit to break up the Oculus/HTC Vive PC VR binary that had developed a year prior, although those Windows VR headsets weren’t just new hardware destined to hook into Steam content. Microsoft made its own Windows Mixed Reality Store which ultimately failed to compete with Steam for developers, which was kind of like a Zune owner somehow getting all their music from iTunes and not Zune Marketplace.
And we’re still early, although that may not be the case for long. Compared to smartphones today, the current XR landscape is toddling out of its infancy. You’d be surprised how much competition there is already, not only across multiple hardware platforms, but entire content ecosystems—something you can’t just grow over night. Currently major contenders are Meta, Sony, HTC, Valve, Pico, Pimax, and Apple starting next year. The future leaders are shaping up to be Sony, Meta and Apple, the last two moving into mixed reality (Meta Quest Pro, Meta Quest 3, Apple Vision Pro) which feature VR displays and color passthrough cameras for AR tasks, while Sony is already in their second-gen PlayStation VR. Things are changing, and Apple jumping into XR could see a host of other companies deciding they want a piece of the pie fairly soon.
Whatever the time frame, eventually the amount of money Microsoft leaves on the table is going to pile up until it can’t be ignored. That’s essentially the strategy the company has decided to take with Xbox at least, with Xbox Game Studio head Matt Booty saying in a recent Hollywood Reporter interview that VR just isn’t big enough yet.
“We have 10 games that have achieved over 10 million players life-to-date, which is a pretty big accomplishment, but that’s the kind of scale that we need to see success for the game and it’s just, it’s not quite there yet with AR, VR,” Booty told the Hollywood Reporter.
So, while we’re no closer to knowing when Microsoft will decide it’s the right time to enter into VR (or MR for that matter), the company is well equipped and funded to break the Zune curse. Whenever Microsoft chooses to compete in consumer XR, any potential failure can’t be blamed on the lack of resources. The company now boasts a vast collection of game studios it can weaponize, which includes the entire Zenimax family of studios, including Bethesda, Arkane Studios, id Software, MachineGames, Tango Gameworks, and ZeniMax Online Studios. Provided the contentious Activision Blizzard acquisition goes through, Microsoft will also own World of Warcraft, Call of Duty, and Diablo franchises. That untapped library of IP and developer talents could make whatever Microsoft decides to bring to the XR table a serious contender.
Just the same, if the megalithic Microsoft can’t overcome what must be a massive internal friction to put out something focused, timely and well-supported, whatever it makes might as well be Zune.
Microsoft technical fellow and mixed reality figurehead Alex Kipman is reportedly leaving the company in the wake of misconduct allegations, leading to a reorganization in its MR efforts.
Late last month Insider reported [paywall] on claims of inappropriate behavior from Kipman, who heads up Microsoft’s HoloLens and mixed reality work. One alleged incident included Kipman watching an “overtly sexualized pillow fight” in VR in front of staff, with another claiming the executive rubbed the shoulders of a female employee as she “looked deeply uncomfortable”.
Another article from Insider today now says that Kipman has resigned from the company. GeekWire has since published an apparent internal email from Scott Guthrie, the head of Microsoft’s Cloud & AI Group, both announcing the departure and a shake-up for the wider mixed reality team. Microsoft has not responded to either article.
The reported email explains that Kipman will remain at Microsoft for two more months to help with the transition, but doesn’t make note of the allegations against him. “Over the last several months, Alex Kipman and I have been talking about the team’s path going forward,” it reads. “We have mutually decided that this is the right time for him to leave the company to pursue other opportunities.”
The rest of the email, meanwhile, explains that Microsoft’s Mixed Reality Hardware teams are being integrated into the Windows + Devices organization. “This move will integrate our Mixed Reality hardware teams as part of Microsoft’s broader end-user device hardware organization,” the email reads. “Both HoloLens and IVAS are built using Windows, and this move further aligns our client platform efforts.”
It also states that the Mixed Reality Presence and Collaboration teams will join the Teams organization.
Earlier this year another report from Insider claimed a HoloLens 3 device had been canceled in favor of a partnership with Samsung, and that Microsoft’s mixed reality efforts were plagued with “confusion and strategic uncertainty”.
Microsoft is supposedly gearing up to field test its HoloLens-based military AR headset, however a new report contends the company is bracing for impact, as it’s expecting negative feedback from soldiers.
Last year, Microsoft announced it had won a United States Army defense contract worth up to $22 billion which would see the development of a so-called Integrated Visual Augmentation System (IVAS), a tactical AR headset for soldiers based on HoloLens 2 technology.
ABusiness Insider report, citing a leaked internal email, maintains that Microsoft has low expectations for its latest version of IVAS, which is set to begin real-world operational tests with the US Army in May.
Prototype testing (2019), Image courtesy CNBC
Microsoft’s IVAS contract has allegedly seen delays and quality problems. A separate Business Insider report from last month alleges its enterprise-focused HoloLens 3 may also be at risk due to internal issues within Microsoft’s mixed reality division surrounding whether HoloLens should serve consumers or continue courting enterprise companies.
A purported Microsoft Teams message from Mixed Reality division head and HoloLens co-inventor Alex Kipman paints a pretty depressing story:
“So depressed, so demoralized, so broken. I’m sure by now you’ve read or heard about one or two of the Business Insider articles that were published on us. On our private roadmap. On our customers’ confidential data … as a consequence of these articles and these individuals shameful actions, someone from finance already came to me to ask if we should lock down and not share so openly our numbers. Someone from marketing already came to me and asked if we should lock down and not share so openly our roadmap. Someone for from our National Intelligence and Security Team already came to me to ask if we should lock down our IVAS work.”
Kipman rebuffed the previously report of unrest, saying “don’t believe what you read on the internet.”
It’s said that soldiers may take issue with the device’s low light and thermal imaging performance, and that user impressions will “continue to be negative as reliability improvements have been minimal from previous events.”
That $22 billion is an upper target and not the full amount granted to Microsoft at present. And it seems confidence in the project isn’t very high at the moment, as US Congress has allegedly frozen $394 million from the Army’s IVAS budget, which Business Insider notes leaves only $405 million—around $200 million shy of what Microsoft supposedly needs to recover development costs.
Additionally, it’s also said some close to the project fear the Army will simply walk away from the contract.
Microsoft’s enterprise-focused HoloLens 3 may be dead in the water, as a recent report maintains that internal divisions have hobbled the company’s efforts to release its next AR headset as planned. In the days following the report’s release, HoloLens co-creator Alex Kipman responded, saying “don’t believe what you read on the internet.”
A Business Insider report from earlier this month maintains that plans to release HoloLens 3 are shifting behind the scenes. This comes alongside an alleged partnership with Samsung that would see the development of a wholly new consumer AR device that is rumored to tether to a Samsung smartphone.
This has allegedly caused divisions within the company surrounding whether HoloLens should serve consumers or continue courting enterprise companies. Rubén Caballero, Microsoft’s mixed reality and AI device engineering VP under Kipman, is reportedly pushing to focus on consumers, whereas the company has historically geared HoloLens for enterprise and the military.
Alex Kipman wearing HoloLens 2, Image courtesy Microsoft
Making an AR headset accessible enough for consumers is a vastly different challenge to producing higher cost enterprise headsets—just ask Magic Leap. If true, it would represent a significant change of course for the HoloLens product line.
There’s some ostensible background radiation related to employee attrition here too. The team has demonstrably thinned out over the last few months, with HoloLens veterans such as principal optical architect Bernard Kress leaving the company recently for Google Labs. A number of other high-profile members, including mixed reality technical fellow Don Box, engineer of computer vision Dave Reed, and former HoloLens engineering director Josh Miller have all left for Meta, along with a reported 70 others over the past year.
The report maintains progress on fulfilling its $22 billion US defense contract, which aims to put HoloLens in battlefield roles over the next 10 years, has also been stymied by internal production issues.
Microsoft Responds
Microsoft spokesperson Frank Shaw reaffirmed HoloLens’ importance to the company, telling Business Insider that HoloLens is “a critical part of our plans for emerging categories like mixed reality and the metaverse. We remain committed to HoloLens and future HoloLens development.”
HoloLens co-creator Alex Kipman also voiced his concern via Twitter, but declined to comment on the specifics.
don't believe what you read on the internet. #HoloLens is doing great and if you search said internet they also said we had cancelled #HoloLens2… which last I checked we shipped with success [)-)
In a follow-up tweet, Kipman says he hopes HoloLens 3 will be as “mind blowing” of an upgrade as the 2019-era HoloLens 2 was over the 2016 version, both of which were class-leading standalone AR headsets.
‘IVAS’ configured HoloLens 2 for US Defense, Image courtesy Microsoft
Kipman’s openness about the existence of the next HoloLens signals at least that Microsoft isn’t throwing in the towel, although the report may suggest Microsoft is moving forward with a more software-forward approach with AR instead of solely hocking its own hardware—pretty much par for the course for the company.
If those claims can be believed, it’s also possible HoloLens 3’s focus has changed so drastically that the device is no longer purely enterprise-focused, but instead split in two separate projects: one to service the US military contract and another to appeal to consumers.
In any case, Samsung’s reported partnership with Microsoft sounds like very familiar territory for the South Korean tech powerhouse. Samsung partnered with Oculus (now Meta) back in 2014 to build Gear VR, a snap-in VR smartphone holder that was a generational leap in terms of low-latency mobile VR.
And as Microsoft no doubt looks to compete with Meta and Apple, both of which are making strides to release AR headsets for consumers in the near future, we’re sure to see its strategy out in the open sooner rather than later.
Reports from Bloomberg and The Wall Street Journal indicate Microsoft and Apple employees are leaving to join Meta Platforms, as part of the latter company’s increased push into AR. To counter, Apple are offering some employees lucrative and unusual stock options as bonuses if they stay.
However, The Wall Street Journal reports that the program faced technical difficulties, particularly involving bringing high-quality night vision to the HoloLens. These strains reportedly enticed some members of the team to consider competitor offers. Despite this, Microsoft told the Wall Street Journal that it has a “strong team and is making progress on the project.” In October, the US Army announced it would move further testing of the project into 2022.
To counter Meta’s recent interest in its employees, Apple is reportedly offering some of its engineers lucrative and unusual stock bonuses to entice them to stay. These options, which would be vested over four years, are being offered to only some engineers in “silicon design, hardware, and select software and operations groups.” The bonus amounts range from $50,000 to $180,000, with “as much as $120,000 in shares.”
The United States Army says it recently conducted a test of its Microsoft HoloLens-based AR platform and will move further testing of the hardware to 2022.
The Integrated Visual Augmentation System or IVAS is based on Microsoft’s HoloLens AR technology and the contract to supply the United Stated military with the technology represents a major vote of confidence in Microsoft’s platform and a key testing ground for the company to develop ruggedized AR tech. A report from Janes suggested the Army is “essentially doing a reset of” the program to figure “out what is the appropriate timeline and where is the technology.”
“The Army is fully committed to its partnership with Microsoft to advance specific technologies to meet operational requirements and maximize warfighter impact,” a press release from the Army states. “The Army conducted an Adversarial Electronic Warfare and Cybersecurity Test in September 2021, and plans to execute testing regularly throughout FY22. This decision allows the Army and Industry team to continue to enhance the IVAS technology platform ensuring Soldiers achieve overmatch in Multi Domain Operations. The Army intends to continue developing and fielding this revolutionary, first-of-its-kind technology in FY22.”
The U.S. Army provided a project timeline with May 2022 listed as the month for an operational test and September of 2022 for the first unit equipped with the hardware. Here’s the full timeline as provided by the Army:
Project Timeline:
OCT / NOV 2020: Soldier Touchpoint 3 (STP 3) & Distinguished Visitor Days
DEC 2020: Rapid Fielding Decision
JAN 2021: Vehicle Integration VE2 – Stryker & Bradley
FEB 2021: Cold Weather Test
MAR 2021: Production Award
MAR 2021: Tropical Weather Test
APR 2021: Soldier Touchpoint 4 (STP 4)
JUL 2021: OT Entrance Criteria User Jury
SEP 2021: Vehicle Integration VE3 – Bradley
SEP 2021: Adversarial Electronic Warfare & Cybersecurity Test
Microsoft reconfirmed it is “absolutely” working on a “consumer journey” for its HoloLens AR headset, even if that’s some ways off.
Speaking to Wall Street Journal, Technical Fellow Alex Kipman touched on the future of the platform. “But you don’t get to lead a new medium of computing if you’re not going to be in consumer,” Kipman said. “So we are absolutely working on a consumer journey for HoloLens. I’m happy to confirm that and say that is a very important part of our strategy.”
Consumer HoloLens Coming
Kipman didn’t give any kind of roadmap for an eventual launch of a consumer HoloLens, but it’s likely not going to happen for a while. The HoloLens 2, which we reviewed earlier this year, is an excellent but expensive ($3,500) AR headset with a lot of features that don’t quite measure up to consumer expectations. Improving these features whilst also bringing the kit’s price down is a challenge not just Microsoft but every hopeful AR competitor faces in the coming years.
Magic Leap learned the hard way when it pivoted its AR glasses away from consumer and towards enterprise, while rumors and speculation around Apple’s AR headset have persisted for years, most recently suggesting it might not arrive until 2025. Even HoloLens itself was first introduced with AR videogames and entertainment apps before Microsoft truly positioned it towards businesses.
As for HoloLens 3, Kipman also assured Microsoft was working on it and that it had to be “another huge leap forward”. Again, no word on when we might see that device.
Would you buy a consumer HoloLens headset? Let us know in the comments below!
Pokémon Go is in a class by itself with near-universal appeal and it will be a guiding light for AR development in the years to come.
Perhaps I’m a bit warped by my attachment to the creatures I started capturing on my Gameboy before the turn of the century. No matter where I went with my family, I was immersed in a world that lived inside my head as much as it did on that simple little monochrome screen.
My Squirtle evolved into Wartortle, who evolved into Blastoise, as he and I worked together to grow our bond and expand our collection of pocket monsters on a journey through the world together. I even had one of those magnifiers with built-in light so I could play at night with a bigger screen, consuming AA batteries as much as I did candy.
Decades later, Pokémon is still played by millions searching for monsters that roam unseen around the physical world. It is difficult to say exactly how many people play Pokémon Go each month as its developer, Niantic, remains a private company so its revenue and userbase can only be estimated by firms like Sensor Tower.
While the below chart is an estimate, the overall trend is likely true — Pokémon Go was a sensation in 2016 when it debuted, crashed the following year, and then steadily improved its revenue and game mechanics to grow overall. The evolution continued even through the COVID-19 pandemic when game changes made it easy to enjoy even while staying physically distant from other people.
The ugly truth of Pokémon Go in 2021 is the augmented reality portion of the game remains one of its worst features. There’s no reason to throw a Poké Ball and try to catch a Pikachu in AR except to make it harder to catch. We only hold our phones vertically like the feature requires when we’re taking a picture, and I’m not taking a picture when I’m catching a monster. Pokémon AR remains a terrible experience. For this to become great, our computers need to become glasses and they need to understand our environment to a degree we’ve still not seen in consumer hardware.
And with Microsoft, Niantic recently teased full multiplayer Pokémon Go on HoloLens 2.
Facebook is working on AR glasses and Apple’s CEO Tim Cook was recently quoted as saying that AR was “critically important” to the future of the company while also stating person-to-person interactions could arguably be “better if we were able to augment our discussion with charts or other things.”
The Pokémon universe will be absolute magic when it is finally realized with true head-worn AR and robust environmental understanding. The idea of leaving home on an adventure and befriending new creatures wherever you go is inviting to many groups of people in a way very few gaming franchises are. And the fact that Pokémon don’t die when they’re defeated in battle (they only faint) doesn’t hurt its appeal either. In my mind the franchise’s core tenets can be thought of as lasting friendship and harmless rivalry, and I can’t think of a layer I want added to reality that would be better than that.
That’s why — after all the technical milestones are met and you can release a Poké Ball from your fingertips and see it hit the ground in a cloud of dust where a giant creature bursts out — Pokémon Go should continue to be a guide shaping the development of AR over the long term.
In the year 2050 will people be catching Pokémon like I’ve described here? I think they will.
