Spatial, a New York-based startup behind the eponymous real-time AR/VR collaboration platform, has secured $14 million in additional financing, bringing its overall funds to $22 million.
The latest funding round was led by White Star Capital, iNovia and Kakao Ventures, with continued participation from Instagram co-founder Mike Krieger and Zynga founder Mark Pincus. Existing Spatial investors also include the likes of Samsung NEXT, Baidu Ventures, and LG Ventures to name a few.
Founded in 2016 by Anand Agarawala and Jinha Lee, the company’s social AR/VR workspace has since gone on to include support for Microsoft HoloLens 2, Oculus Quest, Magic Leap 1, Qualcomm XR2, Android/iPhone mobile device and traditional monitors.
The company emphasizes the platform’s potential to let knowledge workers seamlessly connect through virtual spaces, thereby replacing the need to travel for meetings and collaborations. Users can chat using more lifelike avatars built through a quick 3D scanning process, open browser windows, and view and manipulate 3D objects as if they were in the same room together.
We had a chance to go hands-on at MWC 2019 last year, and again at CES 2020 earlier this month. Spatial has made a compelling argument for its workspace tech, and while it’s equally early days for the company and augmented reality, it’s clear Spatial is headed in an inevitable direction.
Check out the on-stage demo from MWC 2019 last year to get a better idea of what Spatial is all about:
Microsoft began shipping its latest AR headset, HoloLens 2 in November. After a slow initial rollout, devices are getting out into the hands of more developers and reports have surfaced that many units are exhibiting significant color-consistency issues. Microsoft has acknowledged the problems and says its working with customers to understand the cause.
HoloLens 2 is Microsoft’s latest AR headset, boasting a larger field of view, greater resolution, and improved hand-tracking compared to the original HoloLens.
While the original headset wasn’t exactly know for a high level of color-consistency, several HoloLens 2 units that we’ve seen appear to be much worse off, showing an obvious rainbow-like pattern over virtual imagery displayed by the headset.
A through-the-lens view of HoloLens 2 | Image courtesy Reddit user hegemonbill
It’s unclear how widespread the issue is, but Microsoft confirmed that it’s aware of the problem and working to identify the cause. A Microsoft spokesperson shared the following with Road to VR:
“Microsoft continues to invest and innovate in the field of display technology. Microsoft HoloLens 2 contains a new type of display that more than doubles the field of view of the original HoloLens and is the result of a set of balanced display trade-offs. We are aware of reports from some developers experiencing issues with their displays and we’re working closely with them to understand the underlying cause.”
HoloLens creator Alex Kipman responded to a twitter user who posted pictures of the color-consistency issue.
In a string of tweets Kipman said that photos of the headset’s display through a camera wouldn’t look accurate because the headset incorporates eye-tracking into its display. He also encouraged those with issues to reach out to contact him:
Friends, we have a binocular system that forms an image at the back of your eyes not in front of it. Eye tracking is fully in the loop to correct comfort which also includes color.
Eye relief (the distance from lens to your pupil) changes the image quality. Further out you are, worse the image quality becomes in terms of MTF as well as color uniformity.
Taking monocle pictures from a phone (or other camera) is completely outside of our spec and not how the product is experienced.
When you look at it with both eyes, at the right eye relief (somewhere between 12-30 mm from your eyes) with eye tracking turned on, you experience something very different.
If you are having issues experiencing our product, first our apologies, second please get a hold of us (akipman@microsoft.com is your friend) and let’s engage on how we can solve your issues. Team is fully leaned in and listening.
Granted, in all five or so of the HoloLens 2 units which I’ve personally tried, it was immediately apparent that the colors across the display were highly inconsistent, which was the impetus for asking Microsoft if they were aware of the issue. With a $3,500 price tag, I can understand why developers getting headsets with this issue would be concerned.
On the plus side, HoloLens 2 seems to be everything else that Microsoft has promised with regards to improved field of view, resolution, and hand-tracking. It also seems to have exceptional ergonomics (when used with the top strap) thanks to its light weight, balanced design, and large eyebox.
Microsoft today announced that its latest AR headset, HoloLens 2, is finally shipping. The headset can be purchased directly for $3,500 or rented for $125 per month.
Microsoft announced HoloLens 2 back in February and has started selling the headset today. Though pricey at $3,500, HoloLens 2 is actually cheaper than its predecessor which launched at $5,000. In addition to the outright purchase price, Microsoft is also renting headset to businesses for $125 per month, and will later launch a developer edition priced at $100 per month.
Image courtesy Microsoft
Like the original, HoloLens 2 is positioned as an enterprise-focused device. Microsoft envisions companies using the headset for design, remote assistance, training, visualization, collaboration, architecture, construction, and much more.
HoloLens 2 Specs
The official specs confirm the headset’s weight at 566 grams and ‘active use’ battery life at 2–3 hours. Here’s the full spec sheet:
Display
Optics
See-through holographic lenses (waveguides)
Resolution
2k 3:2 light engines
Holographic density
>2.5k radiants (light points per radian)
Eye-based rendering
Display optimization for 3D eye position
Sensors
Head tracking
4 visible light cameras
Eye tracking
2 IR cameras
Depth
1-MP time-of-flight (ToF) depth sensor
IMU
Accelerometer, gyroscope, magnetometer
Camera
8-MP stills, 1080p30 video
Audio and speech
Microphone array
5 channels
Speakers
Built-in spatial sound
Human understanding
Hand tracking
Two-handed fully articulated model, direct manipulation
Eye tracking
Real-time tracking
Voice
Command and control on-device; natural language with internet connectivity
Windows Hello
Enterprise-grade security with iris recognition
Environment understanding
6DoF tracking
World-scale positional tracking
Spatial Mapping
Real-time environment mesh
Mixed Reality Capture
Mixed hologram and physical environment photos and videos
Compute and connectivity
SoC
Qualcomm Snapdragon 850 Compute Platform
HPU
Second-generation custom-built holographic processing unit
Memory
4-GB LPDDR4x system DRAM
Storage
64-GB UFS 2.1
Wi-Fi
Wi-Fi: Wi-Fi 5 (802.11ac 2×2)
Bluetooth
5
USB
USB Type-C
Fit
Single size
Fits over glasses
Weight
566g
Image courtesy WalkingCat
Aside from field of view and resolution improvements, HoloLens 2 also launches out of the gate with full hand-tracking. This is a huge improvement over the ‘air tap’ gesture that was relied upon with the original headset. With hand-tracking users will be able to more naturally and directly interact with augmented reality content.
Epic Games today released a new video featuring a demo for HoloLens 2 that aims to show off just what sort of graphics can be achieved on Microsoft’s latest standalone AR headset. Called Apollo 11: Mission AR, the interactive demo is streamed wirelessly in real-time from networked PCs running the company’s game engine, Unreal Engine.
Unveiled earlier this summer at Microsoft Build 2019, Apollo 11: Mission AR is a recreation of the historic 1969 Apollo 11 mission and lunar landing, showing off the Saturn V’s launch, a reenactment of the lunar landing, and Neil Armstrong’s first steps on the Moon, which Epic says was reconstructed based on data and footage from the actual mission.
Epic says the demo features 7 million polygons in a physically-based rendering environment, and includes fully dynamic lighting and shadows, multi-layered materials, and volumetric effects.
Image courtesy Epic Games
That isn’t done on-device though. To achieve this level of detail, Epic says the experience’s “holographic elements” are actually streamed wirelessly in real-time from networked PCs running UE 4.23, the current version of Unreal Engine.
According to Epic’s HoloLens 2 streaming guide, the headset sends eye tracking, gesture, voice, current device pose, and spatial mapping input to your PC, and then streams rendered frames back to HoloLens 2. This, the company says, is designed to boost app performance, and make development easier since devs won’t need to package and deploy the app on-device before running it, however it’s clear it also allows HoloLens 2 to play host to more graphically involved experiences than were originally intended for the standalone device’s on-board processors.
Image courtesy Epic Games
We reached out to Epic to see whether this could also be achieved via cloud streaming, or if it’s a local machine-only implementation. We’ll update this article as soon as we hear back (see update below).
Released in early September, Unreal Engine 4.23 is the first iteration of the company’s game engine to feature production-ready support for HoloLens 2, which includes tools such as streaming and native deployment, emulator support, finger tracking, gesture recognition, meshing, voice input, and spatial anchor pinning.
Outside of the demo’s visual polish, Epic says Apollo 11: Mission AR also shows support for UE4 Composure, color temperature, and post-processing, plus OCIO LUTs, I/O for AJA video systems, and additional features that streamline mixed reality media production.
Update (2:00 PM ET): An Epic Games spokesperson has left us with this statement regarding cloud rendering for remote PC-to-HoloLens connections:
“While it is technically possible to use the HoloLens 2 Remoting over the Internet, we would strongly recommend against it due to significant latency and uncontrollable network conditions. When using HoloLens 2 Remoting, you should always aim to use a local network to minimize the latency and ensure there are minimal other devices connected to it to maximize the bandwidth available for the HoloLens 2.”
The second-generation HoloLens, first unveiled at Mobile World Congress earlier this year in Barcelona, still doesn’t have a firm release date yet, although it appears that’s about to change pretty soon.
As reported by Reuters, Microsoft’s executive vice president Harry Shum took the stage at the World Artificial Intelligence Conference in Shanghai yesterday, saying that HoloLens 2 will go on sale sometime in September.
When asked to clarify Shum’s reported launch announce, Microsoft issued this statement to several outlets, including Engadget and Tech Radar:
“As we announced in February, Microsoft HoloLens 2 will begin shipping later this year. We have started collecting expressions of interest for HoloLens 2 preorders and, as part of our standard practice for gathering feedback, have shared near-final prototypes with some customers, but have not announced a date for general availability.”
Important to note is the company didn’t disavow the executive flub outright, which could mean Shum simply shared launch plans that weren’t cleared for a public audience. Considering Microsoft admits to having shared near-final versions of the headset with its earliest customers, at the very least it’s clear we’re getting very close to a general launch.
Image courtesy Microsoft
The company’s second standalone AR headset boasts a few important upgrades over the original, although it brings much more than just a bigger FOV. It has better ergonomics, better hand-tracking, better object interaction, and includes both eye-tracking and voice input. Check out our in-depth hands-on with HoloLens 2 for more.
Priced at $3,500, it’s doing all of that at a slightly higher price than the original, which launched in mid-2016 for $3,000. This of course puts both headsets well outside the reach of consumers, placing it squarely in the realm of the enterprise sector, although a general launch will undoubtedly bring HoloLens 2 into the hands of early developers eager to explore just what material benefits the company’s latest AR headset brings to the table.
Microsoft’s first-gen HoloLens, a device initially launched to developers in early 2016, has received its last major OS update, moving on to its Long Term Servicing (LTS) state.
The original HoloLens will continue to receive monthly servicing updates, although the company hasn’t detailed how long these will last.
At this point, if you’re looking for a growing feature set, the company’s upcoming $3,500 HoloLens 2 is your best bet, although it’s admittedly only available for pre-order at this time.
HoloLens (1st gen) is entering Long Term Servicing (LTS) state. Future updates will focus on issue and security fixes, while maintaining feature parity with the Windows 10 October 2018 release for HoloLens (also known as RS5).
Windows 10 October 2018 release brought a number of new features to HoloLens, including a quick actions menu, start/stop video capture, casting to Miracast-enabled devices, and a few more quality of life updates.
There’s still no broader launch date for its predecessor, HoloLens 2, although now with initial OpenXR support, it appears the new headset will be one of the first out the door to feature the new API, which boasts greater interoperability between VR and AR hardware, game engines, and content.
Microsoft today announced they’re releasing a developer-focused HoloLens 2 bundle that aims to make the AR headset and software tools more accessible to developers and businesses.
Aptly named the HoloLens 2 Development Edition, Microsoft is pitching the bundle staring at $99 per month/per user.
The bundle includes the HoloLens 2 hardware, $500.00 in Azure credits for use with the company’s Azure mixed reality services, and a three-month free trial of Unity Pro and the Unity PiXYZ Plugin for CAD data.
The standard HoloLens 2 offering unveiled at MWC this year is still in its pre-order phase, however having the ability to pay a monthly fee versus plonking down the full $3,500 for the headset alone may be more palatable for businesses and developers looking to get their feet wet with the device.
Microsoft sees their HoloLens 2 Development Edition as a way to catalyze what it calls the “third wave of computing,” something it hopes will help them engage three times as many AR developers as it did this past year—more than 20,000 developers, by Microsoft’s reckoning.
Broadening its goal to reach more developers, Microsoft will also be hosting more meetups, programs, events and hacks, such as today’s Mixed Reality Dev Days event.
You can learn more about the HoloLens 2 Development Edition hololens.com to learn more about the HoloLens 2 Development Edition and sign up to stay updated about latest news, mixed reality toolkits, code samples and open source projects.
Microsoft won a decisive $480 million defense contract with the U.S. Army late last year to bring the company’s AR hardware platform to the battlefield. Now CNBC has gone hand-on with the modified headset based on the enterprise-focused HoloLens 2.
Dubbed the Integrated Visual Augmentation System (IVAS), the headset is still in its initial phases. At the moment, IVAS is ostensibly a stock HoloLens 2 sporting custom software, however CNBC learned the headset was also outfitted with a ball-shaped thermal sensor created by Flir, a company creating thermal and visible light imaging hardware; current night vision can be obfuscated by smoke, however thermal imaging doesn’t suffer from that problem, making it an ideal addition to the headset.
The Department of the Army has secured 12 such sensor contracts in total, so there’s no telling what else will come to the headset and in what configuration in the near future.
Image courtesy CNBC
Under Secretary of the Army Ryan McCarthy said he expects a more slimmed-down version of the headset based on HoloLens 2 to materialize as soon as 2022 and 2023, and maintains the Army will begin “fielding it to thousands and thousands of soldiers across the force.”
As with all things military-related, no candid video or images were permitted, however the Army did provide a few approved images of the headset as it is now.
Image courtesy CNBC
Trying on one of the IVAS headsets, CNBC’s Todd Haselton maintains he was able to see 3D images, information, his actual location on a map, and a thermal view of the world around him.
It gave me a birds-eye view of the building I was standing in and also showed a nearby building. It’s like any satellite image you can find online.
But as I turned my head, a small arrow icon representing my location also turned. I could also see several other dots representing my other “squad members” who were also wearing the headsets.
The Army currently intends on using the headsets for training purposes during its rapid iteration phase—it doesn’t fit under a helmet just yet—however it’s clear that the potential advantages will be a boon for soldiers on the ground in the future. Seemingly taking its cues from first-person shooters, Haselton also remarked that the system displays a weapon’s reticle as well.
“Whatever we do on the ground plays off of what we do in training for real-time missions and the real world,” one soldier said. “We might not know what [the battlefield] looks like, but we can predict and take Google images and implement that into the IVAS. It’s a huge boost to rehearsal.”
a mock-up of what the overhead map looked like, Image courtesy CNBC
IVAS also provides what the Army calls “after-action” reports, giving soldiers a summary of their accuracy and performance after a training exercise. The headset can also gather data such as a soldier’s hear rate to improve training and marksmanship.
Speaking to a group of special forces soldiers tasked with IVAS training, the soldiers continuously referred to the headset as a “combat multiplier,” or a supporting technology that significantly increases the relative combat strength of a force while actual force ratios remain constant.
All of this hasn’t gone unchallenged though. After the defense contract was announced in November, a coalition of Microsoft employees wrote an open letter to company CEO Satya Nadella and President Brad Smith, stating that “intent to harm is not an acceptable use of our technology.”
While the company has previously licensed tech to the U.S. Military, it has never crossed the lines into weapons development. With this contract, it does. The application of HoloLens within the IVAS system is designed to help people kill. It will be deployed on the battlefield, and works by turning warface into a simulated “video game,” further distancing soldiers from the grim stakes of war and the reality of bloodshed.
“I appreciate their concerns,” McCarthy told CNBC, referencing the letter, “but these are the same men and women that are protecting their freedoms so they can develop this technology.”
Evidenced by CNBC’s recent hands-on with the soon-to-evolve headset, it appears Microsoft hasn’t given into any of the group’s demands, which stipulated cancelling the IVAS contract, stopping development on all weapons technologies, and appointing an independent ethics review board.
When it comes to the development of videogames and virtual reality (VR) experiences in particular, for most indie developers there’s a simple choice when it comes to game engines. Either Unity or Epic Games’ Unreal Engine 4. For those working with the latter, they’ll be pleased to know that Unreal Engine 4.22 has just arrived, adding all sorts of new features.
When it comes to specific virtual reality (VR) or augmented reality (AR) features in Unreal Engine 4.22, Epic Games has added support HoloLens Remote Streaming which: “allows Unreal applications to run on a Windows desktop PC and stream the rendered result wirelessly to HoloLens over a Wi-Fi connection in real time.”
When it comes to building awesome looking virtual worlds Unreal Engine 4.22 will enable developers to do so thanks to early access support for real-time ray tracing, the technology Nvidia has been shouting about ever since it introduced the RTX series of graphics cards. Ray tracing is a technology being employed in videogames to provide natural realistic looking lighting effects in real-time, making images like those above look almost photorealistic without being too resource intensive. It’s not something you’re going to see in VR titles just yet, but it’s an eventuality. This was showcased at the Game Developers Conference (GDC) 2019 with Troll (trailer at bottom of the article).
Another important step in Unreal Engine 4.22 is the reduction of build times. Epic Games has optimized UnrealBuildTool and UnrealHeaderTool to make C++ iteration times up to 3x faster it claims, releasing the following stats:
Full build (UE4Editor Win64 Development):
Unreal Engine 4.21
Unreal Engine 4.22
Improvement
Total Build Time:
436.90
326.81
30% faster
Compiling UnrealHeaderTool:
46.12
46.30
Generating headers
25.05
15.50
60% faster
Compiling UE4Editor
323.15
257.97
25% faster
UnrealBuildTool overhead
42.58
7.04
600% faster
Incremental build (UE4Editor Win64 Development):
Unreal Engine 4.21
Unreal Engine 4.22
Improvement
Total Build Time:
7.47
2.14
340% faster
Compiling UE4Editor
1.19
1.08
UnrealBuildTool overhead
6.28
1.06
590% faster
Epic Games’ other big announcement recently was the $100,000,000 USD Epic MegaGrants initiative, which aims to assist videogame developers, media and entertainment creators, enterprise professionals, students, educators, and tools developers who are working with Unreal Engine 4.
To learn more about Unreal Engine 4.22 head on over to the Unreal Engine blog. As the company continues to release further improvements VRFocus will keep you updated.
To significant anticipation, Microsoft revealed HoloLens 2 earlier this week at MWC 2019. By all accounts it looks like a beautiful and functional piece of technology and a big step forward for Microsoft’s AR initiative. All of which makes it unfortunate that the company didn’t strive to be clearer when illustrating one of the three key areas in which the headset is said to be improved over its predecessor.
On stage this week at MWC 2019, HoloLens visionary Alex Kipman was the one to officially reveal HoloLens 2. The headset, he said, delivers improvements in three key areas that customers of the original headset had consistently asked for: field of view, comfort, and business ROI right out of the box.
For field of view—how much of your view is covered by the headset’s display—Kipman said that HoloLens 2 delivers “more than double” the field of view of the original HoloLens.
“More Than Double” …What, Exactly?
Image courtesy Microsoft
Within the AR and VR markets, the de facto descriptor used when talking about a headset’s field of view is an angle specified to be the horizontal, vertical, or diagonal extent of the device’s display from the perspective of the viewer. When I hear that one headset has “more than double” the field of view of another, it says to me that one of those angles has increased by a factor of ~2. It isn’t perfect by any means, but it’s how the industry has come to define field of view.
It turns out that’s not what Kipman meant when he said “more than double.” I reached out to Microsoft for clarity and found that what he was actually referring to was not a field of view angle, rather the field of view area, but that wasn’t explained in the presentation at all, just (seemingly intentionally) vague statements of “more than twice the field of view.”
But ok… I get it. Field of view area isn’t a bad way to compare headsets by any means, if somewhat peculiar compared to how other companies rely this information. Not the end of the world.
But then Kipman moved onto a part of the presentation which visually showed the difference between the field of view of HoloLens 1 and HoloLens 2, and that’s when things really became misleading.
From 2x to 5.2x
In the center of the image Microsoft showed the HoloLens 1 field of view, and then drew out the edges of the image to show how much larger the HoloLens 2 field of view was by comparison. Except it was hugely exaggerated.
I pulled the below photo right from the presentation and used a perspective-correct transformation to make sure the shapes are correct relative to the camera angle. Then I overlaid the actual HoloLens 2 field of view (since confirmed to me by Microsoft) as it should have been sized in the comparison:
Photo by Road to VR, based on image courtesy Microsoft
The difference between the actual field of view and what was presented is not just a little bit wrong… it’s like, way wrong. Microsoft made a big deal about the “more than double” increase (in area) compared to HoloLens 1, but managed to misrepresent the difference by an even larger margin, showing a 5.2 times area increase over HoloLens 1 in the visual.
What makes this even more unfortunate is that they attempted to show how the wider field of view would let you see more of a virtual object, but the actual HoloLens 2 field of view in this case would have truncated the example on all sides.
When I asked Microsoft about this, I was told that what was shown just an “illustration,” and only a brief part of the presentation. But it seems surprising that they didn’t take more care to faithfully represent one of the headset’s most important improvements, especially considering this is the only means of comparison short of actually having a HoloLens 1 and 2 in front of you to try.
Initially when I went asking through official Microsoft press channels for specifics on the HoloLens 2 field of view, I was told that “more than double” HoloLens 1 is all the information that would be shared for the time being. When I took to Twitter to air my frustration at the lack of clarity (especially considering that the stated “more than double” was relative to the HoloLens 1 FOV which Microsoft had also been coy about sharing) Alex Kipman responded by pointing to a huge Wired pre-reveal feature article published this week where he confirmed a 34 degree diagonal field of view (16:9) for the original headset, and a 52 degree diagonal (3:2) for HoloLens 2.
But even in an in-depth pre-reveal briefing, Microsoft still didn’t manage to accurately convey the field of view message; the Wired piece reports that the “diagonal field of view has more than doubled” on HoloLens 2—the same thing most people reasonably thought when the company said it on stage this week.
Fool Me Once
This isn’t the first time that Microsoft has been called out for doing a poor job of representing the HoloLens field of view. When the original headset was revealed back in 2015, the company used (admittedly very cool) mixed reality compositing to show a third-person perspective of all the virtual objects floating around the user. This was billed as ‘what the user sees,’ but left out the important detail that with such a small field of view there is significant truncation that is not shown. Microsoft later began adjusting many of its HoloLens marketing visualizations to be more clear about what how the user-perspective actually looked and was impacted by the field of view limitations.
– – — – –
To be clear, this isn’t the end of the world. Companies try to present their products in the best light, and oftentimes that involves stretching the truth. It isn’t clear if Microsoft was being intentionally misleading, or if someone just made a bad mockup, but it does mean that if the only thing you saw was the HoloLens 2 reveal presentation, the field of view is actually significantly smaller than what was shown on stage (and for anyone curious, nearly identical in size to that of Magic Leap).
This will be forgotten in time; HoloLens 2 will go on to do great things. But misrepresenting the field of view doesn’t make it bigger, and people are still going to be asking for even more field of view from HoloLens 3 and beyond. So in the meantime, Microsoft, why not just aim for accuracy and let the device speak for itself.
