Early last month, Apple, Inc. announced that the forthcoming virtual reality (VR) compatibility would come via way of a MacOS update known as High Sierra. Closed beta access to the update came at the Worldwide Developers Conference (WWDC), and now is available to all.
One of the biggest changes incorporated into the High Sierra update is an overhaul of the file organisation system. The new architecture, dubbed the Apple File System, replaces a system designed decades-old Macintosh hardware. The new system is optimised to take advantage of newer technology such as flash-based memory and is configured for better security. Exactly how much of an improvement Apple File System offers will only be known once more people have had the opportunity to work with it.
The operating system also introduced High Efficiency Video Coding (HEVC), and – perhaps most importantly for VR developers – Metal 2. HEVC is a new industry standard for graphics that allows for up to 40 percent greater compression of video data. That means video files will take up significantly less storage space. Metal 2 adds new capabilities to the way the system manages the graphics processor, allowing it to take advantage of machine learning, external GPU support and VR output, including the HTC Vive.
High Sierra also includes optimised support for other VR elements, such as content creation applications like Final Cut Pro X, Unreal Engine 4, and Unity 2017, which all have VR development as a central component of their application.
The High Sierra update is still considered beta, but no longer has any limitations on who may download it. VRFocus will keep you updated with all the latest details on Apple’s VR rollout for Mac format PCs.
Early last month, Apple, Inc. announced that the forthcoming virtual reality (VR) compatibility would come via way of a MacOS update known as High Sierra. Closed beta access to the update came at the Worldwide Developers Conference (WWDC), and now is available to all.
One of the biggest changes incorporated into the High Sierra update is an overhaul of the file organisation system. The new architecture, dubbed the Apple File System, replaces a system designed decades-old Macintosh hardware. The new system is optimised to take advantage of newer technology such as flash-based memory and is configured for better security. Exactly how much of an improvement Apple File System offers will only be known once more people have had the opportunity to work with it.
The operating system also introduced High Efficiency Video Coding (HEVC), and – perhaps most importantly for VR developers – Metal 2. HEVC is a new industry standard for graphics that allows for up to 40 percent greater compression of video data. That means video files will take up significantly less storage space. Metal 2 adds new capabilities to the way the system manages the graphics processor, allowing it to take advantage of machine learning, external GPU support and VR output, including the HTC Vive.
High Sierra also includes optimised support for other VR elements, such as content creation applications like Final Cut Pro X, Unreal Engine 4, and Unity 2017, which all have VR development as a central component of their application.
The High Sierra update is still considered beta, but no longer has any limitations on who may download it. VRFocus will keep you updated with all the latest details on Apple’s VR rollout for Mac format PCs.
With Apple’s increasing interest in VR on Mac, the company is adding essential features to its new Metal 2 graphics API to support high performance VR on MacOS.
Because VR demands high-powered graphics and extremely low latency, specially tuning the rendering pipeline between the GPU and the headset is crucial to maintaining a comfortable and performant VR experience. Many of the key tweaks to the VR rendering pipeline have been established on other platforms, and Apple is bringing many of them to the updated Metal 2 API which the company says now “provides powerful and specialized support for Virtual Reality (VR) rendering.”
At the ‘VR with Metal 2’ session during Apple’s WWDC 2017 conference last month, Apple GPU Software Team member Rav Dhiraj detailed the new features in Metal 2 for VR.
Single-pass Stereo
Photo courtesy Apple
Since VR headsets show a 3D image, they need to render an individual view for each eye; you might think this would require double the rendering work, but with some smart rendering techniques you can be much more efficient. Single-pass stereo allows the GPU to render to the left and right eye with a single draw call instead of one for each eye.
Metal 2 now makes this possible with what Apple calls the Viewport Array feature which allows “per-primitive viewport selection in the vertex shader.” Instead of rendering the left eye image and then the right eye image, developers define a texture that’s the resolution of both eyes next to each other, then render across the entire texture in one pass. Telling the renderer which eye each primitive belongs to ensures that it gets rendered in the correct eye-region of the texture and with the correct offset to account for the slightly different perspective of each (due to the distance between the eyes).
Direct-to-display
Direct-to-display (also known as Direct-mode) allows the GPU to more directly access the VR headset by bypassing parts of the pipeline that would be necessary only for traditional monitors.
Without Direct-to-display, the operating system would see a VR headset like any other monitor and simply mirror or extend the operating system view onto it, including modifying the rendered image with any OS-induced post-processing. With Direct-to-display, the operating system doesn’t identify the headset as a monitor, preserving the existing window layout and display arrangement. This allows a VR compositor (which warps the initially rendered image and prepares it for display on headset) to directly present the rendered image to the headset’s display without the OS introducing any additional latency, performance hits, or image tampering.
Not only does Direct-to-display reduce the complexity of the VR rendering pipeline, it also makes the end-user experience better because developers and users don’t need to fuss with ‘where’ on the OS’s desktop the VR application is displayed—instead it automatically ensures the rendered image lines up perfectly with the VR headset’s display and without any interference from the OS.
– – — – –
Metal 2 will debut with MacOS version ‘High Sierra’ which is presently in beta for developers and due to launch later this year, supporting all Macs compatible with the present ‘Sierra’ version.
On stage at WWDC 2017, Valve’s Nat Brown overviews the inner workings of SteamVR rendering on MacOS | Photo courtesy Apple
Developers building VR apps with tools like Unreal Engine or Unity won’t need to worry about the above features as the MacOS VR integrations of those engines will take advantage of them by default. Developers wanting to building native VR applications or integrate with existing proprietary game/rendering engines on MacOS will want to specifically adapt support for these new VR specific features.
Both features and more are covered in detail during the WWDC session, including an overview of best-practices for VR rendering that will be useful to developers building VR apps from scratch.
Apple’s broad embrace of VR this week shows that the company has had virtual reality support on its roadmap for quite some time. During a session at WWDC, Apple and Valve confirmed they’ve been working together for nearly a year to bring SteamVR and OpenVR to MacOS.
Though you might think that porting SteamVR and OpenVR to MacOS would be relatively straightforward—given that Steam for regular desktop gaming has been on the OS since 2010. But it seems there was quite a bit of development work to be done in order to make it happen, including a few missing capabilities which Apple built into the Metal 2 rendering API, at Valve’s request, in order to make it possible to render high performance VR on MacOS.
Collaborating for VR on MacOS
To that end, the companies have been working together on the project for nearly a year, said Rav Dhiraj, a member of the GPU Software Team at Apple, who detailed Metal 2’s new VR rendering features in a session at the WWDC conference this week.
“We’ve been working really closely with Valve over the last year to align our releases and both SteamVR and OpenVR are available to download in beta form this week,” he announced on stage.
Valve’s Nat Brown joined Dhiraj on stage to talk more about what it took to get SteamVR and OpenVR running in high performance on MacOS.
“Valve and Apple, we started working together more closely about a year ago. Our port to Metal from OpenGL didn’t cost us very much—Metal is a really cool API and it was critical for us to get VR up and performant,” Brown said.
On stage at WWDC 2017, Valve’s Nat Brown overviews the inner workings of SteamVR rendering on MacOS | Photo courtesy Apple
He also confirmed that the companies have been working together on the project for nearly a year, with one of the most significant initial asks from Valve to Apple being the implementation of direct-to-headset rendering capabilities into Metal 2.
“Our biggest request to Apple a year ago was for this direct-to-display feature because it’s critical to ensure that the VR Compositor has the fastest time-predictable path to the headset display panels,” said Brown. “We also really needed super accurate low variance VBL (vertical blank events) so that we could set the cadence of the VR presentation frame timing and predict those poses accurately.”
Apple As An Active Participant
Brown seemed to paint Apple as not only supportive of Valve’s work, but an active participant in the process of getting SteamVR and OpenVR running well on MacOS.
“We hit some speed bumps [during development] around inter-process and inter-thread synchronization. Once everything else was working really well—Metal was blazing fast, we had really tight VBL—but we still were having some synchronization problems, but Apple helped us find better ways to signal and synchronize with low scheduling variance between all the processes and threads involved.”
After digging into the technical challenges of rendering a frame onto a VR headset with low latency, and explaining how developers can install SteamVR on MacOS to begin experimenting, Brown wrapped up his segmented saying, “Thanks to everybody at Apple for making VR shine on MacOS.”
On stage at WWDC 2017, Apple’s Rav Dhiraj explains VR rendering concepts | Photo courtesy Apple
SteamVR and OpenVR for MacOS rely on Metal 2 which is only supported in the forthcoming release of Apple’s new ‘High Sierra’ version of the operating system. Developers can download a beta version of High Sierra today through Apple’s developer portal, and then install SteamVR through the MacOS Steam client and opt into the latest beta version. MacOS High Sierra will be available as a free upgrade on any Mac which supports the current Sierra release, however only the new, top-end iMacs and MacBooks with external GPUs that meet the VR Ready spec will be able to run VR experiences.
Watch This Space
A year’s worth of collaboration on VR between Apple and Valve calls into question Oculus’ involvement with Apple and their intentions (or lack thereof) to support MacOS. It’s possible that the groundwork laid down by Apple and Valve’s work, especially relating to the VR-specific features of Metal 2, could make Oculus’ job of porting their platform to MacOS easier. But it isn’t clear whether or not that effort has begun, nor how long it might take if and when it does start.
For now, outwardly at least, it seems Valve and Apple are forming a strong relationship while Oculus has been not shown up to the veritable table. How this relationship pans out in the long run—especially with rumors of Apple developing its own VR headset—could have far reaching consequences on the future of the VR industry and its key players.
Kurz nach der Ankündigung auf Apples Pressekonferenz zum Auftakt der WWDC 2017 ist es schon da: SteamVR für den Mac. Interessierte können sich die Beta-Version seit gestern herunterladen und installieren. Während SteamVR für alle Anwender offen ist, können derzeit nur Apple-Entwickler das virtuelle Erlebnis auf dem Mac starten. Und es gibt noch weitere Pferdefüße.
Valve SteamVR für macOS High Sierra
Apple hat das kommende Betriebssystem für Mac mit Metal 2 und einer VR-Runtime ausgestattet. Die Beta-Version von macOS High Sierra ist ebenfalls bereits als Download verfügbar. Allerdings ist diese Version noch nicht öffentlich freigegeben. Um sie zu beziehen, benötigen Interessierte einen Entwickler-Account bei dem Hersteller aus Cupertino.
Für Endanwender dürfte SteamVR aber eh noch nicht allzu interessant sein: Bestehende Windows-VR-Titel müssen erst angepasst werden und derzeit bietet Apple lediglich den 27-Zoll-iMac mit einer optionalen Radeon 580 an – die Mindestvoraussetzung für VR. Für Macs mit Thunderbolt-3-Port führt der Hersteller ein VR-Entwickler-Kit im Programm, das aus einem externen Gehäuse, einem USB-Hub, einem 100-US-Dollar-Gutschein für die HTC Vive und einer Radeon 580 besteht. Billig ist das Vergnügen nicht, denn es belastet das Konto mit immerhin 599 US-Dollar. Damit dürfte die tatsächlich installierte VR-Ready-Basis auf Apple-Geräten auch in naher Zukunft äußerst überschaubar bleiben.
Für die Installation von SteamVR benötigt man die Steam-App und in der Library lässt sich unter Tools SteamVR herunterladen.
If you missed the big news yesterday, Apple is now supporting virtual reality (VR) on its Mac platforms thanks to the new OS High Sierra and augmented reality (AR) on iOS with ARKit. During the keynote address the immersive demonstrations from Wingnut ARand ILM used Epic Games’ Unreal Engine, but the other big player in the middleware space, Unity Technologies, will also support Apple’s venture into VR and AR.
Unity is fully integrated with ARKit for iOS, with the company providing an open source bitbucket project for ARKit to get developers started. On the VR side Unity is currently offering an experimental build for content creation with studios able to freely publish to the app store with High Sierra.
Unity editor on macOS with Space Pirate Trainer from I-Illusions in VR Mode
Unity has been working with several studios to test out the software, including Space Pirate Trainer developer I-Illusions. On the Unity blog, I-Illusions’ Dirk Van Welden said: “Overall, the porting of Space Pirate Trainer to macOS with Unity went very smooth. We had it running on macOS under a couple of hours. I previously had some concerns about Metal support, but Unity and Apple made the whole process pretty straightforward. Metal support has been hugely optimized in these past weeks and months and most of our custom shaders were supported out-of-the-box, while we never had Metal in mind while creating those shaders. Great job Unity, Valve and Apple!”
Additionally, Unity has been working with Apple and its API for high performance graphics, Metal 2, to gain extra performance by making use of the software’s new features such as the VR-optimised display pipeline.
Instant messaging is something we do on our smartphones now, but desktop clients are far from dead. In fact, they're currently enjoying something of a renaissance. So, which one should you be using?
An interview with Oculus VR co-founder Nate Mitchell reveals the company still wants to support MacOS, but that may not happen for a while. He said the project currently isn't on the roadmap and will require a huge time investment.
Developer Cindori launched a new MacOS app enabling a virtual three-screen desktop in the Oculus Rift headset. But there are a few catches of course, such as MacOS machines need to be "modern" and sport a discrete graphics processor.
