Former Oculus CTO Reviews Bigscreen Beyond: “like a prop for a futuristic movie”

John Carmack, legendary programmer and former CTO of Oculus, is known for giving his unfiltered thoughts on almost every aspect of the XR industry. While he departed Meta in December, concluding his “decade in VR,” Carmack is still very interested in the medium, as he recently went hands-on with one of the latest PC VR headsets to hit the scene, the slim and light Bigscreen Beyond.

Bigscreen Beyond is a tethered PC VR headset that uses Valve’s SteamVR tracking standard, which starting at $1,000 for just the headset makes it an interesting value proposition for users already hooked into the SteamVR hardware ecosystem. It’s largely praised for its slim and light profile, which is thanks to the inclusion of pancake lenses and micro-OLEDs, serving up 2,560 × 2,560 pixels per eye at 70 to 90 Hz refresh.

You’ve probably already heard what we think of it though. Now for the master:

“Bigscreen Beyond feels like a prop for a futuristic movie, but it works!” Carmack said in a Twitter thread on Monday. “Far and away the smallest and lightest PC VR headset.”

That’s high praise coming from a key figure in the Oculus genesis story, not to mention co-founder and lead programmer of id Software, the studio behind pioneering ’90s 3D games Wolfenstein 3DDoom, and Quake.

Image courtesy John Carmack

To hear all of Carmack’s thoughts on Bigscreen Beyond, we’ve formatted his tweets below for easier reading:

Bigscreen Beyond feels like a prop for a futuristic movie, but it works! Far and away the smallest and lightest PC VR headset.

As a result of the iPhone based face scanning before ordering, the fit is perfect, with zero light leaks. The custom printed facial interface is comfortable, but not breathable, so it isn’t great for fitness activities.

The prescription lens inserts snap in with magnets and work well. The visuals are a trade off vs Quest Pro. The resolution is clearly higher, but there are more internal reflections in the pancake optics, and the quality falls off more toward the edges. There are parts of the view where screens look fantastic, good enough for actual productivity work, but not across the entire view.

I sorely miss integrated audio. Having to mess with headphones severely impacts the minimalist feel of the headset. I know some people have strong opinions, but I still feel Quest made the right decisions around audio.

The cable to the PC and the tracking base stations are the biggest downside. The magic of stand-alone VR is real, and while some people happily trade it away for the raw power and flexibility of a PC, I wouldn’t recommend any PC VR setup as an entry point to VR.

For people considering an upgrade to a PC VR system, Bigscreen Beyond should be in the mix. I am very happy to see this extreme focus on light weight, and I hope it impacts Meta’s future designs.

In a follow-up tweet, Carmack points to a review from Adam Savage’s Tested, which he says “hits most of my points in more depth.”

In it, Tested’s Norman Chan reports back after having lived with the headset for a month, using a development unit as the office’s primary PC VR headset. Chan shows through-the-lens images, and discusses “the good, the bad, and the weird with this unique approach to high-end bespoke VR,” the video’s description reads.

Check out Tested’s review below:

Quest 2 Will Have ‘Long Life’, Project Cambria Still Due In 2022 – Meta

Meta figureheads have reiterated that the Meta Quest 2 will have a “long life”, and will not be replaced by the upcoming Project Cambria.

Consulting CTO John Carmack made that much clear in response to a new blog post from Meta’s VR/AR head, Andrew Bosworth. “An important point here is that the “project Cambria” product will *NOT* replace Quest 2, it will be sold alongside it,” Carmack said. “Quest 2 will have a long life.”

The note comes amongst concerns that Cambria, which is a new high-end standalone headset, will eventually take front and center for Meta, much in the same way that Quest 2 has slowly replaced Quest 1 over the past 14 months. Indeed, a number of apps that have released for Quest 2 this holiday season can’t even be played on Quest 1, including Resident Evil 4 and Medal of Honor.

But Cambria, which was announced at Connect a few months back, isn’t going to be a part of the Quest line of products, and new features like color passthrough and eye and face-tracking suggest the headset will be significantly more expensive than Quest 2’s $299 starting point. The project’s aim is less about widespread adoption and more concerned with implementing features that Meta believes will advance its vision of the metaverse among other uses.

Bosworth’s blog, meanwhile, reconfirmed that Cambria is planned for release sometime in 2022, though there’s still no specific window for that launch. We’ll be keeping a close eye on the headset as we move into the new year and will bring you all the latest.

John Carmack Wants The Next Connect To Be In VR Via Horizon

John Carmack wants the next iteration of Meta’s Connect event to happen in VR via the new Horizon apps.

Carmack said as much in his keynote speech during this year’s event. During the session, he recalled past in-person events in which attendees would come and ask him questions in hallways, or even when he did the same inside VR last year.

“Really what we want to have is something like that session that’s like the real Connect where thousands of people are milling around, some of them cluster around me outside and we all crowd into a room for the big keynotes,” Carmack said. “And you get all of that ability without having to have people fly across the country or across the world to get there. That’s what we’ve always been pitching as the value of VR.”

“So I really do want us to pitch Connect as the north star event of what we’re doing. We’ve got an event that we do every year here, we’ve got a user base for this. We should be doing this in the metaverse. If we can’t hande this, we can’t handle the vision. And we could do this by next year. I thought we could have done it by this year if we really made this a frontline focus. But we didn’t have that focus like we do now a year ago.”

Carmack continued saying he’d be “really disappointed” if this wasn’t a reality in time for next year’s Connect. “I want to be walking around the halls or walking around the stage as my avatar in front of thousands of people getting in the feed across multiple platforms,” he said. “I’m laying that guantlet down right now, we should be able to do that. This should be exactly in line with what our stated mission is.”

The problem, as Carmack goes on to mention, is that the current Quest hardware struggles to handle even Horizon’s current 16-person limit, with character fidelity degrading in crowds. “The magic being reached for by many people is that, well, cloud rendering would let us use much more powerful systems than the current headset systems and we could add a bunch more on there.”

Carmack said he was supportive of the idea, but noted that would be a lot of cost in terms of people not having the internet connection to attend, and that Meta could also explore ideas like hybrid applications.

We’d definitely love to see Connect in VR next year – would you want to attend that way?

Facebook Connect 2021 Schedule: Zuckerberg Keynote, Carmack Unscripted & More

The full line-up and schedule for this year’s Facebook Connect is now available, including Facebook CEO Mark Zuckerberg’s keynote speech and John Carmack’s unscripted talk.

This year, it looks like events and sessions will “unlock” across the Connect schedule– some sessions unlock at specific times, while others simply become available “after the keynote.”

The keynote speech itself unlocks at 10am Pacific on October 28, which will see Mark Zuckerberg and “other Facebook executives” deliver the annual keynote address and “share the vision for the metaverse — a place of new immersive experiences and the next evolution of social technology, built by people like you.” There’s also a Facebook event and Oculus Venues event for the keynote, both of which reveal the keynote title to be “Infinite Horizons” — a nod to the Horizon branding that’s now being used as the moniker for the different branches of Facebook’s social VR platform.

The next biggest event is Consulting CTO John Carmack’s notoriously-unscripted annual speech and Q&A session, which unlocks at 2pm Pacific on October 28, a few hours after the keynote. Last year, Carmack spoke for more than an hour, packing a huge swath of topics into a dense, unscripted talk about the keynote announcements and Facebook’s wider VR strategy.

Here’s a few, but not nearly all, of the other Connect events:

Developer State of the Union with Chris Pruett, Tom Langan, Allison Lee, and Mari Kyle, available from 11:45pm Pacific, October 28.

– Best Practices and Tips for App Lab Success with Chris Lang, available after the keynote

Introduction to Mixed Reality with Britta Hummel and more, available after the keynote

– Do You Really Know App Lab? Tales Of Success From Devs with Rita Turkowski, Daniel Sproll (Realities.io), Julien Pecenicic (Smash Drums), Tommy Maloteaux (Deisim), available after the keynote

– Building for the Future: Social and World AR Experiences with Matthew Roberts and more, available after the keynote

– Bringing Diversity To Your App with Daniel Kamerling, available after the keynote

Build Mixed Reality Experiences with Wei Lyu and more, available after the keynote

– Building Diverse Communities in Horizon Worlds with Lisa Kotecki and more, available after the keynote

Facebook Connect begins on October 28 — you can view the full schedule of talks and register for the event here.

Andrew Bosworth To Take Over As Facebook CTO In 2022

Facebook Vice President of AR/VR, Andrew ‘Boz’ Bosworth, will take over the role of Facebook Chief Technology Officer (CTO) after current CTO Mike Schroepfer announced plans to step down from the role sometime next year.

Schroepfer has been with Facebook for 13 years and after stepping away from his CTO role, he will move to a Senior Fellow role within the company. Here’s a copy of his full Twitter thread announcing his departure:

After 13 years at FB, I’ve decided to step down as CTO and pass the baton to @boztank at some point in 2022. I will stay on as long as it takes to ensure a successful leadership transition.

My new role as Senior Fellow will enable me to stay deeply connected to the company, working on key initiatives including recruiting and developing technical talent and fostering our AI investments in critical technologies like @PyTorch

It has been a privilege to lead our technology teams during a time of incredible growth & advancement. I am proud of what the team has achieved, from unleashing the benefits of AI & bringing VR to life to connecting more people around the world through technology.

I am still incredibly optimistic about the potential for AI and AR/VR to improve the lives of people every day. I am honored to be able to continue to support Facebook’s exciting future in my role as Senior Fellow.

Facebook VP of AR/VR Andrew Bosworth has taken on an increasingly public facing role over the last year or two, becoming a prominent spokesperson for Facebook’s VR/AR efforts. Bosworth has hosted Instagram AMAs (‘ask me anything’) with user-submitted questions on Facebook’s VR/AR efforts (his responses to which often flirt between informative, teasing and promotional all at once), moderated talks with Mark Zuckerberg and John Carmack on various VR and VR-adjacent topics, and dropped public hints and winks towards new products like Quest Pro.

In a statement, Facebook CEO Mark Zuckerberg said that as CTO, Bosworth will continue to lead Facebook Reality Labs:

As our next CTO, Boz will continue leading Facebook Reality Labs and overseeing our work in augmented reality, virtual reality and more, and as part of this transition a few other groups will join Boz’s team as well. This is all foundational to our broader efforts helping to build the metaverse, and I’m excited about the future of this work under Boz’s leadership.

Responding to Schroepfer’s thread on Twitter, Bosworth said, “Congratulations Schrep on an epic 13 year run. Lots to do still and I’m glad you’re sticking around to help.”

Facebook released Quest 2 near the end of 2020 and, last month, doubled the storage of the base $299 model to 128GB. The company also recently partnered to make Ray-Ban brand sunglasses that start at $299 and connect to your phone to answer calls or play music, or capture square stabilized 30 second videos and wide angle photos. As of the end of June 2021, Facebook employed more than 63,000 people with a significant percentage working on VR and AR technologies.

John Carmack’s ‘Brutally Honest’ App Reviews Return As Regular Series

John Carmack’s “brutally honest” VR app reviews are returning as a regular series.

Carmack, originally Oculus CTO, has been a technical guide for VR since before its 2014 acquisition by Facebook and, on occasion over the years, he’s offered direct feedback to VR developers seeking ideas that could help improve their software. The feedback from the long-time game developer isn’t just useful to the developer seeking it, though, as many VR creators dive into his comments for ideas that may be applicable to their own work.

The series debuted with a look at a recently released Oculus Quest App Lab game called Space Shooter. As an example of kind of feedback devs might get, Carmack wrote on the Oculus developer blog about Space Shooter that “Sending the world spinning on death is uncomfortable. Motion sickness should not be used as feedback in a game; it lingers and doesn’t go away when you restart.”

This is the first time we’ve noticed the new title for Carmack — Facebook Reality Labs Executive Advisor. In 2019, Carmack started dedicating the majority of his time to personal artificial intelligence research outside of Facebook, changing his title to “Consulting CTO Oculus VR” even though the startup once known as Oculus remains little more than a brand within the larger VR and AR efforts at the newly minted “Facebook Reality Labs.”  The “Consulting” title is apparently a reference to Sherlock Holmes being a “Consulting Detective” and Carmack commented on Twitter that he likes the former title but “Executive Advisor” is a “more accurate description” of what he does.

“Note that I never did have any executive authority even when I was formally the CTO of Oculus, because I didn’t actually have direct reports (by choice),” he wrote.

Facebook has a form developers can fill out to submit their application for Carmack’s review.

Facebook Is Working On Social Home Environments On Quest

John Carmack says Facebook is working on a social VR experience for the Home environment on Oculus Quest.

Carmack confirmed as much in a tweet posted late last month that’s only just been noticed. He explained that the company originally planned to develop multiplayer interactions in the Oculus Rooms app, which launched on Gear VR and let friends play minigames and watch content together. Those designs would then “roll back” into Home, but Carmack said “we messed up in various ways.”

“The social home experience is being worked on again now,” the developer assured.

Facebook already has its own social VR apps on Quest, including Venues, and has more to come, like Horizons. But the key Home environment, where users shop for new titles, remains a solitary experience. Meanwhile on Oculus Rift, users can fully customize virtual Home environments, invite others to meet them there and play some basic minigames like shooting targets. Quest’s environments aren’t customizable and you can’t even move around them, though you can switch out locations.

Carmack didn’t stamp a timeframe on when a multiplayer Home solution might arrive. That said, Quest is getting major new features fairly regularly – yesterday’s announcement of the v29 software update brought with it plenty of updates.

Would you want multiplayer Home environments on Oculus Quest? Let us know in the comments below.

John Carmack & Andrew Bosworth Twitter Spaces Transcription And Recording

Consulting Oculus CTO John Carmack and Vice President of Facebook Reality Labs Andrew Bosworth held a discussion via Twitter on April 16th.

The talk represents one of the most insightful views into Facebook’s long-term strategy in VR that we’ve ever seen, and we’ve got a transcription of the discussion. The Twitter Spaces system doesn’t archive its broadcasts yet. However, we were able to make a recording when it was happening live and have been listening through carefully for notable takeaways. Those include:

  • Quest Pro with more sensors onboard won’t ship in 2021 while Quest 2 stays in market for “a long while“.
  • Carmack suggested Facebook’s VR headsets could become more fully functional personal computers which could go after the market for Chromebooks and tablets.
  • Carmack suggested Facebook is likely to evolve over several years toward one day offering “a controller-free SKU in the future where we rely just on hand tracking for people that want to use keyboard and mouse and don’t want to pay for the controllers.”
  • Carmack offered a multi-year path toward system-wide simulator sickness mitigation.

Here’s the video and below is our transcription if you want to follow along:

Dedicated Air Link Dongle (0:00)

Andrew Bosworth: Are we considering a dongle to ensure a more solid Quest to PC Air Link?

John Carmack: [Laughs]

Andrew Bosworth: That’s funny, we started with a dongle. We call it Link and Air Link is a evolution on that for people with at least good enough wireless connections, which I think is a lot of people in our community, is what we hope that they’re going to use is just Air Link. But for those who need the stronger connection, you can either buy an Oculus Link cable from us, or, any number of USB-C 3.1 cables.

John Carmack: Actually think that was more around the line of a wifi specific dongle, which of course we found a long and sordid history internally arguing about. where internally this started off a direction that I thought was just going to be a disaster, where, we’ve had these arguments where first it was like, ‘Link’s a bad idea, the quality is going to be lower’ then it’s like ‘Well, you can’t just do Air Link. We know the quality is going to be even worse, but maybe if we design and sell a $200 special wifi dongle, that has custom firmware it would be okay.’ And that was the original plan of record when sort of things went through and I was kind of pitching a fit about that. Eventually we came to, I think the absolutely correct decision, which is no it’ll work on just any wifi . We may yet in the future, make some extra wifi dongle or have some partnership with different firmware flashes for something that can let us get somewhat better performance in congested conditions. But I always pointed to the Virtual Desktop work where we have an existence proof. I kept saying literally, right this very second, there are thousands of people using that and getting value from it, clearly, it’s okay. So this is again, one of those things, the right thing happened in the end. It’s all good. We’re not requiring it. We may yet do something in the future, if we get lots of people using this and it looks like there’s a market for it to do some higher end boutique dongle, that improves performance a bit.

Andrew Bosworth: Yeah, that’s exactly right. And of course, now I understand the question was that was a wifi dongle. Of course it would be a stronger connection. But I kind of feel like you’re in one of two places either you really need a strong connection and you’re using a physical cable or your wifi is strong enough and you use that. But yeah, at some point you could imagine there being a third option somewhere in between.

Quest Pro (02:00)

Andrew Bosworth: We talked about Quest 1 already, which is, I promise you it’s still very well supported – obviously you’ve got two generation older processor there. So there are some limitations on what we can provide to the architecture but when we can bring updates to them with confidence that we can still deliver with within the framework of that headset, we do. People are also asking about the Quest 3 which doesn’t exist yet. Everyone who’s listening to this who’s a reporter, there isn’t a Quest 3, there’s only a Quest 2, but I did hint at an AMA earlier this year about Quest Pro, because we do have a lot of things in development where we want to introduce new functionality to the headset, along the kinds that people theorize that we would want to introduce. That’s a little ways off still, it’s still not going to happen this year. So Quest 2 is going to be in the market for a long while. And it’s going to be the best bet for the most accessible way to get into VR and have a great experience. Are you able to definitively say that VR is mainstream with respect to Quest 2’s first few months? What do you think John, have we hit mainstream yet?

Mainstream VR (02:57)

John Carmack: It’s a sliding spectrum and I think that the success has been everything that we hoped for with Quest 2. Can you comment on some of our internal- how we’ve shifted certain things there? I mean, I don’t want to overstep here.

Andrew Bosworth: You’re absolutely right. Quest 2 has been better than we hoped for. And so we’re shifting and trying to pull supply. Obviously there’s like a global silicon shortage, but we’re trying to increase production to keep up with the demand. We’ve done a much better job with Quest 2 than we were able to do with Quest 1, which is incredible under the pandemic. As a consequence, what you’re seeing us focus so much on is this kind of ‘year of software‘ we’re calling it. Where we’re just having a rolling thunder of product improvements for the Quest 2, which John, I know you’ve been such an advocate for a long time. Just take something that somebody already has and keep adding value to it, especially as you see people leaning into it. And so far as Quest 2 has exceeded expectations outselling all previous 6 DoF headsets combined. We’ve definitely started to make shifts to kind of lean further into it.

John Carmack: Thanks for that little qualification of 6 DoF because I have to keep reminding people that Gear VR out sold every other headset put together multiplied by something. We have a sense of perspective, this is not the iPhone yet, or even the Nintendo Switch, but we are doing really great now. We don’t have to kind of couch it in various ways that let us obfuscate what things actually are, Quest 2 is doing really damn well.

Developer Success (04:13)

Andrew Bosworth: Objectively it’s doing well and that’s why we were so thrilled about the developers who were having success. People don’t believe us because it sounds like a talking point, but like we really care a lot about developers succeeding on our platform and being able to build good businesses there. The fact that people were successfully sideloading things was a justification for building up App Lab to make it easier for both consumers and developers to reach their audiences, something that, John, you were a huge advocate for. And likewise for me, at least , thinking about developers, making a living on VR, which is unthinkable five years ago is now a reality. We’ve got entire teams of people doing that which I think is so exciting.

John Carmack: That is one of the things that I want to point out that really was different on Quest and even more so on Quest 2 is the developer success story. I like to point out how Oculus Go was really quite successful from a sales and user retention standpoint, but developers just didn’t make money on it. It was not something that supported the broader developer ecosystem and a bunch of developers are really doing pretty damn well on Quest. And we’re expecting that to improve in the future.

Andrew Bosworth: We’re so used to the last 15 or 20 years where software, because of the Internet, would just go from zero to a global sensation seeming overnight. This is still hardware, we’re still shipping a thing. It still costs money. People have to work for money. So it’s not going to be the kind of thing that breaks mainstream in the way that we’ve gotten used to things breaking mainstream. But I have been through a lot of products at their elbow, at the time when their curves just went vertical and this is what it feels like. There’s probably false starts. I’m sure there’s examples of things that didn’t carry on that path for whatever reason. And we can certainly still screw it all up by executing poorly, but, Quest 1 to Quest 2 is what it feels like to me, with my experience when a product starts to break to the mainstream.

The Metaverse (05:56)

Andrew Bosworth: Do I think native apps will become the metaverse or the web or some combination? Let’s take the metaverse question more broadly John, how are you thinking about the metaverse?

John Carmack: This is actually something where for the first five years at Oculus I really fought to keep us from attempting to do the metaverse because I just expected it to be a disaster. I expected it to be a honeypot trap for architecture astronauts. Everybody wants to design the metaverse as this infinitely flexible thing. And those discussions just always wind up being pretty far divorced from, sort of on the ground user value that I’m always on about. When people want to design a system, flexible enough to do anything, then it’s not doing any particular thing particularly well. And I’ve always thought that systems like, you know, Fortnite is in many ways closer to the metaverse than anything that we have in VR. Fortnite, Minecraft, Roblox- these things that set out to be games and they built things that people love. So I have a bit of an issue with kind of even considering naming things as an attempt at the metaverse is usually a waving red flag for me. I think it usually signals that people aren’t concentrating on the right thing. I don’t want it as a strategic goal really.

Andrew Bosworth: When we think of the metaverse, we are thinking of it much more generally than is often spoken. I don’t know that it’s a specific single destination, as much as it’s a collection of destinations. At its most abstruse, the abstraction is what we already have on phones. You and I are here on Twitter Spaces we could have been in Clubhouse. What are those, if not, sure, very bare bones dress down limited forms of a metaverse. Where we’re community, we have a stage, and there’s people listening to us. How bizarre is that? So I do think of the metaverse as a more general purpose construct. I think some of it is going to be technical. Like there is a shared space that people can be in together and there’s some rules that govern that space that they all agree to and they all understand some of it will be designed standards to feel continuity from point A to point B, ‘here’s how we do it.’ I do think it’s a very exciting time, I believe in the concept of the metaverse, but not as like this monolithic thing, which I think to your point, is this kind of architectural fantasy, like the ideal city, like there’s no ideal city. Cities that don’t have anything organic about them don’t succeed. I do think there will be some greater fabric that connects us and allows us to travel together and to feel like we’re co-present, which is clearly something we all want to do. Like the demand from the internet of us to connect with one another is undeniable.

Spatialized Audio And Presence (8:29)

Andrew Bosworth: What did Zuck mean by technologies that allow a unique sense of presence? And how does that relate to the hologram issue he discussed when he was on the podcast with Marques Brownlee. I can’t answer what Zuck meant that’s a dangerous game to play, but I can tell you, there are some surprising things about presence, things that make us feel together. Spatialized audio is my number one pick. Spatialized audio is a game changer.

John Carmack: And that’s going to get so much better when we nail out the rest of the latency in it.

Andrew Bosworth: That’s right. And John’s been a tireless advocate of that internally. People don’t realize how much latency affects us. It’s subtle, it’s below our level of conscious understanding, but when you actually survey how people feel it’s something that we emotionally react to. Conversations where you have high latency feel disconnected and jarring and distant and conversations where you have low latency feel better. And if you can then combine that with spatial audio, where everyone’s voice isn’t coming from directly between my ears, it’s amazing. You can start to care around nuanced conversations, where groups of people are having side conversations, then come back together and it feels very organic. That’s my number one, social presence area of enthusiasm. What do you think is going to be the biggest driver of social presence John?

John Carmack: I’d agree that audio and good enough avatars are going to be the thing that makes a significant difference. We have lots of work going on all the way up to the codec avatars at extreme levels of photo realism. And I am a little skeptical about the value proposition being very high on that. I very much like the new avatars that we just rolled out, you know? And I kind of somewhat cynically said, well, fifth time is the charm for us in terms of all the different avatar initiatives we’ve had. But I think we’re at a pretty good place here where it gives a lot of the character customization that people want. And it’s expressive enough that combined with the good hand tracking or controller tracking and the good audio. I think we’re at a sweet spot and I desperately hope we don’t randomize our avatars with another reboot of them the next year.

Andrew Bosworth: No, I think we finally cracked it John. John used to say of our previous Oculus avatars that they were as if they were designed to be creepy as possible.

John Carmack: The radioactive nightmare avatars.

Artificial General Intelligence (10:38)

Andrew Bosworth: Working with John is like this, by the way, for those who are wondering it’s exactly like this he’s the same when he’s talking to you here on Spaces as he is talking to me and I’m, I’m here for it. One of the things that people know about you, John, obviously is what you’re doing while you’re consulting with us, on one hand, you’re also spending a lot of time working on AGI. People want to hear an update, what’s the most interesting stuff that you’ve found so far?

John Carmack: I’m 18 months into it now since leaving the full-time work almost, and I feel like I’ve left my larval phase. I go through this period where I’m kind of cocooned up learning about all the basic things that I need so I can have level conversations with the experts in the field now. And I’ve got my ideas and angles and the things that I’m heading out to go try. It’s at the point now where it really is the research phase, where I have to go try things that nobody’s tried before and hope that they work out and it’s different than all the stuff I’ve done before. Like everything in the gaming, aerospace, and VR space, I’ve always thought, rightly or perhaps wrongly, that I had line of sight to the solutions to what needed to be done and it was just a matter of kind of cranking out, knocking down all the problems and nobody knows what needs to be done here, but I’m still kind of laying out this plan that I think by 2030, there’s a 50/50 chance that someone in the industry, and maybe it’s me, has line of sight and a good glimmer of hope at Artificial General Intelligence. And that’s a world changing thing. So I’m happy with where I’m at right now and I’m committed to this for quite a bit more time.

Andrew Bosworth: Yeah, that makes sense. What about, what about all the naysayers? What about the people who are really concerned about the consequences of AGI?

John Carmack: Well, you know, the wonderful thing about being independently wealthy is I could kind of descend into my bunker with a supercomputer and build it myself without listening to anybody if I don’t want to.

Andrew Bosworth: All right, well that’s something to look forward to for everyone out there who aspires to be as independently wealthy.
A couple of questions here about the future of work. And this is something that you’ve also had really good opinions about that we’ve changed our strategy around, talk to people about how you think about VR and the role it could play in how we work.

Work And The Excuse To Buy (12:42)

John Carmack: My internal pitch for the best super broad strategy for VR, aside from gaming, is screens and people. One of the points that I make is that the investment into conventionally viewed media, whether it’s content, media, applications, things that you do on your screens, your phones, your PCs, your TVs. There’s trillions of dollars of investment that’s gone into that over the last 50 years. And I think it’s just insane to consider that, well, VR is a blank slate people need to come in and reinvent everything. So I think taking advantage of all of those surfaces is something that we absolutely have to do in VR because right now we are still largely an early adopters toy where a lot of people that have VR already have everything else and we’re just adding some new spice, but we need to be a displacement device where we need to be something that somebody hard up for money decides I’m going to buy a VR headset instead of a Chromebook or instead of a tablet. And we need to do everything that those devices do. You know, we need to have similar app libraries. We need to be just as effective with keyboard and mouse. We need it to be something that you could put on your head and do the work that you need to do during a normal day. And I think that we’ve seen the glimmers of light on what we need to do for that. And with Quest 2’s greater screen density we are at the point where it’s becoming reasonable. Ever since DK1 people made demos of “oh, look, I did some programming work inside a headset,” but they were silly gimmicks at the lower resolution. We’re at the point now where it’s still not great but it’s like looking at a 1080P monitor that happens to be gigantic, but you can be flexible. You can do all these great things where it’s not going to be as good as like my home workstation with triple monitors, but we can absolutely make it better than cheap little netbooks that people are hunched over in terrible posture. So I think that on the Quest 2 platform, which as you say, is going to be in market for a while that we can make something that is better than doing work – like some people have to do emergency things on your phone -we can absolutely be better than that. You know, we can be better than the cheap netbooks and we can start competing with some of the better systems. And as we move to future products, whether Quest Pros, Quest Quest 3s, whatever. We could have the software infrastructure in place so that as our optics improve, we really can be looked at as a monitor replacement. And I think that’s how we get truly mainstream when we’re talking hundreds of millions of users.

Andrew Bosworth: There’s this old story in the PC era called “The Reason To Buy And The Excuse To Buy” and the reason to buy a PC was to play games, the excuse to buy was to do spreadsheets. And until you could do both games and spreadsheets you couldn’t buy it because you need both the reason and the excuse. We’ve got plenty of reasons to buy Quest 2 I think for some people they’ll need the excuse. They’ll need to have a justification they use that introduces that technology to them. And that is really critical to breaking fully mainstream to your point where you’re getting out to nine and 10 figures of people in the headsets.

AI In VR (15:44)

Andrew Bosworth: One person asks from a hobbyist point of view, what do you think are the most interesting and realistic AI applications for virtual reality? And it could be high level like applications or low level in terms of functionality.

John Carmack: Internally we use the machine learning technologies for all the perception related things like hand tracking, eye tracking, face tracking, the things that you can’t just apply an algorithm directly to. In fact, there’s an interesting, kind of no man’s land around some of the 6 DoF tracking, where there are areas where you might consider using more machine learning for some of the things that we apply more classical algorithms to. But it’s the clear cut stuff with some of the things like the hand tracking, where there is just no conventional algorithm that can do the job like a machine learning one can. So those are the clear cut cases. We have speculative cases around things like super resolution and view interpolation that could be better versions of the things that we do with some of the various Timewarp solutions. But while there are lots of interesting research papers published about that, like FRL has done some things on super resolution, and it looks all interesting and great until you notice, oh, this is 50,000 operations per pixel generated as opposed to conventional rendering, which might be like 50. So the idea of that making a four times or even 16 times larger image out of that is not yet compelling. But these things do have a habit of magically getting a thousand times faster after people work on them for a while. So there’s interesting aspects there. And then in the game development side, I think that there’s really exciting possibilities on content creation, where we are not that far off from allowing machine learning to be able to take something from sort of Minecraft to AAA level at the push of a button. It’s not there yet, but that is not a crazy thing. We have line of sight to be able to do something like that, where you block something out and then all of a sudden it looks like cyberpunk. It is possible to do that. And I think that’s going to change content creation in some really positive ways. Over the coming decade, I think that’s almost a slam dunk that there is absolutely going to be some things that are creative magnifiers in ways that we haven’t seen in awhile. And then in terms of like classic artificial intelligence, like behavior wise, I’ve always thought that’s one of the weaker aspects for games to take advantage of, because it is amazing when you look behind the curtain how much users do not appreciate or understand what’s going on -in both directions. They don’t understand how complicated some things are. And then if you do something complicated in enemy AI, users will not understand. It is not one of the high leverage things that you can really use in gaming.

Andrew Bosworth: I remember so much Metal Gear Solid was actually one of the games that got me to go into computer science because of the enemy AI at first seemed so clever, they could show the footsteps in the snow and they could do all these things. And then ultimately, obviously it was almost clownishly…

John Carmack: Its a bunch of if statements.

Andrew Bosworth: Yeah, totally. At first it seems so smart. And then like, you know, two levels in, and you’re like, ‘Oh man, this is terrible.’ That’s actually why I pursued artificial intelligence as an undergraduate was that it was Metal Gear Solid. It’s a true story. At the same time, it’s hard to imagine the gameplay, if you were really up against somebody who was, really intelligent or at worst, more like a human- really random, it’s a tougher, tougher gameplay.

Future Hardware (19:00)

Andrew Bosworth: Here’s a good one, John, that I think you and I will disagree on. What’s the next gen hardware feature that you’re most excited about for virtual reality?

John Carmack: There definitely are differences of opinion here across FRL where I think that if we took the specifications of what Quest 2 does today, and we made clearer optics, better ergonomics, longer battery life and cheaper- does the same things just does them better, that would be the killer product, but we have alternate points of view saying, ‘no, we need depth cameras, we need mixed reality sensors, we need eye tracking, we need face tracking.’ You know, maybe we need other things getting on here. We don’t know until we’ve exploited those and really tried hard maybe even for years to see what we can get out of those features. But I think that we’ve got enough signal that you can do so many amazing things with the current setup that continuing to focus on just making what we’ve gotten better, making it cheaper, lighter, smaller, higher resolution, all of these obvious, straightforward things the entire software ecosystem moves forward, everybody knows exactly what they should be targeting. I think that’s the path to as mainstream as we need to get. And then I’m happy to have some Pro version that’s going exploring every sensor in the kitchen sink. I just think that you’ll wind up with 1/10th of the users on there and we should be about kind of maximizing the user base.

Andrew Bosworth: John doesn’t want any new features. That’s not totally fair, he’s right, obviously the optics, displays ergonomics, battery life, and cost are huge features. And actually John has clearly been vindicated on that belief if you look at the last five years of VR development going from the Rift to the Quest 2, with a nearly 10x reduction in price, almost 10x reduction in total cost of system ownership, that is. And from an experience standpoint, we’ve certainly improved the displays, we’ve certainly improved ergonomics in some things, but it is to some degree, it’s a credit to the original Rift team who I think from a standpoint of what was the feature set they needed to deliver really did an excellent job, with things like Go and Gear VR we’ve tried going below that feature set unsuccessfully, and we’ve kind of come back to it.

Importance Of Tracked Controllers (21:13)

John Carmack: I completely missed the value of how much we get out of tracked controllers. My beloved Beat Saber just couldn’t be done on an Oculus Go controller or something, or even do with hand tracking on cameras. So there’s clearly value there. But it’s interesting how we are on this multi-year path to phasing that away as a core feature, where we want to be able to have a controller -free SKU in the future where we rely just on hand tracking for people that want to use keyboard and mouse and don’t want to pay for the controllers. So it is clearly more valuable for gaming than I initially expected. But I think that we will wind up in those cases where lots of the users will not wind up taking advantage of those in the future. And that’ll be nice to have that as a separable feature.

Andrew Bosworth: The good news is we can do both. We can do both, you can have a set of techniques and development that are going to put something out there that has a more featureful presence. And this is going to go at maybe lower volume in terms of the number of units, but also advances the state of the art, inspires developers, I think unlocks a lot more use cases. And then as that technology matures, finds its way into these scale units that get out to so many people, which we want to continue to advance the drum beat on that.

Prioritization (22:24)

Andrew Bosworth: Someone asked earlier what revelation inspired subscriptions. It’s not a revelation. We’ve always wanted to do it. People sometimes think that because Facebook’s a big company and Oculus is well-funded, and it is in fact, John sometimes remarks that we might’ve been too well-funded, allowed people to get out out and about too far, which I think we’re putting reins on. It doesn’t mean you do everything. Prioritization and focus is still important. When you’re prioritizing, it’s not like I’m cutting bad things. I’m cutting things that could be good, but we just have a belief in some other value. And so sequencing how you bring these things out to market, how you put your time and attention into something, matters a lot. So we’ve always believed in subscriptions. We knew we would get here and I’m thrilled to be there this week. We’ve got obviously the explosion of fitness applications, are kind of the really pressing place that we’re going to take advantage, but they’re not the only ones.

Simulator Sickness Mitigation (23:11)

Andrew Bosworth: What are the keys to reducing motion sickness. John, do you want to share anything about the progress we’ve made there?

John Carmack: We had an interesting discussion about this recently where someone in research made some comment about ‘what causes discomfort is when something’s moving in your visual field in a way that’s inconsistent with your vestibular system. Some of the mitigations that we tell people about about, well, like a cockpit around the edges can help because that’s the furthest out to your periphery, that largest motion, if you cover that up, people do the shrinking vignettes when you’re moving. I’m kind of excited about this prospect of if we do a little bit of changes to the engines and pass some more depth information, which we want to do for positional time warp things, anyways, it’s possible for us to do a systems level approach that’s actually aware of the depth next to it. So in a game like Population: One, they offer multiple comfort levels that determine like how much you pull in at the sides and it’s helpful for people. But it’s wasteful in some sense, because if you’re outside, you’ve got the sky at effectively infinity and that causes no impact to your comfort at all when it’s moving there. But the vignette winds up covering it anyways. What we need to do is look at the depth of things relative to your view, how much it’s moving incorrectly relative to the inertial stuff and only fade out things proportional to that relationship. So I think that if you’re ducking for cover right behind a wall, that’s the stuff that can really make you sick if you translate next to it, the sky doesn’t have any impact on it whatsoever. So I think that we could do something system level that could then be uniform across games, which would be great because right now each game has its own mitigation method and it would be good for users if they just realize that, okay, this is the way VR worlds behave when you’re close to something and you slide with the controller, you can expect that to kind of vignette out on the side. So that’s the type of thing that heck it’ll probably take us two years to sort of work something out and push it through developers and get buy in and get people to agree to it. But I think that’s a longterm direction that’s got some real potential.

Competition (25:14)

Andrew Bosworth: Will pass through ever be accessible for developers to integrate into their games? It’s actually funny Guy Godin posted this the other day, his annual reminder that pass through isn’t available. I told him on DM that he wouldn’t have to post it again. It is coming and we’re excited.

Andrew Bosworth: Which companies will you compete against in AR hardware? AR hardware is a little further field. It’s hard to tell how competitive that’s going to be, but let me tell you more approximately in VR. You point to the obvious, point to Sony PSVR, great product. And it’s not that we don’t see ourselves as competitive because, obviously to some degree we are, it’s just that like VR is so small, more people investing in it is a good thing for the developers and thus for the entire VR community. I kind of think the same thing’s going to happen in AR, I’m not saying we won’t compete in these headsets. We will. But to some degree, the investment that the entire industry is putting in is what’s going to be required if we want to see a breakthrough in computing of that magnitude. So for the most part, we’re just much less oppositionally framed than people expect me to be.

John Carmack: I am really happy that Vive exists because they built a whole hardware ecosystem and a software ecosystem that are things that we don’t really have on Oculus. And the world is better for having that outlet for the people that want to do those things. And I know there’s going to continue to be some things that, you know, we are adopting some Steam-like practices in some ways but there’s going to be some aspects of like the way their hardware works, the way their extra trackers can work and things that they just do on Steam that is going to take us years to catch up to in our own system. So it’s good that they’re there and I’m happy for their success.

Andrew Bosworth: Yeah, totally the same. I picked Sony, but just as happy to mention HTC there. All right. We’re out of time, John, I want to wrap up with a question that has come up a bunch of times. What games are you playing right now?

John Carmack: So I play Beat Saber really regularly. I do at least once or twice weekly the public server games, which are a ton of fun. I also play POPULATION: ONE I am not super awesome at it, but I’ve got a little squad of three or four of us that get on and play. So if you see a John Carmack on like usually Thursday nights or Sundays, it probably actually is me, my blue puffy new jacket, but I am having a good deal of fun there. I’m also playing the Walkabout Mini Golf with the same crew, which is also quite a bit of fun.

Oculus Quest Software Improvements Could Aim VR For Chromebook Market

Facebook is trying to increase production of Quest 2 amid a global silicon chip shortage while planning regular software updates for its $299 headset that may put the headset, and its follow-ups, on a path that may convince some buyers to pick up a standalone VR headset instead of a Chromebook or tablet.

A recent talk between Facebook’s vice president in charge of VR, Andrew Bosworth, and consulting technical officer John Carmack offered some of the clearest insights we’ve seen yet into the company’s strategy in VR.

For instance, the talk revealed Facebook is planning to keep Oculus Quest 2 available to buy for a “long while” while making clear a more sensor-laden “Quest Pro” won’t be hitting store shelves this year. The conversation was held in Twitter Spaces, which doesn’t yet archive audio broadcasts. We recorded the interaction, however, and plan to post a full transcription and edited audio file later this week.

We also went through some of the comments in our virtual studio earlier this week which you can check out in our video below.

Oculus Quest’s Year Of Software

Another section of the talk refers to a “year of software” updates meant to deliver “a rolling thunder of product improvements for the Quest 2,” as Bosworth put it.

Back in January we learned Facebook planned monthly updates for Quest with the latest version, software v28, marking a significant milestone with the addition of “Infinite Office” features to bring your desk into VR and the ability to track a bluetooth keyboard with integrated trackpad so it is feasible to actually do some work in VR.

Carmack mentioned the need for Facebook’s VR products “to be something that somebody hard up for money decides I’m going to buy a VR headset instead of a Chromebook or instead of a tablet. And we need to do everything that those devices do. You know, we need to have similar app libraries. We need to be just as effective with keyboard and mouse. We need it to be something that you could put on your head and do the work that you need to do during a normal day.”

Carmack notes this type of use case has been regularly demonstrated for the better part of a decade, but they “were silly gimmicks at the lower resolution” inherent to earlier headsets. He noted the need to further improve the optics, resolution, and weight of headset designs over subsequent products, but argues that Quest 2 is “at the point where it’s becoming reasonable,” and they need to get the “software infrastructure in place” so that when the hardware is good enough they can tap into “hundreds of millions of users.”

“We can absolutely make it better than cheap little netbooks that people are hunched over in terrible posture,” he said.

Here’s the transcribed section of the conversation with notable portions bolded by us:

John Carmack: My internal pitch for the best super broad strategy for VR, aside from gaming, is screens and people. One of the points that I make is that the investment into conventionally viewed media, whether it’s content, media, applications, things that you do on your screens, your phones, your PCs, your TVs. There’s trillions of dollars of investment that’s gone into that over the last 50 years. And I think it’s just insane to consider that, well, VR is a blank slate people need to come in and reinvent everything. So I think taking advantage of all of those surfaces is something that we absolutely have to do in VR because right now we are still largely an early adopters toy where a lot of people that have VR already have everything else and we’re just adding some new spice, but we need to be a displacement device where we need to be something that somebody hard up for money decides I’m going to buy a VR headset instead of a Chromebook or instead of a tablet. And we need to do everything that those devices do. You know, we need to have similar app libraries. We need to be just as effective with keyboard and mouse. We need it to be something that you could put on your head and do the work that you need to do during a normal day. And I think that we’ve seen the glimmers of light on what we need to do for that. And with Quest 2’s greater screen density we are at the point where it’s becoming reasonable. Ever since DK1 people made demos of “oh, look, I did some programming work inside a headset,” but they were silly gimmicks at the lower resolution. We’re at the point now where it’s still not great but it’s like looking at a 1080P monitor that happens to be gigantic, but you can be flexible. You can do all these great things where it’s not going to be as good as like my home workstation with triple monitors, but we can absolutely make it better than cheap little netbooks that people are hunched over in terrible posture. So I think that on the Quest 2 platform, which as you say, is going to be in market for a while that we can make something that is better than doing work -like some people have to do emergency things on your phone we can absolutely be better than that. You know, we can be better than the cheap netbooks and we can start competing with some of the better systems. And as we move to future products, whether Quest Pros, Quest Quest 3s, whatever. We could have the software infrastructure in place so that as our optics improve, we really can be looked at as a monitor replacement. And I think that’s how we get truly mainstream when we’re talking hundreds of millions of users.

Andrew Bosworth: There’s this old story in the PC era about called “The Reason To Buy And The Excuse To Buy” and the reason to buy a PC was to play games, the excuse to buy was to do spreadsheets. And until you could do both games and spreadsheets you couldn’t buy it because you need both the reason and the excuse. We’ve got plenty of reasons to buy Quest 2, I think for some people they’ll need the excuse. They’ll need to have a justification they use that introduces that technology to them. And that is really critical to breaking fully mainstream to the point yeah, to your point where you’re getting out to nine and 10 figures of people in the headsets.

Update: Post updated about 30 minutes after publication with changes to how it refers to current and future Facebook headsets and the overall direction of software updates.

DrBeef: Half-Life 2 Port For Quest 2 Will ‘Probably Never Happen’

In an excerpt from our upcoming interview with community modder Simon ‘DrBeef’ Brown, he said that he would love to port Half-Life 2 to Quest but that it will “probably never happen.”

The reason is mainly practical — all of Team Beef’s ports are possible because the games’ original engines are open source, allowing them to be ported to Android for a Quest release.

Half-Life 2 runs on the Source engine, which is free to use for Steam users but not exactly open source (pun sorta intended) and therefore any Quest ports of the game would not be direct translations of the original. Open source engines also allow modders to port the engines in full without running into any issues.

Here’s DrBeef’s response in full:

UploadVR: I’m wondering if you decided to port something that was Quest 2-only how recent do you think you could go with a game. And is there a dream game that you would like to see ported? Or have you already gotten there?

DrBeef: The problem is there’s a dependency on the actual software being open source. As far as modern games go, I think Doom 3 is kind of — I’m quite happy to be corrected — but Doom 3 is probably one of the most recent well-known popular games that actually has its engine code open source. After that point, I think, id software stopped doing that. There’s not many games studios where they actually open source their engine software. It’s a bit of a shame.

The game I would most like to do, but I think will probably never happen is Half-Life 2. Because I’m slightly ashamed to admit I’ve never played all the way through it. And also I got halfway through playing it on the original dev kit, but never finished it. You know, it’s one of those games that we get asked about all the time. There’s somebody who’s managed to do some sort of Android port somehow. I don’t know how they’ve done it, but it’s, you know, it’s not like a truly open source engine port.

So I think there would be some risks entailed with trying to pick that up. So as it stands, I think that game is out of reach, but, yeah, that would kind of be the dream game to bring to the Quest, but I fear it’s probably never going to happen.

Keep your eyes open for our the rest of our interview with DrBeef, which releases tomorrow on our YouTube channel and here on the site as a transcription.