Mark Zuckerberg and I are confusing the market about VR and AR and the future of all computing: here’s why we need to stop doing that

Mark Zuckerberg talks about the future of VR and AR at Facebook’s recent Oculus Connect conference.

Mark Zuckerberg has been talking a lot about augmented reality and investing a lot, too. Yesterday he did it again in an all-hands meeting that was live streamed to his Facebook account (it was very interesting to watch, by the way). Everyone in Silicon Valley knows that Facebook is spending billions on a future augmented reality headset and operating system, many even point to some new buildings it is building and tell me “that’s where Facebook is going to put its AR team.” Facebook already has internal plans for the next five years and most of these new plans, and excitement, are about augmented reality. It isn’t alone, we know of many companies that are spending billions on same, including Microsoft, Google, Amazon, Sony, Magic Leap, Huawei, and others. The tech industry is gearing up for a new paradigm shift.

At its recent Oculus Connect conference, where it was speaking to virtual reality fans and developers, Facebook made it very clear during its keynote “we are building AR glasses.”

The problem is that saying that confuses the market. Is VR going to turn into AR? Will VR “die?”

I’m responsible for that, too, having co-written a book “The Fourth Transformation” talking about how, in the future, we would “move through” computing while wearing said glasses. I now call the whole field “spatial computing” which includes all computing humans, robots, or virtual beings move through, including augmented and virtual reality.

Note that I’m not talking about the confusion that comes from what the industry is struggling to call next-generation augmented reality. “Mixed reality,” “extended reality,” or something else that some marketer will come along with.

I’ll leave the naming war alone for now. It’s clear there are two separate camps. One VR, one AR, and that will become clearer over time, and trying to force the two together, like Zuckerberg and I were doing is causing massive confusion as we wait for Apple, and others, to ship usable augmented reality devices.

VR is when you see nothing but virtual things. AR is when you see virtual things on top of, or replacing, parts of the real world.

While that belief isn’t wrong, where both Zuckerberg and I have it wrong is hinting that VR will “morph” into this kind of “mixed reality” future where VR and AR will merge and you’ll do both with one device on your face. I’m not alone. Almost everyone in the industry has “AR/VR” listed on their Twitter account. Most of the media that covers VR also covers AR. Almost everyone is hinting the two are joined. Many people in the industry talk as if it is a given that the two will be joined soon.

We are all wrong.

After seeing tons of bleeding edge optics and displays you soon will be wearing on your face, and thinking through the problem, talking with many VCs, big company employees, entrepreneurs, and developers, I realize I made a huge mistake in believing the world would soon be “mixed.” That won’t happen anytime soon. Soon being less than five years.

Why? The two use cases are totally different.

VR is about immersion. Escape from the real world.

AR is about utility. Adding utility to the real world.

The two have completely different goals and will continue to need completely different devices for a long time. At the high end of VR you can plug your headset into a very large and very powerful GPU from, say, NVIDIA. In fact, Facebook announced such a capability even for its standalone Oculus Quest. Truth is that the tiny GPUs in mobile phones simply aren’t good enough for many workloads especially if you are trying to “immerse” your user in, say, a bleeding edge video game.

Also, if you are trying to get high-resolution, wide field of view monitors on someone’s face, you won’t be able to drive those from mobile phones and their small GPUs anytime soon.

You can see this in today’s AR products (and despite what Microsoft or Magic Leap call their devices, they really are augmented reality devices that you wear on your face). The field of view in these devices is about half the field of view in your average VR device. They do NOT do immersion, or, if they do, they don’t do it well. Immersion is the feeling you get when your brain believes you are “inside” and “experiencing” something. When you go to a football game you are “immersed” in a way that you can’t get by watching the same game on TV. That feeling is what VR is aimed at giving you and it delivers it very well. So well, in fact, that when I first experienced VR I experienced vertigo. I’ve never experienced that while watching TV or looking at my computer screen.

This all snapped for me when I met Jon Griffith in the hallway at Oculus Connect. He was carrying around a VR headset. He asked “want to go to Everest?” He carried dozens of cameras up the highest mountain in the world and recorded Sherpa Tenji climbing it without oxygen.

The experience of being inside Griffith’s VR experience was gripping. It “felt” like to me that I was on Everest with him, and not in the San Jose Convention Center. I was so immersed I forgot about the crowds swirling around us and felt the cold and fear as he took us across ladder bridges and up steep cliffs of ice.

You can watch a trailer of his experience on YouTube:

VR is different than where we are going with AR and will not be replaced by AR because of that feeling of immersion.

Here’s the problem that’s coming soon.

Back to the augmented reality devices that Zuckerberg is promising. These augmented reality visors are designed to be worn in the real world and “augment” everything you look at. Qualcomm told me that by the end of 2020 we will see a ton of them announced (Qualcomm makes the core technology that Facebook’s Oculus Quest uses. Literally everyone in the industry other than Magic Leap and Apple uses Qualcomm’s spatial computing tech stack).

The thing is, these new “wearable in the real world” devices have a completely different design set of tradeoffs. They must “look cool.” In other words, they must at least look like a pair of Oakley blade sunglasses. They can’t look nerdy like Google Glass did. They certainly can’t be big and ugly like Oculus Quest/VR headsets are.

They also need to be battery efficient. You don’t want to carry a huge battery on your head. And they need to have displays that show you the real world. You will NEVER wear VR if you are putting your hands into, say, a bandsaw. Too dangerous. You also will never use a VR headset while crossing a street, even if that headset could magically become 1/10th the size of today’s headsets that are big, black, and ugly.

The problem with showing you the real world is you just can’t make an optic, yet, that gives you great immersion and great virtualized monitors. Maybe a decade from now that will happen, maybe even five years from now, that’s possible, but I think I will own two separate devices for a long, long time:

  1. A VR headset for personalized media and experiencing “immersion.”
  2. An AR headset for augmenting the real world and using at shopping malls, school, work, while driving, etc., focused on “utility.” Or, focused on being social with other people (Facebook’s plans for 2025, for instance, include you playing a new kind of virtual frisbee or football game on top of the real world, while at a park, for instance).

It is a mistake to think that VR will stop being interesting five years from now just because augmented reality is a bigger market and even though you can do a lot of 3D things, including playing new kinds of games on top of the real world these will hardly be “as immersive” as a full-blown VR headset hooked up to a gaming-level PC. I’m sorry I made that mistake and will keep that in mind, even though Irena Cronin and I (and our work for clients at our research/consulting firm Infinite Retina) are far more excited by the market potential of augmented reality visors (or, glasses, if that’s what you want to call them, I like thinking of visors because the first devices will struggle to get to the size of the glasses I’m currently wearing).

Now, the deeper question is: will software developers see them as separate, or the same?

We built a Twitter list of about 800 developers in spatial computing (both VR and AR) and are meeting regularly with developers (another was at my house this morning) and you can clearly see a separation in skillset evolving. Some, like Aidan Wolf, are clearly focusing their lives on utilities for augmenting the world. Most others are focusing almost wholly on immersive experiences. Watch this list for a while and you see there is a real difference between people building for AR and those building for VR.

One of the most successful VR developers and that show this demarkation between AR developers and VR ones is STRIVR, who makes VR training for Walmart. Notice how VR is used and why immersion is so important to training. This isn’t about augmenting the real world, it’s about immersing you into the training so you remember what you’ve learned.

Yes, they use similar tools, if not the same tool, as AR developers use: like Unity. Yes they think in 3D.

On the other hand who think more in AR are much more adept at sensor fusion and computer vision techniques. Why? Because they are trying to augment the real world, while those focused on VR are better storytellers, generally, and are more focused on techniques that immerse viewers into a game or media of some kind, like Jon did with his Everest VR experience.

Zuckerberg and I are messing up because we are trying to put both kinds of developers together in an unnatural marriage. For now we should stop it, and I will.

Myself, I’m going to focus more efforts on AR developers like Aidan (even to the point where I’m starting to build new lists of those just focused on augmented reality instead of VR). Why? Irene and I are much more interested in utility and augmenting the real world than, say, Hollywood movies, or video games. I respect those a lot, and love experiencing them on my Oculus Quest, but my professional life will be far more focused on, say, a surgery team that’s using augmented reality/spatial computing to augment their jobs, or a car manufacturer that’s using augmented reality to augment theirs.

Those who are working on immersive media and games, though, deserve to know their work is valued and that they have a strong future, and Zuckerberg, by having Facebook brag about its work in augmented reality without putting it into proper context, and explaining that VR work will remain separate and important for a long time makes them wonder if they have a future and, I can say now that, yes, yes they do have a future and a wonderous one at that. There are tons of new VR products coming and the customer base will continue to grow (albeit at a slower pace than I was expecting to see in 2019 for a variety of reasons).

Even if someday soon we have a huge optics breakthrough (and I’ve seen some that are possible in, say, 2023 or later) I think we still will see innovations in VR-only approaches that will bring better immersion. We should stop pushing the industry to join these two very separate use cases together into some sort of frankenstein device. We should stop forcing developers to be joined in some sort of weird way just because they both use the same tool, Unity or Unreal Engine.

If we do, we will always come up short. A small device probably won’t have great immersion and a bigger, passthrough device, that has amazing immersion won’t be all that great for wearing around the real world.

By forcing them together we will have to compromise and that’s not good for either camp at this point. Virtual reality should be seen as very different than augmented reality, even if both use the same spatial computing grouping of technologies. Zuckerberg should separate the two into separate strategy lines and explain these differences in a much clearer way, at least for the next few years.

The problem with reacting to Apple

Here’s another way to think about it. Apple hates VR, we’ve been told, and sees it as unsafe and unmarketable. It might be right about that, and I believe that’s one reason why Zuckerberg is signaling that he, too, is working on AR. He doesn’t want Apple to be able to position Facebook as “lame.” (Let’s leave aside the fact that Facebook has caused a ton of positioning problems all on its own, thanks to its various privacy and security missteps).

The problem with that is Zuckerberg is actually weakening his own hand that he’s spent billions building: VR and its immersive capabilities. Facebook has spent billions acquiring, and building, its Oculus brand.

Instead of trying to widen this brand to include augmenting the real world, Facebook should make Oculus stand for immersion and keep it only for VR. Yes, Apple will attack VR as “unsafe” and “not useful.” Zuckerberg should reply “yeah, but AR can’t immerse you.” If Oculus stands for immersion, it’ll win. If Oculus tries to play both sides of this coin, it’ll appear weak and confused, in comparison to Apple’s AR-only approach.

It’ll also confuse and piss off developers. There are tons of developers who want to exploit immersion. Andreessen Horowitz, er, A16z, invested tens of millions in Sandbox VR, and Disney has done same for the Void, for instance. These companies make very fun, very immersive experiences you play at your local shopping mall and both are expanding fast. They never will do AR. Why? They are totally about immersion and helping you escape from the real world.

Facebook should keep the Oculus brand focused totally on immersion, while letting its new “Horizon” brand focus on social interactions in virtual and augmented reality, and I recommend it come up with a new brand for its augmented reality devices and keep those separate from Oculus to keep from confusing the marketplace.

Doing that will let them have a stronger position against Apple, and others, like Microsoft, and Magic Leap, who don’t yet have a play in virtual reality, and probably won’t since both of them have decided to do an augmented reality strategy only for now.

Facebook should strengthen its VR brand, not weaken it. Immersion should be the rallying cry. If Facebook can get everyone to understand immersion is magic, then it’ll withstand any attack from Apple or Magic Leap. Plus, that’ll give Facebook the credibility to start up a new brand around the utility of augmented reality and really go to war with Apple, amongst all the others that will soon sprout new spatial computing products.

I will stop weakening VR. It’s huge and important and won’t be killed by AR.

Will Zuckerberg get this too? Time will tell.

One client changes everything. My compliments to Vanessa Camones. She just rose to legendary status in Spatial Computing

Does one client make your career?

I haven’t been able to sleep in the 48 hours since Vanessa Camones brought one of her clients to my home. I can’t tell you who they are. But they showed me some technology that goes way beyond anything I’ve read about in Wired Magazine or seen covered in other press yet. This company builds satellites and planes and lots of other things. Is hugely important to American history and technology. Unfortunately the NDA she had me sign keeps me from saying more.

I’ve been thinking about what they showed me ever since.

You’ll read about it someday in Wired or New York Times but that’s not what I’m writing about today.

Instead, I’m writing about Vanessa Camones.

I’ve known her for years. Decades, even. She’s always been nice to me, always shown me companies that are competent and interesting. Always had a kind word for me and about anyone else in the industry we talked about.

I ran into her a few months back leaving Facebook. She was working on something there that I’m still interested in.

But before Thursday I’ll be honest. Thought she was doing a great job but nothing she was doing rose to the top of my mind as a legendary consultant or world changer. She hadn’t brought me something that I thought the leaders in Spatial Computing industry, or the tech industry overall, should see.

That all changed Thursday.

When companies asked me “who would you hire to do PR or do other go-to-market consulting if you had a world-changing technology?” I wouldn’t have thought of Vanessa.

Today she’s at the top of that list.

Why? One client.

Now, to go all Gary Vaynerchuk on you, she did the hard work for decades. That put her in position to win that client. She never took credit, was happy to stay in the background. I bet this post is making her uncomfortable.

But, as of today, she is now the top go to market strategist in the Spatial Computing industry in my mind.

I no longer look at her that way. One client changed all that.

She, and her client, shook my world Thursday, and reminded me of other times when I was first to see an important technology, like when Siri was launched in my son’s bedroom. Adam Cheyer and Dag Kittlaus are burned into my memory because of that. Or when Elon Musk gave me the first ride in the first Tesla, before he gave his best friend Jason McCabe Calacanis a ride (they still are great friends and Jason, unlike me, had both the ability and risk-taking capability to become one of Tesla’s first investors). Or when Uber was invented by Garrett Camp or Travis Kalanick right in front of me at Loic Le Meur‘s conference in Paris. Or when I was an early user of Instagram, amongst its first 100.

What I saw Thursday will change the world as deeply as any of those and, because Vanessa Camones brought it to me, she has rocketed to legendary status in my mind.

Thank you Vanessa. I’m headed to San Diego on vacation dreaming huge new dreams that I wouldn’t have dreamed before Thursday. Anyone who gives me a new dream (and, honestly, some new nightmares because this tech is so crazy futuristic, Black Mirror has nothing like it) rocks.

Fighting over VR and Facebook’s Oculus Quest

Ahh, a 20-something-year-old guy was over the house tonight. We almost got in a fight over VR. Tried to tell me something other than the Oculus Quest is better after seeing the Quests on the floor and declaring “I would never buy one of those.” I tried to tell him the press, like CNET, was saying it’s the product of the year.

I almost eviscerated him with the research I was doing for our book and research reports we’re working on for our company but remembered that I have been arrogant and wrong like he was many times in the past and that I just don’t need the drama so I put my metaphorical weapons away and went to my corner to sulk and think about where people like him get their opinions.

After he left I said heck with it and wrote this post, but will keep his name out of this.

After all, it’s obvious he had never played with all the headsets yet he was so certain of his opinions and if he had he hadn’t spent more than a few minutes with each, which isn’t enough time to get a real opinion about the ecosystem of each. He got under my skin, mostly because he reminded me of myself.

Another thought. He reminded me of the nerds across my career who have made me feel like I didn’t know anything, so two triggers were pulled. The Unix ones in college who made me feel like the Macintosh I was using wasn’t “a real tool.” They all ended up using one within a few years. Or the people who didn’t want to join me on Twitter. Some of them now are running social media for big corporations. Or the other people who tried to push me to buy a Windows Phone, or other things that ended up failing in the marketplace.

Everyone has opinions, even me, and I’m often wrong too, I remind myself tonight, and even when I’m right no one really cares. Those who are wrong rarely admit they are wrong. Often they dig in their heels deeper before getting a clue that they are on the wrong side of history. Some of my friends kept using Windows Phones for years after the market had declared them dead and lame. One family member is STILL using a Windows Phone and called me a couple of weeks ago worried because Microsoft had announced it wasn’t going to be supported anymore. He was finally accepting that the world had moved on.

So, I quiet my soul, go back to the book I’m writing after interviewing captains of industry with the awesome Irena Cronin all week (these are all people who are doing amazing things in factories with VR and AR and associated tech) and I’ll have the last laugh. Or not. It doesn’t really matter.

But there isn’t a better product right now for the money and for the everyday consumer than the Oculus Quest. It’s a shame that my 20-something debate partner couldn’t shut up long enough to get me to turn mine on. He might have learned something. That’s the real lost opportunity for both of us tonight. I had the other product he was yakking about as well. It isn’t nearly as good for the price.

He would have learned why CNET is naming it the best so far of the standalone systems. He would have understood why I haven’t played our “bigger and more expensive and more ‘pro'” systems since getting the Quest even though they display more polygons and “technically” are better. In usage, though, they aren’t, at least in our house. The tether on the bigger systems just totally kills the experience and for many things the extra polygons don’t really bring much extra to the table. The Quest is so magical that if he had tried it on he would have seen that, but he had already decided it was lame before he got a chance.

We can use the Quest pretty much anywhere we want, while the bigger system is stuck in my office and, since I’m writing a book that system isn’t nearly as accessible to family and friends as the Quest is. Lately it’s been getting a lot more software too, because it’s selling somewhat decently and developers are taking note of reviews like the one in CNET I just posted here.

Ahh, anyone else get wound up about technology like I do?