/ Virtual Reality

Siri, give us a hint about VR

Immersive computing is going to be the biggest digital interface paradigm shift of the next couple of decades and AR and VR are about to take off. This is certain; anyone who has tried the HTC Vive or the Oculus CV1 knows that Virtual Reality is just too cool to fail. If you could empathize with the characters of someone else's creation, draw and invent with unlimited tools and an infinite canvas, and explore worlds either real or imaginary from your living room, why wouldn't you? There is a reason that Valve, Facebook, HTC, Samsung, Microsoft, and countless other Silicon Valley companies have shifted their attention to this space so quickly.

I have heard colleagues and friends claim that Apple has historically been late to the tech table. It does seem true that they haven't always been the first to come up with ideas for platforms. Instead, they innovate by swooping in and releasing mature devices and software, ready for immediate adoption by massive audiences. As Benedict Evans pointed out on the most recent A16z Podcast, Apple didn't announce anything really new at their Sept. 9th media event, but rather repackaged existing ideas and technologies to make them more useful and easily consumable. Apple consistently makes a splash in the market whenever the bleeding edge of what is possible can be squeezed into a spectacular form factor and function as part of daily life. For example, the iPhone was released in 2007, after a series of Samsung, BlackBerry, Palm, and Motorola devices took hold (first with QWERTY keyboards, stylus input, and then touchscreens) between 2003 and 2006 and now it unquestionably defines what a Smartphone should be. Same thing with the Apple II - that was a game changer after the "computer" had already been invented.

Analysts make something of a game out of predicting what Apple will do next. It usually drives me crazy, but most often they look to whatever technology is not quite ready, but has massive potential as a consumer platform and foretell Apple's involvement. I'm going to be the one to speculate this time: I think that we'll see Apple VR within the next ten years.

Gene Munster (of Piper Jaffray) has already theorized quite a bit about Apple's future in VR. An article in Forbes summed up his evidence at the end of last month, which includes Apple's:

  1. acquisition of Metio,
  2. acquisition of PrimeSense,
  3. and AR engineer recruiting.

Apple's VR patent filing this past winter and their VR job listings only reinforce Munster's predictions. Up until now, however, I have been unable to construct a convincing narrative in my own mind about the what future form Apple VR will take, and how VR HMDs and computers will fit in their existing ecosystem of products. Is there a possible roadmap that ends in my purchase of a beautiful set of Apple goggles? Especially a pair that sit alongside my MacBook and iPhone?

As the product line has evolved over the past few years, MacBooks, iPads, and iPhones have become sleeker and more efficient, but at a performance cost. Apple will stop at nothing to improve battery life, focusing on building light and fast products that last all day without a recharge. Apple engineers are slaves to cutting mAh consumption to a trickle.

As a result of Apple devices' lacking GPU power, the major HMD manufacturers (HTC / Valve and Oculus / Facebook) have dropped future support for OS X, with good reason but not without prompting some outrage. If Apple wants to get back into the VR game, they need to release bleeding edge VR technology to consumers, which will only be ready and meet the Apple standard when

  1. hardware is powerful enough to drive a beautiful VR experience,
  2. input has been figured out so we can interact with each other and the digital world in VR,
  3. and the system is precisely motion-tracked but wireless, free of restricting cables which detract from the experience.

If VR meets the above three criteria, then the technology can "get out of the way" entirely, allowing us the full freedom to explore the digital world without distraction.

The iPhone needs to have a long battery life, so it can't be a VR device in the same way that the Samsung Galaxy pairs with the GearVR; I just don't think Jony Ive would allow it. MacBook GPUs are draining less and less power, so they won't fit the bill when it comes to running good VR either. If only Apple were to build a box connected to the internet, wired into your home entertainment system, integrated with a content library, and plugged into the wall to alleviate some of the suffocating power restrictions that constrain current mobile devices...

Call me optimistic or blinded and biased by my job, but I could not help but fixate on VR during the the September 9th Keynote. This is wild speculation but mark my words: the Apple TV will evolve to become an all-encompassing graphics box that will drive all matter of entertainment and interactive digital experiences, including VR. From watching movies to booking flights we are going to rely more and more on smart set-top boxes, the perfect housing for some serious computing brains. Within the next few years, I bet Apple will be able to fit a real GPU into the Apple TV form factor, even if they need to custom-manufacturer it.

And they're already tackling the VR interface, training users to be comfortable talking with Siri instead of giving keyboard input. Keyboards are not built for VR - you can't see your hands when you are wearing those goggles. You can talk, however, and most people talk quite a bit ("woah... wow... look at that!" amid wild gesticulations) when they try VR. What better input device than the user's own voice?

Finally, VR needs to be wireless, with good motion tracking and without the need to set up external trackers every time you want to take the leap out of this world and into the digital multiverse. The living room, next to the 5.1 surround sound system (also connected to your Apple TV) is the perfect place to install such trackers and wireless equipment. HD streaming won't be easy, and I haven't seen technology that can do it with latency as low as is necessary for VR, but someone clever might figure it out (ahem... Apple?... you listening?).

I would be remiss if I did not also mention Apple's extraordinary opportunity to set themselves apart in the VR landscape by marrying custom hardware with custom software. To date, the VR community has struggled to make the technology work on multiple GPU devices and computer platforms. If there is one thing I learned at SIGGRAPH this year, it's that extraordinary rendering performance comes from tight hardware-software integration. John Carmack discussed the problem at length at OC-1 and there were many talks about chip-specific rendering techniques at SIGGRAPH, both for mobile VR and desktop. Apple has long touted their ability to make both the hardware and the software as a key differentiator. Because Metal, their high performance graphics library, now runs on the Apple TV, it makes sense that they will ultimately be able to customize it to provide the best low-level GPU integration possible to provide for a performant VR experience.

Like they did with the PC (iMac), the MP3 player (iPod), the tablet (iPad), phone (iPhone), music store (Apple Music / iTunes), and are doing with the Watch, Apple might provide the linchpin required to secure VR's place in everyday life. The narrative of how they get there is murky, although a plausible story of mass adoption of an awesome Apple product that fits nicely into their product ecosystem certainly piques my interest and makes my ideal future for VR way more real.