Brain Dump: Apple’s Upcoming XR Headset

“Microsoft has a history of releasing a product and a few years later Apple ships the same thing but better and wins the market. We can’t let that happen with the HoloLens”

I joined the HoloLens team a few months before it launched. Almost everyone on the team told me some variation of the above quote and many of those people are now employed in senior positions at Apple.

I’ve since left Microsoft and I’m deeply curious to find out what Apple has in store. For fun and accountability I’m going to offload some wild predictions about their upcoming XR headset.

This post is a brain dump instead of a carefully written article. Later I’ll look back and see how far off the mark I was!

The Hardware

What will Apple’s headset look like? This is a real challenge for Apple. Apple’s brand is sleek, precise, advanced, comfortable, and minimalist. Existing VR headsets are not those things.

Ahead of Apple’s Watch reveal I sketched in my notebook possible designs. I suspected they’d have to make it circular somehow because a square watch would look far too awkward. Apple instead went for the “squircle” shape: a rectangle with soft rounded edges.

I thought it looked ugly.

Untitled

But then the marketing kicked in. They had all these videos of people using it in ways that made it look beautiful. Apple knows how to teach people to see the beauty in their products.

But VR headsets are a challenge to make look comfortable and beautiful. Here’s Facebook’s latest VR headset for comparison:

Untitled

Not a bad design! The white color choice makes it seem airy and light, and it looks a bit more premium than its $399 price.

But if you’re not a gamer the controllers have a strange and intimidating appearance. You might worry that you be isolated from the real world and lose the ability to properly use your hands.

The Quest also doesn’t escape the awkwardness of attaching inches of plastic to your face. Lots of plastic on the face looks both uncomfortable and dorky.

Apple will face a broad skeptical audience and they’ll have to make people think that their headset looks:

Even if Apple’s initial headset is targeted at an early adopter crowd the broader audience will be curious and paying attention.

Fortunately Apple has a stellar brand reputation so people are more willing to trust them.

I am eager to see how they balance making something inherently alien look sleek and approachable.

Wild hardware guesses

The headset will feature some sort of pass-through glow on the outside. A bit like their old laptops:

Untitled

This will help to make the front of the headset appear less isolating. The effect will work even better if it changes to faintly show the colors the person is seeing!

People outside won’t feel quite as cut off from the action. And if it looks like light is bleeding through from the inside of the headset it will hint at the passthrough that lets you see the outside.

The headset may be powered by a separate device carried in your pocket or perhaps even powered by a regular iPhone.

Untitled

Apple does this clever thing where they use similar technologies all over the place and their investment in ultrawideband always struck me as suspicious. Ultrawideband could perhaps be used to wirelessly transmit rendered images to an XR headset from a user's pocket.

I would not put it past Apple to make the headset itself a lighter-weight screen and defer the serious computation to a device you carry in your pocket. This could allow them to make the headset far more comfortable than an Oculus Quest.

And imagine if Apple rolled out a VR headset as an accessory that everyone modern iPhone was compatible with.

I would not be surprised if the headset comes with, or optionally sells, some sort of case to make the experience of using it feel more elegant.

Look at Apple’s AirPods and imagine if they didn’t come with a case:

Untitled

The case makes the headphones feel like a substantial unit, not just a cheap product. Today’s VR headsets feel exposed and tossed aside when they sit with their lenses out. So I wouldn’t be surprised if Apple designed some sort of “second half” that attaches to the lens side of the headset to cover the lenses and perhaps charge the headset.

I don’t think Apple will create game-like controllers. Games are so far outside of Apple’s DNA and I think Apple wants to avoid closely associating their brand to gamers.

Some rumors have alluded to ring attachments on fingers. It feels inelegant to me (and like another thing to lose), but if slight haptics in the fingertips dramatically add to the experience I can see it being included.

If Apple forgoes controllers I think they will introduce something that fulfills a similar role to the Apple Pencil:

Untitled

Most of the time you use your fingers to navigate iPads, but for more precise work you can turn to the Apple Pencil. Similarly with great hand-tracking in VR you’ll be able to do most things, but when you want to do serious precise work you’ll be able to grab some sort of “3D pencil”.

The Software

Apple will craft truly joyful 3D UI and ship a suite of apps to support the device.

The UIs will have light, shadows, and perhaps even some form of global illumination. Apps will look like they’re at home in your lighting conditions.

If your room is illuminated regular apps will look more like natural materials illuminated by your room lights instead of self-illuminating like a screen. If your room is dark apps will look almost bioluminescent as if they’re faintly glowing.

One of the presentation’s “wow” moments will feature familiar app icons that are now beautifully 3D.

The aesthetic of the UI will feature the return of skeuomorphism in the form of shadow, depth, and animation while still featuring ‘flat’ textures and materials.

Many Apple apps that don’t get a complete makeover will still benefit from their their new UI framework which will perhaps be called something like "RealityKit".

Apple may integrate HomeKit, HomePod, and Apple TV to let you control those devices by looking at them and performing hand gestures or voice controls.

It would make for an incredibly cool moment to gaze at a HomePad and see what’s playing floating above it and swiping right to skip tracks.

FaceTime will make an appearance in some form. Navigating the uncanny valley will be challenging but perhaps people in the headset will be able to see 3D construction from others iPhone’s.

There will be a few flagship games that really show off the device. I think Apple will want to launch with an adaptation of an existing aesthetically beautiful series. Monument Valley would be a perfect candidate to show off the new headset in a way that’s beautiful and widely accessible.

Untitled

It’s such a good fit that I bet the Monument Valley developers are working on something for the headset. It will be beautiful.

Otherwise I bet a few flashy but forgettable games are in the works. The device will have great gaming potential but historically Apple has struggled to pick and showcase the games that really resonate with people.

Apple always loves marketing to creatives (and those who aspire to be creatives) and I think they’ll have some sort of creative app shipping with the device. It might be something like GarageBand but you can see more panels at once or it could be a minimalist 3D creation app.

They’ll also feature something with architecture or architects. It just fits too well with Apple’s creative brand and with the possibilities of such a device.

Perhaps a company like Shapr3D will have some iPad integration where a product design can be sketched on an iPad and then rapidly viewed in VR.

I would not surprised if a number of 3D movies and TV shows are announced. We’ll see a director (James Cameron?) talk about how ground-breaking it is to be able to experience quality 3D cinema at home.

Likewise we may see some musicians talk about all the new things they can do with spatial music.

3D audio will play a prominent role in the headset. Apple has been investing in spatial audio with their headphones and AirPods and they’ll adapt that to this headset. This will feature prominently in the movies and games.

There will be a guided meditation experience. It will place you in a beautiful room with a variety of environments with things like colorful spheres, gently moving lines, and other abstract shapes.

Apps vs Worlds

Here’s where I think Apple will go wrong.

There are two key use-cases for XR devices:

Apple will nail the first use case but largely ignore the second. I view immersive worlds as a new frontier for socialization, play, and general entertainment. But I think for them to really take off they need to be:

This fits very poorly onto the AppStore model and Apple is very attached to the AppStore. I suspect instead of embracing “worlds” Apple will focus on apps with a few extra 3D capabilities.

They’ll likely succeed with their vision of the future but make it hard for any competitor to pursue the worlds aspect.

Games are the medium most similar to the this new frontier and it’s so very much not in Apple’s DNA I suspect they’ll miss the value of worlds and view them as simply another category of games.

Further making this problematic is Apple forbids apps that contain a “store”. This means that it’s impossible to build a worlds platform on top of Apple’s platform. Unless somehow you appear kidsy enough that Apple finds you non-threatening (like Roblox).

Another alternative is building a worlds platform on the web. But this requires Apple to implement WebXR in Safari. I worry that Apple will instead choose to treat the web as a document platform and embed the flashiest forms of 3D model viewing they can while neglecting the ability to ship WebXR apps over the web.

Perhaps Meta will find traction with their version of the “worlds” concept (which they call the “metaverse”) but right now they’re struggling. Once Meta’s UI/UX/comfort/fidelity is compared to Apple’s device they’ll find themselves doing some serious soul-searching.

But is missing out on XR worlds bad for Apple? Probably not. If a competitor were to have success with the worlds concept it would put pressure on Apple’s App Store policies.

It’s just a loss for the rest of us as it will delay a new frontier and medium. I’ll gripe about it for a decade.

Price

I’m going to guess $1.5k to $3k. I think this will be seen as a device for early adopters and that price-point reflects that. Apple will want to ship a great device to an audience that loves it rather than ship a good device to a skeptical broader audience.

Name

I think Apple will stick with their branding of "Apple Watch", "Apple TV", "Apple Pencil", etc.

But what word will they attach?

Here are some ideas:

In Conclusion

I predict Apple will ship a great product, face some early skepticism, but succeed over over the years after launch. It will be a great step forward to bring XR headsets into the mainstream but a large step back for immersive worlds.