What comes to mind when you think of 3D user experiences? Gaming? The immersive digital experiences made possible by virtual reality (VR) and headsets like the Oculus Rift? Or perhaps augmented reality (AR), which overlays our view of the real world with 3D graphics via smartglasses like Microsoft's HoloLens?
Surprise! Experts say your smartphone will soon become the dominant platform for 3D, thanks to an escalating trend of what's called mobile AR. Mobile AR is a hybrid of 2D and 3D characterized by an interplay of objects in 3D space with "screen-locked" or fixed elements. It centers on the camera of your phone, tablet or phablet.
There are hundreds of mobile AR apps of varying types and degrees of sophistication. You'll find them in the same place you get your regular 2D apps — in the app store of your mobile device. Want help designing your living room or office space? Try Sotheby's Curate or IKEA Place. Learn about rivers with WWF Free Rivers. Or have fun with Lego in 3D with Lego AR Studio.
Mobile AR took off in a big way last summer when Google and Apple launched their mobile AR platforms — ARCore and ARKit, respectively. The releases sent shockwaves through the 3D industry. Suddenly 4.5 million designers and developers — till now focused on building 2D mobile applications — had a chance to make a dent in the 3D universe.
Here are six ways mobile AR is reshaping the 3D landscape.
1. Accessibility and availability. While high-end VR and AR hardware has struggled to gain traction in the market, mobile AR is available on the hardware we carry around in our pockets every day. For enterprises, mobile AR is riding the bring-your-own-device (BYOD) wave of the past few years. The potential addressable market for mobile AR is already huge. According to ARtillery Intelligence, AR-compatible smartphones will number nearly 2 billion this year and will top 4 billion by 2020.
2. Device familiarity. Dedicated VR and AR devices like the HTC Vive, Sony PlayStation VR, Meta 2 and Magic Leap One take some getting used to, and not just because of price. But we're all familiar with our smartphones and their touch interface. They make a nice entry point for people thinking about 3D, not just for games and entertainment, but as a means of input and computing.
3. Collaborative experiences. AR on your mobile device lends itself to much more collaborative, communal interactions than headsets or smartglasses, which limit the VR/AR experience to the wearer. With mobile AR, you can hand your mobile device to another designer or user, or have them look over your shoulder to share your experience. And it's a safe bet that someone can use their device to collaborate with you because it's just as capable as yours. "We're seeing lots of new workflows and interactions coming off of this," Reynolds says. "You can share experiences with others, but you also have your own free agency with your device."
4. A 3D model of the world. In order for AR experiences to be immersive, a 3D map of the world is needed. Think about it: If a virtual character jumps onto a real sofa or runs into a real wall, we expect the character to bounce or collide when it hits these surfaces. If a virtual character or object moves behind an object in the real world, we expect it to disappear from view (an aspect of AR known as occlusion). "You want people to believe that the virtual content is really in the world with you," says Matt Miesnieks, co-founder and CEO of 6D.ai. "This can only be achieved if you have a virtual model of the world overlaid on the real world." To achieve this, 6D.ai is launching the 6d.ai Reality platform in October in private beta that developers can build on top of. Just as the Waze app gathers traffic data in the background to be shared with all users, any app built on the Reality platform will crowdsource data to create this 3D map of the world. "It's not a nice-to-have feature for AR to work — it has to exist," Miesnieks says.
5. The 5G effect. Today's cellular networks often struggle with the bandwidth requirements of mobile AR. 3D assets, such as models, animations and the accompanying audio, are much heavier than standard web apps. But that will change with 5G, the next generation of cellular technology. 5G will reduce our dependency on Wi-Fi and make mobile AR even more mobile than it already is. Bandwidth and latency become even more critical with shared mobile AR experiences, because 3D assets — and the interactions between collaborators — must be downloaded and distributed over the web to other devices in near real time. "We do most of this over the cloud," Reynolds says.
6. AR for IoT visualization. Mobile AR provides an efficient means of visualizing the positional data made possible by beacons and the contextual data enabled by IoT. In the past, a maintenance worker would have walked around a warehouse full of generators with a clipboard, noting how long generators had been running, which were due for maintenance and so on. But with mobile AR and IoT, the worker could simply hold up a mobile device to see a visual display of streamed data showing which generators needed maintenance. Similarly, in a sports arena, fans could see AR displays of player stats, video clips and more simply by pointing their phone at the number on a player's jersey. "There's a huge ROI potential using mobile AR as a visual interface for these contextual data streams provided by IoT, especially with 5G," Reynolds says.
A stepping stone
And what of the high-end head-worn AR/VR devices and the massive investments already made in them? Will mobile AR displace them permanently? Reynolds sees mobile AR as a stepping stone to greater adoption of these form factors.
"With mobile AR, we have huge market penetration that's already there with very affordable devices," Reynolds says. "All the skills we learn from mobile AR will directly translate to the wearable context. This is how we get a market for 3D."