Feature Story

Augmented Reality Remaking the SmartPhone Experience

New Generation Augmented Reality apps on SmartPhones use. Cameras and Sound to provide new types of information and experience for users.

April 18, 2011

By Bill Bulkeley

When the third Harry Potter book appeared in 1999, one of the startling bits of wizardry at Hogwarts was a magical map that labeled people walking around the school and showed hidden passages and how to get into them.

Today, the wizards developing apps for smart phones are doing similar things with augmented reality.

AR, as it is known, is a technique for enhancing the real world with digital information automatically delivered to a viewing screen. A classic example: in televised football games, the digitally-created yellow line that shows how far the ball carrier needs to rush to get a first down.  The real scene is augmented by digital information.

First generation AR on smart-phones was similar to GPS map systems that could identify gas stations, pizza parlors or parking garages in the vicinity of the car. Now, creators are taking advantage of a phone's camera systems to create more advanced capabilities.

Peter Marx vice president of Business Development and Digital Studio at cellphone technology developer Qualcomm recently wrote on his blog that using the camera image provides "a new type of interactivity which lets developers, publishers, advertisers and others create surprising and compelling experiences which had until now not been possible."

Bewildered by a store-front sign in Buenos Aires? Fire up a $4.99 iPhone app called Word Lens , aim the camera at the sign, and you'll get an instant translation.   It can be very handy for understanding menus.

The creators of the Lonely Planet guidebook series have embraced AR for digital guidebooks. A traveler with a Lonely Planet app and an Android phone can open the camera, frame a streetscape, and see practical information and useful tips about the sites in the viewfinder. The guides cover 25 cities from Boston to Barcelona to Hong Kong for $4.99 each.

Some people are more intrigued with augmenting reality themselves. Sander Veenhof, an Amsterdam-based artist, says he uses AR software from Netherlands-based Layar to get his work "hung" in museums "that are not very accessible," like New York's Museum of Modern Art.  He creates virtual reality sculptures for particular sites, and tells followers to go to the selected site and call up the art work while the smart-phone camera shows the museum walls or ceiling as background. Veenhof has also created a work called Biggar that he claims is the world's largest virtual reality piece. The ever growing infinite series of squares expands to fill the sky.

MIT professor Patti Maes, one of the pioneers of AR, sometimes refers to it as a "sixth sense." A company called Broadcastr.com is trying to go her one better by creating verbal AR. Broadcastr lets individuals and companies dictate stories about places, institutions and stores.  The stories are linked to particular addresses. When someone with the Broadcastr app on his or her phone sees a highlight near them on a map on the smart phone, they can click on the spot and hear the story.  Much of the content is whimsical, but travel guide Fodor's offers spoken information on neighborhoods in New York and San Francisco.

The next phase of augmented reality is likely to make it more personal.  At the recent Mobile World Congress in Barcelona, one of the most talked about products was a cell phone app developed by Viewdle, a Palo Alto-based maker of facial recognition technology.  Using Viewdle, a cell phone owner can take pictures of any of 20 people stored in its memory, and the phone will automatically tag them for uploading to Facebook. 

The just-released Version 3.0 of the location-based application Foursquare, includes an "Explore" tab that lets a user type in what he's looking for and then returns a list of nearby recommendations based on how many "check-ins" the site has received from the user's friends and from all Foursquare users .

Tablets may be the next big market for augmented reality.  While the first iPad didn't have a camera and was a little awkward for using on the move, newer tablets are lighter. Some like the seven-inch business-oriented Cisco Cius also have a smaller form factor. 

With a bigger screen, augmented reality on a tablet could be "more intense," says Amsterdam artist Veenhof.

The wizards at Hogwarts would undoubtedly agree.

The contents or opinions in this feature are independent and do not necessarily represent the views of Cisco. They are offered in an effort to encourage continuing conversations on a broad range of innovative, technology subjects. We welcome your comments and engagement.