Computerworld

The TV is the new tablet: How gesture-based computing is evolving

"You want to be able to point and click on what you want to interact with"

Few people watch television alone today, even when they're by themselves. Most are gravitating toward the multi-screen experience, in which viewers keep a smartphone, tablet or laptop close by so they can access the Web while they watch TV. But as televisions become smarter and gesture-based computing evolves, viewers may be able to mount and control everything they need on the living room wall.

IN PICTURES: 12 touch-less computer navigation projects

Shafa Wala is the co-founder of Tarsier Inc., whose gesture-based technology MoveEye attracted its share of attention at last month's DEMO conference in Santa Clara, Calif. MoveEye enables users to navigate a smart television with hand gestures, basically pointing to and interacting with different points on the television. In a video showing how MoveEye works, a demonstrator wearing connected eyewear carries out common tablet tasks like accessing e-books and playing Angry Birds, then moves on to more intensive functions like playing driving video games and Diablo III.

Though impressive, the concept is hardly new. Microsoft Kinect and Nintendo Wii have made gesture-based interaction famous over the past few years, while the highly anticipated Leap from Leap Motion touts easy touch-less interaction with the PC. Wala admits that these tools are "quite viable," but says Tarsier is specifically targeting the market for smart TV navigation.

"Our opinion is that there was nothing that was expressive enough to navigate a content-rich screen and to navigate media when we do have a content-rich screen," Wala says. "We didn't really have in our minds that we wanted to replace the remote control or replace the keyboard and mouse, but mainly how do we define an expressive way to interact with your TV in the same type of efficient way that you interact with your computer or touchscreen?"

Nevertheless, for more intuitive navigation to become popular, the technology will have to work well enough to convince users to migrate away from traditional input devices. Wala notes that the television remote is already being replaced by tablet apps, such as Comcast's Xfinity TV app for iOS, which offers more efficient remote navigation than traditional options. Similarly, speech recognition technology will need to evolve to the point that "you can just basically eliminate the need for handheld and tabletop input devices altogether," Wala says.

Stephen Prentice, vice-president and Gartner fellow, agrees that user navigation will need to evolve as the content available on the television grows. Prentice has followed the gesture-based computing field for years, and in 2008 he famously predicted that the computer mouse had just five years before it became largely obsolete. Although he admits today that the mouse will remain applicable in certain use cases, he cites the continued decline of Logitech when he says he "got it right."

Prentice says "the role of the TV is changing inside the home," and believes that before long it will become simply "the largest screen in the house." Because of that, he can easily see why consumers would be interested in next-generation navigation capabilities.

"Losing the remote control is so common," Prentice says. "It's interesting to see this generation of super-smart televisions and they all come with a remote that was built in the 1990s, which just seems kind of crazy."

Many modern televisions are equipped with media applications that are accessible with these outdated remote control devices. Wala suggests adapting the same mobile apps found on Apple's App Store or Google's Play store for the television and making them accessible through gesture control. The app stores could even make themselves available in this form factor, enabling users to purchase and download new apps directly on the TV, Wala says. Once the television reaches this level of access to content, gesture navigation will feel as intuitive as touchscreen interaction did on the iPhone, he says.

"You want to be able to point and click on what you want to interact with," Wala says. "You want to see things from your perspective and interact with them in a way that you would on your touchscreen-type device, and Kinect can't do that and Wii Motion can't do that."

Kinect and Wii Motion are more suited for capturing broader gestures, such as full-body motion or swiping from one page on the screen to the next, Wala says. Tarsier doesn't necessarily compete with them, but aims to improve pinpoint navigation for more control over the specific content on the TV.

Wala says Tarsier aims to license the technology to OEMs that can integrate it into televisions before shipping them. He may be onto something; Prentice says the market for gesture-based navigation is still entirely up for grabs.

"To be honest I think the market is wide open. Kinect opened things up and gave people a low-cost way to get in there and experiment," Prentice says. "We don't yet have a sort of dominant player in that space."

More broadly, Prentice believes touch-less, gesture-based computing holds significant advantages over touchscreen technology in certain use cases. In healthcare, for example, the tablet can be an operational godsend, but when multiple doctors and nurses touch them in between examining patients, they can become "germ magnets," Prentice says.

"A touchscreen you don't have to touch, which sounds like a bit of an oxymoron but you understand what I mean, makes a heck of a lot of sense," Prentice says.

Still, the path to mainstream gesture-based computing has its obstacles. Prentice cites the problem of differentiating between gestures meant to control the device and natural motion when users are simply moving around near it. Companies like Tarsier, Leap Motion and Microsoft are undoubtedly tackling those challenges, racing to be on the forefront of the latest user navigation revolution. Once these issues are ironed out, though, gesture-based computing will follow the same path as every other technology, and artifacts like the mouse and keyboard will be relegated to history.

"There are challenges with it, but it's like all these things. People will get used to it very quickly," Prentice says. "It'll come in at the high end, it'll be taken up by the technology progressives and within five or six years everyone will be doing it. And we won't remember anything else."

Colin Neagle covers emerging technologies and the startup scene for Network World. Follow him on Twitter @ntwrkwrldneagle and keep up with the Microsoft, Cisco and Open Source community blogs. Colin's email address is cneagle@nww.com.

Read more about data center in Network World's Data Center section.