The system, developed by a team of computer scientists at the University of Washington, harnesses wi-fi signals from a home router and uses them like sonar to detect movement and translate it into commands for all devices connected to the wireless network. So, for example, a single finger swipe means ‘turn the bathroom lights off,’ moving an arm towards the right means ‘skip the current track’ in iTunes. Or how about turning on the coffee machine in the kitchen with a simple wave from the comfort of your bed or easy chair in the lounge?
The system can already identify nine different whole-body gestures, from pushing and pulling to punching the air, and, during initial tests in a two-bedroom apartment and in an office environment, WiSee was subjected to 900 gestures performed by five users and identified them with 94 percent accuracy. As such, WiSee could be key to bringing the future of the connected home closer to the present.
Gesture control products in your near future
With a slew of products heading to market, from the Leap Motion Controller, to the Myo armband and even the Microsoft Kinect system for the Windows PC, it’s pretty clear that gesture (along with voice) is going to play an important role in the future of computer interaction. As computing moves from a fixed PC at a desk to mobile, pocket-sized devices and even appliances around the home and even in the street, keyboards, mice and even touch will no longer be good enough for controlling the myriad devices that will exist within the ‘Internet of Things’ and that will come to be seen as standard gadgets, features and services of the connected home.
‘Repurposing’ wi-fi signals
Using an ‘adapted’ receiving device that listens to all wireless transimissions within a home or office it matches changes in frequency — waves will slow down when they hit a hand or leg or are disrupted when the fingers of a hand are wiggled — to a set of recognizable gestures.
However, what is so exciting about WiSee is that it works regardless of whether the user is in a fixed location. The Leap Motion Controller has an effective range of 8 cubic feet and a Kinect will only respond to gestures if the user is in the device’s line of sight. Wi-fi signals, on the other hand, pass through walls and permeate an apartment. As lead researcher Shyam Gollakota, a UW assistant professor of computer science and engineering, explains: “This is repurposing wireless signals that already exist in new ways. You can actually use wireless for gesture recognition without needing to deploy more sensors.”
A standard consumer wi-fi network router is so strong that it’s pretty easy to piggy-back a neighbor’s connection if it’s not password protected. Therefore there’s no need for a sensor in every room or even for the user to have line of sight with a camera or controller. “This is the first whole-home gesture recognition system that works without either requiring instrumentation of the user with sensors or deploying cameras in every room,” said Qifan Pu, a collaborator and visiting student at the UW.
And, to make sure this ‘always on’ effect doesn’t lead to unintended gestures deleting movie files or boiling an empty kettle, WiSee’s creators are experimenting with a gesture-password system. A series of gestures will act as log-in for a specific device.
WiSee is currently in the proof of concept stage and will get its first official public demonstration at the 19th Annual International Conference on Mobile Computing and Networking in September.
Until you can watch a video of the system in action here.