WE ARE used to sitting in front of a PC鈥檚 screen while tapping away at a keyboard. But is this really the best way to communicate with a computer?
The Nintendo Wii remote control and Microsoft鈥檚 multi-touch display have begun to change the way we interact with computers, but even these give us feedback in a conventional way. Now Pattie Maes and David Merrill at the Massachusetts Institute of Technology Media Lab are taking things a step further.
鈥淚 wanted to build a new human-computer interface that would take advantage of our abilities to grasp objects, move them around and understand the spatial relationships between them,鈥� Merrill says. 鈥淢any of these skills are underutilised.鈥�
Advertisement
He has now done this by building a set of smart blocks he calls 鈥渟iftables鈥�, each about the size of a toddler鈥檚 play brick. Users feed information into the system simply by shuffling the siftables around, while the siftables themselves relay information back to the user.
Each siftable measures about 5-centimetres square and is fitted with an LCD screen, battery, memory, an accelerometer to detect motion, a Bluetooth radio to communicate with other computers, and an infrared link to detect the presence and orientation of neighbouring siftables. Each block also has a small amount of built-in computing power, comparable to that available in a mid-1980s PC.
A group of siftables can be programmed to either solve problems independently, or take their orders from a desktop computer, in which case the way the pieces are moved around also controls an application running on the computer.
鈥淭he blocks can solve problems independently or take their orders from a desktop computer鈥�
Merrill says the devices are likely to be used initially for games. 鈥淕aming is often an area where new interfaces take hold,鈥� he says, 鈥渂ecause people are more willing to try something new in a play setting than at work.鈥�
He has already developed a siftables word game. Each piece displays a randomly assigned letter, and the user has to move pieces around to form words as quickly as possible. The pieces communicate with each other to keep track of the words being formed, which are checked against a built-in dictionary. When the player forms a valid word, the siftables highlight the relevant pieces.
Producing music is another potential application. Merrill and Josh Kopin of Brown University in Providence, Rhode Island, used siftables to build a user interface for a music sequencer running on a desktop computer. The interface provides three 鈥渧oices鈥� 鈥� lead, drum and bass 鈥� each represented by a different siftable, along with pieces for tempo and volume. Other pieces represent effects such as reverberation.
The user initially arranges a sequence of 鈥渂lank鈥� pieces in a row, which is brought to life by touching the blanks with pieces representing different notes, or notes with special characteristics. Pieces can be tilted to alter their value 鈥� to raise or lower the volume, for example. The siftables communicate with the desktop computer, which plays back the sequence.
The devices could also be used for scientific modelling. For instance, epidemiologists have a technique called compartment modelling in which a population confronted by a disease outbreak is divided into three groups: susceptible, immune, and infected or infectious. Researchers can study the dynamics of the disease using a computer model in which they modify various parameters 鈥� by introducing new disease strains, for example. 鈥淪iftables would make the process of formulating a model interactive and physical in a way the software can鈥檛,鈥� says epidemiologist Caroline Buckee of the University of Oxford.
The position of different siftables could be programmed to represent the value of a particular parameter, and the pieces simply moved above or below an average line to quickly change, say, how infectious a disease strain is. It would feel as intuitive as moving sliders up and down on a stereo.
Different parameters could be rapidly added or removed using other pieces, with the outcome of these changes displayed on a computer screen. 鈥淵ou could rapidly explore how changes to parameters would affect the overall prevalence of the disease, or the number of people you need to vaccinate to prevent an outbreak,鈥� Buckee says.
Just a wave of the hand
In the film Minority Report, Tom Cruise鈥檚 character manipulates information on giant computer displays using hand gestures.
Now 鈥榮 fluid interfaces research group at the Massachusetts Institute of Technology has developed a device that lets you do something similar. Called sixth sense, it offers a way of displaying information on any surface 鈥� a newspaper or wall, for example 鈥� and manipulating it with hand gestures. 鈥淲e wanted to find out if we could merge information into a sense that is always with us,鈥� says Maes鈥檚 student Pranav Mistry.
The device, which is worn around the neck as a pendant, contains a small projector and webcam. The pendant communicates either with a laptop in a backpack or a smartphone connected to a remote server. The webcam monitors the user鈥檚 hand gestures and conveys them to control software running on the laptop or server.
This software uses the projector to initially display a menu. The user taps on an icon, and this gesture is relayed to the software, which opens the selected application. This could be something like a map displayed on the surface, which the user can then control with a wave of their hand.
