SILAS is a friendly-looking golden dog with floppy ears and a pointy tail. If you hold out your hand, Silas will come up to you wagging his tail and sit down, just like a real dog. He will even bring you a ball and try to get you to play.
But Silas is not real. He is made from computer-generated shapes and looks a little as though he escaped from a Disney cartoon. Silas is one of a small but growing number of computer programs that can mimic the behaviour of friends, pets and other objects of human affection. Among them are Julia, with whom you can have long, if strange, conversations, Phink, an unpredictable dolphin-like creature, and Neuro-Baby, a digital infant who can analyse and react appropriately to your moods.
Building 鈥渞elationships鈥 with these digital creatures is, of course, rather an unusual process. With Silas, for example, you have to stand in front of a video camera looking at a projection screen on which you see a picture of yourself, your room and Silas, added in by the computer program. Of course, these are all virtual friends; robot friends with 鈥渞eal鈥 bodies are still a long way off. The first clumsy autonomous robots 鈥 among them Skimer 鈥 are only now emerging from the laboratory. One day, they may turn into the mechanical servants beloved of early science fiction writers.
Advertisement
For the more immediate future, the scientists and artists who are creating digital companions want to explore what is needed to make these computer-generated objects more friendly and engaging to humans. That research is proving that we are often surprisingly willing to read intelligence and intention into the creations of computer programs.
Attempts to build such software started back in the early days of artificial intelligence research. The inspiration to write programs that could provide sufficiently human-like communication to fool other humans came partly from the famous 鈥淚mitation Game鈥 set by British mathematician Alan Turing. His view was that if a computer program mimicked a human so effectively that a real human could not tell whether they were interacting with another human or a computer, then the program was effectively a human being.
Early approaches could tackle only text-based interactions because computer power was far too small to generate a real-time visual image as complex as a dog. Eliza, developed in 1965 by Joseph Weizenbaum at the Massachusetts Institute of Technology, was one of the most famous of the conversational programs, imitating the questioning of a therapist. The program scanned the user鈥檚 punched input for certain words and phrases, then plugged these phrases into sentences from its database. The resulting simple, open-ended questions seemed to come from a human who was listening to what the user was saying:
Human: I feel unhappy.
Eliza: Is it because you feel unhappy that you came to see me?
This was a clever strategy: by adopting a style of discourse designed to conceal the speaker鈥檚 personality, it could also conceal the fact that the speaker had no personality to conceal.
Parry, developed a decade later, approached the problem from the opposite perspective. The program imitated a paranoid, and the user took the role of the therapist. A typical exchange looked like this:
Human: How did you come to be in the hospital?
Parry: I am upset.
Human: How long have you been in the hospital?
Parry: About a week.
Human: In what way are you upset?
Parry: People make me nervous.
Human: In what way do they make you nervous?
Parry: People give me a funny look sometimes.
Parry鈥檚 imitation of a paranoid encouraged the user to accept its non sequiturs and disconnected responses; if the program was unable to find anything in the user鈥檚 input that it could understand, it could simply blurt out something at random without seeming to be acting out of character.
Research on text-based question and answer programs has developed rapidly since then. Julia, developed by Michael Mauldin at Carnegie Mellon University鈥檚 Center for Machine Translation in Pittsburgh, Pennsylvania, has come much closer to imitating human conversation. Julia imitates a normal person 鈥 one that, by coincidence, seems to want to talk about pets:
Human: Do you have a pet at home?
Julia: I鈥檇 rather have a real pet, like a cat. Cats are my favorite, because you can pet cats, and they鈥檙e easy to keep.
Human: Do you know anything about miniature schnauzers?
Julia: I don鈥檛 know anything about miniature schnauzers.
Human: Well, I have one. We call him Pucci. He is a crazy cat 鈥 have you ever had a cat?
Julia: What a coincidence! I have two cats.
Julia uses many of Eliza鈥檚 and Parry鈥檚 tricks. For instance, since asking quetions is much easier than answering them, Julia asks a lot of questions, like Eliza. And like Parry, Julia frequently takes control of the conversation by changing the subject. But Julia adds a number of new tricks. It appears to have a more vivid personality than its predecessors, because its database includes statements that are controversial or funny, such as 鈥淕uinea pigs are about as smart as cucumbers鈥. The database also includes fragments of more extended conversational threads that can make the program seem coherent if the user is fooled into playing along with them.
Julia cannot think. But as long as you don鈥檛 try to move beyond pleasant small talk about pets, you stand a good chance of being fooled.
The key to success is to isolate the characteristics that make interactions seem real. We are often willing to explain away meaningless behaviour as idiosyncratic, and to ascribe intention where none exists, provided we have reason to believe we are dealing with an animate object.
It鈥檚 a dog鈥檚 life
Silas the dog builds on these insights. 鈥淭he research goal behind Silas is to understand how you can build an autonomous creature, like a dog, that seems to do the right thing over time,鈥 says Bruce Blumberg, Silas鈥 creator and a PhD student at the Massachusetts Institute of Technology鈥檚 Media Lab. People readily respond to Silas as though he were real, creating explanations for his behaviour the way they would with a real dog. 鈥淔rom the user鈥檚 perspective,鈥 says Blumberg, 鈥渢here鈥檚 a sentient, intentional being there.鈥
The willingness of humans to see complex emotions in animals is well known to ethologists. 鈥淧eople often ascribe feelings to a real dog that are over and above what they really would admit, if pushed, they believe the dog really feels,鈥 says June McNicholas, a research fellow at the University of Warwick who has worked extensively on the bond between humans and animals. 鈥淚f you鈥檝e had a bad day at work,鈥 she says, 鈥測our dog may seem to respond to this. And you鈥檒l say to yourself 鈥榟e knows I had a bad day at work鈥. But you know that the dog doesn鈥檛 know you had a bad day at work. All he knows is that you are moving and acting differently than you usually do. And you know this.鈥
Reacting to Silas involves similar rationalisations. For instance, Silas is interested in moving objects that are close to him. If you reach out to pat him, your hand will be both moving and close, and Silas will watch it. Silas doesn鈥檛 have any way of knowing whether you鈥檙e touching him or not, but because he鈥檚 watching your hand so intently he 鈥渁ppears鈥 as though he is responding to being patted.
Users tend to explain his response in those terms (鈥淚鈥檓 patting him, he likes it鈥) and the explanation has predictive value (whenever the user tries to pat the dog, the response is the same) and so, in the user鈥檚 mind, Silas likes to be patted.
People find Silas fascinating partly because they enjoy trying to explain what he is doing. 鈥淚f a creature behaves exactly the same every time,鈥 says Blumberg, 鈥渢hat鈥檚 not very interesting. It turns into a robot. On the other hand, if it鈥檚 totally unpredictable, then it seems random, and it鈥檚 hard for the user to develop an explanation with predictive value. The optimal place is where there鈥檚 just enough surprise that you鈥檙e constantly coming up with new explanations that make sense. I think that this is why we like having 鈥榬eal鈥 pets.鈥
Blumberg hopes that his work will soon be built into video games. By making creatures in a game more responsive and by adding autonomous and unpredictable behaviours, they may seem more alive and more interesting. Indeed, Japanese game designers are already experimenting with exactly these components.
Flying dolphin
The Believable Agent Project is Fujitsu鈥檚 foray into the world of virtual creatures. Software designer Makoto Tezuka has created a CD-ROM world called 鈥淭he Other Earth鈥 (TEO), where Phink, a flying dolphin-like creature, lives.
To help you communicate with Phink, Fujitsu provides a 鈥淭EO antenna鈥, a combined infra-red sensor and microphone that connects to your computer and can pick up voice and movement, and a whistle which can be blown to attract Phink if it isn鈥檛 on screen. But Phink will not necessarily respond.
The idea is to give Phink the quality of a wild creature, which can only be 鈥渢amed鈥 if you are really nice to it, and don鈥檛 try and keep Phink as a pet. Users can鈥檛 tell in advance what mood Phink will be in. They can鈥檛 even tell what image they will see when returning to TEO after a couple of days. Time passes on TEO, and the world changes, even when the software is inactive.
Whether TEO, which has not yet been released for sale, is a success remains to be seen. At best, however, TEO鈥檚 ambitions are limited to providing a more engaging game environment. Elsewhere in Japan, video artist Naoko Tosa has much bigger goals. She wants her Neuro-Baby to 鈥渓ive and communicate with modern urban people like ourselves, people who are overwhelmed, if not tortured by the relentless flow of information, and whose peace of mind can only be found in momentary human pleasures鈥.
Tosa is visiting artist at the ATR Media Integration Communications Research Laboratories near Kyoto and is developing her latest Neuro-Baby with researchers from the University of Tokyo. The baby鈥檚 present home is a run-down laboratory, packed with computers, young programmers and cabling in a basement of the decaying Institute of Industrial Sciences in Tokyo.
Where other researchers have tried to add interactive and unpredictable behaviour, Tosa has gone for emotion. On screen, Neuro-Baby appears deceptively simple 鈥 just an animated cartoon infant face with big round eyes, expressive eyebrows, and a small set of utterances provided by a voice synthesiser. But Neuro-Baby is more than it seems: it can analyse voice stress patterns to determine your mood and to respond accordingly. The voice stress analysis is provided by a trainable neural network computer.
Talk to the baby and it responds as if it is sharing the same emotion. The baby鈥檚 reaction is always interesting, often disconcerting. Its emotions, says Tosa, run from 鈥渃ute鈥 to 鈥渟tubborn and introverted鈥 and on through 鈥渃onceited and perverse鈥 to the 鈥渃razy type with violent emotional ups and downs鈥. If you鈥檙e feeling relaxed and peaceful, Neuro-Baby can be sugary-sweet, asking you with a cute smile 鈥淒o you live around here?鈥 Ignore Neuro-Baby, and it can start to irritate as it tries to get your attention, asking 鈥淒o you know any tongue-twisters?鈥 It knows lots and it will tell you them all. Snap at Neuro-Baby and its face grows angry and red, and its head jerks around the screen as it shouts 鈥淪top it! Stop it!鈥 at you. To make interaction more real, Neuro-Baby maintains eye contact. Two video cameras track the eyeballs of both the speaker and the baby so they are always locked onto each other.
An earlier version of Neuro-Baby was exhibited at the SIGGRAPH exhibition in 1993. There, Tosa found that when Americans interacted with Neuro-Baby, it started to act as though it were emotionally unstable. Americans were just too much for it. 鈥淚t was educated with Japanese monotonous voices,鈥 says Tosa. 鈥淎merican intonation was too strong for it.鈥 Now she has made separate emotional models for Japanese and American users, and at this year鈥檚 SIGGRAPH she tried using Neuro-Baby as a cross-cultural emotional communication tool.
When a Japanese and an American communicate through Neuro-Baby, the Japanese sees the American鈥檚 emotions toned down to Japanese levels; the American sees the Japanese participant鈥檚 emotions expressed by Neuro-Baby in the more extreme American way. Neuro-Baby could eventually act as an electronic agent which translates emotions as well as language.
This is ambitious, but Tosa has already achieved considerable fame in Japan for her work in experimental film and video which, she explains, are on the boundary between 鈥渢he visible and the invisible鈥. She see enormous potential for individuals to create digital companions. 鈥淚 wanted to create something like my alter ego,鈥 she says. 鈥淪ome organism which is also like my closest friend or relative.鈥 So Neuro-Baby is also a means of personal expression. 鈥淧eople express their dreams in the media at hand, such as novels, films and drawings,鈥 she explains. Neuro-Baby uses contemporary media to reach the same ends.
Tiny steps
Behind Neuro-Baby鈥檚 artless reactions there is much thought and a lot of computer power. She is now adding a hand which you can shake to the latest model. Sensors monitor the form of your handshake and feed additional information into the computer鈥檚 model of your emotional state. This is Neuro-Baby鈥檚 first tiny step out of the computer. But there is a very long way to go before it, or Silas, can become 鈥渞eal鈥.
Bringing artificial pets into the outside world is far more complex than creating images on a screen. This is why robotic pets are many steps behind virtual ones. Will Wright, the inventor of SimCity, the interactive computer game that allows users to create their own urban dreams, has a passion for robots. But, as he explains, out in the real world, a robot pet can鈥檛 see an object unless it actually detects and interprets the light reflecting off it. 鈥淓dge parsing is fairly computationally intensive,鈥 he says, 鈥渁nd it鈥檚 very hard to do it in real time.鈥
Indeed, Silas鈥檚 ability to identify where the user is standing depends on the much easier technique of 鈥渂ackground subtraction鈥 鈥 the computer knows what the room looks like when it鈥檚 empty. When a person walks into the scene, Silas鈥檚 image-processing software compares the stored image of the empty room with the current scene 鈥 what remains must be the person.
This technique depends on an unchanging background, though. Start moving furniture around, or bring more people into the room, and Silas is likely to get confused. And while Silas is able to process and parse the video image in real time, these tasks are handled by a Silicon Graphics Iris workstation, which is rather more computing power than will fit in a mobile, pet-sized box.
Skimer the robot illustrates these constraints. Kino Coursey, of Daxtron Laboratories in Fort Worth, and the developer of Skimer鈥檚 software, says that the robot can visually identify objects and be trained to follow them around. To do this, one trainer drags a chair, say, while the other uses a joystick to instruct Skimer to move in pursuit of the chair.
Skimer builds a network of associations between the images that it is seeing and the commands it is receiving. For example, when the chair moves out of the robot鈥檚 field of view, moving from right to left, the trainer commands Skimer to turn to the left. Skimer remembers the sequence of images and the commands associated with them. Once Skimer has been trained, it will follow the left-turn command whenever it sees a chair moving across its vision to the left. The result, says Coursey, is that 鈥測ou can drag a chair around in front of it and he鈥檒l follow it anywhere鈥.
But while Skimer does this very well, it can鈥檛 do anything else. And it鈥檚 a pretty hefty contraption, cobbled together from a child鈥檚 six-wheeled riding toy, a camcorder, a computer, and 18 kilos of batteries. Skimer, as Coursey puts it, 鈥渄efinitely belongs in the back yard. You wouldn鈥檛 want him in the house鈥.
Pet care
If Skimer could be made small and agile enough, could it replace the family dog or cat? Erika Friedmann, a specialist in pets and health in Brooklyn College鈥檚 Health and Nutritional Science Department, acknowledges that the autonomy of artificial pets might attract people the way real pets do. 鈥淧eople like pets because they don鈥檛 have to make an effort to get positive feedback from another being,鈥 she says. 鈥淵our pet makes some kind of acknowledgment that you鈥檙e there and entices you to interact with it.鈥
But pets provide people with far more than unstructured entertainment, says Friedmann. They give us someone to care for, someone to touch and fondle. They provide a reason for exercise, a feeling of safety.
McNicholas thinks that this is the fundamental weakness of the artificial dog. 鈥淲hat kind of level of care could someone give a virtual pet?鈥 she asks. 鈥淎 lot of the closeness in a relationship with a pet is based on how dependent the pet is on you.鈥
Even the most complex artificial pet only engages people on one of the many levels that real pets do. Though you can turn off Silas or Neuro-Baby, these aren鈥檛 鈥渄eaths鈥. You can always turn them on again. Artificial pets don鈥檛 depend on your care to keep them alive and well. And, as McNicholas says, 鈥渋f an artificial pet doesn鈥檛 depend on you, you don鈥檛 feel needed鈥. Perhaps that justifies Tosa鈥檚 view of the potential of digital companions: they may not be able to replace pets, but they could give people a new way of expressing themselves.