When Teresa Marrin Nakra slips into her Lycra jacket and begins to conduct, a stream of note-perfect orchestral music pours forth. But if she hands the sensor-stuffed garment to an untrained friend, the music slurs, as if the whole orchestra was suddenly drunk.
There are no musicians in Nakra鈥檚 laboratory at Immersion Music in Haverhill, Massachusetts 鈥� only a cyber-orchestra, made up of groups of instrumental sounds inside a synthesiser.
But just like the real thing, the cyber-orchestra only plays well if it鈥檚 conducted properly, with the conductor鈥檚 right arm signalling volume and the left arm beating time.
Advertisement
鈥淪he plays the system just like an instrument,鈥� says Gary Hill, a teacher of orchestral conducting at Arizona State University. Hill discovered Nakra鈥檚 invention when he was looking for a better way to teach the discipline.
Like all professional conductors, Hill鈥檚 students begin their careers conducting silently in front of a mirror. The only music they produce is in their head.
Nakra developed her conductor鈥檚 jacket while doing research at the Media Lab at MIT. It contains sensors over the arms, back and torso that pick up electrical activity from muscles and detect the wearer鈥檚 heartbeat. It is connected to a PC though wires emerging from the tail of the jacket.
To calibrate the system, Nakra used the jacket to analyse hundreds of hours of action by six conductors. The six included Benjamin Zander of the Boston Philharmonic, Keith Lockhart of the Boston Pops Orchestra, and Nakra herself.
Software running on the PC correlated data from each conductor鈥檚 muscle and heart activity with the volume, tempo and emotional responses each one used for particular musical passages.
The clingy Lycra material of the jacket holds the sensors close to the skin, and the conductors felt comfortable wearing it, Nakra says. 鈥淭he point was to sense movements without changing them.鈥�
When worn by a student the jacket and its PC are linked to a music synthesiser that automatically replays various pieces. As the student conducts the piece, the computer interprets their movements to give the music emotional highs and lows.
In Hill鈥檚 classes, students conduct a series of study pieces designed to test particular skills, like increasing and decreasing the volume, or beating in a particular dynamical style. The computer responses enable them to associate the right muscle memories with musical sounds, and penalises them for tensing in ways that professionals usually avoid. 鈥淚f they generate in a weird way, they get a weird sound,鈥� says Nakra.
Early indications are that the system appears to be halving the time students take to progress from novice to intermediate conductor, from six months to about three.
Hill says it is especially good for teaching students to relax their upper body. Barry Kraus, one of Hill鈥檚 students, agrees. 鈥淵ou learn to make sure you relax enough so your gestures stand out.鈥�