A robotic arm with an attached microphone has learned how to locate noisy objects tossed into a bag, grabbing a set of keys by listening for the telltale clinking sound and picking out a crinkly bag of crisps sight unseen.
鈥淭hat environment is basically like you reach down, you don鈥檛 know where the keys are, but then once you hear the sound of the keys you can kind of localise it,鈥� says at Stanford University in California. 鈥淎nd then by localising, you can grab it out and lift it up.鈥�
Du and his Stanford University colleagues started out with simulations to train the robot to grasp before moving to physical trials. But instead of just trying and failing on its own, the robot learned from a person demonstrating the same task using an Oculus VR headset and controllers.
Advertisement
After the robot had been trained on those demonstrations, it received additional human guidance during sessions in which lab members corrected its actions. This interactive imitation improved the robot鈥檚 performance by 20 per cent, says , another member of the team.
The robot proved equally successful at picking up all sorts of noisy objects, including keys, a bag of crisps and a cluster of pens. By comparison, it failed to locate a piece of cloth that made very little sound upon contact.
The robot鈥檚 use of sound to locate objects suggests that inexpensive microphones could help robots detect objects in the real world, instead of relying on more expensive tactile sensors that provide a sense of touch. The added audio information could prove especially useful when robots must interact with environments where their vision is partially blocked.
鈥淚f you鈥檙e wanting to have a robot actually go into real homes and offices, real environments where people operate, that鈥檚 where the occlusions and general messiness will present itself,鈥� says team member .
Sound can also help robots identify different types of objects. A follow-up experiment showed that the robot could tell the difference between mystery bags filled with either metal screws or polystyrene packing peanuts by first picking up and dropping the bag.
Reference:
