Now the ads watch you (Image: Daniel Allan/Cultura/Alamy)
DON鈥橳 pretend you don鈥檛 want that chocolate bar. Software can now sense how much you鈥檝e been swayed by marketing just by analysing your face as you watch advertisements.
Advertisement
Ad companies are often interested in gauging consumers鈥� reactions to their latest TV spot. Traditionally, this is done by bringing a few customers into an office and asking questions.
But the system made by Affectiva, a start-up in Waltham, Massachusetts, can pick up on hidden emotions just by monitoring face movements. The approach, says Affectiva鈥檚 principal scientist , lets you find out what people actually think from moment to moment while the ad runs, not just what they say once it is over.
鈥淚t provides a way of getting at those more genuine, spontaneous interactions,鈥� he says. 鈥淭his is their visceral response. It鈥檚 not sent through a cognitive filter where they have to evaluate how they feel.鈥�
Affectiva鈥檚 software first pinpoints important facial markers, such as the mouth, eyebrows and the tip of the nose. Then, machine-learning algorithms watch how those regions move or how the skin texture and colour changes over the course of the video. These changes are broken down into discrete expressions indicating shifting emotions.
In a study published this month, McDuff and his colleagues asked 1223 people to give his team access to their home webcams while they watched a series of ads for sweets, pet supplies and groceries. Before and after the ads ran, the subjects filled out online surveys about how likely they were to purchase the products shown. While they watched, the software stayed on the lookout for emotions, such as happiness, surprise or confusion.
Afterwards, the researchers found that they could use the facial data to accurately predict someone鈥檚 survey results 鈥� suggesting that they could rely on the computer鈥檚 analysis alone to know whether an ad was successful (IEEE Transactions on Affective Computing, ).
In the future, McDuff thinks the system could plug into TV services such as Netflix. 鈥淵ou could imagine suggesting TV programmes or movies that people could watch, or ads that they find more enjoyable,鈥� he says.
鈥淵ou could imagine suggesting movies that people could watch, or ads that they find enjoyable鈥�
The Affectiva team has amassed a database of over three million videos of people across different ages, genders and ethnicities. McDuff says that there seem to be subtle variations in emotional responses: women tend to have more positive facial expressions than men, for example. By understanding how different groups respond, companies could put together ads that are fine-tuned for particular audiences.
The data could also help advertisers to tweak their adverts to tie in more closely to viewers鈥� emotions 鈥� for example, by putting in the name of the brand at the moment that elicits the strongest positive reaction.
Automated emotion analysis systems are promising, says Michel Wedel, who studies consumer science at the University of Maryland in College Park. They let advertisers break an ad down moment by moment, to figure out exactly what works and what doesn鈥檛.
鈥淲hat鈥檚 particularly powerful is that they鈥檙e unobtrusive,鈥� he says. 鈥淭hey don鈥檛 rely on introspection or recollection.鈥�
Being able to do research through viewers鈥� home webcams is another advantage, says Wedel, although it won鈥檛 be foolproof. 鈥淧eople could be at the computer eating a sandwich or turning their head, so it could be the case that you can鈥檛 classify their emotions reliably.鈥�
This article appeared in print under the headline 鈥淲hat will you buy next? The computer knows鈥�
