Historians, literary detectives and even courts of law rely on methods that identify the author of a text by their writing style. But a new study suggests that some of these so-called techniques are easily fooled, even by people without linguistic or literary training.
As well as being used to answer literary questions, such as who wrote Shakespeare鈥檚 plays, accept evidence from stylometry on the authorship of written material, including suicide notes and threatening letters. Stylometry even helped to convict 鈥淯nabomber鈥� in 1998.
But the features that stylometry techniques rely on can be easy to imitate, say Michael Brennan and at Drexel University in Philadelphia, Pennsylvania. They have shown that people can successfully confuse stylometry software and hide their identity by imitating the writing style of another person. Until now there had been little research into the weaknesses of these techniques, they say.
Advertisement
Writing attack
The researchers asked 15 people who were not professional writers to submit a 5000 word 鈥榮ignature鈥� text as a sample of their personal writing style. Each volunteer was then asked to write a description of their neighbourhood in a way that masked their personal style, before writing a further passage in the style of novelist and playwright .
Various stylometry techniques were pitted against this deception in an attempt to correctly reveal the true authors of each 鈥渕asked鈥� passage. They ranged from simple techniques, such as measuring word length and analysing punctuation, to more complex methods, such as working out the lexical density, a measure that divides the number of unique words in the document by the total word count.
The methods could identify the author of extracts from signature texts with at least 80 per cent accuracy. But they were no better than random at knowing who wrote a passage when people attempted to hide their writing style. And the techniques consistently identified Cormac McCarthy as the author of the imitations of his work.
鈥淲e would strongly suggest that courts examine their methods of stylometry against the possibility of adversarial attacks,鈥� says Greenstadt.
In the dock
鈥淚t鈥檚 a great paper,鈥� says , a computer scientist and text analyst at Duquesne University in Pittsburgh, Pennsylvania. 鈥淲hen you read a paper and say 鈥榳ell now I know what I鈥檓 studying for the next five years鈥�, they did something right.鈥�
As the study only attacked some of the less complex stylometry techniques, probing the vulnerabilities of others is a 鈥渉uge line of research鈥�, says Juola.
He gives the example of describing a table setting; is the fork placed 鈥渙n鈥�, 鈥渁t鈥� or 鈥渢o鈥� the left of the plate? 鈥淢ost people don鈥檛 necessarily notice which preposition gets used 鈥� and it鈥檚 harder to imitate what you haven鈥檛 noticed.鈥�
Unhelpful filter
Some of the techniques tested by Brennan and Greenstadt discard prepositions because they are deemed to have no information content, says , a computational linguist at the University of Sunderland, UK. This filters out the words that could have helped most, he says.
Brennan and Greenstadt agree that there are more stylometry techniques they could test in the future. 鈥淗owever, it鈥檚 worth noting that our attack methods are not as sophisticated as they could be,鈥� says Greenstadt.
Their volunteer 鈥渁ttackers鈥� lacked formal training in linguistics and had no access to stylometry software. With additional expertise, even the more sophisticated stylometry techniques might be vulnerable.
Brennan and Greenstadt presented on their experiments at the Hacking at Random 2009 conference in Vierhouten, the Netherlands, last week.
