For the first time, humanities scholars can crunch numbers with the best of their natural-science colleagues, thanks to Google鈥檚 鈥渇ossil record鈥� of 5 million books, spanning 500 years.
Already, researchers have traced the accelerating evolution of the English language, mapped the rise and fall of various people, and uncovered patterns of censorship and suppression in Soviet Russia, modern China, and 1950s America 鈥� and that鈥檚 only a beginning.
鈥淭his dataset is going to underwrite a field which is far, far more interesting than anything we could talk about in a single paper,鈥� says , an applied mathematician and bioengineer at Harvard University who led the research along with , also of Harvard.
Advertisement
Until now, scholars in the humanities have tended to read a relatively small number of texts in detail, just hundreds or thousands at the most. This let them form a subjective picture that does not lend itself to statistical analysis. However, in recent years Google has set out to create digitised versions of the full text of millions of books.
From the more than 15 million books digitised to date, Aiden, Michel and colleagues from Google and Harvard selected the 5.2 million with the most reliable data 鈥� a total of more than 500 billion words. If written as a single line of text, this would stretch to the moon and back 10 times. Then the researchers counted up the number of times each word appeared in the dataset during each year from 1800 to 2000.
Lexical dark matter
This let them follow changes in word use over this period, as the total number of English words in use rose from 544,000 in 1900 to more than 1 million in 2000, with the vast majority of that increase coming after 1950. (About 52 per cent of those million words do not appear in standard dictionaries, forming what the researchers call 鈥渓exical dark matter鈥�.)
Similarly, they tracked the mention of people鈥檚 names, a crude measure of fame, and found that people today become famous earlier in life than they used to 鈥揳n average age of 29 in the mid-20th century, down from 43 in the early 19th century. However, fame today is more fleeting, they found.
The data also show clear evidence of censorship, as certain taboo names disappear from use in certain countries: 鈥淭iananmen Square鈥� in China after 1989, for example. Likewise, 鈥淟eon Trotsky鈥� declined sharply in use in Russian books around 1940, and the names of blacklisted Hollywood actors got fewer mentions during anti-communist hysteria in the US.
Quantitative analyses of this sort represent an important new tool for humanities scholars, says Brett Bobley, director of the at the US government鈥檚 National Endowment for Humanities in Washington, DC. 鈥淭here鈥檚 a lot of potential there,鈥� he says. 鈥淲e鈥檙e at a real tipping point.鈥�
Journal reference:
